pg_stat_statements is useful to gather performance information about queries so lets add it to our dockerized postgres database. Using docker-compose Using docker-compose we just need to add the following docker-compose.yaml: docker-compose.yml version: '3.5' services: postgres: container_name: postgres_container image: postgres ports: - "5432:5432" volumes: - /var/lib/postgresql/data # used for query profiling, deactivate for enhanced performance command: postgres -c shared_preload_libraries=pg_stat_statements -c pg_stat_statements.track=all -c max_connections=200 environment: POSTGRES_USER: postgres POSTGRES_PASSWORD: thepassword We simply start our Postgres database with docker-compose: ...

The Goal In the following short article, we want to import data from an existing sitemap XML file into a new Google Sheet document. The sheet must pull the sitemap via HTTP protocol and extract all the URLs from the sitemap and insert them. into the sheet. Implementation Now let’s implement it .. it only takes 1 minute …​ For demonstration purpose, I’m going to use the sitemap from my old blog, to be found at https://www.hascode.com/sitemap.xml. ...

The AWS shapes are included in the PlantUML stdlib .. simply include them as shown here: example.puml @startuml !include <awslib/AWSCommon> !include <awslib/Analytics/ManagedStreamingforKafka> !include <awslib/Database/RDS> !include <awslib/General/Users> !include <awslib/General/InternetGateway> !include <kubernetes/k8s-sprites-labeled-25pct> skinparam linetype ortho title "AWS Context Diagram" package "EKS Kubernetes Cluster" as eks_cluster { component "<$pod>\napp1" as pod1 component "<$pod>\napp2" as pod2 } ManagedStreamingforKafka(kafka_pod, "Amazon MSK", "Apache Kafka") RDS(pg_rds, "PostgreSQL", "Sample Schema") Users(users, "AppUsers","editors, admins") InternetGateway(igw1, "Customer Gateway", "Customer access to internal services") users -> igw1 igw1 --> pod1 igw1 --> pod2 pod1 --> pg_rds pod1 --> kafka_pod pod2 --> pg_rds pod2 --> kafka_pod @enduml ...

C4 models allow us to visualize software architecture by decomposition in containers and components. Viewpoints are organized in hierarchical levels: Context Diagrams (Level 1) Container Diagrams (Level 2) Component Diagrams (Level 3) Code Diagrams (Level 4) C4-PlantUML offers a variety of macros and stereotypes that make modeling fun. An example in PlantUML: sample.puml @startuml !include <c4/C4_Context.puml> !include <c4/C4_Container.puml> left to right direction Person(user, "User") System_Ext(auth, "AuthService", "Provides authentication and authorization via OIDC") System_Boundary(zone1, "Some system boundary") { System(lb, "Load Balancer") System_Boundary(az, "App Cluster") { System(app, "App Servers") { Container(app1, "App1", "Docker", "Does stuff") Container(app2, "App1", "Docker", "Does stuff") ContainerDb(dbSess, "Session DB", "Redis") ContainerDb(db1, "RBMS 1", "AWS RDS Postgres") ContainerDb(db2, "RBMS 2", "AWS RDS Postgres") ' both app servers sync sessions via redis Rel(app1, dbSess, "Uses", "Sync Session") Rel(app2, dbSess, "Uses", "Sync Session") ' both app servers persist data in RDBMS Rel(app1, db1, "Uses", "Persist/query relational data") Rel(app2, db2, "Uses", "Persist/query relational data") } } } Rel(user, lb, "call") Rel(lb, app1, "delegate") Rel(lb, app2, "delegate") Rel(app1, auth, "Verify", "User auth") Rel(app2, auth, "Verify", "User auth") SHOW_FLOATING_LEGEND() @enduml ...

The following hack allows us to draw a diagram in a diagram in a table in a legend :D example-diagram.puml @startuml skinparam legendBackgroundColor transparent skinparam legendBorderColor transparent actor Editor as editor queue Queue as "Message\nQueue" editor -> [Component] : call [Component] --> Queue : publish legend right {{ scale 0.7 skinparam defaultFontSize 14 skinparam BackGroundColor transparent skinparam defaultBackgroundColor white !procedure $entry($type, $label, $scale=1) {{\nscale $scale \nskinparam backgroundcolor transparent\nlabel " " as A\nlabel " " as B\n $type \n}} => $label !endprocedure map "<b>Legend</b>" as legend #white { $entry(":Editor:","\nEditor for Data xyz", 0.6) $entry("[Component]","\nSome Component", 0.8) $entry("queue Queue","\nKafka-Topic", 1) $entry("A -> B","Data-Flow") } }} endlegend @enduml ...

The following WebFilter redirects incoming requests for / to a static HTML file, index.html. ToIndexPageRedirection.java package com.hascode.tutorial; import org.springframework.stereotype.Component; import org.springframework.web.server.ServerWebExchange; import org.springframework.web.server.WebFilter; import org.springframework.web.server.WebFilterChain; import reactor.core.publisher.Mono; @Component public class ToIndexPageRedirection implements WebFilter { @Override public Mono<Void> filter(ServerWebExchange exchange, WebFilterChain chain) { if (exchange.getRequest().getURI().getPath().equals("/")) { return chain.filter( exchange.mutate().request( exchange.getRequest().mutate().path("/index.html").build() ).build() ); } return chain.filter(exchange); } } Ressources: JavaDocs WebFilter

Sometimes this is useful for the diagnosis of configuration and network problems of ones Java application. This is our single-class HTTP client example without the need for external dependencies: HttpTest.java import javax.net.ssl.KeyManagerFactory; import javax.net.ssl.SSLContext; import javax.net.ssl.TrustManagerFactory; import java.io.FileInputStream; import java.io.IOException; import java.net.InetSocketAddress; import java.net.ProxySelector; import java.net.URI; import java.net.URISyntaxException; import java.net.http.HttpClient; import java.net.http.HttpRequest; import java.net.http.HttpRequest.BodyPublishers; import java.net.http.HttpResponse; import java.net.http.HttpResponse.BodyHandlers; import java.security.KeyManagementException; import java.security.KeyStore; import java.security.KeyStoreException; import java.security.NoSuchAlgorithmException; import java.security.UnrecoverableKeyException; import java.security.cert.CertificateException; import java.time.Duration; import java.time.temporal.ChronoUnit; public class HttpTest { public static void main(String[] args) throws URISyntaxException, NoSuchAlgorithmException, KeyManagementException, KeyStoreException, UnrecoverableKeyException, IOException, CertificateException { String bodyPayload = """ ourpayload, json, xml, ... """; String keyStorePath = "/opt/keystore.jks"; String keyStorePassword = "ABCDEFG"; String proxyHost = "ourproxy.proxy"; String uriString = "https://some-service/api"; int proxyPort = 8080; int timeoutInSeconds = 60; KeyStore keyStore = KeyStore.getInstance("JKS"); keyStore.load(new FileInputStream(keyStorePath), keyStorePassword.toCharArray()); KeyManagerFactory keyManagerFactory = KeyManagerFactory.getInstance("PKIX"); keyManagerFactory.init(keyStore, null); TrustManagerFactory trustManagerFactory = TrustManagerFactory.getInstance("PKIX"); // using same keystore for both trustManagerFactory.init(keyStore); SSLContext sslContext = SSLContext.getInstance("TLS"); sslContext.init(keyManagerFactory.getKeyManagers(), trustManagerFactory.getTrustManagers(), null); HttpClient client = HttpClient.newBuilder() .proxy( ProxySelector.of(new InetSocketAddress(proxyHost, proxyPort))) .sslContext(sslContext) .build(); URI uri = new URI(uriString); HttpRequest request = HttpRequest.newBuilder(uri) .POST(BodyPublishers.ofString(bodyPayload)) .timeout(Duration.of(timeoutInSeconds, ChronoUnit.SECONDS)) .build(); System.out.printf("Sending POST request to %s, timeout: %ds%n", uri, timeoutInSeconds); try { HttpResponse response = client.send(request, BodyHandlers.ofString()); System.out.println("Response received..."); System.out.printf("\tResponse-Status: %d%n", response.statusCode()); System.out.printf("\tResponse-Body: %s%n", response.body()); System.out.println("-------------------------------------%n"); } catch (IOException e) { System.err.printf("IOException caught: %s%n", e.getMessage()); e.printStackTrace(System.err); } catch (InterruptedException e) { System.err.printf("IOException caught: %s%n", e.getMessage()); e.printStackTrace(System.err); } } } ...

Let’s say we would like to increase the limit t 10MB …​ Broker Configuration Apply the new limit either by modifying the server.properties like this…​ max.message.bytes=10485760 or apply it to a specific topic using kafka-configs.sh --bootstrap-server localhost:9092 \ --entity-type topics \ --entity-name thetopic \ --alter \ --add-config max.message.bytes=10485760 Producer Configuration for Spring Boot We simply need to add the following line to our application.properties: spring.kafka.producer.properties.max.request.size=spring.kafka.producer.properties.max.request.size The following proof-of-concept demonstrates that without the property, sending a large message fails, with the property it succeeds: ...

Sometimes it is nice to set up an embedded Kafka broker for testing without the need to have Docker installed (e.g. for using testcontainers-lib). The following snippet shows, how to set up an embedded Kafka instance for testing for a Spring Boot project. Setup Using Maven, this is our Spring Boot project with dependencies needed: pom.xml <?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <parent> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-parent</artifactId>(1) <version>2.6.4</version> <relativePath/> <!-- lookup parent from repository --> </parent> <groupId>com.hascode.tutorial</groupId> <artifactId>kafka-testing</artifactId> <version>1.0.0-SNAPSHOT</version> <properties> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding> <java.version>17</java.version> <maven.compiler.source>${java.version}</maven.compiler.source> <maven.compiler.target>${java.version}</maven.compiler.target> <kafka.version>3.1.0</kafka.version>(2) </properties> <dependencies> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter</artifactId> </dependency> <dependency> <groupId>org.springframework.kafka</groupId> <artifactId>spring-kafka</artifactId>(3) </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency> <dependency> <groupId>org.springframework.kafka</groupId> <artifactId>spring-kafka-test</artifactId>(4) <scope>test</scope> </dependency> <dependency>(5) <groupId>org.junit.jupiter</groupId> <artifactId>junit-jupiter-api</artifactId> <scope>test</scope> </dependency> <dependency> <groupId>org.junit.jupiter</groupId> <artifactId>junit-jupiter-engine</artifactId> <scope>test</scope> </dependency> <dependency> <groupId>org.junit.platform</groupId> <artifactId>junit-platform-launcher</artifactId> <scope>test</scope> </dependency> [..] </project> ...

This Linkchecker claims to be way faster than blc and wummel/linkchecker. Using Docker, we may validate our site’s links in no time and without complex setup. docker run --rm tennox/linkcheck hascode.io Resources https://github.com/filiph/linkcheck