Micha Kops’ Tech Notes

Prerequisites We need to have at least Docker installed. Creating the Blog First we’re creating a new directory for our blog and generate the blog structure using William Jackon’s docker-pelican: mkdir my-site cd my-site docker container run -it --rm --entrypoint pelican-quickstart -v ${PWD}:/pelican-site ghcr.io/williamjacksn/pelican Welcome to pelican-quickstart v4.9.1. This script will help you create a new Pelican-based website. Please answer the following questions so this script can generate the files 1 Title: My First Review needed by Pelican. > Where do you want to create your new web site? [.] > What will be the title of this web site? Micha's Tech Notes > Who will be the author of this web site? Micha Kops > What will be the default language of this web site? [C] > Do you want to specify a URL prefix? e.g., https://example.com (Y/n) Y > What is your URL prefix? (see above example; no trailing slash) https > Do you want to enable article pagination? (Y/n) n > What is your time zone? [Europe/Rome] > Do you want to generate a tasks.py/Makefile to automate generation and publishing? (Y/n) > Do you want to upload your website using FTP? (y/N) > Do you want to upload your website using SSH? (y/N) > Do you want to upload your website using Dropbox? (y/N) > Do you want to upload your website using S3? (y/N) > Do you want to upload your website using Rackspace Cloud Files? (y/N) > Do you want to upload your website using GitHub Pages? (y/N) Done. Your new project is available at /pelican-site ...

Installation using homebrew brew install sops manual download download from GitHub https://github.com/getsops/sops/releases Using SOPS Create a sops.yaml sops.yaml creation_rules: # encrypt stuff in .secrets - aws_profile: default kms: arn:aws:kms:eu-central-1:1234567890:key/abcdefg-0123456-abcdefg (1) path_regex: ^./secrets/.*$ (2) We are using AWS KMS for encryption/decryption All files in the directory .secrets will be encrypted Inplace Encrypt sops -e -i .secrets/mysecret.yaml Inplace Decrypt sops -d -i .secrets/mysecret.yaml Complete Example using PGP Install GPG and create a new Key Install the GPG binaries .Linux / apt sudo apt install gnupg ...

Tip Just a quick snippet Maven Integration pom.xml <dependency> <groupId>org.javers</groupId> <artifactId>javers-core</artifactId> <version>${javers.version}</version> </dependency> Calculate Changes in Object Graph package io.hascode; import org.javers.core.Changes; import org.javers.core.Javers; import org.javers.core.JaversBuilder; import org.javers.core.diff.Diff; public <T> Changes diff(T snapshot, T latest) { Javers javers = JaversBuilder.javers().build(); Diff diff = javers.compare(snapshot, latest); return diff.getChanges(); } Resources Javers Website

Figure 1. Kafdrop Topic Viewer In the ever-expanding world of data streaming and event-driven architecture, Apache Kafka has emerged as a cornerstone for reliable and scalable data processing. However, managing and monitoring Kafka clusters can often present its own set of challenges. This is where Kafdrop, a web-based Kafka consumer group and topic viewer, comes to the rescue. With its intuitive interface and insightful visualizations, Kafdrop offers developers and operators an efficient way to gain valuable insights into Kafka clusters. ...

pg_stat_statements is useful to gather performance information about queries so lets add it to our dockerized postgres database. Using docker-compose Using docker-compose we just need to add the following docker-compose.yaml: docker-compose.yml version: '3.5' services: postgres: container_name: postgres_container image: postgres ports: - "5432:5432" volumes: - /var/lib/postgresql/data # used for query profiling, deactivate for enhanced performance command: postgres -c shared_preload_libraries=pg_stat_statements -c pg_stat_statements.track=all -c max_connections=200 environment: POSTGRES_USER: postgres POSTGRES_PASSWORD: thepassword We simply start our Postgres database with docker-compose: ...

The Goal In the following short article, we want to import data from an existing sitemap XML file into a new Google Sheet document. The sheet must pull the sitemap via HTTP protocol and extract all the URLs from the sitemap and insert them. into the sheet. Implementation Now let’s implement it .. it only takes 1 minute …​ For demonstration purpose, I’m going to use the sitemap from my old blog, to be found at https://www.hascode.com/sitemap.xml. ...

The AWS shapes are included in the PlantUML stdlib .. simply include them as shown here: example.puml @startuml !include <awslib/AWSCommon> !include <awslib/Analytics/ManagedStreamingforKafka> !include <awslib/Database/RDS> !include <awslib/General/Users> !include <awslib/General/InternetGateway> !include <kubernetes/k8s-sprites-labeled-25pct> skinparam linetype ortho title "AWS Context Diagram" package "EKS Kubernetes Cluster" as eks_cluster { component "<$pod>\napp1" as pod1 component "<$pod>\napp2" as pod2 } ManagedStreamingforKafka(kafka_pod, "Amazon MSK", "Apache Kafka") RDS(pg_rds, "PostgreSQL", "Sample Schema") Users(users, "AppUsers","editors, admins") InternetGateway(igw1, "Customer Gateway", "Customer access to internal services") users -> igw1 igw1 --> pod1 igw1 --> pod2 pod1 --> pg_rds pod1 --> kafka_pod pod2 --> pg_rds pod2 --> kafka_pod @enduml ...

C4 models allow us to visualize software architecture by decomposition in containers and components. Viewpoints are organized in hierarchical levels: Context Diagrams (Level 1) Container Diagrams (Level 2) Component Diagrams (Level 3) Code Diagrams (Level 4) C4-PlantUML offers a variety of macros and stereotypes that make modeling fun. An example in PlantUML: sample.puml @startuml !include <c4/C4_Context.puml> !include <c4/C4_Container.puml> left to right direction Person(user, "User") System_Ext(auth, "AuthService", "Provides authentication and authorization via OIDC") System_Boundary(zone1, "Some system boundary") { System(lb, "Load Balancer") System_Boundary(az, "App Cluster") { System(app, "App Servers") { Container(app1, "App1", "Docker", "Does stuff") Container(app2, "App1", "Docker", "Does stuff") ContainerDb(dbSess, "Session DB", "Redis") ContainerDb(db1, "RBMS 1", "AWS RDS Postgres") ContainerDb(db2, "RBMS 2", "AWS RDS Postgres") ' both app servers sync sessions via redis Rel(app1, dbSess, "Uses", "Sync Session") Rel(app2, dbSess, "Uses", "Sync Session") ' both app servers persist data in RDBMS Rel(app1, db1, "Uses", "Persist/query relational data") Rel(app2, db2, "Uses", "Persist/query relational data") } } } Rel(user, lb, "call") Rel(lb, app1, "delegate") Rel(lb, app2, "delegate") Rel(app1, auth, "Verify", "User auth") Rel(app2, auth, "Verify", "User auth") SHOW_FLOATING_LEGEND() @enduml ...

The following hack allows us to draw a diagram in a diagram in a table in a legend :D example-diagram.puml @startuml skinparam legendBackgroundColor transparent skinparam legendBorderColor transparent actor Editor as editor queue Queue as "Message\nQueue" editor -> [Component] : call [Component] --> Queue : publish legend right {{ scale 0.7 skinparam defaultFontSize 14 skinparam BackGroundColor transparent skinparam defaultBackgroundColor white !procedure $entry($type, $label, $scale=1) {{\nscale $scale \nskinparam backgroundcolor transparent\nlabel " " as A\nlabel " " as B\n $type \n}} => $label !endprocedure map "<b>Legend</b>" as legend #white { $entry(":Editor:","\nEditor for Data xyz", 0.6) $entry("[Component]","\nSome Component", 0.8) $entry("queue Queue","\nKafka-Topic", 1) $entry("A -> B","Data-Flow") } }} endlegend @enduml ...

The following WebFilter redirects incoming requests for / to a static HTML file, index.html. ToIndexPageRedirection.java package com.hascode.tutorial; import org.springframework.stereotype.Component; import org.springframework.web.server.ServerWebExchange; import org.springframework.web.server.WebFilter; import org.springframework.web.server.WebFilterChain; import reactor.core.publisher.Mono; @Component public class ToIndexPageRedirection implements WebFilter { @Override public Mono<Void> filter(ServerWebExchange exchange, WebFilterChain chain) { if (exchange.getRequest().getURI().getPath().equals("/")) { return chain.filter( exchange.mutate().request( exchange.getRequest().mutate().path("/index.html").build() ).build() ); } return chain.filter(exchange); } } Ressources: JavaDocs WebFilter