Projects & case studies

Reporting queries ran against the same MySQL primary that served the application. As the dataset grew, a single analyst refreshing a dashboard could add noticeable latency to ordinary writes. The goal was to move analytical reads off the transactional database without introducing a nightly batch window.

The pipeline reads the MySQL binary log with Debezium, publishes change events to Kafka, and materialises them in ClickHouse. Deletes and updates are handled with a ReplacingMergeTree keyed on the primary key plus a monotonic version column, so the analytics tables converge on the same state as the source without requiring in-place mutations.

The interesting part was not the happy path but the operational envelope: schema changes, snapshot restarts, replication lag alerting, and deciding what "eventually consistent" is allowed to mean for each dashboard.

Teams frequently rebuild an image per environment, which quietly breaks the guarantee that what was tested is what ships. This pipeline builds a single image, tags it with the commit SHA, and promotes that same digest through staging and production.

Pull requests run lint, unit tests and a container build. Merges to the default branch push the image to a registry, run integration tests against it, and then deploy. Deployment jobs use GitHub environments so production requires an explicit approval and holds its own secrets.

Caching is deliberately conservative: dependency caches are keyed on lockfile hashes, and layer caching is scoped per branch so a poisoned cache cannot silently leak into main.

Adding read replicas is easy; routing to them safely is not. This work introduced an explicit routing layer plus a small set of rules about which reads are allowed to be stale.

Sessions that have just written are pinned to the primary for a bounded window, so a user never fails to see their own change. Everything else — listings, search, reports — is free to read from a replica. Lag is monitored and the router removes a replica from rotation when it falls behind a threshold.

The site you are reading. The frontend is fully static and ships almost no JavaScript; the only dynamic surface is a single Worker endpoint that accepts contact submissions.

That Worker verifies a Turnstile token server-side, applies persistent rate limiting backed by D1, stores the submission with an HMAC-derived IP identifier rather than a raw address, and only then attempts an email notification. Storage and notification are separate states, so a notification failure never loses a message.

D1 persists to disk through Wrangler locally, and the test suite runs offline against a real SQLite database with a stubbed fetch — but email and Turnstile always go through the real services, so a local send is a real send.