Kafka vs Direct DB Writes for the Ledger

Why I put an event log between the API and the database, and what it cost me in latency to get durability I could actually trust.
Kafka vs Direct DB Writes for the Ledger
FIG. 01 — KAFKA VS DIRECT DB WRITES FOR THE LEDGEREST. 2026.02.18 | A.LABS

My first pass had the API server write directly to Postgres on every transaction request. It worked in testing, but under simulated load with concurrent writers, I started seeing occasional lock contention on hot rows — and worse, if the DB write failed partway through, there was no clean way to retry without risking a duplicate.

"Durability is achieved when the writer no longer trusts the database — only the append log."

Where It Broke

The real failure mode showed up during a chaos test where I killed the DB connection mid-write. The API had already told the client 'success' before the commit was confirmed — a classic distributed systems trap. I needed a durability guarantee that didn't depend on the database being reachable at the exact moment of the request.

The Event Log Fix

Now every transaction request first gets appended to a Kafka topic — that append is the actual point of durability. Consumers read from the topic and apply writes to Postgres asynchronously, with idempotency keys to handle replays safely. The API can now return success the moment the event is durably logged, not the moment the DB write completes. Added latency, but I traded it for a guarantee I can actually reason about.

Finis