Working through my Kafka notes, I keep coming back to one question: what exactly have I promised when I say a message is processed?
There are at least two answers. My database may have accepted a change. Separately, Kafka may have recorded how far the consumer group can resume. Those facts can become true at different times.
Put a crash between the two writes
Consider a fictional consumer updating an order projection. It reads offset 42 from one partition, writes the result to a database, and then intends to commit the next position, 43.
Now stop the process after the database transaction succeeds but before the offset commit succeeds. On restart, the earlier committed position can cause record 42 to arrive again. The database write already happened; the consumer’s recovery position does not prove that.
Moving the offset commit before the database write creates the opposite problem: a crash can leave a record behind the recovery position even though its business work never completed. Kafka’s consumer documentation describes this failure window and defines the committed position as the next offset to consume.
The diagram assumes a simple sequence with consecutive offsets. Its numbers illustrate the ordering problem, rather than an algorithm for calculating offsets in every Kafka workload.
Manual acknowledgement is a timing decision
My Spring Cloud Stream notes introduce manual acknowledgement as a way to decide when processing is considered complete. That is useful, but it is not an atomic transaction spanning Kafka and an unrelated database.
I also need to distinguish calling the acknowledgement API from proving that a broker commit completed. The framework’s acknowledgement mode and configuration determine how that request becomes a commit.
For a handler that performs a side effect, the review question is therefore larger than “Did we call acknowledge?” It is “What will this handler do if the same business event arrives again?”
Design the second delivery deliberately
For the fictional order projection, I would consider a processed-event table with a unique event ID. Inside one database transaction, the handler would claim that ID and update the projection. If the ID already exists, it would treat the event as previously applied.
That design needs both writes in the same transaction. Recording the ID first and updating the projection later would introduce another crash gap. A separate “does this ID exist?” query is also insufficient by itself: concurrent handlers could both observe that it does not.
This is a design sketch, not an implementation I tested. It also has a boundary: a local database transaction cannot roll back an email or a remote API call. Those effects need their own duplicate-handling strategy and, where available, an idempotency key understood by the receiving service.
The question I want in my next test
Instead of testing only the successful path, I want to stop the handler immediately after its database commit, restart it, and inspect the business result after redelivery. One order update should remain one order update.
That test would tell me more about correctness than a log line saying “acknowledged.” An offset is a recovery marker. My application still has to define what repeated execution means.
Adapted with AI assistance from my Kafka acknowledgement and offset study notes. The order projection is an illustrative scenario; no broker, crash experiment, or production workload was run for this article.