outbox pattern pattern
When you need to update a database AND publish an event atomically: write the event into an outbox table in the same transaction as the data change. A separate relay reads the outbox and publishes downstream. Avoids the dual-write problem.
When you need to update a database AND publish an event atomically: write the event into an outbox table in the same transaction as the data change. A separate relay reads the outbox and publishes downstream. Avoids the dual-write problem.
symptoms
- data and events drift out of sync
- lost events when the publish step fails after the DB commit
- phantom events when DB rolls back but publish succeeded
causes
- dual writes: commit to DB then publish to Kafka, or vice versa
- no atomic boundary across DB and broker
fixes
- write event row in same DB transaction
- relay process (Debezium, custom poller) publishes
- ensure idempotent consumers
you might say
- outbox the event
- use an outbox table
- transactional outbox