change data capture pattern
Stream every row-level change in a database to other systems by tailing the write-ahead log. The DB's own log becomes the source of truth for replication, search-index sync, cache invalidation, and event-driven integrations.
Stream every row-level change in a database to other systems by tailing the write-ahead log. The DB's own log becomes the source of truth for replication, search-index sync, cache invalidation, and event-driven integrations.
symptoms
- stale search indexes / caches
- fragile dual-writes between DB and downstream
- missed events when application code forgets to publish
causes
- batch ETL too slow
- application-level event publishing is fragile / forgotten
fixes
- Debezium, Postgres logical decoding, MySQL binlog readers
- publish to Kafka, downstream consumers subscribe
you might say
- CDC the table
- Debezium it
- stream the WAL