write-ahead log (pattern) pattern
Before changing data in place, write a record describing the change to a sequential append-only log. On crash, replay the log to recover. The log is the source of truth; everything else is a derivable view. Underpins almost every database, file system, and replication protocol.
Before changing data in place, write a record describing the change to a sequential append-only log. On crash, replay the log to recover. The log is the source of truth; everything else is a derivable view. Underpins almost every database, file system, and replication protocol.
symptoms
- need durability + good write throughput
- need crash recovery without fsync per write
causes
- random writes to data files are slow
- partial writes during crashes corrupt files
fixes
- append-only log, fsync on commit boundary
- periodic checkpoint to flush dirty pages
- log shipping for replication / CDC
you might say
- WAL it
- replay the log
- logical decoding