writeonce/docs/superpowers/specs/2026-08-28-wal-checkpoint-design.md

WAL checkpoint — design

databasev2 3, chain 6. Brainstormed and approved 2026-08-28, after databasev2 4 part A landed.

One sentence: compact the log by rewriting it as one record per live row into a temporary file, then rename it over the live WAL — so recovery is unchanged and crash safety comes from the filesystem.

Decisions taken (the brainstorm's forks, settled)

Fork	Decision
Snapshot format	None. The compacted log is the snapshot, in the existing record grammar
One source or two	One. Rewrite + atomic rename; boot logic is untouched
Trigger	Volume only, as a ratio against the last compaction's own size, with an absolute floor. No timer — see below
Write availability	Stop-the-world, measured against a stated budget rather than assumed acceptable
Composition with group commit	Compaction runs only where nothing is staged — immediately after a barrier
resident: keys (iteration 2)	Compaction rebuilds the offset map as it writes. It cannot be left to discover this later

Why one file, and why Postgres cannot do it

Postgres was read for this (.dev/reference/postgresql), and the conclusion is that its design is unavailable to us — which is what makes the simpler option legitimate rather than lazy.

	PostgreSQL	writeonce
Where data lives	heap/data files; the WAL is a redo tail	the WAL is the only durable form, replayed into RAM
WAL contents	page deltas and full-page images	full row images — apply_record implements UPDATE as remove-then-recreate
Compaction	never; segments before the redo point are recycled by rename or unlinked	possible, because a log of row images is a complete store
Bounded replay	recovery starts at the redo LSN in the control file	recovery starts at byte 0 of a shorter log
Crash safety of the switch	control file written in place, full block, torn writes caught by CRC32C (update_controlfile)	one rename
Trigger	CheckPointTimeout (300 s) or WAL volume (XLogCheckpointNeeded)	volume only
Pause	none; flush is spread over time in a separate process	stop-the-world

Postgres cannot compact its WAL because a compacted redo log is not a store — its records describe changes to pages that live elsewhere. Ours describe whole rows, so the compacted log needs no companion. That single difference removes the control file, the redo pointer, the second recovery source, and the separate process from our design.

What is worth porting is not the architecture but the ordering discipline: publish the new "recovery starts here" atomically and last, so a crash at any instant falls back to the previous state with nothing to undo. Postgres achieves that with a redo pointer computed at checkpoint start and a control file updated at the end. We achieve the same property with rename, in one syscall, because we can swap the entire data set atomically and Postgres cannot.

Correction to a prior exploration doc. docs/plan/exploration/postgresql/buffer-and-checkpoint.md states that Postgres updates its control file by rename ("the same in BasicOpenFile + fsync_parent_path"). It does not — update_controlfile opens the existing file O_WRONLY, writes a zero-padded full block in place, and relies on CRC32C to detect a torn write. That doc also assumes writeonce has segment files ("records before that LSN are known to be in the segment files"), which it does not and, per databasev2 2, deliberately will not. The doc predates the databasev2 direction and should be annotated rather than followed.

The design

Compaction

Run on the owner shard, which owns the WAL. Walk each class's live rows — the bitmap-over-slabs walk that three call sites in db.c already perform — and append one INSERT record per live row to a new file, using the existing append path. No new encoder, no new decoder, no format.

Then: fsync the new file, rename it over the live path, fsync the parent directory (the rename's atomicity is in-kernel; the directory entry is not durable until the parent is synced — Postgres does the same, and the existing exploration doc is right about this part), and reopen the WAL descriptor, because the old one now refers to an unlinked inode.

Every crash point is safe without any recovery logic of ours. Before the rename, the live WAL is untouched and the temp file is garbage. After it, the new log is complete by construction. There is no window in which a reader could see a mixture, so the acceptance criterion — "recovery produces the same consistent store as if the checkpoint had never started" — is satisfied by rename, not by code we must get right.

Two obligations follow. Boot must unlink a stale temp file if one is present, because a crash mid-rewrite leaves one behind and it must never be mistaken for data. And the temp file must be zero-padded beyond its records exactly as the live WAL is, because the tail scan identifies the end of the log by a zero length field.

When it runs, and where in the sequence

The point matters more than the policy. The drain stages records into one buffer and commits them together; compaction rewrites the file those records would land in. So compaction may run only when nothing is staged — in practice, immediately after a barrier, before the next statement is served. Anywhere else and a staged record would either be written to a file about to be replaced, or be lost with it. This is the normative ordering rule that 04-db-binding.md must carry.

Trigger: volume, as a self-tuning ratio. Compact when the WAL's used bytes exceed a multiple of the bytes the last compaction wrote, with an absolute floor so a small store never bothers. The denominator is known exactly — the compactor wrote it — so this needs no estimate of the live set's size, which is not cheaply knowable when rows hold Text. The floor exists because a store whose whole log is a few hundred kilobytes has nothing to reclaim.

No timer, and that is a deliberate difference from Postgres. Postgres needs CheckPointTimeout because its dirty buffers are not durable until flushed — an idle-but-dirty system must still checkpoint or it loses data. Our records are already durable at commit; a checkpoint reclaims space and shortens boot and nothing else. An idle system's log does not grow, so a timer would fire with nothing to do. Adding one would be copying Postgres' mechanism without its reason.

A manual trigger exists for tests, because a policy that can only be observed by waiting is a policy that cannot be tested.

The pause, and how it is judged

Compaction is stop-the-world: the owner shard rewrites the log as one long operation while no statement is served. This is the simplest correct thing, and part A's own experience argues for measuring before buying complexity to avoid it. The dump is O(live rows) encodings plus one write and one barrier, so the expectation is that it is fast — but an expectation is not a measurement, and the proof plan below states the budget it must meet.

If the measured pause exceeds the budget, that is a finding and a follow-up, not something this iteration solves by adding concurrency. The alternative designs (incremental copy, fork-and-dump) cost exactly what Postgres pays, and should only be bought against a number.

The interaction that will otherwise be discovered late

Compaction invalidates every stored WAL offset. Rewriting the log moves every record, so any offset captured from the old file is meaningless afterwards — not stale-but-readable, but pointing at an arbitrary byte of a different file. Iteration 2's resident: keys stores exactly such offsets, one per row, and reads rows back through them.

The compactor therefore rebuilds the offset map as it writes: it is emitting the new records and knows each one's new position, so this is the cheap direction and the only one that keeps both features usable together. The alternative — forbidding compaction while any resident: keys table is live — would mean the feature that exists to handle huge tables is incompatible with the feature that stops their log growing forever.

This is recorded here because iteration 2's storage half is not yet implemented, so nothing will fail today. It will fail later, in a way that looks like data corruption rather than a design gap.

Proof plan

Claim	How it is proven
Space is reclaimed	An aged store shrinks. Measured today on master: seed 20000 gives a 986 614-byte log; 20 000 updates take it to 2 590 262 bytes with the same live rows. Compaction must return it to approximately the former
Replay is bounded	Boot time on the aged store before and after, recorded. Measured today: boot+verify on that store is 155 ms
Crash safety	kill -9 at many instants during compaction, then replay: the store must equal either the pre-compaction or post-compaction state, never a mixture, and no acked write may be missing. This is the criterion the whole design is shaped around, so it gets the crash battery's treatment rather than one case
A stale temp file is harmless	Boot with one present, containing plausible records: it is removed and never read
The pause is known	The stop-the-world pause measured on the largest store the harness builds, recorded as a number with a stated budget — not asserted to be acceptable
The ordering rule holds	Compaction with records staged must be impossible by construction; a test that stages and then requests compaction must find it deferred, not executed
Nothing regressed	The full battery, and specifically part A's wmix legs: compaction must not change the ack contract or the batching it introduced

Out of scope

• A second file, a control file, or a redo pointer. The Postgres shape, priced above and not needed once the log is self-sufficient.
• Avoiding the pause. Incremental or forked dumps are bought against a measurement, not in advance.
• Per-shard compaction policy. One owner shard owns the WAL today; when that changes, this decision is revisited with it.
• Compacting away tombstones across shards, or any cross-shard coordination. There is one log.
• io_uring for the rewrite — part B of databasev2 4, whose premise is already under revision.
• Changing the record grammar. The entire argument for this design is that the grammar already suffices.

Alternatives rejected

Snapshot + WAL tail (the Postgres shape). Rejected because it buys write availability at the cost of a second recovery source, a cutoff offset, a control file with its own torn-write detection, and a crash-safety guarantee that depends on our ordering rather than on rename. Postgres pays this because its log cannot stand alone; ours can.

Compacting in place. Rejected outright: there is no crash point at which a partially rewritten live log is recoverable, and it trades the one property that makes this design defensible for nothing.

A timer trigger. Rejected with a reason rather than on taste: Postgres' timer exists to bound data loss from unflushed buffers, and we have no unflushed buffers. An idle log does not grow.

A ratio against an estimated live-set size. Rejected in favour of the last compaction's measured output, because estimating the live size means estimating Text, and the compactor already knows the true number.