- test_oracle_all_vs_keys_same_update_sequence continues the shared
sequence 3×WO_DELTA_MAX_HOPS steps, alternating scalar and Text, and
asserts the `resident: all` and `resident: keys` rows equal after EVERY
step; the fold's hop count proves the chain terminated at least twice and
never exceeded K; then the keys log replays into a fresh store and is
compared against the oracle once more — the criterion as written, which
the story carried as ⚠ "an expected value, not an oracle table"
- wal.h: wo_wal_append_row_image's comment claimed the flattened image is
written as WO_WAL_INSERT; it is WO_WAL_UPDATE — an INSERT would replay as
a duplicate id; compaction alone writes INSERT, into a FRESH log — as 11
landed it and its story recorded
- story 11: the criterion flips to ✅ naming the test; the sequencing note
and out-of-scope bullet record task 7's 2026-08-30 measurement (16× vs
105× collapse under a cap, 1.53× faster than swapping) — the work stands
- test_wal 6295 → 6880 pass, 0 fail; 21 runtime suites 0 fail
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
(cherry picked from commit d841390f3087a0c2542ddf23f25f15037a7d4d71)
- Second half of the fresh-log seed SEGV: every `*msg = ...` in the fold
was unguarded, and `wo_idx_probe` (table.c:373) borrows with `msg == NULL`
because a candidate that does not fold is simply not a hit; a malformed
record under an index probe was therefore a zero-page write.
- Guard: `const char *sink; if (!msg) msg = &sink;` at the top of the fold;
wal.h documents [msg] as optional. A future malformed record refuses the
candidate by name instead of segfaulting.
- Failing test first: `test_fold_row_at_tolerates_null_msg` (test_wal.c) —
head-only log, fold at offset 0 (schema record) and past the tail with
`msg == NULL` -> -1 both; with a real `msg` the names "record header is
malformed" / "no intact record at that offset" still arrive. Pre-guard:
ASan SEGV `wo_wal_fold_row_at wal.c:1886` from the test.
- Gates: test_wal 6660/0 (was 6650); `make -C runtime test` 21 suites
8462/0 (was 8452); wovm-asan clean; residency `seed` fresh dir + fresh
app.db rc 0 under wovm_asan.
- CODE-LOGIC §Schema migrations bullet extended with the guard + test.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
(cherry picked from commit 1b6750d78db991af464994c2188d219519dfe16f)
- `seed` of docs/examples/residency (`resident: keys`) on a FRESH WO_DATA
segfaulted rc 139 in both the dir and the file form; pre-existing.
- Root cause: the databasev2 12 head record was staged lazily INSIDE the
first `wo_wal_append_*` (wal.c `stage()`), after db.c:78/293 had read
`koff = wo_wal_next_offset(w)`; the first keys-resident row was re-pointed
at the schema record and its first read folded "record header is
malformed"; `wo_idx_probe` borrows with `msg == NULL` -> zero-page write.
- Fix: one helper `stage_schema_head` shared by `stage()`,
`wo_wal_ensure_schema` and `wo_wal_next_offset` (no longer a pure inline):
the head is staged before any caller observes `off + len`. Still lazy,
never for a log that stays empty; head-stage OOM is `wo_wal_stage_fatal`.
db.c untouched; compaction/migrate stage the head explicitly, unaffected.
- Failing test first: `test_keys_resident_fresh_log_first_row` (test_wal.c),
the db.c:78 sequence call for call, then read-by-id, `wo_idx_probe`,
replay. Pre-fix: `koff != 0` FAIL, `wo_row_read` -1 "record header is
malformed", ASan SEGV `wo_wal_fold_row_at wal.c:1871` via `table.c:373`.
- Gates: test_wal 6650/0 (was 6629); `make -C runtime test` 21 suites
8452/0 (was 8431); wovm-asan clean; residency `seed` + restart `order`
under wovm_asan rc 0 on a fresh dir AND a fresh app.db; a control build
with wal.c/wal.h reverted reproduces the SEGV.
- CODE-LOGIC §Schema migrations: "Head before any offset capture" bullet.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
(cherry picked from commit 631007839451fb970e9dec83338d19c09b3043ab)
- `wo_wal_resolve_data_path` (database/src/wal.c|h): an existing directory
or a trailing `/` → `<dir>/shard-0.wal` byte for byte (the `//` after a
trailing slash included); otherwise the path IS the log — opened if a
regular file, created by `wo_wal_open` if absent under an existing parent.
- Refusals as codes for main.c: WO_WAL_PATH_NO_PARENT (out = the parent, so
the line names it), WO_WAL_PATH_NOT_A_FILE (fifo/socket/device),
WO_WAL_PATH_TOO_LONG (today's snprintf truncated silently).
- `parent_dir_of` shared by the resolver and `sync_parent_dir`: the parent
checked at boot IS the parent fsync'd after a compaction/migration rename.
- runtime/src/main.c: the resolver replaces the unconditional
`"%s/shard-0.wal"`; each refusal is one stderr line, exit 2 through the
6a destroy sequence; never mkdir -p. The 6a block is untouched.
- Failing first: test_resolve_data_path — 4× -Werror (implicit declaration
+ three undeclared codes); green after: 21 assertions (dir, trailing
slash, absent file, regular file, bare name, missing parent, parent is a
file, fifo, two too-long).
- `make -C runtime test`: 21 suites, 8374 pass / 0 fail (was 8353/0).
`make -C runtime wovm-asan` clean; smoke: file form seeds + replays with
`app.db` the only artifact; missing parent and fifo refuse rc 2 naming
path + parent; dir and trailing slash unchanged; WO_EPHEMERAL conflict
inherited from 6a.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
(cherry picked from commit b31bd4052624be460de1c18cf208661539397e09)
- main.c builds the compiled schema (names out of the constant pool,
which the database layer never sees), peeks the log head before
replay, and diffs: match replays as-is, add/delete migrates through
the transcode, poisons refuse naming class, field and what to do
- fixed en route: a poisoned class SKIPPED the identity check, so no
transcode ran and replay greeted the shape mismatch with the generic
"corruption" — the exact message this iteration exists to replace.
A poison now forces the transcode, where it either bites with its
text or passes harmlessly when the class has no records
- the schema head is written LAZILY, ahead of the first real record:
an eager head broke the documented "durable: false writes ZERO
bytes" contract by 75 bytes and the residency gate caught it
- end-to-end at the language level: fresh boot seeds, identity
replays, +field migrates with "migrating `Note`: +flag" and reads 0,
retype refuses naming `val`, and the refused log still boots the
previous binary untouched
- gates: wovm-test all green (test_wal 5951/0), woc-test clean,
residency-accept 14/0, site-accept 23/0
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit b21943aa91152ebdcfc72bd4c9fba38730ab1c2f)
- wo_wal_migrate rewrites the log without touching db state: no id
maps, no indexes, no keys-resident logic — the new log replays
through the machinery that already exists and is already tested
- cids remap by name, INCLUDING the ones embedded inside stored owned
values (an owned value carries a cid on the wire); the embed closure
guarantees every nested class is shape-unchanged, so only numbers
move
- surviving fields go to their new slot, deleted fields' values are
freed, added fields take the kind's zero value straight from
enc_val(0)
- a delta on a deleted field is SPLICED out: an offset map (old record
start -> new) rewrites every back pointer, and the dropped delta maps
to its own target so later deltas step over it
- temp + fsync + rename, compaction's own crash discipline; a stale
temp is discarded at start; a torn tail bounds the intact prefix
exactly as replay does
- fixed en route: early `goto corrupt` jumped over initializers, so the
handler freed uninitialized memory — declarations hoisted above the
first jump
- six end-to-end tests: add, delete (ASan watches the freed Text),
reorder with owned fixup, delta splice on a keys-resident chain,
poison-bites-only-with-records, corrupt input
- test_wal 5951 pass, 0 fail
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit b69092a99206e1dcdf6f2dcdb02146939b0564c6)
- wo_schema_diff matches classes and fields by NAME, so declaration
reordering is identity apart from the cid map — the silent
cid-renumbering hole closes as a side effect
- owned-field references (fclass) compare by the NAME the number
resolves to, never the number: a raw compare would false-poison
retype on every pure reorder
- refusals are per-class POISONS carried in the plan, not diff errors:
a poison bites only when a record of the class is met, so a retyped
class with no stored rows never blocks a boot
- poison set: retype, same-shape delete+add (a disguised rename, one
reading destroys a column), vanished class, storage-flag change, and
the embed closure — any class whose old records carry values of a
class whose shape changed, iterated to a fixpoint
- identity plans skip the rewrite entirely; a NEW class in the binary
does not break identity (no records; the head refreshes at the next
compaction)
- ten verdict tests; test_wal 5778 pass, 0 fail
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 63a063b822af381a10c2e599c58cf3455d9c5bf7)
- new record kind 5: class and field NAMES, kinds and the two
encoding-relevant metadata words (field_class, field_elem), CRC-framed
like every record. Index layout deliberately absent: indexes rebuild
from rows at boot and never touch record bytes
- names are byte pointers, not constant-pool indices — the database
layer never sees the module's consts, so the runtime resolves them
once; a decoded schema owns a private copy of its bytes
- wo_wal_set_schema adopts the compiled schema; wo_wal_ensure_schema
writes it as a fresh log's first record; compaction writes it at the
head of every replacement, which is how a legacy log becomes
self-describing without a migration step of its own
- apply_record skips it BEFORE reading cid/id (its class count would be
misread as a cid and bounds-refused); replay does not count it
- schema unset = byte-for-byte today's behaviour: all 5700 prior
assertions pass untouched; four new tests cover roundtrip, fresh-log
head, legacy adoption via compaction, and absent/empty files
- test_wal 5743 pass, 0 fail
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit ba8519fa5e39c6ed6a3504d39e5a372f8decd045)
- flattened row image is WO_WAL_UPDATE, not WO_WAL_INSERT: the row's
original INSERT is already in a live log, so a second one for the same
id is a duplicate replay refuses as corruption. INSERT is right only
for compaction, which builds a fresh log
- remove WO_CKPT_MAX_GARBAGE: with the absolute term at 64 MiB, garbage
large enough to reach a 256 MiB ceiling has already tripped it, so the
branch was unreachable. Postgres needs both constants because it
thresholds on tuples with its pair at opposite ends; this thresholds
on bytes, where one constant does both jobs
- test_delta_chain_flattens_at_k: chain depth stays <= WO_DELTA_MAX_HOPS
across 2K+2 updates, and a reset is observed
- test_delta_chain_flatten_replays: a flattened chain replays correctly
- test_keys_resident_indexed_across_flatten: a delta on an indexed
column composes with flattening, checked at every step across the
bound and after restart. Found no product defect
- test_should_compact_absolute_and_ceiling: pins the absolute term, the
boundary just under it, and the small-log case the ratio still governs
- test_wal 5700 pass / 0 fail; wovm-test and woc-test green
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit f93b5d9db753305c297e868d977670e6d703684c)
TESTS DELIBERATELY HELD at the developer's instruction — logic only.
The existing suite passes (36 suites, 0 fail) but exercises NEITHER new
behaviour: nothing builds a 16-deep chain, and no checkpoint test uses a
log near 64 MiB. Green here means "did not break what existed".
- tier 1: wo_wal_fold_row_at gains hops_out. The walk already visits
every hop, so the depth is free — this is the design's pd_prune_xid,
a cheap "is work worth doing" hint taken from work already happening
- the update path branches on it: past WO_DELTA_MAX_HOPS (16) it writes
a full-row image instead of a delta, terminating the chain. `r`
already holds the complete post-update row because index maintenance
required folding it, so flattening costs bytes, not an extra read
- wo_wal_append_row_image encodes from a caller-held row, as
WO_WAL_INSERT: a chain's base must replay into a database where
nothing precedes it, so replay/compaction/fold need no change
- tier 2: should_compact gains a TRIGGERING absolute term and a ceiling.
Our `floor` SUPPRESSES on a small log — the opposite of postgres's
vac_base_thresh, which triggers on a small absolute problem the
proportion hides. We had the proportion and the suppressor and
neither real guard
- verified by construction, not test: both update entry points converge
on row_apply_field_keys; db.c captures next_offset BEFORE calling in,
so the re-point is transparent to which record type was written
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 1b808abd5942de81c3a6416714d1302384103040)
- wo_wal_pend_repoint's failure was silently discarded (db.c); its
own doc claimed replay reconciles a stale map — false, a second
same-drain update chains past the lost one, permanently. Now
fatal, like wo_wal_stage_fatal; comment corrected
- apply_delta dereferenced db->rt->wal unguarded — NULL rt + any
DELTA record was a crash. Now refuses cleanly (-1)
- wo_wal_replay_ex lent its throwaway view only when rt->wal was
unset, so a live wal's non-empty staging buffer could be folded
against during replay. Now installs unconditionally whenever rt
exists, saving/restoring whatever was there
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit fed9fe8b5fe022f0b22a4170c7fd68008700939f)
- db.c: guard both WO_B_DB_UPDATE_FIELD arms on keys-resident tables —
wo_wal_append_update read a NULL wo_row_ptr there; a live crash, fixed
- ruling override on Task 3: row_apply_field_keys no longer commits or
moves the id map — stages the delta, does the index swap (RAM apply,
unconditional past the shadow-check; a stage failure past that point
is now fatal, like insert). Commit/re-point move to the caller,
mirroring insert. table.c's WAL commit removal is this ruling, not a
regression
- offset passed back via caller-side wo_wal_next_offset(), insert's
koff pattern
- inline arm commits then re-points; request arm records
wo_wal_pend_repoint (own list/name — a drop and a re-point differ),
flushed by wo_db_flush_drops after the barrier
- back_off checks the pending re-point before the durable offset, else
a second update in one drain skips the first delta; verified failing
this way, passing after
- wal.c: fixed a stale comment — keys-resident updates CAN reach
wo_wal_append_update's caller now, they just never call it
- test_wal.c: 2 tests updated for the new contract; new test drives 2
same-row updates via wo_row_update_field_slot in one uncommitted
"drain", checks the value and delta 2's on-disk back-pointer
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 4d13bcebfe51936ba8c608dd9e791c784e5b8983)
- wal.c: wo_wal_fold_row_at now refuses any delta back-pointer that
does not point strictly earlier than the record naming it
(back_off >= cur), instead of capping total hops at off/13+1
- this is the real invariant, not a proxy for it: a step-count bound
lets a forward-pointing back-pointer through in one hop whenever it
happens to land on a genuine record, returning a plausible-but-wrong
row instead of refusing it
- removes the 13-byte-record magic number entirely; no arithmetic
tied to record framing remains in the guard
- wal.h: docblock updated to describe the direction invariant
- test_wal.c: two new tests — self-pointing back-pointer (boundary
case, back_off == cur) and forward-pointing back-pointer to a real
future record for the same row (the actual gap: verified failing
against the old step-count guard, passing after the fix)
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 173dbf28a42d45a8c7f9fe430355f61f70c81935)
- wal.h/wal.c: wo_wal_fold_row_at — THE fold. Walks BACKWARD from an
offset through WO_WAL_DELTA records, remembering the first value
seen per field index (newest wins, since newest is seen first),
stops at the first INSERT/UPDATE, decodes it, overlays resolved
fields. Returns ENGINE-owned values so reads, replay, and
compaction (Tasks 3/5) can all build on the same output.
- Cycle guard: caps the walk at what the log up to the starting
offset could possibly hold (13 = scan_record's own record-size
floor), so a corrupt or malicious back-pointer fails loudly
instead of spinning.
- table.c: wo_row_borrow's keys arm now calls the fold instead of
wo_wal_read_row_at directly, then VM-decodes the result — same
two-stage pattern wo_wal_read_row_at used internally. Per-table
scratch, scratch_busy nested-borrow refusal, and the cid/id
identity check all preserved unchanged.
- resident: all path (wo_row_ptr) untouched.
- test_wal.c: two new tests — deltas on two different fields (changed
fields take the new value, the untouched field keeps its original)
and two deltas on the SAME field (the newer wins, pinning direction
— a reversed fold would pass with the older value instead).
Verified failing pre-implementation (wo_row_borrow returned NULL
since a delta record isn't INSERT/UPDATE) and passing after.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit a60231cde1d49d74743cedbd134d2da11158b70b)
- enum: add WO_WAL_DELTA = 4, existing 1/2/3 untouched (on-disk logs)
- wal.h: document kind 4's payload shape in the format docblock
- wal.h: declare wo_wal_append_delta(w, db, class_id, id, field_idx,
back_off, value) — back-pointer taken as a parameter, not looked up,
keeping the encoder ignorant of table/map state
- wal.c: implement it, modeled on wo_wal_append_insert's shape —
wput_u8/u32/u64 the header fields, enc_val the one field, stage()
- test_wal.c: new test_delta_record — stages a delta after an insert,
commits, then preads the raw record and asserts kind/class/id/
field_idx/back-pointer/value all round-trip; registered in main()
- nothing reads deltas back yet — decode/apply is a later task
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 9c6f832c0534a59e644c53b7cd2850581da12159)
databasev2 2, task 5c. The write and boot halves. Still not exposed: the
loader refuses `resident: keys` until 5d rewires the readers.
GROUP COMMIT FORCED THE DESIGN. A keys-resident payload can only be dropped
once its record is durable, but databasev2 4 deferred the barrier to the drain
— so at append time the bytes are still in the staging buffer and the recorded
offset would pread ZEROS. Dropping at append would have produced rows that
read as garbage, intermittently, only under multi-shard load.
So the drop is recorded, not performed:
- wo_wal gains a pending-drop list, the same shape as the drain's held replies
and for the same reason
- both write paths take the offset BEFORE the append (wo_wal_next_offset) and
record it; the inline path flushes right after its own commit, the request
path's flush runs in the drain immediately after the barrier
- if the process dies before the barrier the list dies with it, which is
correct: nothing was dropped and nothing was lost
- an out-of-memory pend is ignored on purpose — the row simply stays resident,
which is safe
Boot: replay now leaves a keys-resident table pointing at the LOG. Each record
is applied normally, so indexes and uniqueness are built exactly as for any
other table, and the payload is then dropped with THAT record's offset. For an
update the later record wins, because each apply overwrites the map in order —
the rule replay already follows.
Tests: the round trip (insert, commit, drop, read back with Text intact) and
now BOOT — a fresh wo_db replays the store and every row materialises from the
log, count intact, nothing in a slab.
Verified: just wovm-test — 36 suites 0 fail, test_wal 4301 pass, cli_smoke OK.
REMAINING (5d), and precise: every reader still goes through wo_row_ptr, which
for a keys table would index a freed slot. The scans in db.c walk the BITMAP,
and a keys table's bitmap is empty by construction — so a query over one would
today return no rows at all. That, FK restrict, and the @unique shadow are 5d,
and the loader refusal stays until they land.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 08abd09bf88918f2582e74713dc7903beb8aaeb8)
The branch was 17 ahead / 25 behind with 11 conflicting files, and drifting
further: db.c had been rewritten twice on master since (group commit, then
compaction). Resolved rather than rebased so both histories stay legible.
Conflicts, and how each was settled:
- db.c: BOTH semantics kept. Master's fatal path and compaction check now sit
behind the branch's `table_is_durable` predicate, in all three inline arms —
a volatile table reaches neither the barrier nor the compaction check
- db-bench sample: every mode from both sides (growth, growth-verify, randread,
replayseed, wmix) and ONE `boot` mode, which both sides had added
independently
- db-bench.py: all six legs kept. Both sides had also grown the same
WAL-size helper under different names; collapsed into one
- perf-targets: the branch's §5 (RAM ceiling) then master's §6/§7 — master's
numbering had already assumed a §5 it did not have
- story frontmatter: master's `status` (the landing truth) plus the branch's
`readiness` axis. 03 would have read `done` + `refine`, which is a
contradiction — it was brainstormed and landed on master, so `ready`
- board: both standup blocks newest-first; master's chain rows (a superset);
the branch's databasev2 1-2 rows with master's 3-4. Fixed a stray `|` in
master's row 3
- baseline: master's, then REGENERATED from a full campaign — 143 metrics,
132 checks, 0 failures with both sides' legs present
TWO HALF-EXPOSED FEATURES FIXED, because the merge rule is that master gets
no feature that is honoured in name only:
- `resident: keys` PARSED, set a .wob flag, and did nothing: rows stayed fully
resident. A developer could declare a 120 GB table keys-resident, watch it
compile, and be OOM-killed. The loader now REFUSES it with a message naming
what to write instead, until tasks 5c/5d land. The compiler still parses it
and its AST golden still passes, so the grammar work stays tested
- `durable: false` was honoured ONLY on the inline path. wo_db_exec_req had no
guard at all, so a volatile table written from an actor on a worker shard
would still be logged — precisely porch's session-table case, and precisely
what iteration 2 exists to provide. All three request-path arms now carry the
same predicate. Found by reading the merged code, not by a test: the obvious
probe runs main() on the primary and therefore only exercises the inline path
Verified on the merged tree: wovm-test 0, woc-test 0, oop-e2e 122/0,
residency-accept 8/0, db-bench 132/0, linkcheck clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
databasev2 3, task 5.
Full campaign, same workload twice, differing only in whether
checkpointing may fire:
- WAL used 1962358 -> 907094 bytes (2.16x reclaimed)
- boot 114 -> 64 ms (1.78x), median of 3
- stop-the-world pause max 2651us against a STATED 50ms budget
The budget is asserted, not assumed: 50ms is a stall a serving process
can absorb without a client seeing a timeout, and the leg fails if it is
exceeded. The pause is O(live rows) — at ~181 MB/s a 1GB live set implies
~5.5s, which is the number an incremental design must be bought against.
The spec deliberately did not buy it in advance.
FOUND BY MEASURING: the dump was 8x slower than it needed to be. It
flushed through wo_wal_commit, which fdatasyncs, so it paid one barrier
per 256 records. Intermediate durability there is worthless — the temp is
not authoritative until the rename and is fsynced once immediately before
it. With a single final barrier:
- ~107KB live: 23948us -> 2903us
- ~500KB live: 36361us -> 7526us
- ~1.98MB live: 107649us -> 13212us
- marginal ~22 MB/s -> ~181 MB/s, sync-bound to bandwidth-bound
Correctness re-proven after that change: wovm-test 36 suites 0 fail,
test_wal 760 pass including the 40-round kill-during-compaction battery.
Two measurement defects of my own, fixed rather than reported:
- boot measured through the driver's run() helper reported 251ms both
with and without checkpointing — run() samples RSS on a 250ms poll, so
every timing floors at the quantum. Measured directly instead, median
of 3
- ckpt.reclaim_x was recorded as lower-is-better by the default detector,
which would have PASSED "reclaimed nothing" and FAILED an improvement:
the feature's central claim, gated backwards. Now higher-is-better,
gated at 15% while the wall-clock metrics stay wide — waiving them all
would have left the leg ungated, part A's task 4 mistake
- sample gains a `boot` mode that does nothing, so boot time is boot time
- walstats now reports compactions, pause max/total and compacted bytes
- baseline refreshed from the FULL campaign (N=20000, crash_reps=3), and
a fresh full run passes 116 checks 0 failures
- gate bites: reclaim_x doctored to 1.0 -> FAIL on exactly that metric
One flake seen and checked, not papered over: durable.sN.query.ops_sec
failed once at 53% below baseline. It is a read-only metric that touches
no WAL code, and a re-run passed 116/0 with the box at load 1.85.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
databasev2 3, task 4. The plan called this the riskiest task because a race
can pass by luck, so it is argued with mutants rather than green runs.
The battery: a forked child inserts, acks, deletes the oldest so HISTORY
grows while the live set stays ~17, and compacts every 24 iterations. The
parent SIGKILLs at varied instants so kills land before, inside and after
rewrites, then replays and checks the ACKED LIVE SET. The existing
battery's "records >= acks" oracle cannot be reused: collapsing history is
exactly what compaction is for.
A REAL DEFECT IN MY FIRST VERSION, found by the failures and fixed in the
TEST, not by weakening it:
- the child acked deletes AFTER committing them, so a kill in between left
the row legitimately gone on disk while the last ack still said
"inserted" — the parent then demanded a row the engine was right to
remove. Symptom was an acked insert missing near the end of the stream,
~1 run in 3
- deletes now announce INTENT BEFORE committing, so such a row's fate is
simply UNKNOWN to the parent, which is the honest thing to assert. Every
acked insert never marked for deletion must still be present with its
acked value
- the stale-temp assertion was also wrong: it checked for absence after
wo_wal_replay, which never opens the WAL. The guarantee is "removed AT
OPEN", so the test now opens and then asserts. A temp surviving a kill
is expected debris, not a defect
Proven to have teeth, which matters because assertions were softened:
- against the design's rejected alternative (in-place rewrite instead of
the atomic rename) it fails EVERY run, reporting log_records=0 — the
kill landed mid-copy and destroyed the log. That is the corruption
rename exists to prevent
- on correct code: 10 consecutive runs x 40 rounds clean, plus the suite
Also carried the log's PREALLOCATION to the replacement. The WAL is
preallocated so appends never extend the file, which is what lets
fdatasync alone be the ack barrier; a replacement opened with prealloc 0
silently changes that property and the zero-padded tail the scan relies
on. Stated honestly: this is hygiene making the replacement equivalent to
what open() would have produced — I could NOT prove it was the cause of
the observed loss, and the ack race above explains it.
Verified: just wovm-test — 36 suites 0 fail, test_wal 760 pass, cli_smoke OK.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
databasev2 3, task 3.
- wo_wal_should_compact is a PURE decision (used bytes, last compaction's
measured output, floor, ratio) so it is testable without a store —
which is the only way a policy like this gets tested at all. Denominator
is the last compaction's real output, not an estimate of the live set:
estimating would mean estimating Text
- 8 boundary assertions incl. "exactly 3x is not MORE than 3x" and a zero
ratio disabling the policy rather than dividing by nothing
- MUTATION-TESTED instead of observing RED: implementation and test were
written together, so removing the floor check was verified to fail
exactly the two floor assertions. Equivalent evidence, stated plainly
- WO_CHECKPOINT_BYTES / WO_CHECKPOINT_RATIO at boot beside WO_MAILBOX.
The knobs are what make the policy testable — a gate sets a tiny floor
and forces compaction in a few writes instead of megabytes
- NO timer, per the spec: Postgres' CheckPointTimeout bounds loss from
unflushed buffers; our records are durable at commit and an idle log
does not grow
- the ordering rule is now asserted, not trusted: a test stages a record,
requests compaction, and requires REFUSAL with the log untouched and
the staged record still committable afterwards
FOUND AND FIXED a gap in my own wiring. The plan said to call the check
"after the drain's barrier", and I did — but a statement running ON the
owner shard never enters that drain, so WO_SHARDS=1 never compacted and
its log grew forever: measured 536086 bytes where the multi-shard run
held 446024. Now checked after the inline path's commit too (db.c
maybe_compact), where the buffer is equally empty. WO_SHARDS=1 went
536086 -> 260657 bytes. For a checkpoint this mattered more than part A's
equivalent gap: an unbounded log is an operational failure, not just lost
throughput.
Also corrected a measurement of my own: multi-shard logs looked unbounded
(448KB -> 1013KB -> 1647KB across 8k/24k/48k updates). They are not.
Instrumentation showed compaction ran 25 times with zero failures, each
writing MORE than the last, because the live set genuinely grows — wmix's
hist_dump and done-markers are themselves durable inserts. Final log
1631040 against a last compaction of 866432 is a ratio of 1.88, just under
the 2x threshold: the policy holding exactly.
Replies are released BEFORE compaction runs, deliberately: their records
are already durable, and holding them across a stop-the-world rewrite
would add its full duration to their latency for nothing.
Verified: wovm-test 36 suites 0 fail, test_wal 360 pass; db-bench-quick
crash.s1/crash.sN and both restart legs green, and part A still batches
(sN mean 4.16, peak 30).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
databasev2 3, task 1.
- walks each class's live rows via the bitmap-over-slabs pattern db.c
already uses in three places, appending one INSERT per live row through
the EXISTING append path. No second encoder, no new format, and ids are
preserved exactly because wo_wal_append_insert takes the id and reads
the row from the store
- FLUSHES EVERY 256 RECORDS rather than staging the whole store: stage()
grows the staging buffer by doubling and never shrinks it, so a
one-buffer dump would hold the entire store in RAM on top of the store
— the unbounded growth databasev2 1 identified as how this engine dies
- the switch, in order: fsync the temp file, rename over the live path,
fsync the PARENT DIRECTORY (rename's atomicity is in-kernel; the
directory entry is not durable until the parent is synced — Postgres
does the same for the same reason), then reopen the descriptor, because
the old one refers to an unlinked inode
- REFUSES when anything is staged: those records would land in a file
about to be replaced. The caller-side guard is task 3; this is the
backstop
- a failure leaves the ORIGINAL log intact and usable and returns -1. A
failed checkpoint is a missed optimisation, not a durability event, so
it deliberately does NOT take databasev2 4's fatal path
- records the bytes written, so task 3's trigger can compare against a
measured denominator instead of estimating the live set (which would
mean estimating Text)
Recovery is untouched — the result is an ordinary log in the ordinary
grammar, replayed from byte 0. Crash safety comes from rename, not from
code of ours.
Test asserts BOTH halves, on purpose:
- the log shrinks: 43 records (3 inserts + 40 updates of the SAME row, so
history grows while the live set does not) -> 3 records, fewer bytes
- AND a fresh replay reproduces the store: every id present, and row 0
carries the 40th update's value rather than its original. "It got
shorter" is also true of a truncating bug, so the replay comparison is
what actually proves it
- and the WAL stays usable after the swap: a further append lands after
the compacted records, giving 4 on the next check
Verified: just wovm-test — 36 suites 0 fail, test_wal 315 pass (was 165),
cli_smoke OK.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
databasev2 4 part A, task 6. Mostly documentation, plus one real fix the
full battery caught.
THE FIX. The drain held EVERY DB reply until the barrier — including
reads, which stage nothing and have no stake in durability. That parked
readers behind an fsync for no reason: durable.sN.mixread.p99 rose from
~1043us to 4057us. Only a statement that actually staged a record now has
its reply held. Caught by the gate, not by review.
THE TRADE, recorded rather than smoothed over. What remains is inherent: a
barrier blocks the owner shard LONGER (more records per fsync) though LESS
OFTEN, so anything queued behind one waits. Three full runs of the same
build gave durable.sN.mixread.p99 of 1043 / 2318 / 4147us and wmix.p99 of
8758 / 20000us — a 2-4x spread with the box near idle. So part A buys ~3x
write throughput at the cost of a longer, noisier tail on the owner shard,
and that is the strongest argument for part B (submit and keep serving).
- durable.sN.*.p99us tolerance widened to 100% WITH the reason in the
code: a 2-4x-variable tail gated at 50% gates the disk, not the engine.
The floor is the real guard and is not slack — mixread's (4172us) came
within 25us of tripping on the worst run. Baseline refreshed; a fresh
full run then passed 106 checks 0 failures
EXIT STATUS MOVED 3 -> 74 (sysexits EX_IOERR). 3 and 4 are already used by
SAMPLES for their own meanings — db-bench's own `verify` exits 3 on a
checksum mismatch, and it is the gate that exercises durability, so a
durability abort exiting 3 would have been indistinguishable from the
mismatch it should help diagnose. The low range belongs to programs.
Docs:
- story: progress, the payoff measured two ways, the cost side, criteria
split met/outstanding, and a "part B — its premise changed" section:
it was justified by "close the 66x gap", but that gap is two problems
and only the concurrent one was a batching problem
- board: standup entry in the six-question shape; both databasev2 4 rows
rewritten. They had said "close the 66x gap" — recorded as MIS-STATED
rather than quietly renumbered
- 00-wob-format.md and 04-db-binding.md: the normative failure contract
("a failed WAL commit traps WO_T_IO after un-applying the row") was
false; corrected, along with the tick-scoped group commit that never
happened
- database/src/CODE-LOGIC.md: where the barrier runs and why there, why
replies are held, why the inline path is asymmetric, the one failure
rule, and how to measure it
- db-bench README: the wmix mode, the env knobs, and the tmpfs warning
Battery: wovm-test 36 suites 0 fail, woc-test, oop-e2e 119/0,
db-bench 106/0, linkcheck clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
databasev2 4 part A, task 4. Scope extended with developer approval: the
plan authorised touching the sample only for observability, but no
existing leg has enough concurrent durable writes to exercise group
commit at all, so the payoff was unevaluable either way.
The finding that forced it:
- `mix` writes on one op in ten with C=4 (all_mode calls mix_mode(n/10,
4); Mixer writes on i % 10 == 9), so the quick run performs 20 writes
total. Measured mean batch 1.01 over 3112 barriers, peak 3
- that is a property of the WORKLOAD, not the mechanism: peak 3 of a
possible 4 shows batches form whenever writes actually coincide
- `wmix N C` added: every op a durable write, C at once. Updates rather
than inserts, so it is comparable to mixwrite and the row count stays
flat. Histogram kind 2 — a replayed store still holds the seeding run's
kind-0/1 Hist rows and merging those would report someone else's
latencies
- WO_WAL_STATS=1 prints one line at exit: batches, records, peak_batch,
peak_staged. Opt-in, because it would otherwise pollute every durable
program's output. Counters live in wo_wal; no builtin, the numbers are
diagnostic and not part of the language
Measured, and it scales with concurrency exactly as designed:
- C = 4 / 16 / 64 -> mean batch 1.13 / 1.76 / 5.35, peak 3 / 10 / 39
- the gate's own legs: durable.s1 5412 records over 5412 barriers (mean
1.0, peak 1 — the inline path, one barrier per statement BY DESIGN),
durable.sN 7757 over 2296 (mean 3.38, peak 28) at 2x the throughput
- peak staged 1372 B settles the no-cap decision with a number: the batch
is tiny, so the upstream mailbox bound is sufficient
- mean_batch/peak_batch are higher-is-better (the default detector would
have called bigger batches worse)
- only the batch SHAPE metrics are waived to 100%; wmix throughput and
latency keep real tolerances (15% s1, 50% sN) — a blanket waiver would
have left the entire new leg ungated
- the live assertion `mean > 1.0` on the sN leg is what catches inertness
- gate bites: sN wmix ops_sec -60% -> FAIL on exactly that metric, 1 of 86
Verified: db-bench-quick 89 checks 0 failures; baseline refreshed (86
metrics).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
databasev2 4 part A, task 2. The core change, and mostly deletion.
- the REQUEST path (wo_db_exec_req) no longer commits after each append.
Applying to RAM and staging stay exactly where they were
- wo_vm_adopt holds each DB reply envelope in a local FIFO instead of
pushing it as the statement finishes. Pushing there would unpark the
requester before its record is durable — the ack contract this
iteration exists to make literally true rather than true by accident
of every batch having one member
- at the end of the drain: ONE wo_wal_commit_fatal for everything staged,
then every held reply. Locals rather than per-shard state: nothing
needs to outlive the batch it describes
- "did this statement stage anything" is asked of the buffer, not guessed
from the opcode, and that count is what the failure diagnostic reports
- the drain commits unconditionally when anything is staged, because the
inline path relies on finding the buffer empty (task 3 documents that)
- staging failure on the request path is now FATAL via wo_wal_stage_fatal:
the row is already in RAM and of the three verbs only insert could undo
itself, so continuing means RAM ahead of disk. One rule
- wal_die is now shared by both fatal points
Verified — the ack contract is the thing that could break, so it is what
was tested:
- just wovm-test: 36 suites (18 x both dispatch flavors) 0 fail, cli_smoke OK
- just db-bench-quick: 85 checks, 0 failures. The legs that matter:
crash.sN.0 — 612 acked rows all present after kill -9, which is the
BATCHING path (multi-shard requests, held replies, one barrier);
crash.s1.0 — 800 acked rows; restart.s1 and restart.sN replay byte-true
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
databasev2 4 part A, task 1.
- wo_wal gains `path`: the abort diagnostic is worthless without naming
the file it could not write. strdup'd in open, freed in close; NULL is
tolerated so the message degrades rather than crashes
- wo_wal_commit now reports WHICH half failed — WO_WAL_ERR_WRITE for
pwrite, WO_WAL_ERR_SYNC for fdatasync. A short write and a device
refusing the flush are different operational problems and the operator
needs the right one named
- wo_wal_commit_fatal(w, nrec): commits, or prints one diagnostic naming
the operation, path, errno and record count, then exits
WO_EXIT_DURABILITY (3 — 1 is a trap, 2 is a refusal, so this takes a
third of its own)
- retrying is not offered, deliberately: on Linux a failed fsync may have
already discarded the dirty pages, so a second call can report success
having written nothing. Replay is the recovery that works
- test_wal: a failed commit is DETECTED, reports the write error
specifically, keeps the batch staged (a failed commit consumes
nothing), and the WAL knows its own path. 165 pass (was 156)
DISCLOSED GAP: the exit path itself is not exercised. Forcing a real
fdatasync failure needs a full or read-only filesystem, which the gate
cannot arrange without mount privileges. No fault-injection switch was
added — shipping a binary that can be told to kill itself is the worse
trade, and the spec rejected it.
Verified: just wovm-test — 36 suites (18 x both dispatch flavors) 0 fail,
cli_smoke OK.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Task 5b of docs/superpowers/plans/2026-08-26-table-residency.md, whose Task 5
is now split 5a-5d (plan updated in this commit).
- the offset twin of wo_row_read (table.c:721): same out-gate contract —
every value handed back is a FRESH VM allocation — but resolved from a file
position instead of the id hash
- fits entirely in wal.c because everything it needs was already public:
scan_record and dec_val are local, and wo_val_decode_vm / wo_db_val_free
are exported at table.h:183-187. Two decode stages, since the record and
the VM speak different dialects: dec_val -> engine slots -> VM copies, with
the engine slots freed as scratch on every path
- ZERO storage change. Nothing calls it yet; that is the point of separating
it from 5c, so the read path can be proven before the slabs are touched
- refuses rather than guessing, each case distinguishable: no intact record
at the offset, a malformed header, a decode failure, trailing bytes, and a
REMOVE tombstone. That last one matters most — handing a tombstone back as
a row would read a deleted row as live
Tested by deep field comparison, not by "it parsed": 24 rows with a nil Text
every third row, each read back BY OFFSET and compared field by field,
including the string bytes. Plus all three refusal paths — tombstone, a
mid-record offset (the silent-wrong-row failure this guards), and past the
intact prefix.
The free-on-every-path claim is VERIFIED, not assumed: removing the free
produced 3 LeakSanitizer reports; restoring it returns to 0. Worth doing
because "ASan is clean" only means something if the harness would have
complained.
PLAN SPLIT: Task 5's storage half was written as if it were plumbing. Measured
instead: wo_row_ptr returns a db_row* into a slab with 11 call sites, table.c
has 37 slab references, db.c:105-181 scans slabs directly, enc_val serialises
FROM the slab, and no operation exists that drops a payload while keeping
index entries. Note this is the OPPOSITE half from the earlier retraction —
the record FORMAT needed nothing, the record STORAGE genuinely is deep. 5c
(id->offset map + drop-payload-keep-index) and 5d (rewiring the call sites,
scans, @unique/FK across the boundary) get their own write-ups.
Gates: test_wal 3654/0 (was 3428), all 18 runtime suites 0 fail under
ASan+UBSan, oop-e2e 119/0, residency 8/0, employee 8/0, db-actor 8/0.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Task 5a of docs/superpowers/plans/2026-08-26-table-residency.md. The read
path itself is NOT in this commit; see the note below.
- the offset problem is far smaller than the spec feared. `w->off` is the
durable tail and `w->len` the staged bytes, and wo_wal_commit pwrites the
whole batch AT off before advancing it — so a record staged now lands at
exactly off+len, knowable at append time with no deferral to flush
- shipped as an inline accessor rather than new out-params on the three
append functions, so the 156 existing WAL checks keep their signatures
- correct across both awkward cases, and both are now unit-pinned:
a failed commit leaves off unadvanced so the record still lands where it
was promised, and wo_wal_open positions off at the end of the INTACT
prefix so offsets are always relative to validated data
- test_offset_capture asserts the recovered ID per record, not merely that a
record parses — a wrong offset reads a NEIGHBOURING record, which passes
its own CRC and returns the wrong row silently. 400 records across
repeated buffer growth (stage() doubles from 4096) and uneven commit
batches, so offsets are exercised mid-buffer and right after a flush
The failed-commit test caught MY OWN misunderstanding: I asserted
next_offset was unchanged after a failed commit. It is not, and should not
be — the record is still staged, so next_offset correctly points PAST it.
The invariant that matters is that the durable tail did not move, which is
what it now asserts.
Gates: test_wal 3428/0 (was 3426), all 18 runtime suites 0 fail under
ASan+UBSan, cli_smoke OK, oop-e2e 119/0, residency 8/0.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Task 4 of docs/superpowers/plans/2026-08-26-table-residency.md — the first
behavioural change in the iteration.
- db.c: one predicate, `table_is_durable`, gating the three EXISTING mutation
sites. Kept as a function rather than an inlined condition so
database/src/CODE-LOGIC.md's "nothing else may mutate storage" claim keeps
holding — the choke points stayed three
- the ack contract is untouched for durable tables: RAM applied, record
staged, one commit before the ack, and a failed commit still removes the row
- replay: a log holding records for a class the image now declares volatile is
a real migration case, not corruption. apply_record returns -2 (distinct
from -1), wo_wal_replay_ex reports the class id, and main.c names it and
exits 2. `wo_wal_replay` stays as the NULL wrapper, so all 156 WAL unit
checks are untouched
- measured, not asserted: 50 inserts wrote 1500 WAL bytes into a durable
table and ZERO into a volatile one. The file's SIZE proves nothing (it is
fallocate'd to 1 MiB up front), so the gate measures the non-zero prefix
BUG I INTRODUCED AND CAUGHT: the mismatch message first printed the class name
with %s, but wo_str.data is `char data[]` with NO NUL terminator (obj.h) — a
buffer over-read. Now %.*s with the explicit length, and re-verified under
ASan.
New gate `just residency` (8 checks), because everything above was otherwise
a one-off manual measurement: restart behaviour, the zero-byte write path, the
mismatch refusal (exit 2, names the class, NOT reported as corruption), and
both compile-time refusals. Its own first run failed two checks for a bug in
the script rather than the feature — `woc | grep` under `set -o pipefail`
returns woc's exit 1 even when grep matches, since woc exits 1 whenever it
reports diagnostics. Captures first now, with the reason noted inline.
Also new: corpus run/table-volatile-inprocess pins that a volatile table is a
FULL table in-process — same @unique enforcement, same index probe, same query
surface. Only survival differs, and that is unobservable from inside one
process.
Gates: woc-test 557/0, 18 runtime suites 0 fail, cli_smoke OK, oop-e2e 119/0
(was 118), residency 8/0, employee 8/0, db-actor 8/0, site 21/0, ASan clean on
the new replay path.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- wo_row_update_field: encode new value, unique re-check against a
shadow BEFORE any mutation (violating update leaves the row
untouched, DB_ERR_UNIQUE), index entries moved old-hash -> new-hash,
old engine value freed; proven by test_table (unique refusal keeps
the row, released key becomes insertable)
- WAL UPDATE record: full-row re-log, replay = replace (remove +
re-create same id); prefix/suffix delta recorded as later
optimization; test_wal replays insert+update to the updated state
- builtins 62 DB_UPDATE_FIELD (cid,id,field,value) and 63 DB_DELETE
(cid,id), commit-before-ack like insert, WO_T_UNIQUE/WO_T_DB/WO_T_IO
mapping; dispatch range 61..63; loader arities; runner mirror
- plan Task 5 marked superseded-in-part with the recorded deviation:
the language surface (reads, queries, row views, delete statement)
is 9b's, where the comprehension design put it -- no interim brace-
select grammar to retire later
- gates: test_table 839/0, test_wal 102/0, 15 suites, oop-e2e 73/0
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- database/src/wal.{c,h}: framed records len|crc32|payload|mark
("WOL1" written last -- no mark, no record), typed-row payloads
walking the class-table kinds (nested records, containers, nil
encodings), little-endian like the loader
- commit order verbatim from the shipped phase-D pattern: RAM apply,
stage, ONE pwrite + ONE fdatasync for the batch, ack after -- group
commit is everything staged riding one sync
- replay decodes straight into engine-owned values (no VM at boot)
and re-enters rows through the choke-point row API, so Task 4's
indexes will rebuild for free; next_id advances past replayed ids
this shard owns (wo_row_create_raw)
- torn tail = short/CRC-fail/no-mark/zero-len: intact prefix applies,
tear dropped whole, wo_wal_open positions AT the tear so the next
commit overwrites it; CRC-valid-but-undecodable = corruption, loud
- wo_wal_check: offline oracle, no engine needed -- the crash
battery's verifier
- test_wal 90/0 ASan+UBSan incl. five crash-battery rounds (fork,
insert/commit/ack-over-pipe, SIGKILL mid-stream: zero acked-but-
missing, zero acked-but-wrong); all runtime suites green, oop-e2e
71/0; binding doc WAL section + CODE-LOGIC + plan Task 2 checked
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
