Task 5b of docs/superpowers/plans/2026-08-26-table-residency.md, whose Task 5
is now split 5a-5d (plan updated in this commit).
- the offset twin of wo_row_read (table.c:721): same out-gate contract —
every value handed back is a FRESH VM allocation — but resolved from a file
position instead of the id hash
- fits entirely in wal.c because everything it needs was already public:
scan_record and dec_val are local, and wo_val_decode_vm / wo_db_val_free
are exported at table.h:183-187. Two decode stages, since the record and
the VM speak different dialects: dec_val -> engine slots -> VM copies, with
the engine slots freed as scratch on every path
- ZERO storage change. Nothing calls it yet; that is the point of separating
it from 5c, so the read path can be proven before the slabs are touched
- refuses rather than guessing, each case distinguishable: no intact record
at the offset, a malformed header, a decode failure, trailing bytes, and a
REMOVE tombstone. That last one matters most — handing a tombstone back as
a row would read a deleted row as live
Tested by deep field comparison, not by "it parsed": 24 rows with a nil Text
every third row, each read back BY OFFSET and compared field by field,
including the string bytes. Plus all three refusal paths — tombstone, a
mid-record offset (the silent-wrong-row failure this guards), and past the
intact prefix.
The free-on-every-path claim is VERIFIED, not assumed: removing the free
produced 3 LeakSanitizer reports; restoring it returns to 0. Worth doing
because "ASan is clean" only means something if the harness would have
complained.
PLAN SPLIT: Task 5's storage half was written as if it were plumbing. Measured
instead: wo_row_ptr returns a db_row* into a slab with 11 call sites, table.c
has 37 slab references, db.c:105-181 scans slabs directly, enc_val serialises
FROM the slab, and no operation exists that drops a payload while keeping
index entries. Note this is the OPPOSITE half from the earlier retraction —
the record FORMAT needed nothing, the record STORAGE genuinely is deep. 5c
(id->offset map + drop-payload-keep-index) and 5d (rewiring the call sites,
scans, @unique/FK across the boundary) get their own write-ups.
Gates: test_wal 3654/0 (was 3428), all 18 runtime suites 0 fail under
ASan+UBSan, oop-e2e 119/0, residency 8/0, employee 8/0, db-actor 8/0.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Task 5a of docs/superpowers/plans/2026-08-26-table-residency.md. The read
path itself is NOT in this commit; see the note below.
- the offset problem is far smaller than the spec feared. `w->off` is the
durable tail and `w->len` the staged bytes, and wo_wal_commit pwrites the
whole batch AT off before advancing it — so a record staged now lands at
exactly off+len, knowable at append time with no deferral to flush
- shipped as an inline accessor rather than new out-params on the three
append functions, so the 156 existing WAL checks keep their signatures
- correct across both awkward cases, and both are now unit-pinned:
a failed commit leaves off unadvanced so the record still lands where it
was promised, and wo_wal_open positions off at the end of the INTACT
prefix so offsets are always relative to validated data
- test_offset_capture asserts the recovered ID per record, not merely that a
record parses — a wrong offset reads a NEIGHBOURING record, which passes
its own CRC and returns the wrong row silently. 400 records across
repeated buffer growth (stage() doubles from 4096) and uneven commit
batches, so offsets are exercised mid-buffer and right after a flush
The failed-commit test caught MY OWN misunderstanding: I asserted
next_offset was unchanged after a failed commit. It is not, and should not
be — the record is still staged, so next_offset correctly points PAST it.
The invariant that matters is that the durable tail did not move, which is
what it now asserts.
Gates: test_wal 3428/0 (was 3426), all 18 runtime suites 0 fail under
ASan+UBSan, cli_smoke OK, oop-e2e 119/0, residency 8/0.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Task 4 of docs/superpowers/plans/2026-08-26-table-residency.md — the first
behavioural change in the iteration.
- db.c: one predicate, `table_is_durable`, gating the three EXISTING mutation
sites. Kept as a function rather than an inlined condition so
database/src/CODE-LOGIC.md's "nothing else may mutate storage" claim keeps
holding — the choke points stayed three
- the ack contract is untouched for durable tables: RAM applied, record
staged, one commit before the ack, and a failed commit still removes the row
- replay: a log holding records for a class the image now declares volatile is
a real migration case, not corruption. apply_record returns -2 (distinct
from -1), wo_wal_replay_ex reports the class id, and main.c names it and
exits 2. `wo_wal_replay` stays as the NULL wrapper, so all 156 WAL unit
checks are untouched
- measured, not asserted: 50 inserts wrote 1500 WAL bytes into a durable
table and ZERO into a volatile one. The file's SIZE proves nothing (it is
fallocate'd to 1 MiB up front), so the gate measures the non-zero prefix
BUG I INTRODUCED AND CAUGHT: the mismatch message first printed the class name
with %s, but wo_str.data is `char data[]` with NO NUL terminator (obj.h) — a
buffer over-read. Now %.*s with the explicit length, and re-verified under
ASan.
New gate `just residency` (8 checks), because everything above was otherwise
a one-off manual measurement: restart behaviour, the zero-byte write path, the
mismatch refusal (exit 2, names the class, NOT reported as corruption), and
both compile-time refusals. Its own first run failed two checks for a bug in
the script rather than the feature — `woc | grep` under `set -o pipefail`
returns woc's exit 1 even when grep matches, since woc exits 1 whenever it
reports diagnostics. Captures first now, with the reason noted inline.
Also new: corpus run/table-volatile-inprocess pins that a volatile table is a
FULL table in-process — same @unique enforcement, same index probe, same query
surface. Only survival differs, and that is unobservable from inside one
process.
Gates: woc-test 557/0, 18 runtime suites 0 fail, cli_smoke OK, oop-e2e 119/0
(was 118), residency 8/0, employee 8/0, db-actor 8/0, site 21/0, ASan clean on
the new replay path.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- wo_row_update_field: encode new value, unique re-check against a
shadow BEFORE any mutation (violating update leaves the row
untouched, DB_ERR_UNIQUE), index entries moved old-hash -> new-hash,
old engine value freed; proven by test_table (unique refusal keeps
the row, released key becomes insertable)
- WAL UPDATE record: full-row re-log, replay = replace (remove +
re-create same id); prefix/suffix delta recorded as later
optimization; test_wal replays insert+update to the updated state
- builtins 62 DB_UPDATE_FIELD (cid,id,field,value) and 63 DB_DELETE
(cid,id), commit-before-ack like insert, WO_T_UNIQUE/WO_T_DB/WO_T_IO
mapping; dispatch range 61..63; loader arities; runner mirror
- plan Task 5 marked superseded-in-part with the recorded deviation:
the language surface (reads, queries, row views, delete statement)
is 9b's, where the comprehension design put it -- no interim brace-
select grammar to retire later
- gates: test_table 839/0, test_wal 102/0, 15 suites, oop-e2e 73/0
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- database/src/wal.{c,h}: framed records len|crc32|payload|mark
("WOL1" written last -- no mark, no record), typed-row payloads
walking the class-table kinds (nested records, containers, nil
encodings), little-endian like the loader
- commit order verbatim from the shipped phase-D pattern: RAM apply,
stage, ONE pwrite + ONE fdatasync for the batch, ack after -- group
commit is everything staged riding one sync
- replay decodes straight into engine-owned values (no VM at boot)
and re-enters rows through the choke-point row API, so Task 4's
indexes will rebuild for free; next_id advances past replayed ids
this shard owns (wo_row_create_raw)
- torn tail = short/CRC-fail/no-mark/zero-len: intact prefix applies,
tear dropped whole, wo_wal_open positions AT the tear so the next
commit overwrites it; CRC-valid-but-undecodable = corruption, loud
- wo_wal_check: offline oracle, no engine needed -- the crash
battery's verifier
- test_wal 90/0 ASan+UBSan incl. five crash-battery rounds (fork,
insert/commit/ack-over-pipe, SIGKILL mid-stream: zero acked-but-
missing, zero acked-but-wrong); all runtime suites green, oop-e2e
71/0; binding doc WAL section + CODE-LOGIC + plan Task 2 checked
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
