Task 5c step 1 of docs/superpowers/plans/2026-08-26-table-residency.md, as a
PURE REFACTOR: no storage change, no keys-table anywhere. Borrow is wo_row_ptr
plus a seam, every release is a no-op. Provable on its own before the storage
change it exists to enable.
DESIGN SETTLED BY READING THE STRUCTURES, and both answers make 5c smaller:
- the id hash needs NO new storage. `hvals` is already uint64 holding
slot+1 with 0 = empty (table.h), so offset+1 fits the same field, and the
interpretation is per-table because a table is wholly `all` or wholly
`keys`. No parallel map
- secondary indexes need NO change. `db_ibucket.ids` stores row IDS, not slot
indices, and table.c resolves them through the id hash. I had told the
developer these pointed at slab slots — that was wrong, and it is why this
is one shared accessor rather than 11 rewrites
- the real coupling is the unique shadow: idx_add_row and
row_apply_field_slot both FETCH the conflicting row and compare columns.
Both now borrow/release, so a keys-table's non-resident conflict will be
found rather than silently skipped — a unique check that only examines
resident rows is a correctness hole, not a limitation
The scratch lives on `db_table`, not on the stack and not per call. Per call
would allocate once per candidate inside a bucket loop, turning an O(1) probe
into an allocation storm; a stack buffer is unsafe because the loader bounds
field_cnt at 65535 (loader.c:189), so the worst case is ~512 KB. It is safe
per-table because the store is single-writer, and a `busy` flag is there to
catch a nested borrow rather than let it alias silently. Freed in
table_destroy.
Gates: all 18 runtime suites 0 fail under ASan+UBSan (test_table 856/0,
test_wal 3654/0), oop-e2e 119/0, residency 8/0, employee 8/0, db-actor 8/0.
And the pure-refactor proof the plan asked for: `db-bench --quick` 85/0, every
resident read/query/seed/write floor held — a refactor that moves a number is
not a refactor.
Remaining wo_row_ptr sites for 5d: 6 in table.c, 2 in db.c, 2 in wal.c.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Task 5b of docs/superpowers/plans/2026-08-26-table-residency.md, whose Task 5
is now split 5a-5d (plan updated in this commit).
- the offset twin of wo_row_read (table.c:721): same out-gate contract —
every value handed back is a FRESH VM allocation — but resolved from a file
position instead of the id hash
- fits entirely in wal.c because everything it needs was already public:
scan_record and dec_val are local, and wo_val_decode_vm / wo_db_val_free
are exported at table.h:183-187. Two decode stages, since the record and
the VM speak different dialects: dec_val -> engine slots -> VM copies, with
the engine slots freed as scratch on every path
- ZERO storage change. Nothing calls it yet; that is the point of separating
it from 5c, so the read path can be proven before the slabs are touched
- refuses rather than guessing, each case distinguishable: no intact record
at the offset, a malformed header, a decode failure, trailing bytes, and a
REMOVE tombstone. That last one matters most — handing a tombstone back as
a row would read a deleted row as live
Tested by deep field comparison, not by "it parsed": 24 rows with a nil Text
every third row, each read back BY OFFSET and compared field by field,
including the string bytes. Plus all three refusal paths — tombstone, a
mid-record offset (the silent-wrong-row failure this guards), and past the
intact prefix.
The free-on-every-path claim is VERIFIED, not assumed: removing the free
produced 3 LeakSanitizer reports; restoring it returns to 0. Worth doing
because "ASan is clean" only means something if the harness would have
complained.
PLAN SPLIT: Task 5's storage half was written as if it were plumbing. Measured
instead: wo_row_ptr returns a db_row* into a slab with 11 call sites, table.c
has 37 slab references, db.c:105-181 scans slabs directly, enc_val serialises
FROM the slab, and no operation exists that drops a payload while keeping
index entries. Note this is the OPPOSITE half from the earlier retraction —
the record FORMAT needed nothing, the record STORAGE genuinely is deep. 5c
(id->offset map + drop-payload-keep-index) and 5d (rewiring the call sites,
scans, @unique/FK across the boundary) get their own write-ups.
Gates: test_wal 3654/0 (was 3428), all 18 runtime suites 0 fail under
ASan+UBSan, oop-e2e 119/0, residency 8/0, employee 8/0, db-actor 8/0.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- found during the pre-execution review of the plan, before any code
- the claim was wrong in both spec and plan: table.c's db_val_encode builds
the IN-MEMORY slot; the FILE record is a separate encoding in wal.c and has
been flat since iteration 9. enc_val inlines every kind recursively with no
pointer anywhere; dec_val reads it back; a record is
`WO_WAL_INSERT | class_id | id | <value per field>` in the
len|crc|payload|mark frame; scan_record already preads and CRC-verifies a
record at an arbitrary offset
- so the row encoding needs NO change, and Task 5 (a "self-contained,
offset-based" rewrite billed as the iteration's substantive engineering) is
DELETED, not reduced. 8 tasks -> 7, and the highest-risk task is gone
- the real difficulty is where the spec never looked: wo_wal_append_insert
stages into a 1 MiB buffer, so a record's final offset is unknown until
flush. Threading an accurate offset back through a buffered writer —
correct across partial flush, failed commit and torn tail — is now Task 5's
first two steps, with a unit test that straddles a buffer boundary and a
case asserting no offset is published for a record that never reached disk
- dependent claims corrected: the mmap alternative's premise, the read-path
bullet (now names scan_record/dec_val), and the self-review coverage table,
which records the retraction rather than quietly dropping the row
- root cause worth noting: reading one layer and inferring another. Second
time this iteration — the first was assuming WO_HEAP_MB bounded table
storage when it bounds the VM arena
- no code written yet; linkcheck 0 broken / 0 anchors
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- 8 tasks, 71 steps, from the 2026-08-26 table-residency spec
- deliberately contains NO code: discarded.md records "raw code in plan
documents" as rejected, and all six preceding plans have zero fences.
Stated in the header so it does not read as an omission. Every step
instead names the exact file and line region plus the required behaviour
- task order is by testable deliverable, not by layer:
1 grammar + defaults (no existing golden may move)
2 the cross-table check — a durable ref into a volatile table is refused
3 .wob v7: descriptor carries both properties, loader refuses the
meaningless combination so it never reaches the engine
4 durable:false skips the WAL at the three existing choke points in db.c;
replay refuses on mismatch rather than resurrecting rows
5 self-contained offset-based records — the one real rewrite, since
table.c returns a malloc'd address as the slot word today
6 resident:keys read path: id->offset map, pread, sequential scan;
@unique and FK-restrict across the boundary are the correctness core
7 the two runtime refusals — durable with no WO_DATA (silent data loss
today), and the resident-footprint budget
8 measure, baseline, crash battery, docs, closeout
- self-review table maps every spec section to a task. Two gaps found and
closed: the escape hatch for an intentionally ephemeral run (a refusal
with no way forward is worse than the loss it replaces), and persisting
the offset map in databasev2 3's snapshot
- linkcheck 0 broken / 0 anchors
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
