docs(db2-keys): spec — delta records for keys-resident updates

- updates append a DELTA (class, id, field, value, back-pointer), not a
  full row. The workload decides it: a product catalogue changes one
  narrow field of a wide row on every order, so a full-row append would
  rewrite every field to move one integer on a shop's hottest path
- the back-pointer keeps the id map at one slot per row, which is the
  mode's whole premise; a map growing per update would defeat it
- ONE fold function, three callers (read, replay, compaction). Three
  implementations of one rule is how they drift, and a fold that differs
  between reading and replaying is a database that changes its mind at
  boot. Named as the design's principal risk
- indexed columns MAY change: price is exactly what a catalogue indexes,
  so forbidding it would be a restriction users meet immediately
- no chain cap, deliberately. Compaction already rewrites live rows, so
  every checkpoint resets every chain, and deltas grow the log which
  pulls the next checkpoint forward — the workload that lengthens chains
  triggers the fold that shortens them
- the risk that accepts: one hot SKU under an otherwise quiet write
  rate. Task 7's benchmark must include it
- supersedes the 2026-08-26 spec's one-line full-row Update sketch,
  marked in place rather than left as a second design in the tree

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit c9a88b05c4c62a5df13253686c990aaccc3f9cbb)

docs/superpowers/specs/2026-08-26-table-residency-design.md

@@ -124,8 +124,15 @@ indexes are very much resident.
   each field. Both functions already exist in `wal.c` and are already exercised
   by replay; the change is that they are called on demand rather than only at
   boot.
-- **Update** — append a new record, repoint the offset. The superseded record
+- **Update** — ~~append a new record, repoint the offset~~. **SUPERSEDED
-  becomes garbage, reclaimed by the checkpoint.
+  2026-08-30** by
+  [`2026-08-30-keys-resident-delta-updates-design.md`](2026-08-30-keys-resident-delta-updates-design.md).
+  This line was written before any of the mode was built and describes a
+  full-row append, which the motivating workload rejects: a product catalogue
+  changes one narrow field of a wide row on every order, so a full-row append
+  rewrites every field to move one integer. Updates append a **delta** carrying
+  a back-pointer, folded on read, with compaction doing the fold that keeps
+  chains short. The superseded-record-becomes-garbage half is still true.
 - **Delete** — append a tombstone, drop the id from the map and every index.
 - **Scan** — a sequential walk of the log, which is the case log-structured
   storage is best at. Cost changes from memory-speed to sequential-disk; the

docs/superpowers/specs/2026-08-30-keys-resident-delta-updates-design.md

@@ -0,0 +1,170 @@
+# `resident: keys` updates — delta records folded on read
+
+Design settled 2026-08-30. Closes the last gap in
+[databasev2 2](../../stories/databasev2/02-table-storage-modes.md), which is
+the only reason the loader still refuses `resident: keys` outright.
+
+**Supersedes** the one-line sketch in
+[`2026-08-26-table-residency-design.md`](2026-08-26-table-residency-design.md)
+— "Update: append a new record, repoint the offset". That was written before
+any of the mode was built, and it describes the full-row shape this design
+rejects.
+
+## The workload that decides the shape
+
+The docs used to motivate this mode with a 120 GB audit table. That was a bad
+example, because an audit log is append-only and therefore argues for nothing:
+if rows are never updated, the missing update path costs nothing and the mode
+could simply have been declared append-only.
+
+The real case is a **product catalogue**, and it is the one in
+`docs/examples/residency`. Its shape decides everything below:
+
+- The id is stable. A SKU does not change.
+- Other fields change constantly. Every order moves `stock`.
+- Reads are hot. Browsing a catalogue is the dominant operation.
+- Rows are wide. Name, price, description, and whatever else a shop carries.
+- The table is the one that outgrows RAM first, which is why it wants this mode.
+
+So an update touches **one narrow field of a wide row, frequently, on the
+hottest write path a shop has**, while reads must stay fast. Appending the whole
+row per sale would rewrite every field to move one integer. That is the argument
+for a delta.
+
+## The design
+
+### The record
+
+A fourth WAL record kind beside `WO_WAL_INSERT` (1), `WO_WAL_REMOVE` (2) and
+`WO_WAL_UPDATE` (3). It carries the class id, the row id, the field index, the
+encoded new value, and a **back-pointer**: the log offset of the record this one
+supersedes.
+
+The back-pointer is what keeps the id map at exactly one slot per row. That is
+not an optimisation — a map that grew per update would defeat the mode's entire
+premise, which is that the map is the only resident part.
+
+### The fold, and why it lives in one function
+
+Reading a row means walking the chain **backward** from the newest record,
+taking first-seen-wins per field, until a full-row record terminates it, then
+materialising that base and overlaying the collected fields.
+
+Three callers need exactly this: reads (`wo_row_borrow`), replay, and
+compaction. They must agree perfectly — a fold that differs between reading and
+replaying is a database that changes its mind at boot. **This is the design's
+principal risk**, and the mitigation is that the fold is written once and all
+three call it. No caller may implement its own walk.
+
+### Update
+
+Borrow the row, which folds it. That read is needed regardless: index
+maintenance requires the old value, and this design allows indexed columns to
+change (see below). Then append the delta, commit, and re-point the map at the
+new record.
+
+The two existing entry points — the VM-value path and the engine-value slot
+path — converge on one internal append. They currently both refuse keys tables;
+that refusal is replaced, not duplicated.
+
+### Indexed columns may change
+
+Allowed deliberately. On a product table `price` is exactly the kind of column
+you would index, so forbidding it would be a restriction users meet immediately
+rather than a theoretical one.
+
+The cost is that index maintenance runs on every delta: the old value comes from
+the row just folded, the new one from the delta itself. `@unique` probing
+already borrows each bucket candidate, and those borrows now fold — so a unique
+check against a heavily-updated table walks chains. Disclosed, not hidden.
+
+### Crash safety
+
+Commit precedes re-point, and that ordering is the whole argument:
+
+- Crash **after** the barrier, before the re-point: the delta is durable, the
+  in-memory map is stale, and replay rebuilds the map from the log — which sees
+  the delta and lands on the same state. Correct.
+- Crash **before** the barrier: the delta never reached the log and the row is
+  unchanged. Correct.
+
+Under group commit an update stages like an insert, and the re-point defers to
+the post-barrier flush — a pending-repoint list mirroring the pending-drop list
+that already exists in `wo_wal` (`pend`/`pend_len`/`pend_cap`).
+
+### Compaction is the bound
+
+Compaction already walks live ids and rewrites each row. It now **folds** each
+row and writes a single full-row record, so every checkpoint resets every chain
+to length zero.
+
+This is what makes read cost bounded without a cap: a read is
+`1 + deltas-since-the-last-compaction` preads, and deltas grow the log, which
+pulls the next compaction forward on the existing `WO_CHECKPOINT_BYTES` and
+`WO_CHECKPOINT_RATIO` policy. The workload that lengthens chains is the same one
+that triggers the fold that shortens them.
+
+**Settled deliberately: no hard chain cap.** The self-limiting property above is
+real, and a cap is a number nobody would tune. The risk this accepts is stated
+under Risks.
+
+### Replay
+
+A delta at replay re-points the row's offset, exactly as a dropped payload
+does today. Because indexed columns may change, replay must also fold the delta
+into the indexes it built from the base record — so boot work grows with chain
+length. Chains are short by the compaction argument, and this is the same fold
+function reads use.
+
+## What this does NOT change
+
+- The id map's shape — one offset per row, unchanged.
+- The insert path, the read path for unmodified rows, deletes, scans, or
+  checkpoint survival. All of those shipped and are tested.
+- `resident: all`. Nothing here touches the fully-resident path.
+
+## Risks
+
+- **The three folds drifting apart.** Mitigated by one function, and the
+  acceptance criteria below test the same row through all three paths rather
+  than testing the fold once.
+- **One hot row under an otherwise quiet write rate.** This is the shape that
+  breaks the compaction argument: a single SKU selling far faster than the rest
+  lengthens its chain without growing the log enough to trigger a checkpoint.
+  Deliberately not solved with a cap. **Task 7's benchmark must include it**,
+  because it is the case where this design is worst and it is a realistic
+  inventory shape, not a contrived one.
+- **Unique probing on a hot table.** Each bucket candidate now folds. Bounded by
+  the same checkpoint argument, but it multiplies: candidates times chain length.
+
+## Acceptance criteria
+
+- **Given** a keys-resident row, **when** one field is updated, **then** a read
+  returns the new value for that field and the original values for every other.
+- **Given** a row updated N times, **when** it is read, **then** the result
+  equals the same row's value in a fully-resident table subjected to the same
+  updates. The resident table is the oracle.
+- **Given** a row with a delta chain, **when** the process restarts, **then**
+  replay produces the same row the pre-restart read produced.
+- **Given** a row with a delta chain, **when** compaction runs, **then** the row
+  reads identically afterwards and its chain is length zero.
+- **Given** a delta that changes an indexed column, **when** the row is queried
+  through that index, **then** it is found by the new value and not by the old —
+  before and after a restart, and before and after a compaction.
+- **Given** a crash between the commit barrier and the re-point, **when** the
+  process restarts, **then** the update is present.
+- **Given** an update to a `resident: all` table, **when** it runs, **then**
+  nothing about its behaviour or its log records changes.
+- **Given** `resident: keys` on a table, **when** the program loads, **then** it
+  is **accepted** — the loader refusal is lifted by this work, and that lift is
+  the deliverable.
+
+## Out of scope
+
+- A hard delta-chain cap, and any read-triggered fold. Settled above.
+- Changing the compaction policy knobs. They already exist and this design
+  leans on them unchanged.
+- `resident: keys` performance measurement. That is
+  [databasev2 2](../../stories/databasev2/02-table-storage-modes.md)'s task 7,
+  and it decides whether this mode is worth having at all — a question this
+  design does not answer and must not be read as answering.