diff --git a/.claude/agents/database-developer.md b/.claude/agents/database-developer.md index 159416e..cdfd71e 100644 --- a/.claude/agents/database-developer.md +++ b/.claude/agents/database-developer.md @@ -14,7 +14,8 @@ You are the database engineer for writeonce's embedded engine. Doctrine (non-negotiable): - C11 + libc only. No new dependencies, no atomics on the data path. -- RAM is authoritative; the WAL makes it durable. An ack means the +- The log is authoritative; residency is a declared per-table policy + (principle 7, amended 2026-08-26). An ack means the commit fsynced. Replay is whole-or-not-at-all; torn tails drop. - The engine and the VM heap are two memory worlds crossed only by copy (the out-gate: wo_val_decode_vm always copies; rows never hold diff --git a/docs/00-principles.md b/docs/00-principles.md index aaa6a3b..f044386 100644 --- a/docs/00-principles.md +++ b/docs/00-principles.md @@ -68,17 +68,42 @@ binary embeds its own source, so prod is always self-describing. events; a database that is also the app must not blink. *Enforced by:* [the blue-green spec](superpowers/specs/2026-08-03-blue-green-vm-design.md). -## 7. RAM is authoritative; the WAL makes it durable +## 7. The log is authoritative; residency is a declared per-table policy + +**Amended 2026-08-26.** This principle read "RAM is authoritative; the WAL +makes it durable. All reads serve from memory." The durability half was never +under strain and is unchanged. The residency half was false for a real +workload, so it is now a declaration rather than a law. + +**Durability, unconditional:** every mutation is WAL-logged and fsynced before +acknowledgment; boot replays the log; a torn tail is dropped whole by CRC; an +ack means the commit reached disk. Mirrors (Postgres) are reconstructible +backups that reads and acks never depend on. None of this is per-table and +none of it is negotiable. + +**Residency, declared:** what a table keeps in memory is stated at the +declaration site. The default keeps every row resident and serves reads at +memory speed. A table that cannot fit says so, and then only its indexes are +resident while rows are read from the log by offset — the kernel page cache is +the hot copy, which is why the engine uses `pread` and deliberately not +`O_DIRECT`. + +*Why the amendment:* the original wording is right for a knowledge-management +app and simply false for a 120 GB order table on a 32 GB host. A doctrine a +real workload cannot satisfy does not get followed, it gets ignored — and the +failure it produced was an OOM kill, which is the least debuggable outcome +available. The fix keeps one storage engine and one source of truth: the log +*is* the database, and RAM is how much of it you choose to serve fast. What was +rejected in 2026-08-18 and stays rejected is a *second* engine — a paged +B-tree with its own buffer pool ([`plan/discarded.md`](plan/discarded.md)). +Reading rows from the log we already write is not that. -All reads serve from memory. Every mutation is WAL-logged and fsynced -before acknowledgment; boot replays the log. Mirrors (Postgres) are -reconstructible backups that reads and acks never depend on. -*Why:* one source of truth with predictable latency; durability is a -sequential append, not a storage engine bolted to the side. *Enforced by:* [the db-engine binding plan](superpowers/plans/2026-08-01-db-engine-binding.md) -(typed WAL + boot replay, shipped); the mirror-is-backup doctrine is -recorded in [`plan/discarded.md`](plan/discarded.md) (the Rust-era WAL -and mirror plans 11/16 were removed with that track 2026-08-18). +(typed WAL + boot replay, shipped); the residency declaration and its +enforcement are [databasev2 2](stories/databasev2/02-table-storage-modes.md); +the mirror-is-backup doctrine is recorded in +[`plan/discarded.md`](plan/discarded.md) (the Rust-era WAL and mirror plans +11/16 were removed with that track 2026-08-18). ## 8. Samples force the grammar diff --git a/docs/guides/database-developer-subagent.md b/docs/guides/database-developer-subagent.md index 36aaa88..85124c6 100644 --- a/docs/guides/database-developer-subagent.md +++ b/docs/guides/database-developer-subagent.md @@ -42,7 +42,8 @@ You are the database engineer for writeonce's embedded engine. Doctrine (non-negotiable): - C11 + libc only. No new dependencies, no atomics on the data path. -- RAM is authoritative; the WAL makes it durable. An ack means the +- The log is authoritative; residency is a declared per-table policy + (principle 7, amended 2026-08-26). An ack means the commit fsynced. Replay is whole-or-not-at-all; torn tails drop. - The engine and the VM heap are two memory worlds crossed only by copy (the out-gate: wo_val_decode_vm always copies; rows never hold diff --git a/docs/plan/discarded.md b/docs/plan/discarded.md index 1b9e83a..17d9d6f 100644 --- a/docs/plan/discarded.md +++ b/docs/plan/discarded.md @@ -43,7 +43,7 @@ Status board: [`00-status.md`](../stories/00-status.md) · Doctrine: [`../00-pri | **Self-hosted `.wo` deploy logic** | Bootstrap problem — the deploy path cannot be written in the language whose deployment it implements. Recorded as a much-later possibility. | | **Script-based / destructive schema migrations (v1)** | Additive-only auto-diff is what makes rollback unconditional: the previous version ignores fields and classes it never knew. Destructive changes reject at propose time. | | **Portability abstractions over Linux** | Targeting one kernel lets the runtime use its sharpest primitives directly instead of the lowest common denominator. Principle 9. | -| **Mirrors on the commit path** | The Postgres mirror is a reconstructible backup. RAM is authoritative; reads and acks never depend on it. Principle 7. | +| **Mirrors on the commit path** | The Postgres mirror is a reconstructible backup. Reads and acks never depend on it — the log is authoritative. Principle 7. | ## Process / docs @@ -72,7 +72,7 @@ goes to find what took each one's place. | --- | --- | | `09-concurrency-scaleout.md` | [`08-shard-actor-runtime.md`](../stories/language-runtime-database/08-shard-actor-runtime.md) + [`11-fibers.md`](../stories/language-runtime-database/11-fibers.md) — the arc, landed 2026-08-21 | | `10-storage-foundations.md`, `11-wal-and-recovery.md` | [`09-database-engine.md`](../stories/language-runtime-database/09-database-engine.md) (typed WAL + replay) and [`22-durability-throughput-scale.md`](../stories/language-runtime-database/22-durability-throughput-scale.md) (the measurements) | -| `12-engine-disk-cutover.md` | Nothing — RAM stays authoritative by doctrine (principle 7). The disk story is the WAL; reclamation is [`databasev2 3, WAL checkpoint`](../stories/databasev2/03-wal-checkpoint.md) | +| `12-engine-disk-cutover.md` | **Partly revisited 2026-08-26.** The disk story is still the WAL and a paged B-tree with its own buffer pool stays rejected. But principle 7's *residency* half was amended: a table may now declare that only its indexes are resident and its rows are read from the log by offset — [databasev2 2](../stories/databasev2/02-table-storage-modes.md). Reclamation remains [`databasev2 3, WAL checkpoint`](../stories/databasev2/03-wal-checkpoint.md) | | `13-class-model-live-pricing.md` | [`09b-table-relations-query.md`](../stories/language-runtime-database/09b-table-relations-query.md) — `@table`, `ref`/`backlink`, the compiler-checked query surface | | `07-inotify-content-watcher.md` | [`07-logwatcher-proof.md`](../stories/language-runtime-database/07-logwatcher-proof.md) — the log-watcher sample polls via `fs.stat`; inotify was never surfaced as a builtin | | `08-sendfile-static-assets.md` | Nothing. `sendfile` is not exposed; static assets are served as `Text` through `net.write` | diff --git a/docs/stories/00-status.md b/docs/stories/00-status.md index 8ca6f94..843fdc1 100644 --- a/docs/stories/00-status.md +++ b/docs/stories/00-status.md @@ -604,7 +604,7 @@ story file); slots in when scheduled — nothing in the chain depends on it. ### ▸ databasev2 — the database beyond RAM -New 2026-08-26. **The problem:** RAM is authoritative (principle 7) and nothing +New 2026-08-26. **The problem:** rows were resident unconditionally and nothing declares a budget. Rows live in `malloc`'d slabs whose addresses are stable forever; there is no eviction, spill or paging anywhere in `database/src/`; the WAL never checkpoints so boot replays all history; and durability is one diff --git a/docs/stories/databasev2/02-table-storage-modes.md b/docs/stories/databasev2/02-table-storage-modes.md index f2ad592..d46290c 100644 --- a/docs/stories/databasev2/02-table-storage-modes.md +++ b/docs/stories/databasev2/02-table-storage-modes.md @@ -19,6 +19,14 @@ status: refine > global switch forces "everything is precious" or "nothing is", and the > developer pays for the wrong one either way. +> **⚠ Superseded in part, 2026-08-26.** The brainstorm settled on a different +> shape than this document describes: one log-structured engine where the WAL +> *is* the row store, with residency declared per table (`resident: all` / +> `resident: index`) rather than a three-valued `mode:` enum including `cold`. +> Principle 7 was amended accordingly — the log is authoritative, residency is +> the declaration. This file is rewritten once the grammar is approved; read the +> [track index](00-story.md) and `docs/00-principles.md` §7 as current. + ## Goals - **Move the storage decision to the declaration site.** `@table(mode: ...)`, diff --git a/docs/stories/databasev2/04-io-uring-commit.md b/docs/stories/databasev2/04-io-uring-commit.md index 0dec19f..5d8a819 100644 --- a/docs/stories/databasev2/04-io-uring-commit.md +++ b/docs/stories/databasev2/04-io-uring-commit.md @@ -93,9 +93,14 @@ chain: 5 - **io_uring for the network/accept path.** This iteration is the WAL write path only; the socket side is the shard-actor runtime's and the network layer's concern. -- **io_uring for reads.** RAM is authoritative — reads never touch a - descriptor (phase-B doctrine), so there is nothing to accelerate on the - read path. This is a write-durability optimization, full stop. +- **io_uring for reads.** For a fully-resident table reads never touch a + descriptor, so there is nothing to accelerate on the read path. This is a + write-durability optimization, full stop. **Note (2026-08-26):** principle 7's + residency half was amended, so a table declaring `resident: index` + ([databasev2 2](02-table-storage-modes.md)) *does* `pread` rows from the log — + and accelerating that read path with io_uring becomes a real, separate + question. It is not this iteration's, and it should not be folded in: this + one is about the commit path and its acceptance is a durability number. - **Registered buffers / fixed files / SQPOLL tuning** beyond what the benchmark shows is worth it. Start with the plain submit/complete model; add ring features only when 22's number says a specific one pays. diff --git a/docs/stories/databasev2/07-single-file-db.md b/docs/stories/databasev2/07-single-file-db.md index 7eeb598..5e6a3b8 100644 --- a/docs/stories/databasev2/07-single-file-db.md +++ b/docs/stories/databasev2/07-single-file-db.md @@ -17,7 +17,7 @@ status: refine > **Inserted 2026-08-22** (developer ask: "can the persistent db be in > file.db form?"). The truth is already almost there: `WO_DATA=` > holds exactly ONE file (`shard-0.wal`) — the entire persistent state, -> since RAM is authoritative and no data pages exist. This iteration +> since the log is the whole store and no data pages exist. This iteration > makes the surface say so: point `WO_DATA` at a file and THAT file is > the store. Small, driver-only, independent of the concurrency chain. @@ -50,7 +50,7 @@ status: refine ## Out Of Scope -- A paged database file (SQLite's shape) — RAM is authoritative; the +- A paged database file (SQLite's shape) — still rejected; the disk story is the WAL, full stop. - Checkpoint/compaction — [iteration 32](03-wal-checkpoint.md)'s; its rename-swap (write snapshot+tail to a NEW file, fsync, `rename()` diff --git a/docs/stories/language-runtime-database/38-content-platform-capabilities.md b/docs/stories/language-runtime-database/38-content-platform-capabilities.md index 6222b20..86d90fc 100644 --- a/docs/stories/language-runtime-database/38-content-platform-capabilities.md +++ b/docs/stories/language-runtime-database/38-content-platform-capabilities.md @@ -116,7 +116,7 @@ status: refine - **A plugin/app ecosystem.** In-runtime recompile is [iteration 26](26-blue-green-deploy.md)'s; nothing here loads code at run time. -- **Scale claims.** RAM is authoritative (64 MiB arena by default, +- **Scale claims.** Rows are resident by default (64 MiB arena default, `WO_HEAP_MB`), so this is a team-scale platform and the README states the row ceiling it was measured at. diff --git a/docs/superpowers/specs/2026-08-26-table-residency-design.md b/docs/superpowers/specs/2026-08-26-table-residency-design.md new file mode 100644 index 0000000..48820d1 --- /dev/null +++ b/docs/superpowers/specs/2026-08-26-table-residency-design.md @@ -0,0 +1,286 @@ +# databasev2 2 — table residency: the log as row store (for review) + +> **Status:** design approved in brainstorm 2026-08-26, pending review of this +> document. Implements [databasev2 2](../../stories/databasev2/02-table-storage-modes.md), +> which is rewritten to match once this is approved. Amends +> [principle 7](../../00-principles.md) — already applied. +> +> Driving requirement, from the developer: **a 120 GB order table on a 32 GB +> host.** Not a tuning problem; no eviction policy fixes it. Small data-driven +> applications are well served by today's resident default and must not regress. + +## Decisions taken (the brainstorm's forks, settled) + +| Fork | Decision | +| --- | --- | +| One enum or two keys | **Two keys.** `durable:` and `resident:` answer two different developer questions ("do I need this after a restart?", "does it fit in RAM?"). One enum forces a name for each *combination*, which is what made a third value unreadable. | +| Mode vocabulary | **`resident: all \| index`** and **`durable: true \| false`**. No `cold`, `tiered`, `paged`, `mmap` or `buffer` in the grammar. | +| Optional or mandatory | **Optional, both default to today's behaviour** (`durable: true`, `resident: all`). All 28 existing declarations compile unchanged; no goldens reblessed. | +| Which storage architecture | **One engine, log-structured.** The WAL already holds every row; keep an in-RAM id→offset map and read rows back with `pread`. No second engine. | +| Row cache | **None in user space.** The kernel page cache is the hot copy — the repo's own stated position in `exploration/postgresql/buffer-and-checkpoint.md`: "`pread` against an fd that already has its page cached is a memcpy… the page cache is the one cache we want", and the reason the engine avoids `O_DIRECT`. | +| `@unique` on a non-resident table | **Allowed; its index is unconditionally resident.** Settled here rather than deferred — see Constraints. | +| Budget unit | **Bytes** (estimated resident footprint). Rows is the meaningless unit: a text-heavy row and an Int-only row differ by an order of magnitude, so a row count cannot bound RAM. | +| Rejected architectures | `mmap` and a buffer pool stay out — see Alternatives rejected. `discarded.md`'s paged-engine rejection is amended to *partly revisited*, not reversed. | + +## The problem, read off the engine + +Facts, each verified in source rather than assumed: + +- Rows live in `malloc`'d slabs of `DB_SLAB_ROWS` (256), allocated as a table + grows and freed only at table teardown. The free-slot list recycles removed + slots, so a delete-heavy table plateaus; a growing table only grows. +- **The ceiling is process RSS and nothing declares it.** `WO_HEAP_MB` + (default 64 MiB) bounds the VM object arena; table storage is separate + `malloc`. No knob says "this database may use at most N". +- No eviction, spill, paging or LRU exists anywhere in `database/src`. +- **Durability is process-global.** `main.c` opens one `shard-0.wal` when + `WO_DATA` is set. `db.c` guards every WAL append with a null check on + `vm->rt.wal`, so with no `WO_DATA` **every table is silently volatile** — a + program can declare nothing and lose everything. +- An allocation failure is clean: every `malloc` in the row encoder is checked + and `DB_ERR_OOM` maps to `WO_T_OOM`, a catchable trap. The dangerous exit is + the one *before* that — swap thrash, which carries no error signal at all. + +The measurements that bound the design, from iteration 22: durable inserts +≈4.5k/s against RAM ≈297k/s (the 66× fsync gap); reads 1.3M ops/s at p50 1µs +after the index probe landed. The read number is what a resident table buys and +what must not regress for tables that keep it. + +### Why the arithmetic works + +A 120 GB table at ~500 B/row is ≈240M rows. An id→offset entry is 16 bytes, so +the resident index is **≈3.8 GB** — comfortable inside 32 GB with room for two +secondary indexes of similar cost. That ratio is the whole design: **indexes +stay resident, rows do not.** It buys roughly two orders of magnitude of table +size, not infinity, and the spec says so plainly because a design sold as +unlimited gets deployed as if it were. + +## The design + +### Grammar + +Two new `@table` arguments, both optional. The parser's argument loop already +matches `name` and `index` and rejects anything else with a catalogued +diagnostic; these are two more arms and an updated message. `Ast.table_cfg` +grows two fields beside `table_name` and `indexes`. + +| Argument | Values | Default | Meaning | +| --- | --- | --- | --- | +| `durable` | `true`, `false` | `true` | `false` skips the WAL append entirely: no record, no fsync, ack from RAM, table empty after restart. | +| `resident` | `all`, `index` | `all` | `index` keeps the id map and every secondary index in RAM; rows are read from the log by offset. | + +`true`/`false` are already keyword tokens; `all`/`index` are parsed as the same +bare identifiers the `index:` argument's column list already accepts. No lexer +change. + +**One wart, surfaced rather than buried:** `resident: index` puts the word +`index` in value position while `index:` is also a key, so +`@table(index: [customer], resident: index)` reads awkwardly on first +encounter. The parser distinguishes them structurally and there is no +ambiguity, but a reviewer may prefer `resident: keys` or `resident: index_only`. +Flagged for the review of this document; the mechanism is unaffected either +way. + +### The four combinations + +| `durable` | `resident` | Status | +| --- | --- | --- | +| `true` | `all` | Today's behaviour. The default. Reads at memory speed. | +| `false` | `all` | Volatile scratch: sessions, rate-limit counters, idempotency keys. Skips the 66× fsync cost. What porch 1–3 need. | +| `true` | `index` | The 120 GB case. Rows in the log, indexes resident, `pread` on read. | +| `false` | `index` | **Refused at compile time.** Rows would have nowhere to be read from. | + +### Read, write and recovery paths + +- **Insert** — unchanged for `resident: all`. For `resident: index`: encode and + append the record as today, then record id→offset in the resident map instead + of retaining the row in a slab. The WAL append is already the durable write; + this stops discarding its payload. +- **Read by id** — resident map lookup, then `pread` at the offset, verify CRC, + decode into fresh VM values. The decode path already exists + (`wo_val_decode_vm` always copies; rows never hand out interior pointers), so + the change is where the bytes come from. +- **Update** — append a new record, repoint the offset. The superseded record + becomes garbage, reclaimed by the checkpoint. +- **Delete** — append a tombstone, drop the id from the map and every index. +- **Scan** — a sequential walk of the log, which is the case log-structured + storage is best at. Cost changes from memory-speed to sequential-disk; the + query surface is unchanged in *meaning* and materially different in *cost*. +- **Boot** — replay rebuilds the offset map by scanning the log. Correct, and + O(entire history), which is the honest cost of shipping this before + [databasev2 3](../../stories/databasev2/03-wal-checkpoint.md). + +### Row encoding: the one real rewrite + +A row slot today holds raw pointers. `table.c`'s `WO_K_TEXT` case allocates a +`db_text` and returns its address as the slot word; owned, multi and map do the +same. Pointers minted by a dead process are meaningless in a file, so the +on-disk record must be **self-contained and offset-based**: every heap value +inlined into the record with internal references expressed as offsets from the +record's own start. + +This is confined to `db_val_encode`/`db_val_decode` and the record framing. It +is the substantive engineering in this iteration and the place to expect the +bugs. `wal.c` already frames records as `len|crc|payload|mark`, so the framing +exists; what changes is that the payload must be readable standalone rather than +only replayable. + +### Constraints across the residency boundary + +- **`@unique` is allowed, and its index is unconditionally resident.** A shadow + check cannot scan slabs that are not there, so correctness requires the unique + column's index in RAM. Cost is one hash entry per row — the same order as the + id map, so a unique column roughly doubles the resident index. Stated at the + declaration so the developer can see what they bought. Silently checking only + resident rows is the one outcome that must never ship. +- **Foreign-key restrict works unchanged.** It is a secondary-index probe, and + secondary indexes are resident. +- **`ref` navigation works unchanged**, at the cost of a `pread` per hop. +- **A `resident: all` table may hold a `ref` into a `resident: index` table** + and vice versa — both are durable, so neither evaporates. This is the case + that a `durable: false` table genuinely breaks, below. + +### What the compiler enforces + +Four refusals, each a catalogued `WO-E1xx` diagnostic added in the same change +as the code — not afterwards: + +1. `durable: false` with `resident: index` — the meaningless combination. +2. An unknown value for either argument, or either argument given twice. +3. **A `durable: true` table holding a `ref` into a `durable: false` table.** + A persistent row cannot reference one that evaporates on restart; FK restrict + cannot save it and the dangling reference is provable from the class table. + This is the highest-value check in the iteration. +4. A `cold`, `tiered`, `ram` or other retired mode word — refused with a message + naming the two real arguments, so the vocabulary explored during the + brainstorm does not become folklore. + +### What the runtime enforces + +- **`durable: true` with no `WO_DATA` is a startup refusal.** Today this + combination silently loses everything, which is the worst failure mode in the + current engine. A program that declares durability and is given nowhere to put + it must not start. This is independent of residency and is arguably the most + valuable single line in the spec. +- **A byte budget that exists by default, breached loudly.** The budget bounds + estimated resident footprint across all tables. On breach the program refuses + with a message naming the largest offending table and the exact annotation to + add, so the 120 GB developer meets a diagnostic at 32 GB rather than the OOM + killer. + + **The default must not be "no budget"** — that was a contradiction in the + first draft of this spec: a budget nobody sets cannot produce the diagnostic + that is this design's main deliverable, and the ERP developer would still meet + the OOM killer. So the default is a **fraction of host-detected available + memory**, overridable by an environment variable and by a per-program + declaration. Choosing that fraction is the one number this spec cannot supply: + it comes from [databasev2 1](../../stories/databasev2/01-ram-ceiling-measurement.md)'s + swap-onset measurement, which is why 1 sequences before 2. Until 1 lands, + implement the mechanism with a conservative placeholder fraction and treat the + value as unset rather than settled. + + Consequence to accept deliberately: a program that today grows past that + fraction and survives on a large host will now refuse. That is the intended + behaviour change — it converts an invisible slide into swap into a startup + error — but it *is* a behaviour change, and it is the second of the two + breaks listed under Migration. +- `WO_DATA` remains the data-directory location. It stops being the durability + switch; the declaration is. + +### Format + +The class descriptor carries both properties, so the runtime never re-derives +them. This moves `WOB_VERSION` (currently 6) and the format contract in the same +commit as the code. An older image is refused on version rather than misread. + +## Why this shape + +- **It keeps one engine and one source of truth.** The log *is* the database. + Nothing here adds a second storage system, which is what the 2026-08-18 + rejection was actually about. +- **It costs nothing for tables that do not use it.** A `resident: all` table + takes the same path it takes today. The 1.3M ops/s read baseline is the + regression gate, and a measurable regression there is grounds to reject the + implementation rather than tune it. +- **The failure mode becomes a diagnostic.** Two of the three exits + characterised in databasev2 1 — swap thrash and the OOM killer — are replaced + by a refusal that names the fix. +- **It is declared, not automatic.** No threshold heuristic, no performance + cliff the compiler cannot explain. Consistent with a language whose thesis is + that the compiler tells you the truth. + +## Alternatives rejected + +| Alternative | Why not | +| --- | --- | +| **`mmap` the row file** | Requires offset-based rows *and* gives up precise ack-after-fsync for the kernel's flush schedule. The repo's own mmap study only ever proposed it read-only for segment lookups. If rows become self-contained anyway, mmap is a possible later optimisation of the read path — recorded, not adopted. | +| **Buffer pool with dirty-page tracking** | This is the Rust-era phase-12 design in `exploration/postgresql/buffer-and-checkpoint.md` (`CachedRow { bytes, dirty }`, `WO_CACHE_ROWS` LRU) that died with that track. It duplicates the kernel page cache, and the page cache is explicitly the cache this project wants. | +| **Paged B-tree engine** | Rejected 2026-08-18 and still rejected. Reading rows from the log we already write is not this. | +| **A three-valued `mode:` enum** | Needs a name per combination. The brainstorm demonstrated that the third name is unwriteable before its mechanism is decided. | +| **Automatic spill at a threshold** | No annotation needed, but unpredictable cliffs and magic the compiler cannot explain. | +| **Disk-backed by default** | Costs every application the 1µs read, including the small ones explicitly said to be well served today. | + +## Proof plan + +Acceptance is the story's Given/When/Then list; this is how each is exercised. + +- **Compatibility** — all 28 existing `@table` declarations compile untouched; + `just employee`, `just db-actor`, `just web-app`, `just site`, `oop-accept` + unchanged; no golden reblessed. +- **Volatility** — a `durable: false` table produces no WAL growth (measured, + not asserted) and is empty after restart while durable siblings replay intact. +- **Residency correctness** — a `resident: index` table larger than the + configured budget returns every row correctly by id, byte-identical including + every heap-valued column, and scans in full. +- **Constraints across the boundary** — `@unique` refuses a duplicate whose + conflicting row is not resident; FK restrict refuses a delete whose only + referrer is not resident. Corpus fixtures, both. +- **Compiler refusals** — a `compile-fail` fixture per diagnostic, each pinning + the exact code. +- **Runtime refusals** — `durable: true` with no `WO_DATA` fails at startup; + a budget breach names the table and the annotation. +- **Crash safety** — `kill -9` mid-append and mid-checkpoint on a + `resident: index` table; replay loses no acked write and no row appears twice. +- **Performance** — new baseline rows for the `resident: index` read path with + its amplification versus resident, published in `perf-targets.md` as a number + a developer can plan around; and a regression check that resident tables did + not move. +- **Sanitisers** — ASan on the new decode path, which is where the bugs are. + +## Out of scope + +- **Checkpoint and compaction** — [databasev2 3](../../stories/databasev2/03-wal-checkpoint.md). + This spec's boot cost is O(history) until 3 lands, and 3's snapshot should + persist the offset map so boot stops rescanning. That coupling is stated in + both documents. +- **Eviction policy and a resident row cache** — [databasev2 5](../../stories/databasev2/05-bounded-tables-eviction.md). + This design needs neither: rows are either all resident or none are. +- **io_uring on the read path** — a real question that only exists after this + lands; noted in [databasev2 4](../../stories/databasev2/04-io-uring-commit.md), + not folded into it. +- **Per-shard residency** — the owner shard owns the store and the log. A + `durable: false` table arguably need not live on the owner at all, which would + be dramatically faster and a different consistency story. Recorded as a + candidate, deliberately not decided here. +- **Migrating an existing dataset between settings**, and a WAL written when a + table had different settings: refuse clearly on mismatch, do not convert. +- **`transaction { }` and `@table` feature flags** — language iteration 18, + approved spec, left whole. + +## Migration + +Nothing to convert. Both arguments default to present behaviour, so every +existing program, manifest, corpus fixture and golden compiles and runs +unchanged. + +**Two deliberate behaviour changes**, both converting a silent failure into a +loud one, and both listed here so neither arrives as a surprise: + +1. `durable: true` (the default) with no `WO_DATA` becomes a startup refusal. + Today it silently discards every write. +2. Total estimated resident footprint crossing the default budget fraction + becomes a startup or insert refusal. Today the program slides into swap with + no signal and is eventually killed. + +Both are opt-out-able by explicit declaration. Neither is a data-format change, +so a rollback is a binary swap with no migration.