diff --git a/docs/examples/residency/README.md b/docs/examples/residency/README.md new file mode 100644 index 0000000..b3b6a13 --- /dev/null +++ b/docs/examples/residency/README.md @@ -0,0 +1,88 @@ +# residency — per-table storage + +What [databasev2 2](../../stories/databasev2/02-table-storage-modes.md) added: +`durable` and `resident`, declared per `@table` instead of one environment +variable for the whole process. + +Before it, `WO_DATA` was the only switch. Set, and every table is WAL-logged; +unset, and none are. Applications are not uniform — a session table is +disposable, an orders table is precious, and a 120 GB audit table does not fit +in RAM at all. One global switch forces "everything is precious" or "nothing +is", and you pay for whichever is wrong. + +## Run it + +``` +just residency +``` + +The gate writes the example's own output to `/tmp/residency.log`, banner- +separated, so you can `tail -F` it while it runs. + +Or by hand, which is the whole demonstration — the same program twice against +one data directory: + +``` +compiler/_build/default/bin/woc --emit docs/examples/residency/main.wo -o /tmp/residency.wob +mkdir -p /tmp/residency-data # WO_DATA must exist; wovm will not create it +WO_DATA=/tmp/residency-data runtime/wovm /tmp/residency.wob seed +WO_DATA=/tmp/residency-data runtime/wovm /tmp/residency.wob +``` + +``` +seeded: orders=3 sessions=3 +after restart: orders=3 sessions=0 +ok: durable replayed, volatile did not +``` + +The second run inserts nothing. Three orders come back from the log; zero +sessions do, because they were never written to it. Both tables were filled by +the same loop — only the annotation differs, so the difference after the +restart is the annotation's doing and nothing else's. + +## The three modes + +| Declaration | Meaning | State today | +| --- | --- | --- | +| `durable: true` (default) | WAL-logged, replayed at boot | ✅ works | +| `durable: false` | never written to the log; costs no disk and no fsync; empty after a restart | ✅ works | +| `resident: all` (default) | every row's payload lives in RAM | ✅ works | +| `resident: keys` | the id map stays resident, the payload lives in the WAL and is read back by offset | ⛔ **refused at load** | + +## Why `resident: keys` is refused + +It is the mode the track exists for — a table larger than RAM. Storage, the +read paths, deletes and checkpoint survival all work. **Updating such a row +does not:** the row has no slab slot to mutate, so a write would land in a +scratch buffer and be discarded *silently*. Doing it properly is +read-modify-append. + +So the loader refuses the annotation rather than honouring it in name only. +Uncomment the `AuditEntry` block in `main.wo` and you get: + +``` +wovm: class 0 declares `resident: keys`, which is INCOMPLETE: rows are stored +and read keys-only, but UPDATING one is not implemented (it needs +read-modify-append). Remove it until databasev2 2 lands updates; +`resident: all` is what runs +``` + +Note **where** that comes from: `woc` compiles it happily and emits a `.wob`. +The annotation is a load-time property, so the compiler is green and `wovm` +exits 2. + +Refusing at load rather than at the first update is deliberate. A developer who +declared a 120 GB table keys-resident, saw it compile, and shipped would find +the gap in production. That judgement earned its keep in a way nobody had +written down: an audit before relaxing the refusal found that `delete` on such +a table was reading a WAL byte offset as a slab index and freeing whatever it +landed on — memory corruption, not a missing feature. It is fixed and pinned by +a test that SEGVs against the old code, but the refusal is what stood in front +of it. + +## What this example does NOT show + +The mode-mismatch startup refusal, the zero-WAL-bytes measurement, and the two +compile-time refusals are proven by `scripts/residency-accept.sh` against +purpose-built snippets, because each needs a deliberately broken program or a +byte-level assertion on the log file. This example is the readable half. diff --git a/docs/examples/residency/main.wo b/docs/examples/residency/main.wo new file mode 100644 index 0000000..4ed35a2 --- /dev/null +++ b/docs/examples/residency/main.wo @@ -0,0 +1,109 @@ +-- residency — databasev2 2's per-table storage, demonstrated across a +-- restart. +-- +-- Before this iteration, durability was ONE environment variable for a whole +-- process: WO_DATA set and every @table is WAL-logged, or unset and none are. +-- Real applications are not uniform. A session table is disposable, an orders +-- table is precious. One global switch forces "everything is precious" or +-- "nothing is", and the developer pays for the wrong one. +-- +-- Run it twice against the same WO_DATA directory: +-- +-- mkdir -p /tmp/residency-data -- WO_DATA must exist already +-- WO_DATA=/tmp/residency-data wovm residency.wob seed +-- WO_DATA=/tmp/residency-data wovm residency.wob +-- +-- The second run inserts nothing. Orders come back from the log; Sessions do +-- not, because they were never written to it. + +-- Precious: WAL-logged, replayed at boot. `durable: true` is the default, and +-- is written out here only because this example is about the annotation. +@table(name: "orders", index: [sku], durable: true, resident: all) +class Order { + sku: Text + cents: Int +} + +-- Scratch: never written to the WAL, so it costs no disk and no fsync, and it +-- is EMPTY after a restart. That is the point — not a bug to work around. +@table(name: "sessions", index: [token], durable: false) +class Session { + token: Text + uid: Int +} + +-- WHAT DOES NOT COMPILE YET, and why it is written here rather than omitted: +-- +-- @table(name: "audit", index: [at], durable: true, resident: keys) +-- class AuditEntry { +-- at: Int +-- what: Text +-- } +-- +-- `resident: keys` is the mode for a table too large for RAM: the id map stays +-- resident, the row payload lives in the WAL and is read back by offset. The +-- storage, the read paths, deletes and checkpoint survival all work today. +-- UPDATING such a row does not — the row has no slab slot to mutate, and doing +-- it properly means read-modify-append. The loader therefore REFUSES the +-- annotation outright rather than honouring it in name only: +-- +-- wovm: class 0 declares `resident: keys`, which is INCOMPLETE: rows are +-- stored and read keys-only, but UPDATING one is not implemented (it needs +-- read-modify-append). Remove it until databasev2 2 lands updates; +-- `resident: all` is what runs +-- +-- Note WHERE it is refused: the compiler accepts it and emits a .wob — the +-- annotation is a load-time property, so `woc` is green and `wovm` exits 2. +-- +-- Refusing at load rather than at the first update is deliberate: a developer +-- who declared a 120 GB table keys-resident, saw it compile, and shipped would +-- discover the gap in production. + +fn count_orders() -> Int { + let n = 0; + for _o in from o in Order select o { n = n + 1; } + return n; +} + +fn count_sessions() -> Int { + let n = 0; + for _s in from s in Session select s { n = n + 1; } + return n; +} + +fn report(label: Text) { + print("${label}: orders=${count_orders()} sessions=${count_sessions()}"); +} + +fn main(args: multi Text) -> Int { + if len(args) > 0 { + if args[0] == "seed" { + -- Both tables get the same number of rows, from the same code path. + -- Only the annotation differs, so anything that differs after the + -- restart is the annotation's doing and nothing else's. + let i = 0; + while i < 3 { + insert Order { sku: "sku-${i}", cents: 100 + i }; + insert Session { token: "tok-${i}", uid: i }; + i = i + 1; + } + report("seeded"); + return 0; + } + } + + -- Second run: no inserts at all. Whatever is here came from the log. + report("after restart"); + + -- The claim this example exists to make, checked rather than narrated. + if count_sessions() != 0 { + print("UNEXPECTED: a durable:false table survived a restart"); + return 1; + } + if count_orders() == 0 { + print("UNEXPECTED: a durable:true table did not replay"); + return 1; + } + print("ok: durable replayed, volatile did not"); + return 0; +} diff --git a/docs/examples/residency/wo.toml b/docs/examples/residency/wo.toml new file mode 100644 index 0000000..454268f --- /dev/null +++ b/docs/examples/residency/wo.toml @@ -0,0 +1,6 @@ +name = "residency" +version = "0.1.0" +description = "databasev2 2: per-table storage — durable: true|false and resident: all|keys, shown across a restart" + +[runtime] +wo = ">= 0.1" diff --git a/docs/stories/databasev2/02-table-storage-modes.md b/docs/stories/databasev2/02-table-storage-modes.md index 7c5a54c..680ec28 100644 --- a/docs/stories/databasev2/02-table-storage-modes.md +++ b/docs/stories/databasev2/02-table-storage-modes.md @@ -9,7 +9,9 @@ readiness: ready > Part of [Story — databasev2: the database beyond RAM](00-story.md). > Spec: [`2026-08-26-table-residency-design.md`](../../superpowers/specs/2026-08-26-table-residency-design.md) -> · plan: [`2026-08-26-table-residency.md`](../../superpowers/plans/2026-08-26-table-residency.md). +> · plan: [`2026-08-26-table-residency.md`](../../superpowers/plans/2026-08-26-table-residency.md) +> · runnable example: [`docs/examples/residency`](../../examples/residency/README.md), +> gated by `just residency`. > > **The language enrichment this track exists for.** Before this, durability was > one environment variable for a whole process: `WO_DATA` set and every `@table` diff --git a/scripts/residency-accept.sh b/scripts/residency-accept.sh index 8ecd535..57332ee 100755 --- a/scripts/residency-accept.sh +++ b/scripts/residency-accept.sh @@ -133,6 +133,46 @@ grep -q 'WO-E224' <<<"$dref_out" \ && ok "durable ref into a volatile table is WO-E224" \ || bad "dangling ref refused" "got: $dref_out" +# ---- 5. the doc example actually runs, and its README's claim holds -------- +# The example is the readable half of this gate. An example no gate runs rots, +# and its README quotes the loader's refusal verbatim — so that message is +# checked here too, not merely trusted. +LOG=/tmp/residency.log +: > "$LOG" +echo "residency-accept: example output -> $LOG (tail -F it)" +EX="docs/examples/residency" +{ + echo "===================== residency example =====================" +} >> "$LOG" +if "$WOC" --emit "$EX/main.wo" -o "$WORK/residency.wob" >>"$LOG" 2>&1; then + ok "the doc example compiles" + mkdir -p "$WORK/exdata" + { + echo "--------------------- run 1: seed ---------------------" + WO_DATA="$WORK/exdata" "$WOVM" "$WORK/residency.wob" seed 2>&1 + echo "--------------------- run 2: restart ------------------" + } >> "$LOG" + ex2="$(WO_DATA="$WORK/exdata" "$WOVM" "$WORK/residency.wob" 2>&1)" + printf '%s\n' "$ex2" >> "$LOG" + grep -q 'orders=3 sessions=0' <<<"$ex2" \ + && ok "example: durable replayed, volatile did not" \ + || bad "example restart" "got: $ex2" +else + bad "the doc example compiles" "see $LOG" +fi + +# the README quotes this message; drift between them is a doc bug +printf '@table(name: "audit", durable: true, resident: keys)\nclass A {\n at: Int\n}\nfn main() -> Int { return 0; }\n' > "$WORK/keys.wo" +if "$WOC" --emit "$WORK/keys.wo" -o "$WORK/keys.wob" >>"$LOG" 2>&1; then + keys_out="$(WO_DATA="$WORK/exdata" "$WOVM" "$WORK/keys.wob" 2>&1)" + printf '%s\n' "$keys_out" >> "$LOG" + grep -q 'resident: keys.*INCOMPLETE' <<<"$keys_out" \ + && ok "resident: keys is refused at LOAD with the documented message" \ + || bad "keys refusal message" "got: $keys_out" +else + bad "resident: keys compiles (the refusal is load-time, not compile-time)" "woc rejected it" +fi + echo echo "residency-accept: $((pass + fail)) checks, $fail failures" [[ $fail -eq 0 ]] || exit 1