docs(db2-keys): a runnable example for per-table storage
- docs/examples/residency: one program, two tables filled by the same loop, differing only in the annotation. Run twice against one WO_DATA and orders replay while sessions do not - the example checks its own claim (exits 1 if a durable:false table survives, or a durable:true one fails to replay) rather than narrating it in a print - resident: keys is written out as a commented block with the loader's exact refusal, so the frontier is visible in the example rather than only in a story. It documents WHERE the refusal happens: woc compiles it and emits a .wob; wovm exits 2, because the annotation is a load-time property - residency-accept gains two legs: the example runs and its restart claim holds, and the refusal message the README quotes is checked so doc and code cannot drift apart - the gate writes the example's output to /tmp/residency.log, banner-separated, for tail -F - README commands verified verbatim; they needed mkdir -p because wovm will not create WO_DATA Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> (cherry picked from commit c9c7e03e62c3918cb65ef7994d1e33a0c5337b71)
This commit is contained in:
parent
7ad52937b2
commit
516bd8362d
5 changed files with 246 additions and 1 deletions
88
docs/examples/residency/README.md
Normal file
88
docs/examples/residency/README.md
Normal file
|
|
@ -0,0 +1,88 @@
|
||||||
|
# residency — per-table storage
|
||||||
|
|
||||||
|
What [databasev2 2](../../stories/databasev2/02-table-storage-modes.md) added:
|
||||||
|
`durable` and `resident`, declared per `@table` instead of one environment
|
||||||
|
variable for the whole process.
|
||||||
|
|
||||||
|
Before it, `WO_DATA` was the only switch. Set, and every table is WAL-logged;
|
||||||
|
unset, and none are. Applications are not uniform — a session table is
|
||||||
|
disposable, an orders table is precious, and a 120 GB audit table does not fit
|
||||||
|
in RAM at all. One global switch forces "everything is precious" or "nothing
|
||||||
|
is", and you pay for whichever is wrong.
|
||||||
|
|
||||||
|
## Run it
|
||||||
|
|
||||||
|
```
|
||||||
|
just residency
|
||||||
|
```
|
||||||
|
|
||||||
|
The gate writes the example's own output to `/tmp/residency.log`, banner-
|
||||||
|
separated, so you can `tail -F` it while it runs.
|
||||||
|
|
||||||
|
Or by hand, which is the whole demonstration — the same program twice against
|
||||||
|
one data directory:
|
||||||
|
|
||||||
|
```
|
||||||
|
compiler/_build/default/bin/woc --emit docs/examples/residency/main.wo -o /tmp/residency.wob
|
||||||
|
mkdir -p /tmp/residency-data # WO_DATA must exist; wovm will not create it
|
||||||
|
WO_DATA=/tmp/residency-data runtime/wovm /tmp/residency.wob seed
|
||||||
|
WO_DATA=/tmp/residency-data runtime/wovm /tmp/residency.wob
|
||||||
|
```
|
||||||
|
|
||||||
|
```
|
||||||
|
seeded: orders=3 sessions=3
|
||||||
|
after restart: orders=3 sessions=0
|
||||||
|
ok: durable replayed, volatile did not
|
||||||
|
```
|
||||||
|
|
||||||
|
The second run inserts nothing. Three orders come back from the log; zero
|
||||||
|
sessions do, because they were never written to it. Both tables were filled by
|
||||||
|
the same loop — only the annotation differs, so the difference after the
|
||||||
|
restart is the annotation's doing and nothing else's.
|
||||||
|
|
||||||
|
## The three modes
|
||||||
|
|
||||||
|
| Declaration | Meaning | State today |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `durable: true` (default) | WAL-logged, replayed at boot | ✅ works |
|
||||||
|
| `durable: false` | never written to the log; costs no disk and no fsync; empty after a restart | ✅ works |
|
||||||
|
| `resident: all` (default) | every row's payload lives in RAM | ✅ works |
|
||||||
|
| `resident: keys` | the id map stays resident, the payload lives in the WAL and is read back by offset | ⛔ **refused at load** |
|
||||||
|
|
||||||
|
## Why `resident: keys` is refused
|
||||||
|
|
||||||
|
It is the mode the track exists for — a table larger than RAM. Storage, the
|
||||||
|
read paths, deletes and checkpoint survival all work. **Updating such a row
|
||||||
|
does not:** the row has no slab slot to mutate, so a write would land in a
|
||||||
|
scratch buffer and be discarded *silently*. Doing it properly is
|
||||||
|
read-modify-append.
|
||||||
|
|
||||||
|
So the loader refuses the annotation rather than honouring it in name only.
|
||||||
|
Uncomment the `AuditEntry` block in `main.wo` and you get:
|
||||||
|
|
||||||
|
```
|
||||||
|
wovm: class 0 declares `resident: keys`, which is INCOMPLETE: rows are stored
|
||||||
|
and read keys-only, but UPDATING one is not implemented (it needs
|
||||||
|
read-modify-append). Remove it until databasev2 2 lands updates;
|
||||||
|
`resident: all` is what runs
|
||||||
|
```
|
||||||
|
|
||||||
|
Note **where** that comes from: `woc` compiles it happily and emits a `.wob`.
|
||||||
|
The annotation is a load-time property, so the compiler is green and `wovm`
|
||||||
|
exits 2.
|
||||||
|
|
||||||
|
Refusing at load rather than at the first update is deliberate. A developer who
|
||||||
|
declared a 120 GB table keys-resident, saw it compile, and shipped would find
|
||||||
|
the gap in production. That judgement earned its keep in a way nobody had
|
||||||
|
written down: an audit before relaxing the refusal found that `delete` on such
|
||||||
|
a table was reading a WAL byte offset as a slab index and freeing whatever it
|
||||||
|
landed on — memory corruption, not a missing feature. It is fixed and pinned by
|
||||||
|
a test that SEGVs against the old code, but the refusal is what stood in front
|
||||||
|
of it.
|
||||||
|
|
||||||
|
## What this example does NOT show
|
||||||
|
|
||||||
|
The mode-mismatch startup refusal, the zero-WAL-bytes measurement, and the two
|
||||||
|
compile-time refusals are proven by `scripts/residency-accept.sh` against
|
||||||
|
purpose-built snippets, because each needs a deliberately broken program or a
|
||||||
|
byte-level assertion on the log file. This example is the readable half.
|
||||||
109
docs/examples/residency/main.wo
Normal file
109
docs/examples/residency/main.wo
Normal file
|
|
@ -0,0 +1,109 @@
|
||||||
|
-- residency — databasev2 2's per-table storage, demonstrated across a
|
||||||
|
-- restart.
|
||||||
|
--
|
||||||
|
-- Before this iteration, durability was ONE environment variable for a whole
|
||||||
|
-- process: WO_DATA set and every @table is WAL-logged, or unset and none are.
|
||||||
|
-- Real applications are not uniform. A session table is disposable, an orders
|
||||||
|
-- table is precious. One global switch forces "everything is precious" or
|
||||||
|
-- "nothing is", and the developer pays for the wrong one.
|
||||||
|
--
|
||||||
|
-- Run it twice against the same WO_DATA directory:
|
||||||
|
--
|
||||||
|
-- mkdir -p /tmp/residency-data -- WO_DATA must exist already
|
||||||
|
-- WO_DATA=/tmp/residency-data wovm residency.wob seed
|
||||||
|
-- WO_DATA=/tmp/residency-data wovm residency.wob
|
||||||
|
--
|
||||||
|
-- The second run inserts nothing. Orders come back from the log; Sessions do
|
||||||
|
-- not, because they were never written to it.
|
||||||
|
|
||||||
|
-- Precious: WAL-logged, replayed at boot. `durable: true` is the default, and
|
||||||
|
-- is written out here only because this example is about the annotation.
|
||||||
|
@table(name: "orders", index: [sku], durable: true, resident: all)
|
||||||
|
class Order {
|
||||||
|
sku: Text
|
||||||
|
cents: Int
|
||||||
|
}
|
||||||
|
|
||||||
|
-- Scratch: never written to the WAL, so it costs no disk and no fsync, and it
|
||||||
|
-- is EMPTY after a restart. That is the point — not a bug to work around.
|
||||||
|
@table(name: "sessions", index: [token], durable: false)
|
||||||
|
class Session {
|
||||||
|
token: Text
|
||||||
|
uid: Int
|
||||||
|
}
|
||||||
|
|
||||||
|
-- WHAT DOES NOT COMPILE YET, and why it is written here rather than omitted:
|
||||||
|
--
|
||||||
|
-- @table(name: "audit", index: [at], durable: true, resident: keys)
|
||||||
|
-- class AuditEntry {
|
||||||
|
-- at: Int
|
||||||
|
-- what: Text
|
||||||
|
-- }
|
||||||
|
--
|
||||||
|
-- `resident: keys` is the mode for a table too large for RAM: the id map stays
|
||||||
|
-- resident, the row payload lives in the WAL and is read back by offset. The
|
||||||
|
-- storage, the read paths, deletes and checkpoint survival all work today.
|
||||||
|
-- UPDATING such a row does not — the row has no slab slot to mutate, and doing
|
||||||
|
-- it properly means read-modify-append. The loader therefore REFUSES the
|
||||||
|
-- annotation outright rather than honouring it in name only:
|
||||||
|
--
|
||||||
|
-- wovm: class 0 declares `resident: keys`, which is INCOMPLETE: rows are
|
||||||
|
-- stored and read keys-only, but UPDATING one is not implemented (it needs
|
||||||
|
-- read-modify-append). Remove it until databasev2 2 lands updates;
|
||||||
|
-- `resident: all` is what runs
|
||||||
|
--
|
||||||
|
-- Note WHERE it is refused: the compiler accepts it and emits a .wob — the
|
||||||
|
-- annotation is a load-time property, so `woc` is green and `wovm` exits 2.
|
||||||
|
--
|
||||||
|
-- Refusing at load rather than at the first update is deliberate: a developer
|
||||||
|
-- who declared a 120 GB table keys-resident, saw it compile, and shipped would
|
||||||
|
-- discover the gap in production.
|
||||||
|
|
||||||
|
fn count_orders() -> Int {
|
||||||
|
let n = 0;
|
||||||
|
for _o in from o in Order select o { n = n + 1; }
|
||||||
|
return n;
|
||||||
|
}
|
||||||
|
|
||||||
|
fn count_sessions() -> Int {
|
||||||
|
let n = 0;
|
||||||
|
for _s in from s in Session select s { n = n + 1; }
|
||||||
|
return n;
|
||||||
|
}
|
||||||
|
|
||||||
|
fn report(label: Text) {
|
||||||
|
print("${label}: orders=${count_orders()} sessions=${count_sessions()}");
|
||||||
|
}
|
||||||
|
|
||||||
|
fn main(args: multi Text) -> Int {
|
||||||
|
if len(args) > 0 {
|
||||||
|
if args[0] == "seed" {
|
||||||
|
-- Both tables get the same number of rows, from the same code path.
|
||||||
|
-- Only the annotation differs, so anything that differs after the
|
||||||
|
-- restart is the annotation's doing and nothing else's.
|
||||||
|
let i = 0;
|
||||||
|
while i < 3 {
|
||||||
|
insert Order { sku: "sku-${i}", cents: 100 + i };
|
||||||
|
insert Session { token: "tok-${i}", uid: i };
|
||||||
|
i = i + 1;
|
||||||
|
}
|
||||||
|
report("seeded");
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
-- Second run: no inserts at all. Whatever is here came from the log.
|
||||||
|
report("after restart");
|
||||||
|
|
||||||
|
-- The claim this example exists to make, checked rather than narrated.
|
||||||
|
if count_sessions() != 0 {
|
||||||
|
print("UNEXPECTED: a durable:false table survived a restart");
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
if count_orders() == 0 {
|
||||||
|
print("UNEXPECTED: a durable:true table did not replay");
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
print("ok: durable replayed, volatile did not");
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
6
docs/examples/residency/wo.toml
Normal file
6
docs/examples/residency/wo.toml
Normal file
|
|
@ -0,0 +1,6 @@
|
||||||
|
name = "residency"
|
||||||
|
version = "0.1.0"
|
||||||
|
description = "databasev2 2: per-table storage — durable: true|false and resident: all|keys, shown across a restart"
|
||||||
|
|
||||||
|
[runtime]
|
||||||
|
wo = ">= 0.1"
|
||||||
|
|
@ -9,7 +9,9 @@ readiness: ready
|
||||||
|
|
||||||
> Part of [Story — databasev2: the database beyond RAM](00-story.md).
|
> Part of [Story — databasev2: the database beyond RAM](00-story.md).
|
||||||
> Spec: [`2026-08-26-table-residency-design.md`](../../superpowers/specs/2026-08-26-table-residency-design.md)
|
> Spec: [`2026-08-26-table-residency-design.md`](../../superpowers/specs/2026-08-26-table-residency-design.md)
|
||||||
> · plan: [`2026-08-26-table-residency.md`](../../superpowers/plans/2026-08-26-table-residency.md).
|
> · plan: [`2026-08-26-table-residency.md`](../../superpowers/plans/2026-08-26-table-residency.md)
|
||||||
|
> · runnable example: [`docs/examples/residency`](../../examples/residency/README.md),
|
||||||
|
> gated by `just residency`.
|
||||||
>
|
>
|
||||||
> **The language enrichment this track exists for.** Before this, durability was
|
> **The language enrichment this track exists for.** Before this, durability was
|
||||||
> one environment variable for a whole process: `WO_DATA` set and every `@table`
|
> one environment variable for a whole process: `WO_DATA` set and every `@table`
|
||||||
|
|
|
||||||
|
|
@ -133,6 +133,46 @@ grep -q 'WO-E224' <<<"$dref_out" \
|
||||||
&& ok "durable ref into a volatile table is WO-E224" \
|
&& ok "durable ref into a volatile table is WO-E224" \
|
||||||
|| bad "dangling ref refused" "got: $dref_out"
|
|| bad "dangling ref refused" "got: $dref_out"
|
||||||
|
|
||||||
|
# ---- 5. the doc example actually runs, and its README's claim holds --------
|
||||||
|
# The example is the readable half of this gate. An example no gate runs rots,
|
||||||
|
# and its README quotes the loader's refusal verbatim — so that message is
|
||||||
|
# checked here too, not merely trusted.
|
||||||
|
LOG=/tmp/residency.log
|
||||||
|
: > "$LOG"
|
||||||
|
echo "residency-accept: example output -> $LOG (tail -F it)"
|
||||||
|
EX="docs/examples/residency"
|
||||||
|
{
|
||||||
|
echo "===================== residency example ====================="
|
||||||
|
} >> "$LOG"
|
||||||
|
if "$WOC" --emit "$EX/main.wo" -o "$WORK/residency.wob" >>"$LOG" 2>&1; then
|
||||||
|
ok "the doc example compiles"
|
||||||
|
mkdir -p "$WORK/exdata"
|
||||||
|
{
|
||||||
|
echo "--------------------- run 1: seed ---------------------"
|
||||||
|
WO_DATA="$WORK/exdata" "$WOVM" "$WORK/residency.wob" seed 2>&1
|
||||||
|
echo "--------------------- run 2: restart ------------------"
|
||||||
|
} >> "$LOG"
|
||||||
|
ex2="$(WO_DATA="$WORK/exdata" "$WOVM" "$WORK/residency.wob" 2>&1)"
|
||||||
|
printf '%s\n' "$ex2" >> "$LOG"
|
||||||
|
grep -q 'orders=3 sessions=0' <<<"$ex2" \
|
||||||
|
&& ok "example: durable replayed, volatile did not" \
|
||||||
|
|| bad "example restart" "got: $ex2"
|
||||||
|
else
|
||||||
|
bad "the doc example compiles" "see $LOG"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# the README quotes this message; drift between them is a doc bug
|
||||||
|
printf '@table(name: "audit", durable: true, resident: keys)\nclass A {\n at: Int\n}\nfn main() -> Int { return 0; }\n' > "$WORK/keys.wo"
|
||||||
|
if "$WOC" --emit "$WORK/keys.wo" -o "$WORK/keys.wob" >>"$LOG" 2>&1; then
|
||||||
|
keys_out="$(WO_DATA="$WORK/exdata" "$WOVM" "$WORK/keys.wob" 2>&1)"
|
||||||
|
printf '%s\n' "$keys_out" >> "$LOG"
|
||||||
|
grep -q 'resident: keys.*INCOMPLETE' <<<"$keys_out" \
|
||||||
|
&& ok "resident: keys is refused at LOAD with the documented message" \
|
||||||
|
|| bad "keys refusal message" "got: $keys_out"
|
||||||
|
else
|
||||||
|
bad "resident: keys compiles (the refusal is load-time, not compile-time)" "woc rejected it"
|
||||||
|
fi
|
||||||
|
|
||||||
echo
|
echo
|
||||||
echo "residency-accept: $((pass + fail)) checks, $fail failures"
|
echo "residency-accept: $((pass + fail)) checks, $fail failures"
|
||||||
[[ $fail -eq 0 ]] || exit 1
|
[[ $fail -eq 0 ]] || exit 1
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue