- loader's resident:keys refusal said "rows are still fully resident"
and "until tasks 5c/5d land". Both false since f606fc9. Corrected to
name the real blocker: UPDATE needs read-modify-append
- databasev2 00-story: the sequence graph drew 2->3->4, which reads as
3 needing 2 and 4 needing 3. Both backwards, and it still drew the
2->5->6 path the 2026-08-27 amendment retired. Redrawn stating only
real dependencies, with 4 and 3 shown as composing rather than
ordered, and the execution order that actually happened
- databasev2 03: the hazard and its Outstanding entry both claimed
nothing fails "because iteration 2's storage half is unimplemented".
Marked discharged, and recorded that the hazard named only half the
danger — the bitmap walk would have dropped keys rows outright
- databasev2 06: pending -> hold (largely superseded, revisit only on
a measurement); dated its 5c/5d references
- porch 01: rewritten to the settled shape. readiness ready, status
in-progress, phases B and C marked superseded with why
- porch 01 claimed time.after "is still a reserved builtin id". False —
builtin 90, implemented. That claim is what made the iteration look
cheaper than it is
- porch README gains honest ledger rows for both features (partial,
being rebuilt), not shipped
- skill-catalog README pointed at a story path that moved tracks;
linkcheck now 0 broken
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit b3d8c403e1d19ac27ec966de85cb293e0765795c)
- wo_row_read and the @unique shadow probe go through borrow/release;
release runs before every exit, including wo_row_read's early return
- updates on a keys table refused explicitly in wo_row_update_field and
the slot variant: no slab slot to mutate, and writing the borrow's
scratch would discard the write silently. Needs read-modify-append
- compaction walked the bitmap, which a keys row has no bit in — every
such row would have been dropped from the new log. Now walks
wo_row_next_id and re-points each row to where it lands
- moves records byte-for-byte (copy_record) rather than decoding: a
borrowed row holds VM values, enc_val expects engine values, and ASan
caught that mismatch as a 4294967292-byte memcpy
- wo_row_set_offset updates a value in place and never rehashes, so a
wo_row_next_id cursor stays valid while compaction re-points
- a compaction that fails after moving rows is fatal: the map would name
an unlinked temp file, and the intact log replays correctly
- test_keys_resident_survives_compaction pins both failure modes; rows
rewrite in hash order so offsets really move
- loader still refuses resident: keys — updates are not implemented
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit f606fc9b76d983cac2b348f03f7d4f01433cd905)
The branch was 17 ahead / 25 behind with 11 conflicting files, and drifting
further: db.c had been rewritten twice on master since (group commit, then
compaction). Resolved rather than rebased so both histories stay legible.
Conflicts, and how each was settled:
- db.c: BOTH semantics kept. Master's fatal path and compaction check now sit
behind the branch's `table_is_durable` predicate, in all three inline arms —
a volatile table reaches neither the barrier nor the compaction check
- db-bench sample: every mode from both sides (growth, growth-verify, randread,
replayseed, wmix) and ONE `boot` mode, which both sides had added
independently
- db-bench.py: all six legs kept. Both sides had also grown the same
WAL-size helper under different names; collapsed into one
- perf-targets: the branch's §5 (RAM ceiling) then master's §6/§7 — master's
numbering had already assumed a §5 it did not have
- story frontmatter: master's `status` (the landing truth) plus the branch's
`readiness` axis. 03 would have read `done` + `refine`, which is a
contradiction — it was brainstormed and landed on master, so `ready`
- board: both standup blocks newest-first; master's chain rows (a superset);
the branch's databasev2 1-2 rows with master's 3-4. Fixed a stray `|` in
master's row 3
- baseline: master's, then REGENERATED from a full campaign — 143 metrics,
132 checks, 0 failures with both sides' legs present
TWO HALF-EXPOSED FEATURES FIXED, because the merge rule is that master gets
no feature that is honoured in name only:
- `resident: keys` PARSED, set a .wob flag, and did nothing: rows stayed fully
resident. A developer could declare a 120 GB table keys-resident, watch it
compile, and be OOM-killed. The loader now REFUSES it with a message naming
what to write instead, until tasks 5c/5d land. The compiler still parses it
and its AST golden still passes, so the grammar work stays tested
- `durable: false` was honoured ONLY on the inline path. wo_db_exec_req had no
guard at all, so a volatile table written from an actor on a worker shard
would still be logged — precisely porch's session-table case, and precisely
what iteration 2 exists to provide. All three request-path arms now carry the
same predicate. Found by reading the merged code, not by a test: the obvious
probe runs main() on the primary and therefore only exercises the inline path
Verified on the merged tree: wovm-test 0, woc-test 0, oop-e2e 122/0,
residency-accept 8/0, db-bench 132/0, linkcheck clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
databasev2 3, task 6. Documentation, plus three gate-tolerance
corrections that are justified rather than silent.
- 04-db-binding.md: the NORMATIVE rule — compaction may run only where
nothing is staged (a correctness requirement, not scheduling), recovery
is unchanged, and a failed compaction is a missed optimisation rather
than a durability event
- database/src/CODE-LOGIC.md: why one file and not snapshot-plus-tail
(Postgres CANNOT compact — page deltas; ours are full row images, so a
compacted log IS a store), why rename is the whole crash-safety story,
why the dump flushes but does NOT fsync when it does, why the
replacement is preallocated, and where the trigger is checked
- README: the checkpoint knobs, the extended walstats line, the boot mode
- story -> status: done, with criteria split met/outstanding
- board: standup entry in the six-question shape, both rows rewritten
THE OBLIGATION IS AT THE COMPACTOR, not only in a spec: compaction moves
every record, so it invalidates every WAL offset iteration 2's
`resident: keys` stores, and the loop that knows each record's new
position must rebuild that map. Nothing fails today because that storage
half is unimplemented — it would fail later, looking like corruption.
Board claim corrected before it shipped: I wrote that the concurrency
chain is "complete". It is not — chain 5 stays in-progress because
databasev2 4's part B was never done and its premise was invalidated by
part A. Every link has landed its PLANNED work; that is a different
statement.
Gate tolerances, each with the measurement that justifies it:
- ckpt.pause_us_max is no longer gated relatively. The raw pause scales
with the live set and this workload's live set is not fixed (wmix's
hist_dump inserts a row per latency bucket), so gating it gates the
box. Added ckpt.pause_us_per_mb — the engine's own rate, gated for
real, and the metric that would have caught the 8x dump regression —
with the absolute 50ms budget still guarding the raw pause
- ram.*.msgrate 15% -> 70%. PRE-EXISTING, and measured: 10.7M-17.9M
msgs/sec across ten full runs, several predating this work — a 1.67x
spread against a 15% gate
- durable.sN.*.p99us 100% -> 300%, with more evidence than the first
widening: mixread 1043/2318/4147us, mixwrite 1623/4446us on the same
build. Floors stay the real guard and are not slack
Battery: wovm-test 36 suites 0 fail, woc-test, oop-e2e 119/0,
db-bench 117 checks 0 failures, linkcheck clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Plan for the approved spec. Code-free per the repo convention
(docs/plan/discarded.md:54); the executor writes the code.
- T1 wo_wal_compact: walk live rows via the bitmap, append one INSERT
each through the EXISTING append path, fsync, rename over the live log,
fsync the parent dir, reopen the descriptor. Test asserts BOTH that the
log shrank AND that a replay reproduces the same rows/ids/values —
shorter alone is worthless, a truncating bug also passes that
- T2 a stale temp file is removed at open and never read. The test uses
PLAUSIBLE records, not garbage: garbage would be rejected anyway and
would prove nothing
- T3 the trigger as a PURE decision (used bytes, last compaction's
measured output, floor) so it is unit-testable without a store; env
knobs for floor and ratio, which is what makes the policy testable at
all. No timer, with the reason. The check is called only where nothing
is staged, asserted by a test that stages and expects deferral
- T4 kill -9 DURING compaction, extending the existing fork-based crash
battery. Asserts the PROPERTY — the store equals the pre- or the
post-compaction content, never a mixture, and every acked id survives.
Run repeatedly and state the count: it is a race, one green run proves
little
- T5 measure space reclaimed, boot before/after, and the stop-the-world
PAUSE against a stated budget. If the pause exceeds it, stop and report
— the alternatives are bought against that number, not before it
- T6 closeout, including the normative ordering rule in 04-db-binding.md
Constraints carried from the spec into every task:
- recovery must NOT change; a task editing the replay path should stop
- the dump must FLUSH PERIODICALLY. stage() grows the staging buffer by
doubling, so dumping a whole store through one buffer would hold the
entire store in RAM — the unbounded growth databasev2 1 identified as
how this engine dies
- a FAILED compaction is a missed optimisation, not a durability event,
so it must not take databasev2 4's fatal path
- gate tolerances must not be waived wholesale (part A's T4 made that
mistake), and the baseline is full-mode — writing a quick-mode baseline
over it is a regression part A also made
Deliberately NOT a task: rebuilding the `resident: keys` offset map. It
cannot be implemented against a feature that does not exist yet, so T6
records it as an obligation at the compactor and in the story instead of
a stub nobody can test.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
databasev2 3, chain 6. Brainstormed 2026-08-28 after databasev2 4 part A
landed.
Design: compact the log by rewriting it as one record per live row into a
temp file, fsync, rename over the live WAL, fsync the parent dir, reopen.
Recovery is COMPLETELY UNCHANGED — boot still opens one file and replays
it — and the crash criterion ("the same store as if the checkpoint had
never started") is satisfied by rename, not by code we must get right.
Read .dev/reference/postgresql for this. The finding is that PG's design
is UNAVAILABLE to us, which is what makes the simpler option legitimate:
- PG never compacts its WAL; segments before the redo point are recycled
by rename or unlinked. Its records are page deltas, so a compacted redo
log is not a store — hence heap files, a control file, a redo pointer,
a second recovery source and a separate process
- ours are FULL ROW IMAGES (apply_record implements UPDATE as
remove-then-recreate), so a compacted log IS a complete store. That one
difference deletes all of the above from the design
- what IS worth porting is the ordering discipline: publish the new
"recovery starts here" atomically and LAST, so a crash falls back. PG
needs a start-of-checkpoint redo pointer plus an end-of-checkpoint
control file update; we get the same property from one rename, because
we can swap the whole data set atomically and PG cannot
Forks settled:
- no snapshot format — the compacted log is the snapshot, existing grammar,
so no new encoder or decoder and the dump reuses wo_wal_append_insert
- one source, not two
- volume-only trigger, as a ratio against the LAST compaction's measured
output (the denominator is known exactly; estimating the live set would
mean estimating Text) with an absolute floor. NO TIMER — PG's exists to
bound loss from unflushed buffers and we have none; an idle log does not
grow. Copying the mechanism without the reason was the trap
- stop-the-world, with the pause measured against a stated budget rather
than assumed acceptable; alternatives are bought against a number
- compaction may run ONLY where nothing is staged (right after a barrier),
or a staged record lands in a file about to be replaced. Normative
Recorded before it can be found late: compaction invalidates every WAL
offset iteration 2's `resident: keys` stores, so the compactor rebuilds the
offset map as it writes. Nothing breaks today because that storage half is
unimplemented — it would break later, looking like corruption.
Also corrected exploration/postgresql/buffer-and-checkpoint.md, which was
wrong on two counts: PG does NOT update its control file by rename (in-place
full-block write + CRC32C), and its checkpoint sketch assumes writeonce has
segment files, which it does not and deliberately will not.
Grounding measured on master: seed 20000 leaves a 986614-byte log; 20000
updates take it to 2590262 bytes with the SAME live rows, and boot+verify on
that store is 155ms.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
databasev2 4 part A, task 6. Mostly documentation, plus one real fix the
full battery caught.
THE FIX. The drain held EVERY DB reply until the barrier — including
reads, which stage nothing and have no stake in durability. That parked
readers behind an fsync for no reason: durable.sN.mixread.p99 rose from
~1043us to 4057us. Only a statement that actually staged a record now has
its reply held. Caught by the gate, not by review.
THE TRADE, recorded rather than smoothed over. What remains is inherent: a
barrier blocks the owner shard LONGER (more records per fsync) though LESS
OFTEN, so anything queued behind one waits. Three full runs of the same
build gave durable.sN.mixread.p99 of 1043 / 2318 / 4147us and wmix.p99 of
8758 / 20000us — a 2-4x spread with the box near idle. So part A buys ~3x
write throughput at the cost of a longer, noisier tail on the owner shard,
and that is the strongest argument for part B (submit and keep serving).
- durable.sN.*.p99us tolerance widened to 100% WITH the reason in the
code: a 2-4x-variable tail gated at 50% gates the disk, not the engine.
The floor is the real guard and is not slack — mixread's (4172us) came
within 25us of tripping on the worst run. Baseline refreshed; a fresh
full run then passed 106 checks 0 failures
EXIT STATUS MOVED 3 -> 74 (sysexits EX_IOERR). 3 and 4 are already used by
SAMPLES for their own meanings — db-bench's own `verify` exits 3 on a
checksum mismatch, and it is the gate that exercises durability, so a
durability abort exiting 3 would have been indistinguishable from the
mismatch it should help diagnose. The low range belongs to programs.
Docs:
- story: progress, the payoff measured two ways, the cost side, criteria
split met/outstanding, and a "part B — its premise changed" section:
it was justified by "close the 66x gap", but that gap is two problems
and only the concurrent one was a batching problem
- board: standup entry in the six-question shape; both databasev2 4 rows
rewritten. They had said "close the 66x gap" — recorded as MIS-STATED
rather than quietly renumbered
- 00-wob-format.md and 04-db-binding.md: the normative failure contract
("a failed WAL commit traps WO_T_IO after un-applying the row") was
false; corrected, along with the tick-scoped group commit that never
happened
- database/src/CODE-LOGIC.md: where the barrier runs and why there, why
replies are held, why the inline path is asymmetric, the one failure
rule, and how to measure it
- db-bench README: the wmix mode, the env knobs, and the tmpfs warning
Battery: wovm-test 36 suites 0 fail, woc-test, oop-e2e 119/0,
db-bench 106/0, linkcheck clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Plan for the approved spec. Code-free per the repo convention
(docs/plan/discarded.md:54); the executor writes the code.
- T1 a failed barrier is detected and fatal — one entry point that names
the operation, errno, WAL path and batch size, then exits. The abort
path itself stays unexercised and the task says so rather than buying
coverage with a fault-injection switch
- T2 the barrier moves to the drain point and replies are held; the
request path stops committing per append. Riskiest task, and its risk
is one place: the crash legs. Plan says STOP if they fail, do not
adjust the test
- T3 the inline path takes the same fatal rule but keeps its own barrier,
with a comment explaining the asymmetry so the next reader does not
"fix" it. Looks like a no-op; without it the two paths disagree, which
is the unevenness the spec exists to remove
- T4 prove batches actually form BEFORE measuring the payoff — otherwise
a win gets attributed to the wrong cause. Also records peak staged
bytes, settling the no-cap decision with a number
- T5 measure, gate, write it down. If the payoff is absent, say so and
stop: part B must not start on an unproven premise
- T6 closeout, including the error catalogue — WO_T_IO leaving the write
path is language-visible and must be written down
Spec corrected while planning: it pointed at durable.s1.seed as the
payoff. Wrong, structurally — worker shards hold no WAL, so a queue only
exists when other shards write, and a serial writer has nothing to batch
with. The real target is durable.sN.mixwrite: 480 ops/s at p99 5888us
against s1's 1023 at p99 664, so adding shards currently makes durable
writing WORSE. That inversion is a better argument for the iteration than
the one the story recorded.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Brainstormed 2026-08-28. The iteration is split: part A batches, part B
(io_uring submission) is deferred until A's measurement says whether the
blocking boundary still dominates.
The story's premise needed correcting first:
- it says "replace fsync-per-commit with io_uring group-commit", but the
engine commits per STATEMENT — db.c calls wo_wal_commit right after
every append, all six sites, so each row change is one pwrite + one
fdatasync
- so two independent wins were being carried as one, and only the second
needs io_uring. The staging buffer already holds any number of records;
today it never holds more than one. Part A is mostly deleting calls
- iteration 22's numbers say A is where the payoff is: durable writes
4460 ops/s, mixwrite 1023 ops/s p99 664us, against 1.28M ops/s reads
Forks settled:
- batch boundary is QUEUE-DRAIN, not the tick this story had recorded: a
tick adds latency to a lone writer, taxing an idle system to serve a
busy one. Queue-drain self-tunes and needs no knob
- shard 0 holds each reply envelope instead of sending it, commits once
when the queue empties, then releases all — so a writer is acked after
the barrier carrying ITS record, which today is true only because
every batch has one member
- a failure between "RAM mutated" and "record durable" is a FATAL,
diagnosed abort. This replaces uneven behaviour that already exists:
insert rolls back, update and delete do not and say so in a comment
("RAM ahead of disk"). Batching would have multiplied that
- consequence stated, not slipped in: WO_T_IO leaves the write path
- no batch cap initially; peak staged bytes is measured so the question
is settled by a number
One gap disclosed rather than hidden: forcing a real fdatasync failure
needs mount privileges, so the unit test proves the error is DETECTED and
the abort itself stays covered by inspection.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Iteration 24 closes, absorbing 31 and 34. No code in this commit.
- stories 24, 31, 34 -> `status: done`, each with a landing banner. 24's
records the gate numbers and BOTH disclosed deviations: monitor takes
three arguments (the caller may be `main`, which has no mailbox) and a
v1 `call` reply is a typed scalar (which is what let the agreement be
checked at compile time, WO-E226). 31's notes it landed INSIDE 24 and
that a fifth mechanism it never anticipated came out of proving the
gate — the drain guarantee (40). 34's names the gap it did NOT close:
still no RNG, so CSRF/sessions stay blocked
- board: in-progress row cleared, marker doc deleted (convention), the
standup entry in the six-question shape, chain note — next link is
databasev2 4 (io_uring group-commit, chain 5)
- graph: PUBSUB2 (pub/sub + WebSockets, "rejected until here") -> done
- porch ledger: a WebSocket/pub-sub row added; the cancellation row now
says what it actually waits on rather than repeating "the arc"; the
README's "no WebSockets/SSE" limitation was stale — WebSockets are
supported, SSE and chunked encoding are not
- CODE-LOGIC: runtime/src gains the actor-lifecycle section (call, death,
the cap counter's sender/home-thread split, the monitor walk, the timer
list), the drain guarantee, and the digest section; docs/examples/chat
gains its own — actor topology, WHY two actors per connection, fd
ownership, and the shutdown choreography
Battery after the doc edits: wovm-test 36 suites 0 fail, woc-test exit 0,
oop-e2e 119/0, chat 11/0, web-app 46/0, linkcheck clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- narrative said "five of ten tasks landed" and named the branch as the
live location; T4/T5 (ids 89/90) had landed and the slice merged to
master 2026-08-27 (60414a1, fast-forward)
- records what was verified ON master: chat 11/0 at the full 1000-client
soak, runtime 36 suites 0 fail, compiler 556 checks, corpus 119 checks
- T10 closeout is what still holds stories 24/31/34 open
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A message sent before the stop flag is observed must be delivered and run
before the engine stops. One rule; a spin count could never express it.
- root cause in `shard_main` (runtime/src/vm.c): NEXT_RUNNABLE() already
stated the contract — "a WORKER on stop keeps DRAINING ... so queued
shutdown messages (close frames!) still run" — but the IDLE branch
contradicted it, calling fib_reap_all and breaking on WO_IO_STOP,
abandoning its inbox for wo_engine_stop() to free wholesale
- an actor between messages is exactly that idle case, which is why a WARM
soak server hid it: warm shards held live fibers and took the right path
- fix: while the primary's drain window is open, an idle worker adopts its
inbox and runs what arrives; sched_yield on an empty poll so a drain
cannot burn a core per shard and starve the actors it exists to let run
- unreachable at WO_SHARDS=1: wo_engine_stop returns early at nshards <= 1
Measured:
- fresh-server SIGTERM drain: 5 of 16 failing before, 20 of 20 clean after
- `just chat` at the FULL 1000-client soak: 11 checks, 0 failures, both
WO_IO backends, ASan clean with zero leaks
- the fd leg settled at scale too: 1000 connections left the count at 44,
unchanged after 20 more — lazy per-shard init, not a leak
- runtime battery 36 suites (18 x both dispatch flavors) 0 fail;
compiler 556 checks 0 fail
- story: docs/stories/language-runtime-database/40-shutdown-drain-guarantee.md
(chain 3 with 31, status done), board row, slice marker updated
- outstanding and named: a pin below the gate needs new multithreaded test
infrastructure — nothing in runtime/test/ drives wo_engine_start/stop and
no corpus fixture can trigger a stop
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Audit found 4 of 10 iterations citing it1 and 4 carrying stale claims the
measurement contradicts.
- 05: framing was contradicted, not merely incomplete. Its goal expected a
gradient to detect ("back-pressure before the cliff"); there is no cliff
— SIGKILL with swap off, exit 0 with swap on, and read latency STEPS
(1us -> 487us) rather than departing. Heading and goal rewritten; the
measurement makes the goal stronger, not weaker
- 05: budget must be bytes — 3.3x footprint spread — with headroom for
index doublings, else it fires during a rehash
- 05: new goal — eviction policy QUALITY is decisive, since getting the
resident set wrong costs 273x, not a few percent
- 06: its revival question now has a reference point. 273x is the KERNEL
SWAP path; `resident: keys` preads via page cache and must beat it. This
file revives only if 5c/5d lands near 273x rather than well below
- 04: write path is not where pressure bites (append ~1%, read 273x), so
the io_uring question that matters is iteration 2's deferred read-path
one, not group-commit
- 00-story: problem statement asserted the store "refuses the insert
rather than dying". Corrected in place — a banner above it was not
enough, a skimmer never reaches it
- residency spec: "swap thrash and the OOM killer" named exits that were
not measured; replaced with silence-or-a-corpse
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Closes the last gap in databasev2 1; gives databasev2 3 its "before".
- `boot` mode: does NOTHING. WO_DATA replay runs before main, so a mode
with no work measures replay plus a fixed startup
- `replayseed N M`: N inserts + M updates — same live rows, longer log
- `replay` leg: empty-store startup floor measured and SUBTRACTED, then
two shapes timed, median of 3 boots each
- premise check: updates must actually append WAL records, else the two
shapes are one measurement and the penalty means nothing
- WAL bytes = non-zero prefix, never file size (fallocate'd to 1 MiB)
- per-record cost stored in NANOseconds: as us it rounded 5.5 and 5.3 to
6 and 5, too coarse for the number a checkpoint exists to improve
- 148 checks, 0 failures; gate bites on a doctored ns_per_record
Measured — same 20 000 live rows, different history:
- 20 000 records: 980 035 B WAL, 110 ms replay, 5.5 us/record
- 40 000 records: 1 960 035 B WAL, 211 ms replay, 5.3 us/record
- 1.9x boot cost for an IDENTICAL dataset; per-record cost flat, so
replay is linear in records not rows
- extrapolated: 10M records ~55 s of boot, 100M ~9 min
- databasev2 3 correction: it planned to use "22's aged-store replay
numbers", which never existed — 22 proved restart correctness, never
timed it
- databasev2 3 hazard recorded: compaction rewrites the log and moves
every record, so it invalidates every `resident: keys` offset — an
arbitrary byte in a rewritten file, not stale-but-readable
- databasev2 1 -> status: done
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- `randread N R` in the sample: fill N rows, read R across the WHOLE range
- Weyl order `i*2654435761 mod n` — no RNG in the language, none needed;
both legs read the SAME key order so residency is the only variable
- `randread` driver leg: control (256 MiB, does not bind) vs over-cap
(6 MiB + swap), sizes kept modest — quick resolves it in ~5s
- gates the RATIO, not the absolutes: over-cap reads/sec belongs to the
box's swap device, the factor between two runs belongs to the engine
- reads must all resolve (hits == R) or the leg fails; a collapse measured
over unresolved reads is noise
- 133 checks, 0 failures; gate bites on a doctored collapse_x
Measured — this closes the gap the swap leg left:
- resident 1 851 166 reads/sec, p50 0us p99 1us
- over-cap 6 771 reads/sec, p50 128us p99 487us
- 273x throughput, ~480x p99, all 20 000 reads resolving in both
- so the two access patterns sit ~270x apart under identical pressure:
append-mostly insert ~1%, random read 273x
- departure is a STEP not a curve (1us -> 487us, nothing between), which
is why p99_departure_decile finds no knee — there is none
- caveat recorded, NOT inherited: this is demand-paged anonymous memory
through swap (4 KiB/fault, no readahead). `resident: keys` preads via
the page cache — should be better, but databasev2 2 task 7 must measure
its own read path. New criterion added there
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- `Wide` text-heavy reference shape beside Int-only `Item`
- `growth N int|text`: per-decile RSS read from own /proc/self/status
- `growth-verify`: survivor of a crash must be a contiguous intact prefix
- four footprint legs under a rootless cgroup v2 cap, swap on/off
- `ceiling` leg: die at the cap, then replay must come back intact
- footprint read as median-of-marginals; doublings a separate metric
- 121 checks, 0 failures; footprint gated ±10%, kill-timing ±100%
Measured, and it inverted two of the iteration's own predictions:
- footprint 96.5-100 B/row Int vs 320.6-324 B/row text = 3.3x, NOT the
"order of magnitude" three docs asserted
- table storage has NO checked ceiling: SIGKILL signal 9, not a catchable
WO_T_OOM. overcommit lets malloc succeed; kernel kills on page touch
- swap is NOT latency collapse: 900k rows 148s capped-with-swap vs 150s
uncapped. Append-mostly never re-touches cold pages
- ack-after-fsync survives an OOM kill: ~40k rows, no holes, no corruption
- iteration 2's budget dependency is REMOVED not satisfied — there is no
"swap onset" to derive a fraction from
- fix: subprocess returncode -9 was labelled a "checked refusal"; 137 is
the shell spelling of the same signal
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- `readiness: ready | refine` is a SECOND axis, orthogonal to status.
`ready` = the brainstorm is complete and the decisions are LOCKED (a spec
approved, or the forks explicitly confirmed). `refine` = open forks remain
and it cannot be planned yet
- `status: refine` RETIRED because it carried both meanings at once, so a held
iteration with an approved spec (language 18, 26) was indistinguishable from
one nobody had thought about. status is now purely where the WORK is:
done | in-progress | pending | hold — `pending` was already the board's own
rendering word, so nothing new was invented
- all 47 iterations classified from EVIDENCE in their own text, not by guess:
"the four forks are SETTLED" / "spec + plan approved" / "Approved spec:" for
ready; "Forks the spec must settle" / "no spec exists yet" for refine. Every
shipped iteration is ready by definition. 19 done, 5 in-progress, 15
pending, 8 hold; 27 ready, 20 refine
- two iterations moved refine -> in-progress rather than -> pending: language
31 and 34 are absorbed into 24 and work on them is literally happening, which
the board already showed as 🔄 while their frontmatter said otherwise. That
disagreement is now gone
- board legend, board-views' frontmatter contract, and two new Dataview
queries updated — the useful one being `readiness: ready AND status:
pending`, the startable set
WHAT THE NEW AXIS IMMEDIATELY SURFACED: of 15 pending iterations, exactly ONE
is startable — databasev2 4, io_uring group-commit, whose forks were confirmed
settled 2026-08-20. Everything else pending needs a brainstorm first. That was
invisible while one key carried both meanings, and it is now on the board.
Also caught by the sweep, unrelated to readiness but found by cross-checking
frontmatter against the board: SIX duplicate rows. Every iteration moved into
databasev2 was still listed in the LANGUAGE pending table under its retired id
(23, 32, 33, 20, 21, 27) as well as its new one. Stale copies removed. And two
databasev2 rows made claims the sweep contradicts — iteration 1 was billed
"startable today" while its forks are open, and 6 still called itself the
ceiling-raiser after 2 took that role.
Docs only. linkcheck 0 broken / 0 anchors.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Flagged by the developer: the iteration still described the pre-brainstorm
three-mode design behind a "superseded in part" banner while four tasks had
landed against it.
- iteration 2 rewritten around the shape as built: two keys
(`durable: true|false`, `resident: all|keys`), not a `mode:` enum with
`cold`. status refine -> in-progress
- added a task-by-task progress table with commit hashes, and split the
acceptance criteria into MET (each with how it was verified, not just that
it passed — e.g. the goldens-unchanged claim is `git diff` over golden/
being empty after a WOC_BLESS run, since blessing rewrites all of them) and
OUTSTANDING with the task that owns each
- kept the history rather than deleting it: the three-mode replacement, the
"one real rewrite" that was fiction, and the opposite half that turned out
genuinely deep. An iteration file is where that record belongs
- board row rewritten to agree; the track index's "the lever" section, its
principle-7 paragraph and its sequence rationale all still taught the dead
three-mode design
ITERATION 6 IS NOW LARGELY SUPERSEDED, and bannered as such rather than
quietly gutted. `resident: keys` is the ceiling-raiser and it lives in
iteration 2 (tasks 5c/5d). More than relocated: 6's premise — a user-space
resident working set with faulting and 5's eviction policy — was specifically
REJECTED by the spec in favour of the kernel page cache, since a pread against
a cached page is a memcpy. What may still be left is recorded honestly: revisit
only with a measurement showing the page cache insufficient. Its fork list
survives, especially "does the language surface the fault cost at the use
site", which is still open and still the largest question about what writeonce
is. The sequence rationale is amended too — it had 6 as the ceiling-raiser and
5 as a prerequisite on the critical path; neither holds.
Docs only. linkcheck 0 broken / 0 anchors.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- `resident: all | keys` replaces `resident: all | index`. Two reasons beyond
taste: it kills the collision with the `index:` argument
(`@table(index: [customer], resident: index)` read badly), and it puts both
values on ONE axis — each now answers "what row data stays resident",
where `all`/`index` mixed a quantity with a structure name
- accurate as well as clearer: what stays resident is the id->offset map, the
secondary indexes and the unique shadows — all key structures; row payloads
are exactly what leaves. `resident: none` was rejected as overclaiming,
since the indexes very much are resident
- checked for collisions: neither `all` nor `keys` is a keyword or a builtin
(`key_at`/`val_at` exist, bare `keys` does not)
- the spec's wart note became a recorded decision; the rejected spelling is
kept quoted so the rationale still reads
- fixes a bug I introduced in the 2026-08-26 track move: all six moved
iterations carried a banner reading "Part of [Story — the database beyond
RAM]" whose link pointed at the LANGUAGE arc — correct target, lying text,
the exact failure mode the link audit warned about. Banners now point at
the databasev2 story, and the original "Part of" line says plainly which
track the iteration was authored in before the move
- linkcheck 0 broken / 0 anchors
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- driving case: a 120 GB order table on a 32 GB host. Not a tuning problem;
no eviction policy fixes it. Developer accepted reconsidering the principle
- principle 7 rewritten: durability half UNCHANGED and unconditional
(WAL-logged, fsync before ack, CRC-dropped torn tail); residency half
demoted from law to per-table declaration. Old wording quoted in place so
the amendment is legible, with the reason: a doctrine a real workload
cannot satisfy gets ignored, and the failure it produced was an OOM kill
- spec: docs/superpowers/specs/2026-08-26-table-residency-design.md
One log-structured engine — the WAL already holds every row, so keep an
in-RAM id->offset map and pread rows back. No second engine, no user-space
row cache (the kernel page cache is the hot copy, which is already this
repo's stated position and why it avoids O_DIRECT)
- arithmetic that makes it work: 240M rows x 16 B of index = ~3.8 GB
resident in 32 GB. Indexes stay resident, rows do not. Buys ~2 orders of
magnitude, not infinity — stated plainly in the spec
- grammar: two optional keys, `durable: true|false` and `resident: all|index`,
both defaulting to today's behaviour, so all 28 existing @table
declarations compile untouched and no golden is reblessed
- rejected, with reasons recorded: mmap (rows are pointer-bearing —
table.c returns (uintptr_t)t as the slot word), buffer pool (the Rust-era
phase-12 design that died with that track), paged B-tree (stays rejected),
a three-valued enum, automatic spill, disk-backed-by-default
- self-review caught the budget defaulting to "none" while promising the ERP
developer a diagnostic instead of the OOM killer — contradiction fixed:
the budget defaults to a fraction of host memory, and its value comes from
databasev2 1's swap-onset measurement
- live docs that contradicted the amendment updated (subagent doctrine,
its guide, discarded.md's two rows, iteration 04's read claim, 07, 38);
dated specs/plans left as records. linkcheck 0 broken / 0 anchors
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- docs/stories/databasev2/, numbered from 1. Six PENDING database iterations
moved from the language track and renumbered, keeping the old id in
`was_language_iteration:` so a search for "iteration 32" still finds it:
32 -> 3 WAL checkpoint, 23 -> 4 io_uring commit, 33 -> 7 single-file store,
27 -> 8 query grammar, 20 -> 9 cross-program, 21 -> 10 keypair auth.
Done work (9, 9b, 22) stays as v1 history; language 18 left whole
- the problem, read off the engine not guessed: rows are malloc'd slabs with
addresses stable forever, NO eviction/spill/paging anywhere in database/src,
the WAL never checkpoints so boot replays all history, and durability is one
process-global WO_DATA so no table can say it matters more than another.
An allocation failure IS a clean catchable WO_T_OOM — but swap thrash
arrives first and carries no error signal at all, which is the real hazard
- four new iterations:
1 measure the ceiling FIRST (curve not cliff; the three exits; kill -9 at
exhaustion) — every later default should follow from a number
2 `@table(mode: ram | durable | cold)` — the grammar ask. Small surface
(Ast.table_cfg gains a key, the parser already rejects unknown args), big
semantics: `durable` defaults so nothing changes silently, and the
compiler refuses a durable row holding a `ref` into a ram table
5 bounded tables + refuse/evict/back-pressure, shedding BEFORE the OS acts
6 cold tiering — mostly forks, incl. whether the language surfaces the
fault cost and whether @unique on cold is refused outright. A paged
B-tree stays rejected: if tiering needs one, reject tiering
- 39 links repointed, link TEXT renumbered to track-local ids; arc gains one
pointer row replacing the six moved; board + board-views cover three tracks
- linkcheck 0 broken / 0 anchors; no code blocks in any story
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- docs/stories/porch/ — a TRACK folder, not a status folder: status still
lives only in frontmatter. Adds `track: porch` so a query over
docs/stories/ can tell a porch 3 from a language 3
- 00-story.md carries the sequence, the dependency graph, and a table of
what the track explicitly does NOT own (binding -> 29, cache -> 18,
proxy -> 38, metrics -> 30, TLS/templates -> doctrine)
- eight iterations, each with phases, per-phase tasks, Given/When/Then
criteria, out-of-scope and the forks a spec must settle:
1 store-backed middleware (limiter + idempotency — needs nothing new,
first on purpose so the store pattern is proven cheaply)
2 randomness + cookies (phase A is language-track: a CSPRNG builtin;
`Resp.headers` being a map cannot emit two Set-Cookie lines)
3 sessions 4 CSRF 5 routing/response ergonomics (independent)
6 streaming core (the seam 7 and 8 wait on; chunked-request refusal
must survive) 7 SSE + compression 8 static + lifecycle hooks
- language iteration 39 -> status: hold, retitled superseded, with a row
mapping each of its goals to the porch iteration that took it. Kept, not
deleted: the Fiber study cites it and its randomness argument is what
this track is built on
- board gains a porch section; board-views gains porch and both-track
Dataview queries; porch README and the Fiber study §7 point at the track
- no code blocks in any story (plans carry concept and actions in words);
linkcheck 0 broken / 0 anchors
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- docs/examples/writeonce-serve -> docs/examples/porch (git mv, history kept);
`[deps]` key and import are now `porch` / `porch/http` / `porch/router`
- name history preserved on the library README, not rewritten into dated
records: writeonce-framework -> writeonce-serve (08-25) -> porch (08-26).
Stories, specs, plans and the audit reports keep the older name by the
repo's own convention; only live docs and every path link were rewritten
- left alone deliberately: `internal/serve.wo`, `pub fn serve`, `serve_conn`,
`app.serve(...)` — those are functions, not the module name
- web-app/wo.toml comment corrected: it claimed hyphens are not identifier
characters and named a key this file never used. lexer.ml's `is_ident_cont`
DOES accept `-` (an internal dash is part of the identifier, which is why
binary minus needs spaces), so a hyphenated key would be legal too
- site now teaches the name: package card, the two-deps chapter and the
handlers-are-classes chapter say `porch`; site-accept asserted the old
/packages/serve route and caught the rename, as a gate should
- gates: web-app 46/0, site 21/0, deps-accept 8/0, oop-e2e 116/0,
linkcheck 0 broken / 0 anchors; porch typechecks entry-less as kind=library
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- README: shipped concurrency/HTTP/WebSockets sat in the roadmap as "not yet
available"; "no package manager" contradicted [deps]; the deps example
would not have compiled (the key IS the module name)
- runtime/README: leads with wovm, wo-rt.c demoted to a historical section;
dropped 2 nonexistent recipes, crates/rt, @gc refcounting, 13 suites -> 18
- employee + log-watcher READMEs claimed "does not compile"; both are gates
- error catalog: +10 emitted codes incl WO-E250, the only diagnostic the
shipped query surface raises; recorded why the sweep rotted
- language-surface: group-by parses, then the typechecker refuses it
- 00-code-review + 00-link-audit re-run; history kept, not rewritten
- 48 dead Rust-era exploration links de-linked rather than re-pointed (their
prose names the retired plan by number); successor map -> discarded.md
- 08-project-structure: compiler/plan/ never existed; corpus has 9 dirs, 5 empty
- releasing.md: dropped a --draft step the workflow never had
- new docs/00-doc-audit.md: findings + disposition, incl one row where the
audit was wrong and the doc it accused was right
- status folders removed: 34 stories flat, status only in frontmatter; 252
links recomputed from resolved paths; board/board-views/structure retaught
- story 24 -> in-progress, since frontmatter is now the only truth
- new iteration 38: fs mutation verbs + net.connect, the two capability
families no iteration owned
- new iteration 39: gofiber/fiber v3.5.0 parity study. The ledger called
CSRF/sessions unblocked by iteration 34's HMAC, but the runtime has no
source of randomness at all
- linkcheck skips .dev/.superpowers: 0 broken paths, 0 bad anchors
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- rename the two libraries: writeonce-framework -> writeonce-serve
(`use serve`), wo-html -> writeonce-view (`use view`). Names say the
ROLE now; every sample, script, gate and live doc follows
- stories/specs/plans keep the old names: they are dated records, and
both library READMEs carry a "renamed 2026-08-25" note
- serve/http/files.wo: StaticFiles { dir, max_bytes } — traversal
refused not normalised, extension content types, attachment
disposition for archives. Lifted out of the shop, which had said in
a comment that it belonged in the framework
- shop drops its private copy and mounts the framework's
- site: /dl/*path over $WO_DIST (default ./dist), 16 MiB ceiling
- /install gains supported systems — Linux x86-64, glibc >= 2.38,
not musl — read off `file` and the binaries' GLIBC_ symbol
versions, not off a wish list; plus GitHub release as primary,
/dl as mirror, and the sha256 verify step
- site-accept: 17 -> 21 checks (supported systems, gzip download with
a binary-safe probe, checksum, /dl traversal 404)
Verified on 192.168.0.165: the real 960,820-byte tarball downloads
as application/gzip and its sha256 matches the published digest.
Gates: oop-accept MET, site 21/0, web-app 46/0, fibers 10/0,
db-actor 8/0; shop rebuilt and its /assets served by the framework.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- lexer: backtick raw text literal — content verbatim, no escape
processing, common source margin removed at lex time; `${ }` raw and
`{{ }}` auto-escaping holes
- `{{ e }}` desugars to `esc(${e})` in parser.ml — a Call on the `esc`
in scope, so types/owner/emit/.wob/VM are untouched
- WO-E004 unterminated raw literal; WO-E005 newline inside "..." —
closes a hole where a missing quote silently ate the rest of the file
- wo-html: `Component` interface, `render_all`, `Layout`, README
- framework: `ok_html` joins ok_text/ok_json in http/types.wo
- site + shop restructured to one-feature-one-module MVC (view +
controller per directory, model at the root, bootstrap-only main)
- removed the filler `pad: Int` convention — verified unnecessary for
plain classes, interface dispatch, containers and actors
- corrected recorded claims: gap #1 blocks neither the build nor the
layout; a class crosses module lines, only a free fn is scoped
- docs/guides/language-surface.md — the full grammar inventory
- story 37 landed and moved to done/
Gates: oop-accept MET, oop-e2e 116/0, woc-test 556/0, site 11/0,
web-app 46/0, fibers 10/0, db-actor 8/0
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- separate view.html files dropped; every render() now carries its
'-- 37 target:' literal above the hand-lowered body — the pair is
the DX referendum in one file
- story 37 re-pointed: raw multi-line literal + {{ }} auto-escaped
typed holes + {!! !!} raw slots; structural control stays if/for;
w:if/w:for and .html files demoted to later; forks revised
- rebuild verified on untouched toolchain
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- runtime ids 91-95: net.read_dl/accept_dl/write_dl (per-call deadline,
nil/false = the EXPECTED timeout; ms<=0 = old behavior bit for bit),
net.listen_unix (unlink-before-bind, O_NONBLOCK on the listener —
probe-found: accept4's flag covers accepted sockets only), net.peer
- plane: one-op-per-park stays law — deadlines ride one per-shard
TIMEOUT tick (sentinel user_data) + post-CQE expiry sweep +
POLL_REMOVE tombstone; epoll's deadline scan grew the fd-park case;
fibers POOL instead of freeing mid-run (stale-CQE UAF); plain parks
zero park_deadline (no stale sleep deadlines)
- probe: all five seams verified on BOTH WO_IO backends (timeout
timing exact, peer round-trip, unix rebind)
- framework: parse_request grows first_ms/read_ms; serve_conn — the
keep-alive loop with deadlines where parked idle conns are LEGAL
(close-when-idle RETIRED); App.handle_conn exposes it; plain serve()
unchanged for simple apps
- web-app: app-owned accept_dl loop + ConnWorker actor per connection
(each builds its own App; cross-shard placement rides the DB actor);
WA_IDLE_MS knob; gate grows to 41 checks — two slow requests served
in PARALLEL, stalled client evicted at the idle deadline, slow-loris
torn at the read deadline (400)
- docs: story 35 -> done with banner; SQE/CQE design spec LANDED (was
the review doc); ledger rows (timeouts/unix/keep-alive/peer), graph
(NETSEAM cleared, KEEPAL done), builtin-surface rows, runtime
CODE-LOGIC section, board entry
- battery 13/13 fresh-built
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- story 34: premise fixed — iteration 36 landed bitwise/hex, digests
and HMAC now expressible in pure .wo; C-builtin vs pure-.wo is the
story's brainstorm call, not an impossibility
- dependency graph: crypto gate names story 34; net-seam gate names
story 35; radix gate corrected — 22 benched the DB, router scan
still unmeasured, perf-targets entry first
- framework README ledger: timeouts/unix/peer rows point at story 35;
ETag row at story 34; path-matching row repointed off iteration 22
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- brings time.ticks builtin (id 84), bench sample + campaign driver
(scripts/db-bench.py), just db-bench/db-bench-quick recipes, first
baseline recorded, story 22 to done/, postgres study cards
- conflicts resolved: board in-progress table (iteration 36 +
framework rows kept, db-bench row now "22 landed"; dangling order
anchor repointed); story 36 moved back to in-progress/ (dir-rename
inference dragged it to done/ — 36 still awaits the manual pass)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- owns the framework ledger's three seam rows; deadlines compose with
the arc's park plane (POLL_ADD+TIMEOUT fork recorded); framework
knobs explicitly out of scope; stalled-client soak in acceptance
- board + table rows (held seqs bumped); pairs naturally with 24
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- SHA-1 (WS-handshake hard req, RFC 6455 worked example as acceptance),
SHA-256, HMAC-SHA256 over Bytes; hand-rolled C per doctrine, FIPS/RFC
vector fixtures; four forks recorded (namespace, shape, source, file)
- gates chain item 24; digest floor for held 21 + ETag row; 24's
dependency note repointed; board + table rows (held seqs bumped)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- file path IS the wal; dir form byte-identical; one fork (nonexistent
path semantics) leaning recorded; off-chain, driver-only
- board + story table rows (held tail seqs bumped)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- bench/baseline.json: 74 metrics from the first full campaign;
tolerances tuned by a two-run repeatability check (mix* 50%,
read/query 35%, rest 15% — rationale in _config)
- gate bites: --check mode; doctored copy fails, both real runs 74/0
- headline: durable seed 4.5k/s vs ram 297k/s (23's case); reads
O(table) at ~1.5k/s; mixread 1280 vs 21 ops/s single-vs-multi
(the arc's price); msgrate 13.4M vs 2.45M (mutex-inbox number)
- arc delta recorded in story 8; findings in sample README
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- five-property map in marker doc: atomicity/durability/recovery
proven or held (18); concurrency control = stage 3's property;
space reclamation RAM done (slot reuse), disk = new story 32
- story 08: three stage-3 criteria (one commit per write RPC +
ack-after-owner-fsync, workers WAL-free + replay-before-serve,
no torn reads under TSan corpus); arc plan stage 3 carries them
- refine/32-wal-checkpoint.md: snapshot + truncate, bounded replay,
crash-during-checkpoint safe; four forks; after 23
- chain now stage 3 -> 22 -> 31 -> 24 -> 23 -> 32 in all 10 docs;
boards + seq bumps synced
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
