Commit graph

473 commits

Author SHA1 Message Date
62d29d6a77 docs(24): T10 closeout — stories done, board, graph, ledger, CODE-LOGIC
Iteration 24 closes, absorbing 31 and 34. No code in this commit.

- stories 24, 31, 34 -> `status: done`, each with a landing banner. 24's
  records the gate numbers and BOTH disclosed deviations: monitor takes
  three arguments (the caller may be `main`, which has no mailbox) and a
  v1 `call` reply is a typed scalar (which is what let the agreement be
  checked at compile time, WO-E226). 31's notes it landed INSIDE 24 and
  that a fifth mechanism it never anticipated came out of proving the
  gate — the drain guarantee (40). 34's names the gap it did NOT close:
  still no RNG, so CSRF/sessions stay blocked
- board: in-progress row cleared, marker doc deleted (convention), the
  standup entry in the six-question shape, chain note — next link is
  databasev2 4 (io_uring group-commit, chain 5)
- graph: PUBSUB2 (pub/sub + WebSockets, "rejected until here") -> done
- porch ledger: a WebSocket/pub-sub row added; the cancellation row now
  says what it actually waits on rather than repeating "the arc"; the
  README's "no WebSockets/SSE" limitation was stale — WebSockets are
  supported, SSE and chunked encoding are not
- CODE-LOGIC: runtime/src gains the actor-lifecycle section (call, death,
  the cap counter's sender/home-thread split, the monitor walk, the timer
  list), the drain guarantee, and the digest section; docs/examples/chat
  gains its own — actor topology, WHY two actors per connection, fd
  ownership, and the shutdown choreography

Battery after the doc edits: wovm-test 36 suites 0 fail, woc-test exit 0,
oop-e2e 119/0, chat 11/0, web-app 46/0, linkcheck clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 00:06:31 +02:00
7833dd5740 feat(gates): example apps log to /tmp/<example-app>.log so it can be tailed
Every gate wrote its server output into a per-run mktemp dir that its own
cleanup trap deletes on exit — nothing to follow during the run, nothing
to read after it.

- chat -> /tmp/chat.log, web-app -> /tmp/web-app.log,
  site -> /tmp/site.log, log-watcher -> /tmp/log-watcher.log
- truncated once at gate start, appended for the rest of the run, so one
  file holds the whole run in order
- each gate PRINTS the path as its first line, with the tail -F command
- legs are banner-separated and name their port and env
  (===== leg 2 - port 18902 - env WO_IO=epoll =====)

Appending breaks readiness detection unless it is leg-scoped:

- serve() used to grep the whole file for `listening`, which after the
  switch to append would match an EARLIER leg and return before the new
  server was up. It now records the line count first and searches only
  tail -n "+$LEGFROM"; the ASan scan is scoped the same way
- log-watcher's checks grep per-invocation files, so those are kept and
  the output is teed into both — process substitution adds no pipeline
  stage, so $! is still the command's pid the gate kills and waits on
- its one SYNCHRONOUS invocation appends after it finishes rather than
  teeing: the grep on the next line would race tee's flush

Verified, all green: chat 11/0, web-app 46/0, site 21/0, log-watcher 7/0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 23:53:10 +02:00
d87846354e docs(board): iteration 24 is 9 of 10 and on master; only T10 closeout remains
- narrative said "five of ten tasks landed" and named the branch as the
  live location; T4/T5 (ids 89/90) had landed and the slice merged to
  master 2026-08-27 (60414a1, fast-forward)
- records what was verified ON master: chat 11/0 at the full 1000-client
  soak, runtime 36 suites 0 fail, compiler 556 checks, corpus 119 checks
- T10 closeout is what still holds stories 24/31/34 open

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 23:44:56 +02:00
60414a1754 feat(runtime): the shutdown drain guarantee — iteration 40
A message sent before the stop flag is observed must be delivered and run
before the engine stops. One rule; a spin count could never express it.

- root cause in `shard_main` (runtime/src/vm.c): NEXT_RUNNABLE() already
  stated the contract — "a WORKER on stop keeps DRAINING ... so queued
  shutdown messages (close frames!) still run" — but the IDLE branch
  contradicted it, calling fib_reap_all and breaking on WO_IO_STOP,
  abandoning its inbox for wo_engine_stop() to free wholesale
- an actor between messages is exactly that idle case, which is why a WARM
  soak server hid it: warm shards held live fibers and took the right path
- fix: while the primary's drain window is open, an idle worker adopts its
  inbox and runs what arrives; sched_yield on an empty poll so a drain
  cannot burn a core per shard and starve the actors it exists to let run
- unreachable at WO_SHARDS=1: wo_engine_stop returns early at nshards <= 1

Measured:

- fresh-server SIGTERM drain: 5 of 16 failing before, 20 of 20 clean after
- `just chat` at the FULL 1000-client soak: 11 checks, 0 failures, both
  WO_IO backends, ASan clean with zero leaks
- the fd leg settled at scale too: 1000 connections left the count at 44,
  unchanged after 20 more — lazy per-shard init, not a leak
- runtime battery 36 suites (18 x both dispatch flavors) 0 fail;
  compiler 556 checks 0 fail

- story: docs/stories/language-runtime-database/40-shutdown-drain-guarantee.md
  (chain 3 with 31, status done), board row, slice marker updated
- outstanding and named: a pin below the gate needs new multithreaded test
  infrastructure — nothing in runtime/test/ drives wo_engine_start/stop and
  no corpus fixture can trigger a stop

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 23:43:45 +02:00
3bc85d85f3 docs(slice): marker reflects T4/T5 landed, the drain blocker, and the stale-artifact trap
- T4 monitor + T5 time.after landed in 4092074 (ids 89/90); marker still
  listed them pending because it came from master, which lacks that commit
- records the branch baseline (18 suites x 2 flavors, 0 fail) and the gate
  at 11 of 12 legs green
- names the stale-artifact trap: after a branch switch, compiler/_build and
  runtime/build hold the OTHER branch's binaries, and a v7-vs-v6 mismatch
  surfaces only as "no listener"
- the drain guarantee is now the single named blocker; nothing else in the
  slice should land before it

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 23:28:25 +02:00
4af1e8bcdd fix(chat gate): every leg starts its own server — and it found a real bug
Gate defects, all measured:

- fd check was core-count dependent: `fds_before + 8` read LAZY per-shard
  init as a leak. Shards init on first fiber, each taking one io_uring +
  one eventfd, capped at nproc; on 20 cores the first wave legitimately
  adds 18. Measured 26 -> 44 after 20 clients, still 44 after 40 more.
  Replaced with the invariant the check is for: a second wave must not
  raise the count. Core-count independent, and catches a slow leak that
  any fixed slack would hide
- a failed leg ORPHANED its server: drain inherited $SRV from the soak
  leg, so its python died on int("") and the soak server was never
  killed — its listener then broke the next run's soak on the same port.
  drain now starts its own server; cleanup kills every server a run
  started, matched on the run's unique temp dir
- two legs the plan requires were missing: WO_SHARDS=1 (the single-shard
  control) and WO_MAILBOX=8 (drop-slow-member backpressure). Both added,
  both green. The mailbox leg shrinks the slow client's SO_RCVBUF so it
  needs no sleeps
- chat adopted the porch naming (use porch/..., [deps] key) after the
  rename landed on master

Decoupling the legs exposed a REAL drain bug, traced and documented in
docs/2026-08-27-chat-drain-finding.md, NOT fixed here:

- on a FRESH server the SIGTERM drain is flaky: 5 of 16 runs left a
  client at EOF with no close frame and no diagnostic
- traced: main -> Registry -> Room -> Writer. Registry runs (diag
  confirms), the Room NEVER processes its shutdown message, so the
  Writer's close branch never runs. Clients that do get a frame are
  saved by their own Reader seeing env.stopping()
- ruled out: the spin budget (a 1s wall-clock deadline still failed 2 of
  12 — reverted, it fixed nothing and cost 1s per shutdown),
  dummy_writer() spawning during shutdown, and write failure
- the fix is an engine guarantee — a send issued before the stop flag is
  delivered — which belongs to the actor lifecycle, not a spin count

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 23:27:46 +02:00
ebc3522c40 Merge branch 'master' into chat-ws-lifecycle 2026-08-27 23:11:49 +02:00
3507aafea3 docs(databasev2): propagate iteration 1's findings to every consumer
Audit found 4 of 10 iterations citing it1 and 4 carrying stale claims the
measurement contradicts.

- 05: framing was contradicted, not merely incomplete. Its goal expected a
  gradient to detect ("back-pressure before the cliff"); there is no cliff
  — SIGKILL with swap off, exit 0 with swap on, and read latency STEPS
  (1us -> 487us) rather than departing. Heading and goal rewritten; the
  measurement makes the goal stronger, not weaker
- 05: budget must be bytes — 3.3x footprint spread — with headroom for
  index doublings, else it fires during a rehash
- 05: new goal — eviction policy QUALITY is decisive, since getting the
  resident set wrong costs 273x, not a few percent
- 06: its revival question now has a reference point. 273x is the KERNEL
  SWAP path; `resident: keys` preads via page cache and must beat it. This
  file revives only if 5c/5d lands near 273x rather than well below
- 04: write path is not where pressure bites (append ~1%, read 273x), so
  the io_uring question that matters is iteration 2's deferred read-path
  one, not group-commit
- 00-story: problem statement asserted the store "refuses the insert
  rather than dying". Corrected in place — a banner above it was not
  enough, a skimmer never reaches it
- residency spec: "swap thrash and the OOM killer" named exits that were
  not measured; replaced with silence-or-a-corpse

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 22:47:37 +02:00
5b1a8c96a1 feat(db-bench): replay baseline — boot cost tracks history, not data
Closes the last gap in databasev2 1; gives databasev2 3 its "before".

- `boot` mode: does NOTHING. WO_DATA replay runs before main, so a mode
  with no work measures replay plus a fixed startup
- `replayseed N M`: N inserts + M updates — same live rows, longer log
- `replay` leg: empty-store startup floor measured and SUBTRACTED, then
  two shapes timed, median of 3 boots each
- premise check: updates must actually append WAL records, else the two
  shapes are one measurement and the penalty means nothing
- WAL bytes = non-zero prefix, never file size (fallocate'd to 1 MiB)
- per-record cost stored in NANOseconds: as us it rounded 5.5 and 5.3 to
  6 and 5, too coarse for the number a checkpoint exists to improve
- 148 checks, 0 failures; gate bites on a doctored ns_per_record

Measured — same 20 000 live rows, different history:

- 20 000 records:  980 035 B WAL, 110 ms replay, 5.5 us/record
- 40 000 records: 1 960 035 B WAL, 211 ms replay, 5.3 us/record
- 1.9x boot cost for an IDENTICAL dataset; per-record cost flat, so
  replay is linear in records not rows
- extrapolated: 10M records ~55 s of boot, 100M ~9 min

- databasev2 3 correction: it planned to use "22's aged-store replay
  numbers", which never existed — 22 proved restart correctness, never
  timed it
- databasev2 3 hazard recorded: compaction rewrites the log and moves
  every record, so it invalidates every `resident: keys` offset — an
  arbitrary byte in a rewritten file, not stale-but-readable
- databasev2 1 -> status: done

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 21:25:24 +02:00
a873cf7331 feat(db-bench): random-read-over-cap leg — the 273x collapse
- `randread N R` in the sample: fill N rows, read R across the WHOLE range
- Weyl order `i*2654435761 mod n` — no RNG in the language, none needed;
  both legs read the SAME key order so residency is the only variable
- `randread` driver leg: control (256 MiB, does not bind) vs over-cap
  (6 MiB + swap), sizes kept modest — quick resolves it in ~5s
- gates the RATIO, not the absolutes: over-cap reads/sec belongs to the
  box's swap device, the factor between two runs belongs to the engine
- reads must all resolve (hits == R) or the leg fails; a collapse measured
  over unresolved reads is noise
- 133 checks, 0 failures; gate bites on a doctored collapse_x

Measured — this closes the gap the swap leg left:

- resident 1 851 166 reads/sec, p50 0us p99 1us
- over-cap    6 771 reads/sec, p50 128us p99 487us
- 273x throughput, ~480x p99, all 20 000 reads resolving in both
- so the two access patterns sit ~270x apart under identical pressure:
  append-mostly insert ~1%, random read 273x
- departure is a STEP not a curve (1us -> 487us, nothing between), which
  is why p99_departure_decile finds no knee — there is none

- caveat recorded, NOT inherited: this is demand-paged anonymous memory
  through swap (4 KiB/fault, no readahead). `resident: keys` preads via
  the page cache — should be better, but databasev2 2 task 7 must measure
  its own read path. New criterion added there

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 20:56:19 +02:00
0c9b2c45d8 feat(db-bench): measure the RAM ceiling — databasev2 1
- `Wide` text-heavy reference shape beside Int-only `Item`
- `growth N int|text`: per-decile RSS read from own /proc/self/status
- `growth-verify`: survivor of a crash must be a contiguous intact prefix
- four footprint legs under a rootless cgroup v2 cap, swap on/off
- `ceiling` leg: die at the cap, then replay must come back intact
- footprint read as median-of-marginals; doublings a separate metric
- 121 checks, 0 failures; footprint gated ±10%, kill-timing ±100%

Measured, and it inverted two of the iteration's own predictions:

- footprint 96.5-100 B/row Int vs 320.6-324 B/row text = 3.3x, NOT the
  "order of magnitude" three docs asserted
- table storage has NO checked ceiling: SIGKILL signal 9, not a catchable
  WO_T_OOM. overcommit lets malloc succeed; kernel kills on page touch
- swap is NOT latency collapse: 900k rows 148s capped-with-swap vs 150s
  uncapped. Append-mostly never re-touches cold pages
- ack-after-fsync survives an OOM kill: ~40k rows, no holes, no corruption
- iteration 2's budget dependency is REMOVED not satisfied — there is no
  "swap onset" to derive a fraction from

- fix: subprocess returncode -9 was labelled a "checked refusal"; 137 is
  the shell spelling of the same signal

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 20:18:47 +02:00
1fe808b7a4 docs(stories): add readiness, retire status: refine, sweep all 47 iterations
- `readiness: ready | refine` is a SECOND axis, orthogonal to status.
  `ready` = the brainstorm is complete and the decisions are LOCKED (a spec
  approved, or the forks explicitly confirmed). `refine` = open forks remain
  and it cannot be planned yet
- `status: refine` RETIRED because it carried both meanings at once, so a held
  iteration with an approved spec (language 18, 26) was indistinguishable from
  one nobody had thought about. status is now purely where the WORK is:
  done | in-progress | pending | hold — `pending` was already the board's own
  rendering word, so nothing new was invented
- all 47 iterations classified from EVIDENCE in their own text, not by guess:
  "the four forks are SETTLED" / "spec + plan approved" / "Approved spec:" for
  ready; "Forks the spec must settle" / "no spec exists yet" for refine. Every
  shipped iteration is ready by definition. 19 done, 5 in-progress, 15
  pending, 8 hold; 27 ready, 20 refine
- two iterations moved refine -> in-progress rather than -> pending: language
  31 and 34 are absorbed into 24 and work on them is literally happening, which
  the board already showed as 🔄 while their frontmatter said otherwise. That
  disagreement is now gone
- board legend, board-views' frontmatter contract, and two new Dataview
  queries updated — the useful one being `readiness: ready AND status:
  pending`, the startable set

WHAT THE NEW AXIS IMMEDIATELY SURFACED: of 15 pending iterations, exactly ONE
is startable — databasev2 4, io_uring group-commit, whose forks were confirmed
settled 2026-08-20. Everything else pending needs a brainstorm first. That was
invisible while one key carried both meanings, and it is now on the board.

Also caught by the sweep, unrelated to readiness but found by cross-checking
frontmatter against the board: SIX duplicate rows. Every iteration moved into
databasev2 was still listed in the LANGUAGE pending table under its retired id
(23, 32, 33, 20, 21, 27) as well as its new one. Stale copies removed. And two
databasev2 rows made claims the sweep contradicts — iteration 1 was billed
"startable today" while its forks are open, and 6 still called itself the
ceiling-raiser after 2 took that role.

Docs only. linkcheck 0 broken / 0 anchors.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 16:54:45 +02:00
f72b3310a8 refactor(db): wo_row_borrow/wo_row_release — one read path for both residencies
Task 5c step 1 of docs/superpowers/plans/2026-08-26-table-residency.md, as a
PURE REFACTOR: no storage change, no keys-table anywhere. Borrow is wo_row_ptr
plus a seam, every release is a no-op. Provable on its own before the storage
change it exists to enable.

DESIGN SETTLED BY READING THE STRUCTURES, and both answers make 5c smaller:

- the id hash needs NO new storage. `hvals` is already uint64 holding
  slot+1 with 0 = empty (table.h), so offset+1 fits the same field, and the
  interpretation is per-table because a table is wholly `all` or wholly
  `keys`. No parallel map
- secondary indexes need NO change. `db_ibucket.ids` stores row IDS, not slot
  indices, and table.c resolves them through the id hash. I had told the
  developer these pointed at slab slots — that was wrong, and it is why this
  is one shared accessor rather than 11 rewrites
- the real coupling is the unique shadow: idx_add_row and
  row_apply_field_slot both FETCH the conflicting row and compare columns.
  Both now borrow/release, so a keys-table's non-resident conflict will be
  found rather than silently skipped — a unique check that only examines
  resident rows is a correctness hole, not a limitation

The scratch lives on `db_table`, not on the stack and not per call. Per call
would allocate once per candidate inside a bucket loop, turning an O(1) probe
into an allocation storm; a stack buffer is unsafe because the loader bounds
field_cnt at 65535 (loader.c:189), so the worst case is ~512 KB. It is safe
per-table because the store is single-writer, and a `busy` flag is there to
catch a nested borrow rather than let it alias silently. Freed in
table_destroy.

Gates: all 18 runtime suites 0 fail under ASan+UBSan (test_table 856/0,
test_wal 3654/0), oop-e2e 119/0, residency 8/0, employee 8/0, db-actor 8/0.
And the pure-refactor proof the plan asked for: `db-bench --quick` 85/0, every
resident read/query/seed/write floor held — a refactor that moves a number is
not a refactor.

Remaining wo_row_ptr sites for 5d: 6 in table.c, 2 in db.c, 2 in wal.c.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 16:33:36 +02:00
51dd42f9db docs(databasev2): rewrite iteration 2 to match what was designed and built
Flagged by the developer: the iteration still described the pre-brainstorm
three-mode design behind a "superseded in part" banner while four tasks had
landed against it.

- iteration 2 rewritten around the shape as built: two keys
  (`durable: true|false`, `resident: all|keys`), not a `mode:` enum with
  `cold`. status refine -> in-progress
- added a task-by-task progress table with commit hashes, and split the
  acceptance criteria into MET (each with how it was verified, not just that
  it passed — e.g. the goldens-unchanged claim is `git diff` over golden/
  being empty after a WOC_BLESS run, since blessing rewrites all of them) and
  OUTSTANDING with the task that owns each
- kept the history rather than deleting it: the three-mode replacement, the
  "one real rewrite" that was fiction, and the opposite half that turned out
  genuinely deep. An iteration file is where that record belongs
- board row rewritten to agree; the track index's "the lever" section, its
  principle-7 paragraph and its sequence rationale all still taught the dead
  three-mode design

ITERATION 6 IS NOW LARGELY SUPERSEDED, and bannered as such rather than
quietly gutted. `resident: keys` is the ceiling-raiser and it lives in
iteration 2 (tasks 5c/5d). More than relocated: 6's premise — a user-space
resident working set with faulting and 5's eviction policy — was specifically
REJECTED by the spec in favour of the kernel page cache, since a pread against
a cached page is a memcpy. What may still be left is recorded honestly: revisit
only with a measurement showing the page cache insufficient. Its fork list
survives, especially "does the language surface the fault cost at the use
site", which is still open and still the largest question about what writeonce
is. The sequence rationale is amended too — it had 6 as the ceiling-raiser and
5 as a prerequisite on the critical path; neither holds.

Docs only. linkcheck 0 broken / 0 anchors.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 13:10:30 +02:00
18a56f24ec feat(db): wo_wal_read_row_at — materialise a row from a log offset
Task 5b of docs/superpowers/plans/2026-08-26-table-residency.md, whose Task 5
is now split 5a-5d (plan updated in this commit).

- the offset twin of wo_row_read (table.c:721): same out-gate contract —
  every value handed back is a FRESH VM allocation — but resolved from a file
  position instead of the id hash
- fits entirely in wal.c because everything it needs was already public:
  scan_record and dec_val are local, and wo_val_decode_vm / wo_db_val_free
  are exported at table.h:183-187. Two decode stages, since the record and
  the VM speak different dialects: dec_val -> engine slots -> VM copies, with
  the engine slots freed as scratch on every path
- ZERO storage change. Nothing calls it yet; that is the point of separating
  it from 5c, so the read path can be proven before the slabs are touched
- refuses rather than guessing, each case distinguishable: no intact record
  at the offset, a malformed header, a decode failure, trailing bytes, and a
  REMOVE tombstone. That last one matters most — handing a tombstone back as
  a row would read a deleted row as live

Tested by deep field comparison, not by "it parsed": 24 rows with a nil Text
every third row, each read back BY OFFSET and compared field by field,
including the string bytes. Plus all three refusal paths — tombstone, a
mid-record offset (the silent-wrong-row failure this guards), and past the
intact prefix.

The free-on-every-path claim is VERIFIED, not assumed: removing the free
produced 3 LeakSanitizer reports; restoring it returns to 0. Worth doing
because "ASan is clean" only means something if the harness would have
complained.

PLAN SPLIT: Task 5's storage half was written as if it were plumbing. Measured
instead: wo_row_ptr returns a db_row* into a slab with 11 call sites, table.c
has 37 slab references, db.c:105-181 scans slabs directly, enc_val serialises
FROM the slab, and no operation exists that drops a payload while keeping
index entries. Note this is the OPPOSITE half from the earlier retraction —
the record FORMAT needed nothing, the record STORAGE genuinely is deep. 5c
(id->offset map + drop-payload-keep-index) and 5d (rewiring the call sites,
scans, @unique/FK across the boundary) get their own write-ups.

Gates: test_wal 3654/0 (was 3428), all 18 runtime suites 0 fail under
ASan+UBSan, oop-e2e 119/0, residency 8/0, employee 8/0, db-actor 8/0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 10:27:40 +02:00
3290c7d117 feat(db): wo_wal_next_offset — exact record offsets, proven
Task 5a of docs/superpowers/plans/2026-08-26-table-residency.md. The read
path itself is NOT in this commit; see the note below.

- the offset problem is far smaller than the spec feared. `w->off` is the
  durable tail and `w->len` the staged bytes, and wo_wal_commit pwrites the
  whole batch AT off before advancing it — so a record staged now lands at
  exactly off+len, knowable at append time with no deferral to flush
- shipped as an inline accessor rather than new out-params on the three
  append functions, so the 156 existing WAL checks keep their signatures
- correct across both awkward cases, and both are now unit-pinned:
  a failed commit leaves off unadvanced so the record still lands where it
  was promised, and wo_wal_open positions off at the end of the INTACT
  prefix so offsets are always relative to validated data
- test_offset_capture asserts the recovered ID per record, not merely that a
  record parses — a wrong offset reads a NEIGHBOURING record, which passes
  its own CRC and returns the wrong row silently. 400 records across
  repeated buffer growth (stage() doubles from 4096) and uneven commit
  batches, so offsets are exercised mid-buffer and right after a flush

The failed-commit test caught MY OWN misunderstanding: I asserted
next_offset was unchanged after a failed commit. It is not, and should not
be — the record is still staged, so next_offset correctly points PAST it.
The invariant that matters is that the durable tail did not move, which is
what it now asserts.

Gates: test_wal 3428/0 (was 3426), all 18 runtime suites 0 fail under
ASan+UBSan, cli_smoke OK, oop-e2e 119/0, residency 8/0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 08:43:10 +02:00
b74e13d21e feat(db): durable:false skips the WAL append and replay
Task 4 of docs/superpowers/plans/2026-08-26-table-residency.md — the first
behavioural change in the iteration.

- db.c: one predicate, `table_is_durable`, gating the three EXISTING mutation
  sites. Kept as a function rather than an inlined condition so
  database/src/CODE-LOGIC.md's "nothing else may mutate storage" claim keeps
  holding — the choke points stayed three
- the ack contract is untouched for durable tables: RAM applied, record
  staged, one commit before the ack, and a failed commit still removes the row
- replay: a log holding records for a class the image now declares volatile is
  a real migration case, not corruption. apply_record returns -2 (distinct
  from -1), wo_wal_replay_ex reports the class id, and main.c names it and
  exits 2. `wo_wal_replay` stays as the NULL wrapper, so all 156 WAL unit
  checks are untouched
- measured, not asserted: 50 inserts wrote 1500 WAL bytes into a durable
  table and ZERO into a volatile one. The file's SIZE proves nothing (it is
  fallocate'd to 1 MiB up front), so the gate measures the non-zero prefix

BUG I INTRODUCED AND CAUGHT: the mismatch message first printed the class name
with %s, but wo_str.data is `char data[]` with NO NUL terminator (obj.h) — a
buffer over-read. Now %.*s with the explicit length, and re-verified under
ASan.

New gate `just residency` (8 checks), because everything above was otherwise
a one-off manual measurement: restart behaviour, the zero-byte write path, the
mismatch refusal (exit 2, names the class, NOT reported as corruption), and
both compile-time refusals. Its own first run failed two checks for a bug in
the script rather than the feature — `woc | grep` under `set -o pipefail`
returns woc's exit 1 even when grep matches, since woc exits 1 whenever it
reports diagnostics. Captures first now, with the reason noted inline.

Also new: corpus run/table-volatile-inprocess pins that a volatile table is a
FULL table in-process — same @unique enforcement, same index probe, same query
surface. Only survival differs, and that is unobservable from inside one
process.

Gates: woc-test 557/0, 18 runtime suites 0 fail, cli_smoke OK, oop-e2e 119/0
(was 118), residency 8/0, employee 8/0, db-actor 8/0, site 21/0, ASan clean on
the new replay path.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 08:23:49 +02:00
477f8d1b2b feat(wob): v7 — class descriptor carries durability and residency
Task 3 of docs/superpowers/plans/2026-08-26-table-residency.md.

- NO LAYOUT CHANGE. The plan said to add descriptor fields; the descriptor
  already had a `flags` u32 with only bit0 used, so both properties ride
  spare bits (WO_CLASSF_VOLATILE 0x02, WO_CLASSF_RESIDENT_KEYS 0x04). A v7
  class record is byte-identical in shape to a v6 one, which is a much
  smaller and safer change than the plan assumed
- both spelled as the NON-default, so a zero flags word means exactly what
  every pre-v7 image meant: durable, every row resident. A non-@table class
  has both clear by construction
- the loader refuses the meaningless pair (bit1+bit2) independently of woc,
  on the standing principle that what the loader accepts the interpreter
  trusts. Verified by FORGING the flags word in an otherwise valid image,
  since woc will not emit one: flags=6 gives "durable:false with
  resident:keys", flags=8 still gives "unknown flags"
- WOB_VERSION 6 -> 7. Kept because an OLDER runtime reading a v7 image would
  otherwise treat a volatile table as durable and quietly disagree with its
  own source. loader.c's check is exact-match, so a v6 image is refused
  rather than read with the bits clear — verified by patching a v7 header
  back down to 6

GAP FOUND AND CLOSED: woc ACCEPTED `durable: false, resident: keys`. Task 1's
steps covered duplicates and bad values but never the combination, and the
plan had only put that refusal in the loader. The spec wants both, so the
compiler now refuses it too (WO-E102, checked after the argument list is
complete since it is a property of the pair). A compile error is the one a
developer can act on.

VERSION DRIFT: the constant lives in FOUR places, not one. wob.h,
emit.ml:157, disasm.ml:186, and compiler/test/runner.ml:2405 — the last is a
deliberately independent reimplementation of the loader battery, and it
caught the drift as 14 failures rather than silently passing. Its flags mask
and the combination refusal are now in sync too, which is the point of it
being independent rather than shared.

Gates: woc-test 557/0, 18 runtime suites 0 fail, 18 ISO-flavour suites 0
fail, cli_smoke OK, oop-e2e 118/0, employee 8/0, db-actor 8/0, site 21/0.
Zero goldens moved (git diff over golden/ empty).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 23:22:26 +02:00
e120129f2c feat(woc): WO-E224 — refuse a durable ref into a volatile table
Task 2 of docs/superpowers/plans/2026-08-26-table-residency.md.

- the check lives in `check_field_types`, which already runs over the raw AST
  (so the diagnostic lands at the field's own position, once per declaration)
  in Pass 2 with `syms` fully built
- only the durable -> volatile direction is refused. volatile -> durable is
  legal: the referencing row is the one that disappears, so nothing is left
  holding a stale id
- the message names both classes and both escapes, because "this is wrong" is
  less useful than "make Session durable, or declare Order volatile too"
- sees through a `?` wrapper, so `?ref S` is caught too
- code picked as 24 by sweeping `<stage>_prefix ^ "NN"` — 01-23, 25, 26 and
  50 were taken, so 24 was a genuine hole. Grepping the literal WO-E224
  would have found nothing, which is how ten codes once went missing
- catalogued in the same commit, and the completeness sweep re-run: 53
  emitted, 54 catalogued (the extra is retired WO-W201), none missing

PLAN CORRECTION: the plan's second step said to apply the same check to
`backlink` fields. Dropped — a backlink is "NOT a stored column" (ast.ml:72),
so after a restart it resolves to an EMPTY COLLECTION, which is a legal state
indistinguishable from "nothing references me". There is no id to dangle.
Implementing it would have refused correct programs; a spurious diagnostic is
worse than a missing one. A run fixture now pins that the backlink shape stays
legal, so the check cannot silently grow over-broad later.

Gates: woc-test 557/0, oop-e2e 118/0 (was 116 — one compile-fail and one run
fixture added), employee 8/0, db-actor 8/0; employee, db-bench, db-actor,
porch, log-watcher and gc-cycle all still typecheck.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 23:09:40 +02:00
37f7267115 feat(woc): @table durable/resident arguments, defaults preserve behaviour
Task 1 of docs/superpowers/plans/2026-08-26-table-residency.md.

- ast.ml: `table_cfg` gains `durable : bool` (default true) and
  `resident : residency` (ResAll | ResKeys, default ResAll) — both defaulting
  to the pre-existing behaviour, which is what lets every @table written
  before this compile byte-identically
- parser.ml: `durable:` takes the existing KwTrue/KwFalse tokens; `resident:`
  takes the bare identifiers `all`/`keys`. Given-twice tracked by local seen
  flags rather than option fields, so "absent" and "explicitly the default"
  stay distinguishable without the AST carrying an option nobody reads
- five new WO-E102 causes, all catalogued in the same commit: durable twice,
  resident twice, an unknown resident value, `resident: index` (the
  pre-review spelling, with a message naming its replacement), and a retired
  design word (mode/store/ram/cold/tiered/paged/mmap/buffer) which gets a
  message stating the two real keys instead of a generic "unknown argument"
- dump.ml prints each property ONLY when it differs from its default.
  Printing unconditionally would have moved every pre-existing golden, which
  this iteration is not allowed to do
- new golden compiler/test/golden/ast/table-residency.wo covers all four
  shapes, including a table declaring `resident: all` explicitly and
  correctly dumping nothing for it
- verified, not assumed: `git diff --stat` over compiler/test/golden/ is
  EMPTY after a WOC_BLESS run, so all 30 pre-existing goldens are untouched.
  woc-test 557/0 (was 556), oop-e2e 116/0, employee 8/0; employee, db-bench,
  db-actor and porch all still typecheck
- docs: language-surface's @table row now matches what the parser accepts

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 23:03:36 +02:00
1ee7cce597 docs: retract the row-encoding rewrite — the flat record format already exists
- found during the pre-execution review of the plan, before any code
- the claim was wrong in both spec and plan: table.c's db_val_encode builds
  the IN-MEMORY slot; the FILE record is a separate encoding in wal.c and has
  been flat since iteration 9. enc_val inlines every kind recursively with no
  pointer anywhere; dec_val reads it back; a record is
  `WO_WAL_INSERT | class_id | id | <value per field>` in the
  len|crc|payload|mark frame; scan_record already preads and CRC-verifies a
  record at an arbitrary offset
- so the row encoding needs NO change, and Task 5 (a "self-contained,
  offset-based" rewrite billed as the iteration's substantive engineering) is
  DELETED, not reduced. 8 tasks -> 7, and the highest-risk task is gone
- the real difficulty is where the spec never looked: wo_wal_append_insert
  stages into a 1 MiB buffer, so a record's final offset is unknown until
  flush. Threading an accurate offset back through a buffered writer —
  correct across partial flush, failed commit and torn tail — is now Task 5's
  first two steps, with a unit test that straddles a buffer boundary and a
  case asserting no offset is published for a record that never reached disk
- dependent claims corrected: the mmap alternative's premise, the read-path
  bullet (now names scan_record/dec_val), and the self-review coverage table,
  which records the retraction rather than quietly dropping the row
- root cause worth noting: reading one layer and inferring another. Second
  time this iteration — the first was assuming WO_HEAP_MB bounded table
  storage when it bounds the VM arena
- no code written yet; linkcheck 0 broken / 0 anchors

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 23:00:18 +02:00
3586650baa docs: implementation plan for databasev2 2 — table residency
- 8 tasks, 71 steps, from the 2026-08-26 table-residency spec
- deliberately contains NO code: discarded.md records "raw code in plan
  documents" as rejected, and all six preceding plans have zero fences.
  Stated in the header so it does not read as an omission. Every step
  instead names the exact file and line region plus the required behaviour
- task order is by testable deliverable, not by layer:
  1 grammar + defaults (no existing golden may move)
  2 the cross-table check — a durable ref into a volatile table is refused
  3 .wob v7: descriptor carries both properties, loader refuses the
    meaningless combination so it never reaches the engine
  4 durable:false skips the WAL at the three existing choke points in db.c;
    replay refuses on mismatch rather than resurrecting rows
  5 self-contained offset-based records — the one real rewrite, since
    table.c returns a malloc'd address as the slot word today
  6 resident:keys read path: id->offset map, pread, sequential scan;
    @unique and FK-restrict across the boundary are the correctness core
  7 the two runtime refusals — durable with no WO_DATA (silent data loss
    today), and the resident-footprint budget
  8 measure, baseline, crash battery, docs, closeout
- self-review table maps every spec section to a task. Two gaps found and
  closed: the escape hatch for an intentionally ephemeral run (a refusal
  with no way forward is worse than the loss it replaces), and persisting
  the offset map in databasev2 3's snapshot
- linkcheck 0 broken / 0 anchors

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 22:51:49 +02:00
566559cf70 docs: name the residency value keys, not index (review change)
- `resident: all | keys` replaces `resident: all | index`. Two reasons beyond
  taste: it kills the collision with the `index:` argument
  (`@table(index: [customer], resident: index)` read badly), and it puts both
  values on ONE axis — each now answers "what row data stays resident",
  where `all`/`index` mixed a quantity with a structure name
- accurate as well as clearer: what stays resident is the id->offset map, the
  secondary indexes and the unique shadows — all key structures; row payloads
  are exactly what leaves. `resident: none` was rejected as overclaiming,
  since the indexes very much are resident
- checked for collisions: neither `all` nor `keys` is a keyword or a builtin
  (`key_at`/`val_at` exist, bare `keys` does not)
- the spec's wart note became a recorded decision; the rejected spelling is
  kept quoted so the rationale still reads
- fixes a bug I introduced in the 2026-08-26 track move: all six moved
  iterations carried a banner reading "Part of [Story — the database beyond
  RAM]" whose link pointed at the LANGUAGE arc — correct target, lying text,
  the exact failure mode the link audit warned about. Banners now point at
  the databasev2 story, and the original "Part of" line says plainly which
  track the iteration was authored in before the move
- linkcheck 0 broken / 0 anchors

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 22:47:24 +02:00
da30aa6527 docs: amend principle 7 — the log is authoritative, residency is declared
- driving case: a 120 GB order table on a 32 GB host. Not a tuning problem;
  no eviction policy fixes it. Developer accepted reconsidering the principle
- principle 7 rewritten: durability half UNCHANGED and unconditional
  (WAL-logged, fsync before ack, CRC-dropped torn tail); residency half
  demoted from law to per-table declaration. Old wording quoted in place so
  the amendment is legible, with the reason: a doctrine a real workload
  cannot satisfy gets ignored, and the failure it produced was an OOM kill
- spec: docs/superpowers/specs/2026-08-26-table-residency-design.md
  One log-structured engine — the WAL already holds every row, so keep an
  in-RAM id->offset map and pread rows back. No second engine, no user-space
  row cache (the kernel page cache is the hot copy, which is already this
  repo's stated position and why it avoids O_DIRECT)
- arithmetic that makes it work: 240M rows x 16 B of index = ~3.8 GB
  resident in 32 GB. Indexes stay resident, rows do not. Buys ~2 orders of
  magnitude, not infinity — stated plainly in the spec
- grammar: two optional keys, `durable: true|false` and `resident: all|index`,
  both defaulting to today's behaviour, so all 28 existing @table
  declarations compile untouched and no golden is reblessed
- rejected, with reasons recorded: mmap (rows are pointer-bearing —
  table.c returns (uintptr_t)t as the slot word), buffer pool (the Rust-era
  phase-12 design that died with that track), paged B-tree (stays rejected),
  a three-valued enum, automatic spill, disk-backed-by-default
- self-review caught the budget defaulting to "none" while promising the ERP
  developer a diagnostic instead of the OOM killer — contradiction fixed:
  the budget defaults to a fraction of host memory, and its value comes from
  databasev2 1's swap-onset measurement
- live docs that contradicted the amendment updated (subagent doctrine,
  its guide, discarded.md's two rows, iteration 04's read claim, 07, 38);
  dated specs/plans left as records. linkcheck 0 broken / 0 anchors

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 22:42:27 +02:00
746dc2b42b docs(databasev2): third track — the database beyond RAM, with per-table storage modes
- docs/stories/databasev2/, numbered from 1. Six PENDING database iterations
  moved from the language track and renumbered, keeping the old id in
  `was_language_iteration:` so a search for "iteration 32" still finds it:
  32 -> 3 WAL checkpoint, 23 -> 4 io_uring commit, 33 -> 7 single-file store,
  27 -> 8 query grammar, 20 -> 9 cross-program, 21 -> 10 keypair auth.
  Done work (9, 9b, 22) stays as v1 history; language 18 left whole
- the problem, read off the engine not guessed: rows are malloc'd slabs with
  addresses stable forever, NO eviction/spill/paging anywhere in database/src,
  the WAL never checkpoints so boot replays all history, and durability is one
  process-global WO_DATA so no table can say it matters more than another.
  An allocation failure IS a clean catchable WO_T_OOM — but swap thrash
  arrives first and carries no error signal at all, which is the real hazard
- four new iterations:
  1 measure the ceiling FIRST (curve not cliff; the three exits; kill -9 at
    exhaustion) — every later default should follow from a number
  2 `@table(mode: ram | durable | cold)` — the grammar ask. Small surface
    (Ast.table_cfg gains a key, the parser already rejects unknown args), big
    semantics: `durable` defaults so nothing changes silently, and the
    compiler refuses a durable row holding a `ref` into a ram table
  5 bounded tables + refuse/evict/back-pressure, shedding BEFORE the OS acts
  6 cold tiering — mostly forks, incl. whether the language surfaces the
    fault cost and whether @unique on cold is refused outright. A paged
    B-tree stays rejected: if tiering needs one, reject tiering
- 39 links repointed, link TEXT renumbered to track-local ids; arc gains one
  pointer row replacing the six moved; board + board-views cover three tracks
- linkcheck 0 broken / 0 anchors; no code blocks in any story

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 20:52:48 +02:00
01df75245f docs(porch): give the framework its own story track, iterations 1-8
- docs/stories/porch/ — a TRACK folder, not a status folder: status still
  lives only in frontmatter. Adds `track: porch` so a query over
  docs/stories/ can tell a porch 3 from a language 3
- 00-story.md carries the sequence, the dependency graph, and a table of
  what the track explicitly does NOT own (binding -> 29, cache -> 18,
  proxy -> 38, metrics -> 30, TLS/templates -> doctrine)
- eight iterations, each with phases, per-phase tasks, Given/When/Then
  criteria, out-of-scope and the forks a spec must settle:
  1 store-backed middleware (limiter + idempotency — needs nothing new,
    first on purpose so the store pattern is proven cheaply)
  2 randomness + cookies (phase A is language-track: a CSPRNG builtin;
    `Resp.headers` being a map cannot emit two Set-Cookie lines)
  3 sessions   4 CSRF   5 routing/response ergonomics (independent)
  6 streaming core (the seam 7 and 8 wait on; chunked-request refusal
    must survive)   7 SSE + compression   8 static + lifecycle hooks
- language iteration 39 -> status: hold, retitled superseded, with a row
  mapping each of its goals to the porch iteration that took it. Kept, not
  deleted: the Fiber study cites it and its randomness argument is what
  this track is built on
- board gains a porch section; board-views gains porch and both-track
  Dataview queries; porch README and the Fiber study §7 point at the track
- no code blocks in any story (plans carry concept and actions in words);
  linkcheck 0 broken / 0 anchors

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 20:17:33 +02:00
ffd791d05b refactor(porch): name the web framework porch, fix the wo.toml identifier claim
- docs/examples/writeonce-serve -> docs/examples/porch (git mv, history kept);
  `[deps]` key and import are now `porch` / `porch/http` / `porch/router`
- name history preserved on the library README, not rewritten into dated
  records: writeonce-framework -> writeonce-serve (08-25) -> porch (08-26).
  Stories, specs, plans and the audit reports keep the older name by the
  repo's own convention; only live docs and every path link were rewritten
- left alone deliberately: `internal/serve.wo`, `pub fn serve`, `serve_conn`,
  `app.serve(...)` — those are functions, not the module name
- web-app/wo.toml comment corrected: it claimed hyphens are not identifier
  characters and named a key this file never used. lexer.ml's `is_ident_cont`
  DOES accept `-` (an internal dash is part of the identifier, which is why
  binary minus needs spaces), so a hyphenated key would be legal too
- site now teaches the name: package card, the two-deps chapter and the
  handlers-are-classes chapter say `porch`; site-accept asserted the old
  /packages/serve route and caught the rename, as a gate should
- gates: web-app 46/0, site 21/0, deps-accept 8/0, oop-e2e 116/0,
  linkcheck 0 broken / 0 anchors; porch typechecks entry-less as kind=library

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 19:36:34 +02:00
c0b0dbb846 docs: audit all markdown against the code, fix findings, flatten status folders
- README: shipped concurrency/HTTP/WebSockets sat in the roadmap as "not yet
  available"; "no package manager" contradicted [deps]; the deps example
  would not have compiled (the key IS the module name)
- runtime/README: leads with wovm, wo-rt.c demoted to a historical section;
  dropped 2 nonexistent recipes, crates/rt, @gc refcounting, 13 suites -> 18
- employee + log-watcher READMEs claimed "does not compile"; both are gates
- error catalog: +10 emitted codes incl WO-E250, the only diagnostic the
  shipped query surface raises; recorded why the sweep rotted
- language-surface: group-by parses, then the typechecker refuses it
- 00-code-review + 00-link-audit re-run; history kept, not rewritten
- 48 dead Rust-era exploration links de-linked rather than re-pointed (their
  prose names the retired plan by number); successor map -> discarded.md
- 08-project-structure: compiler/plan/ never existed; corpus has 9 dirs, 5 empty
- releasing.md: dropped a --draft step the workflow never had
- new docs/00-doc-audit.md: findings + disposition, incl one row where the
  audit was wrong and the doc it accused was right
- status folders removed: 34 stories flat, status only in frontmatter; 252
  links recomputed from resolved paths; board/board-views/structure retaught
- story 24 -> in-progress, since frontmatter is now the only truth
- new iteration 38: fs mutation verbs + net.connect, the two capability
  families no iteration owned
- new iteration 39: gofiber/fiber v3.5.0 parity study. The ledger called
  CSRF/sessions unblocked by iteration 34's HMAC, but the runtime has no
  source of randomness at all
- linkcheck skips .dev/.superpowers: 0 broken paths, 0 bad anchors

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 19:20:22 +02:00
b3f8ed9985 feat(site): source links to the repo, deployment layout documented
- "View source" and the nav GitHub link pointed at the user profile
  (github.com/shoneyj); both now point at github.com/shoneyJ/writeonce
- README: what actually has to reach the host — the self-contained
  binary plus dist/ (served by /dl) and data/ (WO_DATA) — and the four
  environment variables, with SITE_HOST left UNSET behind a proxy so
  the process binds loopback
- says plainly that dist/ must hold the PUBLISHED release assets: the
  build is not byte-reproducible, so a local tarball would not match
  the published .sha256 and the mirror would disagree with GitHub

Prepared and verified locally: docs/examples/site/dist/ holds the real
v0.1.0 assets (digest matches the release) and target/site serves them
byte-identically. Both directories are gitignored.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 08:45:15 +02:00
8f3824409b fix(site): glibc floor is 2.35, not 2.38 — read off the release
The published v0.1.0 binaries need less than the page claimed:
woc imports up to GLIBC_2.35, wovm up to GLIBC_2.34. The page said
2.38, which was measured on a dev workstation (glibc 2.39) before CI
existed — and understating support turns working platforms away.

- supported systems: glibc 2.35+, covering Ubuntu 22.04+, Debian 12+,
  Fedora 36+
- RHEL 9 (2.34) runs wovm but not woc: build elsewhere, copy the
  self-contained binary
- say plainly that the floor is set by the machine that BUILT the
  release, which is why CI pins ubuntu-22.04
- site-accept follows the new string

This is the pinned-runner decision paying off: 2.38 -> 2.35.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 07:42:59 +02:00
3f9ce2bf32 fix(ci): do not pin dune — use whatever setup-ocaml provides
Some checks failed
release / release (push) Has been cancelled
The pinned `opam install dune.3.14.0` failed. setup-ocaml installs a
dune of its own for caching, so requesting an exact older version is a
downgrade the solver refuses — which also means run 1's
`dune: command not found` was only ever a PATH problem, fixed by
`opam exec --`.

- probe with `opam exec -- dune --version`, install only if absent
- echo the resolved version so the log says what built the release
- any dune >= 3.14 satisfies `(lang dune 3.14)`

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 07:39:51 +02:00
4f89d42948 fix(ci): install dune and build inside the opam env
First run failed with `dune: command not found`.

- ocaml/setup-ocaml provides a compiler and opam, not dune. dune is an
  ordinary opam package and this project has no .opam file for the
  action to infer one from, so nothing pulled it in
- add `opam install -y dune.3.14.0`, pinned to the version
  compiler/dune-project targets (`(lang dune 3.14)`)
- run the build as `opam exec -- ./scripts/mkdist.sh`: the script calls
  dune internally, so it needs the opam environment on PATH

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 07:17:59 +02:00
72cd510676 ci: workflow_dispatch is a real dry run
- manual runs skip the tag guard (there is no tag on a dispatch, so
  GITHUB_REF_NAME is the branch and the guard always failed) and skip
  publishing
- a dispatch now builds, verifies the digest, smoke-tests the
  extracted toolchain and reports the glibc floor, then stops
- replaces the throwaway-tag rehearsal in the checklist: no tag to
  delete, no draft release to clean up

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 06:01:26 +02:00
c47d91c153 docs(releasing): what the hosted runner costs
- public repos: standard runners free, unlimited minutes; only larger
  (4-core+) runners bill there and this workflow does not use one
- private: included minutes per plan, then per-minute; Linux x1 vs
  Windows x2 / macOS x10; each job rounds up to the next minute
- sized from a measurement: cold mkdist.sh is 3.4s on 20 cores, so
  under a minute on a 2-core runner — setup-ocaml dominates, ~3-10
  min per release, and it only runs on a tag
- release assets do not count against Actions artifact storage
- flags that rates drift; check the billing page

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 05:23:33 +02:00
3dcadefbfd docs(releasing): why the release job stays on a hosted runner
- self-hosted works technically: outbound HTTPS only, no inbound
  ports, honours HTTPS_PROXY/NO_PROXY — a box behind a proxy is fine
- but it defeats the pinned-runner decision: the build host sets the
  glibc floor, so a workstation runner (2.39 here) puts it back to
  2.38+ and drops Ubuntu 22.04 / Debian 12 / RHEL 9
- and a workstation-built release is unattested
- records what self-hosting accepts: jobs run as the starting user,
  with that user's ~/.ssh, credentials and network reach — including
  hosts named in ~/.ssh/config; worst on public repos, where a
  stranger's PR runs code on the runner
- if unavoidable: dedicated VM, unprivileged user, --ephemeral,
  segmented network, treat .credentials as a secret

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 05:16:43 +02:00
a705622f4a docs(releasing): first-time pipeline readiness checklist
- states plainly what does NOT trigger it: builds run on a
  GitHub-hosted runner, not locally, and only on a `v*` tag push —
  pushing master releases nothing
- 14 numbered steps: get the workflow onto GitHub, enable Actions,
  allow ocaml/setup-ocaml, the 403/workflow-permissions fallback,
  a --draft rehearsal on a throwaway tag, cleanup, then the real tag
- calls out that the rehearsal tag is EXPECTED to fail the tag/VERSION
  guard, and how to rehearse the full job instead
- step 8/9: read the runner's glibc floor and reconcile
  install/view.wo with it — the runner, not the dev machine, decides
  who can run the release
- lists the three likely first-run failures

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 05:04:39 +02:00
463f897e5f ci: release workflow — no gh auth login, tag-triggered
- .github/workflows/release.yml: builds, verifies and publishes on a
  `v*` tag. `permissions: contents: write` on the injected
  GITHUB_TOKEN replaces `gh auth login`; no PAT, nothing to rotate
- runs-on ubuntu-22.04 DELIBERATELY: the build host's glibc caps which
  symbol versions the binaries import, and that cap is the floor every
  user needs. 22.04 (2.35) includes Ubuntu 22.04 / Debian 12 / RHEL 9;
  24.04 (2.39) would exclude them
- guards that fail instead of publishing: tag vs VERSION, produced
  asset name vs the filename /install links, sha256, and a smoke test
  that builds a hello project with the binaries INSIDE the tarball
- reports the shipped glibc floor so the claim on /install is checkable
  from a build log
- releasing.md: pipeline route up front, manual route kept; GH_TOKEN
  recipe for non-GitHub CI

Not run — this repo has no CI history and Actions cannot execute
locally. Every guard's shell was dry-run here against the real dist.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 04:58:40 +02:00
844bcc1067 docs(releasing): full gh auth login section
- gh's credential is separate from git's: SSH keys let you push but
  not call the API, so a machine that pushes can still fail to release
- the five interactive prompts and what to answer, with SSH as the
  protocol to match this repo's existing remote
- headless path: PAT scopes (classic repo/read:org/gist, fine-grained
  Contents: read and write), --with-token from a 600 file, GH_TOKEN
  for automation
- verify with `gh repo view shoneyJ/writeonce` — proves the token
  reaches THIS repo, not just that it is valid

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 04:54:57 +02:00
b11f964eec docs: release runbook — build, verify, publish on GitHub
- docs/guides/releasing.md: the steps from `just dist` to a working
  download button
- pins the constraint that matters: the asset filename and tag must
  match the URL /install links, or the button 404s
- includes verifying the tarball with the binaries INSIDE it, tagging
  the built commit, `gh release create` with both files, the web-UI
  path, and a curl check of the exact link the site uses
- notes dist/ is gitignored, the shoneyJ/shoneyj path-case difference,
  and what a version bump must touch in install/view.wo

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 04:51:14 +02:00
ef37b8ffa2 feat(serve+view): file serving, downloads, supported systems; rename
- rename the two libraries: writeonce-framework -> writeonce-serve
  (`use serve`), wo-html -> writeonce-view (`use view`). Names say the
  ROLE now; every sample, script, gate and live doc follows
- stories/specs/plans keep the old names: they are dated records, and
  both library READMEs carry a "renamed 2026-08-25" note
- serve/http/files.wo: StaticFiles { dir, max_bytes } — traversal
  refused not normalised, extension content types, attachment
  disposition for archives. Lifted out of the shop, which had said in
  a comment that it belonged in the framework
- shop drops its private copy and mounts the framework's
- site: /dl/*path over $WO_DIST (default ./dist), 16 MiB ceiling
- /install gains supported systems — Linux x86-64, glibc >= 2.38,
  not musl — read off `file` and the binaries' GLIBC_ symbol
  versions, not off a wish list; plus GitHub release as primary,
  /dl as mirror, and the sha256 verify step
- site-accept: 17 -> 21 checks (supported systems, gzip download with
  a binary-safe probe, checksum, /dl traversal 404)

Verified on 192.168.0.165: the real 960,820-byte tarball downloads
as application/gzip and its sha256 matches the published digest.

Gates: oop-accept MET, site 21/0, web-app 46/0, fibers 10/0,
db-actor 8/0; shop rebuilt and its /assets served by the framework.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 04:41:00 +02:00
3afdafa5c4 feat(site): logo, favicon, install guide, package catalogue
- layout/logo.wo: the mark as inline SVG — dark tile, two-stroke "W"
  (white then accent blue). One source: nav brand + /favicon.svg
- favicon/controller.wo: GET /favicon.svg, image/svg+xml, day cache
- install/: GET /install — toolchain tarball, PATH, verify, first
  project, build/run, adding a dep. Copy from the real install README
- packages/: GET /packages + /packages/:name — catalogue with the
  [deps] line, what each library gives you, and a usage snippet.
  Index cards are child components (multi Component)
- wo-html: page_head(title, head, body) and a `head` slot on Layout —
  a favicon link or meta tag had nowhere else to go; page() passes ""
- header: Install/Tutorial/Packages/GitHub, brand shows the mark
- main.wo: SITE_HOST picks the interface (loopback default), bound
  address printed at startup
- site-accept: 11 -> 17 checks (install, packages x2, 404, favicon,
  inline logo)

Verified on 192.168.0.165:8080 — every route, favicon bytes, and the
mark rasterised at 256px and 32px.

Gates: oop-accept MET, site 17/0, web-app 46/0, fibers 10/0,
db-actor 8/0; shop rebuilt clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 04:05:00 +02:00
23550e7021 feat(lang+wo-html): raw text literals, component layer, MVC samples
- lexer: backtick raw text literal — content verbatim, no escape
  processing, common source margin removed at lex time; `${ }` raw and
  `{{ }}` auto-escaping holes
- `{{ e }}` desugars to `esc(${e})` in parser.ml — a Call on the `esc`
  in scope, so types/owner/emit/.wob/VM are untouched
- WO-E004 unterminated raw literal; WO-E005 newline inside "..." —
  closes a hole where a missing quote silently ate the rest of the file
- wo-html: `Component` interface, `render_all`, `Layout`, README
- framework: `ok_html` joins ok_text/ok_json in http/types.wo
- site + shop restructured to one-feature-one-module MVC (view +
  controller per directory, model at the root, bootstrap-only main)
- removed the filler `pad: Int` convention — verified unnecessary for
  plain classes, interface dispatch, containers and actors
- corrected recorded claims: gap #1 blocks neither the build nor the
  layout; a class crosses module lines, only a free fn is scoped
- docs/guides/language-surface.md — the full grammar inventory
- story 37 landed and moved to done/

Gates: oop-accept MET, oop-e2e 116/0, woc-test 556/0, site 11/0,
web-app 46/0, fibers 10/0, db-actor 8/0

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 03:58:41 +02:00
a4743edcc6 fix(shop): views markup-first within today's grammar
- render() bodies: one HTML line per 'h = h ..' statement, single-
  quoted attributes, ${} holes, esc() on data — el() chains gone
- discovered + recorded gap #3: no multi-line expressions or literals
  (leading/trailing .. and paren grouping all reject at NEWLINE) —
  exactly the tax story 37's raw literal deletes
- 37-target comment blocks dropped (bodies now self-explanatory; the
  README states the delta); rebuilt + full buy-flow re-smoked (178.0
  total, stock 12->10, 409, traversal 404)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 00:23:39 +02:00
bbf5fb66d3 feat(shop): 37 target = in-class raw template literals (developer form)
- separate view.html files dropped; every render() now carries its
  '-- 37 target:' literal above the hand-lowered body — the pair is
  the DX referendum in one file
- story 37 re-pointed: raw multi-line literal + {{ }} auto-escaped
  typed holes + {!! !!} raw slots; structural control stays if/for;
  w:if/w:for and .html files demoted to later; forks revised
- rebuild verified on untouched toolchain

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 00:15:38 +02:00
5a2e6402c8 feat(shop): story-37 target templates beside the hand-lowering
- view.html per feature + layout app/header/footer.html: the markup-
  first form woc will compile ({{}} auto-escaped, w:if/w:for,
  {!! !!} slots, w:component sections); inert today, verified not to
  disturb the build
- README: pair is the DX referendum — .html is the target feel,
  view.wo is today's cost; doctrine line reworded (templates compile
  or don't exist)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 00:01:11 +02:00
89df517613 feat(shop): the program template — MVC on disk; story 37 redirected
- docs/examples/shop: types.wo model, per-feature view modules
  (render() classes), root controller files, layout shell/header/
  footer, static assets controller + real style.css, README with
  Angular file map + run + DX referendum notes
- buy flow proven by hand: stock check/decrement, 409s, traversal
  404, css typed, WAL-durable; builds on the UNPATCHED toolchain
- two language gaps recorded, not fixed (per directive): pub+@table
  cannot combine (controllers forced into root module); @view
  projection classes absent
- story 37 REDIRECTED per Vue-SFC review: view.html compiled by woc
  ({{}} auto-escaped, w:if/w:for, typed against view class, no
  runtime engine); render()-as-concatenation failed the referendum;
  forks revised; shop named the acceptance consumer

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-23 23:57:44 +02:00
67cd039533 docs: fix stale story-35 links (file moved to done/ at landing)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-23 19:36:18 +02:00
8f96584682 docs: story 37 — wo-html components (MVC view layer, Angular format studied)
- Component interface (render->Text), layouts/slots, site migrates
  as acceptance; framework stays micro (view layer = library)
- Angular map recorded: inputs/templates/ngFor/projection translate,
  DI/bindings/client-side rejected by doctrine (no closures, no JS)
- four forks for the spec; unscheduled, off-chain; table + board rows

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-23 19:35:41 +02:00
b19042bca0 feat(site): go.dev-style homepage + shared nav/footer chrome
- wo-html grows generic nav_bar/card/btn_link + the utility classes
  they need (sticky nav, footer, 2-col grid, hero sizes); library
  stays content-free
- home: hero (tagline + Get started/View source CTAs), real actor+
  table code showcase, four why-cards, chapters strip (gate's
  'Learn writeonce' anchor kept); every page shares nav + footer
- site gate 11/0 unchanged; loopback bind untouched

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-23 19:11:21 +02:00
735fd270db feat: chat sample + gate (T8/T9, IN PROGRESS) + stop-drain semantics
- docs/examples/chat: registry (call consumer) / room / reader+writer
  actor pair per connection over ws_accept + wsframe; presence,
  broadcast, cross-room isolation, mailbox-full = drop-from-room;
  reader tail sends hardened (a full writer no longer orphans the fd)
- RUNTIME SEMANTICS CHANGE (the drain): SIGTERM no longer kills parked
  fibers from outside — the plane WAKES them and each wait RESOLVES
  (deadline'd waits answer their timeout result, sleeps return early,
  plain waits answer WO_SYS_STOPPED and unwind THAT fiber alone; main's
  STOPPED still ends the program). Workers keep adopting their inboxes
  after stop until eng_shutdown. This is what lets a program drain:
  chat's close frames now reach clients (byte-verified 0x88), then
  main returns and the reap runs
- also: SIGPIPE ignored process-wide (EPIPE trap instead of death);
  two-phase engine teardown (real drops while arenas+routing live,
  settle passes for routed frees) — fixes the registry-map leak and
  the drain UAF ASan found
- gate scripts/chat-accept.sh + just chat: handshake independently
  verified, functional matrix on BOTH backends, 1k-hot-room soak
  (1000/1000 in ~35ms), drain close-frames, SIGTERM exit 0, ASan leg
  clean. OPEN: soak-fds check (18 fds settle slower than the window)
  + full battery after the semantics change — NOT yet run
- committed for manual testing at the user's request

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-23 09:53:21 +02:00