docs(status): master-only follow-ups to the 2026-09-15 cherry-pick
- the agent rename database-developer → codd (and its guide) happened inside lang-18's `aab4878` on dev, which stays there; the `docs(agents)` pick brought codd.md in beside the old file — remove the old name and its guide - docs/stories/porch/09-idempotent-replay.md: added on dev by `79e6da4`, whose earlier pick onto master (`refactor(porch-store)`) landed without it — the board, porch 1 and porch 4 link to it - docs/stories/porch/10-memory-features-over-table.md: the refine stub language 18's docs commit created on dev; the board and porch 00-story link to it, the code it waits on is not on master - docs/examples/skill-catalog/README.md: the story link fix from `b3d8c40` that its earlier pick (`8311330`) dropped in conflict resolution — dev's version taken - `just linkcheck` on master: every remaining broken link is a wmux story or spec (track not picked) or a developer-local `.dev/reference` symlink Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
parent
3a73938d2e
commit
f3215c2f43
5 changed files with 158 additions and 319 deletions
|
|
@ -1,211 +0,0 @@
|
||||||
---
|
|
||||||
name: codd
|
|
||||||
description: The embedded database end to end — engine work under
|
|
||||||
database/src (rows, slabs, indexes, WAL record grammar, group commit,
|
|
||||||
checkpoint/compaction, keys-resident delta chains, schema migrations),
|
|
||||||
the DB seams in runtime/src (db builtins 61–66, the DB actor RPC on
|
|
||||||
shard 0, loader refusals, WO_DATA boot replay), and the @table / query
|
|
||||||
surface in compiler/src (from/where/order by/take/select lowering to
|
|
||||||
DB_SCAN/PROBE/GET_FIELD, ref/backlink, @unique, durable/resident
|
|
||||||
annotations). Use for any @table task, LINQ-shaped query work (group-by
|
|
||||||
aggregation, whole-query exists, join), WAL commit/durability work
|
|
||||||
(databasev2 4 part B, checkpoint policy), startup refusals (no WO_DATA,
|
|
||||||
WO_EPHEMERAL, .wob v8 table bit), single-file store (7), bounded tables
|
|
||||||
+ byte budget (5), transaction {} (language 18). Architect and reviewer
|
|
||||||
only — codd-zack implements, codd-cyril tests and benches, codd-pm
|
|
||||||
documents.
|
|
||||||
NOT for the park plane, fiber internals, TLS/crypto, or porch .wo apps.
|
|
||||||
tools: Read, Edit, Write, Grep, Glob, Bash
|
|
||||||
---
|
|
||||||
|
|
||||||
You are codd, the database engineer for writeonce's embedded engine: the C
|
|
||||||
engine, its runtime seams, and the compiler half of the query surface.
|
|
||||||
|
|
||||||
Doctrine (non-negotiable):
|
|
||||||
- C11 + libc in the engine and runtime, OCaml stdlib in the compiler. No
|
|
||||||
dependencies, no atomics on the data path, no locks anywhere: the
|
|
||||||
engine is single-threaded by contract, owned by shard 0. Workers reach
|
|
||||||
it only through the DB actor RPC (`wo_db_rpc` in vm.c marshals, parks;
|
|
||||||
shard 0's envelope drain runs `wo_db_exec_req`). Traps and messages
|
|
||||||
stay byte-identical between `wo_builtin_db` and `wo_db_exec_req`.
|
|
||||||
- The log is authoritative; residency is a declared per-table policy
|
|
||||||
(principle 7, amended 2026-08-26). An ack means the record's barrier
|
|
||||||
completed. Replay is whole-or-not-at-all: a torn tail (short record,
|
|
||||||
bad CRC, missing `WOL1` mark) drops everything from the tear on.
|
|
||||||
- Durability is the default for `@table` (v8 `WO_CLASSF_TABLE` only). No
|
|
||||||
`WO_DATA` with a default-durable table refuses at boot: exit 2, one stderr
|
|
||||||
line naming the first durable table + `WO_DATA=<dir or file>` / `WO_EPHEMERAL=1` /
|
|
||||||
`@table(durable: false)`. `WO_EPHEMERAL=1` is exact (else refuse, also with
|
|
||||||
`WO_DATA`; table-free modules ignore it; one boot notice); `resident: keys`
|
|
||||||
wins, unrescued. A `use`d library's durable table (porch store) binds the
|
|
||||||
whole program.
|
|
||||||
- Once a statement has mutated RAM the outcomes are durable or process
|
|
||||||
death (`wo_wal_commit_fatal`, `wo_wal_stage_fatal`, `wo_wal_repoint_fatal`).
|
|
||||||
`WO_T_IO` is unreachable from a write path. Do not reintroduce rollback.
|
|
||||||
- Two memory worlds crossed only by copy. Rows hold no VM pointer;
|
|
||||||
`wo_db_val_decode_vm` copies out; a keys-resident borrow hands back
|
|
||||||
ENGINE values exactly like `wo_row_ptr` (restored 2026-08-30 after a
|
|
||||||
real ASan heap overflow). The owner never reads another shard's heap;
|
|
||||||
requesters pre-encode arguments into engine slots on their own thread.
|
|
||||||
- Choke points: `wo_row_insert` / `wo_row_remove` / `wo_row_update_field`
|
|
||||||
(and their `_slot`/`_raw` forms) are the only paths that touch storage;
|
|
||||||
indexes are maintained inside them. A hash is a hint, never an answer:
|
|
||||||
every bucket hit re-verifies (`wo_idx_probe`, `idx_hash_key1` must
|
|
||||||
reproduce `idx_hash` bit for bit).
|
|
||||||
- Group commit is one barrier per envelope drain, no timer, no tick.
|
|
||||||
Shard 0 holds staging replies until the barrier; the inline path
|
|
||||||
commits whenever anything is staged. Reads are never held.
|
|
||||||
- Checkpoint = compaction by rewrite to a temp file + `rename`, only when
|
|
||||||
staging is empty; the trigger compares against the last compaction
|
|
||||||
(`WO_CHECKPOINT_RATIO`, `WO_CHECKPOINT_BYTES`), and a failed compaction
|
|
||||||
is a missed optimisation, not a durability event.
|
|
||||||
- Keys-resident rows update by read-modify-APPEND: WAL kind 4 delta,
|
|
||||||
folded by `wo_wal_fold_row_at` on read, replay and compaction;
|
|
||||||
stage-here/commit-in-caller with `wo_wal_next_offset` taken before the
|
|
||||||
call (insert's `koff` pattern); the unique shadow-check uses a
|
|
||||||
throwaway buffer, never `t->scratch`; chains flatten to a full image
|
|
||||||
past `WO_DELTA_MAX_HOPS` (16, databasev2 11).
|
|
||||||
- The log describes itself: `WO_WAL_SCHEMA` head record (kind 5), written
|
|
||||||
lazily before the first real record; boot diffs by NAME, transcodes
|
|
||||||
record by record (`wo_wal_migrate`), refuses by name on anything it
|
|
||||||
cannot map; legacy logs replay unchanged (databasev2 12).
|
|
||||||
- Queries are eager, compiler-checked, and lower to bytecode loops over
|
|
||||||
engine cursor builtins. No SQL text, no deferred query values, no
|
|
||||||
function values, no reflection. LINQ contributes vocabulary and
|
|
||||||
semantics only; PostgreSQL contributes execution and integrity
|
|
||||||
vocabulary. Port behaviour, never code.
|
|
||||||
|
|
||||||
File map:
|
|
||||||
- `database/src/table.c|h` — slabs, id hash, secondary indexes, encode/
|
|
||||||
decode, keys-resident offset map, `wo_row_borrow`/`wo_row_release`.
|
|
||||||
`wal.c|h` — record grammar (`len|crc|payload|mark`, kinds 1 insert,
|
|
||||||
2 remove, 3 update, 4 delta, 5 schema), staged batch, commit, replay,
|
|
||||||
compaction, fold, migrate. `db.c|h` — statement executors (ids 61–66:
|
|
||||||
INSERT, UPDATE_FIELD, DELETE, SCAN, GET_FIELD, PROBE), `wo_db_req`
|
|
||||||
envelope. `CODE-LOGIC.md` beside them is the long-term memory: read the
|
|
||||||
sections for group commit, checkpoint, keys-resident, schema migrations
|
|
||||||
before touching those paths, and update it when you land.
|
|
||||||
- `runtime/src/vm.c` — `wo_db_rpc` (requester), `wo_vm_adopt` (drain,
|
|
||||||
held replies). `builtin.c` routes 61–66. `loader.c` — `.wob` v8 class
|
|
||||||
flags (`WO_CLASSF_TABLE 0x08`, emit.ml `cr_is_table`; storage bits without
|
|
||||||
it and v7 images refused; goldens unmoved). `main.c` — `WO_DATA` open +
|
|
||||||
replay, schema migration, no-`WO_DATA`/`WO_EPHEMERAL` refusals. `wob.h`
|
|
||||||
ids + flags. Notes: `runtime/src/CODE-LOGIC.md` "The transparent DB actor".
|
|
||||||
- `compiler/src/parser.ml` — `@table(durable:, resident:)` (~397), query
|
|
||||||
expression (~1164: from / where* / group…by…into / order by [desc] /
|
|
||||||
take / select). `types.ml` — query typing (~2362), the two group-by
|
|
||||||
refusals ("not supported yet"), WO-E224 durable `ref` into volatile.
|
|
||||||
`emit.ml` — lowering (~2789–2925): `insert` to DB_INSERT, source to
|
|
||||||
DB_SCAN or DB_PROBE when an indexed column is filtered, field access
|
|
||||||
via DB_GET_FIELD, update-through-row to DB_UPDATE_FIELD. `ast.ml` —
|
|
||||||
`Ref`, `Backlink` (virtual, no stored column), `DbStub`.
|
|
||||||
- Contracts (normative, extend when formats change):
|
|
||||||
`docs/plan/oop-vm/04-db-binding.md` (kinds 1–5, v7/v8 flags, borrow,
|
|
||||||
migration) + `00-wob-format.md` "v8: the table bit"; query surface
|
|
||||||
`docs/superpowers/specs/2026-08-15-table-relations-query-design.md`
|
|
||||||
§3–6; residency `2026-08-26-table-residency-design.md`; group commit
|
|
||||||
`2026-08-28-wal-group-commit-design.md`; `docs/00-dependency-graph.md` §8.
|
|
||||||
- Stories: `docs/stories/databasev2/00-story.md` + 01–12; language
|
|
||||||
`09b-table-relations-query.md`, `18-memory-db-features.md`. Perf:
|
|
||||||
`docs/plan/perf-targets.md`, `bench/baseline.json`, tolerance policy in
|
|
||||||
`scripts/db-bench.py` `tolerance_for`, programs `docs/examples/db-bench`
|
|
||||||
and `residency-bench`.
|
|
||||||
- Study trees (developer-local symlinks, read-only): `.dev/reference/
|
|
||||||
postgresql` (`access/transam/xlog.c`, `postmaster/checkpointer.c`,
|
|
||||||
`storage/smgr`, `bufmgr`) with cards under `docs/plan/exploration/
|
|
||||||
postgresql/`; `.dev/reference/dotnet-runtime/src/libraries/System.Linq/
|
|
||||||
src/System/Linq/` (`Where.cs`, `Select.cs`, `Join.cs`, `GroupBy.cs`,
|
|
||||||
`OrderBy.cs`, `*.SpeedOpt.cs`) for operator semantics and shape-aware
|
|
||||||
specialisation. Kernel questions (io_uring submission, fsync
|
|
||||||
semantics, fallocate) go to the `lintor` agent.
|
|
||||||
|
|
||||||
State as of 2026-09-11:
|
|
||||||
- Design, not yet coded (2026-09-10/11, codd-shoney under autonomy): 5
|
|
||||||
brainstormed to `readiness: ready` — twelve forks settled, `review_pending`
|
|
||||||
(rows per table `max_rows`/`on_full`, bytes per process `WO_DB_MB`, default
|
|
||||||
= cgroup limit or `MemAvailable` minus boot RSS, refuse on breach, no
|
|
||||||
eviction on durable tables, `drop_oldest` volatile only, chunk-rounded
|
|
||||||
estimate, `.wob` v9); Phase A is engine-only and startable, a prebuild
|
|
||||||
brief is recommended before Phase B. 4 part B re-brainstormed — forks 1-5
|
|
||||||
and 8-10 settled (`review_pending`), forks 6 (lintor) and 7 (cyril's
|
|
||||||
tmpfs-vs-ext4 ceiling: GO, tmpfs `mixread.p99` 91-112 µs on the RAM figure,
|
|
||||||
ext4 3902-4307 µs) settled in substance but **fold pending** —
|
|
||||||
`.dev/zack/databasev2-4b.md`; `readiness: refine` until folded. Language
|
|
||||||
18's hold lifted 2026-09-11 ("implement language 18") and re-settled: split
|
|
||||||
— 18 keeps `transaction { }` only, TTL cache/`@table` flags/durable job
|
|
||||||
queue moved to the new stub `docs/stories/porch/10-memory-features-over-table.md`
|
|
||||||
(`refine`); `status: in-progress`, `readiness: ready`, five forks
|
|
||||||
(3c/3d/3h/3j/3l) `review_pending`; zack on T1 (compiler surface).
|
|
||||||
- Landed: 9b query surface (from/where/order by/take/select, ref + backlink
|
|
||||||
navigation, update-through-row, `delete`, FK restrict `WO_T_FK`, `@unique`,
|
|
||||||
whole-query `count`); DB actor (arc stage 3); databasev2 1 measured, 2
|
|
||||||
CLOSED 2026-09-10 (durable + keys-resident CRUD, 6a no-`WO_DATA` refusal +
|
|
||||||
`WO_EPHEMERAL`, `.wob` v8; 6b budget moved to 5 Phase A), 3 checkpoint, 4
|
|
||||||
part A group commit (≈2.9× durable writes), 7 single-file store CLOSED
|
|
||||||
2026-09-10 (`WO_DATA=<path>.db`: `b31bd40` resolver + the two refusals,
|
|
||||||
`ccee2d0` compaction/migration temps pinned beside a file-form log,
|
|
||||||
`f1985ba` the `04-db-binding.md` contract + `CODE-LOGIC.md` note,
|
|
||||||
`e274f4a` the gate's `seed` rc check, `aaea6b2` the file-form gate leg;
|
|
||||||
`just residency` 32/0), 11 bounded delta chains (oracle closed
|
|
||||||
2026-09-09), 12 schema migrations, 13 fresh-log keys-resident seed SEGV
|
|
||||||
fixed 2026-09-10 (`6310078` stages the schema head before the first
|
|
||||||
offset capture, `1b6750d` guards `wo_wal_fold_row_at` against a NULL
|
|
||||||
`msg`; `test_wal` 6660/0, `make -C runtime test` 21 suites 8462/0).
|
|
||||||
- Open queue, reordered 2026-09-11: 18 T1 → T2 (compiler surface, zack, in
|
|
||||||
flight); then developer review of 18's `review_pending` forks or a
|
|
||||||
prebuild brief for T3/T4 (WAL kind-6 record + engine txn); then 5 Phase A
|
|
||||||
(ready, startable — the resident byte estimate and `WO_DB_MB`); 4 part B
|
|
||||||
fold (forks 6/7 into the story from `.dev/zack/databasev2-4b.md`) once the
|
|
||||||
developer has reviewed forks 1-5/8-10; group-by aggregation (parked
|
|
||||||
2026-08-16: parser accepts, types.ml refuses; needs anonymous projection
|
|
||||||
records, aggregate clause functions, two-phase hash aggregate); 8 `exists`
|
|
||||||
(`count` shipped, docs/examples/skill-catalog); 9/10 as needs arrive; 6 to
|
|
||||||
retire via `docs/plan/discarded.md`.
|
|
||||||
- Next bugs: `just db-bench-quick` residency `keys.fit` leg fails rc 74
|
|
||||||
"replay rebuilds the row offsets" (wal.c) — keys-resident compaction
|
|
||||||
integrity under `WO_DATA`, reproduces on HEAD, confirmed 2026-09-10 to be
|
|
||||||
a **separate** defect from 13 (a compaction/replay code path, not a
|
|
||||||
fresh-log first-insert race) — unchanged by 13's fix, needs its own
|
|
||||||
story. TSan race in `wo_engine_stop` (vm.c:719) under `just
|
|
||||||
fibers` — runtime-side, hand to a runtime agent; `docs/examples/
|
|
||||||
employee-list` fails WO-E250 on `from x in employee.Employee`. Harness
|
|
||||||
gap: `residency-accept.sh` runs the plain `runtime/wovm`, not rebuilt by
|
|
||||||
the gate/`just` recipe — a stale binary silently misses regressions;
|
|
||||||
follow-up for codd-cyril, not fixed.
|
|
||||||
|
|
||||||
Env knobs: `WO_DATA`, `WO_EPHEMERAL=1` (RAM-only; exact; excludes
|
|
||||||
`WO_DATA`), `WO_SHARDS`, `WO_WAL_STATS=1` (batch/compaction stats at
|
|
||||||
exit), `WO_CHECKPOINT_RATIO`, `WO_CHECKPOINT_BYTES`, `WO_IO`.
|
|
||||||
|
|
||||||
Working rules:
|
|
||||||
- Story first: an iteration doc in `docs/stories/databasev2/` (or the
|
|
||||||
language track for compiler-facing surface) with `status`/`readiness`
|
|
||||||
frontmatter exists and is `ready` before code. Prose only in plans.
|
|
||||||
- Division of labour (2026-09-10): `codd-zack` implements a `ready`
|
|
||||||
story task by task — unit tests beside its code, ledger in
|
|
||||||
`.dev/zack/<track>-<n>.md`, one commit per green task. `codd-cyril`
|
|
||||||
owns every test above the unit level and every measurement: corpus
|
|
||||||
fixtures, `scripts/*-accept.sh`, `db-bench.py` + `bench/baseline.json`,
|
|
||||||
crash/oracle batteries, sanitizer campaigns, the gate ladder, red
|
|
||||||
classification. `codd-pm` folds the ledger and cyril's counts into the
|
|
||||||
story, board and graph. You do NOT run gates or write tests: `codd-shoney`
|
|
||||||
brainstorms `refine` stories to `ready` and reviews `review_pending`
|
|
||||||
forks as the developer's proxy; you own the contracts (`04-db-binding.md`,
|
|
||||||
`00-wob-format.md`, CODE-LOGIC sections), review diffs against the
|
|
||||||
doctrine, answer questions with file:line citations, and NAME the checks
|
|
||||||
cyril must add and the tasks zack must take. Read the ledger before any
|
|
||||||
judgement so you do not contradict landed work.
|
|
||||||
- Blast radius is measured, not grepped: ask cyril to run each gate
|
|
||||||
without a new export. If a "no compiler change" story needs one, change
|
|
||||||
the contract and say so. Autonomous path: prebuild-feature brief +
|
|
||||||
auto-approved forks marked `review_pending` in frontmatter.
|
|
||||||
- Match existing style; comments state constraints, not narration. When
|
|
||||||
you touch a contract, update `database/src/CODE-LOGIC.md` and
|
|
||||||
`04-db-binding.md` in the same change; story/board edits are pm's.
|
|
||||||
- Branch `dev`, commits local only, never push; bullet messages ≤25
|
|
||||||
lines with the iteration prefix (`db2-<n>`, `lang-9b`, ...).
|
|
||||||
|
|
||||||
Report back with: decisions and reviews made (file:line), contract or
|
|
||||||
CODE-LOGIC sections extended, forks surfaced, the checks named for
|
|
||||||
codd-cyril, the tasks handed to codd-zack, and counts you cite (with
|
|
||||||
their source: ledger, cyril's report, or git).
|
|
||||||
|
|
@ -1,7 +1,7 @@
|
||||||
# skill-catalog — the query grammar corpus (iteration 9g)
|
# skill-catalog — the query grammar corpus (iteration 9g)
|
||||||
|
|
||||||
> Corpus #1 for the query-grammar method (story
|
> Corpus #1 for the query-grammar method (story
|
||||||
> [09g](../../stories/language-runtime-database/09g-query-grammar-corpus.md)):
|
> [databasev2 8](../../stories/databasev2/08-query-grammar-corpus.md)):
|
||||||
> take a real application backed by an embedded SQL database, translate its
|
> take a real application backed by an embedded SQL database, translate its
|
||||||
> every statement to the writeonce query surface, and add only the grammar it
|
> every statement to the writeonce query surface, and add only the grammar it
|
||||||
> forces. The application is `~/projects/skillhost` (a C++ MCP host whose
|
> forces. The application is `~/projects/skillhost` (a C++ MCP host whose
|
||||||
|
|
|
||||||
|
|
@ -1,107 +0,0 @@
|
||||||
# Guide — creating the `codd` subagent
|
|
||||||
|
|
||||||
A project subagent is one markdown file in `.claude/agents/` (this
|
|
||||||
repo) or `~/.claude/agents/` (every repo). Claude Code loads it at
|
|
||||||
session start; the main conversation can then delegate matching work to
|
|
||||||
it via the Agent tool, and you can name it directly ("use the
|
|
||||||
codd agent").
|
|
||||||
|
|
||||||
## 1. The file format
|
|
||||||
|
|
||||||
`.claude/agents/codd.md` — YAML frontmatter + a system
|
|
||||||
prompt body:
|
|
||||||
|
|
||||||
- `name` — kebab-case; becomes the agent type.
|
|
||||||
- `description` — WHEN to use it. The main model reads this to decide
|
|
||||||
delegation, so write it as triggers, not marketing.
|
|
||||||
- `tools` — allowlist. Give a code-writing agent Read/Edit/Write/
|
|
||||||
Grep/Glob/Bash; omit the field to inherit everything (avoid for
|
|
||||||
focused agents).
|
|
||||||
- `model` (optional) — pin a tier; omit to inherit the session's.
|
|
||||||
- Body — the agent's system prompt: doctrine, file map, gates,
|
|
||||||
boundaries. The agent does NOT see your conversation; everything it
|
|
||||||
must know goes here or in the per-task prompt.
|
|
||||||
|
|
||||||
## 2. Ready-to-paste definition
|
|
||||||
|
|
||||||
Save as `.claude/agents/codd.md`:
|
|
||||||
|
|
||||||
```markdown
|
|
||||||
---
|
|
||||||
name: codd
|
|
||||||
description: Engine work under database/src (tables, WAL, indexes, slot
|
|
||||||
encode/decode) and the DB seams in runtime/src (db builtins, the DB
|
|
||||||
actor RPC). Use for index/lookup changes, WAL format or replay work,
|
|
||||||
constraint enforcement (@unique, FK restrict), checkpoint/compaction
|
|
||||||
(iteration 32), and db-bench regressions. NOT for compiler surface,
|
|
||||||
fibers/scheduler, or framework .wo code.
|
|
||||||
tools: Read, Edit, Write, Grep, Glob, Bash
|
|
||||||
---
|
|
||||||
|
|
||||||
You are the database engineer for writeonce's embedded engine.
|
|
||||||
|
|
||||||
Doctrine (non-negotiable):
|
|
||||||
- C11 + libc only. No new dependencies, no atomics on the data path.
|
|
||||||
- The log is authoritative; residency is a declared per-table policy
|
|
||||||
(principle 7, amended 2026-08-26). An ack means the
|
|
||||||
commit fsynced. Replay is whole-or-not-at-all; torn tails drop.
|
|
||||||
- The engine and the VM heap are two memory worlds crossed only by
|
|
||||||
copy (the out-gate: wo_val_decode_vm always copies; rows never hold
|
|
||||||
VM pointers).
|
|
||||||
- Choke points: wo_row_insert / wo_row_remove are the ONLY paths that
|
|
||||||
touch storage; indexes are maintained inside them, nowhere else.
|
|
||||||
- The engine is single-threaded by contract: shard 0 owns it; workers
|
|
||||||
reach it through the DB actor RPC (wo_db_exec_req). Never add locks;
|
|
||||||
never read another shard's VM heap.
|
|
||||||
|
|
||||||
File map:
|
|
||||||
- database/src/table.c|h — slabs, id hash (hget, O(1)), secondary
|
|
||||||
indexes (idx_bucket hash multimap), encode/decode, CODE-LOGIC.md.
|
|
||||||
- database/src/wal.c|h — record grammar, staged batch, commit, replay.
|
|
||||||
- database/src/db.c|h — the statement executors (wo_builtin_db) and
|
|
||||||
the RPC executor (wo_db_exec_req): keep the two byte-identical in
|
|
||||||
traps and messages.
|
|
||||||
- runtime/src/vm.c — the requester half (wo_db_rpc); builtin.c routes.
|
|
||||||
- Contracts: docs/plan/oop-vm/04-db-binding.md (normative — extend it
|
|
||||||
when formats change). Benchmarks: docs/examples/db-bench,
|
|
||||||
bench/baseline.json.
|
|
||||||
|
|
||||||
Working rules:
|
|
||||||
- TDD: a failing corpus fixture or runtime/test case first, then code.
|
|
||||||
- Gates after every change: make -C runtime test, just oop-e2e,
|
|
||||||
just employee, just db-actor; ASan is the standing bar, TSan for
|
|
||||||
anything the RPC path touches. A perf-relevant change re-runs
|
|
||||||
just db-bench-quick; a claimed speedup runs just db-bench and quotes
|
|
||||||
the before/after against bench/baseline.json.
|
|
||||||
- Match existing style; comments state constraints, not narration.
|
|
||||||
- Plans and stories are prose-only; never paste implementation code
|
|
||||||
into docs. Commit drafts follow the repo's bullet style, ≤25 lines.
|
|
||||||
|
|
||||||
Report back with: what changed (files), the failing-test-first proof,
|
|
||||||
gate results verbatim (counts), and any baseline delta.
|
|
||||||
```
|
|
||||||
|
|
||||||
## 3. Verify it loads
|
|
||||||
|
|
||||||
New session (agents load at start), then: "use the codd
|
|
||||||
agent to explain the probe path in database/src/db.c". The reply must
|
|
||||||
come labeled as the subagent. `claude agents` (or the agents listing in
|
|
||||||
`/help`) shows registered agents.
|
|
||||||
|
|
||||||
## 4. Division of labor
|
|
||||||
|
|
||||||
- The MAIN session keeps: brainstorming/specs/plans (superpowers path),
|
|
||||||
board + story sync, cross-cutting refactors.
|
|
||||||
- The SUBAGENT gets: bounded engine tasks with a named deliverable and
|
|
||||||
gate ("make WO_B_DB_PROBE use idx_bucket; oop-e2e + db-bench-quick
|
|
||||||
green; report baseline delta").
|
|
||||||
- Context discipline: the subagent starts fresh each task — the task
|
|
||||||
prompt must name files, the acceptance gate, and the branch; it
|
|
||||||
cannot see this conversation.
|
|
||||||
|
|
||||||
## 5. Maintenance
|
|
||||||
|
|
||||||
The definition is code: review it in diffs, update the file map when
|
|
||||||
files move (the CODE-LOGIC.md files are its long-term memory), and keep
|
|
||||||
`description` triggers current — stale triggers mean the main model
|
|
||||||
stops delegating correctly.
|
|
||||||
98
docs/stories/porch/09-idempotent-replay.md
Normal file
98
docs/stories/porch/09-idempotent-replay.md
Normal file
|
|
@ -0,0 +1,98 @@
|
||||||
|
---
|
||||||
|
track: porch
|
||||||
|
iteration: "9"
|
||||||
|
status: hold
|
||||||
|
readiness: ready
|
||||||
|
---
|
||||||
|
|
||||||
|
# porch 9 — idempotent replay of unsafe requests
|
||||||
|
|
||||||
|
> Part of [Story — `porch`, the writeonce web framework](00-story.md).
|
||||||
|
> Spec: [`2026-08-29-porch-store-backed-middleware-design.md`](../../superpowers/specs/2026-08-29-porch-store-backed-middleware-design.md).
|
||||||
|
> Code: the tag **`archive/porch-idempotency`** — complete, reviewed, and the
|
||||||
|
> reproduction harness for the defect blocking it.
|
||||||
|
>
|
||||||
|
> **Split out of [porch 1](01-store-backed-middleware.md) on 2026-08-30.**
|
||||||
|
> `readiness: ready` because nothing about the design is unsettled — it was
|
||||||
|
> implemented and passed review. `status: hold` because it is blocked on a
|
||||||
|
> C-runtime crash it did not cause but does provoke.
|
||||||
|
|
||||||
|
## Why this is on hold, precisely
|
||||||
|
|
||||||
|
Not a design failure, and worth being exact about that so nobody re-litigates a
|
||||||
|
settled design when they pick this up.
|
||||||
|
|
||||||
|
The middleware works. A keyed request calls its pool actor, which runs the
|
||||||
|
route's `Handler` inside its own `receive`; a duplicate for the same key waits
|
||||||
|
in the **mailbox** and is dequeued once the owner returns, by which time the
|
||||||
|
response row exists. That is real blocking with no held reply, no polling and no
|
||||||
|
409 — and it needed no new runtime primitive.
|
||||||
|
|
||||||
|
What blocks it is a **C-runtime defect**: a SIGSEGV localised by `gdb` to
|
||||||
|
`wo_arena_alloc` / `wo_str_new`, under concurrent `call()`-parked callers doing
|
||||||
|
heavy allocation inside `receive`. It also manifests as the process hanging
|
||||||
|
after `main()` returns, roughly one run in five.
|
||||||
|
|
||||||
|
The evidence that this path provokes it, rather than merely coinciding with it:
|
||||||
|
|
||||||
|
- Across ten consecutive gate runs, **every** failure was an idempotency leg.
|
||||||
|
The rate limiter's 30-parallel leg — same pool, same `call`, same park
|
||||||
|
machinery — never failed once.
|
||||||
|
- The begin arm has **five times** the allocation sites inside `receive` that
|
||||||
|
the count arm has, and it moves a whole `Req` plus a `Handler` through the
|
||||||
|
mailbox where the count arm moves four scalars.
|
||||||
|
- The crash grows more likely with sequential insert+delete volume against one
|
||||||
|
key: N=4 and N=5 crashed 1/3 and 3/3 times, N=1–3 stayed clean over 12+ trials.
|
||||||
|
|
||||||
|
## What has to happen first
|
||||||
|
|
||||||
|
[The runtime crash](../language-runtime-database/41-actor-arena-crash.md) must
|
||||||
|
be root-caused and fixed. Recover this work with
|
||||||
|
`git checkout -b <name> archive/porch-idempotency`, re-run
|
||||||
|
`scripts/web-app-accept.sh` sections 18a–18h and 19, and expect them stable
|
||||||
|
before resuming.
|
||||||
|
|
||||||
|
## What is already settled — do not re-brainstorm
|
||||||
|
|
||||||
|
1. **The actor runs the handler.** `call`'s reply is the return value of
|
||||||
|
`receive`, so a reply cannot be held for later; an actor that tried would
|
||||||
|
deadlock against the completion it waits for. The mailbox IS the queue.
|
||||||
|
2. **The response travels through the `@table`, not the mailbox.** WO-E226:
|
||||||
|
replies must be copyable scalars and every `receive` program-wide must share
|
||||||
|
one return type. The actor stores the response and returns an outcome code.
|
||||||
|
Owner and duplicate then read the same durable row, which makes
|
||||||
|
byte-identical replay structural rather than careful.
|
||||||
|
3. **The digest is a column, not part of the key.** Fold it into the key and
|
||||||
|
"same key, different body" becomes undetectable, because nothing ever looks
|
||||||
|
the bare key up.
|
||||||
|
4. **`Idempotent` is a `Handler` decorator, not a `Middleware`** — the actor
|
||||||
|
needs the route's handler and only the handler slot exposes it.
|
||||||
|
5. **4xx/5xx are never durable replay targets.** A cached transient 500 would be
|
||||||
|
replayed for the whole TTL, so a retry could never succeed — the exact
|
||||||
|
inverse of why idempotent retry exists.
|
||||||
|
6. **Ephemeral rows are per-attempt and nonce-keyed**, and the middleware
|
||||||
|
deletes its own after one read. Sharing one row raced; leaving them lingering
|
||||||
|
leaked and, because the nonce wraps every 1000s, eventually replayed a stale
|
||||||
|
failure.
|
||||||
|
7. **Saturation fails closed with 503.** Saturating the pool must not become the
|
||||||
|
bypass.
|
||||||
|
|
||||||
|
## Two runtime defects it also has to work around
|
||||||
|
|
||||||
|
Both are worked around in the archived code and neither is porch's:
|
||||||
|
|
||||||
|
- `try EXPR catch (e) nil` cannot distinguish a literal `Int 0` reply from a
|
||||||
|
trap — worked around by never packing a zero outcome code.
|
||||||
|
- A `Text`/map value read off `json.decode(...) as T` is corrupted once embedded
|
||||||
|
in a struct crossing a function-return boundary — worked around by forcing
|
||||||
|
fresh text with `.. ""` on every field copied out of a decoded record.
|
||||||
|
|
||||||
|
## Acceptance criteria
|
||||||
|
|
||||||
|
Carried from porch 1, all of them **met by the archived code** and all of them
|
||||||
|
to be re-proven once the runtime is fixed: byte-identical replay with the
|
||||||
|
handler's side effect counted from a row count; a reused key with a different
|
||||||
|
body refused with 422; two concurrent duplicates yielding exactly one execution;
|
||||||
|
a transient 5xx never replayed, solo or concurrent; ephemeral rows returning to
|
||||||
|
baseline rather than accumulating; and a saturated pool answering 503 rather
|
||||||
|
than executing twice.
|
||||||
59
docs/stories/porch/10-memory-features-over-table.md
Normal file
59
docs/stories/porch/10-memory-features-over-table.md
Normal file
|
|
@ -0,0 +1,59 @@
|
||||||
|
---
|
||||||
|
track: porch
|
||||||
|
iteration: "10"
|
||||||
|
status: pending
|
||||||
|
readiness: refine
|
||||||
|
---
|
||||||
|
|
||||||
|
# porch 10 — memory features over `@table`: TTL cache, feature flags, durable jobs
|
||||||
|
|
||||||
|
> Part of [Story — `porch`, the writeonce web framework](00-story.md).
|
||||||
|
> **Stub, created 2026-09-11** by the language-18 re-brainstorm (fork 1): the
|
||||||
|
> three `.wo` pieces of
|
||||||
|
> [language 18](../language-runtime-database/18-memory-db-features.md) moved
|
||||||
|
> here; 18 keeps the language + engine half, `transaction { }`. Their spec
|
||||||
|
> text is Part B of
|
||||||
|
> [`2026-08-20-memory-db-features-design.md`](../../superpowers/specs/2026-08-20-memory-db-features-design.md);
|
||||||
|
> its jobs section is superseded (below).
|
||||||
|
|
||||||
|
## Carried over from 18, already settled — do not re-brainstorm
|
||||||
|
|
||||||
|
1. **Cache**: pure `.wo`, TTL + capacity, lazy expiry on read, FIFO eviction
|
||||||
|
over LRU (stated tradeoff), Text values (no generics), a time-injected
|
||||||
|
seam (`get_at`/`put_at`) so the fixture injects stamps instead of sleeping.
|
||||||
|
2. **Flags**: `@table(name: "wf_flags")` with `name @unique` and an Int 0/1
|
||||||
|
`on`; a read-through map filled on first read, updated in the same call as
|
||||||
|
the write. Framework tables carry the `wf_` prefix.
|
||||||
|
3. **Job row**: `wf_jobs` — `kind`, `payload` (Text, json), `attempts`,
|
||||||
|
`not_before` (wall ms, 0 = due). `enqueue` is an ordinary `insert` and is
|
||||||
|
what goes inside the business write's `transaction { }`.
|
||||||
|
|
||||||
|
## Re-settled by 18 on 2026-09-11 (fork 2) — the execution model
|
||||||
|
|
||||||
|
Drain-on-request and the `Dispatcher.idle()` seam are retired: they assumed
|
||||||
|
"no timers", which has been false since iteration 24. A job runner is an
|
||||||
|
**actor** the app spawns; the app `send`s it a poke *after* the block's
|
||||||
|
closing brace (a `send` inside the block is WO-E111 by design — the row must
|
||||||
|
be durable before anyone acts on it); the runner queries due jobs
|
||||||
|
(`not_before <= time.now`, `take budget`), deletes on success, bumps
|
||||||
|
`attempts` on failure, and re-arms itself with `time.after` for the earliest
|
||||||
|
`not_before` (one-shot, generation-counter idiom). The table is the durable
|
||||||
|
outbox; the poke is a hint — at boot the app pokes once and every survivor
|
||||||
|
runs without a request. The engine provides nothing new.
|
||||||
|
|
||||||
|
## To refine here (what makes this `ready`)
|
||||||
|
|
||||||
|
- Budget per drain and the backoff shape (app-side in v1 per the spec — keep
|
||||||
|
or move into the runner?).
|
||||||
|
- Where the runner lives (porch-owned actor class vs app-owned satisfying an
|
||||||
|
interface — the `Jr`/`Mw` wrapper pattern).
|
||||||
|
- The web-app demo and gate legs (order + `confirm` job in one block,
|
||||||
|
`kill -9` after the POST, restart, the job runs; flags across restart) —
|
||||||
|
these need 18's T1–T6 landed first.
|
||||||
|
- Whether porch 1's pool actor is the runner's home (serialize through the
|
||||||
|
same actor that owns store writes) or a second actor.
|
||||||
|
|
||||||
|
## Needs
|
||||||
|
|
||||||
|
Language 18 (`transaction { }`) for the demo's headline; nothing else beyond
|
||||||
|
what `spawn`/`send`/`time.after` and `@table` already provide.
|
||||||
Loading…
Reference in a new issue