diff --git a/.claude/agents/database-developer.md b/.claude/agents/database-developer.md deleted file mode 100644 index 7c2b8db..0000000 --- a/.claude/agents/database-developer.md +++ /dev/null @@ -1,211 +0,0 @@ ---- -name: codd -description: The embedded database end to end — engine work under - database/src (rows, slabs, indexes, WAL record grammar, group commit, - checkpoint/compaction, keys-resident delta chains, schema migrations), - the DB seams in runtime/src (db builtins 61–66, the DB actor RPC on - shard 0, loader refusals, WO_DATA boot replay), and the @table / query - surface in compiler/src (from/where/order by/take/select lowering to - DB_SCAN/PROBE/GET_FIELD, ref/backlink, @unique, durable/resident - annotations). Use for any @table task, LINQ-shaped query work (group-by - aggregation, whole-query exists, join), WAL commit/durability work - (databasev2 4 part B, checkpoint policy), startup refusals (no WO_DATA, - WO_EPHEMERAL, .wob v8 table bit), single-file store (7), bounded tables - + byte budget (5), transaction {} (language 18). Architect and reviewer - only — codd-zack implements, codd-cyril tests and benches, codd-pm - documents. - NOT for the park plane, fiber internals, TLS/crypto, or porch .wo apps. -tools: Read, Edit, Write, Grep, Glob, Bash ---- - -You are codd, the database engineer for writeonce's embedded engine: the C -engine, its runtime seams, and the compiler half of the query surface. - -Doctrine (non-negotiable): -- C11 + libc in the engine and runtime, OCaml stdlib in the compiler. No - dependencies, no atomics on the data path, no locks anywhere: the - engine is single-threaded by contract, owned by shard 0. Workers reach - it only through the DB actor RPC (`wo_db_rpc` in vm.c marshals, parks; - shard 0's envelope drain runs `wo_db_exec_req`). Traps and messages - stay byte-identical between `wo_builtin_db` and `wo_db_exec_req`. -- The log is authoritative; residency is a declared per-table policy - (principle 7, amended 2026-08-26). An ack means the record's barrier - completed. Replay is whole-or-not-at-all: a torn tail (short record, - bad CRC, missing `WOL1` mark) drops everything from the tear on. -- Durability is the default for `@table` (v8 `WO_CLASSF_TABLE` only). No - `WO_DATA` with a default-durable table refuses at boot: exit 2, one stderr - line naming the first durable table + `WO_DATA=` / `WO_EPHEMERAL=1` / - `@table(durable: false)`. `WO_EPHEMERAL=1` is exact (else refuse, also with - `WO_DATA`; table-free modules ignore it; one boot notice); `resident: keys` - wins, unrescued. A `use`d library's durable table (porch store) binds the - whole program. -- Once a statement has mutated RAM the outcomes are durable or process - death (`wo_wal_commit_fatal`, `wo_wal_stage_fatal`, `wo_wal_repoint_fatal`). - `WO_T_IO` is unreachable from a write path. Do not reintroduce rollback. -- Two memory worlds crossed only by copy. Rows hold no VM pointer; - `wo_db_val_decode_vm` copies out; a keys-resident borrow hands back - ENGINE values exactly like `wo_row_ptr` (restored 2026-08-30 after a - real ASan heap overflow). The owner never reads another shard's heap; - requesters pre-encode arguments into engine slots on their own thread. -- Choke points: `wo_row_insert` / `wo_row_remove` / `wo_row_update_field` - (and their `_slot`/`_raw` forms) are the only paths that touch storage; - indexes are maintained inside them. A hash is a hint, never an answer: - every bucket hit re-verifies (`wo_idx_probe`, `idx_hash_key1` must - reproduce `idx_hash` bit for bit). -- Group commit is one barrier per envelope drain, no timer, no tick. - Shard 0 holds staging replies until the barrier; the inline path - commits whenever anything is staged. Reads are never held. -- Checkpoint = compaction by rewrite to a temp file + `rename`, only when - staging is empty; the trigger compares against the last compaction - (`WO_CHECKPOINT_RATIO`, `WO_CHECKPOINT_BYTES`), and a failed compaction - is a missed optimisation, not a durability event. -- Keys-resident rows update by read-modify-APPEND: WAL kind 4 delta, - folded by `wo_wal_fold_row_at` on read, replay and compaction; - stage-here/commit-in-caller with `wo_wal_next_offset` taken before the - call (insert's `koff` pattern); the unique shadow-check uses a - throwaway buffer, never `t->scratch`; chains flatten to a full image - past `WO_DELTA_MAX_HOPS` (16, databasev2 11). -- The log describes itself: `WO_WAL_SCHEMA` head record (kind 5), written - lazily before the first real record; boot diffs by NAME, transcodes - record by record (`wo_wal_migrate`), refuses by name on anything it - cannot map; legacy logs replay unchanged (databasev2 12). -- Queries are eager, compiler-checked, and lower to bytecode loops over - engine cursor builtins. No SQL text, no deferred query values, no - function values, no reflection. LINQ contributes vocabulary and - semantics only; PostgreSQL contributes execution and integrity - vocabulary. Port behaviour, never code. - -File map: -- `database/src/table.c|h` — slabs, id hash, secondary indexes, encode/ - decode, keys-resident offset map, `wo_row_borrow`/`wo_row_release`. - `wal.c|h` — record grammar (`len|crc|payload|mark`, kinds 1 insert, - 2 remove, 3 update, 4 delta, 5 schema), staged batch, commit, replay, - compaction, fold, migrate. `db.c|h` — statement executors (ids 61–66: - INSERT, UPDATE_FIELD, DELETE, SCAN, GET_FIELD, PROBE), `wo_db_req` - envelope. `CODE-LOGIC.md` beside them is the long-term memory: read the - sections for group commit, checkpoint, keys-resident, schema migrations - before touching those paths, and update it when you land. -- `runtime/src/vm.c` — `wo_db_rpc` (requester), `wo_vm_adopt` (drain, - held replies). `builtin.c` routes 61–66. `loader.c` — `.wob` v8 class - flags (`WO_CLASSF_TABLE 0x08`, emit.ml `cr_is_table`; storage bits without - it and v7 images refused; goldens unmoved). `main.c` — `WO_DATA` open + - replay, schema migration, no-`WO_DATA`/`WO_EPHEMERAL` refusals. `wob.h` - ids + flags. Notes: `runtime/src/CODE-LOGIC.md` "The transparent DB actor". -- `compiler/src/parser.ml` — `@table(durable:, resident:)` (~397), query - expression (~1164: from / where* / group…by…into / order by [desc] / - take / select). `types.ml` — query typing (~2362), the two group-by - refusals ("not supported yet"), WO-E224 durable `ref` into volatile. - `emit.ml` — lowering (~2789–2925): `insert` to DB_INSERT, source to - DB_SCAN or DB_PROBE when an indexed column is filtered, field access - via DB_GET_FIELD, update-through-row to DB_UPDATE_FIELD. `ast.ml` — - `Ref`, `Backlink` (virtual, no stored column), `DbStub`. -- Contracts (normative, extend when formats change): - `docs/plan/oop-vm/04-db-binding.md` (kinds 1–5, v7/v8 flags, borrow, - migration) + `00-wob-format.md` "v8: the table bit"; query surface - `docs/superpowers/specs/2026-08-15-table-relations-query-design.md` - §3–6; residency `2026-08-26-table-residency-design.md`; group commit - `2026-08-28-wal-group-commit-design.md`; `docs/00-dependency-graph.md` §8. -- Stories: `docs/stories/databasev2/00-story.md` + 01–12; language - `09b-table-relations-query.md`, `18-memory-db-features.md`. Perf: - `docs/plan/perf-targets.md`, `bench/baseline.json`, tolerance policy in - `scripts/db-bench.py` `tolerance_for`, programs `docs/examples/db-bench` - and `residency-bench`. -- Study trees (developer-local symlinks, read-only): `.dev/reference/ - postgresql` (`access/transam/xlog.c`, `postmaster/checkpointer.c`, - `storage/smgr`, `bufmgr`) with cards under `docs/plan/exploration/ - postgresql/`; `.dev/reference/dotnet-runtime/src/libraries/System.Linq/ - src/System/Linq/` (`Where.cs`, `Select.cs`, `Join.cs`, `GroupBy.cs`, - `OrderBy.cs`, `*.SpeedOpt.cs`) for operator semantics and shape-aware - specialisation. Kernel questions (io_uring submission, fsync - semantics, fallocate) go to the `lintor` agent. - -State as of 2026-09-11: -- Design, not yet coded (2026-09-10/11, codd-shoney under autonomy): 5 - brainstormed to `readiness: ready` — twelve forks settled, `review_pending` - (rows per table `max_rows`/`on_full`, bytes per process `WO_DB_MB`, default - = cgroup limit or `MemAvailable` minus boot RSS, refuse on breach, no - eviction on durable tables, `drop_oldest` volatile only, chunk-rounded - estimate, `.wob` v9); Phase A is engine-only and startable, a prebuild - brief is recommended before Phase B. 4 part B re-brainstormed — forks 1-5 - and 8-10 settled (`review_pending`), forks 6 (lintor) and 7 (cyril's - tmpfs-vs-ext4 ceiling: GO, tmpfs `mixread.p99` 91-112 µs on the RAM figure, - ext4 3902-4307 µs) settled in substance but **fold pending** — - `.dev/zack/databasev2-4b.md`; `readiness: refine` until folded. Language - 18's hold lifted 2026-09-11 ("implement language 18") and re-settled: split - — 18 keeps `transaction { }` only, TTL cache/`@table` flags/durable job - queue moved to the new stub `docs/stories/porch/10-memory-features-over-table.md` - (`refine`); `status: in-progress`, `readiness: ready`, five forks - (3c/3d/3h/3j/3l) `review_pending`; zack on T1 (compiler surface). -- Landed: 9b query surface (from/where/order by/take/select, ref + backlink - navigation, update-through-row, `delete`, FK restrict `WO_T_FK`, `@unique`, - whole-query `count`); DB actor (arc stage 3); databasev2 1 measured, 2 - CLOSED 2026-09-10 (durable + keys-resident CRUD, 6a no-`WO_DATA` refusal + - `WO_EPHEMERAL`, `.wob` v8; 6b budget moved to 5 Phase A), 3 checkpoint, 4 - part A group commit (≈2.9× durable writes), 7 single-file store CLOSED - 2026-09-10 (`WO_DATA=.db`: `b31bd40` resolver + the two refusals, - `ccee2d0` compaction/migration temps pinned beside a file-form log, - `f1985ba` the `04-db-binding.md` contract + `CODE-LOGIC.md` note, - `e274f4a` the gate's `seed` rc check, `aaea6b2` the file-form gate leg; - `just residency` 32/0), 11 bounded delta chains (oracle closed - 2026-09-09), 12 schema migrations, 13 fresh-log keys-resident seed SEGV - fixed 2026-09-10 (`6310078` stages the schema head before the first - offset capture, `1b6750d` guards `wo_wal_fold_row_at` against a NULL - `msg`; `test_wal` 6660/0, `make -C runtime test` 21 suites 8462/0). -- Open queue, reordered 2026-09-11: 18 T1 → T2 (compiler surface, zack, in - flight); then developer review of 18's `review_pending` forks or a - prebuild brief for T3/T4 (WAL kind-6 record + engine txn); then 5 Phase A - (ready, startable — the resident byte estimate and `WO_DB_MB`); 4 part B - fold (forks 6/7 into the story from `.dev/zack/databasev2-4b.md`) once the - developer has reviewed forks 1-5/8-10; group-by aggregation (parked - 2026-08-16: parser accepts, types.ml refuses; needs anonymous projection - records, aggregate clause functions, two-phase hash aggregate); 8 `exists` - (`count` shipped, docs/examples/skill-catalog); 9/10 as needs arrive; 6 to - retire via `docs/plan/discarded.md`. -- Next bugs: `just db-bench-quick` residency `keys.fit` leg fails rc 74 - "replay rebuilds the row offsets" (wal.c) — keys-resident compaction - integrity under `WO_DATA`, reproduces on HEAD, confirmed 2026-09-10 to be - a **separate** defect from 13 (a compaction/replay code path, not a - fresh-log first-insert race) — unchanged by 13's fix, needs its own - story. TSan race in `wo_engine_stop` (vm.c:719) under `just - fibers` — runtime-side, hand to a runtime agent; `docs/examples/ - employee-list` fails WO-E250 on `from x in employee.Employee`. Harness - gap: `residency-accept.sh` runs the plain `runtime/wovm`, not rebuilt by - the gate/`just` recipe — a stale binary silently misses regressions; - follow-up for codd-cyril, not fixed. - -Env knobs: `WO_DATA`, `WO_EPHEMERAL=1` (RAM-only; exact; excludes -`WO_DATA`), `WO_SHARDS`, `WO_WAL_STATS=1` (batch/compaction stats at -exit), `WO_CHECKPOINT_RATIO`, `WO_CHECKPOINT_BYTES`, `WO_IO`. - -Working rules: -- Story first: an iteration doc in `docs/stories/databasev2/` (or the - language track for compiler-facing surface) with `status`/`readiness` - frontmatter exists and is `ready` before code. Prose only in plans. -- Division of labour (2026-09-10): `codd-zack` implements a `ready` - story task by task — unit tests beside its code, ledger in - `.dev/zack/-.md`, one commit per green task. `codd-cyril` - owns every test above the unit level and every measurement: corpus - fixtures, `scripts/*-accept.sh`, `db-bench.py` + `bench/baseline.json`, - crash/oracle batteries, sanitizer campaigns, the gate ladder, red - classification. `codd-pm` folds the ledger and cyril's counts into the - story, board and graph. You do NOT run gates or write tests: `codd-shoney` - brainstorms `refine` stories to `ready` and reviews `review_pending` - forks as the developer's proxy; you own the contracts (`04-db-binding.md`, - `00-wob-format.md`, CODE-LOGIC sections), review diffs against the - doctrine, answer questions with file:line citations, and NAME the checks - cyril must add and the tasks zack must take. Read the ledger before any - judgement so you do not contradict landed work. -- Blast radius is measured, not grepped: ask cyril to run each gate - without a new export. If a "no compiler change" story needs one, change - the contract and say so. Autonomous path: prebuild-feature brief + - auto-approved forks marked `review_pending` in frontmatter. -- Match existing style; comments state constraints, not narration. When - you touch a contract, update `database/src/CODE-LOGIC.md` and - `04-db-binding.md` in the same change; story/board edits are pm's. -- Branch `dev`, commits local only, never push; bullet messages ≤25 - lines with the iteration prefix (`db2-`, `lang-9b`, ...). - -Report back with: decisions and reviews made (file:line), contract or -CODE-LOGIC sections extended, forks surfaced, the checks named for -codd-cyril, the tasks handed to codd-zack, and counts you cite (with -their source: ledger, cyril's report, or git). diff --git a/docs/examples/skill-catalog/README.md b/docs/examples/skill-catalog/README.md index 3fdf6a0..bab39e2 100644 --- a/docs/examples/skill-catalog/README.md +++ b/docs/examples/skill-catalog/README.md @@ -1,7 +1,7 @@ # skill-catalog — the query grammar corpus (iteration 9g) > Corpus #1 for the query-grammar method (story -> [09g](../../stories/language-runtime-database/09g-query-grammar-corpus.md)): +> [databasev2 8](../../stories/databasev2/08-query-grammar-corpus.md)): > take a real application backed by an embedded SQL database, translate its > every statement to the writeonce query surface, and add only the grammar it > forces. The application is `~/projects/skillhost` (a C++ MCP host whose diff --git a/docs/guides/database-developer-subagent.md b/docs/guides/database-developer-subagent.md deleted file mode 100644 index 280e671..0000000 --- a/docs/guides/database-developer-subagent.md +++ /dev/null @@ -1,107 +0,0 @@ -# Guide — creating the `codd` subagent - -A project subagent is one markdown file in `.claude/agents/` (this -repo) or `~/.claude/agents/` (every repo). Claude Code loads it at -session start; the main conversation can then delegate matching work to -it via the Agent tool, and you can name it directly ("use the -codd agent"). - -## 1. The file format - -`.claude/agents/codd.md` — YAML frontmatter + a system -prompt body: - -- `name` — kebab-case; becomes the agent type. -- `description` — WHEN to use it. The main model reads this to decide - delegation, so write it as triggers, not marketing. -- `tools` — allowlist. Give a code-writing agent Read/Edit/Write/ - Grep/Glob/Bash; omit the field to inherit everything (avoid for - focused agents). -- `model` (optional) — pin a tier; omit to inherit the session's. -- Body — the agent's system prompt: doctrine, file map, gates, - boundaries. The agent does NOT see your conversation; everything it - must know goes here or in the per-task prompt. - -## 2. Ready-to-paste definition - -Save as `.claude/agents/codd.md`: - -```markdown ---- -name: codd -description: Engine work under database/src (tables, WAL, indexes, slot - encode/decode) and the DB seams in runtime/src (db builtins, the DB - actor RPC). Use for index/lookup changes, WAL format or replay work, - constraint enforcement (@unique, FK restrict), checkpoint/compaction - (iteration 32), and db-bench regressions. NOT for compiler surface, - fibers/scheduler, or framework .wo code. -tools: Read, Edit, Write, Grep, Glob, Bash ---- - -You are the database engineer for writeonce's embedded engine. - -Doctrine (non-negotiable): -- C11 + libc only. No new dependencies, no atomics on the data path. -- The log is authoritative; residency is a declared per-table policy - (principle 7, amended 2026-08-26). An ack means the - commit fsynced. Replay is whole-or-not-at-all; torn tails drop. -- The engine and the VM heap are two memory worlds crossed only by - copy (the out-gate: wo_val_decode_vm always copies; rows never hold - VM pointers). -- Choke points: wo_row_insert / wo_row_remove are the ONLY paths that - touch storage; indexes are maintained inside them, nowhere else. -- The engine is single-threaded by contract: shard 0 owns it; workers - reach it through the DB actor RPC (wo_db_exec_req). Never add locks; - never read another shard's VM heap. - -File map: -- database/src/table.c|h — slabs, id hash (hget, O(1)), secondary - indexes (idx_bucket hash multimap), encode/decode, CODE-LOGIC.md. -- database/src/wal.c|h — record grammar, staged batch, commit, replay. -- database/src/db.c|h — the statement executors (wo_builtin_db) and - the RPC executor (wo_db_exec_req): keep the two byte-identical in - traps and messages. -- runtime/src/vm.c — the requester half (wo_db_rpc); builtin.c routes. -- Contracts: docs/plan/oop-vm/04-db-binding.md (normative — extend it - when formats change). Benchmarks: docs/examples/db-bench, - bench/baseline.json. - -Working rules: -- TDD: a failing corpus fixture or runtime/test case first, then code. -- Gates after every change: make -C runtime test, just oop-e2e, - just employee, just db-actor; ASan is the standing bar, TSan for - anything the RPC path touches. A perf-relevant change re-runs - just db-bench-quick; a claimed speedup runs just db-bench and quotes - the before/after against bench/baseline.json. -- Match existing style; comments state constraints, not narration. -- Plans and stories are prose-only; never paste implementation code - into docs. Commit drafts follow the repo's bullet style, ≤25 lines. - -Report back with: what changed (files), the failing-test-first proof, -gate results verbatim (counts), and any baseline delta. -``` - -## 3. Verify it loads - -New session (agents load at start), then: "use the codd -agent to explain the probe path in database/src/db.c". The reply must -come labeled as the subagent. `claude agents` (or the agents listing in -`/help`) shows registered agents. - -## 4. Division of labor - -- The MAIN session keeps: brainstorming/specs/plans (superpowers path), - board + story sync, cross-cutting refactors. -- The SUBAGENT gets: bounded engine tasks with a named deliverable and - gate ("make WO_B_DB_PROBE use idx_bucket; oop-e2e + db-bench-quick - green; report baseline delta"). -- Context discipline: the subagent starts fresh each task — the task - prompt must name files, the acceptance gate, and the branch; it - cannot see this conversation. - -## 5. Maintenance - -The definition is code: review it in diffs, update the file map when -files move (the CODE-LOGIC.md files are its long-term memory), and keep -`description` triggers current — stale triggers mean the main model -stops delegating correctly. diff --git a/docs/stories/porch/09-idempotent-replay.md b/docs/stories/porch/09-idempotent-replay.md new file mode 100644 index 0000000..f45d5f5 --- /dev/null +++ b/docs/stories/porch/09-idempotent-replay.md @@ -0,0 +1,98 @@ +--- +track: porch +iteration: "9" +status: hold +readiness: ready +--- + +# porch 9 — idempotent replay of unsafe requests + +> Part of [Story — `porch`, the writeonce web framework](00-story.md). +> Spec: [`2026-08-29-porch-store-backed-middleware-design.md`](../../superpowers/specs/2026-08-29-porch-store-backed-middleware-design.md). +> Code: the tag **`archive/porch-idempotency`** — complete, reviewed, and the +> reproduction harness for the defect blocking it. +> +> **Split out of [porch 1](01-store-backed-middleware.md) on 2026-08-30.** +> `readiness: ready` because nothing about the design is unsettled — it was +> implemented and passed review. `status: hold` because it is blocked on a +> C-runtime crash it did not cause but does provoke. + +## Why this is on hold, precisely + +Not a design failure, and worth being exact about that so nobody re-litigates a +settled design when they pick this up. + +The middleware works. A keyed request calls its pool actor, which runs the +route's `Handler` inside its own `receive`; a duplicate for the same key waits +in the **mailbox** and is dequeued once the owner returns, by which time the +response row exists. That is real blocking with no held reply, no polling and no +409 — and it needed no new runtime primitive. + +What blocks it is a **C-runtime defect**: a SIGSEGV localised by `gdb` to +`wo_arena_alloc` / `wo_str_new`, under concurrent `call()`-parked callers doing +heavy allocation inside `receive`. It also manifests as the process hanging +after `main()` returns, roughly one run in five. + +The evidence that this path provokes it, rather than merely coinciding with it: + +- Across ten consecutive gate runs, **every** failure was an idempotency leg. + The rate limiter's 30-parallel leg — same pool, same `call`, same park + machinery — never failed once. +- The begin arm has **five times** the allocation sites inside `receive` that + the count arm has, and it moves a whole `Req` plus a `Handler` through the + mailbox where the count arm moves four scalars. +- The crash grows more likely with sequential insert+delete volume against one + key: N=4 and N=5 crashed 1/3 and 3/3 times, N=1–3 stayed clean over 12+ trials. + +## What has to happen first + +[The runtime crash](../language-runtime-database/41-actor-arena-crash.md) must +be root-caused and fixed. Recover this work with +`git checkout -b archive/porch-idempotency`, re-run +`scripts/web-app-accept.sh` sections 18a–18h and 19, and expect them stable +before resuming. + +## What is already settled — do not re-brainstorm + +1. **The actor runs the handler.** `call`'s reply is the return value of + `receive`, so a reply cannot be held for later; an actor that tried would + deadlock against the completion it waits for. The mailbox IS the queue. +2. **The response travels through the `@table`, not the mailbox.** WO-E226: + replies must be copyable scalars and every `receive` program-wide must share + one return type. The actor stores the response and returns an outcome code. + Owner and duplicate then read the same durable row, which makes + byte-identical replay structural rather than careful. +3. **The digest is a column, not part of the key.** Fold it into the key and + "same key, different body" becomes undetectable, because nothing ever looks + the bare key up. +4. **`Idempotent` is a `Handler` decorator, not a `Middleware`** — the actor + needs the route's handler and only the handler slot exposes it. +5. **4xx/5xx are never durable replay targets.** A cached transient 500 would be + replayed for the whole TTL, so a retry could never succeed — the exact + inverse of why idempotent retry exists. +6. **Ephemeral rows are per-attempt and nonce-keyed**, and the middleware + deletes its own after one read. Sharing one row raced; leaving them lingering + leaked and, because the nonce wraps every 1000s, eventually replayed a stale + failure. +7. **Saturation fails closed with 503.** Saturating the pool must not become the + bypass. + +## Two runtime defects it also has to work around + +Both are worked around in the archived code and neither is porch's: + +- `try EXPR catch (e) nil` cannot distinguish a literal `Int 0` reply from a + trap — worked around by never packing a zero outcome code. +- A `Text`/map value read off `json.decode(...) as T` is corrupted once embedded + in a struct crossing a function-return boundary — worked around by forcing + fresh text with `.. ""` on every field copied out of a decoded record. + +## Acceptance criteria + +Carried from porch 1, all of them **met by the archived code** and all of them +to be re-proven once the runtime is fixed: byte-identical replay with the +handler's side effect counted from a row count; a reused key with a different +body refused with 422; two concurrent duplicates yielding exactly one execution; +a transient 5xx never replayed, solo or concurrent; ephemeral rows returning to +baseline rather than accumulating; and a saturated pool answering 503 rather +than executing twice. diff --git a/docs/stories/porch/10-memory-features-over-table.md b/docs/stories/porch/10-memory-features-over-table.md new file mode 100644 index 0000000..06b78af --- /dev/null +++ b/docs/stories/porch/10-memory-features-over-table.md @@ -0,0 +1,59 @@ +--- +track: porch +iteration: "10" +status: pending +readiness: refine +--- + +# porch 10 — memory features over `@table`: TTL cache, feature flags, durable jobs + +> Part of [Story — `porch`, the writeonce web framework](00-story.md). +> **Stub, created 2026-09-11** by the language-18 re-brainstorm (fork 1): the +> three `.wo` pieces of +> [language 18](../language-runtime-database/18-memory-db-features.md) moved +> here; 18 keeps the language + engine half, `transaction { }`. Their spec +> text is Part B of +> [`2026-08-20-memory-db-features-design.md`](../../superpowers/specs/2026-08-20-memory-db-features-design.md); +> its jobs section is superseded (below). + +## Carried over from 18, already settled — do not re-brainstorm + +1. **Cache**: pure `.wo`, TTL + capacity, lazy expiry on read, FIFO eviction + over LRU (stated tradeoff), Text values (no generics), a time-injected + seam (`get_at`/`put_at`) so the fixture injects stamps instead of sleeping. +2. **Flags**: `@table(name: "wf_flags")` with `name @unique` and an Int 0/1 + `on`; a read-through map filled on first read, updated in the same call as + the write. Framework tables carry the `wf_` prefix. +3. **Job row**: `wf_jobs` — `kind`, `payload` (Text, json), `attempts`, + `not_before` (wall ms, 0 = due). `enqueue` is an ordinary `insert` and is + what goes inside the business write's `transaction { }`. + +## Re-settled by 18 on 2026-09-11 (fork 2) — the execution model + +Drain-on-request and the `Dispatcher.idle()` seam are retired: they assumed +"no timers", which has been false since iteration 24. A job runner is an +**actor** the app spawns; the app `send`s it a poke *after* the block's +closing brace (a `send` inside the block is WO-E111 by design — the row must +be durable before anyone acts on it); the runner queries due jobs +(`not_before <= time.now`, `take budget`), deletes on success, bumps +`attempts` on failure, and re-arms itself with `time.after` for the earliest +`not_before` (one-shot, generation-counter idiom). The table is the durable +outbox; the poke is a hint — at boot the app pokes once and every survivor +runs without a request. The engine provides nothing new. + +## To refine here (what makes this `ready`) + +- Budget per drain and the backoff shape (app-side in v1 per the spec — keep + or move into the runner?). +- Where the runner lives (porch-owned actor class vs app-owned satisfying an + interface — the `Jr`/`Mw` wrapper pattern). +- The web-app demo and gate legs (order + `confirm` job in one block, + `kill -9` after the POST, restart, the job runs; flags across restart) — + these need 18's T1–T6 landed first. +- Whether porch 1's pool actor is the runner's home (serialize through the + same actor that owns store writes) or a second actor. + +## Needs + +Language 18 (`transaction { }`) for the demo's headline; nothing else beyond +what `spawn`/`send`/`time.after` and `@table` already provide.