docs: concurrency chain re-refined + resequenced
- order now stage 3 -> 22 -> 31 -> 24 -> 23: correctness before measurement (multi-shard DB traps WO_T_DB today) - stories 08/11 catch up to landed arc stages 1+2 (plan of record, deviations, settled open questions) - 22 gains multi-shard + mutex-inbox targets; deltas owed retroactively - 23 rides arc's per-shard ring (T4); old-id order string superseded - 24 depends on 31; 19 landed so Bytes ready - new story: refine/31-actor-lifecycle.md (request/response, bounded mailboxes, death/supervision, timers) - 00-story table + 00-status pending resequenced; held rows link hold/; ids stay immutable, no renames Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
parent
38e695c3b9
commit
68f90229b8
8 changed files with 293 additions and 110 deletions
|
|
@ -65,10 +65,11 @@ routing/security gaps is **framework v1**, tracked item-by-item in the
|
|||
(✅/🔶/⬜/⏸/🔧 per feature — timeouts and Unix sockets need `net` runtime
|
||||
seams, crypto hashes need C builtins since the language has no bitwise
|
||||
operators, streaming/cancellation park behind 8/11). The memory-rich
|
||||
features are **framework v2** = iteration 18 (spec APPROVED 2026-08-20,
|
||||
plan next): TTL cache, @table flags, durable job queue with
|
||||
drain-on-request, `transaction { }` over the WAL's staged batch. After 18,
|
||||
the order resumes at 20/21. Edges: [00-dependency-graph.md](00-dependency-graph.md).
|
||||
features are **framework v2** = iteration 18 (⏸ HELD 2026-08-21 with spec
|
||||
approved + plan authored intact): TTL cache, @table flags, durable job
|
||||
queue with drain-on-request, `transaction { }` over the WAL's staged
|
||||
batch. The pending order is now the concurrency chain (see *Pending*
|
||||
below). Edges: [00-dependency-graph.md](00-dependency-graph.md).
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -168,24 +169,26 @@ that sequences its tasks. Read one, approve, then the next starts.
|
|||
| 6 | [Program mode + stdlib](stories/language-runtime-database/done/06-program-mode-stdlib.md) | ✅ (the surface log-watcher uses) |
|
||||
| 7 | [log-watcher proof](stories/language-runtime-database/done/07-logwatcher-proof.md) | ✅ **landed 2026-08-15** — executable, not merely compilable: zero ASan leaks in all three modes, SIGTERM ends parked syscalls, fds flat, `LW_SOAK` gate; `just log-watcher` 7/0 |
|
||||
| 7b | [Inferred GC + mark-sweep](stories/language-runtime-database/done/07b-inferred-gc-mark-sweep.md) | ✅ **landed 2026-08-18** — `@gc` gone (WO-E104), GC-ness inferred, RC replaced by incremental mark-sweep, `.wob` v4; supersedes iteration 2's RC memory model |
|
||||
| 8 | [Shard-actor runtime](stories/language-runtime-database/refine/08-shard-actor-runtime.md) | ⬜ |
|
||||
| 8 | [Shard-actor runtime](stories/language-runtime-database/refine/08-shard-actor-runtime.md) | 🔄 arc stages 1+2 landed 2026-08-20 (branch concurrency-arc); stage 3 (transparent DB actor) = **first in the concurrency chain** |
|
||||
| 9 | [Database engine](stories/language-runtime-database/done/09-database-engine.md) | 🔄 engine complete (storage/WAL/indexes/insert-update-delete); reads land with 9b |
|
||||
| 9b | [`@table`, relations, query](stories/language-runtime-database/done/09b-table-relations-query.md) | 🔄 query surface + relations + FK done (branch query-surface); group-by parked |
|
||||
| 19 | [Float + Bytes](stories/language-runtime-database/done/19-missing-scalar-types.md) | ✅ **landed 2026-08-20** — `.wob` v5: Float constant tag, field kinds 6/7, opcodes 34-41 (IEEE-quiet f64), builtins 70-83. Full stack: literals, arithmetic, `@table` column, WAL bit-exact replay, json fractions in / shortest-round-trip out, `?Float` reserved-NaN nil, total-order index (NaN last, `-0.0` == `+0.0`), Bytes + base64. No implicit Int/Float mixing (WO-E201); `float`/`trunc` are the only bridges. Proof: web-app price is a real Float (`{"price":9.99}`), `just web-app` 23/0; corpus 103/0 |
|
||||
| 20 | [Cross-program tables](stories/language-runtime-database/refine/20-cross-program-tables.md) | 🔄 channel done (branch ipc-attach); manifest+binding pending |
|
||||
| 21 | [Keypair attach auth](stories/language-runtime-database/refine/21-keypair-attach-auth.md) | 🔄 crypto+handshake done (branch keypair-auth); manifest pending |
|
||||
| 22 | [Durability, throughput, scale](stories/language-runtime-database/refine/22-durability-throughput-scale.md) | ⬜ needs a spec first |
|
||||
| 23 | [io_uring group-commit](stories/language-runtime-database/refine/23-io-uring-commit.md) | ⬜ after 8 + 22 |
|
||||
| 27 | [Query grammar corpus](stories/language-runtime-database/refine/27-query-grammar-corpus.md) | ⬜ needs a spec first |
|
||||
| 10 | [HTTP service layer](stories/language-runtime-database/25-http-service.md) | ⬜ | Hold |
|
||||
| 11 | [Fibers](stories/language-runtime-database/refine/11-fibers.md) | ⬜ | Hold |
|
||||
| 12 | [Blue-green deploy](stories/language-runtime-database/26-blue-green-deploy.md) | ⬜ | Hold |
|
||||
| 13 | [Compile-time metaprogramming](stories/language-runtime-database/refine/29-compile-time-metaprogramming.md) | ⬜ needs a spec first |
|
||||
| 14 | [skillhost host workload](stories/language-runtime-database/refine/28-skillhost-host-workload.md) | ⬜ gaps recorded (branch query-grammar found skillhost needs no new query grammar); each gap a candidate iteration |
|
||||
| 11 | [Fibers](stories/language-runtime-database/refine/11-fibers.md) | 🔄 stages 1+2 landed 2026-08-20 with 8 (`just fibers` 8/0); closes with the arc's stage 3 |
|
||||
| 22 | [Durability, throughput, scale](stories/language-runtime-database/refine/22-durability-throughput-scale.md) | ⬜ needs a spec first — second in chain, after arc stage 3 |
|
||||
| 31 | [Actor lifecycle](stories/language-runtime-database/refine/31-actor-lifecycle.md) | ⬜ needs a spec first — third in chain (story written 2026-08-21) |
|
||||
| 24 | [chat: WebSocket workload](stories/language-runtime-database/refine/24-chat-websocket-workload.md) | ⬜ fourth in chain — the arc's acceptance; after 31 |
|
||||
| 23 | [io_uring group-commit](stories/language-runtime-database/refine/23-io-uring-commit.md) | ⬜ last in chain, after stage 3 + 22 |
|
||||
| 20 | [Cross-program tables](stories/language-runtime-database/hold/20-cross-program-tables.md) | ⏸ hold (2026-08-21); channel done (branch ipc-attach keeps its manifest) |
|
||||
| 21 | [Keypair attach auth](stories/language-runtime-database/hold/21-keypair-attach-auth.md) | ⏸ hold (2026-08-21); crypto+handshake done (branch keypair-auth keeps its manifest) |
|
||||
| 25 | [HTTP service layer](superpowers/plans/2026-08-01-http-service-layer.md) | ⏸ hold (2026-08-21) — story file removed; the plan doc remains |
|
||||
| 26 | [Blue-green deploy](stories/language-runtime-database/hold/26-blue-green-deploy.md) | ⏸ hold (2026-08-21) |
|
||||
| 27 | [Query grammar corpus](stories/language-runtime-database/hold/27-query-grammar-corpus.md) | ⏸ hold (2026-08-21) |
|
||||
| 28 | [skillhost host workload](stories/language-runtime-database/hold/28-skillhost-host-workload.md) | ⏸ hold (2026-08-21); gaps recorded (branch query-grammar found skillhost needs no new query grammar) |
|
||||
| 29 | [Compile-time metaprogramming](stories/language-runtime-database/hold/29-compile-time-metaprogramming.md) | ⏸ hold (2026-08-21) |
|
||||
| 15 | [deps: `wo.toml [deps]`](stories/language-runtime-database/done/15-deps-package-manager.md) | ✅ **landed 2026-08-18** (branch web-framework): [deps] inline tables, git-binary fetch, wo.lock pinning, offline-when-locked, --update-deps, WO-E106/E107; `just deps-accept` 8/0 |
|
||||
| 16 | [web framework](stories/language-runtime-database/done/16-web-framework.md) | ✅ **landed 2026-08-19** — writeonce-framework (HTTP/1.1 + router + Handler/Middleware) consumed by web-app through [deps]; h2c parked (§C) behind 8/23/11. **v1 polish landed 2026-08-20** (branch framework-v1): get/post/put/delete_ helpers, 405+Allow, HEAD, Logging middleware, set_header; `just web-app` 16/0; fixed the interp-borrowed-field emitter crash en route. **Auth-in-core landed 2026-08-20**: http/auth.wo (Bearer/Basic, ct_eq, req.principal), web-app dogfoods BearerAuth, gate 17/0 |
|
||||
| 17 | [library projects + `internal/`](stories/language-runtime-database/done/17-library-projects-internal.md) | ✅ **landed 2026-08-20** — `kind = "library"` in `wo.toml` (default `program`, so every existing manifest is byte-identical; unknown value = WO-E109 exit 2); `woc <dir>` on a library runs the FULL pipeline entry-less and writes nothing, retiring iteration 16's `--emit` workaround; the no-entry build error names the kind; lib+bin dual works. Go's `internal/` rule as **WO-E108** at the consumer's own `use`, dep-boundary-only — the library imports its own interior freely. Framework reorganized: `internal/{parse,serve}.wo` behind the line, `http/form.wo` split out to keep `media_type`/`form_values` public. Driver-only change; VM/`.wob`/GC untouched. `just web-app` **26/0** (3 new checks), every standing gate unchanged |
|
||||
| 18 | [framework v2: memory-rich features](stories/language-runtime-database/hold/18-memory-db-features.md) | 🔄 **spec APPROVED 2026-08-20, plan next** ([spec](superpowers/specs/2026-08-20-memory-db-features-design.md)): TTL cache + @table flags + durable job queue (drain-on-request) + `transaction { }` over the WAL's staged batch; pub/sub REJECTED until 8/11 |
|
||||
| 18 | [framework v2: memory-rich features](stories/language-runtime-database/hold/18-memory-db-features.md) | ⏸ hold (2026-08-21); spec approved + plan authored, both held intact ([spec](superpowers/specs/2026-08-20-memory-db-features-design.md), [plan](superpowers/plans/2026-08-20-framework-v2-memory-features.md)): TTL cache + @table flags + durable job queue (drain-on-request) + `transaction { }` over the WAL's staged batch; pub/sub REJECTED until 8/11 |
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -360,53 +363,54 @@ The C proving-ground work (`exploration/c-runtime/`, phases A–F: 859k reads/s,
|
|||
|
||||
## Pending
|
||||
|
||||
### Implementation order (re-sequenced 2026-08-20 — code-review pass)
|
||||
### Implementation order (re-sequenced 2026-08-21 — concurrency chain)
|
||||
|
||||
Replaces the framework-goal ordering. Basis: the verified findings in
|
||||
[`00-code-review.md`](00-code-review.md) — measure before optimizing, close
|
||||
correctness holes before adding surface, stop stacking features on
|
||||
unmeasured ground. IDs below are post-renumber; the authoritative table with
|
||||
per-row reasoning is
|
||||
[`00-story.md`](stories/language-runtime-database/00-story.md).
|
||||
Everything still pending IS the runtime-concurrency chain. Basis: the
|
||||
2026-08-20 code-review pass (measure before optimizing, close correctness
|
||||
holes before adding surface), amended 2026-08-21 by developer decision:
|
||||
**stage 3 before 22** — correctness first, then one benchmark campaign
|
||||
covers single- and multi-shard. The authoritative table with per-row
|
||||
reasoning is [`00-story.md`](stories/language-runtime-database/00-story.md).
|
||||
|
||||
Dependency rules that still force the shape: 23 explicitly after 8 + 22;
|
||||
21's plan folds into 20's; 26 only after 9 + 25; 11 rides 8's shard
|
||||
scheduler; h2c parked behind 8/23/11.
|
||||
Dependency rules that force the shape: 23 after stage 3 + 22 (the ring is
|
||||
the arc's, the baseline is 22's); 24 after 31 (chat is dishonest without
|
||||
lifecycle); h2c stays parked behind the chain; the held tail keeps its own
|
||||
precedence notes for resumption.
|
||||
|
||||
1. **22** — the measurement backbone, and now first: it has never run, so
|
||||
every performance claim on this project is unsourced. No
|
||||
`bench/baseline.json`, no `just db-bench`; `runtime/bench/` is the
|
||||
retired C prototype's harness.
|
||||
2. **8+11 stage 3** — transparent DB RPC, then 22 re-run for the
|
||||
concurrency delta. Reframed as a correctness fix: worker VMs are
|
||||
zero-initialized, so a DB statement off the primary traps `WO_T_DB`.
|
||||
A multi-shard program that touches the database is broken today.
|
||||
3. **30** (new) — observability, CI, fuzz: runtime counters + a profiler
|
||||
hook, 22's harness run per change instead of by hand, a fuzz target on
|
||||
the parser and `.wob` loader. No iteration covered any of this.
|
||||
4. **19** — Float + Bytes; small, and it gates 24 (WS frames) and the
|
||||
crypto fork (digests).
|
||||
5. **31** (new) — actor lifecycle: request/response (`send` is one-way and
|
||||
callers `sleep` to await), bounded mailboxes (the FIFO only grows),
|
||||
actor death/supervision, timers beyond `time.sleep`.
|
||||
6. **24** — chat, the arc's acceptance; honest only after 19 + 31.
|
||||
7. **23** — io_uring group-commit; explicitly after 8 + 22.
|
||||
8. **25** — HTTP service layer; `service` blocks lower onto the framework
|
||||
instead of a parallel stack.
|
||||
9. **18** — framework v2 (transaction{} + cache/flags/jobs); spec APPROVED
|
||||
2026-08-20 but **demoted from first**: more surface on a framework with
|
||||
one consumer, and its cache stores `Text` because there are no generics.
|
||||
10. **27, then 26** — query grammar from corpora (likely collapses to
|
||||
"confirm `len(query)` + add `exists`"), then blue-green.
|
||||
11. **20 then 21** — demoted hard: new distribution surface while there is
|
||||
no TLS, no crypto primitives, and the multi-shard DB still traps. The
|
||||
half-done branches (ipc-attach, keypair-auth) keep their manifests.
|
||||
12. **28, then 29 + parked drain** — skillhost is no longer the driving
|
||||
workload; then metaprogramming (spec first), group-by, ADT roster,
|
||||
WO-E225, held by the 2026-08-08 scope directive.
|
||||
1. **8+11 stage 3** — transparent DB actor. A correctness fix, not an
|
||||
optimization: worker VMs are zero-initialized, so a DB statement off
|
||||
the primary traps `WO_T_DB` — a multi-shard program touching the
|
||||
database is broken today. Plan of record:
|
||||
[`2026-08-20-shard-fiber-arc.md`](superpowers/plans/2026-08-20-shard-fiber-arc.md)
|
||||
(stages 1+2 landed 2026-08-20, branch `concurrency-arc`).
|
||||
2. **22** — the measurement backbone: restart-persistence proof + baseline
|
||||
benchmark (durable + RAM-only), single- AND multi-shard in one
|
||||
campaign, plus the stage-2 mutex-inbox number (rings only if the mutex
|
||||
costs). It has never run — no `bench/baseline.json`, no `just db-bench`;
|
||||
the arc's stages 1+2 delta is recorded retroactively.
|
||||
3. **31** — actor lifecycle
|
||||
([story](stories/language-runtime-database/refine/31-actor-lifecycle.md),
|
||||
written 2026-08-21): request/response (`send` is one-way and callers
|
||||
`sleep` to await), bounded mailboxes (the FIFO only grows), actor
|
||||
death/supervision, timers beyond `time.sleep`.
|
||||
4. **24** — chat, the arc's acceptance; honest only after 31 (19 landed
|
||||
2026-08-20 — Bytes carries the frames).
|
||||
5. **23** — io_uring group-commit; the WAL's WRITE+FSYNC chains ride the
|
||||
arc's per-shard ring (T4); after 22's baseline — the payoff, measured.
|
||||
|
||||
**30** — observability, CI, fuzz: named 2026-08-20, still row-only (no
|
||||
story file); slots in when scheduled — nothing in the chain depends on it.
|
||||
|
||||
⏸ **Held** (2026-08-21, developer decision): 18, 20, 21, 25, 26, 27, 28,
|
||||
29 — stories in
|
||||
[`hold/`](stories/language-runtime-database/hold/) (25's story file
|
||||
removed; its [plan doc](superpowers/plans/2026-08-01-http-service-layer.md)
|
||||
remains). Half-done branches (ipc-attach, keypair-auth) keep their
|
||||
manifests.
|
||||
|
||||
✅ **17** — landed 2026-08-20 (unparked and executed): `kind = "library"`,
|
||||
check mode, and the `internal/` dep boundary (WO-E108). Driver-only.
|
||||
✅ **19** — landed 2026-08-20: Float + Bytes, `.wob` v5.
|
||||
|
||||
### Language track — sequenced, on the critical path
|
||||
|
||||
|
|
|
|||
|
|
@ -71,6 +71,20 @@ adding surface, and stop stacking features on unmeasured ground.
|
|||
or timers behind `spawn`/`send`.
|
||||
- **18, 20, 21 demoted.** All three add surface; none answer a named gap.
|
||||
|
||||
RE-SEQUENCED 2026-08-21 (third pass — the concurrency chain). Everything
|
||||
still pending IS the runtime-concurrency chain; order:
|
||||
**stage 3 → 22 → 31 → 24 → 23**. Changes from the second pass:
|
||||
|
||||
- **The arc's stage 3 moves ahead of 22** — correctness before
|
||||
measurement: a multi-shard program touching the database traps
|
||||
`WO_T_DB` today, and fixing that first lets one benchmark campaign
|
||||
cover single- and multi-shard honestly.
|
||||
- **31 has its story file** ([refine/31-actor-lifecycle.md](refine/31-actor-lifecycle.md));
|
||||
30 stays a row until it is scheduled.
|
||||
- **Holds landed** (developer decision, 2026-08-21): 18, 20, 21, 26, 27,
|
||||
28, 29 moved to `hold/`; 25's story file removed (its plan doc remains
|
||||
in superpowers). 19 landed 2026-08-20.
|
||||
|
||||
| Seq | # | Iteration | Delivers |
|
||||
| --- | --- | --- | --- |
|
||||
| 1 | 1 | [Principles doc](done/01-principles-doc.md) | `docs/00-principles.md` — the doctrine page every later slice links back to |
|
||||
|
|
@ -85,21 +99,21 @@ adding surface, and stop stacking features on unmeasured ground.
|
|||
| 10 | 9b | [`@table`, relations, query](done/09b-table-relations-query.md) | `@table` real storage; `ref`/`backlink`/`multi`; compiler-checked queries |
|
||||
| 11 | 15 | [deps: `wo.toml [deps]`](done/15-deps-package-manager.md) | exact-rev git deps + `wo.lock` + `.wo-deps`; flat-only, offline once locked |
|
||||
| 12 | 16 | [web framework](done/16-web-framework.md) | the `.wo` framework v1 (router, middleware, auth, all three body hooks) consumed via `[deps]` |
|
||||
| 13 | 22 | [Durability, throughput, scale](refine/22-durability-throughput-scale.md) | restart-persistence proof, benchmarks, ~1M rows — the baseline the arc and 23 sign against. **Promoted to first pending (was seq 18)**: it has never run, so every performance claim on this project is currently unsourced. *(was 9e)* |
|
||||
| 14 | 8+11 | [Shard-actor runtime](refine/08-shard-actor-runtime.md) · [Fibers](refine/11-fibers.md) | THE ARC (stages 1+2 landed 2026-08-20: fibers/budget/actors/io_uring plane; pinned shards, envelope sends, home-routed frees, WO-E222); stage 3 = transparent DB RPC + 22 re-run. **Stage 3 is a correctness hole, not an optimization**: worker shards are zero-initialized, so a DB statement off the primary traps `WO_T_DB`. |
|
||||
| 13 | 8+11 | [Shard-actor runtime](refine/08-shard-actor-runtime.md) · [Fibers](refine/11-fibers.md) | THE ARC (stages 1+2 landed 2026-08-20: fibers/budget/actors/io_uring plane; pinned shards, envelope sends, home-routed frees, WO-E222); stage 3 = transparent DB RPC. **Promoted above 22 (2026-08-21): stage 3 is a correctness hole, not an optimization** — worker shards are zero-initialized, so a DB statement off the primary traps `WO_T_DB`. |
|
||||
| 14 | 22 | [Durability, throughput, scale](refine/22-durability-throughput-scale.md) | restart-persistence proof, benchmarks, ~1M rows — the baseline the arc and 23 sign against. It has never run, so every performance claim on this project is currently unsourced; runs after stage 3 so one campaign covers single- and multi-shard, plus the stage-2 mutex-inbox number. *(was 9e)* |
|
||||
| 15 | 30 | Observability, CI, fuzz *(no story file yet)* | **NEW** — runtime counters + a profiler hook, 22's harness wired to run per change instead of by hand, and a fuzz target on the parser and `.wob` loader. The whole proof-maturity gap had no iteration to point at. |
|
||||
| 16 | 19 | [Float + Bytes](done/19-missing-scalar-types.md) | **LANDED 2026-08-20** — `.wob` v5; the full stack: IEEE-quiet f64 through literals/VM/@table/WAL/json + Bytes as the binary carrier, no implicit mixing, total-order indexes. Unblocks 24 (WS frames) and the crypto fork (digests). *(was 20)* |
|
||||
| 17 | 31 | Actor lifecycle *(no story file yet)* | **NEW** — request/response (today `send` is one-way and callers `sleep` to await), bounded mailboxes with backpressure (today the FIFO just grows), actor death/supervision, and timers beyond `time.sleep`. 24 cannot be written honestly without these. |
|
||||
| 17 | 31 | [Actor lifecycle](refine/31-actor-lifecycle.md) | request/response (today `send` is one-way and callers `sleep` to await), bounded mailboxes with backpressure (today the FIFO just grows), actor death/supervision, and timers beyond `time.sleep`. 24 cannot be written honestly without these. *(story written 2026-08-21)* |
|
||||
| 18 | 24 | [chat: WebSocket workload](refine/24-chat-websocket-workload.md) | the arc's acceptance: WS upgrade + frames (SHA-1 via crypto fork, Bytes via 19), rooms/broadcast, 1k clients, drain-clean. *(was 19)* |
|
||||
| 19 | 23 | [io_uring group-commit](refine/23-io-uring-commit.md) | WAL WRITE+FSYNC chains on the arc's per-shard rings; fsync fallback kept (after 22 + the arc). *(was 9f)* |
|
||||
| 20 | 25 | [HTTP service layer](25-http-service.md) | `service` blocks lower onto the framework (after 9b + 20 by their own precedence notes). *(was 10)* |
|
||||
| 20 | 25 | [HTTP service layer](../../superpowers/plans/2026-08-01-http-service-layer.md) | `service` blocks lower onto the framework (after 9b + 20 by their own precedence notes). **HELD 2026-08-21** — story file removed; the plan doc remains. *(was 10)* |
|
||||
| 21 | 18 | [framework v2: memory-rich features](hold/18-memory-db-features.md) | spec+plan approved: TTL cache, @table flags, durable job queue, `transaction { }` over the WAL's staged batch. **Demoted from seq 14**: more surface on a framework with one consumer, and the cache still stores `Text` because there are no generics |
|
||||
| 22 | 27 | [Query grammar corpus](refine/27-query-grammar-corpus.md) | grow the query grammar from real corpora; likely collapses to "confirm `len(query)` + add `exists`"; precedes 28. *(was 9g)* |
|
||||
| 23 | 26 | [Blue-green deploy](26-blue-green-deploy.md) | two VM slots, in-runtime compile, atomic switch, resident rollback (plan authored after 9 + 25). *(was 12)* |
|
||||
| 24 | 20 | [Cross-program tables](refine/20-cross-program-tables.md) | attach to a running program's database over local IPC; owner stays the single writer (channel half-built). **Demoted from seq 16**: new distribution surface while there is no TLS, no crypto, and the multi-shard DB still traps. *(was 9c)* |
|
||||
| 25 | 21 | [Keypair attach auth](refine/21-keypair-attach-auth.md) | program identity is a keypair; mutual challenge–response at attach (crypto half-built; plan folds into 20's). **Demoted with 20** — and it needs crypto primitives that do not exist. *(was 9d)* |
|
||||
| 26 | 28 | [skillhost host workload](refine/28-skillhost-host-workload.md) | host-shaped driving workload naming runtime gaps — demoted with the framework goal. *(was 14)* |
|
||||
| 27 | 29 | [Compile-time metaprogramming](refine/29-compile-time-metaprogramming.md) | `@derive(...)` from class-table metadata; held with the parked drain by the 2026-08-08 scope directive. *(was 13)* |
|
||||
| 22 | 27 | [Query grammar corpus](hold/27-query-grammar-corpus.md) | grow the query grammar from real corpora; likely collapses to "confirm `len(query)` + add `exists`"; precedes 28. *(was 9g)* |
|
||||
| 23 | 26 | [Blue-green deploy](hold/26-blue-green-deploy.md) | two VM slots, in-runtime compile, atomic switch, resident rollback (plan authored after 9 + 25). *(was 12)* |
|
||||
| 24 | 20 | [Cross-program tables](hold/20-cross-program-tables.md) | attach to a running program's database over local IPC; owner stays the single writer (channel half-built). **Demoted from seq 16**: new distribution surface while there is no TLS, no crypto, and the multi-shard DB still traps. *(was 9c)* |
|
||||
| 25 | 21 | [Keypair attach auth](hold/21-keypair-attach-auth.md) | program identity is a keypair; mutual challenge–response at attach (crypto half-built; plan folds into 20's). **Demoted with 20** — and it needs crypto primitives that do not exist. *(was 9d)* |
|
||||
| 26 | 28 | [skillhost host workload](hold/28-skillhost-host-workload.md) | host-shaped driving workload naming runtime gaps — demoted with the framework goal. *(was 14)* |
|
||||
| 27 | 29 | [Compile-time metaprogramming](hold/29-compile-time-metaprogramming.md) | `@derive(...)` from class-table metadata; held with the parked drain by the 2026-08-08 scope directive. *(was 13)* |
|
||||
| ✅ | 17 | [library projects + `internal/`](done/17-library-projects-internal.md) | **LANDED 2026-08-20** — `kind = "library"` + entry-less check mode (retires the `--emit` workaround) and Go's `internal/` rule as WO-E108 at the consumer's `use`; driver-only, VM/GC untouched. `just web-app` 26/0 |
|
||||
|
||||
|
||||
|
|
@ -131,11 +145,9 @@ list, and a pointer to the plan document that already sequences its tasks.
|
|||
nothing MVC-scale — and iteration 17 (library kind + `internal/`) is
|
||||
**parked** with spec + plan ready on branch `library-internal`. The
|
||||
v1 slices landed 2026-08-20 (`just web-app` 21/0 after polish, auth,
|
||||
form, multipart). Implementation order for everything still pending:
|
||||
**18 (spec approved)** → 20/21 → 22 → 8 → 23 → 11 (+ h2c unparks) →
|
||||
10 → 12 → 27 → 14 → 13 + parked drain; 17 parked, slots anywhere after
|
||||
16 on directive. The iterations table above carries this order
|
||||
row-by-row; edges live in
|
||||
form, multipart). The implementation-order list this note once carried
|
||||
is superseded — the iterations table above is the authority
|
||||
(re-sequenced 2026-08-20 twice, then 2026-08-21); edges live in
|
||||
[`docs/00-dependency-graph.md`](../../00-dependency-graph.md).
|
||||
- **Iteration 7b (inserted 2026-08-11)** sits after the critical path
|
||||
deliberately: it delays nothing on the log-watcher line, and it must precede
|
||||
|
|
|
|||
|
|
@ -10,8 +10,18 @@
|
|||
> workload is **iteration 24 (chat: WebSocket pub/sub)**. The pre-existing
|
||||
> plan (`docs/superpowers/plans/2026-08-01-shard-actor-vm-runtime.md`)
|
||||
> predates inferred GC (7b), the unified surface, and the DB decision —
|
||||
> it is a source of ideas, NOT the plan of record; the arc starts with a
|
||||
> fresh brainstorm → spec → plan.
|
||||
> it is a source of ideas, NOT the plan of record.
|
||||
>
|
||||
> **RE-SEQUENCED 2026-08-21** (developer decision): the brainstorm →
|
||||
> spec → plan happened. The arc's plan of record is
|
||||
> [`2026-08-20-shard-fiber-arc.md`](../../../superpowers/plans/2026-08-20-shard-fiber-arc.md)
|
||||
> (spec: [`2026-08-20-shard-fiber-arc-design.md`](../../../superpowers/specs/2026-08-20-shard-fiber-arc-design.md)),
|
||||
> and **stages 1+2 LANDED 2026-08-20** on branch `concurrency-arc`
|
||||
> (T1–T6, seven disclosed deviations recorded in the plan). Remaining
|
||||
> scope: **stage 3, the transparent DB actor** — a correctness fix, not
|
||||
> an optimization: worker VMs are zero-initialized, so a DB statement
|
||||
> off the primary shard traps `WO_T_DB`. Concurrency-chain order:
|
||||
> **stage 3 → 22 → 31 → 24 → 23**.
|
||||
|
||||
## Settled decisions (2026-08-20)
|
||||
|
||||
|
|
@ -33,8 +43,11 @@
|
|||
3. **Driving workload: chat** (iteration 24) — rooms, broadcast, N
|
||||
concurrent WebSocket clients, one binary. The arc's acceptance is the
|
||||
chat sample's, not only synthetic corpora.
|
||||
4. **Order: 22 → the 8+11 arc → 23.** The io_uring write path waits for
|
||||
the arc (its batch boundary is the shard tick) and for 22's baseline.
|
||||
4. **Order — SUPERSEDED 2026-08-21.** Originally 22 → the 8+11 arc → 23;
|
||||
in fact the arc's stages 1+2 landed before 22 ever ran (accepted
|
||||
deviation — the before/after delta is owed and lands as 22's
|
||||
multi-shard pass). Current order: **stage 3 → 22 → 31 → 24 → 23**.
|
||||
23 still waits for the arc's tick boundary and 22's baseline.
|
||||
|
||||
## Goals
|
||||
|
||||
|
|
@ -102,9 +115,12 @@
|
|||
|
||||
## Proposed Solution
|
||||
|
||||
Fresh brainstorm → spec → plan for the WHOLE arc (8+11), staged: the
|
||||
pinned-worker scheduler + shard-stamped heaps + MPSC mailboxes + the
|
||||
unified spawn/send surface and the traced-send rejection; fibers on that
|
||||
scheduler (part 2's document); the DB-actor migration; then iteration 24
|
||||
proves it. The 2026-08-01 plan is reference material for the mailbox and
|
||||
heap-stamping shapes only.
|
||||
Execute stage 3 of the plan of record
|
||||
([`2026-08-20-shard-fiber-arc.md`](../../../superpowers/plans/2026-08-20-shard-fiber-arc.md)):
|
||||
the DB-actor migration — engine calls off the owner shard become message
|
||||
sends with parked replies, closing the `WO_T_DB` hole. Stages 1+2
|
||||
(scheduler, fibers, unified spawn/send, WO-E222 traced-send rejection,
|
||||
cross-shard envelopes with home-routed frees) landed 2026-08-20. After
|
||||
stage 3: 22 measures, then iteration 24 proves the arc. Actor lifecycle
|
||||
(request/response, backpressure, supervision, timers) is deliberately
|
||||
NOT the arc's scope — it is [iteration 31](31-actor-lifecycle.md).
|
||||
|
|
|
|||
|
|
@ -20,9 +20,21 @@
|
|||
> time delivery, main-return reap, fiber-trap isolation, and parked
|
||||
> `net`/`time` builtins on the io_uring-first per-shard I/O plane
|
||||
> (`WO_IO=uring|epoll`, epoll fallback proven). Demonstrated by
|
||||
> `docs/examples/fibers` (`just fibers` 8/0). The shard-context criteria
|
||||
> (TID assertions, cross-shard sends, blue-green drain reuse) close with
|
||||
> the arc's stage 2.
|
||||
> `docs/examples/fibers` (`just fibers` 8/0).
|
||||
>
|
||||
> **STAGE-2 SUBSTANCE LANDED 2026-08-20** (branch `concurrency-arc`,
|
||||
> T5–T6): pinned thread-per-core shards, envelope sends with ownership
|
||||
> move, round-robin placement, home-routed frees, WO-E222 on EVERY
|
||||
> spawn/send (placement makes any actor potentially remote), TID-verified
|
||||
> shard context. Deviations disclosed in the plan of record
|
||||
> ([`2026-08-20-shard-fiber-arc.md`](../../../superpowers/plans/2026-08-20-shard-fiber-arc.md)):
|
||||
> the inbox is a mutex-guarded list + eventfd (rings arrive only if
|
||||
> iteration 22 measures the mutex as a cost), the deterministic corpus
|
||||
> pins `WO_SHARDS=1`, two TSan races and one teardown SEGV fixed.
|
||||
>
|
||||
> **RE-SEQUENCED 2026-08-21**: what remains for the chain is the arc's
|
||||
> stage 3 (transparent DB actor — [iteration 8](08-shard-actor-runtime.md)),
|
||||
> then measurement. Order: **stage 3 → 22 → 31 → 24 → 23**.
|
||||
|
||||
## Goals
|
||||
|
||||
|
|
@ -90,15 +102,20 @@
|
|||
matches the stdlib posture).
|
||||
- Vision origin: [blue-green vision §3](../../../plan/exploration/blue-green-vm/00-vision.md);
|
||||
iteration 8's scheduler is the substrate this extends.
|
||||
- Open questions to settle in the spec — REDUCED 2026-08-20: the spawn
|
||||
surface is settled (the unified actor address, iteration 8 decision 2).
|
||||
Still open for the arc's spec: budget size and check granularity,
|
||||
parked-fiber drop semantics, run-queue fairness (FIFO v1), and how a
|
||||
parked fiber's borrow state interacts with the shard's GC safepoints.
|
||||
- Open questions — REDUCED AGAIN 2026-08-21: the spawn surface, budget
|
||||
size/granularity (back-edge accounting — see the plan's livelock
|
||||
deviation), run-queue fairness (FIFO), and parked-fiber drop semantics
|
||||
(fiber-trap isolation + main-return reap) are all settled by the landed
|
||||
implementation. Still open: how a parked fiber's borrow state interacts
|
||||
with the shard's GC safepoints — tracked for stage 3 / the collector's
|
||||
next pass. Lifecycle surface (request/response, backpressure,
|
||||
supervision, timers) is [iteration 31](31-actor-lifecycle.md)'s, not
|
||||
this story's.
|
||||
|
||||
## Proposed Solution
|
||||
|
||||
- No implementation plan exists yet — this iteration starts with the
|
||||
brainstorming → spec → writing-plans chain (the superpowers path every
|
||||
prior iteration followed), then executes that plan. The research note
|
||||
above is the brainstorm's entry material.
|
||||
- The plan exists and its fiber stages are landed:
|
||||
[`2026-08-20-shard-fiber-arc.md`](../../../superpowers/plans/2026-08-20-shard-fiber-arc.md)
|
||||
(stages 1+2 complete 2026-08-20). This story closes when the arc's
|
||||
stage 3 lands and iteration 22 records the delta; the chat workload
|
||||
([iteration 24](24-chat-websocket-workload.md)) is the proof.
|
||||
|
|
|
|||
|
|
@ -11,6 +11,17 @@
|
|||
> iteration's benchmark and showing the number moved the right way.
|
||||
>
|
||||
> **No spec exists yet.** The forks in *Info* are genuine decisions.
|
||||
>
|
||||
> **RE-SEQUENCED 2026-08-21** (developer decision): runs AFTER the arc's
|
||||
> stage 3 — the transparent DB actor is a correctness hole (a multi-shard
|
||||
> program touching the database traps `WO_T_DB` today), and fixing it
|
||||
> first lets ONE benchmark campaign cover single- and multi-shard
|
||||
> honestly. The arc's stages 1+2 landed 2026-08-20 unmeasured; their
|
||||
> delta is recorded retroactively against this iteration's first
|
||||
> baseline. New measurement target since stage 2: the mutex-guarded
|
||||
> inbox + eventfd (the plan's deviation — lock-free rings arrive only if
|
||||
> this number says the mutex costs). Chain order:
|
||||
> **stage 3 → 22 → 31 → 24 → 23**.
|
||||
|
||||
## Goals
|
||||
|
||||
|
|
@ -122,15 +133,18 @@ exists to close.
|
|||
- **Brainstorm the spec**, settling the four forks; then a plan whose first
|
||||
task is the harness and the baseline file, because nothing downstream means
|
||||
anything without them.
|
||||
- **Sequence the whole performance arc around this iteration:**
|
||||
1. 9b lands → employee compiles and runs → **22 restart-persistence** and
|
||||
**22 baseline benchmark** (single-thread, both durable and RAM-only).
|
||||
2. **7b** (inferred GC + mark-sweep) → re-run 22, record the delta (does
|
||||
tracing change the write path's tail latency?).
|
||||
3. **8** (shard-actor, thread-per-core) → re-run 22 at the connection/
|
||||
concurrency scale it unlocks, record the delta.
|
||||
4. **23** (io_uring group-commit) → re-run 22's durable write number, record
|
||||
the delta against the fsync-per-commit baseline — the payoff.
|
||||
- **Sequence the performance chain around this iteration** (rewritten
|
||||
2026-08-21 — the first version predated 7b and the arc landing first):
|
||||
1. Already landed unmeasured: 9b, **7b** (inferred GC + mark-sweep,
|
||||
2026-08-18), the arc's **stages 1+2** (fibers + shards, 2026-08-20).
|
||||
Their deltas are owed retroactively against the first baseline.
|
||||
2. Arc **stage 3** (transparent DB actor) lands → **22 runs**:
|
||||
restart-persistence proof + baseline benchmark, durable and
|
||||
RAM-only, single- AND multi-shard, plus the mutex-inbox number.
|
||||
3. **31** (actor lifecycle), then **24** (chat) → re-run the
|
||||
concurrency-facing numbers at the connection scale chat unlocks.
|
||||
4. **23** (io_uring group-commit) → re-run 22's durable write number,
|
||||
record the delta against the fsync-per-commit baseline — the payoff.
|
||||
- The benchmark harness and its baseline live under `bench/` (or the existing
|
||||
`runtime/bench/`), and `just` gets a `db-bench` recipe kept off the fast
|
||||
path, exactly like `log-watcher::soak`.
|
||||
|
|
|
|||
|
|
@ -22,8 +22,12 @@
|
|||
> accumulated; (4) startup auto-probe + an env override so CI proves both
|
||||
> paths on one kernel — AMENDED: the override is the arc-wide
|
||||
> `WO_IO=uring|epoll` (the arc's T4 owns the probe and the per-shard
|
||||
> ring; `WO_WAL_MODE` is subsumed). Position: AFTER the 8+11 arc (order
|
||||
> settled 2026-08-20: 9e → 8+11 → 9f). AMENDED 2026-08-20 (io_uring-first
|
||||
> ring; `WO_WAL_MODE` is subsumed). Position — RE-SEQUENCED 2026-08-21:
|
||||
> LAST in the concurrency chain, **stage 3 → 22 → 31 → 24 → 23**
|
||||
> (supersedes the 2026-08-20 old-id ordering "9e → 8+11 → 9f"); the
|
||||
> per-shard ring already exists (arc T4 landed 2026-08-20,
|
||||
> `WO_IO=uring|epoll`) — this iteration adds the WAL's WRITE+FSYNC
|
||||
> chains to it. AMENDED 2026-08-20 (io_uring-first
|
||||
> directive): the WAL's WRITE+FSYNC chains ride the SAME per-shard ring
|
||||
> T4 creates for fiber parking — one event loop per shard, readiness ops
|
||||
> and durability ops together, exactly the linux reference project's
|
||||
|
|
|
|||
|
|
@ -6,6 +6,12 @@
|
|||
> **Inserted 2026-08-20** (concurrency-chain refinement): the 8+11 arc's
|
||||
> driving workload, the role log-watcher played for iterations 3–7. Needs
|
||||
> its spec AFTER the arc's — it lands at the arc's end and proves it.
|
||||
>
|
||||
> **RE-SEQUENCED 2026-08-21**: fourth in the chain,
|
||||
> **stage 3 → 22 → 31 → 24 → 23** — chat cannot be written honestly
|
||||
> before [iteration 31](31-actor-lifecycle.md) (request/response,
|
||||
> bounded mailboxes, actor death, timers). Iteration 19 LANDED
|
||||
> 2026-08-20, so Bytes is available for frame parse/serialize.
|
||||
|
||||
## Why this iteration exists
|
||||
|
||||
|
|
@ -54,11 +60,15 @@ proxy — the proxy story extends to WS pass-through, documented).
|
|||
|
||||
## Info
|
||||
|
||||
- Dependencies: the 8+11 arc (fibers + cross-shard send), the crypto
|
||||
builtins fork (SHA-1 for the upgrade handshake — note: the ledger's
|
||||
crypto slice lists SHA-256/512; the WS handshake specifically needs
|
||||
SHA-1, so the builtin set must include it), and the framework's parse
|
||||
seam (upgrade is an HTTP request until it isn't).
|
||||
- Dependencies: the 8+11 arc (fibers + cross-shard send — stages 1+2
|
||||
landed 2026-08-20, stage 3 pending), [iteration 31](31-actor-lifecycle.md)
|
||||
(request/response, backpressure, death, timers — the mechanisms rooms
|
||||
and presence are made of), iteration 19 (LANDED 2026-08-20 — Bytes
|
||||
carries the frames), the crypto builtins fork (SHA-1 for the upgrade
|
||||
handshake — note: the ledger's crypto slice lists SHA-256/512; the WS
|
||||
handshake specifically needs SHA-1, so the builtin set must include
|
||||
it), and the framework's parse seam (upgrade is an HTTP request until
|
||||
it isn't).
|
||||
- Unparks on landing: the framework ledger's WebSocket/pub-sub rows and
|
||||
the iteration-18 rejection note ("pub/sub REJECTED until 8/11").
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,106 @@
|
|||
# Iteration 31 — actor lifecycle: request/response, backpressure, death, timers
|
||||
|
||||
> Format: fiberloom `product/story-iteration-template`. Part of
|
||||
> [Story — one language, one runtime, one database, one binary](../00-story.md).
|
||||
>
|
||||
> **Inserted 2026-08-21** (concurrency-chain re-sequence; the iteration
|
||||
> was named as "new 31" in the 2026-08-20 code-review re-sequence — this
|
||||
> is its story file). Third in the chain,
|
||||
> **stage 3 → 22 → 31 → 24 → 23**: chat
|
||||
> ([iteration 24](24-chat-websocket-workload.md)) cannot be written
|
||||
> honestly without these four mechanisms.
|
||||
|
||||
## Why this iteration exists
|
||||
|
||||
The arc's stages 1+2 shipped `spawn`/`send` mechanism without lifecycle:
|
||||
`send` is one-way and callers `sleep`-poll to await an answer; the
|
||||
mailbox FIFO grows without bound (a hot sender can exhaust a shard's
|
||||
memory); an actor that traps dies silently (nobody learns, nothing
|
||||
restarts, its mailbox rots); and there is no timer surface beyond a
|
||||
fiber blocking in `time.sleep`. Every real serving program — chat first —
|
||||
is made of request/response turns, bounded queues, death notices, and
|
||||
deadlines. Without this iteration the arc is a demo, not a runtime.
|
||||
|
||||
## Goals
|
||||
|
||||
- **Request/response over one-way sends.** A caller can send and park
|
||||
until the reply arrives — one surface, no `sleep`-polling, no second
|
||||
concurrency vocabulary. Ownership rules unchanged: the request moves,
|
||||
the reply moves back.
|
||||
- **Bounded mailboxes with a stated backpressure policy.** A mailbox has
|
||||
a cap; what happens at the cap (park the sender vs error) is decided by
|
||||
the spec, one policy, doctrine-pure — no silent unbounded growth
|
||||
anywhere in the runtime.
|
||||
- **Actor death is observable.** A trap or normal exit produces a signal
|
||||
another actor can receive; a fiber-trap already isolates (stage 1) —
|
||||
this makes the fact of death deliverable, so a supervisor CAN be
|
||||
written in `.wo`.
|
||||
- **Timers as messages.** A deadline or interval delivers to a mailbox
|
||||
like any other send, riding the shard's existing io_uring/epoll
|
||||
timeout plumbing (arc T4) — no new event loop.
|
||||
|
||||
## Acceptance Criteria (draft — the spec refines)
|
||||
|
||||
- What to achieve?
|
||||
- **Given** an actor that answers requests,
|
||||
- **when** a caller awaits the reply,
|
||||
- **then** the caller's fiber parks (the shard serves other fibers,
|
||||
TID-verified), resumes with the moved reply, and never busy-waits.
|
||||
- What to achieve?
|
||||
- **Given** a mailbox at its cap,
|
||||
- **when** another send arrives,
|
||||
- **then** the stated backpressure policy fires deterministically,
|
||||
memory stays bounded (RSS flat under a hot-sender soak), and no
|
||||
message is silently dropped.
|
||||
- What to achieve?
|
||||
- **Given** an actor that traps mid-message,
|
||||
- **when** it dies,
|
||||
- **then** its drop maps run (ASan zero leaks), a death signal
|
||||
reaches the observer that asked for one, and a supervisor written
|
||||
in `.wo` can respawn it.
|
||||
- What to achieve?
|
||||
- **Given** a timer armed by an actor,
|
||||
- **when** it fires,
|
||||
- **then** the actor receives it as an ordinary message on its own
|
||||
shard, and cancelling before expiry means it never delivers.
|
||||
|
||||
## Out Of Scope
|
||||
|
||||
- Supervision TREES / OTP-scale restart policy — a `.wo` library once
|
||||
death signals exist, not runtime policy.
|
||||
- Priorities and custom scheduling — the reduction budget stays the only
|
||||
fairness mechanism.
|
||||
- Distributed (cross-process) supervision — no network layer exists.
|
||||
- Changing the ownership-move rule or the unified address surface —
|
||||
iteration 8's decisions stand.
|
||||
|
||||
## Info
|
||||
|
||||
Forks the spec must settle:
|
||||
|
||||
1. **The request/response surface.** A reply-address baked into the
|
||||
message shape vs a runtime-level call that parks — and what the
|
||||
compiler checks (does a request type name its reply type?).
|
||||
2. **The backpressure policy at the cap.** Park the sender (natural with
|
||||
fibers, risks deadlock cycles) vs fail the send (explicit, pushes
|
||||
handling to the program). One policy, stated; not configurable per
|
||||
mailbox in v1.
|
||||
3. **The death-signal shape.** Erlang's link (bidirectional, dies
|
||||
together) vs monitor (one-way notice) — likely monitor-only v1.
|
||||
4. **The timer surface.** Builtin (`time.after` delivering a message) vs
|
||||
actor-spawned sleeper fiber — and cancellation semantics.
|
||||
|
||||
Sources: the 2026-08-20 code-review findings (the gaps this iteration
|
||||
answers), the arc plan's stage-2 deviations
|
||||
([`2026-08-20-shard-fiber-arc.md`](../../../superpowers/plans/2026-08-20-shard-fiber-arc.md)
|
||||
— the unbounded FIFO is deviation 4's mutex inbox), and BEAM precedent
|
||||
already surveyed in
|
||||
[`docs/plan/exploration/fibers/00-fibers.md`](../../../plan/exploration/fibers/00-fibers.md).
|
||||
|
||||
## Proposed Solution
|
||||
|
||||
Brainstorm → spec → plan after the arc's stage 3 lands and iteration 22
|
||||
has its baseline (the mailbox-cap and inbox-ring decisions want 22's
|
||||
mutex number). The spec is written against iteration 24's needs — chat
|
||||
names the lifecycle mechanisms it consumes, this iteration names chat as
|
||||
its first honest consumer.
|
||||
Loading…
Reference in a new issue