diff --git a/docs/00-dependency-graph.md b/docs/00-dependency-graph.md index e096bc1..e722942 100644 --- a/docs/00-dependency-graph.md +++ b/docs/00-dependency-graph.md @@ -59,7 +59,7 @@ flowchart TD I9g["27 query grammar corpus (⏸ hold; likely collapses)"]:::parked I30["30 observability, CI, fuzz — release-only CI exists; per-change gates + fuzz open (no story file)"]:::open GAPS["28's gap fan-out, what is LEFT of it: fs metadata, FFI-vs-out-of-process (bounded subprocess + stdio transport moved to 42)"]:::open - I42["42 bounded subprocess: bound proc.run (deadline, caps, fiber-parked), streaming form, owner-bound reaping"]:::open + I42["42 bounded subprocess ✅ 2026-09-01: proc.run bounded + parked (pidfd), proc.run_dl; streaming form deferred by name"]:::done DRAIN["parked drain, what is LEFT of it: WO-E225 roster, ADT roster, group-by aggregates"]:::parked FOUND --> I7 diff --git a/docs/stories/00-status.md b/docs/stories/00-status.md index 608fa6f..e692e44 100644 --- a/docs/stories/00-status.md +++ b/docs/stories/00-status.md @@ -67,6 +67,51 @@ behind this board; live Obsidian Dataview views: ## ▶ NEXT PLAN +### Landed 2026-09-01 — iteration 42, bounded subprocess (brainstorm to gate in one day) + +**Implemented last time (2026-09-01):** iteration +[42](language-runtime-database/42-bounded-subprocess.md) end to end — +`proc.run` reworked from shard-blocking to parked (pipe read ends + +pidfd behind one epoll fd, the `_dl` retry mould), bounds everywhere +(30 s / 1 MiB / 64 KiB defaults; per-shard ceiling 32; every violation +kills the child and traps `WO_T_IO` naming the bound), owner-bound +reaping (`fib_reap`/`wo_vm_destroy`/stop all sweep), and `proc.run_dl` +(id 96) stating bounds per call. New suite `runtime/test/test_proc.c` +(128 checks) and `docs/examples/subprocess` + `just subprocess` +(12 checks). [Spec](../superpowers/specs/2026-09-01-bounded-subprocess-design.md) +· [plan](../superpowers/plans/2026-09-01-bounded-subprocess.md). + +**Key findings (measured, not asserted):** the suspected drain deadlock +was REAL — a child writing 200 KB to stdout while holding stderr open +hung the old `proc.run` until the test's 5 s alarm (stdout silently +truncated at 8,192 bytes, exit code lost to SIGPIPE); the parked rework +answers the same child in 15 ms. A `ping` request was answered in 2 ms +while a `sleep 2` child was parked on the same shard. One thousand +sequential spawns left the fd table byte-flat. SIGTERM with a `sleep 30` +child live: clean exit 0, child pid verifiably gone from outside. + +**Learned:** a new sysio builtin id is THREE registrations, not one — +the wob.h enum, the loader's arity table, and builtin.c's dispatch +range; missing any of them surfaces as `unknown stdlib builtin` from a +perfectly valid image. And glibc 2.35 (the release build floor) has no +pidfd wrappers — raw `syscall(SYS_pidfd_open/…_send_signal)` or the +release build breaks. + +**Dependencies unblocked:** the streaming form (long-lived children, +output as mailbox messages) now has its registry/pidfd/cap machinery +built; the tmux/alacritty studies' stage A and the zen study's CDP +driver (stage C′) queue behind that plus their own named gaps +(PTY/termios/fd-passing; ws-client). Iteration 28's "bounded subprocess +first" ordering item is spent. + +**Next steps:** cherry-pick lang42 to master when declared ready; the +startable set otherwise unchanged. The exploration studies' next +builtin-sized item is the WebSocket client (zen C′). + +**`.dev/reference` used:** alacritty, tmux, zen-browser (the three +parity studies that promoted this gap to an iteration); the kernel's own +pidfd/epoll interfaces for the mechanics. + ### Landed 2026-08-30 — keys-resident delta updates DONE, loader refusal lifted **Implemented last time (2026-08-30):** the six-task @@ -1048,7 +1093,7 @@ check mode, and the `internal/` dep boundary (WO-E108). Driver-only. | 23 | io_uring group-commit write path — batched durability overlapped on shard threads, fsync fallback | **no spec yet** — brainstorm after iterations 8 + 22 | | 27 | Query grammar from real embedded-DB corpora — whole-query count + correlated exists, driven by the skillhost SQL catalogue; add only what a corpus uses | **no spec yet** — three forks; may collapse to "confirm len(query) + add exists" | | 14 | skillhost host workload — port skillhost (MCP host + confined script runner) to writeonce; drives the missing host capabilities into the open (bounded subprocess, stdin/stdout transport, fs metadata, FFI-vs-out-of-process) | **no spec yet** — gaps recorded in the iteration; each gap brainstormed on demand, bounded-subprocess first | -| 42 | [Bounded subprocess](language-runtime-database/42-bounded-subprocess.md) — `proc.run` exists (`sysio.c`) but shard-blocking, deadline-less, silently truncating; this bounds it in place (deadline, output caps, per-shard ceiling, owner-bound reaping via pidfd in the io_uring loop, fiber parked); streaming form deferred by name; 28's leading gap promoted with four consumers | ✅ spec approved 2026-09-01 — [spec](../superpowers/specs/2026-09-01-bounded-subprocess-design.md); `readiness: ready`, plan next | +| 42 | [Bounded subprocess](language-runtime-database/42-bounded-subprocess.md) — `proc.run` bounded in place (deadline, output caps, per-shard ceiling, owner-bound reaping via pidfd, fiber parked) + `proc.run_dl`; streaming form deferred by name | ✅ **DONE 2026-09-01** — [spec](../superpowers/specs/2026-09-01-bounded-subprocess-design.md) · [plan](../superpowers/plans/2026-09-01-bounded-subprocess.md); test_proc 128/0, `just subprocess` 12/0; see NEXT PLAN | | 17 | library projects + dependency privacy — `wo.toml` kind = "library" (checkable without entry, dual lib+bin) + Go-style `internal/` at the [deps] boundary; framework reorg demonstrates both | ✅ **landed 2026-08-20** — [spec](../superpowers/specs/2026-08-20-library-kind-internal-design.md) · [plan](../superpowers/plans/2026-08-20-library-kind-internal.md) | | 10 | HTTP service layer | [plan 6](../superpowers/plans/2026-08-01-http-service-layer.md) | | 11 | Fibers | vision §3, [blue-green exploration](../plan/exploration/blue-green-vm/00-vision.md) | diff --git a/docs/stories/language-runtime-database/42-bounded-subprocess.md b/docs/stories/language-runtime-database/42-bounded-subprocess.md index bb0dbc6..fc94ffa 100644 --- a/docs/stories/language-runtime-database/42-bounded-subprocess.md +++ b/docs/stories/language-runtime-database/42-bounded-subprocess.md @@ -1,7 +1,7 @@ --- track: language-runtime-database iteration: "42" -status: pending +status: done readiness: ready --- @@ -84,6 +84,20 @@ Bounds surface: bare `proc.run(cmd, args)` gets named defaults (30 s, exceeding any bound kills the child and raises a catchable error naming the bound — silent truncation is removed. +## Progress + +**DONE 2026-09-01, same day as the brainstorm.** Everything the spec +names landed: the parked rework (pidfd + epoll bundle + `wo_child` +registry in `sysio.c`/`vm.h`/`vm.c`), `proc.run_dl` end to end (wob.h id +96, loader arity row, builtin dispatch range, `types.ml` row — no +`emit.ml` change, as the net `_dl` precedent predicted), and the +suspected drain deadlock proven red against the old code (5.0 s hang to +the alarm, stdout truncated at 8192) before the rework dissolved it +(15 ms). Gates: `test_proc` 128/0 inside a fully green 19-suite ASan run, +woc-test 557/0, `just subprocess` 12/0 first run (ping answered in 2 ms +while a sleep-2 child was parked; SIGTERM left no child), `just site` +23/0 untouched. Mechanics written up in `runtime/src/CODE-LOGIC.md`. + ## Acceptance criteria — firmed in the spec, normative form there - Given a child that exits normally, when it is run, then its exit code is diff --git a/runtime/src/CODE-LOGIC.md b/runtime/src/CODE-LOGIC.md index 454c479..3e20f48 100644 --- a/runtime/src/CODE-LOGIC.md +++ b/runtime/src/CODE-LOGIC.md @@ -80,6 +80,39 @@ fills fields by index. The field order is therefore a contract, written beside each case in `sysio.c` and mirrored in `compiler/src/types.ml`'s predeclared records. Change one side and the other silently writes to the wrong slot. +## Bounded subprocess (iteration 42) + +`proc.run` (id 56) and `proc.run_dl` (id 96) share one case in `sysio.c`: +the first entry validates argv, forks with `execvp`, and claims a +`wo_child` slot in the shard's `wo_vm` (32 per shard — the concurrency +ceiling, failing closed by name). The two pipe read ends (parent side +`O_NONBLOCK`) and a pidfd for the child sit behind ONE `epoll` fd the +fiber parks on — the plane watches a single fd per fiber, and the bundle +turns three waits into it. The slot is the cross-park state (the `_dl` +retry protocol re-executes the builtin); `fb->proc_st` is how a re-entry +finds it. + +Every entry drains whatever is ready into growable buffers bounded by +the caps (defaults 1 MiB stdout / 64 KiB stderr; `run_dl` states them +per call), then `waitpid(WNOHANG)`: reaped means final-drain-and-answer; +alive means park with the deadline armed (`dl_active`/`dl_at`, default +30 000 ms). Any bound violation KILLS the child +(`pidfd_send_signal` — raw `syscall`, the glibc 2.35 build floor has no +wrappers), reaps, releases and traps `WO_T_IO` naming the bound and its +value. Silent truncation is gone — the pre-42 sequential drain also +deadlocked against a child that filled stdout past the old fixed cap +while holding stderr open (proven by `test_proc`'s chatty-child leg +before the rework). + +Ownership: a child belongs to the fiber that spawned it. `fib_reap` +calls `wo_proc_abandon` (a reaped fiber's child dies with it), +`wo_vm_destroy` calls `wo_proc_reap_all` (no child outlives its shard), +and a `stop_pending()` entry kills before returning `WO_SYS_STOPPED` — +iteration 40's drain guarantee extends to subprocesses. A zombie or an +orphan is a bug by definition; `test_proc` pins all of it (deadline, +caps, ceiling, thousand-spawn fd flatness, stop/unwind), and +`scripts/subprocess-accept.sh` proves the language-level half. + ## Class metadata and json (`.wob` v2) The class table carries, per field, its name constant, the class it refers to