From 8bcd24969eac6e228786682a387cf8350577954c Mon Sep 17 00:00:00 2001 From: "shoney.arickathil" Date: Tue, 1 Sep 2026 21:34:17 +0200 Subject: [PATCH] docs(lang42): story, spec and plan for bounded subprocess MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - claim the lang42 prefix; iteration 42 story (readiness: ready), approved spec, and the 11-task implementation plan - board: pending row for 42; graph: node 42 with green edges (11, 24) - graph: porch track section added (same sweep) - parity studies that motivated 42: alacritty, tmux, zen-browser under docs/plan/exploration/ — staged path, gap lists, refused routes Co-Authored-By: Claude Fable 5 --- docs/00-dependency-graph.md | 50 ++- docs/00-git-commit-history.md | 1 + .../alacritty/00-alacritty-parity.md | 123 +++++++ docs/plan/exploration/tmux/00-tmux-parity.md | 104 ++++++ .../zen-browser/00-zen-browser-parity.md | 86 +++++ docs/stories/00-status.md | 1 + .../42-bounded-subprocess.md | 126 +++++++ .../plans/2026-09-01-bounded-subprocess.md | 320 ++++++++++++++++++ .../2026-09-01-bounded-subprocess-design.md | 143 ++++++++ 9 files changed, 953 insertions(+), 1 deletion(-) create mode 100644 docs/plan/exploration/alacritty/00-alacritty-parity.md create mode 100644 docs/plan/exploration/tmux/00-tmux-parity.md create mode 100644 docs/plan/exploration/zen-browser/00-zen-browser-parity.md create mode 100644 docs/stories/language-runtime-database/42-bounded-subprocess.md create mode 100644 docs/superpowers/plans/2026-09-01-bounded-subprocess.md create mode 100644 docs/superpowers/specs/2026-09-01-bounded-subprocess-design.md diff --git a/docs/00-dependency-graph.md b/docs/00-dependency-graph.md index ca86ea3..e096bc1 100644 --- a/docs/00-dependency-graph.md +++ b/docs/00-dependency-graph.md @@ -58,7 +58,8 @@ flowchart TD I38["38 content platform capabilities: fs mutation verbs + net.connect"]:::open I9g["27 query grammar corpus (⏸ hold; likely collapses)"]:::parked I30["30 observability, CI, fuzz — release-only CI exists; per-change gates + fuzz open (no story file)"]:::open - GAPS["28's gap fan-out: bounded subprocess, stdin/stdout transport, fs metadata, FFI-vs-out-of-process"]:::open + GAPS["28's gap fan-out, what is LEFT of it: fs metadata, FFI-vs-out-of-process (bounded subprocess + stdio transport moved to 42)"]:::open + I42["42 bounded subprocess: bound proc.run (deadline, caps, fiber-parked), streaming form, owner-bound reaping"]:::open DRAIN["parked drain, what is LEFT of it: WO-E225 roster, ADT roster, group-by aggregates"]:::parked FOUND --> I7 @@ -98,6 +99,8 @@ flowchart TD I9g --> I28 I7 --> I28 I28 --> GAPS + I11 --> I42 + I24 --> I42 I16 --> I38 I32 --> I38 I36 -.reopens the pure-wo HMAC question.-> I34 @@ -269,6 +272,51 @@ critical path (the only engine + language work); jobs compose on it; the demo and gate close it. Fiber-scheduled jobs and cancellation→rollback appear in graph 2 — they need iteration 11 as well as 18. +## 5. porch — the web framework track + +States live on [the board's porch section](stories/00-status.md); every +pending node is `readiness: refine` (no approved spec), so a brainstorm +precedes any plan. Three independent roots: **2** (the auth chain), +**6** (the streaming chain), **5** (anytime, no incoming edges). **9** is +the track's only `readiness: ready` item and is held. + +```mermaid +flowchart TD + classDef done fill:#1a7f37,color:#fff,stroke:none + classDef refine fill:#eac54f,color:#000,stroke:none + classDef held fill:#6e7781,color:#fff,stroke:none + classDef lang fill:#8250df,color:#fff,stroke:none + + CSPRNG["language track: CSPRNG builtin (id 96+) — porch 2's Phase A"]:::lang + P1["porch 1 store-backed middleware ✅ 2026-08-30"]:::done + P2["porch 2 randomness + cookies (repeated Set-Cookie/Vary headers, Cookie: parsing, signed cookies)"]:::refine + P3["porch 3 sessions"]:::refine + P4["porch 4 CSRF"]:::refine + P5["porch 5 routing + response ergonomics"]:::refine + P6["porch 6 streaming core"]:::refine + P7["porch 7 SSE + compression"]:::refine + P8["porch 8 static files + lifecycle"]:::refine + P9["porch 9 idempotent replay (⏸ hold; readiness: ready)"]:::held + + CSPRNG --> P2 + P2 --> P3 + P2 --> P4 + P3 --> P4 + P6 --> P7 + P6 --> P8 + P2 --> P7 + P5 --> P7 + P1 -.re-scope 79e6da4: replay-on-retry split out of 1.-> P9 +``` + +The two non-obvious edges are stated in porch 7's own story: gzip's +`Accept-Encoding` negotiation reuses the q-value ranking iteration 5 adds, +and `Vary: Accept-Encoding` needs iteration 2's repeated-header work to +accumulate with other `Vary` contributions. Porch 2's Phase A is +language-track work (the CSPRNG builtin) — the cross-track edge this graph +exists to make visible. Streaming responses' runtime prerequisites +(fibers, iteration 11) are already green in graph 2. + ## Maintenance rule When an iteration or slice lands, update its node's class here in the diff --git a/docs/00-git-commit-history.md b/docs/00-git-commit-history.md index 94c9cbe..c8fd935 100644 --- a/docs/00-git-commit-history.md +++ b/docs/00-git-commit-history.md @@ -43,6 +43,7 @@ features cannot collide. | `lang41` | runtime: unadopted shard must not impersonate shard 0 | on `dev` (`9dca0b4`); independent of the residency stack, not picked | | `porch-store` | porch store tables, Limiter and Idempotent middleware (Phases A, B, C) | on `dev` (`519d411`, `5b1e82a`, `aee7926`). **In progress**: Phase C was uncommitted work from a parallel session, committed as-is, and calls `json.decode`/`json.encode` with no `use json` import | | `query-corpus` | databasev2 query-grammar corpus #1 | on `dev` (`4c82461`). Conclusion was "no new grammar needed" | +| `lang42` | iteration 42 — bounded subprocess: `proc.run` bounded + parked (pidfd, caps, ceiling, owner-bound reaping), `proc.run_dl`; carries the alacritty/tmux/zen parity studies and the porch dependency-graph section from the same sweep | on `dev` 2026-09-01 | ## Cherry-picks onto master diff --git a/docs/plan/exploration/alacritty/00-alacritty-parity.md b/docs/plan/exploration/alacritty/00-alacritty-parity.md new file mode 100644 index 0000000..5bd45ee --- /dev/null +++ b/docs/plan/exploration/alacritty/00-alacritty-parity.md @@ -0,0 +1,123 @@ +# Alacritty parity — what the language needs to build a terminal-class application + +Source: [`.dev/reference/alacritty/`](../../../../.dev/reference/alacritty/) +(shallow clone, surveyed 2026-09-01). Companion study to +[`fiber/00-fiber-parity.md`](../fiber/00-fiber-parity.md), which asked the same +question for the web-framework surface. This one asks it for a native, +event-driven, GPU-rendered desktop application — the workload class writeonce +is furthest from today. + +## What alacritty actually is (measured, not summarized from its README) + +33,699 lines of Rust across a 4-crate workspace, ~22 direct dependencies: + +- **`alacritty_terminal`** — the portable core, no GUI anywhere in it: PTY + creation and lifecycle (`tty/unix.rs`, 448 lines — openpty, fork/exec of the + shell, window-size ioctls, child reaping), a byte-level escape-sequence + parser (the `vte` crate), the grid data structure, and an event loop over + the `polling` crate (epoll/kqueue on arbitrary fds). +- **`alacritty`** — the application: windowing (`winit`), OpenGL context + creation (`glutin`, EGL/WGL), glyph rasterization (`crossfont`), a + timer-driven scheduler (`scheduler.rs` — cursor blink, repaint deadlines), + config with live reload (`notify` file watching), clipboard, signal + handling (`signal-hook`), CLI (`clap`). +- **`alacritty_config`/`_derive`** — config plumbing (a proc-macro crate; + writeonce's analogue would be compile-time codegen, which `@table` already + does for a different domain). + +The split matters more than the line count: the terminal CORE is a +files-and-processes program with zero graphics, and the GUI shell around it is +where every heavyweight dependency lives. + +## What already fits — the part that costs nothing + +- **The concurrency model is a better fit than alacritty's own.** Alacritty + runs a PTY-reader thread and a window-event thread synchronized through + `parking_lot` locks around the grid. Shard actors with ownership-move + messages express this without shared state: a reader actor owns the parser, + sends grid deltas; a display actor owns the grid. Iteration 24's chat app + already proved the shape (fd-driven actor, fan-out, lifecycle). +- **Single self-contained binary** — alacritty's distribution story is + writeonce's existing one. +- **Bytes + bitwise operators** (iterations 19, 36) — the VTE parser is a byte + state machine; the primitive layer for it exists. +- **`fn main`, exit codes, env, fs, time** (plan 9 stdlib) — config discovery, + CLI-shaped startup. + +## The gaps, ordered by what unblocks what + +Each maps onto an existing iteration where one exists; only two items are +genuinely new. + +1. **PTY + bounded subprocess** — the heart of the core. Openpty, fork/exec + with the child on the slave side as its controlling terminal, resize + ioctls, orderly child shutdown. This is exactly iteration 28's named gap + fan-out ("bounded subprocess, stdin/stdout transport") plus the PTY-specific + ioctls. ~450 lines of Rust in the reference; a C-builtin family in + writeonce's existing style (crypto/net precedent: small id-numbered + builtins, no general FFI). +2. **Readiness on arbitrary fds.** The runtime's io_uring loop watches sockets + it created. A PTY master fd — and later a display-server fd — must be + registrable in the same loop, parking the owning fiber until readable. + Alacritty needs nothing fancier (its `polling` crate is the same shape); + this is a seam widening, not a new subsystem. +3. **Signals as events.** SIGCHLD (child died) and SIGWINCH (resize) must + arrive as mailbox messages, the way iteration 24 handles fd events. Today + signals are runtime-internal (SIGTERM drain). Small, but nothing else can + substitute for it. +4. **Unicode width + UTF-8 decode in the stdlib.** The grid is addressed in + cells; every printed byte-run needs "how many columns". A data-table + problem, not a design problem — but without it a terminal misrenders + immediately. (Alacritty: `unicode-width` crate.) +5. **`time.mono` returns, plus a timer wheel.** Cut 2026-08-10 with "returns + when a workload needs monotonic math" — this workload is that consumer. + `time.after` (id 90) exists; a repaint/blink scheduler needs monotonic + deadlines that survive wall-clock jumps. +6. **Outbound connection + fd passing + shared-memory buffers.** Iteration 38 + already names `net.connect`. Wayland is *a unix-socket protocol*: with + connect, SCM_RIGHTS fd passing, and an mmap/shm builtin, a Wayland client + with software rendering (wl_shm) is expressible in pure `.wo` — a windowed + terminal with **no C dependency linked at all**. This is the outside-the-box + route the reference makes visible: alacritty predates it culturally (X11 + era) and pays for GL instead. +7. **The GPU fork — decide late.** OpenGL/EGL and font rasterization + (freetype/fontconfig) cannot be spoken over a socket; they are C ABI or + nothing. Three options, same fork iteration 28 already recorded as + "FFI-vs-out-of-process": (a) general FFI — largest doctrine change, rejected + until a second consumer demands it; (b) subsystem C builtins (the crypto + precedent) — a `gfx`/`font` builtin family; (c) an out-of-process render + server writeonce talks to over its own socket — fits the actor model, + keeps the language pure, costs a second process. Software rendering via + route 6 defers this fork entirely: glyph rasterization from a pre-baked + bitmap font atlas is pure byte math. + +## The maturity path, as driving workloads (the project's own method) + +Each stage is a shippable proof, ordered so no stage waits on the fork in 7: + +- **Stage A — headless terminal**: PTY builtins + signals + fd readiness + (items 1–3). Proof: a `.wo` program spawns a shell, feeds it a script, + captures and asserts the output — a `script`/`expect` clone. Closes + iteration 28's bounded-subprocess gap as a side effect. +- **Stage B — VTE grid**: parser + grid in pure `.wo` (item 4). Proof: replay + recorded terminal sessions (vttest, asciinema casts) and assert final grid + state against the reference implementation's. +- **Stage C — multiplexer**: A + B + existing net = a tmux-lite: sessions + survive detach, clients attach over a unix socket. No graphics, real + product, exercises everything server-side writeonce is already good at. + *Corrected by the tmux study (2026-09-01): this stage also needs + SCM_RIGHTS fd passing (item 6's builtin, promoted here), termios adoption + of the client's own tty, and the terminfo fork — see + [`../tmux/00-tmux-parity.md`](../tmux/00-tmux-parity.md).* +- **Stage D — windowed, software-rendered**: item 6 (connect, fd passing, + shm) + a Wayland client library in `.wo`. First pixel on screen with zero + linked C. +- **Stage E — GPU**: only now decide item 7, with D as the measured baseline + that says whether GL is worth an FFI doctrine change. + +## What this does NOT recommend + +No general FFI now (one consumer, and stages A–D never need it); no bundled +font rasterizer until D shows bitmap atlases failing; no attempt at winit-class +cross-platform windowing — Linux/Wayland first, the same way the runtime is +Linux/io_uring first. diff --git a/docs/plan/exploration/tmux/00-tmux-parity.md b/docs/plan/exploration/tmux/00-tmux-parity.md new file mode 100644 index 0000000..b7a120d --- /dev/null +++ b/docs/plan/exploration/tmux/00-tmux-parity.md @@ -0,0 +1,104 @@ +# tmux parity — the multiplexer as the language's next driving workload + +Source: [`.dev/reference/tmux/`](../../../../.dev/reference/tmux/) (shallow +clone, surveyed 2026-09-01). Companion to +[`alacritty/00-alacritty-parity.md`](../alacritty/00-alacritty-parity.md), +whose stage C ("tmux-lite") this study scopes for real. tmux is the more +instructive reference of the two: it needs **no graphics at all**, so the +entire program sits inside the territory the alacritty study's stages A–C +cover — it is the proof target, not a stepping stone to one. + +## What tmux actually is (measured) + +~112,000 lines of C (100,027 in the 153 top-level `.c` files; the rest is +`compat/` shims and headers). Exactly **two** external dependencies — +libevent (fd loop + buffers) and ncurses used *only* as a terminfo(5) reader +(`tty-term.c` calls `setupterm`/`tigetstr` and nothing curses-like). Every +other need is vendored in `compat/`: OpenBSD's imsg framing, `forkpty` for +five platforms, `daemon`. + +The architecture is one daemonized **server** owning every session, window, +pane and PTY (`server.c:190` forks it), and a deliberately thin **client** +(808 lines): the client connects over a unix socket and **passes its own +terminal fd to the server** with SCM_RIGHTS (`compat/imsg-buffer.c:798`, +`proc.c` — `imsgbuf_allow_fdpass`); from then on the server writes escape +sequences directly to the client's tty. Detach survives because the client +process is disposable — the server never was attached to a terminal it +doesn't hold as a passed fd. + +Where the lines actually go — the multiplexer kernel is small, the UX is not: + +- **Kernel, ~27k**: `server-client.c` 3,280 · `input.c` 3,745 (the escape + parser) · `tty.c` 3,221 (the output driver) · `screen-write.c` 3,204 · + `window.c` 2,893 · `grid.c` 1,839 · `layout.c` 2,022 (the pane split + tree) · `tty-keys.c` 1,883 (key decoding) · `utf8.c` 1,043 · plus + session/spawn/proc/job plumbing. +- **A command LANGUAGE, ~12k**: 63 `cmd-*.c` files behind a yacc grammar + (`cmd-parse.y`) — the config file is just commands; `format.c` is 7,182 + lines implementing a template DSL with ~380 variables; `options-table.c` + 2,070 lines of typed settings. +- **Interactive UX, ~20k**: `window-copy.c` 7,300 (copy mode is the single + biggest file in tmux) · choose-tree/customize/menus/prompt. + +## What writeonce already answers + +- **Client/server over a unix socket** — `listen_unix` landed (iteration 35); + the request/actor shape is the porch daily bread. +- **Server = actor tree.** tmux multiplexes everything through one + single-threaded libevent loop with callbacks; sessions, windows, panes and + clients as actors with ownership-move messages is the same topology with + the concurrency written down instead of implied. Iteration 24 (chat rooms, + fan-out, lifecycle, SIGTERM drain) already proved every piece of the + pattern. +- **The DSL layer is free.** tmux hand-rolls a command grammar, a format + template language and a typed options table (~12k lines) because C has no + expression language to lend. writeonce's config/scripting surface can be + the language itself — `${ }` interpolation replaces `format.c`, and a + `@table` of settings replaces `options-table.c`. +- **Durable sessions beyond tmux.** tmux state dies with the server; a + writeonce multiplexer's session/layout tables can be `durable: true` for + free — scrollback in the WAL is the kind of trick the storage engine + exists for. Feature, not parity. + +## Gaps — mostly shared with the alacritty list, two new, one promoted + +1. **PTY + bounded subprocess, signals, arbitrary-fd readiness, unicode + width, monotonic timers** — identical to alacritty study gaps 1–5; + `spawn.c`/`job.c`/`compat/fdforkpty.c` are the reference reading. +2. **fd passing (SCM_RIGHTS) — PROMOTED.** The alacritty study placed it in + stage D (Wayland). Wrong stage: detach/attach — the whole point of a + multiplexer — is built on handing the client's tty fd across a unix + socket. Second consumer found; the builtin belongs to the multiplexer + stage. (The alacritty study is corrected in place.) +3. **NEW — termios control of an existing terminal.** The client must put + *its own* tty into raw mode and restore it on exit (tmux: + `cfmakeraw`, `tcgetattr`/`tcsetattr`). The PTY gap covers creating + terminals; this is adopting one you were given. Small builtin family, + nothing else substitutes. +4. **NEW — the terminfo fork.** tmux answers "what escape sequences does + THIS client's terminal speak" from the terminfo database. Two honest + options: read the compiled terminfo format in pure `.wo` (a documented + binary file — a parser, not a linked library; ncurses would NOT be + imported) or emit a fixed xterm-256color profile and refuse exotic + terminals by name. Decide at brainstorm; start fixed, the refusal names + the gap. +5. **Daemonization** — fork-and-detach with the socket handed over. Cheap, + and arguably skippable first (a foreground server under systemd was good + enough for writeonce.de). + +## Corrected staged path (supersedes the alacritty study's stage C sizing) + +- **Stage A/B unchanged** — headless PTY runner, then the VTE grid replayed + against recorded sessions (`input.c` + `grid.c` are the behaviours to pin). +- **Stage C — the multiplexer** now carries its real bill: A + B **plus** + fd passing (gap 2), termios adoption (gap 3), and the terminfo decision + (gap 4). Proof: detach, kill the client, reattach from another terminal, + scrollback intact — then restart the *server* and reattach with layout and + scrollback replayed from the WAL, which is the demo tmux cannot give. +- **Stage D/E unchanged** (Wayland shm, then the GPU fork) — and stage D + gets gap 2 for free once C lands. + +Scope honesty: parity with tmux the product is ~100k lines including a 7k +copy mode and 20k of chooser UX — not the goal. The kernel a driving +workload needs is the ~27k-line column, and the DSL third of tmux dissolves +into language features writeonce already has. diff --git a/docs/plan/exploration/zen-browser/00-zen-browser-parity.md b/docs/plan/exploration/zen-browser/00-zen-browser-parity.md new file mode 100644 index 0000000..2cff4a0 --- /dev/null +++ b/docs/plan/exploration/zen-browser/00-zen-browser-parity.md @@ -0,0 +1,86 @@ +# Zen Browser parity — what a browser-class application actually asks of a language + +Source: [`.dev/reference/zen-browser/`](../../../../.dev/reference/zen-browser/) +(shallow clone of zen-browser/desktop, surveyed 2026-09-01). Third study in the +series after [`alacritty`](../alacritty/00-alacritty-parity.md) and +[`tmux`](../tmux/00-tmux-parity.md) — and the one whose answer is different in +kind, which is why it is worth having. + +## What Zen actually is (measured) + +Zen is not a browser codebase. It is a **Firefox overlay**: `surfer.json` pins +`product: firefox, version: 154.0.1`, `npm run download` fetches the Firefox +source into `engine/` at build time, and the repo contributes **256 patch +files** plus a 33 MB `src/` tree that is copied over it — ~246,000 lines +counting JS/MJS/CSS/XHTML/patches (616 `.js` files, 568 Fluent localization +files, 48 CSS). There is no rendering, layout, JS-engine, or networking code +in the repository at all; Gecko (tens of millions of lines of C++/Rust) +arrives as a downloaded dependency and is built with Mozilla's own `mach`. + +Zen's value-add lives in `src/zen/`: workspaces ("spaces"), split view, +compact mode, glance, folders, session store, sync — all UI composition and +state, written in JavaScript **because the engine ships a JavaScript host and +Zen's code runs inside it**. The engine is also the app platform. + +## The lesson, stated plainly + +Nobody — including a successful, well-staffed browser project — writes a +browser. They skin an engine. So "mature the language until it can build an +application such as this" resolves to three different questions, and only one +of them is real work for writeonce: + +1. **Build the engine in `.wo`?** Not a maturity path, a decades-long refusal. + The gap list it produces ("general FFI, C++ interop, a GPU pipeline, a JS + VM…") is the alacritty study's stage-E fork multiplied by a thousand, with + no intermediate shippable stage. Rejected as a driving workload — it cannot + drive, only sink. +2. **Embed an engine (CEF/WebKitGTK) behind FFI?** The heaviest possible FFI + consumer. Same fork as alacritty item 7, and this study deliberately does + not promote it: an embedded engine's API surface is enormous and unstable, + the worst first FFI customer imaginable. +3. **Drive an engine out-of-process.** Chromium and Firefox both expose a + remote-debugging protocol (CDP; Firefox now speaks it too) over a local + socket or a stdio pipe. A writeonce program that spawns a browser, connects, + and drives it — kiosk shells, scrapers, site-acceptance drivers of the kind + `site-accept.sh` fakes with curl today — is the browser-class workload that + is actually reachable, and it is exactly the actor/protocol shape the + runtime is built around. This is Playwright's architecture, minus the + Node.js. + +## What route 3 needs — mostly the existing list, one genuinely new gap + +- **Bounded subprocess with stdio transport** — spawning the browser with a + pipe transport is the third consumer of iteration 28's named gap (after the + alacritty and tmux studies' PTY stages; this one does not even need a PTY). +- **`net.connect`** — iteration 38, already named, for the debugging socket. +- **NEW — WebSocket CLIENT.** CDP speaks WebSocket; iteration 24 landed the + server side (`ws_accept`, frames) but nothing performs an outbound upgrade + handshake. Small — the frame code is the hard half and already exists. +- **JSON both ways** — exists. Actor-per-session/per-tab supervision, timeouts + (`time.after`), graceful teardown — all landed in iteration 24. + +Notably absent: nothing graphical, no terminfo, no termios, no fd passing. +Route 3 is *closer* than the tmux stage. + +## What Zen's own feature layer says about writeonce + +Zen's ~250k-line overlay is session stores, workspace state, sync, and +declarative UI — in writeonce terms: `@table` rows (durable by declaration +since databasev2 2), and wo-html components. The project's existing bet — +that the browser is the *client* and the application lives server-side in +porch — already covers the useful half of what Zen builds. The other half of +Zen's lesson is about hosting: Gecko wins as a platform because it embeds a +scripting language with capability boundaries, which is iteration 28's +skillhost direction (writeonce as the confined host, not the confined guest). + +## Verdict for the maturity path + +The browser study adds **one builtin-sized gap** (WebSocket client) and +**one workload** to the staged path from the alacritty study — call it +**stage C′, the browser driver**, parallel to stage C (it needs subprocess + +`net.connect` + ws-client, none of stage C's tty machinery). Proof when it +lands: replace a curl leg of `site-accept.sh` with a `.wo` driver that opens +writeonce.de in a real browser, asserts the rendered chapter list, and tears +the browser down cleanly — the site gate exercising the language's own +browser-automation story end to end. Engine-building and engine-embedding +stay refused, by name, with this study as the reason. diff --git a/docs/stories/00-status.md b/docs/stories/00-status.md index 0f63af7..608fa6f 100644 --- a/docs/stories/00-status.md +++ b/docs/stories/00-status.md @@ -1048,6 +1048,7 @@ check mode, and the `internal/` dep boundary (WO-E108). Driver-only. | 23 | io_uring group-commit write path — batched durability overlapped on shard threads, fsync fallback | **no spec yet** — brainstorm after iterations 8 + 22 | | 27 | Query grammar from real embedded-DB corpora — whole-query count + correlated exists, driven by the skillhost SQL catalogue; add only what a corpus uses | **no spec yet** — three forks; may collapse to "confirm len(query) + add exists" | | 14 | skillhost host workload — port skillhost (MCP host + confined script runner) to writeonce; drives the missing host capabilities into the open (bounded subprocess, stdin/stdout transport, fs metadata, FFI-vs-out-of-process) | **no spec yet** — gaps recorded in the iteration; each gap brainstormed on demand, bounded-subprocess first | +| 42 | [Bounded subprocess](language-runtime-database/42-bounded-subprocess.md) — `proc.run` exists (`sysio.c`) but shard-blocking, deadline-less, silently truncating; this bounds it in place (deadline, output caps, per-shard ceiling, owner-bound reaping via pidfd in the io_uring loop, fiber parked); streaming form deferred by name; 28's leading gap promoted with four consumers | ✅ spec approved 2026-09-01 — [spec](../superpowers/specs/2026-09-01-bounded-subprocess-design.md); `readiness: ready`, plan next | | 17 | library projects + dependency privacy — `wo.toml` kind = "library" (checkable without entry, dual lib+bin) + Go-style `internal/` at the [deps] boundary; framework reorg demonstrates both | ✅ **landed 2026-08-20** — [spec](../superpowers/specs/2026-08-20-library-kind-internal-design.md) · [plan](../superpowers/plans/2026-08-20-library-kind-internal.md) | | 10 | HTTP service layer | [plan 6](../superpowers/plans/2026-08-01-http-service-layer.md) | | 11 | Fibers | vision §3, [blue-green exploration](../plan/exploration/blue-green-vm/00-vision.md) | diff --git a/docs/stories/language-runtime-database/42-bounded-subprocess.md b/docs/stories/language-runtime-database/42-bounded-subprocess.md new file mode 100644 index 0000000..bb0dbc6 --- /dev/null +++ b/docs/stories/language-runtime-database/42-bounded-subprocess.md @@ -0,0 +1,126 @@ +--- +track: language-runtime-database +iteration: "42" +status: pending +readiness: ready +--- + +# 42 — bounded subprocess: spawn, supervise, and reap a child process + +> Part of [Story — one language, one runtime, one database](00-story.md). +> Spec: [`2026-09-01-bounded-subprocess-design.md`](../../superpowers/specs/2026-09-01-bounded-subprocess-design.md) +> (brainstormed and approved 2026-09-01; the six forks below are settled). +> First named as iteration [28](28-skillhost-host-workload.md)'s leading gap +> ("bounded subprocess, stdin/stdout transport"), promoted to its own +> iteration on 2026-09-01 when three further consumers arrived at once — the +> [alacritty](../../plan/exploration/alacritty/00-alacritty-parity.md), +> [tmux](../../plan/exploration/tmux/00-tmux-parity.md) and +> [zen-browser](../../plan/exploration/zen-browser/00-zen-browser-parity.md) +> studies all bottom out on it as their first domino. +> +> **The problem.** A one-shot `proc.run` already exists +> (`runtime/src/sysio.c`, `WO_B_PROC_RUN`: fork/execvp on an argv vector, +> stdout/stderr captured through pipes, exit code returned) — but it is +> bounded in nothing that matters and blocking in the one place it must not +> be. It runs on the shard thread, not parked in a fiber, so one slow child +> stalls every actor on that shard; it has no deadline, and its `waitpid` +> loop notices a runtime stop only if a signal happens to interrupt it; its +> output caps (8,192 / 4,096 bytes, fixed) truncate silently rather than +> refuse; there is no stdin, no streaming, no concurrency ceiling, and no +> story for what happens to a child when its owner dies. Suspected but not +> yet reproduced: a child that fills the stdout pipe past the cap while the +> parent is waiting for stderr EOF deadlocks both — needs a test before it +> is claimed. Every workload class beyond "server that owns all its state" +> — a script runner (28), a terminal (alacritty stage A), a multiplexer +> (tmux), a browser driver (zen C′) — begins by starting a process and ends +> by being responsible for it. + +## Why *bounded* — the doctrine half + +The feature is deliberately not "exec". Every resource a child can consume +carries a declared ceiling, and exceeding it is a refusal by name, never a +hang or an OOM — the same fail-closed posture porch 1 proved for pool +saturation and iteration 31 for mailbox caps: + +- **Time** — a deadline after which the child is killed and the caller + resumes with a named timeout error. +- **Output bytes** — a cap on captured stdout/stderr; past it the child is + killed and the error names the cap, because an unbounded pipe buffer is an + unbounded allocation. +- **Concurrency** — a ceiling on live children per program; at the ceiling, + spawning fails closed rather than queueing invisibly. +- **Lifetime** — a child is owned by the actor that spawned it. Owner dies, + child is killed and reaped. Program stops, children are terminated before + exit (iteration 40's drain guarantee extends to them). A zombie or an + orphan is a bug by definition, not a caveat. + +## Info — the forks, settled (brainstorm 2026-09-01) + +1. **Retrofit, one-shot only.** The existing `proc.run` gains the bounds + in place (deadline, declared output caps, fiber parking, owner-bound + reaping, per-shard ceiling). The streaming form — long-lived child, + stdout as mailbox messages, what tmux/alacritty/zen need — is the named + follow-up, building on this slice's registry and pidfd machinery. +2. **Transport** — died with fork 1: a one-shot form streams nothing. + Settled by the follow-up when it arrives. +3. **pidfd, no signal seam.** Child exit is a pollable fd (`pidfd_open`, + Linux 5.3+) registered in the shard's io_uring beside the pipes and the + deadline timer; the fiber parks in the iteration 35 `_dl` mould. The + general signals-as-events seam stays deferred by name to its real + consumer (the multiplexer study's stage C). +4. **Argv only; a shell form is refused permanently.** A caller wanting a + shell types `sh -c` into their own argv and owns the quoting risk; + consumer #1 is a confined script runner and must never get a + shell-shaped API. +5. **Env and cwd stay inherited, zero knobs.** No consumer demands + control; skillhost confinement owns env scrubbing when it arrives, and + a clean-env knob breaks `execvp` path search. +6. **Ids.** `WO_B_PROC_RUN` keeps its id and gains parking + defaults; the + extended bounded form takes the next free id from 96+ (89/90 remain + 31's holes); no `.wob` bump (38's precedent). + +Bounds surface: bare `proc.run(cmd, args)` gets named defaults (30 s, +1 MiB stdout, 64 KiB stderr); an extended form states them per call; +exceeding any bound kills the child and raises a catchable error naming +the bound — silent truncation is removed. + +## Acceptance criteria — firmed in the spec, normative form there + +- Given a child that exits normally, when it is run, then its exit code is + an ordinary value in the language and both output streams are readable. +- Given a child that outruns its deadline, when the deadline passes, then + the child is dead (verified by pid absence, not assumed), the caller has a + named timeout error, and no fd has leaked. +- Given a child whose output exceeds the byte cap, then the refusal names + the cap and the child is dead — measured with a deliberately chatty child. +- Given an owning actor that dies while its child lives, then the child is + reaped — verified from the outside via the process table. +- Given SIGTERM to the program while children live, then every child is + terminated before the program exits — the drain gate (iteration 40's + battery) gains a subprocess leg. +- Given a spawn loop of one thousand sequential children, then the + program's fd count is flat (the iteration 24 measurement style) and rss + does not grow with the loop. +- A runnable example under `docs/examples/` with a `just` gate, in the + residency/chat mould — the gate is the acceptance, not the unit tests + alone. + +## Out of scope, by name + +- **PTY allocation** — pipes only. Giving the child a pseudo-terminal is + the alacritty/tmux stage-A follow-up and has its own ioctl surface. +- **fd passing (SCM_RIGHTS), termios, terminfo** — the tmux study's stage-C + bill, separate iterations. +- **WebSocket client** — zen C′'s remaining item, not process work. +- **A general signal API for user code** — only if fork 3 resolves toward + the private reaping path; otherwise this iteration carries the seam but + not the surface. +- **Pipelines between children** — composition can wait for a consumer. + +## Consumers, for the record + +Iteration [28](28-skillhost-host-workload.md) (script runner, "bounded +subprocess first" was already its stated order); alacritty study stage A +(headless PTY runner — needs this plus PTY); tmux study (spawn half of its +kernel); zen study stage C′ (spawn the browser, then drive it). Four +consumers is the most any single named gap has accumulated. diff --git a/docs/superpowers/plans/2026-09-01-bounded-subprocess.md b/docs/superpowers/plans/2026-09-01-bounded-subprocess.md new file mode 100644 index 0000000..0ad8678 --- /dev/null +++ b/docs/superpowers/plans/2026-09-01-bounded-subprocess.md @@ -0,0 +1,320 @@ +# Bounded Subprocess (iteration 42) Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use +> superpowers:subagent-driven-development (recommended) or +> superpowers:executing-plans to implement this plan task-by-task. Steps use +> checkbox (`- [ ]`) syntax for tracking. +> +> **Project rule (overrides the plan-skill template):** plan docs carry +> concept, reason and actions in words — no implementation or test code +> blocks. Each step names the exact functions, fields, ids and expected +> outcomes; the implementer writes the code at the keyboard, matching the +> anchors cited here. + +**Goal:** `proc.run` becomes bounded (deadline, output caps, per-shard +ceiling, owner-bound reaping) and parked (fiber waits, shard never stalls), +plus an extended `proc.run_dl` stating bounds per call. + +**Architecture:** rework `WO_B_PROC_RUN` in `runtime/src/sysio.c` from +blocking drain + `waitpid` into the iteration 35 `_dl` parking mould: after +the fork, the two pipe read ends and a pidfd for the child are bundled +behind one epoll fd; the fiber parks on that bundle fd with the existing +`fb->dl_active`/`fb->dl_at` deadline sweep armed; re-entry after each wake +drains whatever is ready into growable capped buffers, and the pidfd firing +means reap-and-return. A per-shard child registry in `wo_vm` carries the +cross-park state and serves the ceiling, engine-stop kill, and unwind +cleanup. + +**Tech Stack:** C (runtime), OCaml (compiler stdlib table), `.wo` (example +app), bash (gate script), just (recipes). + +**Spec:** `docs/superpowers/specs/2026-09-01-bounded-subprocess-design.md` +— the plan argues from it; read both. + +## Global Constraints + +- Linux only; pidfd needs runtime kernel ≥ 5.3 (Ubuntu 22.04 ships 5.15). +- glibc 2.35 (the release build floor) has NO `pidfd_open` / + `pidfd_send_signal` wrappers (they arrived in 2.36) — call both via raw + `syscall(SYS_pidfd_open, …)` / `syscall(SYS_pidfd_send_signal, …)`. +- Defaults, verbatim from the spec: deadline 30 000 ms, stdout cap + 1 048 576 bytes, stderr cap 65 536 bytes, ceiling 32 children per shard. +- Bound violations kill the child and trap as `WO_T_IO` with a message + naming the bound and its value; silent truncation is removed. +- New builtin id: `WO_B_PROC_RUN_DL = 96` (89/90 stay iteration 31's + reserved holes). No `.wob` version bump — `WOB_VERSION` does not move. +- No new dependencies, no new threads, no signal handlers. +- All work on `dev`; every commit title prefixed `feat(lang42):` / + `fix(lang42):` / `docs(lang42):`; bullet-point commit bodies, ≤25 lines. +- Builds and tests only through just recipes: `just wovm-build`, + `just wovm-test`, `just woc-build`, `just woc-test`. +- Runtime test binaries build with ASan+UBSan (runtime/Makefile does this + for every `test/test_*.c` automatically) — a leak or race is a failure. + +--- + +### Task 1: claim the prefix, commit the standing docs + +**Files:** +- Modify: `docs/00-git-commit-history.md` (prefix registry table) +- Already-edited, to commit: story 42, spec, `docs/stories/00-status.md` + row, `docs/00-dependency-graph.md` node, the three exploration studies + under `docs/plan/exploration/{alacritty,tmux,zen-browser}/`, this plan. + +**Interfaces:** none — bookkeeping. + +- [ ] **Step 1:** Add a `lang42` row to the prefix registry table in + `docs/00-git-commit-history.md`: feature "iteration 42 — bounded + subprocess (proc.run bounded + parked, proc.run_dl)", status "on `dev`". +- [ ] **Step 2:** Commit all listed docs as one + `docs(lang42): story, spec and plan for bounded subprocess` commit + (bullets: story+spec+plan added; board row; graph node; the three + parity studies that motivated it). + +### Task 2: baseline suite — pin what `proc.run` already does + +**Files:** +- Create: `runtime/test/test_proc.c` (auto-picked by the Makefile's + `$(wildcard test/test_*.c)` — no build wiring needed) + +**Interfaces:** +- Consumes: the test harness macros in `runtime/test/t.h` (`T_EQ`, + `T_CHECK`), `wo_db_init`-style setup patterns from `test_builtin.c`, and + direct builtin dispatch the way `test_builtin.c` invokes cases — operands + in registers, `WO_B_PROC_RUN` (id 56) takes cmd Text, `multi Text` args, + and the Proc class id, returns the record {code, out, err}. +- Produces: `test_proc.c` as the home for every later runtime leg. + +- [ ] **Step 1:** Write three green legs against CURRENT behavior, copying + `test_builtin.c`'s VM/fiber setup: (a) a child that exits 0 with known + stdout — assert code 0 and the exact bytes; (b) a child that exits with a + known nonzero code — assert the code; (c) a nonexistent command — assert + code 127 (the execvp-failed convention already in the code). +- [ ] **Step 2:** `just wovm-test` — all three legs pass, whole suite + stays green, ASan clean. +- [ ] **Step 3:** Commit `feat(lang42): pin proc.run's current contract in + test_proc`. + +### Task 3: the parked rework — pidfd + epoll bundle + registry + +The core task. The suspected sequential-drain deadlock is proven first, +then dissolved by the rework. + +**Files:** +- Modify: `runtime/src/sysio.c` (the `WO_B_PROC_RUN` case, currently + ~lines 723–815) +- Modify: `runtime/src/vm.h` (the per-shard registry in `wo_vm`, one + pointer field on `wo_fiber` for the in-flight entry) +- Modify: `runtime/src/vm.c` (engine-stop sweep over the registry) +- Test: `runtime/test/test_proc.c` + +**Interfaces:** +- Consumes: `WO_SYS_PARKED` re-entry convention (`sysio.c:510` READ_DL is + the model: fill `fb->park_fd`, arm `fb->dl_active`/`fb->dl_at` once — + guarded so re-entry does not re-arm — return `WO_SYS_PARKED`; the plane + re-runs the builtin on wake); `stop_pending()`; `wo_str_new`. +- Produces: a registry entry type (name it `wo_child`) holding pid, pidfd, + the bundle epoll fd, both pipe fds, two growable buffers with their caps, + the deadline, and the owning fiber pointer; `wo_vm` gains a fixed array + of 32 `wo_child` slots plus a live count; `wo_fiber` gains a pointer to + its in-flight entry (NULL when none). Task 4–8 legs and Task 9's + `WO_B_PROC_RUN_DL` all reuse exactly this machinery. + +- [ ] **Step 1 (the red test):** In `test_proc.c`, add the deadlock leg: a + child (use `sh -c` in the test only) that writes ~200 KiB to stdout and + one line to stderr, stderr kept open until stdout completes. Guard the + leg with a wall-clock check: it must complete within 5 s. Under the + current sequential drain the parent stops reading stdout at 8,192 bytes, + the child blocks on a full pipe, and stderr never reaches EOF. +- [ ] **Step 2:** `just wovm-test` — the new leg FAILS (hangs into the + guard) while everything else stays green. This is the bug proven. +- [ ] **Step 3 (the rework):** Rewrite the `WO_B_PROC_RUN` case: pipes + opened `O_NONBLOCK` on the parent side; after the fork, + `syscall(SYS_pidfd_open, pid, 0)`; create one epoll fd and register both + pipe read ends and the pidfd; claim a registry slot (fail closed with + `WO_T_IO` naming the 32-per-shard ceiling if none — the message text the + Task 6 leg asserts); stash caps and buffers in the slot, point the fiber + at it. First entry and every re-entry then run the same drain: read each + ready pipe into its growable buffer; a buffer passing its cap means kill + (`syscall(SYS_pidfd_send_signal, pidfd, SIGKILL, 0, 0)`), reap, release + the slot, trap `WO_T_IO` naming the cap and value. Pidfd readable means + exited: reap via `waitpid` (now non-blocking — the pidfd said so), do a + final drain of both pipes to EOF (bounded by the caps), release, build + the Proc record exactly as today. Nothing ready and child alive: park on + the bundle fd with the deadline armed (30 000 ms default), return + `WO_SYS_PARKED`. Deadline re-entry with `dl_at` passed: kill, reap, + release, trap `WO_T_IO` naming the deadline. `stop_pending()` on any + entry: kill, reap, release, return `WO_SYS_STOPPED`. +- [ ] **Step 4:** Engine-stop sweep: where `vm.c`'s worker loop observes + the stop flag, kill + reap every live registry entry on that shard — + covers fibers that never get rescheduled. +- [ ] **Step 5:** `just wovm-test` — deadlock leg green, Task 2 baseline + legs still green (same results from the parked path), suite ASan clean. +- [ ] **Step 6:** Commit `feat(lang42): proc.run parks — pidfd + epoll + bundle + child registry` (bullets: the deadlock repro and its dissolve; + raw syscalls because glibc 2.35). + +### Task 4: the deadline leg + +**Files:** Test: `runtime/test/test_proc.c`; fix (if red exposes drift): +`runtime/src/sysio.c`. + +**Interfaces:** consumes Task 3's machinery unchanged; the leg drives the +default arming path. + +- [ ] **Step 1 (red first if Task 3 left a gap):** a child sleeping 10 s, + run with the deadline forced low for the test (drive the builtin with a + small `dl` the way the harness passes operands — until Task 9 the + extended operands are reachable only from C, which is fine here). Assert: + the trap message names the deadline and its value; then assert the pid is + GONE — `kill(pid, 0)` returns ESRCH — measured, not assumed. +- [ ] **Step 2:** `just wovm-test` green (implement/adjust the kill path if + Step 1 caught drift). Also add the fiber-progress assertion: while the + sleeping child runs, a second fiber on the same VM increments a counter — + assert it advanced before the deadline fired (the shard was never + blocked). +- [ ] **Step 3:** Commit `feat(lang42): deadline kills, parked shard keeps + scheduling`. + +### Task 5: the output-cap leg + +**Files:** Test: `runtime/test/test_proc.c`; fix: `runtime/src/sysio.c`. + +- [ ] **Step 1:** a child writing unbounded output against a small stdout + cap passed from the harness. Assert: `WO_T_IO` whose message names the + cap and its value; pid gone (ESRCH); and — fd hygiene — the shard's open + fd count returns to its pre-call value (count `/proc/self/fd` entries + before and after). +- [ ] **Step 2:** Same for the stderr cap. +- [ ] **Step 3:** `just wovm-test` green. Commit `feat(lang42): output caps + refuse by name, no truncation`. + +### Task 6: the ceiling leg + +**Files:** Test: `runtime/test/test_proc.c`. + +- [ ] **Step 1:** 32 fibers each spawn a child sleeping 2 s; a 33rd spawn + must trap `WO_T_IO` naming the ceiling while the 32 keep running to + completion unharmed. Then all 32 complete with code 0. +- [ ] **Step 2:** `just wovm-test` green (the slot-claim refusal exists + since Task 3; this pins it). Commit `feat(lang42): per-shard ceiling + fails closed at 32`. + +### Task 7: the churn leg — fds flat over a thousand runs + +**Files:** Test: `runtime/test/test_proc.c`. + +- [ ] **Step 1:** loop one thousand sequential short-lived children + (the iteration 24 measurement style): record the `/proc/self/fd` entry + count before, assert the count after equals it, and assert no registry + slot remains claimed. +- [ ] **Step 2:** `just wovm-test` green. Commit `feat(lang42): a thousand + spawns leave the fd table flat`. + +### Task 8: stop and unwind reap their children + +**Files:** Test: `runtime/test/test_proc.c`; fix: `runtime/src/sysio.c`, +`runtime/src/vm.c`. + +- [ ] **Step 1:** park a fiber on a child sleeping 10 s, set the stop flag + the way `test_fiber.c` does, drive the loop; assert `WO_SYS_STOPPED` + surfaced AND the child pid is gone. Second leg: a child owned by a fiber + that is unwound (the `test_unwind.c` pattern) is also gone. +- [ ] **Step 2:** `just wovm-test` green. Commit `feat(lang42): stop and + unwind kill the children they own`. + +### Task 9: `proc.run_dl` — the per-call bounds surface + +**Files:** +- Modify: `runtime/src/wob.h` (enum entry `WO_B_PROC_RUN_DL = 96`, comment + stating the operand order and the nil-never contract: bounds violations + trap, they do not nil) +- Modify: `runtime/src/sysio.c` (a thin case: parse deadline_ms, out_cap, + err_cap operands, then fall into Task 3's core with those instead of the + defaults) +- Modify: `compiler/src/types.ml:315` region (one row beside `proc.run`: + module `proc`, member `run_dl`, arity 5, id 96, same nullable-Proc + return and Proc record class as the existing row) +- Test: `runtime/test/test_proc.c` (drive id 96 with explicit bounds); + compiler: `just woc-test` must stay green — the generic builtin path + needs no `emit.ml` change (the net `_dl` family at ids 91–93 landed with + table rows alone; verify by reading their absence from `emit.ml` before + assuming). + +**Interfaces:** +- Produces: `proc.run_dl(cmd, args, deadline_ms, out_cap, err_cap)` in the + language, the name the example app and every future consumer calls. + +- [ ] **Step 1 (red):** runtime leg driving id 96 with a tight explicit + deadline — fails while the case is absent. +- [ ] **Step 2:** add the wob.h entry and the sysio.c case; green. +- [ ] **Step 3:** add the types.ml row; `just woc-build && just woc-test` + green. +- [ ] **Step 4:** Commit `feat(lang42): proc.run_dl — deadline and caps at + the call site`. + +### Task 10: the example app and its gate + +**Files:** +- Create: `docs/examples/subprocess/` (wo.toml + one `.wo` source: an + actor that serves `proc.run`/`proc.run_dl` results — a fast command, a + deliberate deadline hit caught with try/catch, a deliberate cap hit + caught, and a long-running child for the drain leg; logs to + `/tmp/subprocess.log` per the examples convention, announced and + banner-separated) +- Create: `scripts/subprocess-accept.sh` (the gate) +- Modify: `justfile` (recipe `subprocess` running the gate, in the + `residency`/`site` recipe mould) + +**Interfaces:** +- Consumes: `proc.run` and `proc.run_dl` exactly as Task 9 shipped them. + +- [ ] **Step 1:** write the app and gate with these legs: fast command + returns its output through the actor; deadline violation is caught in + `.wo` and reported (proves catchability from the language, not just from + C); cap violation likewise; concurrency — a slow child runs while a + second request is answered (wall-clock assertion that the second answer + did not wait for the child); SIGTERM while a `sleep 30` child lives — + the gate records the child pid, stops the app, asserts the app exited + cleanly AND the child pid is gone (iteration 40's battery gains its + subprocess leg here). +- [ ] **Step 2:** `just subprocess` — all legs green, printed as + `subprocess-accept: N checks, 0 failures`. +- [ ] **Step 3:** Commit `feat(lang42): subprocess example + gate` (app, + script, recipe). + +### Task 11: close-out + +**Files:** +- Modify: `runtime/src/CODE-LOGIC.md` (the proc.run section: parked + lifecycle, the registry, the raw-syscall note, the deadlock that was) +- Modify: `docs/stories/language-runtime-database/42-bounded-subprocess.md` + (frontmatter `status: done`; Progress note of what landed vs the spec) +- Modify: `docs/stories/00-status.md` (NEXT PLAN entry answering the six + standup questions — implemented, key findings measured, learned, + unblocked (the streaming form, tmux/alacritty/zen stages), next steps, + `.dev/reference` used: alacritty/tmux/zen-browser; flip the pending row) +- Modify: `docs/00-dependency-graph.md` (node 42 class → done, same change + as the board row per the maintenance rule) + +- [ ] **Step 1:** run the full belt: `just wovm-test`, `just woc-test`, + `just subprocess`, plus `just site` untouched-but-verified. All green, + outputs quoted in the board entry, not asserted. +- [ ] **Step 2:** write CODE-LOGIC.md beside the code (project rule) and + the three doc updates. +- [ ] **Step 3:** Commit `docs(lang42): close out iteration 42`. + +--- + +## Self-review (done at write time) + +- Spec coverage: every acceptance criterion has a task — normal exit (2), + deadlock repro-then-fix (3), deadline + progress (4), caps (5), ceiling + (6), fd-flat thousand (7), stop/unwind (8), per-call bounds (9), drain + leg + example gate (10). Out-of-scope items appear in no task. +- Placeholders: none; every step names its files, ids, messages and + expected outcomes. +- Consistency: registry type `wo_child`, bundle-epoll park, id 96, caps + 30 000 ms / 1 MiB / 64 KiB / 32 used identically in tasks 3–10. +- One verify-before-assuming flag left deliberately in Task 9: confirm the + net `_dl` rows needed no `emit.ml` change before mirroring them. diff --git a/docs/superpowers/specs/2026-09-01-bounded-subprocess-design.md b/docs/superpowers/specs/2026-09-01-bounded-subprocess-design.md new file mode 100644 index 0000000..89fddc3 --- /dev/null +++ b/docs/superpowers/specs/2026-09-01-bounded-subprocess-design.md @@ -0,0 +1,143 @@ +# Bounded subprocess v1 — the one-shot form, bounded and parked + +> Brainstormed and approved 2026-09-01. Story: +> [language 42](../../stories/language-runtime-database/42-bounded-subprocess.md). +> Scope settled during brainstorm: bound the existing one-shot `proc.run` +> only; the streaming form (long-lived child, output as mailbox messages) is +> the named follow-up. Exit notification via pidfd, no signal seam. Bounds as +> defaults plus per-call override, refusal by name. Argv only, no shell form +> ever. Env and cwd stay inherited, no knobs. + +## The problem + +`proc.run` exists (`runtime/src/sysio.c`, `WO_B_PROC_RUN`) but is bounded in +nothing that matters and blocks in the one place it must not: it runs the +fork, the pipe drain, and the `waitpid` on the shard thread itself, so one +slow child stalls every actor and fiber on that shard. It has no deadline — +a child that never exits parks the shard forever, and the `waitpid` loop +notices an engine stop only if a signal happens to interrupt it. Its output +caps are fixed constants (8,192 bytes stdout, 4,096 stderr) that truncate +silently. Nothing reaps a child when the engine stops. And the sequential +drain (stdout to cap, then stderr to EOF) is suspected of deadlocking +against a child that fills the stdout pipe while holding stderr open — to be +proven by a failing test before the fix is claimed. + +Four consumers arrived at once — iteration 28's script runner and the +alacritty/tmux/zen-browser exploration studies — all bottoming out on this +as their first domino. + +## The model, and why + +**Bounded means: every resource a child can consume carries a declared +ceiling, and exceeding one is a refusal that names the ceiling — never a +hang, a truncation, or an OOM.** Same fail-closed posture as porch 1's pool +saturation and iteration 31's mailbox caps. + +- **Time.** Every run has a deadline. At the deadline the child is killed, + reaped, and the caller gets a catchable error naming the deadline and its + value. +- **Output bytes.** Stdout and stderr each carry a cap. Past a cap the + child is killed and the error names the cap — silent truncation is + removed, which is the one observable behavior change for existing + callers (code relying on truncation was relying on a bug). +- **Concurrency.** A per-shard ceiling on live children (default 32). At + the ceiling a spawn fails closed with a named error rather than queueing. +- **Lifetime.** A child is owned by the fiber that spawned it. The fiber + unwinding for any reason — actor death, engine stop — kills and reaps the + child. Engine stop kills every registered child before the process exits: + iteration 40's drain guarantee extends to subprocesses. A zombie or an + orphan is a bug by definition. + +**Call surface.** `proc.run(cmd, args)` keeps its shape and its `Proc` +record (`code`, `out`, `err`). Bare calls get the named defaults: 30 s +deadline, 1 MiB stdout cap, 64 KiB stderr cap. An extended form states +`deadline_ms`, `out_cap` and `err_cap` per call — deadline is intrinsically +per-call (a browser run and a `true` are not the same ask), which is why the +bounds do not live in `wo.toml`. Precedent: mailbox caps — a default that a +declaration can override. + +**Argv only, no shell form, ever.** The capability a shell would add +already exists through argv (a caller can spawn `sh` with `-c` explicitly, +owning the quoting risk by having typed it); a shell builtin adds nothing +but an injection-shaped API, and consumer #1 is a *confined* script runner. +The existing argv limits (62 arguments, 512 bytes each) stay — nothing +asked for more. + +**Env and cwd stay inherited, zero knobs.** No consumer demands control; +28's confinement story owns env scrubbing when it arrives, and a clean-env +knob has hidden teeth (`execvp` path search dies with an emptied +environment). Recorded as the skillhost follow-up, not built. + +## Mechanics + +**Parking, not blocking.** The builtin becomes a parking builtin in the +iteration 35 mould (the `_dl` net family): after the fork it registers the +two pipe read ends, a pidfd for the child (`pidfd_open`), and a deadline +timer with the shard's io_uring, then parks the calling fiber. The shard +thread never waits on a child. Readiness events append into growable +buffers up to the cap; the pidfd firing means the child exited (reap, +finish, unpark); the timer firing first means deadline (kill via +`pidfd_send_signal`, reap, unpark with the error). Both pipes are serviced +concurrently by readiness, which structurally removes the suspected +sequential-drain deadlock. + +**Why pidfd and not SIGCHLD or a reaper thread.** A pidfd is a pollable fd +— exactly what the runtime's loop already multiplexes — with no +process-wide signal handler, no delivery-shard question, no masking rules, +and no race between concurrent children. The general signals-as-events +seam stays deferred by name to its real consumer (the multiplexer study's +stage C). A reaper thread would introduce a thread class the runtime does +not have, for no gain over the fd. Kernel floor: `pidfd_open` is Linux +5.3+, well under the runtime's existing io_uring floor. + +**The child registry.** Each shard keeps a registry of its live children +(pid + pidfd + owning fiber). Spawn checks the ceiling against it and fails +closed at 32. Engine stop walks it and kills before exit. Fiber unwind +(actor death, cancellation) kills its own entry through the same path. One +mechanism, three callers. + +**Builtin ids.** `WO_B_PROC_RUN` keeps its id and gains the parking +semantics and defaults. The extended bounded form takes the next free id +from 96 up (89/90 remain iteration 31's reserved holes). No `.wob` version +bump — iteration 38's precedent: new builtin ids alone do not move the +format. + +## Acceptance criteria + +- A child that exits normally: exit code, stdout and stderr surface as + values — the existing behavior, now from a parked fiber; a second actor + on the same shard demonstrably makes progress while a slow child runs. +- A child that outruns its deadline: caller resumes with the named timeout + error, and the pid is verified absent from the process table — measured, + not assumed. +- A child that exceeds an output cap: refusal names the cap and its value; + the child is dead; no fd leaked. +- The deadlock repro: a child that fills stdout past the pipe capacity + while holding stderr open. The test is written against the CURRENT code + first and must fail (hang) there; it passes under the new mechanics. +- The drain leg: SIGTERM to a program with live children — every child is + terminated before the program exits. Iteration 40's battery gains this + leg. +- The ceiling leg: the 33rd concurrent spawn on one shard fails closed + with the named error while the 32 live ones are unaffected. +- One thousand sequential spawns: fd count flat (the iteration 24 + measurement style), rss not growing with the loop. +- A runnable example under `docs/examples/subprocess` behind a `just` + recipe, logging to `/tmp/.log` per convention — the gate is the + acceptance, unit tests alone are not. + +## Out of scope (each with its reason) + +- **The streaming form** — long-lived children with output as mailbox + messages. The follow-up this slice deliberately excludes; tmux, alacritty + and the zen CDP driver queue behind it. The registry, pidfd wait and + cap machinery built here are its foundation. +- **PTY allocation** — pipes only; the pseudo-terminal surface (openpty, + resize ioctls) is the alacritty/tmux stage-A follow-up. +- **A general signal API** — deferred to its real consumer; this slice's + child-exit path needs no signals at all. +- **Env/cwd control** — skillhost confinement's item, recorded above. +- **stdin transport** — no consumer in this slice's set feeds a child + interactively; the streaming form owns it. +- **Pipelines between children** — composition waits for a consumer. +- **A shell convenience form** — refused permanently, not deferred.