writeonce/docs/superpowers/specs/2026-09-01-bounded-subprocess-design.md
shoney.arickathil 8bcd24969e docs(lang42): story, spec and plan for bounded subprocess
- claim the lang42 prefix; iteration 42 story (readiness: ready), approved
  spec, and the 11-task implementation plan
- board: pending row for 42; graph: node 42 with green edges (11, 24)
- graph: porch track section added (same sweep)
- parity studies that motivated 42: alacritty, tmux, zen-browser under
  docs/plan/exploration/ — staged path, gap lists, refused routes

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-01 22:19:25 +02:00

7.8 KiB

Bounded subprocess v1 — the one-shot form, bounded and parked

Brainstormed and approved 2026-09-01. Story: language 42. Scope settled during brainstorm: bound the existing one-shot proc.run only; the streaming form (long-lived child, output as mailbox messages) is the named follow-up. Exit notification via pidfd, no signal seam. Bounds as defaults plus per-call override, refusal by name. Argv only, no shell form ever. Env and cwd stay inherited, no knobs.

The problem

proc.run exists (runtime/src/sysio.c, WO_B_PROC_RUN) but is bounded in nothing that matters and blocks in the one place it must not: it runs the fork, the pipe drain, and the waitpid on the shard thread itself, so one slow child stalls every actor and fiber on that shard. It has no deadline — a child that never exits parks the shard forever, and the waitpid loop notices an engine stop only if a signal happens to interrupt it. Its output caps are fixed constants (8,192 bytes stdout, 4,096 stderr) that truncate silently. Nothing reaps a child when the engine stops. And the sequential drain (stdout to cap, then stderr to EOF) is suspected of deadlocking against a child that fills the stdout pipe while holding stderr open — to be proven by a failing test before the fix is claimed.

Four consumers arrived at once — iteration 28's script runner and the alacritty/tmux/zen-browser exploration studies — all bottoming out on this as their first domino.

The model, and why

Bounded means: every resource a child can consume carries a declared ceiling, and exceeding one is a refusal that names the ceiling — never a hang, a truncation, or an OOM. Same fail-closed posture as porch 1's pool saturation and iteration 31's mailbox caps.

  • Time. Every run has a deadline. At the deadline the child is killed, reaped, and the caller gets a catchable error naming the deadline and its value.
  • Output bytes. Stdout and stderr each carry a cap. Past a cap the child is killed and the error names the cap — silent truncation is removed, which is the one observable behavior change for existing callers (code relying on truncation was relying on a bug).
  • Concurrency. A per-shard ceiling on live children (default 32). At the ceiling a spawn fails closed with a named error rather than queueing.
  • Lifetime. A child is owned by the fiber that spawned it. The fiber unwinding for any reason — actor death, engine stop — kills and reaps the child. Engine stop kills every registered child before the process exits: iteration 40's drain guarantee extends to subprocesses. A zombie or an orphan is a bug by definition.

Call surface. proc.run(cmd, args) keeps its shape and its Proc record (code, out, err). Bare calls get the named defaults: 30 s deadline, 1 MiB stdout cap, 64 KiB stderr cap. An extended form states deadline_ms, out_cap and err_cap per call — deadline is intrinsically per-call (a browser run and a true are not the same ask), which is why the bounds do not live in wo.toml. Precedent: mailbox caps — a default that a declaration can override.

Argv only, no shell form, ever. The capability a shell would add already exists through argv (a caller can spawn sh with -c explicitly, owning the quoting risk by having typed it); a shell builtin adds nothing but an injection-shaped API, and consumer #1 is a confined script runner. The existing argv limits (62 arguments, 512 bytes each) stay — nothing asked for more.

Env and cwd stay inherited, zero knobs. No consumer demands control; 28's confinement story owns env scrubbing when it arrives, and a clean-env knob has hidden teeth (execvp path search dies with an emptied environment). Recorded as the skillhost follow-up, not built.

Mechanics

Parking, not blocking. The builtin becomes a parking builtin in the iteration 35 mould (the _dl net family): after the fork it registers the two pipe read ends, a pidfd for the child (pidfd_open), and a deadline timer with the shard's io_uring, then parks the calling fiber. The shard thread never waits on a child. Readiness events append into growable buffers up to the cap; the pidfd firing means the child exited (reap, finish, unpark); the timer firing first means deadline (kill via pidfd_send_signal, reap, unpark with the error). Both pipes are serviced concurrently by readiness, which structurally removes the suspected sequential-drain deadlock.

Why pidfd and not SIGCHLD or a reaper thread. A pidfd is a pollable fd — exactly what the runtime's loop already multiplexes — with no process-wide signal handler, no delivery-shard question, no masking rules, and no race between concurrent children. The general signals-as-events seam stays deferred by name to its real consumer (the multiplexer study's stage C). A reaper thread would introduce a thread class the runtime does not have, for no gain over the fd. Kernel floor: pidfd_open is Linux 5.3+, well under the runtime's existing io_uring floor.

The child registry. Each shard keeps a registry of its live children (pid + pidfd + owning fiber). Spawn checks the ceiling against it and fails closed at 32. Engine stop walks it and kills before exit. Fiber unwind (actor death, cancellation) kills its own entry through the same path. One mechanism, three callers.

Builtin ids. WO_B_PROC_RUN keeps its id and gains the parking semantics and defaults. The extended bounded form takes the next free id from 96 up (89/90 remain iteration 31's reserved holes). No .wob version bump — iteration 38's precedent: new builtin ids alone do not move the format.

Acceptance criteria

  • A child that exits normally: exit code, stdout and stderr surface as values — the existing behavior, now from a parked fiber; a second actor on the same shard demonstrably makes progress while a slow child runs.
  • A child that outruns its deadline: caller resumes with the named timeout error, and the pid is verified absent from the process table — measured, not assumed.
  • A child that exceeds an output cap: refusal names the cap and its value; the child is dead; no fd leaked.
  • The deadlock repro: a child that fills stdout past the pipe capacity while holding stderr open. The test is written against the CURRENT code first and must fail (hang) there; it passes under the new mechanics.
  • The drain leg: SIGTERM to a program with live children — every child is terminated before the program exits. Iteration 40's battery gains this leg.
  • The ceiling leg: the 33rd concurrent spawn on one shard fails closed with the named error while the 32 live ones are unaffected.
  • One thousand sequential spawns: fd count flat (the iteration 24 measurement style), rss not growing with the loop.
  • A runnable example under docs/examples/subprocess behind a just recipe, logging to /tmp/<app>.log per convention — the gate is the acceptance, unit tests alone are not.

Out of scope (each with its reason)

  • The streaming form — long-lived children with output as mailbox messages. The follow-up this slice deliberately excludes; tmux, alacritty and the zen CDP driver queue behind it. The registry, pidfd wait and cap machinery built here are its foundation.
  • PTY allocation — pipes only; the pseudo-terminal surface (openpty, resize ioctls) is the alacritty/tmux stage-A follow-up.
  • A general signal API — deferred to its real consumer; this slice's child-exit path needs no signals at all.
  • Env/cwd control — skillhost confinement's item, recorded above.
  • stdin transport — no consumer in this slice's set feeds a child interactively; the streaming form owns it.
  • Pipelines between children — composition waits for a consumer.
  • A shell convenience form — refused permanently, not deferred.