docs(lang42): story, spec and plan for bounded subprocess
- claim the lang42 prefix; iteration 42 story (readiness: ready), approved spec, and the 11-task implementation plan - board: pending row for 42; graph: node 42 with green edges (11, 24) - graph: porch track section added (same sweep) - parity studies that motivated 42: alacritty, tmux, zen-browser under docs/plan/exploration/ — staged path, gap lists, refused routes Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
parent
bed1167ca9
commit
8bcd24969e
9 changed files with 953 additions and 1 deletions
|
|
@ -58,7 +58,8 @@ flowchart TD
|
|||
I38["38 content platform capabilities: fs mutation verbs + net.connect"]:::open
|
||||
I9g["27 query grammar corpus (⏸ hold; likely collapses)"]:::parked
|
||||
I30["30 observability, CI, fuzz — release-only CI exists; per-change gates + fuzz open (no story file)"]:::open
|
||||
GAPS["28's gap fan-out: bounded subprocess, stdin/stdout transport, fs metadata, FFI-vs-out-of-process"]:::open
|
||||
GAPS["28's gap fan-out, what is LEFT of it: fs metadata, FFI-vs-out-of-process (bounded subprocess + stdio transport moved to 42)"]:::open
|
||||
I42["42 bounded subprocess: bound proc.run (deadline, caps, fiber-parked), streaming form, owner-bound reaping"]:::open
|
||||
DRAIN["parked drain, what is LEFT of it: WO-E225 roster, ADT roster, group-by aggregates"]:::parked
|
||||
|
||||
FOUND --> I7
|
||||
|
|
@ -98,6 +99,8 @@ flowchart TD
|
|||
I9g --> I28
|
||||
I7 --> I28
|
||||
I28 --> GAPS
|
||||
I11 --> I42
|
||||
I24 --> I42
|
||||
I16 --> I38
|
||||
I32 --> I38
|
||||
I36 -.reopens the pure-wo HMAC question.-> I34
|
||||
|
|
@ -269,6 +272,51 @@ critical path (the only engine + language work); jobs compose on it; the
|
|||
demo and gate close it. Fiber-scheduled jobs and cancellation→rollback
|
||||
appear in graph 2 — they need iteration 11 as well as 18.
|
||||
|
||||
## 5. porch — the web framework track
|
||||
|
||||
States live on [the board's porch section](stories/00-status.md); every
|
||||
pending node is `readiness: refine` (no approved spec), so a brainstorm
|
||||
precedes any plan. Three independent roots: **2** (the auth chain),
|
||||
**6** (the streaming chain), **5** (anytime, no incoming edges). **9** is
|
||||
the track's only `readiness: ready` item and is held.
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
classDef done fill:#1a7f37,color:#fff,stroke:none
|
||||
classDef refine fill:#eac54f,color:#000,stroke:none
|
||||
classDef held fill:#6e7781,color:#fff,stroke:none
|
||||
classDef lang fill:#8250df,color:#fff,stroke:none
|
||||
|
||||
CSPRNG["language track: CSPRNG builtin (id 96+) — porch 2's Phase A"]:::lang
|
||||
P1["porch 1 store-backed middleware ✅ 2026-08-30"]:::done
|
||||
P2["porch 2 randomness + cookies (repeated Set-Cookie/Vary headers, Cookie: parsing, signed cookies)"]:::refine
|
||||
P3["porch 3 sessions"]:::refine
|
||||
P4["porch 4 CSRF"]:::refine
|
||||
P5["porch 5 routing + response ergonomics"]:::refine
|
||||
P6["porch 6 streaming core"]:::refine
|
||||
P7["porch 7 SSE + compression"]:::refine
|
||||
P8["porch 8 static files + lifecycle"]:::refine
|
||||
P9["porch 9 idempotent replay (⏸ hold; readiness: ready)"]:::held
|
||||
|
||||
CSPRNG --> P2
|
||||
P2 --> P3
|
||||
P2 --> P4
|
||||
P3 --> P4
|
||||
P6 --> P7
|
||||
P6 --> P8
|
||||
P2 --> P7
|
||||
P5 --> P7
|
||||
P1 -.re-scope 79e6da4: replay-on-retry split out of 1.-> P9
|
||||
```
|
||||
|
||||
The two non-obvious edges are stated in porch 7's own story: gzip's
|
||||
`Accept-Encoding` negotiation reuses the q-value ranking iteration 5 adds,
|
||||
and `Vary: Accept-Encoding` needs iteration 2's repeated-header work to
|
||||
accumulate with other `Vary` contributions. Porch 2's Phase A is
|
||||
language-track work (the CSPRNG builtin) — the cross-track edge this graph
|
||||
exists to make visible. Streaming responses' runtime prerequisites
|
||||
(fibers, iteration 11) are already green in graph 2.
|
||||
|
||||
## Maintenance rule
|
||||
|
||||
When an iteration or slice lands, update its node's class here in the
|
||||
|
|
|
|||
|
|
@ -43,6 +43,7 @@ features cannot collide.
|
|||
| `lang41` | runtime: unadopted shard must not impersonate shard 0 | on `dev` (`9dca0b4`); independent of the residency stack, not picked |
|
||||
| `porch-store` | porch store tables, Limiter and Idempotent middleware (Phases A, B, C) | on `dev` (`519d411`, `5b1e82a`, `aee7926`). **In progress**: Phase C was uncommitted work from a parallel session, committed as-is, and calls `json.decode`/`json.encode` with no `use json` import |
|
||||
| `query-corpus` | databasev2 query-grammar corpus #1 | on `dev` (`4c82461`). Conclusion was "no new grammar needed" |
|
||||
| `lang42` | iteration 42 — bounded subprocess: `proc.run` bounded + parked (pidfd, caps, ceiling, owner-bound reaping), `proc.run_dl`; carries the alacritty/tmux/zen parity studies and the porch dependency-graph section from the same sweep | on `dev` 2026-09-01 |
|
||||
|
||||
## Cherry-picks onto master
|
||||
|
||||
|
|
|
|||
123
docs/plan/exploration/alacritty/00-alacritty-parity.md
Normal file
123
docs/plan/exploration/alacritty/00-alacritty-parity.md
Normal file
|
|
@ -0,0 +1,123 @@
|
|||
# Alacritty parity — what the language needs to build a terminal-class application
|
||||
|
||||
Source: [`.dev/reference/alacritty/`](../../../../.dev/reference/alacritty/)
|
||||
(shallow clone, surveyed 2026-09-01). Companion study to
|
||||
[`fiber/00-fiber-parity.md`](../fiber/00-fiber-parity.md), which asked the same
|
||||
question for the web-framework surface. This one asks it for a native,
|
||||
event-driven, GPU-rendered desktop application — the workload class writeonce
|
||||
is furthest from today.
|
||||
|
||||
## What alacritty actually is (measured, not summarized from its README)
|
||||
|
||||
33,699 lines of Rust across a 4-crate workspace, ~22 direct dependencies:
|
||||
|
||||
- **`alacritty_terminal`** — the portable core, no GUI anywhere in it: PTY
|
||||
creation and lifecycle (`tty/unix.rs`, 448 lines — openpty, fork/exec of the
|
||||
shell, window-size ioctls, child reaping), a byte-level escape-sequence
|
||||
parser (the `vte` crate), the grid data structure, and an event loop over
|
||||
the `polling` crate (epoll/kqueue on arbitrary fds).
|
||||
- **`alacritty`** — the application: windowing (`winit`), OpenGL context
|
||||
creation (`glutin`, EGL/WGL), glyph rasterization (`crossfont`), a
|
||||
timer-driven scheduler (`scheduler.rs` — cursor blink, repaint deadlines),
|
||||
config with live reload (`notify` file watching), clipboard, signal
|
||||
handling (`signal-hook`), CLI (`clap`).
|
||||
- **`alacritty_config`/`_derive`** — config plumbing (a proc-macro crate;
|
||||
writeonce's analogue would be compile-time codegen, which `@table` already
|
||||
does for a different domain).
|
||||
|
||||
The split matters more than the line count: the terminal CORE is a
|
||||
files-and-processes program with zero graphics, and the GUI shell around it is
|
||||
where every heavyweight dependency lives.
|
||||
|
||||
## What already fits — the part that costs nothing
|
||||
|
||||
- **The concurrency model is a better fit than alacritty's own.** Alacritty
|
||||
runs a PTY-reader thread and a window-event thread synchronized through
|
||||
`parking_lot` locks around the grid. Shard actors with ownership-move
|
||||
messages express this without shared state: a reader actor owns the parser,
|
||||
sends grid deltas; a display actor owns the grid. Iteration 24's chat app
|
||||
already proved the shape (fd-driven actor, fan-out, lifecycle).
|
||||
- **Single self-contained binary** — alacritty's distribution story is
|
||||
writeonce's existing one.
|
||||
- **Bytes + bitwise operators** (iterations 19, 36) — the VTE parser is a byte
|
||||
state machine; the primitive layer for it exists.
|
||||
- **`fn main`, exit codes, env, fs, time** (plan 9 stdlib) — config discovery,
|
||||
CLI-shaped startup.
|
||||
|
||||
## The gaps, ordered by what unblocks what
|
||||
|
||||
Each maps onto an existing iteration where one exists; only two items are
|
||||
genuinely new.
|
||||
|
||||
1. **PTY + bounded subprocess** — the heart of the core. Openpty, fork/exec
|
||||
with the child on the slave side as its controlling terminal, resize
|
||||
ioctls, orderly child shutdown. This is exactly iteration 28's named gap
|
||||
fan-out ("bounded subprocess, stdin/stdout transport") plus the PTY-specific
|
||||
ioctls. ~450 lines of Rust in the reference; a C-builtin family in
|
||||
writeonce's existing style (crypto/net precedent: small id-numbered
|
||||
builtins, no general FFI).
|
||||
2. **Readiness on arbitrary fds.** The runtime's io_uring loop watches sockets
|
||||
it created. A PTY master fd — and later a display-server fd — must be
|
||||
registrable in the same loop, parking the owning fiber until readable.
|
||||
Alacritty needs nothing fancier (its `polling` crate is the same shape);
|
||||
this is a seam widening, not a new subsystem.
|
||||
3. **Signals as events.** SIGCHLD (child died) and SIGWINCH (resize) must
|
||||
arrive as mailbox messages, the way iteration 24 handles fd events. Today
|
||||
signals are runtime-internal (SIGTERM drain). Small, but nothing else can
|
||||
substitute for it.
|
||||
4. **Unicode width + UTF-8 decode in the stdlib.** The grid is addressed in
|
||||
cells; every printed byte-run needs "how many columns". A data-table
|
||||
problem, not a design problem — but without it a terminal misrenders
|
||||
immediately. (Alacritty: `unicode-width` crate.)
|
||||
5. **`time.mono` returns, plus a timer wheel.** Cut 2026-08-10 with "returns
|
||||
when a workload needs monotonic math" — this workload is that consumer.
|
||||
`time.after` (id 90) exists; a repaint/blink scheduler needs monotonic
|
||||
deadlines that survive wall-clock jumps.
|
||||
6. **Outbound connection + fd passing + shared-memory buffers.** Iteration 38
|
||||
already names `net.connect`. Wayland is *a unix-socket protocol*: with
|
||||
connect, SCM_RIGHTS fd passing, and an mmap/shm builtin, a Wayland client
|
||||
with software rendering (wl_shm) is expressible in pure `.wo` — a windowed
|
||||
terminal with **no C dependency linked at all**. This is the outside-the-box
|
||||
route the reference makes visible: alacritty predates it culturally (X11
|
||||
era) and pays for GL instead.
|
||||
7. **The GPU fork — decide late.** OpenGL/EGL and font rasterization
|
||||
(freetype/fontconfig) cannot be spoken over a socket; they are C ABI or
|
||||
nothing. Three options, same fork iteration 28 already recorded as
|
||||
"FFI-vs-out-of-process": (a) general FFI — largest doctrine change, rejected
|
||||
until a second consumer demands it; (b) subsystem C builtins (the crypto
|
||||
precedent) — a `gfx`/`font` builtin family; (c) an out-of-process render
|
||||
server writeonce talks to over its own socket — fits the actor model,
|
||||
keeps the language pure, costs a second process. Software rendering via
|
||||
route 6 defers this fork entirely: glyph rasterization from a pre-baked
|
||||
bitmap font atlas is pure byte math.
|
||||
|
||||
## The maturity path, as driving workloads (the project's own method)
|
||||
|
||||
Each stage is a shippable proof, ordered so no stage waits on the fork in 7:
|
||||
|
||||
- **Stage A — headless terminal**: PTY builtins + signals + fd readiness
|
||||
(items 1–3). Proof: a `.wo` program spawns a shell, feeds it a script,
|
||||
captures and asserts the output — a `script`/`expect` clone. Closes
|
||||
iteration 28's bounded-subprocess gap as a side effect.
|
||||
- **Stage B — VTE grid**: parser + grid in pure `.wo` (item 4). Proof: replay
|
||||
recorded terminal sessions (vttest, asciinema casts) and assert final grid
|
||||
state against the reference implementation's.
|
||||
- **Stage C — multiplexer**: A + B + existing net = a tmux-lite: sessions
|
||||
survive detach, clients attach over a unix socket. No graphics, real
|
||||
product, exercises everything server-side writeonce is already good at.
|
||||
*Corrected by the tmux study (2026-09-01): this stage also needs
|
||||
SCM_RIGHTS fd passing (item 6's builtin, promoted here), termios adoption
|
||||
of the client's own tty, and the terminfo fork — see
|
||||
[`../tmux/00-tmux-parity.md`](../tmux/00-tmux-parity.md).*
|
||||
- **Stage D — windowed, software-rendered**: item 6 (connect, fd passing,
|
||||
shm) + a Wayland client library in `.wo`. First pixel on screen with zero
|
||||
linked C.
|
||||
- **Stage E — GPU**: only now decide item 7, with D as the measured baseline
|
||||
that says whether GL is worth an FFI doctrine change.
|
||||
|
||||
## What this does NOT recommend
|
||||
|
||||
No general FFI now (one consumer, and stages A–D never need it); no bundled
|
||||
font rasterizer until D shows bitmap atlases failing; no attempt at winit-class
|
||||
cross-platform windowing — Linux/Wayland first, the same way the runtime is
|
||||
Linux/io_uring first.
|
||||
104
docs/plan/exploration/tmux/00-tmux-parity.md
Normal file
104
docs/plan/exploration/tmux/00-tmux-parity.md
Normal file
|
|
@ -0,0 +1,104 @@
|
|||
# tmux parity — the multiplexer as the language's next driving workload
|
||||
|
||||
Source: [`.dev/reference/tmux/`](../../../../.dev/reference/tmux/) (shallow
|
||||
clone, surveyed 2026-09-01). Companion to
|
||||
[`alacritty/00-alacritty-parity.md`](../alacritty/00-alacritty-parity.md),
|
||||
whose stage C ("tmux-lite") this study scopes for real. tmux is the more
|
||||
instructive reference of the two: it needs **no graphics at all**, so the
|
||||
entire program sits inside the territory the alacritty study's stages A–C
|
||||
cover — it is the proof target, not a stepping stone to one.
|
||||
|
||||
## What tmux actually is (measured)
|
||||
|
||||
~112,000 lines of C (100,027 in the 153 top-level `.c` files; the rest is
|
||||
`compat/` shims and headers). Exactly **two** external dependencies —
|
||||
libevent (fd loop + buffers) and ncurses used *only* as a terminfo(5) reader
|
||||
(`tty-term.c` calls `setupterm`/`tigetstr` and nothing curses-like). Every
|
||||
other need is vendored in `compat/`: OpenBSD's imsg framing, `forkpty` for
|
||||
five platforms, `daemon`.
|
||||
|
||||
The architecture is one daemonized **server** owning every session, window,
|
||||
pane and PTY (`server.c:190` forks it), and a deliberately thin **client**
|
||||
(808 lines): the client connects over a unix socket and **passes its own
|
||||
terminal fd to the server** with SCM_RIGHTS (`compat/imsg-buffer.c:798`,
|
||||
`proc.c` — `imsgbuf_allow_fdpass`); from then on the server writes escape
|
||||
sequences directly to the client's tty. Detach survives because the client
|
||||
process is disposable — the server never was attached to a terminal it
|
||||
doesn't hold as a passed fd.
|
||||
|
||||
Where the lines actually go — the multiplexer kernel is small, the UX is not:
|
||||
|
||||
- **Kernel, ~27k**: `server-client.c` 3,280 · `input.c` 3,745 (the escape
|
||||
parser) · `tty.c` 3,221 (the output driver) · `screen-write.c` 3,204 ·
|
||||
`window.c` 2,893 · `grid.c` 1,839 · `layout.c` 2,022 (the pane split
|
||||
tree) · `tty-keys.c` 1,883 (key decoding) · `utf8.c` 1,043 · plus
|
||||
session/spawn/proc/job plumbing.
|
||||
- **A command LANGUAGE, ~12k**: 63 `cmd-*.c` files behind a yacc grammar
|
||||
(`cmd-parse.y`) — the config file is just commands; `format.c` is 7,182
|
||||
lines implementing a template DSL with ~380 variables; `options-table.c`
|
||||
2,070 lines of typed settings.
|
||||
- **Interactive UX, ~20k**: `window-copy.c` 7,300 (copy mode is the single
|
||||
biggest file in tmux) · choose-tree/customize/menus/prompt.
|
||||
|
||||
## What writeonce already answers
|
||||
|
||||
- **Client/server over a unix socket** — `listen_unix` landed (iteration 35);
|
||||
the request/actor shape is the porch daily bread.
|
||||
- **Server = actor tree.** tmux multiplexes everything through one
|
||||
single-threaded libevent loop with callbacks; sessions, windows, panes and
|
||||
clients as actors with ownership-move messages is the same topology with
|
||||
the concurrency written down instead of implied. Iteration 24 (chat rooms,
|
||||
fan-out, lifecycle, SIGTERM drain) already proved every piece of the
|
||||
pattern.
|
||||
- **The DSL layer is free.** tmux hand-rolls a command grammar, a format
|
||||
template language and a typed options table (~12k lines) because C has no
|
||||
expression language to lend. writeonce's config/scripting surface can be
|
||||
the language itself — `${ }` interpolation replaces `format.c`, and a
|
||||
`@table` of settings replaces `options-table.c`.
|
||||
- **Durable sessions beyond tmux.** tmux state dies with the server; a
|
||||
writeonce multiplexer's session/layout tables can be `durable: true` for
|
||||
free — scrollback in the WAL is the kind of trick the storage engine
|
||||
exists for. Feature, not parity.
|
||||
|
||||
## Gaps — mostly shared with the alacritty list, two new, one promoted
|
||||
|
||||
1. **PTY + bounded subprocess, signals, arbitrary-fd readiness, unicode
|
||||
width, monotonic timers** — identical to alacritty study gaps 1–5;
|
||||
`spawn.c`/`job.c`/`compat/fdforkpty.c` are the reference reading.
|
||||
2. **fd passing (SCM_RIGHTS) — PROMOTED.** The alacritty study placed it in
|
||||
stage D (Wayland). Wrong stage: detach/attach — the whole point of a
|
||||
multiplexer — is built on handing the client's tty fd across a unix
|
||||
socket. Second consumer found; the builtin belongs to the multiplexer
|
||||
stage. (The alacritty study is corrected in place.)
|
||||
3. **NEW — termios control of an existing terminal.** The client must put
|
||||
*its own* tty into raw mode and restore it on exit (tmux:
|
||||
`cfmakeraw`, `tcgetattr`/`tcsetattr`). The PTY gap covers creating
|
||||
terminals; this is adopting one you were given. Small builtin family,
|
||||
nothing else substitutes.
|
||||
4. **NEW — the terminfo fork.** tmux answers "what escape sequences does
|
||||
THIS client's terminal speak" from the terminfo database. Two honest
|
||||
options: read the compiled terminfo format in pure `.wo` (a documented
|
||||
binary file — a parser, not a linked library; ncurses would NOT be
|
||||
imported) or emit a fixed xterm-256color profile and refuse exotic
|
||||
terminals by name. Decide at brainstorm; start fixed, the refusal names
|
||||
the gap.
|
||||
5. **Daemonization** — fork-and-detach with the socket handed over. Cheap,
|
||||
and arguably skippable first (a foreground server under systemd was good
|
||||
enough for writeonce.de).
|
||||
|
||||
## Corrected staged path (supersedes the alacritty study's stage C sizing)
|
||||
|
||||
- **Stage A/B unchanged** — headless PTY runner, then the VTE grid replayed
|
||||
against recorded sessions (`input.c` + `grid.c` are the behaviours to pin).
|
||||
- **Stage C — the multiplexer** now carries its real bill: A + B **plus**
|
||||
fd passing (gap 2), termios adoption (gap 3), and the terminfo decision
|
||||
(gap 4). Proof: detach, kill the client, reattach from another terminal,
|
||||
scrollback intact — then restart the *server* and reattach with layout and
|
||||
scrollback replayed from the WAL, which is the demo tmux cannot give.
|
||||
- **Stage D/E unchanged** (Wayland shm, then the GPU fork) — and stage D
|
||||
gets gap 2 for free once C lands.
|
||||
|
||||
Scope honesty: parity with tmux the product is ~100k lines including a 7k
|
||||
copy mode and 20k of chooser UX — not the goal. The kernel a driving
|
||||
workload needs is the ~27k-line column, and the DSL third of tmux dissolves
|
||||
into language features writeonce already has.
|
||||
86
docs/plan/exploration/zen-browser/00-zen-browser-parity.md
Normal file
86
docs/plan/exploration/zen-browser/00-zen-browser-parity.md
Normal file
|
|
@ -0,0 +1,86 @@
|
|||
# Zen Browser parity — what a browser-class application actually asks of a language
|
||||
|
||||
Source: [`.dev/reference/zen-browser/`](../../../../.dev/reference/zen-browser/)
|
||||
(shallow clone of zen-browser/desktop, surveyed 2026-09-01). Third study in the
|
||||
series after [`alacritty`](../alacritty/00-alacritty-parity.md) and
|
||||
[`tmux`](../tmux/00-tmux-parity.md) — and the one whose answer is different in
|
||||
kind, which is why it is worth having.
|
||||
|
||||
## What Zen actually is (measured)
|
||||
|
||||
Zen is not a browser codebase. It is a **Firefox overlay**: `surfer.json` pins
|
||||
`product: firefox, version: 154.0.1`, `npm run download` fetches the Firefox
|
||||
source into `engine/` at build time, and the repo contributes **256 patch
|
||||
files** plus a 33 MB `src/` tree that is copied over it — ~246,000 lines
|
||||
counting JS/MJS/CSS/XHTML/patches (616 `.js` files, 568 Fluent localization
|
||||
files, 48 CSS). There is no rendering, layout, JS-engine, or networking code
|
||||
in the repository at all; Gecko (tens of millions of lines of C++/Rust)
|
||||
arrives as a downloaded dependency and is built with Mozilla's own `mach`.
|
||||
|
||||
Zen's value-add lives in `src/zen/`: workspaces ("spaces"), split view,
|
||||
compact mode, glance, folders, session store, sync — all UI composition and
|
||||
state, written in JavaScript **because the engine ships a JavaScript host and
|
||||
Zen's code runs inside it**. The engine is also the app platform.
|
||||
|
||||
## The lesson, stated plainly
|
||||
|
||||
Nobody — including a successful, well-staffed browser project — writes a
|
||||
browser. They skin an engine. So "mature the language until it can build an
|
||||
application such as this" resolves to three different questions, and only one
|
||||
of them is real work for writeonce:
|
||||
|
||||
1. **Build the engine in `.wo`?** Not a maturity path, a decades-long refusal.
|
||||
The gap list it produces ("general FFI, C++ interop, a GPU pipeline, a JS
|
||||
VM…") is the alacritty study's stage-E fork multiplied by a thousand, with
|
||||
no intermediate shippable stage. Rejected as a driving workload — it cannot
|
||||
drive, only sink.
|
||||
2. **Embed an engine (CEF/WebKitGTK) behind FFI?** The heaviest possible FFI
|
||||
consumer. Same fork as alacritty item 7, and this study deliberately does
|
||||
not promote it: an embedded engine's API surface is enormous and unstable,
|
||||
the worst first FFI customer imaginable.
|
||||
3. **Drive an engine out-of-process.** Chromium and Firefox both expose a
|
||||
remote-debugging protocol (CDP; Firefox now speaks it too) over a local
|
||||
socket or a stdio pipe. A writeonce program that spawns a browser, connects,
|
||||
and drives it — kiosk shells, scrapers, site-acceptance drivers of the kind
|
||||
`site-accept.sh` fakes with curl today — is the browser-class workload that
|
||||
is actually reachable, and it is exactly the actor/protocol shape the
|
||||
runtime is built around. This is Playwright's architecture, minus the
|
||||
Node.js.
|
||||
|
||||
## What route 3 needs — mostly the existing list, one genuinely new gap
|
||||
|
||||
- **Bounded subprocess with stdio transport** — spawning the browser with a
|
||||
pipe transport is the third consumer of iteration 28's named gap (after the
|
||||
alacritty and tmux studies' PTY stages; this one does not even need a PTY).
|
||||
- **`net.connect`** — iteration 38, already named, for the debugging socket.
|
||||
- **NEW — WebSocket CLIENT.** CDP speaks WebSocket; iteration 24 landed the
|
||||
server side (`ws_accept`, frames) but nothing performs an outbound upgrade
|
||||
handshake. Small — the frame code is the hard half and already exists.
|
||||
- **JSON both ways** — exists. Actor-per-session/per-tab supervision, timeouts
|
||||
(`time.after`), graceful teardown — all landed in iteration 24.
|
||||
|
||||
Notably absent: nothing graphical, no terminfo, no termios, no fd passing.
|
||||
Route 3 is *closer* than the tmux stage.
|
||||
|
||||
## What Zen's own feature layer says about writeonce
|
||||
|
||||
Zen's ~250k-line overlay is session stores, workspace state, sync, and
|
||||
declarative UI — in writeonce terms: `@table` rows (durable by declaration
|
||||
since databasev2 2), and wo-html components. The project's existing bet —
|
||||
that the browser is the *client* and the application lives server-side in
|
||||
porch — already covers the useful half of what Zen builds. The other half of
|
||||
Zen's lesson is about hosting: Gecko wins as a platform because it embeds a
|
||||
scripting language with capability boundaries, which is iteration 28's
|
||||
skillhost direction (writeonce as the confined host, not the confined guest).
|
||||
|
||||
## Verdict for the maturity path
|
||||
|
||||
The browser study adds **one builtin-sized gap** (WebSocket client) and
|
||||
**one workload** to the staged path from the alacritty study — call it
|
||||
**stage C′, the browser driver**, parallel to stage C (it needs subprocess +
|
||||
`net.connect` + ws-client, none of stage C's tty machinery). Proof when it
|
||||
lands: replace a curl leg of `site-accept.sh` with a `.wo` driver that opens
|
||||
writeonce.de in a real browser, asserts the rendered chapter list, and tears
|
||||
the browser down cleanly — the site gate exercising the language's own
|
||||
browser-automation story end to end. Engine-building and engine-embedding
|
||||
stay refused, by name, with this study as the reason.
|
||||
|
|
@ -1048,6 +1048,7 @@ check mode, and the `internal/` dep boundary (WO-E108). Driver-only.
|
|||
| 23 | io_uring group-commit write path — batched durability overlapped on shard threads, fsync fallback | **no spec yet** — brainstorm after iterations 8 + 22 |
|
||||
| 27 | Query grammar from real embedded-DB corpora — whole-query count + correlated exists, driven by the skillhost SQL catalogue; add only what a corpus uses | **no spec yet** — three forks; may collapse to "confirm len(query) + add exists" |
|
||||
| 14 | skillhost host workload — port skillhost (MCP host + confined script runner) to writeonce; drives the missing host capabilities into the open (bounded subprocess, stdin/stdout transport, fs metadata, FFI-vs-out-of-process) | **no spec yet** — gaps recorded in the iteration; each gap brainstormed on demand, bounded-subprocess first |
|
||||
| 42 | [Bounded subprocess](language-runtime-database/42-bounded-subprocess.md) — `proc.run` exists (`sysio.c`) but shard-blocking, deadline-less, silently truncating; this bounds it in place (deadline, output caps, per-shard ceiling, owner-bound reaping via pidfd in the io_uring loop, fiber parked); streaming form deferred by name; 28's leading gap promoted with four consumers | ✅ spec approved 2026-09-01 — [spec](../superpowers/specs/2026-09-01-bounded-subprocess-design.md); `readiness: ready`, plan next |
|
||||
| 17 | library projects + dependency privacy — `wo.toml` kind = "library" (checkable without entry, dual lib+bin) + Go-style `internal/` at the [deps] boundary; framework reorg demonstrates both | ✅ **landed 2026-08-20** — [spec](../superpowers/specs/2026-08-20-library-kind-internal-design.md) · [plan](../superpowers/plans/2026-08-20-library-kind-internal.md) |
|
||||
| 10 | HTTP service layer | [plan 6](../superpowers/plans/2026-08-01-http-service-layer.md) |
|
||||
| 11 | Fibers | vision §3, [blue-green exploration](../plan/exploration/blue-green-vm/00-vision.md) |
|
||||
|
|
|
|||
126
docs/stories/language-runtime-database/42-bounded-subprocess.md
Normal file
126
docs/stories/language-runtime-database/42-bounded-subprocess.md
Normal file
|
|
@ -0,0 +1,126 @@
|
|||
---
|
||||
track: language-runtime-database
|
||||
iteration: "42"
|
||||
status: pending
|
||||
readiness: ready
|
||||
---
|
||||
|
||||
# 42 — bounded subprocess: spawn, supervise, and reap a child process
|
||||
|
||||
> Part of [Story — one language, one runtime, one database](00-story.md).
|
||||
> Spec: [`2026-09-01-bounded-subprocess-design.md`](../../superpowers/specs/2026-09-01-bounded-subprocess-design.md)
|
||||
> (brainstormed and approved 2026-09-01; the six forks below are settled).
|
||||
> First named as iteration [28](28-skillhost-host-workload.md)'s leading gap
|
||||
> ("bounded subprocess, stdin/stdout transport"), promoted to its own
|
||||
> iteration on 2026-09-01 when three further consumers arrived at once — the
|
||||
> [alacritty](../../plan/exploration/alacritty/00-alacritty-parity.md),
|
||||
> [tmux](../../plan/exploration/tmux/00-tmux-parity.md) and
|
||||
> [zen-browser](../../plan/exploration/zen-browser/00-zen-browser-parity.md)
|
||||
> studies all bottom out on it as their first domino.
|
||||
>
|
||||
> **The problem.** A one-shot `proc.run` already exists
|
||||
> (`runtime/src/sysio.c`, `WO_B_PROC_RUN`: fork/execvp on an argv vector,
|
||||
> stdout/stderr captured through pipes, exit code returned) — but it is
|
||||
> bounded in nothing that matters and blocking in the one place it must not
|
||||
> be. It runs on the shard thread, not parked in a fiber, so one slow child
|
||||
> stalls every actor on that shard; it has no deadline, and its `waitpid`
|
||||
> loop notices a runtime stop only if a signal happens to interrupt it; its
|
||||
> output caps (8,192 / 4,096 bytes, fixed) truncate silently rather than
|
||||
> refuse; there is no stdin, no streaming, no concurrency ceiling, and no
|
||||
> story for what happens to a child when its owner dies. Suspected but not
|
||||
> yet reproduced: a child that fills the stdout pipe past the cap while the
|
||||
> parent is waiting for stderr EOF deadlocks both — needs a test before it
|
||||
> is claimed. Every workload class beyond "server that owns all its state"
|
||||
> — a script runner (28), a terminal (alacritty stage A), a multiplexer
|
||||
> (tmux), a browser driver (zen C′) — begins by starting a process and ends
|
||||
> by being responsible for it.
|
||||
|
||||
## Why *bounded* — the doctrine half
|
||||
|
||||
The feature is deliberately not "exec". Every resource a child can consume
|
||||
carries a declared ceiling, and exceeding it is a refusal by name, never a
|
||||
hang or an OOM — the same fail-closed posture porch 1 proved for pool
|
||||
saturation and iteration 31 for mailbox caps:
|
||||
|
||||
- **Time** — a deadline after which the child is killed and the caller
|
||||
resumes with a named timeout error.
|
||||
- **Output bytes** — a cap on captured stdout/stderr; past it the child is
|
||||
killed and the error names the cap, because an unbounded pipe buffer is an
|
||||
unbounded allocation.
|
||||
- **Concurrency** — a ceiling on live children per program; at the ceiling,
|
||||
spawning fails closed rather than queueing invisibly.
|
||||
- **Lifetime** — a child is owned by the actor that spawned it. Owner dies,
|
||||
child is killed and reaped. Program stops, children are terminated before
|
||||
exit (iteration 40's drain guarantee extends to them). A zombie or an
|
||||
orphan is a bug by definition, not a caveat.
|
||||
|
||||
## Info — the forks, settled (brainstorm 2026-09-01)
|
||||
|
||||
1. **Retrofit, one-shot only.** The existing `proc.run` gains the bounds
|
||||
in place (deadline, declared output caps, fiber parking, owner-bound
|
||||
reaping, per-shard ceiling). The streaming form — long-lived child,
|
||||
stdout as mailbox messages, what tmux/alacritty/zen need — is the named
|
||||
follow-up, building on this slice's registry and pidfd machinery.
|
||||
2. **Transport** — died with fork 1: a one-shot form streams nothing.
|
||||
Settled by the follow-up when it arrives.
|
||||
3. **pidfd, no signal seam.** Child exit is a pollable fd (`pidfd_open`,
|
||||
Linux 5.3+) registered in the shard's io_uring beside the pipes and the
|
||||
deadline timer; the fiber parks in the iteration 35 `_dl` mould. The
|
||||
general signals-as-events seam stays deferred by name to its real
|
||||
consumer (the multiplexer study's stage C).
|
||||
4. **Argv only; a shell form is refused permanently.** A caller wanting a
|
||||
shell types `sh -c` into their own argv and owns the quoting risk;
|
||||
consumer #1 is a confined script runner and must never get a
|
||||
shell-shaped API.
|
||||
5. **Env and cwd stay inherited, zero knobs.** No consumer demands
|
||||
control; skillhost confinement owns env scrubbing when it arrives, and
|
||||
a clean-env knob breaks `execvp` path search.
|
||||
6. **Ids.** `WO_B_PROC_RUN` keeps its id and gains parking + defaults; the
|
||||
extended bounded form takes the next free id from 96+ (89/90 remain
|
||||
31's holes); no `.wob` bump (38's precedent).
|
||||
|
||||
Bounds surface: bare `proc.run(cmd, args)` gets named defaults (30 s,
|
||||
1 MiB stdout, 64 KiB stderr); an extended form states them per call;
|
||||
exceeding any bound kills the child and raises a catchable error naming
|
||||
the bound — silent truncation is removed.
|
||||
|
||||
## Acceptance criteria — firmed in the spec, normative form there
|
||||
|
||||
- Given a child that exits normally, when it is run, then its exit code is
|
||||
an ordinary value in the language and both output streams are readable.
|
||||
- Given a child that outruns its deadline, when the deadline passes, then
|
||||
the child is dead (verified by pid absence, not assumed), the caller has a
|
||||
named timeout error, and no fd has leaked.
|
||||
- Given a child whose output exceeds the byte cap, then the refusal names
|
||||
the cap and the child is dead — measured with a deliberately chatty child.
|
||||
- Given an owning actor that dies while its child lives, then the child is
|
||||
reaped — verified from the outside via the process table.
|
||||
- Given SIGTERM to the program while children live, then every child is
|
||||
terminated before the program exits — the drain gate (iteration 40's
|
||||
battery) gains a subprocess leg.
|
||||
- Given a spawn loop of one thousand sequential children, then the
|
||||
program's fd count is flat (the iteration 24 measurement style) and rss
|
||||
does not grow with the loop.
|
||||
- A runnable example under `docs/examples/` with a `just` gate, in the
|
||||
residency/chat mould — the gate is the acceptance, not the unit tests
|
||||
alone.
|
||||
|
||||
## Out of scope, by name
|
||||
|
||||
- **PTY allocation** — pipes only. Giving the child a pseudo-terminal is
|
||||
the alacritty/tmux stage-A follow-up and has its own ioctl surface.
|
||||
- **fd passing (SCM_RIGHTS), termios, terminfo** — the tmux study's stage-C
|
||||
bill, separate iterations.
|
||||
- **WebSocket client** — zen C′'s remaining item, not process work.
|
||||
- **A general signal API for user code** — only if fork 3 resolves toward
|
||||
the private reaping path; otherwise this iteration carries the seam but
|
||||
not the surface.
|
||||
- **Pipelines between children** — composition can wait for a consumer.
|
||||
|
||||
## Consumers, for the record
|
||||
|
||||
Iteration [28](28-skillhost-host-workload.md) (script runner, "bounded
|
||||
subprocess first" was already its stated order); alacritty study stage A
|
||||
(headless PTY runner — needs this plus PTY); tmux study (spawn half of its
|
||||
kernel); zen study stage C′ (spawn the browser, then drive it). Four
|
||||
consumers is the most any single named gap has accumulated.
|
||||
320
docs/superpowers/plans/2026-09-01-bounded-subprocess.md
Normal file
320
docs/superpowers/plans/2026-09-01-bounded-subprocess.md
Normal file
|
|
@ -0,0 +1,320 @@
|
|||
# Bounded Subprocess (iteration 42) Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use
|
||||
> superpowers:subagent-driven-development (recommended) or
|
||||
> superpowers:executing-plans to implement this plan task-by-task. Steps use
|
||||
> checkbox (`- [ ]`) syntax for tracking.
|
||||
>
|
||||
> **Project rule (overrides the plan-skill template):** plan docs carry
|
||||
> concept, reason and actions in words — no implementation or test code
|
||||
> blocks. Each step names the exact functions, fields, ids and expected
|
||||
> outcomes; the implementer writes the code at the keyboard, matching the
|
||||
> anchors cited here.
|
||||
|
||||
**Goal:** `proc.run` becomes bounded (deadline, output caps, per-shard
|
||||
ceiling, owner-bound reaping) and parked (fiber waits, shard never stalls),
|
||||
plus an extended `proc.run_dl` stating bounds per call.
|
||||
|
||||
**Architecture:** rework `WO_B_PROC_RUN` in `runtime/src/sysio.c` from
|
||||
blocking drain + `waitpid` into the iteration 35 `_dl` parking mould: after
|
||||
the fork, the two pipe read ends and a pidfd for the child are bundled
|
||||
behind one epoll fd; the fiber parks on that bundle fd with the existing
|
||||
`fb->dl_active`/`fb->dl_at` deadline sweep armed; re-entry after each wake
|
||||
drains whatever is ready into growable capped buffers, and the pidfd firing
|
||||
means reap-and-return. A per-shard child registry in `wo_vm` carries the
|
||||
cross-park state and serves the ceiling, engine-stop kill, and unwind
|
||||
cleanup.
|
||||
|
||||
**Tech Stack:** C (runtime), OCaml (compiler stdlib table), `.wo` (example
|
||||
app), bash (gate script), just (recipes).
|
||||
|
||||
**Spec:** `docs/superpowers/specs/2026-09-01-bounded-subprocess-design.md`
|
||||
— the plan argues from it; read both.
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- Linux only; pidfd needs runtime kernel ≥ 5.3 (Ubuntu 22.04 ships 5.15).
|
||||
- glibc 2.35 (the release build floor) has NO `pidfd_open` /
|
||||
`pidfd_send_signal` wrappers (they arrived in 2.36) — call both via raw
|
||||
`syscall(SYS_pidfd_open, …)` / `syscall(SYS_pidfd_send_signal, …)`.
|
||||
- Defaults, verbatim from the spec: deadline 30 000 ms, stdout cap
|
||||
1 048 576 bytes, stderr cap 65 536 bytes, ceiling 32 children per shard.
|
||||
- Bound violations kill the child and trap as `WO_T_IO` with a message
|
||||
naming the bound and its value; silent truncation is removed.
|
||||
- New builtin id: `WO_B_PROC_RUN_DL = 96` (89/90 stay iteration 31's
|
||||
reserved holes). No `.wob` version bump — `WOB_VERSION` does not move.
|
||||
- No new dependencies, no new threads, no signal handlers.
|
||||
- All work on `dev`; every commit title prefixed `feat(lang42):` /
|
||||
`fix(lang42):` / `docs(lang42):`; bullet-point commit bodies, ≤25 lines.
|
||||
- Builds and tests only through just recipes: `just wovm-build`,
|
||||
`just wovm-test`, `just woc-build`, `just woc-test`.
|
||||
- Runtime test binaries build with ASan+UBSan (runtime/Makefile does this
|
||||
for every `test/test_*.c` automatically) — a leak or race is a failure.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: claim the prefix, commit the standing docs
|
||||
|
||||
**Files:**
|
||||
- Modify: `docs/00-git-commit-history.md` (prefix registry table)
|
||||
- Already-edited, to commit: story 42, spec, `docs/stories/00-status.md`
|
||||
row, `docs/00-dependency-graph.md` node, the three exploration studies
|
||||
under `docs/plan/exploration/{alacritty,tmux,zen-browser}/`, this plan.
|
||||
|
||||
**Interfaces:** none — bookkeeping.
|
||||
|
||||
- [ ] **Step 1:** Add a `lang42` row to the prefix registry table in
|
||||
`docs/00-git-commit-history.md`: feature "iteration 42 — bounded
|
||||
subprocess (proc.run bounded + parked, proc.run_dl)", status "on `dev`".
|
||||
- [ ] **Step 2:** Commit all listed docs as one
|
||||
`docs(lang42): story, spec and plan for bounded subprocess` commit
|
||||
(bullets: story+spec+plan added; board row; graph node; the three
|
||||
parity studies that motivated it).
|
||||
|
||||
### Task 2: baseline suite — pin what `proc.run` already does
|
||||
|
||||
**Files:**
|
||||
- Create: `runtime/test/test_proc.c` (auto-picked by the Makefile's
|
||||
`$(wildcard test/test_*.c)` — no build wiring needed)
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: the test harness macros in `runtime/test/t.h` (`T_EQ`,
|
||||
`T_CHECK`), `wo_db_init`-style setup patterns from `test_builtin.c`, and
|
||||
direct builtin dispatch the way `test_builtin.c` invokes cases — operands
|
||||
in registers, `WO_B_PROC_RUN` (id 56) takes cmd Text, `multi Text` args,
|
||||
and the Proc class id, returns the record {code, out, err}.
|
||||
- Produces: `test_proc.c` as the home for every later runtime leg.
|
||||
|
||||
- [ ] **Step 1:** Write three green legs against CURRENT behavior, copying
|
||||
`test_builtin.c`'s VM/fiber setup: (a) a child that exits 0 with known
|
||||
stdout — assert code 0 and the exact bytes; (b) a child that exits with a
|
||||
known nonzero code — assert the code; (c) a nonexistent command — assert
|
||||
code 127 (the execvp-failed convention already in the code).
|
||||
- [ ] **Step 2:** `just wovm-test` — all three legs pass, whole suite
|
||||
stays green, ASan clean.
|
||||
- [ ] **Step 3:** Commit `feat(lang42): pin proc.run's current contract in
|
||||
test_proc`.
|
||||
|
||||
### Task 3: the parked rework — pidfd + epoll bundle + registry
|
||||
|
||||
The core task. The suspected sequential-drain deadlock is proven first,
|
||||
then dissolved by the rework.
|
||||
|
||||
**Files:**
|
||||
- Modify: `runtime/src/sysio.c` (the `WO_B_PROC_RUN` case, currently
|
||||
~lines 723–815)
|
||||
- Modify: `runtime/src/vm.h` (the per-shard registry in `wo_vm`, one
|
||||
pointer field on `wo_fiber` for the in-flight entry)
|
||||
- Modify: `runtime/src/vm.c` (engine-stop sweep over the registry)
|
||||
- Test: `runtime/test/test_proc.c`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `WO_SYS_PARKED` re-entry convention (`sysio.c:510` READ_DL is
|
||||
the model: fill `fb->park_fd`, arm `fb->dl_active`/`fb->dl_at` once —
|
||||
guarded so re-entry does not re-arm — return `WO_SYS_PARKED`; the plane
|
||||
re-runs the builtin on wake); `stop_pending()`; `wo_str_new`.
|
||||
- Produces: a registry entry type (name it `wo_child`) holding pid, pidfd,
|
||||
the bundle epoll fd, both pipe fds, two growable buffers with their caps,
|
||||
the deadline, and the owning fiber pointer; `wo_vm` gains a fixed array
|
||||
of 32 `wo_child` slots plus a live count; `wo_fiber` gains a pointer to
|
||||
its in-flight entry (NULL when none). Task 4–8 legs and Task 9's
|
||||
`WO_B_PROC_RUN_DL` all reuse exactly this machinery.
|
||||
|
||||
- [ ] **Step 1 (the red test):** In `test_proc.c`, add the deadlock leg: a
|
||||
child (use `sh -c` in the test only) that writes ~200 KiB to stdout and
|
||||
one line to stderr, stderr kept open until stdout completes. Guard the
|
||||
leg with a wall-clock check: it must complete within 5 s. Under the
|
||||
current sequential drain the parent stops reading stdout at 8,192 bytes,
|
||||
the child blocks on a full pipe, and stderr never reaches EOF.
|
||||
- [ ] **Step 2:** `just wovm-test` — the new leg FAILS (hangs into the
|
||||
guard) while everything else stays green. This is the bug proven.
|
||||
- [ ] **Step 3 (the rework):** Rewrite the `WO_B_PROC_RUN` case: pipes
|
||||
opened `O_NONBLOCK` on the parent side; after the fork,
|
||||
`syscall(SYS_pidfd_open, pid, 0)`; create one epoll fd and register both
|
||||
pipe read ends and the pidfd; claim a registry slot (fail closed with
|
||||
`WO_T_IO` naming the 32-per-shard ceiling if none — the message text the
|
||||
Task 6 leg asserts); stash caps and buffers in the slot, point the fiber
|
||||
at it. First entry and every re-entry then run the same drain: read each
|
||||
ready pipe into its growable buffer; a buffer passing its cap means kill
|
||||
(`syscall(SYS_pidfd_send_signal, pidfd, SIGKILL, 0, 0)`), reap, release
|
||||
the slot, trap `WO_T_IO` naming the cap and value. Pidfd readable means
|
||||
exited: reap via `waitpid` (now non-blocking — the pidfd said so), do a
|
||||
final drain of both pipes to EOF (bounded by the caps), release, build
|
||||
the Proc record exactly as today. Nothing ready and child alive: park on
|
||||
the bundle fd with the deadline armed (30 000 ms default), return
|
||||
`WO_SYS_PARKED`. Deadline re-entry with `dl_at` passed: kill, reap,
|
||||
release, trap `WO_T_IO` naming the deadline. `stop_pending()` on any
|
||||
entry: kill, reap, release, return `WO_SYS_STOPPED`.
|
||||
- [ ] **Step 4:** Engine-stop sweep: where `vm.c`'s worker loop observes
|
||||
the stop flag, kill + reap every live registry entry on that shard —
|
||||
covers fibers that never get rescheduled.
|
||||
- [ ] **Step 5:** `just wovm-test` — deadlock leg green, Task 2 baseline
|
||||
legs still green (same results from the parked path), suite ASan clean.
|
||||
- [ ] **Step 6:** Commit `feat(lang42): proc.run parks — pidfd + epoll
|
||||
bundle + child registry` (bullets: the deadlock repro and its dissolve;
|
||||
raw syscalls because glibc 2.35).
|
||||
|
||||
### Task 4: the deadline leg
|
||||
|
||||
**Files:** Test: `runtime/test/test_proc.c`; fix (if red exposes drift):
|
||||
`runtime/src/sysio.c`.
|
||||
|
||||
**Interfaces:** consumes Task 3's machinery unchanged; the leg drives the
|
||||
default arming path.
|
||||
|
||||
- [ ] **Step 1 (red first if Task 3 left a gap):** a child sleeping 10 s,
|
||||
run with the deadline forced low for the test (drive the builtin with a
|
||||
small `dl` the way the harness passes operands — until Task 9 the
|
||||
extended operands are reachable only from C, which is fine here). Assert:
|
||||
the trap message names the deadline and its value; then assert the pid is
|
||||
GONE — `kill(pid, 0)` returns ESRCH — measured, not assumed.
|
||||
- [ ] **Step 2:** `just wovm-test` green (implement/adjust the kill path if
|
||||
Step 1 caught drift). Also add the fiber-progress assertion: while the
|
||||
sleeping child runs, a second fiber on the same VM increments a counter —
|
||||
assert it advanced before the deadline fired (the shard was never
|
||||
blocked).
|
||||
- [ ] **Step 3:** Commit `feat(lang42): deadline kills, parked shard keeps
|
||||
scheduling`.
|
||||
|
||||
### Task 5: the output-cap leg
|
||||
|
||||
**Files:** Test: `runtime/test/test_proc.c`; fix: `runtime/src/sysio.c`.
|
||||
|
||||
- [ ] **Step 1:** a child writing unbounded output against a small stdout
|
||||
cap passed from the harness. Assert: `WO_T_IO` whose message names the
|
||||
cap and its value; pid gone (ESRCH); and — fd hygiene — the shard's open
|
||||
fd count returns to its pre-call value (count `/proc/self/fd` entries
|
||||
before and after).
|
||||
- [ ] **Step 2:** Same for the stderr cap.
|
||||
- [ ] **Step 3:** `just wovm-test` green. Commit `feat(lang42): output caps
|
||||
refuse by name, no truncation`.
|
||||
|
||||
### Task 6: the ceiling leg
|
||||
|
||||
**Files:** Test: `runtime/test/test_proc.c`.
|
||||
|
||||
- [ ] **Step 1:** 32 fibers each spawn a child sleeping 2 s; a 33rd spawn
|
||||
must trap `WO_T_IO` naming the ceiling while the 32 keep running to
|
||||
completion unharmed. Then all 32 complete with code 0.
|
||||
- [ ] **Step 2:** `just wovm-test` green (the slot-claim refusal exists
|
||||
since Task 3; this pins it). Commit `feat(lang42): per-shard ceiling
|
||||
fails closed at 32`.
|
||||
|
||||
### Task 7: the churn leg — fds flat over a thousand runs
|
||||
|
||||
**Files:** Test: `runtime/test/test_proc.c`.
|
||||
|
||||
- [ ] **Step 1:** loop one thousand sequential short-lived children
|
||||
(the iteration 24 measurement style): record the `/proc/self/fd` entry
|
||||
count before, assert the count after equals it, and assert no registry
|
||||
slot remains claimed.
|
||||
- [ ] **Step 2:** `just wovm-test` green. Commit `feat(lang42): a thousand
|
||||
spawns leave the fd table flat`.
|
||||
|
||||
### Task 8: stop and unwind reap their children
|
||||
|
||||
**Files:** Test: `runtime/test/test_proc.c`; fix: `runtime/src/sysio.c`,
|
||||
`runtime/src/vm.c`.
|
||||
|
||||
- [ ] **Step 1:** park a fiber on a child sleeping 10 s, set the stop flag
|
||||
the way `test_fiber.c` does, drive the loop; assert `WO_SYS_STOPPED`
|
||||
surfaced AND the child pid is gone. Second leg: a child owned by a fiber
|
||||
that is unwound (the `test_unwind.c` pattern) is also gone.
|
||||
- [ ] **Step 2:** `just wovm-test` green. Commit `feat(lang42): stop and
|
||||
unwind kill the children they own`.
|
||||
|
||||
### Task 9: `proc.run_dl` — the per-call bounds surface
|
||||
|
||||
**Files:**
|
||||
- Modify: `runtime/src/wob.h` (enum entry `WO_B_PROC_RUN_DL = 96`, comment
|
||||
stating the operand order and the nil-never contract: bounds violations
|
||||
trap, they do not nil)
|
||||
- Modify: `runtime/src/sysio.c` (a thin case: parse deadline_ms, out_cap,
|
||||
err_cap operands, then fall into Task 3's core with those instead of the
|
||||
defaults)
|
||||
- Modify: `compiler/src/types.ml:315` region (one row beside `proc.run`:
|
||||
module `proc`, member `run_dl`, arity 5, id 96, same nullable-Proc
|
||||
return and Proc record class as the existing row)
|
||||
- Test: `runtime/test/test_proc.c` (drive id 96 with explicit bounds);
|
||||
compiler: `just woc-test` must stay green — the generic builtin path
|
||||
needs no `emit.ml` change (the net `_dl` family at ids 91–93 landed with
|
||||
table rows alone; verify by reading their absence from `emit.ml` before
|
||||
assuming).
|
||||
|
||||
**Interfaces:**
|
||||
- Produces: `proc.run_dl(cmd, args, deadline_ms, out_cap, err_cap)` in the
|
||||
language, the name the example app and every future consumer calls.
|
||||
|
||||
- [ ] **Step 1 (red):** runtime leg driving id 96 with a tight explicit
|
||||
deadline — fails while the case is absent.
|
||||
- [ ] **Step 2:** add the wob.h entry and the sysio.c case; green.
|
||||
- [ ] **Step 3:** add the types.ml row; `just woc-build && just woc-test`
|
||||
green.
|
||||
- [ ] **Step 4:** Commit `feat(lang42): proc.run_dl — deadline and caps at
|
||||
the call site`.
|
||||
|
||||
### Task 10: the example app and its gate
|
||||
|
||||
**Files:**
|
||||
- Create: `docs/examples/subprocess/` (wo.toml + one `.wo` source: an
|
||||
actor that serves `proc.run`/`proc.run_dl` results — a fast command, a
|
||||
deliberate deadline hit caught with try/catch, a deliberate cap hit
|
||||
caught, and a long-running child for the drain leg; logs to
|
||||
`/tmp/subprocess.log` per the examples convention, announced and
|
||||
banner-separated)
|
||||
- Create: `scripts/subprocess-accept.sh` (the gate)
|
||||
- Modify: `justfile` (recipe `subprocess` running the gate, in the
|
||||
`residency`/`site` recipe mould)
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `proc.run` and `proc.run_dl` exactly as Task 9 shipped them.
|
||||
|
||||
- [ ] **Step 1:** write the app and gate with these legs: fast command
|
||||
returns its output through the actor; deadline violation is caught in
|
||||
`.wo` and reported (proves catchability from the language, not just from
|
||||
C); cap violation likewise; concurrency — a slow child runs while a
|
||||
second request is answered (wall-clock assertion that the second answer
|
||||
did not wait for the child); SIGTERM while a `sleep 30` child lives —
|
||||
the gate records the child pid, stops the app, asserts the app exited
|
||||
cleanly AND the child pid is gone (iteration 40's battery gains its
|
||||
subprocess leg here).
|
||||
- [ ] **Step 2:** `just subprocess` — all legs green, printed as
|
||||
`subprocess-accept: N checks, 0 failures`.
|
||||
- [ ] **Step 3:** Commit `feat(lang42): subprocess example + gate` (app,
|
||||
script, recipe).
|
||||
|
||||
### Task 11: close-out
|
||||
|
||||
**Files:**
|
||||
- Modify: `runtime/src/CODE-LOGIC.md` (the proc.run section: parked
|
||||
lifecycle, the registry, the raw-syscall note, the deadlock that was)
|
||||
- Modify: `docs/stories/language-runtime-database/42-bounded-subprocess.md`
|
||||
(frontmatter `status: done`; Progress note of what landed vs the spec)
|
||||
- Modify: `docs/stories/00-status.md` (NEXT PLAN entry answering the six
|
||||
standup questions — implemented, key findings measured, learned,
|
||||
unblocked (the streaming form, tmux/alacritty/zen stages), next steps,
|
||||
`.dev/reference` used: alacritty/tmux/zen-browser; flip the pending row)
|
||||
- Modify: `docs/00-dependency-graph.md` (node 42 class → done, same change
|
||||
as the board row per the maintenance rule)
|
||||
|
||||
- [ ] **Step 1:** run the full belt: `just wovm-test`, `just woc-test`,
|
||||
`just subprocess`, plus `just site` untouched-but-verified. All green,
|
||||
outputs quoted in the board entry, not asserted.
|
||||
- [ ] **Step 2:** write CODE-LOGIC.md beside the code (project rule) and
|
||||
the three doc updates.
|
||||
- [ ] **Step 3:** Commit `docs(lang42): close out iteration 42`.
|
||||
|
||||
---
|
||||
|
||||
## Self-review (done at write time)
|
||||
|
||||
- Spec coverage: every acceptance criterion has a task — normal exit (2),
|
||||
deadlock repro-then-fix (3), deadline + progress (4), caps (5), ceiling
|
||||
(6), fd-flat thousand (7), stop/unwind (8), per-call bounds (9), drain
|
||||
leg + example gate (10). Out-of-scope items appear in no task.
|
||||
- Placeholders: none; every step names its files, ids, messages and
|
||||
expected outcomes.
|
||||
- Consistency: registry type `wo_child`, bundle-epoll park, id 96, caps
|
||||
30 000 ms / 1 MiB / 64 KiB / 32 used identically in tasks 3–10.
|
||||
- One verify-before-assuming flag left deliberately in Task 9: confirm the
|
||||
net `_dl` rows needed no `emit.ml` change before mirroring them.
|
||||
143
docs/superpowers/specs/2026-09-01-bounded-subprocess-design.md
Normal file
143
docs/superpowers/specs/2026-09-01-bounded-subprocess-design.md
Normal file
|
|
@ -0,0 +1,143 @@
|
|||
# Bounded subprocess v1 — the one-shot form, bounded and parked
|
||||
|
||||
> Brainstormed and approved 2026-09-01. Story:
|
||||
> [language 42](../../stories/language-runtime-database/42-bounded-subprocess.md).
|
||||
> Scope settled during brainstorm: bound the existing one-shot `proc.run`
|
||||
> only; the streaming form (long-lived child, output as mailbox messages) is
|
||||
> the named follow-up. Exit notification via pidfd, no signal seam. Bounds as
|
||||
> defaults plus per-call override, refusal by name. Argv only, no shell form
|
||||
> ever. Env and cwd stay inherited, no knobs.
|
||||
|
||||
## The problem
|
||||
|
||||
`proc.run` exists (`runtime/src/sysio.c`, `WO_B_PROC_RUN`) but is bounded in
|
||||
nothing that matters and blocks in the one place it must not: it runs the
|
||||
fork, the pipe drain, and the `waitpid` on the shard thread itself, so one
|
||||
slow child stalls every actor and fiber on that shard. It has no deadline —
|
||||
a child that never exits parks the shard forever, and the `waitpid` loop
|
||||
notices an engine stop only if a signal happens to interrupt it. Its output
|
||||
caps are fixed constants (8,192 bytes stdout, 4,096 stderr) that truncate
|
||||
silently. Nothing reaps a child when the engine stops. And the sequential
|
||||
drain (stdout to cap, then stderr to EOF) is suspected of deadlocking
|
||||
against a child that fills the stdout pipe while holding stderr open — to be
|
||||
proven by a failing test before the fix is claimed.
|
||||
|
||||
Four consumers arrived at once — iteration 28's script runner and the
|
||||
alacritty/tmux/zen-browser exploration studies — all bottoming out on this
|
||||
as their first domino.
|
||||
|
||||
## The model, and why
|
||||
|
||||
**Bounded means: every resource a child can consume carries a declared
|
||||
ceiling, and exceeding one is a refusal that names the ceiling — never a
|
||||
hang, a truncation, or an OOM.** Same fail-closed posture as porch 1's pool
|
||||
saturation and iteration 31's mailbox caps.
|
||||
|
||||
- **Time.** Every run has a deadline. At the deadline the child is killed,
|
||||
reaped, and the caller gets a catchable error naming the deadline and its
|
||||
value.
|
||||
- **Output bytes.** Stdout and stderr each carry a cap. Past a cap the
|
||||
child is killed and the error names the cap — silent truncation is
|
||||
removed, which is the one observable behavior change for existing
|
||||
callers (code relying on truncation was relying on a bug).
|
||||
- **Concurrency.** A per-shard ceiling on live children (default 32). At
|
||||
the ceiling a spawn fails closed with a named error rather than queueing.
|
||||
- **Lifetime.** A child is owned by the fiber that spawned it. The fiber
|
||||
unwinding for any reason — actor death, engine stop — kills and reaps the
|
||||
child. Engine stop kills every registered child before the process exits:
|
||||
iteration 40's drain guarantee extends to subprocesses. A zombie or an
|
||||
orphan is a bug by definition.
|
||||
|
||||
**Call surface.** `proc.run(cmd, args)` keeps its shape and its `Proc`
|
||||
record (`code`, `out`, `err`). Bare calls get the named defaults: 30 s
|
||||
deadline, 1 MiB stdout cap, 64 KiB stderr cap. An extended form states
|
||||
`deadline_ms`, `out_cap` and `err_cap` per call — deadline is intrinsically
|
||||
per-call (a browser run and a `true` are not the same ask), which is why the
|
||||
bounds do not live in `wo.toml`. Precedent: mailbox caps — a default that a
|
||||
declaration can override.
|
||||
|
||||
**Argv only, no shell form, ever.** The capability a shell would add
|
||||
already exists through argv (a caller can spawn `sh` with `-c` explicitly,
|
||||
owning the quoting risk by having typed it); a shell builtin adds nothing
|
||||
but an injection-shaped API, and consumer #1 is a *confined* script runner.
|
||||
The existing argv limits (62 arguments, 512 bytes each) stay — nothing
|
||||
asked for more.
|
||||
|
||||
**Env and cwd stay inherited, zero knobs.** No consumer demands control;
|
||||
28's confinement story owns env scrubbing when it arrives, and a clean-env
|
||||
knob has hidden teeth (`execvp` path search dies with an emptied
|
||||
environment). Recorded as the skillhost follow-up, not built.
|
||||
|
||||
## Mechanics
|
||||
|
||||
**Parking, not blocking.** The builtin becomes a parking builtin in the
|
||||
iteration 35 mould (the `_dl` net family): after the fork it registers the
|
||||
two pipe read ends, a pidfd for the child (`pidfd_open`), and a deadline
|
||||
timer with the shard's io_uring, then parks the calling fiber. The shard
|
||||
thread never waits on a child. Readiness events append into growable
|
||||
buffers up to the cap; the pidfd firing means the child exited (reap,
|
||||
finish, unpark); the timer firing first means deadline (kill via
|
||||
`pidfd_send_signal`, reap, unpark with the error). Both pipes are serviced
|
||||
concurrently by readiness, which structurally removes the suspected
|
||||
sequential-drain deadlock.
|
||||
|
||||
**Why pidfd and not SIGCHLD or a reaper thread.** A pidfd is a pollable fd
|
||||
— exactly what the runtime's loop already multiplexes — with no
|
||||
process-wide signal handler, no delivery-shard question, no masking rules,
|
||||
and no race between concurrent children. The general signals-as-events
|
||||
seam stays deferred by name to its real consumer (the multiplexer study's
|
||||
stage C). A reaper thread would introduce a thread class the runtime does
|
||||
not have, for no gain over the fd. Kernel floor: `pidfd_open` is Linux
|
||||
5.3+, well under the runtime's existing io_uring floor.
|
||||
|
||||
**The child registry.** Each shard keeps a registry of its live children
|
||||
(pid + pidfd + owning fiber). Spawn checks the ceiling against it and fails
|
||||
closed at 32. Engine stop walks it and kills before exit. Fiber unwind
|
||||
(actor death, cancellation) kills its own entry through the same path. One
|
||||
mechanism, three callers.
|
||||
|
||||
**Builtin ids.** `WO_B_PROC_RUN` keeps its id and gains the parking
|
||||
semantics and defaults. The extended bounded form takes the next free id
|
||||
from 96 up (89/90 remain iteration 31's reserved holes). No `.wob` version
|
||||
bump — iteration 38's precedent: new builtin ids alone do not move the
|
||||
format.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- A child that exits normally: exit code, stdout and stderr surface as
|
||||
values — the existing behavior, now from a parked fiber; a second actor
|
||||
on the same shard demonstrably makes progress while a slow child runs.
|
||||
- A child that outruns its deadline: caller resumes with the named timeout
|
||||
error, and the pid is verified absent from the process table — measured,
|
||||
not assumed.
|
||||
- A child that exceeds an output cap: refusal names the cap and its value;
|
||||
the child is dead; no fd leaked.
|
||||
- The deadlock repro: a child that fills stdout past the pipe capacity
|
||||
while holding stderr open. The test is written against the CURRENT code
|
||||
first and must fail (hang) there; it passes under the new mechanics.
|
||||
- The drain leg: SIGTERM to a program with live children — every child is
|
||||
terminated before the program exits. Iteration 40's battery gains this
|
||||
leg.
|
||||
- The ceiling leg: the 33rd concurrent spawn on one shard fails closed
|
||||
with the named error while the 32 live ones are unaffected.
|
||||
- One thousand sequential spawns: fd count flat (the iteration 24
|
||||
measurement style), rss not growing with the loop.
|
||||
- A runnable example under `docs/examples/subprocess` behind a `just`
|
||||
recipe, logging to `/tmp/<app>.log` per convention — the gate is the
|
||||
acceptance, unit tests alone are not.
|
||||
|
||||
## Out of scope (each with its reason)
|
||||
|
||||
- **The streaming form** — long-lived children with output as mailbox
|
||||
messages. The follow-up this slice deliberately excludes; tmux, alacritty
|
||||
and the zen CDP driver queue behind it. The registry, pidfd wait and
|
||||
cap machinery built here are its foundation.
|
||||
- **PTY allocation** — pipes only; the pseudo-terminal surface (openpty,
|
||||
resize ioctls) is the alacritty/tmux stage-A follow-up.
|
||||
- **A general signal API** — deferred to its real consumer; this slice's
|
||||
child-exit path needs no signals at all.
|
||||
- **Env/cwd control** — skillhost confinement's item, recorded above.
|
||||
- **stdin transport** — no consumer in this slice's set feeds a child
|
||||
interactively; the streaming form owns it.
|
||||
- **Pipelines between children** — composition waits for a consumer.
|
||||
- **A shell convenience form** — refused permanently, not deferred.
|
||||
Loading…
Reference in a new issue