docs: jarvis track, runtime-v2 7/8/9, lang-41 fix design, fiber scope-gap
- jarvis (00-story): 6th track, 2nd software built with writeonce — an AI assistant; direct-HTTPS design; blockers named (net.connect + TLS) - runtime-v2 7 observability + 8 symmetric cipher: moved from the language track (were 30/43); 9 in-process TLS: created from the gap jarvis surfaces, RETIRES the "TLS is the proxy's job" doctrine (both directions) - language 41 (arena hang): fix design to ready — marshal cross-shard messages (root), align the shard_id % nshards route/compare + assert bound; poison-on-free + minimal fixture as follow-ups - fiber scope-gap analysis (plan/exploration/fiber/01): porch vs fiber, what porch lacks, would developers prefer porch - board + dependency-graph synced (porch 2-8 ready; rv2 table; §5/§5a graphs) (cherry picked from commit 203470ceb2a151fe3584931cd4237af3f96a9f29)
This commit is contained in:
parent
e91a3704fe
commit
cc1c82b2ef
10 changed files with 1181 additions and 43 deletions
|
|
@ -274,48 +274,103 @@ appear in graph 2 — they need iteration 11 as well as 18.
|
|||
|
||||
## 5. porch — the web framework track
|
||||
|
||||
States live on [the board's porch section](stories/00-status.md); every
|
||||
pending node is `readiness: refine` (no approved spec), so a brainstorm
|
||||
precedes any plan. Three independent roots: **2** (the auth chain),
|
||||
**6** (the streaming chain), **5** (anytime, no incoming edges). **9** is
|
||||
the track's only `readiness: ready` item and is held.
|
||||
States live on [the board's porch section](stories/00-status.md). **The whole
|
||||
track (2–8) is `readiness: ready`** as of the 2026-09-06 brainstorm; **1** is
|
||||
done, **9** is held (blocked on the lang-41 arena hang, not an enhancement).
|
||||
Three independent roots: **2** (the auth chain), **6** (the streaming chain),
|
||||
**5** (anytime, no incoming edges at all — not even iteration 2).
|
||||
|
||||
This graph makes the **cross-track language edges** visible: the three builtins
|
||||
the track needs, each drawn as a `lang` node feeding the story that owns it.
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
classDef done fill:#1a7f37,color:#fff,stroke:none
|
||||
classDef refine fill:#eac54f,color:#000,stroke:none
|
||||
classDef ready fill:#0969da,color:#fff,stroke:none
|
||||
classDef held fill:#6e7781,color:#fff,stroke:none
|
||||
classDef lang fill:#8250df,color:#fff,stroke:none
|
||||
|
||||
CSPRNG["language track: CSPRNG builtin (id 96+) — porch 2's Phase A"]:::lang
|
||||
P1["porch 1 store-backed middleware ✅ 2026-08-30"]:::done
|
||||
P2["porch 2 randomness + cookies (repeated Set-Cookie/Vary headers, Cookie: parsing, signed cookies)"]:::refine
|
||||
P3["porch 3 sessions"]:::refine
|
||||
P4["porch 4 CSRF"]:::refine
|
||||
P5["porch 5 routing + response ergonomics"]:::refine
|
||||
P6["porch 6 streaming core"]:::refine
|
||||
P7["porch 7 SSE + compression"]:::refine
|
||||
P8["porch 8 static files + lifecycle"]:::refine
|
||||
P9["porch 9 idempotent replay (⏸ hold; readiness: ready)"]:::held
|
||||
RB["language work: random_bytes builtin (bare-name, id 84/90) — porch 2 Phase A"]:::lang
|
||||
DFL["language work: deflate + crc32 builtins (C) — porch 7 Phase C"]:::lang
|
||||
TU["language work: time.utc builtin (gmtime sibling of time.local) — porch 8 Phase A"]:::lang
|
||||
|
||||
CSPRNG --> P2
|
||||
P1["porch 1 store-backed middleware ✅ 2026-08-30"]:::done
|
||||
P2["porch 2 randomness + cookies"]:::ready
|
||||
P3["porch 3 sessions"]:::ready
|
||||
P4["porch 4 CSRF"]:::ready
|
||||
P5["porch 5 routing + response ergonomics (zero upstream deps)"]:::ready
|
||||
P6["porch 6 streaming core"]:::ready
|
||||
P7["porch 7 SSE + compression"]:::ready
|
||||
P8["porch 8 static files + lifecycle"]:::ready
|
||||
P9["porch 9 idempotent replay (⏸ hold; readiness: ready)"]:::held
|
||||
L41["language 41 actor-arena hang (in-progress)"]:::held
|
||||
|
||||
RB --> P2
|
||||
P2 --> P3
|
||||
P2 --> P4
|
||||
P3 --> P4
|
||||
P6 --> P7
|
||||
P6 --> P8
|
||||
P2 --> P7
|
||||
P5 --> P7
|
||||
P5 --> P8
|
||||
DFL --> P7
|
||||
TU --> P8
|
||||
L41 -.blocks.-> P9
|
||||
P1 -.re-scope 79e6da4: replay-on-retry split out of 1.-> P9
|
||||
```
|
||||
|
||||
The two non-obvious edges are stated in porch 7's own story: gzip's
|
||||
`Accept-Encoding` negotiation reuses the q-value ranking iteration 5 adds,
|
||||
and `Vary: Accept-Encoding` needs iteration 2's repeated-header work to
|
||||
accumulate with other `Vary` contributions. Porch 2's Phase A is
|
||||
language-track work (the CSPRNG builtin) — the cross-track edge this graph
|
||||
exists to make visible. Streaming responses' runtime prerequisites
|
||||
(fibers, iteration 11) are already green in graph 2.
|
||||
Edges corrected by the 2026-09-06 brainstorm: `P5 --> P7` (gzip's
|
||||
`Accept-Encoding` reuses iteration 5's q-value ranking) stays, but the old
|
||||
`P2 --> P7` edge is **gone** — story 7 decided `Vary` accumulates by comma-join
|
||||
(iteration 5's shape), not iteration 2's repeated-header work. `P5 --> P8` is the
|
||||
`Download`/`Attachment` helper. The three `lang` nodes are the track's entire
|
||||
language bill; each is a builtin with a named consumer, none shipped as
|
||||
decoration.
|
||||
|
||||
## 5a. porch's language-driven gaps (out-of-scope features and the language stories that own them)
|
||||
|
||||
These are the features fiber ships that porch deliberately does **not** — each
|
||||
excluded because a language primitive does not exist yet. Every edge points from
|
||||
the owning language story to the porch feature it would unblock (see the
|
||||
[porch↔fiber scope-gap analysis](plan/exploration/fiber/01-porch-vs-fiber-scope-gap.md)).
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
classDef done fill:#1a7f37,color:#fff,stroke:none
|
||||
classDef refine fill:#eac54f,color:#000,stroke:none
|
||||
classDef held fill:#6e7781,color:#fff,stroke:none
|
||||
classDef gap fill:#cf222e,color:#fff,stroke:none
|
||||
|
||||
L29["language 29 @derive (⏸ hold)"]:::held
|
||||
L38["language 38 net.connect (refine)"]:::refine
|
||||
L43["runtime-v2 8 symmetric cipher (refine, NEW 2026-09-06)"]:::refine
|
||||
L30["runtime-v2 7 observability (refine, moved from language 30, 2026-09-06)"]:::refine
|
||||
L18["language 18 TTL cache + transaction{} (⏸ hold)"]:::held
|
||||
L31["language 31 cancellation ✅ (landed in 24)"]:::done
|
||||
|
||||
BIND["typed request binding (fiber Bind)"]:::gap
|
||||
PROXY["reverse proxy + outbound HTTP client"]:::gap
|
||||
ENC["encrypted cookies (fiber encryptcookie)"]:::gap
|
||||
METRICS["metrics / pprof / expvar endpoints"]:::gap
|
||||
CACHE["cache middleware + recovery rollback"]:::gap
|
||||
TIMEOUT["per-handler timeout + streaming backpressure"]:::gap
|
||||
|
||||
L29 --> BIND
|
||||
L38 --> PROXY
|
||||
L43 --> ENC
|
||||
L30 --> METRICS
|
||||
L18 --> CACHE
|
||||
L31 --> TIMEOUT
|
||||
```
|
||||
|
||||
31 (cancellation) is already green — per-handler timeout and streaming
|
||||
backpressure are unblocked at the language level and wait only on a porch slice
|
||||
to consume them. The other five gaps are gated on an upstream story: two
|
||||
brand-new runtime-v2 iterations (8 cipher, 7 observability — moved out of the
|
||||
language track 2026-09-06), two language iterations on hold (29, 18), one pending
|
||||
a spec (38). Landed enablers the
|
||||
track already consumed — 34 (crypto digests), 36 (bit operators), 35 (net
|
||||
seams) — are green in graphs 1–3 and not repeated here.
|
||||
|
||||
## 6. wmux — the multiplexer track (wmux 1) and its gap chain
|
||||
|
||||
|
|
|
|||
130
docs/plan/exploration/fiber/01-porch-vs-fiber-scope-gap.md
Normal file
130
docs/plan/exploration/fiber/01-porch-vs-fiber-scope-gap.md
Normal file
|
|
@ -0,0 +1,130 @@
|
|||
# porch vs fiber — the scope gap, and who prefers which
|
||||
|
||||
> Companion to [00-fiber-parity.md](00-fiber-parity.md). Written 2026-09-06,
|
||||
> after the porch track (stories 1–8) was brainstormed to `ready`. It reads
|
||||
> fiber v3 (`.dev/reference/fiber`, commit `3ca9a9d`) against porch's **approved
|
||||
> scope** — what ships today, plus what stories 1–8 add — and asks what remains
|
||||
> missing, what fiber does better, and whether a developer would choose porch.
|
||||
> An exploration doc: no status banner by convention.
|
||||
|
||||
## The 32 middleware, mapped against approved scope
|
||||
|
||||
Nine already had counterparts before the track. The track adds eight more. The
|
||||
rest are excluded, and the exclusions divide into *principled* (a different
|
||||
philosophy) and *blocked* (a primitive porch does not have yet).
|
||||
|
||||
- **Shipped before the track (9):** cors, basicauth, keyauth (bearer), helmet
|
||||
(SecurityHeaders), hostauthorization (HostAllow), etag, static, recover
|
||||
(trap=500), logger.
|
||||
- **Added by the track (8):** compress + sse (story 7), csrf (4), session (3),
|
||||
favicon + healthcheck + rewrite + skip + redirect (5/8), requestid (5),
|
||||
limiter (story 1, already done).
|
||||
- **Blocked on the lang-41 runtime hang (1):** idempotency — built, reviewed,
|
||||
gate-passed, reverted; it is porch story 9, held, not a design gap.
|
||||
- **Excluded, owner named (rest):** cache/paginate (language 18), expvar/pprof
|
||||
(runtime-v2 7 observability), proxy (language 38, needs `net.connect`),
|
||||
encryptcookie (runtime-v2 8 symmetric cipher), adaptor (no net/http
|
||||
ecosystem), earlydata (TLS at the
|
||||
proxy), timeout (per-handler, needs cancellation — language 31), envvar +
|
||||
responsetime (niche helpers, unscoped).
|
||||
|
||||
## What porch lacks even after the whole track lands
|
||||
|
||||
Ranked by how much a real app feels it.
|
||||
|
||||
1. **Typed request binding.** Fiber's `Bind().Body/Query/Cookie(&struct)` is the
|
||||
single biggest ergonomic gap. porch cannot do it without reflection
|
||||
(principle 13 forbids it) until `@derive` lands (language iteration 29). Today
|
||||
a porch handler pulls fields out of maps by hand. This is the one place fiber
|
||||
is *dramatically* more pleasant, and it is felt on every form and every JSON
|
||||
endpoint.
|
||||
2. **Outbound HTTP: client and reverse proxy.** Fiber ships an HTTP client and a
|
||||
`proxy` middleware. porch has no `net.connect`, so neither exists — an app
|
||||
that calls another service or fronts one cannot be written in porch at all.
|
||||
Owner: language iteration 38.
|
||||
3. **Encrypted cookies.** Fiber's `encryptcookie` is AES-GCM; porch has digests
|
||||
but no symmetric cipher, so it offers signed-and-readable only. Fine for a
|
||||
session id, not for a payload an app wants to hide from the client.
|
||||
4. **Per-handler timeouts.** Fiber wraps a handler in a deadline. porch has
|
||||
per-*call* net deadlines (iteration 35) but no per-handler timeout, because
|
||||
cancelling a running handler needs the actor-lifecycle cancellation that
|
||||
language iteration 31 owns.
|
||||
5. **Runtime templating.** Fiber renders views at request time against a dozen
|
||||
template engines. porch rejects this by doctrine — markup is a compile-time
|
||||
literal (`writeonce-view`). A principled choice, but it rules out the
|
||||
user-editable-template use case entirely.
|
||||
6. **The ecosystem.** `adaptor` plugs fiber into all of Go's `net/http` universe;
|
||||
its session/cache/limiter middleware take pluggable storage drivers (Redis,
|
||||
Postgres, dozens more). porch has none of that surface and, by single-binary
|
||||
doctrine, does not want most of it — but it means no drop-in Redis, no
|
||||
community middleware, no third-party integrations.
|
||||
7. **Minor, unscoped helpers:** pagination, response-time header, env-var
|
||||
exposure. Each is a few lines an app can write; none is in the track.
|
||||
|
||||
## What fiber is better at, even where porch has a counterpart
|
||||
|
||||
- **Maturity and ecosystem.** Battle-tested, huge community, pluggable storage
|
||||
behind every stateful middleware, and the whole Go module world one adaptor
|
||||
away. This is fiber's decisive, structural advantage and porch will not close
|
||||
it.
|
||||
- **Ergonomics.** Binding, generic helpers, reflection-driven convenience — less
|
||||
hand-written glue per endpoint.
|
||||
- **Raw performance ceiling.** fasthttp is extreme; a bytecode VM with
|
||||
single-threaded-per-connection serving will not match its throughput on a
|
||||
synthetic benchmark. (porch trades this for a different model, below.)
|
||||
- **Configurability.** Case-sensitivity, strict-slash routing, prefork, storage
|
||||
backends — many knobs. porch is deliberately opinionated with few.
|
||||
- **All-in-one networking.** Client, proxy, TLS termination, HTTP/2 in one
|
||||
process. porch delegates TLS/HTTP2 to a front proxy by doctrine.
|
||||
|
||||
## Where porch is actually better
|
||||
|
||||
The honest counter-case, because "prefer" is not decided on fiber's axes alone.
|
||||
|
||||
- **Durable by default, in one binary.** Sessions, rate-limiting and idempotency
|
||||
ride the WAL and survive a restart with no Redis, no external store. Fiber's
|
||||
defaults are in-memory — a restart logs everyone out and resets every counter;
|
||||
durability means operating a second system. porch's whole stateful surface is
|
||||
durable with zero extra infrastructure.
|
||||
- **One static binary, zero dependencies.** Hand-rolled crypto, no CGO, no
|
||||
driver matrix. Deploy a file.
|
||||
- **Safety by construction.** No reflection, ownership/borrow checking, and a
|
||||
house style of secure-by-default (HttpOnly/SameSite defaults, session-id
|
||||
rotation on login, refusal classes distinguishable in logs but opaque in the
|
||||
body) and refuse-the-unconstructible (an incoherent heartbeat/idle pair, a
|
||||
streaming route under header-mutating middleware — both rejected at
|
||||
construction, never silently half-applied).
|
||||
- **A concurrency model that fits the hard parts.** Actors/fibers/shards make
|
||||
SSE fan-out and WebSocket rooms natural rather than bolted on, and idle
|
||||
connections park for almost nothing.
|
||||
- **Honesty about limits.** Every gap above is named with an owner; nothing
|
||||
degrades silently.
|
||||
|
||||
## Would developers prefer porch over fiber?
|
||||
|
||||
It depends on who is asking, and the answer is not "porch wins on features."
|
||||
|
||||
- **A Go developer choosing a framework today: no.** Not on parity terms. Fiber
|
||||
wins on ecosystem, binding ergonomics, maturity, performance ceiling, storage
|
||||
flexibility and sheer breadth. porch cannot out-fiber fiber at fiber's own
|
||||
game, and trying would be the wrong goal.
|
||||
- **A developer already choosing writeonce: yes, and gladly.** porch is a
|
||||
coherent, durable-by-default, single-binary framework with strong security
|
||||
defaults, written *in* the language it serves. Inside the ecosystem there is no
|
||||
contest — it is the framework, and a good one.
|
||||
- **A developer choosing on values, either language aside: sometimes.** Someone
|
||||
who weights one-binary durability with no Redis, memory safety without GC
|
||||
pauses, opinionated secure defaults, and named-not-hidden limits above
|
||||
ecosystem breadth may genuinely prefer porch. That is a real but narrow
|
||||
constituency.
|
||||
|
||||
**The honest positioning.** porch is not a fiber-killer and the approved scope
|
||||
does not try to be one. It is deliberately *complete enough* to prove the
|
||||
language can carry a serious web framework, with a distinct thesis —
|
||||
durable-by-default, single-binary, safe-by-construction — that fiber does not
|
||||
compete on. Its remaining gaps are almost all one upstream primitive away
|
||||
(`net.connect` → client/proxy, `@derive` → binding, a cipher → encrypted
|
||||
cookies, cancellation → handler timeouts), which means the ceiling is set by the
|
||||
language track, not by porch's design. Developers will prefer porch when they
|
||||
have already bought the thesis; they will prefer fiber when they are shopping on
|
||||
breadth.
|
||||
|
|
@ -54,7 +54,13 @@ settled", so a held iteration with an approved spec was indistinguishable from
|
|||
one nobody had thought about. **The startable set is `readiness: ready` and
|
||||
`status: pending`.**
|
||||
|
||||
**As of the 2026-08-27 sweep that set has exactly one member:**
|
||||
**Startable-set counts below are STALE** — the paragraph that follows is the
|
||||
2026-08-27 sweep and predates the entire **wmux** (23 rungs, ~17 done/partial
|
||||
as of 2026-09-05) and **runtime-v2** (6 iterations, all done) tracks, plus
|
||||
lang 42. Treat the per-track tables further down as the current truth; this
|
||||
snapshot is kept for its explanation of the two-axis model, not its numbers.
|
||||
|
||||
**As of the 2026-08-27 sweep (STALE — see above):**
|
||||
[databasev2 4, io_uring group-commit](databasev2/04-io-uring-commit.md) — its
|
||||
four forks were confirmed settled on 2026-08-20 and nothing has started. Across
|
||||
47 iterations: 19 done, 5 in-progress, 15 pending, 8 hold; 27 `ready`, 20
|
||||
|
|
@ -70,6 +76,287 @@ behind this board; live Obsidian Dataview views:
|
|||
|
||||
## ▶ NEXT PLAN
|
||||
|
||||
### Brainstormed 2026-09-06 — the porch track (2–8) and language 41's fix, both to `ready`
|
||||
|
||||
**What happened this session (docs only, no code):** the whole
|
||||
[porch track](porch/00-story.md) 2–8 was brainstormed to `readiness: ready`
|
||||
against `.dev/reference/fiber`; the track's language bill is three small builtins
|
||||
(`random_bytes`, `deflate`/`crc32`, `time.utc`), plus two gaps moved out to
|
||||
runtime-v2 ([7 observability](runtime-v2/07-observability.md),
|
||||
[8 symmetric cipher](runtime-v2/08-symmetric-cipher.md)). And
|
||||
[language 41](language-runtime-database/41-actor-arena-crash.md) — the arena
|
||||
hang — was **root-caused and its fix designed to `ready`**.
|
||||
|
||||
**Language 41, settled:** the hang is a **double free from a broken invariant**.
|
||||
`wo_db_rpc` marshals ("VM heaps are never read cross-shard"), but cross-shard
|
||||
actor `send`/`call` pointer-shares the message into the receiver's shard — a
|
||||
worker then drops an object in the sender's arena. **Fix = marshal cross-shard
|
||||
messages** (copy into the receiver's arena, matching the DB RPC), which
|
||||
eliminates the class by construction without needing the exact aliasing site;
|
||||
**plus** aligning the `shard_id % nshards` route/compare mismatch and asserting
|
||||
`shard_id < nshards`. Poison-on-free and a minimal corpus fixture are named
|
||||
follow-ups. Proven against `archive/porch-idempotency` 18a–18h/19.
|
||||
|
||||
**Next steps:** implement language 41's marshal fix (unblocks
|
||||
[porch 9](porch/09-idempotent-replay.md), already written); then porch is
|
||||
buildable — [porch 5](porch/05-routing-response-ergonomics.md) has zero upstream
|
||||
deps and is the natural start, with `random_bytes` (porch 2) opening the 3→4
|
||||
chain.
|
||||
|
||||
### Landed 2026-09-04 — wmux: switch-client + choose-session (rung 21)
|
||||
|
||||
**Implemented (2026-09-04):** an in-session `switch-client -t B` / `-l`
|
||||
moves a live client between sessions — the Input `call`s the session (sync,
|
||||
so a refusal keeps the client on A), a `reg` handle threaded into every
|
||||
session resolves B, the fds hand off WITHOUT closing, B adopts them and
|
||||
spawns a fresh Input, the old Input exits only on success; B-occupied
|
||||
refuses, missing→error, per-session `last_session` for `-l`. Plus a native
|
||||
**`choose-session` picker** (bound `prefix o`): lists sessions on the status
|
||||
row, a digit switches. Brainstormed + story-refined first (readiness ready),
|
||||
then built. Landmine: a `?actor RMsg` nullable field reordered the checkpoint
|
||||
schema (spurious restart migration) — fixed with a non-nilable `me` +
|
||||
`DeadReg` placeholder. Gate 52 → 54/0; committed + reinstalled. Full fuzzy
|
||||
`sesh connect` (fzf + zoxide/config dirs) still needs the tmux-compatible
|
||||
CLI shim — that's rung 23.
|
||||
|
||||
### Landed 2026-09-04 — wmux: theming, active-pane border, automatic-rename
|
||||
|
||||
**Implemented (2026-09-04):** three developer-requested UI features, each
|
||||
gated (`just wmux` 50 → 52/0) + committed + reinstalled.
|
||||
- **Theming (rung 22, first slice)** — a tmux-style style engine
|
||||
(`style_sgr`: `fg=`/`bg=` named/`bright*`/`colourN`, plus
|
||||
bold/dim/italic/underscore/reverse), read from durable options
|
||||
(`opt_style`). Applied to `status-style`, `window-status-current-style`,
|
||||
`pane-active-border-style` — all `set-option`-able, restart-durable.
|
||||
- **Active-pane border** — the split divider is now the active border:
|
||||
drawn in `pane-active-border-style` (green default) as box-drawing, with
|
||||
a marker pointing at the focused pane (`◄`/`►` / `▲`/`▼`). Focus changes
|
||||
(click / `select-pane` / cycle) redraw so it follows immediately. This
|
||||
answers the "panes not clickable" report — clicks always worked (the
|
||||
cursor moved), there was just no visible cue.
|
||||
- **Automatic-rename** — the VTE captures the pane's OSC 0/1/2 title; the
|
||||
Window pushes it to a non-durable `AutoName` table; `window_list` shows
|
||||
it unless a manual `rename-window` overrides. So starship/vim/bash
|
||||
setting the title renames the window (refreshed on the status tick).
|
||||
|
||||
**Also fixed same day:** the popup mouse wheel (was dropped — now forwarded
|
||||
to the popup's app as translated SGR) and mouse re-arm after a full-screen
|
||||
app disables it (`\e[?1000l` on exit); both gated.
|
||||
|
||||
**`.dev/reference`:** the developer's live lazydocker session + screenshots;
|
||||
tmux's `pane-active-border-style` / `automatic-rename` / style-string shape.
|
||||
|
||||
**Still the big ones (each its own focused run, NOT rushed):** N-way panes
|
||||
(3+) + break/swap-pane (rung 10 full), sesh `switch-client` (20), plugin
|
||||
ports thumbs/fzf/fzf-url (21), control-mode `%notifications` (14),
|
||||
run-shell/if-shell, pane/layout persistence (16). And the rest of the new
|
||||
[mouse-UX rung 19](wmux/19-mouse-ux.md): status-line click → window,
|
||||
drag-resize, drag-select.
|
||||
|
||||
### Landed 2026-09-04 — wmux: lazydocker popup renders clean (OSC + flicker)
|
||||
|
||||
**Reported:** the developer ran `lazydocker` via `display-popup`; it showed
|
||||
`8;;` garbage smeared across every border/panel and flickered — while the
|
||||
same lazydocker in tmux was pixel-clean.
|
||||
|
||||
**Root-caused (measured, not guessed):** captured lazydocker's real pty
|
||||
bytes and replayed a 24 KB slice through the VTE. Two gaps, both gated now:
|
||||
- **OSC dropped.** lazydocker wraps every bordered element in an OSC-8
|
||||
hyperlink (`\e]8;;URI\e\…\e]8;;\e\`). The ESC dispatch had no `\e]` arm,
|
||||
so it dropped `\e]` and printed the `8;;` payload as cells. Now `\e]…`
|
||||
consumes the string to its `ST`/`BEL` terminator (covers OSC 0/2 title +
|
||||
OSC 52 clipboard); `\e(`/`\e)`/`\e*`/`\e+` eat their charset byte too (no
|
||||
literal `B`/`0`). The garbage was **repaint-only** — a focused pane
|
||||
passes OSC raw to the client's own terminal, which handles it.
|
||||
- **Flicker.** `PopReader` read 4 KB, so a ~24 KB frame split into ~6
|
||||
partial repaints. It now reads 64 K and drains already-available bytes
|
||||
(3 ms poll) → one frame, one paint.
|
||||
|
||||
**Verified:** replay of the real capture shows box-drawing/colours/text
|
||||
intact, zero `8;;`. New gate leg runs an OSC-8 line in a popup (the
|
||||
repaint path) and asserts clean. `just wmux` **46 → 47/0**. Binary
|
||||
rebuilt (`woc build … -o wmux`) + reinstalled to `~/.local/lib/wmux/wmux`.
|
||||
|
||||
**`.dev/reference`:** a live `lazydocker` capture via a python pty harness;
|
||||
the developer's screenshots (wmux vs tmux side by side).
|
||||
|
||||
**Next:** the big remaining parity items are unchanged (N-way panes, sesh
|
||||
switch-client, plugin ports) — see the marathon entry below.
|
||||
|
||||
### Landed 2026-09-03 — wmux tmux-parity marathon (7 rungs, all gated)
|
||||
|
||||
**Implemented (2026-09-03, "complete all rungs" multi-rung push):** the
|
||||
switch-blocking tmux features, each gated (`just wmux` 45 → 46/0) +
|
||||
committed + reinstalled to `~/.local/lib/wmux/wmux`, in dependency order:
|
||||
- **Rung 10 core** — `split-window -h` (side-by-side) + `-v`; directional
|
||||
`select-pane -L/R/U/D` (h/j/k/l); zoom (`resize-pane -Z`). Fixed a
|
||||
spawn-time winsize race (spawn panes at their band size).
|
||||
- **Rung 12** — system **clipboard** (OSC 52 on yank, verified base64);
|
||||
**mouse** (SGR enable on attach, wheel→copy-scroll, click→select-pane).
|
||||
- **Rung 20** (committed at the time as "rung 19") — **display-popup**: a
|
||||
session-owned modal float running a command (lazygit/lazydocker/sesh) in
|
||||
a bordered box, reaped on exit.
|
||||
- **Rung 13** — copy-mode **char selection** (vi `v`/`y`, highlighted,
|
||||
multi-line range yank → buffer + OSC 52).
|
||||
- **Rung 15** — terminfo-lite: accept the common TERM families (tmux/
|
||||
screen/alacritty/kitty/…), still refuse `dumb`.
|
||||
- **Rung 17 (full)** — **`#(shell-command)`** (cached, `time.after`-
|
||||
refreshed) PLUS the recursive expander: `#{?cond,a,b}` conditionals,
|
||||
`#{b:}`/`#{d:}` modifiers, `#{time}` clock, `#{host_short}`.
|
||||
- **Window names** — durable `rename-window`, shown as `[idx:name*]`,
|
||||
`#{window_name}`; **last-window** (`prev`).
|
||||
- (Earlier same day) **Rung 18** — key tables (`bind-key -n`, Meta/named
|
||||
keys); and the screen-completeness cluster (UTF-8, sizing, alt-screen,
|
||||
erase 0/1, SGR reset, O(n log n) replay).
|
||||
|
||||
**Config:** `~/.config/wmux/wmux.conf` maps the developer's tmux binds
|
||||
(prefix C-a, `-n M-h/M-l`, h/j/k/l select-pane, z zoom, %, G/D/T popups),
|
||||
auto-loaded by the launcher.
|
||||
|
||||
**Daily-drivable now on xterm.** Remaining for FULL parity (larger, each
|
||||
its own effort) — rung numbers corrected to final: **N-way panes (3+)** +
|
||||
break/swap-pane (rung 10 full); ~~format conditionals/modifiers (17)~~ and
|
||||
~~theming (22)~~ and ~~sesh switch-client (21)~~ have since landed;
|
||||
**plugin ports** thumbs/fzf/fzf-url + the tmux-compat CLI shim (rung 23,
|
||||
popup+capture-pane ready); control-mode `%notifications` (14),
|
||||
run-shell/if-shell, pane/layout persistence (16).
|
||||
|
||||
**`.dev/reference`:** the developer's `~/.tmux.conf` + plugins (the target
|
||||
config), tmux `popup.c`/`window-copy.c`/`tty.c` shapes, runtime `proc`/
|
||||
`term`/`time` seams.
|
||||
|
||||
### Landed 2026-09-03 — wmux screen-completeness + rung 18 (key tables, first slice)
|
||||
|
||||
**Implemented (2026-09-03):** a burst of real-usage fixes driven by
|
||||
running wmux with the developer's live starship/eza + tmux setup, plus
|
||||
the first slice of the new [key-tables rung 18](wmux/18-key-tables.md).
|
||||
- **Dynamic sizing** — the session sizes to the client's terminal via
|
||||
`term.size` at attach (was a fixed 80×23 box); gate 120×40 → 39×120.
|
||||
- **Screen completeness (vte.wo)** — the alternate screen (`\e[?1049h/l`,
|
||||
so btop/vim stop bleeding into the shell), scroll region + cursor
|
||||
save/restore, erase-display modes 0/1 (`\e[J` clears stale lines below
|
||||
the cursor), a per-row SGR reset (no colour-bleed blank rows), and
|
||||
**UTF-8 decoding** (one cell per glyph via `term.width` — nerd-font/CJK/
|
||||
emoji render instead of `000`).
|
||||
- **O(n log n) boot replay** — killed an O(n²) startup CPU burst.
|
||||
- **Rung 18 slice** — a no-prefix `RootBind` table, `key_code` (Meta +
|
||||
named keys), a tty key decoder in the Input actor, `bind-key -n`/`-T`.
|
||||
|
||||
**Key findings (measured):** a byte-based VTE mangles every multi-byte
|
||||
glyph — `feed()` now decodes UTF-8 lead+continuation bytes into one cell
|
||||
and `term.width(cp)` sets the advance. `\e[K` erases with the CURRENT
|
||||
SGR, so an un-reset colour painted whole blank rows once the grid filled
|
||||
the screen. And a **compiler bug** surfaced: `self.f = self.f .. x`
|
||||
(self-referential field concat-assign) miscompiles — worked around with a
|
||||
local temp, `emit.ml` fix tracked. Gate `just wmux` 37 → 42, 0 failures.
|
||||
|
||||
**Dependencies unblocked:** rung 18's decoder + tables are the seam the
|
||||
copy-mode-vi table and `-r` repeat extend; UTF-8 + sizing make the VTE
|
||||
usable for real prompts/TUIs. The audit's siblings have since landed with
|
||||
final numbers — 17 formats-v2, 20 display-popup, 21 sesh switch-client, 22
|
||||
theming all DONE; 23 plugin ports (the tmux-compat CLI shim) remains.
|
||||
|
||||
**Next steps:** finish rung 18 (copy-mode-vi + `-r`, with rungs 10/13), or
|
||||
the paused rung 10–22 story map; the `emit.ml` self-concat fix is a
|
||||
standalone language follow-up.
|
||||
|
||||
**`.dev/reference` used:** the developer's own `~/.tmux.conf` + plugins
|
||||
(the config the fixes had to render), and the runtime's `term`/`sysio`
|
||||
seams (`term.size`/`term.width`, EIO).
|
||||
|
||||
### Landed 2026-09-02 — wmux 16 (first slice): the Window owns + reaps its panes
|
||||
|
||||
**Implemented last time (2026-09-02):** the pane-ownership + reaping fix,
|
||||
a discrete slice of [rung 16](wmux/16-durability-polish.md) surfaced by
|
||||
live usage — a pane whose child exited on its own (a shell exiting, a
|
||||
command pane finishing) was left a `<defunct>` zombie until the session
|
||||
was killed. The `Window` actor now spawns its OWN panes: `make_window`
|
||||
and `do_split` `call` the Window (`kind 0` / `kind 6`), it `spawn_pane`s
|
||||
in-actor (so `owner_actor` = the Window) and returns the reader fd. A
|
||||
pane's Reader, on EOF/EIO, sends `kind 8`; the Window `wait_dl`s the
|
||||
child on its own shard to reap it and marks it dead. `DeadWin` and the
|
||||
`kind 9` replay-feed became dead code and were removed. Gate leg
|
||||
`attach-zombie` added; `just wmux` 36 → 37, 0 failures.
|
||||
|
||||
**Key findings (measured, not asserted):** the leak was structural, not
|
||||
a missing `wait()` — the runtime's child slots are PER-SHARD, so
|
||||
`proc.wait_dl(id)` only works from the actor that spawned the child.
|
||||
Instrumentation proved it: two live panes reported the SAME shard-local
|
||||
id (64), and the reader's `wait_dl` trapped `"process id is not a live
|
||||
child"` because it ran on a different shard. So the kill-time `wait_dl`
|
||||
was silently failing too; only owner-actor death (kill-session, server
|
||||
exit) was actually reaping. Moving the spawn into the Window put the
|
||||
child and its reaper on one shard. Verified: after a command pane and a
|
||||
window shell both exit, the server has zero defunct children; 0 traps.
|
||||
|
||||
**Learned:** an actor cannot fetch its own address (no self primitive),
|
||||
so the Reader's window address is threaded from the spawn site — but the
|
||||
child fd can ride back through a synchronous `call` return (WMsg receive
|
||||
returns Int), which let the Window own the spawn while the caller (which
|
||||
holds the window address) wires the Reader. That `call`-returns-a-fd
|
||||
shape is the clean way to keep ownership and wiring in the right actors.
|
||||
|
||||
**Dependencies unblocked:** the rest of rung 16 (pane/layout
|
||||
persistence, killw compaction, named buffers) is unchanged and still
|
||||
pending. The Window-owns-panes shape also makes per-pane resize and
|
||||
future pane persistence cleaner (the Window is now the single owner).
|
||||
|
||||
**Next steps:** the remaining wmux burn-down — rung 10 (layout tree) or
|
||||
rung 12 (resize + mouse); then cherry-pick the wmux track dev→master
|
||||
when the ladder is declared ready.
|
||||
|
||||
**`.dev/reference` used:** the runtime's own `sysio.c`
|
||||
(`WO_B_PROC_WAIT_DL`, `proc_slot_by_id`, `owner_actor`) — to source the
|
||||
per-shard child-ownership model that dictated the fix.
|
||||
|
||||
### Landed 2026-09-02 — wmux 11: options/formats/keys + two baseline bug fixes
|
||||
|
||||
**Implemented last time (2026-09-02):** wmux
|
||||
[rung 11](wmux/11-formats-options-keys.md) — behaviour became durable
|
||||
DATA. A `Setting {key, val}` options table (`set-option`, `wmux_opt`)
|
||||
and a `Bind {key, cmd}` key table (`bind-key`), both seeded idempotently
|
||||
at boot and replayed after a restart; a `#{...}` status-format expander
|
||||
(`format()`); and ONE `run_command`/`run_session_command` dispatcher
|
||||
that the CLI, control mode, the `C-b :` prompt, key bindings and the
|
||||
`WMUX_CONF` config file all feed. Folded rung 14's command-prompt race
|
||||
fix by moving the line editor into the Input actor. Pure `.wo`, zero
|
||||
runtime work. `just wmux` 30→36 checks, 0 failures.
|
||||
|
||||
**Key findings (measured, not asserted):** the tmux options/format/keys
|
||||
DSL (~12k lines in tmux) collapses to two durable tables, a ~40-line
|
||||
`#{...}` walker, and one dispatcher — and unlike tmux the config SURVIVES
|
||||
a server restart (proven: `set-option prefix C-t` + `bind-key X` are in
|
||||
effect after SIGTERM). The prompt race the command-pane rung disclosed is
|
||||
gone: with the flag and keystrokes in one actor, `C-b : split <cmd>` runs
|
||||
the whole line. The gate's old "command prompt neww" leg was a false
|
||||
positive — its `[1` needle matched an ANSI cursor escape, not a window;
|
||||
the new legs assert on the expanded format text instead.
|
||||
|
||||
**Learned (two latent baseline bugs the new paths exposed, both fixed):**
|
||||
(1) `net.read_dl` returns nil on a timeout but TRAPS on a hard error, and
|
||||
a dead PTY master returns EIO, so the pane Reader's `catch (e) nil;
|
||||
continue` pinned a core at 100% the moment ANY command pane's child
|
||||
exited — a pre-existing runaway confirmed identical on the pre-rung-11
|
||||
build. Fix: the catch `return`s (the parser allows a `{ … }` catch arm),
|
||||
stopping the reader like EOF; same guard added to the Input tty read.
|
||||
(2) `kill-session` fired `kind 4` at every window at once and each
|
||||
scanned the shared `Chunk where c.sess` bucket and deleted — colliding
|
||||
cursors trapped WO-5 "no such row"; chunk cleanup moved into the
|
||||
serializing session. Only reachable once prompt-`neww` opened real
|
||||
windows.
|
||||
|
||||
**Dependencies unblocked:** rung 14 narrows to the control-mode command
|
||||
surface + `%notifications` (its prompt-race scope is done). The unified
|
||||
dispatcher is the seam rungs 12–16 extend (each new verb is added once).
|
||||
|
||||
**Next steps:** the remaining wmux burn-down — rung 10 (layout tree) or
|
||||
rung 12 (resize + mouse, runtime already ready via rt2 3/6); then
|
||||
cherry-pick the wmux track dev→master when the ladder is declared ready.
|
||||
|
||||
**`.dev/reference` used:** tmux (`options.c`, `format.c`, `key-bindings.c`
|
||||
— the option/format/key shapes wmux compresses); the runtime's own
|
||||
`sysio.c` `read_dl` (to source the EIO-vs-timeout distinction).
|
||||
|
||||
### Landed 2026-09-02 — runtime-v2 COMPLETE: all five iterations in one run
|
||||
|
||||
**Implemented last time (2026-09-02):** the whole
|
||||
|
|
@ -629,8 +916,9 @@ asset nobody had built.
|
|||
|
||||
**Next steps:** the live slice is iteration 24, untouched by this. CI is
|
||||
release-only — no workflow runs the gates per change, which remains the
|
||||
open half of iteration 30 (observability, CI, fuzz — still no story
|
||||
file).
|
||||
open half of iteration 30 (observability — now
|
||||
[runtime-v2 7](runtime-v2/07-observability.md), moved there 2026-09-06;
|
||||
CI and fuzz are tooling, split out).
|
||||
|
||||
**`.dev/reference` used:** none — GitHub Actions' own docs and the
|
||||
runner images' glibc versions were the only sources.
|
||||
|
|
@ -1231,8 +1519,10 @@ check mode, and the `internal/` dep boundary (WO-E108). Driver-only.
|
|||
|
||||
New 2026-09-01. The I/O plane learned sockets in 8/11/35 and files in 6;
|
||||
this track adds the missing third — **processes, terminals, signals** —
|
||||
five builtin-sized seams, each `runtime/src/` work with a `types.ml` row
|
||||
as its whole compiler cost (the iteration 42 precedent). Iteration 42
|
||||
first five builtin-sized seams, each `runtime/src/` work with a `types.ml`
|
||||
row as its whole compiler cost (the iteration 42 precedent); the track then
|
||||
grew a 6th (terminal measurement) and, 2026-09-06, a 7th and 8th (observability,
|
||||
moved from language 30; a symmetric cipher) both `refine`. Iteration 42
|
||||
(bounded subprocess, ✅ on `master` 2026-09-01) opened the arc from the
|
||||
language track before it had a name. **All five `readiness: ready`** —
|
||||
one track-wide brainstorm settled every fork
|
||||
|
|
@ -1251,19 +1541,45 @@ starts. Edges in [dependency graph section 6](../00-dependency-graph.md).
|
|||
| 4 | [termios adoption](runtime-v2/04-termios.md) | ✅ **DONE 2026-09-02** — `term.raw/restore`; restore proven a runtime obligation twice (DIV0 while raw, and the double-raw refusal itself) |
|
||||
| 5 | [fd passing](runtime-v2/05-fd-passing.md) | ✅ **DONE 2026-09-02** — `net.send_fd`/`recv_fd`/`connect_unix`; a tty crossed the socket, was raw'd through the received copy and restored at destroy — the wmux handover in miniature |
|
||||
| 6 | [term.size + term.width](runtime-v2/06-term-size-width.md) | ✅ **DONE 2026-09-02** — TIOCGWINSZ read twin (nil = not a tty) and libc wcwidth under C.UTF-8; the only runtime work the whole wmux parity ladder needs |
|
||||
| 7 | [observability](runtime-v2/07-observability.md) | ⬜ `refine` — **moved here 2026-09-06** from language iteration 30 (`was_language_iteration: 30`). Runtime metrics/gauges, a `pprof`-equivalent profile, stack-trace-on-trap; consumers named (porch [8](porch/08-static-and-lifecycle.md)/[39](language-runtime-database/39-web-framework-parity.md), databasev2 [5](databasev2/05-bounded-tables-eviction.md), the limiter's lazy expiry). Forks: counters-only vs profiling, exposition format, pull vs push, trace-on-trap as a separable first slice. Stretches the track's charter (instrumentation, not processes/terminals/signals) — noted in the story |
|
||||
| 8 | [symmetric cipher](runtime-v2/08-symmetric-cipher.md) | ⬜ `refine` — **created 2026-09-06** from the [porch↔fiber scope-gap](../plan/exploration/fiber/01-porch-vs-fiber-scope-gap.md); a cipher builtin fits the builtin-seam shape. AEAD for encrypted cookies (porch [2](porch/02-randomness-and-cookies.md) out-of-scope) + data-at-rest; extends [34](language-runtime-database/34-crypto-builtins.md)'s digests, needs porch 2's `random_bytes` for nonces. Load-bearing fork: AES-256-GCM (expected, hard constant-time in software) vs ChaCha20-Poly1305 (easier hand-roll, no-dep doctrine fit) |
|
||||
| 9 | [in-process TLS](runtime-v2/09-in-process-tls.md) | ⬜ `refine` — **created 2026-09-07** from the gap [jarvis](jarvis/00-story.md) surfaces. TLS **both directions** (outbound client + inbound termination), **retiring the "TLS is the proxy's job" doctrine** (recorded in 34/38/porch). The track's heaviest seam — **not** builtin-sized, and likely a **vendored-lib exception** to no-external-deps (TLS is the one thing not to hand-roll). Load-bearing fork (left open): vendor mbedTLS/BearSSL vs hand-roll a subset. Sits on language 38's `net.connect`; gates jarvis entirely |
|
||||
|
||||
### ▸ wmux — the terminal multiplexer track
|
||||
|
||||
New 2026-09-01, from [the tmux parity study](../plan/exploration/tmux/00-tmux-parity.md).
|
||||
First of the *softwares built with writeonce* tracks: the product is an
|
||||
end-user program, not a library. Its runtime prerequisites are the
|
||||
[runtime-v2 track](runtime-v2/00-story.md) above — iteration 42 was the
|
||||
first domino; runtime-v2 1–5 remain, streaming-subprocess first — plus
|
||||
the VTE grid + unicode width work wmux 1 itself owns.
|
||||
New 2026-09-01, from [the tmux parity study](../plan/exploration/tmux/00-tmux-parity.md);
|
||||
**re-scoped 2026-09-02 to FULL tmux parity** as a nine-rung ladder
|
||||
([ladder spec](../superpowers/specs/2026-09-02-wmux-ladder-design.md)),
|
||||
serving the recorded goal: acceptance by Linux-based developers. Runtime
|
||||
prerequisites ALL landed (runtime-v2 1–6); every rung past 1 is pure
|
||||
`.wo`. Durability is the ladder-wide differentiator — every rung's
|
||||
state replays after a server restart, which tmux loses by design.
|
||||
|
||||
| # | Iteration | State |
|
||||
| --- | --- | --- |
|
||||
| 1 | [wmux](wmux/01-wmux.md) *(was language 43)* | ⬜ `refine` — five forks recorded (terminfo, v1 surface without split panes, scrollback residency, command surface, streaming verb shape). The beyond-tmux leg: durable sessions replay layout + scrollback after a server RESTART |
|
||||
| 1 | [foundation](wmux/01-wmux.md) *(was language 43)* | ✅ **DONE 2026-09-02** — sessions, attach by fd-handover, durable capped chunk log, restart replay; `just wmux` 19/0 under a real PTY harness incl. the beyond-tmux restart-replay leg |
|
||||
| 2 | [the screen](wmux/02-the-screen.md) | ✅ **DONE 2026-09-02** — VTE grid (vte.wo); reattach/restart paint the screen, proven grid-specific in the gate |
|
||||
| 3 | [windows + status](wmux/03-windows-and-status.md) | ✅ **DONE 2026-09-02** — actor-per-window, status line, C-b c/n/p/digit; windows durable across restart |
|
||||
| 4 | [split panes](wmux/04-split-panes.md) | ✅ **DONE 2026-09-02** — vertical 2-pane split, focus, composite grid render (N-way/horizontal deferred) |
|
||||
| 5 | [copy mode](wmux/05-copy-mode.md) | ✅ **DONE 2026-09-02** — scrollback, copy-mode paging, yank to a durable paste buffer, paste |
|
||||
| 6 | [multi-client](wmux/06-multi-client.md) | ✅ **DONE 2026-09-02** — multi-client mirroring (min-size/live-resize/mouse deferred) |
|
||||
| 7 | [command system](wmux/07-command-system.md) | ✅ **DONE 2026-09-02** — C-b : prompt (neww/split/next/prev/killw) + config file (session directive) |
|
||||
| 8 | [hooks + control](wmux/08-hooks-and-control.md) | ✅ **DONE 2026-09-02** — control-mode line protocol + session-created hooks delivered as actor messages |
|
||||
| 9 | [parity audit](wmux/09-parity-audit.md) | ✅ **DONE 2026-09-02** — the tmux-vs-wmux catalog: every gap a named follow-up or a refusal by name |
|
||||
| 10 | [layout tree](wmux/10-layout-tree.md) | 🟡 `refine` — **first slice DONE 2026-09-03**: horizontal `split-window -h`, directional `select-pane -L/R/U/D` (h/j/k/l), zoom (`resize-pane -Z`); rung-22 added the active-pane border marker. Still 2-pane max — N-way (3+), swap/break-pane, presets, durable layout pending |
|
||||
| 11 | [formats + options + keys](wmux/11-formats-options-keys.md) | ✅ **DONE 2026-09-02** — durable options + `bind-key` tables, `#{...}` status format, one `run_command` dispatcher behind CLI/control/prompt/keys/config; folded rung 14's prompt-race fix. Fixed two baseline bugs: a PTY-EIO reader 100%-CPU spin and a concurrent kill-session chunk race. `just wmux` 36/0 |
|
||||
| 12 | [resize + mouse](wmux/12-resize-and-mouse.md) | 🟡 `refine` — **attach-time sizing + SGR mouse DONE 2026-09-03/04**: sizes to the client's terminal via `term.size` (was fixed 80×23); wheel→copy-scroll, click→select-pane, status-row click→window; mouse re-armed after a full-screen app. Live SIGWINCH resize + multi-client min-size still pending |
|
||||
| 13 | [copy selection + search](wmux/13-copy-selection-search.md) | 🟡 `refine` — **char-range selection DONE 2026-09-03**: vi `v`/`y`, highlighted, multi-line yank → buffer + OSC 52. Only incremental search + rectangle select pending |
|
||||
| 14 | [control surface + prompt fix](wmux/14-control-and-prompt.md) | ⬜ `refine` — broaden control mode. **Prompt-race fix DONE in rung 11**; this rung narrows to the control-mode command surface + `%notifications` |
|
||||
| 15 | [terminfo](wmux/15-terminfo.md) | 🟡 `refine` — **terminfo-lite DONE 2026-09-03**: a TERM allowlist (xterm/screen/tmux/alacritty/kitty/…; refuses only `dumb`) retired the foreign-`TERM` refusal. Full compiled-terminfo parsing still pending |
|
||||
| 16 | [durability polish](wmux/16-durability-polish.md) | 🟡 `refine` — pane/layout persistence, killw compaction, buffers STILL pending. **First slice DONE 2026-09-02**: the Window owns + reaps its panes (spawns them in-actor so `wait_dl` works on its shard), fixing a `<defunct>` zombie leak; gate leg `attach-zombie`, 37/0 |
|
||||
| 17 | [formats v2](wmux/17-formats.md) | ✅ **DONE 2026-09-03** — the full status format engine: `#(shell)` (cached, timer-refreshed), recursive `#{...}` with `#{?cond,a,b}` conditionals, `#{b:}`/`#{d:}` modifiers, `#{time}`/`#{host_short}`/real `#{window_name}` |
|
||||
| 18 | [key tables](wmux/18-key-tables.md) *(new, from the config audit)* | 🟡 `refine` — **first slice DONE 2026-09-03**: no-prefix root table (`bind-key -n`), Meta + named keys (`key_code`), a tty key decoder in the Input actor; gate `bind-key -n M-h` fires without prefix, 42/0. `copy-mode-vi` table, `-r` repeat still pending |
|
||||
| 19 | [mouse-driven UX](wmux/19-mouse-ux.md) *(new)* | 🟡 `refine` — **active-pane border highlight DONE 2026-09-04** (the split divider is the focus border + a direction marker; click flips it) plus **status-row click → select window** (2026-09-04). Still pending: drag-resize, drag-select. Forks open — brainstorm the rest before build |
|
||||
| 20 | [display-popup](wmux/20-display-popup.md) | ✅ **DONE 2026-09-03/04** — session-owned modal float (`-E`), `-B` borderless (flush app frame), rounded gray border, popup mouse-wheel forwarding; drove the VTE OSC-swallow + popup frame-coalesce fixes. (Committed as "rung 19" then renumbered) |
|
||||
| 21 | [sesh + switch-client](wmux/21-sesh-switch-client.md) | ✅ **DONE 2026-09-04** — in-session `switch-client -t B` / `-l`: sync `call` hand-off (Input exits only on success), `reg` threaded into sessions, fds handed off without close, B adopts + spawns a fresh Input; B-occupied refuses, missing→error. Plus a native **`choose-session` picker** (prefix `o`) — lists sessions on the status row, a digit switches. Fixed a `?actor RMsg` nullable-wrapper schema reorder (spurious restart migration) with a non-nilable `me`. Gate 54/0. (Full fuzzy `sesh connect` needs the tmux-compat CLI — rung 23) |
|
||||
| 22 | [theming](wmux/22-theming.md) | ✅ **DONE 2026-09-04** — a tmux-style `style_sgr` engine (fg/bg named/bright/colourN + attrs) read from durable options; applied to `status-style`, `window-status-current-style`, `pane-active-border-style` (active border). Plus **automatic-rename** (window name follows the pane's OSC title). Later surface (`mode-style`, per-window format styling, `message-style`) can extend it |
|
||||
| 23 | [plugin ports](wmux/23-plugin-ports.md) | ⬜ `refine` — run the developer's tmux plugins (thumbs, fzf, fzf-url) under wmux via `capture-pane` + a popup picker. Forks open (port each vs a `tmux` compat shim, which plugins first, capture scope). Rides rung 20 + rung 12 |
|
||||
|
||||
### Language track — sequenced, on the critical path
|
||||
|
||||
|
|
|
|||
107
docs/stories/jarvis/00-story.md
Normal file
107
docs/stories/jarvis/00-story.md
Normal file
|
|
@ -0,0 +1,107 @@
|
|||
# Story — `jarvis`, the writeonce AI assistant
|
||||
|
||||
The sixth track, and the second whose product is an end-user *program* rather
|
||||
than a language capability — after [`wmux`](../wmux/00-story.md), the terminal
|
||||
multiplexer. Where wmux proves writeonce can build the tool a developer lives
|
||||
in, jarvis proves it can build the tool of the moment: an AI assistant, written
|
||||
end to end in `.wo`, durable and single-binary by construction. It serves the
|
||||
same north star — adoption by Linux developers — by meeting them where the
|
||||
attention is.
|
||||
|
||||
Numbering restarts at 1 and is local to this track; frontmatter carries
|
||||
`track: jarvis`. Status rules are the repo's, unchanged: `status:` in
|
||||
frontmatter is the only place state lives, no directory encodes it.
|
||||
|
||||
## The problem, stated once
|
||||
|
||||
An assistant's whole job is to reach a model, and the runtime cannot reach
|
||||
anything outbound. It learned to *listen* — sockets in 8/11/35, unix sockets and
|
||||
peer address in 35 and runtime-v2 — but it has never learned to *dial*: there is
|
||||
no `net.connect` (outbound TCP), confirmed against `runtime/src/wob.h` (the net
|
||||
builtins stop at listen/accept/read/write plus the unix-socket client), and no
|
||||
outbound TLS client anywhere. An LLM API is HTTPS on a remote host. jarvis
|
||||
therefore does not begin until two things exist.
|
||||
|
||||
The design chosen is **direct outbound HTTPS** — jarvis dials the LLM API
|
||||
itself, keeping the pure single-binary story. That gates the whole track on
|
||||
language work:
|
||||
|
||||
- **`net.connect`** — outbound TCP —
|
||||
[language 38](../language-runtime-database/38-content-platform-capabilities.md),
|
||||
written but not yet built.
|
||||
- **An outbound TLS client** — HTTPS over that socket — now owned by
|
||||
runtime-v2 [9](../runtime-v2/09-in-process-tls.md) (in-process TLS), created
|
||||
2026-09-07 from this gap. It **retires the standing "TLS is the proxy's job"
|
||||
doctrine**, giving the runtime TLS both directions — the load-bearing choice
|
||||
jarvis's direct-HTTPS design forced into the open.
|
||||
|
||||
A **local-gateway alternative was considered and set aside**: jarvis could speak
|
||||
to a small companion process over a unix socket (`net.connect_unix`, id 107) or
|
||||
spawn one (`proc.spawn`, id 97) — both exist today — and let that companion do
|
||||
the HTTPS, exactly as inbound TLS terminates at a proxy. It is buildable now.
|
||||
It was rejected in favour of the single-binary story, in which the assistant
|
||||
owns its own connection rather than shipping a second executable.
|
||||
|
||||
## Architecture
|
||||
|
||||
browser ⇄ jarvis (a porch app) ⇄ [blocked seam: net.connect + TLS] ⇄ LLM API
|
||||
|
||||
Requests arrive at a porch web app; the answer streams the other way, token by
|
||||
token, LLM → jarvis → browser, over porch's SSE. Conversation state is durable
|
||||
in a `@table`, so history survives a restart with no external store — the
|
||||
writeonce differentiator wmux already showed for session state, applied to chat.
|
||||
|
||||
## The iterations
|
||||
|
||||
Ordered by dependency; the first rung is the whole end-to-end seam, and nothing
|
||||
past it is worth building until that seam is proven.
|
||||
|
||||
| # | Iteration | Delivers | Needs |
|
||||
| --- | --- | --- | --- |
|
||||
| 1 | the chat loop | a prompt sent to one LLM, tokens streamed back to the browser, the conversation persisted durably | the outbound seam (language 38 + TLS); porch 2/3/6/7; wo-html |
|
||||
| 2 | tool use / the agent loop | function-calling and multi-step orchestration through actors — where "assistant" becomes "agent" | 1 |
|
||||
| 3 | retrieval (RAG) | embeddings + vector search over a document set; carries its own sub-gap — an embeddings call over the same outbound path, plus a vector store (pure-`.wo` or a new primitive, decided in that story) | 1, and the embeddings/vector decision |
|
||||
|
||||
Sketched, not committed — named so the shape is visible, not to schedule them:
|
||||
**model routing / multi-model** (choose a backend per request) and an **MCP
|
||||
client** (jarvis as an MCP host, calling tools over the protocol) — both
|
||||
on-brand, both later.
|
||||
|
||||
Only iteration 1's scope is settled by this overview; every iteration file is
|
||||
written and refined to `ready` before its code lands, per the repo's story
|
||||
discipline.
|
||||
|
||||
## Dependencies
|
||||
|
||||
Consumed, and already `ready` or shipped:
|
||||
|
||||
| Needs | From |
|
||||
| --- | --- |
|
||||
| signed cookies, session id | porch [2](../porch/02-randomness-and-cookies.md) |
|
||||
| durable conversation history, revocable sessions | porch [3](../porch/03-sessions.md) + `@table` |
|
||||
| incremental response writes | porch [6](../porch/06-streaming-core.md) |
|
||||
| token streaming to the browser | porch [7](../porch/07-sse-and-compression.md) (SSE) |
|
||||
| the chat UI | `wo-html` / `writeonce-view` |
|
||||
|
||||
Blockers, which must land before iteration 1 starts:
|
||||
|
||||
| Blocker | Owner |
|
||||
| --- | --- |
|
||||
| outbound TCP (`net.connect`) | language [38](../language-runtime-database/38-content-platform-capabilities.md) |
|
||||
| outbound TLS client | runtime-v2 [9](../runtime-v2/09-in-process-tls.md) — in-process TLS, created 2026-09-07 from this gap; **retires the proxy-termination doctrine** |
|
||||
|
||||
## What this track does NOT own
|
||||
|
||||
| Not jarvis's | Why |
|
||||
| --- | --- |
|
||||
| local, in-process model inference | needs an ML runtime and heavy FFI — against the no-external-dependency doctrine |
|
||||
| the local-gateway companion process | considered and rejected (above) in favour of the single-binary story |
|
||||
| voice / audio in or out | a separate surface with its own capture and codec story; no rung asks for it |
|
||||
| starting before the blockers land | like [porch 9](../porch/09-idempotent-replay.md) waiting on language 41, jarvis waits on the outbound seam — documented, not worked around |
|
||||
|
||||
## Review protocol
|
||||
|
||||
Same as every track: the developer reads one iteration, approves or amends, and
|
||||
the next starts only after approval. Each iteration is an unsplittable value
|
||||
slice with phases, Given/When/Then acceptance criteria, and an out-of-scope
|
||||
list, proven by a gate before it is called done.
|
||||
|
|
@ -2,7 +2,7 @@
|
|||
track: language-runtime-database
|
||||
iteration: "41"
|
||||
status: in-progress
|
||||
readiness: refine
|
||||
readiness: ready
|
||||
---
|
||||
|
||||
# 41 — the actor arena crash: a SIGSEGV under concurrent parked callers
|
||||
|
|
@ -65,13 +65,229 @@ So the hang is its own defect and needs its own investigation. It was not
|
|||
caught in this pass: a `gdb` attach needs the hung process held open, and eight
|
||||
scripted attempts to catch one in the act did not land inside the time budget.
|
||||
|
||||
**Where to look first.** `wo_engine_stop` sets `eng_shutdown`, wakes each
|
||||
### Status 2026-09-05 — the hang is ROOT-CAUSED. It is a double free.
|
||||
|
||||
Neither of the two candidates below was right, and the third guess in this
|
||||
file — "two shards route the same payload to each other" — is half right: it is
|
||||
**one** shard routing to *itself*.
|
||||
|
||||
`gdb` is still unusable here (`ptrace_scope=1` blocks a sibling tracer), so the
|
||||
evidence came from `/proc` plus counters compiled into the runtime. Two CPU
|
||||
samples a second apart during a live hang: `Threads: 1`, state `R`, `wchan 0`,
|
||||
utime 157 → 212 and stime 148 → 193. One thread spinning at 100%, every worker
|
||||
already joined — so the hang is past `pthread_join`, and `main()` had returned.
|
||||
|
||||
**The livelock, measured.** A counter around `while (eng_settle_inboxes() > 0)`
|
||||
shows it returning **12, forever**: `moved_total` is exactly 12 × passes at one
|
||||
million passes. Dumping those envelopes names the mechanism:
|
||||
|
||||
```
|
||||
L41 env: inbox=19 vm_rt_shard=19 kind=2 obj_shard=32019 class_id=-3888 nshards=20
|
||||
```
|
||||
|
||||
`obj_shard=32019` is not a shard. It is the **high 16 bits of a pointer**
|
||||
(`0x7d13`), and `class_id` is that pointer's low 32 bits. `wo_arena_free` frees
|
||||
a block by writing the freelist next-pointer over its first 8 bytes — which is
|
||||
exactly `class_id` (0..3), `shard_id` (4..5), `flags`, `pad`. So the payload
|
||||
being settled is **an already-freed block**, and the header being read is a
|
||||
freelist link.
|
||||
|
||||
That garbage id is what makes it spin rather than crash or leak.
|
||||
`wo_route_free` pushes to `INBOX[shard_id % WO_ENG_MAX_SHARDS]`, but
|
||||
`wo_drop_obj` compares the **unmasked** `shard_id` against `rt->shard_id`.
|
||||
`32019 % 64 == 19`, so the envelope lands back in the very inbox it came from,
|
||||
is judged foreign again, and routes again. Any id ≥ 64 congruent to a live
|
||||
shard mod 64 livelocks the settle loop.
|
||||
|
||||
**Where the freed block enters.** A `backtrace()` on any route whose
|
||||
`shard_id >= nshards` puts the origin in the RUN, not in teardown:
|
||||
|
||||
```
|
||||
main → wo_vm_call → vm_run (vm.c:2163) → wo_vm_adopt (vm.c:174, case 2,
|
||||
a home-routed free) → wo_drop_obj → class_free (gc.c:66)
|
||||
→ wo_drop_kind → wo_drop_obj → wo_route_free
|
||||
```
|
||||
|
||||
The parent object is intact; its **fields** are not:
|
||||
|
||||
```
|
||||
L41 BADFIELD: parent class_id=0 shard=0 | field #0 kind=4 target_shard=29774
|
||||
L41 BADFIELD: parent class_id=0 shard=0 | field #2 kind=4 target_shard=29774
|
||||
L41 BADFIELD: parent class_id=0 shard=0 | field #3 kind=4 target_shard=29774
|
||||
```
|
||||
|
||||
Kind 4 is `WO_K_MULTI`. A shard-0 object owns three `multi` containers that
|
||||
live in **another shard's arena** and have already been freed. Freeing the
|
||||
parent drops them a second time. This is why the allocation-heavy arm was the
|
||||
only one that ever failed and the four-scalar arm never did: **containers
|
||||
crossing the mailbox are the trigger**, not `call` and not allocation volume as
|
||||
such. Slot recycling under insert+delete churn only decides how fast the freed
|
||||
block gets reused, which is why N=4/N=5 looked causal.
|
||||
|
||||
### Which side drops first — settled 2026-09-06
|
||||
|
||||
Measured, not inferred. The instrument is a per-arena history table: every
|
||||
`wo_arena_alloc` (both the bump and freelist paths) and every `wo_arena_free`
|
||||
records the class the block carried plus two return addresses, keyed by block
|
||||
address. A dangling pointer's history then names the free that orphaned it. The
|
||||
earlier suspicion — the parked-`call` re-execution dropping its moved argument
|
||||
twice — was **wrong**.
|
||||
|
||||
The block that starts the livelock has this history, all of it in **shard 1's**
|
||||
arena:
|
||||
|
||||
```
|
||||
-- PARENT 0x73cdc7fff1e0 --
|
||||
#0 ALLOC bump via wo_obj_new+0x53
|
||||
#1 FREE as user(14) via class_free <- wo_drop_obj+0xcb
|
||||
```
|
||||
|
||||
**First drop: the object's own home shard, legitimately.** Shard 1 allocated it
|
||||
as class 14 and freed it through the ordinary owned-graph path. Nothing is wrong
|
||||
up to here.
|
||||
|
||||
**Second drop: a worker executing the compiled `DROP` opcode**, walking a
|
||||
*different* owner graph that still reaches the same block:
|
||||
|
||||
```
|
||||
wo_vm_serve → vm_run (vm.c:2153, CASE(DROP)) → wo_drop_obj
|
||||
→ class_free (gc.c:67) → wo_drop_kind → wo_drop_obj
|
||||
→ multi_free (gc.c:42) → wo_drop_kind → wo_drop_obj
|
||||
→ class_free → wo_drop_kind → wo_drop_obj → wo_route_free ← stale
|
||||
```
|
||||
|
||||
So the defect is **an owned subtree reachable from two owner graphs**: one shard
|
||||
1 has already destroyed, one still live on a worker. It is shared where it
|
||||
should have been transferred. The `multi_free` frame in the middle is why only
|
||||
the container-carrying arm ever failed.
|
||||
|
||||
**Why the second drop lands on shard 0 rather than trapping.** `wo_arena_free`
|
||||
writes the freelist next-pointer over the block's first 8 bytes. When the block
|
||||
is the **tail** of its size class that pointer is NULL, so the header reads back
|
||||
`class_id 0, shard_id 0, flags 0` — and class 0 is a *valid* class index. The
|
||||
worker sees `shard_id 0 != its own`, routes the free to shard 0; shard 0 adopts
|
||||
it (`wo_vm_adopt` case 2), matches `0 == 0`, concludes "we are home", and runs
|
||||
`class_free` with **class 0's** field kinds over a dead class-14 object. Class
|
||||
0's kinds say fields #0/#2/#3 are `WO_K_MULTI`; the slots actually hold the dead
|
||||
object's Text pointers, already freed by shard 1. Those carry freelist links as
|
||||
headers, so their `shard_id` is a pointer's high 16 bits — 29645, 32019 — and
|
||||
the modulo alias sends them back to the inbox they came from, forever.
|
||||
|
||||
Three distinct defects, in fix order:
|
||||
|
||||
1. **The aliased subtree** — the root cause. A child owned by two graphs.
|
||||
Not yet localised to the code path that creates the alias; that is the next
|
||||
question, and the `DROP` site plus the `multi_free` frame are where to look.
|
||||
2. **A freed block is indistinguishable from a live class-0 object.** A NULL
|
||||
freelist link forges a valid header. A poison class id (or a free bit in
|
||||
`flags`) would turn every one of these into an immediate, named trap instead
|
||||
of a silent misinterpretation — cheap, and it would have caught this on the
|
||||
first run.
|
||||
3. **The modulo alias.** `wo_route_free` pushes to `INBOX[shard_id % 64]` while
|
||||
`wo_drop_obj` compares the unmasked id. Mask consistently, or refuse to route
|
||||
an id ≥ `nshards`.
|
||||
|
||||
**Two defects, and the order matters.** Bounding the settle loop would stop the
|
||||
hang and leave a double free behind, turning a visible spin into a silent
|
||||
corruption. Fix the ownership bug first; the modulo alias is a real second
|
||||
defect worth its own fix (mask consistently, or refuse to route an id ≥
|
||||
`nshards`), but it is not the root cause.
|
||||
|
||||
Reproduction, all of it scripted: worktree at `archive/porch-idempotency`,
|
||||
cherry-pick `9dca0b4` onto it, then loop `scripts/web-app-accept.sh`. The
|
||||
`/proc` dump, the settle counter, the envelope trace and the bad-field trace are
|
||||
each a few lines against `vm.c` and `gc.c`.
|
||||
|
||||
**Where to look first (superseded — kept for the record).**
|
||||
`wo_engine_stop` sets `eng_shutdown`, wakes each
|
||||
worker's eventfd, then `pthread_join`s. Two candidates worth eliminating before
|
||||
anything else: a worker blocked in `wo_io_wait` with a fiber parked on a `call`
|
||||
whose reply will never arrive, and the primary spinning in
|
||||
`while (eng_settle_inboxes() > 0) {}` if two shards can route the same payload
|
||||
to each other indefinitely. The second is cheap to rule out with a counter.
|
||||
|
||||
## Fix design — settled 2026-09-06 (`readiness: ready`)
|
||||
|
||||
The root cause is a **broken invariant**, not a stray double free. `wo_db_rpc`
|
||||
states the invariant plainly (`vm.c:281`): *the args are ENCODED into engine
|
||||
slots on THIS thread — VM heaps are never read cross-shard.* The DB path
|
||||
marshals. But cross-shard actor `send`/`call` (`vm.c:1098-1112`) does
|
||||
`e->payload = msg_val` — it **pointer-shares** the message into the receiver's
|
||||
shard. A worker then reads and eventually drops an object that lives in the
|
||||
sender's arena, and the double free, the class-0 forge and the modulo livelock
|
||||
are all downstream of that single violation.
|
||||
|
||||
### Decision 1 — marshal cross-shard messages (root fix)
|
||||
|
||||
Cross-shard `send` and `call` copy the message into the receiver's arena on the
|
||||
crossing, exactly as `wo_db_rpc` already marshals its args. No pointer crosses
|
||||
an arena boundary, so the double-free class is eliminated **by construction** —
|
||||
and, importantly, the exact aliasing *site* need not be localised, because the
|
||||
fix removes the shared pointer rather than the specific graph that aliased it.
|
||||
It restores the "heaps are never read cross-shard" invariant the actor path
|
||||
currently breaks, and it closes the latent hazard beyond the double free: a
|
||||
worker reading sender-arena fields is unsafe under GC or compaction even when
|
||||
the ownership happens to be clean. The cost is a copy per cross-shard message —
|
||||
the same cost the DB RPC already pays, and correctness outranks the zero-copy
|
||||
the current path was reaching for. Same-shard send is unchanged (the arena is
|
||||
shared, the pointer move is correct — which is why `WO_SHARDS=1` never failed).
|
||||
|
||||
### Decision 2 — fix the modulo alias and bound the shard id (this iteration)
|
||||
|
||||
`wo_route_free` pushes to `INBOX[shard_id % nshards]` while `wo_drop_obj`
|
||||
compares the **unmasked** `shard_id` against `rt->shard_id`; that mismatch is
|
||||
what makes a stale free self-route forever instead of resolving. The two sites
|
||||
are made consistent, **and** both assert `shard_id < nshards` — an out-of-range
|
||||
id is impossible for a live object, so hitting it is a corrupt or freed header
|
||||
and must trap loudly rather than route somewhere. This lands with the root fix
|
||||
because it is the guard that would have turned the original silent livelock into
|
||||
an immediate diagnostic.
|
||||
|
||||
### Decision 3 — poison-on-free is a follow-up, not this iteration
|
||||
|
||||
The deeper defensive fix — stamping a freed block's header (a free bit in
|
||||
`flags`, or a poison `class_id`) so a NULL freelist link can never forge a valid
|
||||
class-0 object — is deferred to its own story. Decision 2's bounds assert already
|
||||
catches the specific corrupt-header shape this bug produces at route time; the
|
||||
general poison is broader and separable.
|
||||
|
||||
### Decision 4 — prove against the existing repro; a minimal corpus fixture is a follow-up
|
||||
|
||||
The marshal fix is proven against the `archive/porch-idempotency` reproduction:
|
||||
recover the branch, cherry-pick `9dca0b4`, and run `scripts/web-app-accept.sh`
|
||||
sections 18a–18h and 19 to stability (the idempotency legs that failed one run
|
||||
in six). A minimal, deterministic corpus fixture — a cross-shard `send` of an
|
||||
object carrying an owned subtree (`multi`/`Text`), both sides then dropping,
|
||||
under `WO_SHARDS>1` and ASan — is worth pinning but is its own follow-up; it is
|
||||
not required to land the fix.
|
||||
|
||||
### Phases
|
||||
|
||||
- **A — marshal.** Make cross-shard `send`/`call` (kinds 0 and 5) copy the
|
||||
payload into the receiver's arena on the crossing, mirroring `wo_db_rpc`. The
|
||||
monitor path (kind 7) carries a payload too and is audited the same way.
|
||||
Same-shard paths untouched.
|
||||
- **B — the modulo/bounds guard.** Align the `shard_id` comparison in
|
||||
`wo_drop_obj` with the masking in `wo_route_free`, and assert `shard_id <
|
||||
nshards` at both the route and the home-check.
|
||||
- **C — prove and close.** Re-run the archive repro's 18a–18h/19 to stability
|
||||
under ASan, confirm the settle loop no longer spins, and unblock
|
||||
[porch 9](../porch/09-idempotent-replay.md).
|
||||
|
||||
### Acceptance criteria
|
||||
|
||||
- **Given** a cross-shard `send`/`call` of an object with an owned subtree,
|
||||
**when** both the sender's graph and the receiver drop, **then** each block is
|
||||
freed exactly once and no free is routed across an arena boundary.
|
||||
- **Given** the `archive/porch-idempotency` repro under `WO_SHARDS>1` and ASan,
|
||||
**when** sections 18a–18h and 19 run repeatedly, **then** they are stable —
|
||||
the one-run-in-six idempotency hang is gone — and the settle loop terminates.
|
||||
- **Given** a header carrying a `shard_id >= nshards`, **when** it reaches the
|
||||
drop/route path, **then** it traps loudly rather than self-routing.
|
||||
- **Given** every same-shard workload, **when** the runtime battery and corpus
|
||||
run, **then** they are byte-for-byte unchanged — the marshal cost falls only
|
||||
on the cross-shard path.
|
||||
|
||||
## The original symptom
|
||||
|
||||
Two shapes, believed at the time to share one cause:
|
||||
|
|
@ -136,5 +352,17 @@ fixed.
|
|||
|
||||
## Out of scope
|
||||
|
||||
Fixing the porch feature that found it. That is [porch 9](../porch/09-idempotent-replay.md),
|
||||
and it is already written; it only needs this to land first.
|
||||
- **Fixing the porch feature that found it.** That is
|
||||
[porch 9](../porch/09-idempotent-replay.md), already written; it only needs
|
||||
this to land first.
|
||||
- **Poison-on-free** (decision 3) — a separate defensive story: stamp a freed
|
||||
header so a NULL freelist link can never forge a valid class-0 object,
|
||||
trapping any stale drop rather than misreading it. Needs a language-track
|
||||
number when picked up.
|
||||
- **A minimal deterministic corpus fixture** (decision 4) — a cross-shard `send`
|
||||
of an object with an owned subtree, both sides dropping, under `WO_SHARDS>1` +
|
||||
ASan. Worth pinning; its own follow-up.
|
||||
- **The two smaller runtime defects found alongside** (above): the
|
||||
`try EXPR catch (e) nil` Int-0-vs-trap ambiguity and the `json.decode ... as T`
|
||||
cross-return-boundary corruption. Both worked around in the archived code;
|
||||
each deserves its own minimal fixture and fix, neither blocks this.
|
||||
|
|
|
|||
|
|
@ -73,7 +73,7 @@ risky work starts.
|
|||
| typed binding of query/params/form into a class | language: [`@derive`](../language-runtime-database/29-compile-time-metaprogramming.md) — reflection is forbidden by principle 13 |
|
||||
| TTL cache, `transaction { }`, durable job queue | language: [iteration 18](../language-runtime-database/18-memory-db-features.md) |
|
||||
| a `proxy` middleware | language: [iteration 38](../language-runtime-database/38-content-platform-capabilities.md) — needs `net.connect`, which does not exist |
|
||||
| metrics, profiling, per-change CI, fuzzing | language iteration 30 (no story file yet) |
|
||||
| metrics, profiling, per-change CI, fuzzing | [runtime-v2 7](../runtime-v2/07-observability.md) — observability (was language iteration 30; metrics/profiling/trace-on-trap; CI + fuzz are tooling, split out) |
|
||||
| TLS, HTTP/2 | nobody — proxy-terminated by doctrine |
|
||||
| a runtime template engine | nobody — rejected; markup is a compile-time literal (`writeonce-view`) |
|
||||
| a radix-tree router | nobody yet — waiting on a *measurement*, not a decision |
|
||||
|
|
|
|||
|
|
@ -34,7 +34,17 @@ cross a unix socket. Five seams, each builtin-sized, each in
|
|||
| 4 | [termios adoption](04-termios.md) | the process's OWN tty into raw mode and back — adopting a terminal it was given |
|
||||
| 5 | [fd passing](05-fd-passing.md) | SCM_RIGHTS over unix sockets — detach/attach's foundation, and the Wayland stage's later |
|
||||
|
||||
**ALL FIVE LANDED 2026-09-02, one execution run** (plan:
|
||||
The track then grew past its original five seams — same shape (a builtin in
|
||||
`runtime/src/` with a `types.ml` row), broader than processes/terminals/signals:
|
||||
|
||||
| # | Iteration | What it adds |
|
||||
| --- | --- | --- |
|
||||
| 6 | [term.size + term.width](06-term-size-width.md) | the two terminal-measurement verbs the wmux ladder asked for |
|
||||
| 7 | [observability](07-observability.md) | metrics/gauges, a `pprof`-equivalent profile, stack-trace-on-trap (moved from language iteration 30, 2026-09-06) — `refine` |
|
||||
| 8 | [symmetric cipher](08-symmetric-cipher.md) | AEAD (encrypt/decrypt) for encrypted cookies and data at rest, extending iteration 34's digests (moved from language iteration 43, 2026-09-06) — `refine` |
|
||||
| 9 | [in-process TLS](09-in-process-tls.md) | TLS **both directions** — an outbound client (dial HTTPS) and inbound termination — **retiring the "TLS is the proxy's job" doctrine**; created 2026-09-07 from the gap [jarvis](../jarvis/00-story.md) surfaces. The track's heaviest seam (not builtin-sized; likely a vendored-lib exception) — `refine` |
|
||||
|
||||
**ALL FIVE [1–5] LANDED 2026-09-02, one execution run** (plan:
|
||||
[`2026-09-01-runtime-v2.md`](../../superpowers/plans/2026-09-01-runtime-v2.md);
|
||||
three implementation amendments in the spec's History). Gates:
|
||||
`test_proc` 193/0 + `test_term` 60/0 inside a fully green ASan suite on
|
||||
|
|
|
|||
88
docs/stories/runtime-v2/07-observability.md
Normal file
88
docs/stories/runtime-v2/07-observability.md
Normal file
|
|
@ -0,0 +1,88 @@
|
|||
---
|
||||
track: runtime-v2
|
||||
iteration: "7"
|
||||
was_language_iteration: "30"
|
||||
status: pending
|
||||
readiness: refine
|
||||
---
|
||||
|
||||
# runtime-v2 7 — observability: metrics, profiling, and traces on trap
|
||||
|
||||
> Moved 2026-09-06 from the language track (was language iteration 30, the
|
||||
> number a dozen docs still point at) into runtime-v2, whose builtin-sized-seam
|
||||
> shape it fits. It stretches the track's original processes/terminals/signals
|
||||
> charter — observability is runtime instrumentation of the VM, GC and shards —
|
||||
> but the track already grew past its first five seams. **`readiness: refine`** —
|
||||
> the gap, its consumers and its forks are named here, nothing is brainstormed to
|
||||
> `ready` yet.
|
||||
|
||||
## Why this exists
|
||||
|
||||
The runtime has no observability surface. A healthcheck answers one bit
|
||||
(up/not-up); nothing exposes counters, gauges, latencies, memory, or a
|
||||
profile. Every attempt to reason about the system's behaviour at runtime hits
|
||||
the same wall, which is why the number is referenced from five directions at
|
||||
once:
|
||||
|
||||
- **porch** excludes `expvar`, `pprof` and metrics endpoints by pointing here
|
||||
([story 8](../porch/08-static-and-lifecycle.md),
|
||||
[39](../language-runtime-database/39-web-framework-parity.md)) — a healthcheck is one bit, not
|
||||
observability.
|
||||
- **databasev2** cannot observe table size without it
|
||||
([bounded-tables 5](../databasev2/05-bounded-tables-eviction.md)) and leans on
|
||||
it for per-change benchmark CI
|
||||
([databasev2 story](../databasev2/00-story.md)); the RAM-ceiling study read
|
||||
RSS from `/proc` by hand ([databasev2 1](../databasev2/01-ram-ceiling-measurement.md))
|
||||
precisely because this does not exist.
|
||||
- The **rate limiter**'s ephemeral-row expiry is lazy "because porch has no
|
||||
timer and iteration 30 owns" the sweep story (that "iteration 30" is now this
|
||||
one — [porch 1](../porch/01-store-backed-middleware.md)).
|
||||
|
||||
Fiber ships `expvar` and `pprof` as middleware; the equivalents here are runtime
|
||||
work, because the numbers they expose (allocations, fiber counts, shard load,
|
||||
GC pauses) live in the C runtime, not in `.wo`.
|
||||
|
||||
## What it should deliver (scope to be refined)
|
||||
|
||||
- **Runtime counters and gauges** — allocations, arena high-water, live fiber and
|
||||
actor counts, per-shard load, GC pause totals, request counters — exposed
|
||||
through one endpoint the app can mount.
|
||||
- **A profiling story** — CPU and heap sampling, the `pprof` equivalent, so a hot
|
||||
path can be found rather than guessed at.
|
||||
- **A stack trace on trap** — today a trap is a 500 and a line; a trace at the
|
||||
trap site is the cheapest debugging win and may be separable from the metrics
|
||||
work.
|
||||
|
||||
## Forks the brainstorm must settle
|
||||
|
||||
1. **Counters only, or profiling too?** Counters and gauges are a bounded, mostly
|
||||
`.wo`-plus-a-few-builtins surface; CPU/heap profiling needs sampling
|
||||
machinery in the runtime and is a much larger commitment. Splitting profiling
|
||||
into its own iteration is a legitimate outcome.
|
||||
2. **Exposition format.** Prometheus text (the ops-standard, scrape-friendly),
|
||||
an `expvar`-style JSON blob, or both. The format decides who can consume it
|
||||
without a translator.
|
||||
3. **Pull endpoint or push.** A mounted `/metrics` endpoint (pull) fits the
|
||||
proxy-fronted, single-binary model; a push to a collector needs
|
||||
`net.connect`, which does not exist (iteration 38) — so pull is almost
|
||||
certainly the answer, but say so.
|
||||
4. **Is stack-trace-on-trap in this iteration at all?** It is separable, it is
|
||||
the highest debugging value per line, and it touches the trap path rather than
|
||||
the metrics path — a candidate to land first and alone.
|
||||
|
||||
## Out of scope (named, owned elsewhere)
|
||||
|
||||
- **Per-change CI and fuzzing.** Frequently lumped under the old "iteration 30"
|
||||
but they are tooling and process, not a runtime surface; they belong to a
|
||||
CI/ops story, not this one. The benchmark *harness* already exists (language
|
||||
iteration 22).
|
||||
- **Distributed tracing / OpenTelemetry export.** Needs `net.connect`
|
||||
(iteration 38) and a wire protocol; a later slice if a consumer appears.
|
||||
- **Alerting, dashboards.** Downstream of exposition, not the runtime's job.
|
||||
|
||||
## Info
|
||||
|
||||
Consumers exist and are named above, so this is not a primitive shipped as
|
||||
decoration. Ordering: stack-trace-on-trap has no dependency and could lead;
|
||||
counters/gauges are next; profiling is the heaviest and most separable. Nothing
|
||||
here depends on the porch track — the dependency runs the other way.
|
||||
87
docs/stories/runtime-v2/08-symmetric-cipher.md
Normal file
87
docs/stories/runtime-v2/08-symmetric-cipher.md
Normal file
|
|
@ -0,0 +1,87 @@
|
|||
---
|
||||
track: runtime-v2
|
||||
iteration: "8"
|
||||
status: pending
|
||||
readiness: refine
|
||||
---
|
||||
|
||||
# runtime-v2 8 — a symmetric cipher: authenticated encryption for cookies and data at rest
|
||||
|
||||
> Created 2026-09-06 from the porch-vs-fiber scope-gap analysis
|
||||
> ([exploration](../../plan/exploration/fiber/01-porch-vs-fiber-scope-gap.md)) as
|
||||
> a runtime-v2 iteration — a hand-rolled cipher builtin with a `types.ml` row is
|
||||
> exactly the track's builtin-sized-seam shape (the iteration 42 precedent). (It
|
||||
> is not a language-track iteration; the language number 43 was already spent on
|
||||
> the wmux foundation.) **`readiness: refine`** — the
|
||||
> gap, its consumer and its forks are named; not brainstormed to `ready`.
|
||||
|
||||
## Why this exists
|
||||
|
||||
The runtime has digests only — SHA-1, SHA-256, HMAC-SHA256, base64
|
||||
([iteration 34](../language-runtime-database/34-crypto-builtins.md)) — and, from the porch track,
|
||||
`random_bytes` ([porch 2](../porch/02-randomness-and-cookies.md)). Those let a
|
||||
program *authenticate* and *sign* a value, and *mint* a random one. None of them
|
||||
let it *encrypt* — turn a plaintext into a ciphertext only the key-holder can
|
||||
read.
|
||||
|
||||
That absence is a named porch limit:
|
||||
[porch 2](../porch/02-randomness-and-cookies.md) scopes out **encrypted
|
||||
cookies** explicitly — "Fiber's `encryptcookie` needs a symmetric cipher, and
|
||||
the runtime has digests only. Signed-and-readable is honest and sufficient for a
|
||||
session id; encrypting a payload is a separate ask with a separate primitive
|
||||
behind it." This is that separate primitive.
|
||||
|
||||
Signed-and-readable (what porch has) is correct for a session id — the client
|
||||
may see it, it just may not forge it. Encryption is for the case where the
|
||||
*payload itself* must be hidden from the client: an encrypted cookie carrying
|
||||
app state, or a database field encrypted at rest.
|
||||
|
||||
## What it should deliver (scope to be refined)
|
||||
|
||||
- **An AEAD primitive** — authenticated encryption with associated data — as one
|
||||
or two builtins in the crypto family beside `hmac_sha256`: encrypt (key,
|
||||
nonce, associated-data, plaintext) → ciphertext+tag, and decrypt returning the
|
||||
plaintext or nil on any authentication failure. AEAD, not a bare cipher,
|
||||
because unauthenticated encryption is a footgun that ships.
|
||||
- **The porch consumer**: an `encryptcookie`-equivalent — a cookie whose value is
|
||||
encrypted, not merely signed — layered on iteration 2's cookie machinery.
|
||||
|
||||
## Forks the brainstorm must settle
|
||||
|
||||
1. **Which cipher? This is the load-bearing fork.** AES-256-GCM is what browsers,
|
||||
fiber and every peer expect — but constant-time AES in pure software (no
|
||||
AES-NI intrinsics) is genuinely hard to get right. ChaCha20-Poly1305
|
||||
(RFC 8439) is modern, is far easier to implement constant-time in portable C,
|
||||
and is what a from-scratch no-dependency runtime should probably prefer — at
|
||||
the cost of being less "expected." The runtime hand-rolls its crypto (the
|
||||
SHA-256 precedent, no external dependency), which weighs toward ChaCha.
|
||||
2. **Nonce management.** A reused nonce is catastrophic for both GCM and ChaCha.
|
||||
Caller-supplied nonces put that footgun in every app; a builtin-generated
|
||||
random nonce (drawing on iteration 2's `random_bytes`, prepended to the
|
||||
ciphertext, as fiber's `NewGCMWithRandomNonce` does) removes it. Leaning
|
||||
builtin-generated — so this iteration is ordered after porch 2's builtin.
|
||||
3. **Key handling.** A raw 32-byte key (from `random_bytes`, carried as base64 in
|
||||
config, the `encryptcookie.GenerateKey` shape) with a length check, versus a
|
||||
passphrase-plus-KDF. Leaning raw key with validation; a KDF is its own ask.
|
||||
4. **Hand-roll versus vendor.** Doctrine is no external dependencies. A
|
||||
hand-rolled ChaCha20-Poly1305 in C is bounded and well-specified; hand-rolled
|
||||
AES-GCM is more error-prone. This fork is the practical face of fork 1.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- **Asymmetric crypto** (RSA, ECDH, signatures beyond HMAC). A different, much
|
||||
larger surface with no current consumer.
|
||||
- **Key rotation, a KMS, envelope encryption.** Operational key management is its
|
||||
own story if a consumer appears.
|
||||
- **TLS.** Proxy-terminated by doctrine; this cipher is for application payloads,
|
||||
not the transport.
|
||||
- **Compression before encryption** (the CRIME/BREACH interaction). A caller
|
||||
concern to document, not a primitive.
|
||||
|
||||
## Info
|
||||
|
||||
One named consumer today (encrypted cookies), with database-field-at-rest as a
|
||||
plausible second — enough to not be decoration, not so much as to over-build.
|
||||
Depends on iteration 2's `random_bytes` (for the nonce, fork 2) and extends
|
||||
iteration 34's crypto builtins. Pure compute — no actors, not exposed to the
|
||||
lang-41 hang.
|
||||
117
docs/stories/runtime-v2/09-in-process-tls.md
Normal file
117
docs/stories/runtime-v2/09-in-process-tls.md
Normal file
|
|
@ -0,0 +1,117 @@
|
|||
---
|
||||
track: runtime-v2
|
||||
iteration: "9"
|
||||
status: pending
|
||||
readiness: refine
|
||||
---
|
||||
|
||||
# runtime-v2 9 — in-process TLS: retiring the proxy-termination doctrine
|
||||
|
||||
> Created 2026-09-07 from the gap [`jarvis`](../jarvis/00-story.md) surfaces — an
|
||||
> assistant must dial an LLM over HTTPS, and the runtime has no outbound TLS. The
|
||||
> developer chose the **full overturn**: the runtime gains TLS **both
|
||||
> directions**, and the standing "TLS is the proxy's job" doctrine is retired.
|
||||
> **`readiness: refine`** — the gap, its consumers and its forks are named here;
|
||||
> the implementation is deliberately left as this story's load-bearing fork, not
|
||||
> settled.
|
||||
|
||||
## Why this exists — and what it overturns
|
||||
|
||||
Three documents record the same standing decision, and this story reverses it:
|
||||
|
||||
- *"TLS — permanently the proxy's job (framework doctrine)"* —
|
||||
[language 34](../language-runtime-database/34-crypto-builtins.md) (crypto
|
||||
builtins, line ~83).
|
||||
- *"TLS — proxy-terminated, by doctrine, unchanged… the story says so out loud
|
||||
rather than implying HTTPS clients"* —
|
||||
[language 38](../language-runtime-database/38-content-platform-capabilities.md)
|
||||
(which adds `net.connect` as **plaintext** outbound TCP and explicitly refuses
|
||||
HTTPS).
|
||||
- *"TLS, HTTP/2 | nobody — proxy-terminated by doctrine"* —
|
||||
[porch](../porch/00-story.md)'s "what this track does NOT own".
|
||||
|
||||
The doctrine was reasonable while nothing in-tree needed to *dial* anything: a
|
||||
front proxy terminates inbound TLS, and there were no outbound callers. jarvis
|
||||
breaks that — its whole job is to reach a remote API — and the developer's
|
||||
direct-HTTPS choice for it means the runtime, not a companion, owns the
|
||||
connection. Rather than carve out a one-directional exception, the decision is to
|
||||
give the runtime TLS in **both** directions: outbound so a `.wo` program can dial
|
||||
HTTPS, and inbound so porch can terminate TLS itself instead of mandating a
|
||||
proxy in front of every deployment.
|
||||
|
||||
This is **not a builtin-sized seam** like the rest of this track. TLS 1.3 plus
|
||||
X.509 certificate validation is a large, security-critical subsystem — the one
|
||||
place the runtime's hand-roll-everything habit (the sha256 precedent) should not
|
||||
be assumed to extend. That tension is the load-bearing fork below.
|
||||
|
||||
## What it should deliver (scope to be refined)
|
||||
|
||||
- **Outbound TLS client** — a `.wo` program dials an HTTPS endpoint: a TLS
|
||||
handshake over the TCP socket `net.connect` (language 38) provides, with server
|
||||
certificate validation against a trust store. jarvis's direct path, and
|
||||
language 38's deliberately-excluded HTTPS half.
|
||||
- **Inbound TLS server** — porch terminates TLS on its own listener (cert + key
|
||||
loaded at startup), retiring the "put a proxy in front" requirement for a
|
||||
single-binary deployment.
|
||||
- **Certificate validation and a trust store** — X.509 chain verification,
|
||||
hostname/SNI checks outbound; certificate + private-key loading inbound. This
|
||||
is where most of the risk and most of the code live.
|
||||
|
||||
## Forks the brainstorm must settle
|
||||
|
||||
1. **Implementation source — the load-bearing fork (left open by direction).**
|
||||
Vendor a small, audited TLS library (mbedTLS or BearSSL) compiled into the
|
||||
single static binary — correct and maintainable, but a build-time external
|
||||
dependency, an explicit exception to the no-external-deps doctrine the runtime
|
||||
otherwise holds — versus hand-rolling a TLS 1.3 subset plus X.509 in C, which
|
||||
matches the sha256 precedent but is thousands of security-critical lines and a
|
||||
hand-rolled certificate validator is a CVE factory (strongly discouraged). The
|
||||
honest lean is vendor-a-lib; TLS is exactly the thing not to hand-roll. This
|
||||
fork decides whether the whole "single static binary, no external deps" story
|
||||
gains a footnote.
|
||||
2. **Phasing.** Outbound first (jarvis's actual need) with inbound to follow, or
|
||||
both together since the handshake machinery and the vendored library are
|
||||
shared and only the client-vs-server role and validation direction differ.
|
||||
3. **Trust store and cert provisioning.** Where the outbound trust anchors come
|
||||
from (the system CA bundle, and its path across distros), and how the inbound
|
||||
side is handed its certificate and key (files, env, a reload story).
|
||||
4. **TLS version and cipher policy.** TLS 1.3 only (simplest, modern, smallest
|
||||
attack surface) versus 1.2+1.3 (broader reach). Leaning 1.3-only.
|
||||
|
||||
## Consumers
|
||||
|
||||
Named, so this is not a capability shipped as decoration:
|
||||
|
||||
- **[jarvis 1](../jarvis/00-story.md)** — outbound HTTPS to the LLM API (the
|
||||
reason this story exists).
|
||||
- **porch** — inbound TLS termination, retiring the mandatory front proxy for a
|
||||
single-binary deployment.
|
||||
- **language 38** — the outbound HTTPS half it excluded by doctrine; this story
|
||||
is where that exclusion is lifted.
|
||||
|
||||
## Dependencies
|
||||
|
||||
- **[language 38](../language-runtime-database/38-content-platform-capabilities.md)**
|
||||
— `net.connect` (outbound TCP) is the socket the outbound handshake runs over;
|
||||
the client half of this story sits directly on it.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- **HTTP/2.** A separate protocol concern, parked behind language iteration 23
|
||||
regardless; TLS is its prerequisite, not its owner.
|
||||
- **Mutual TLS / client certificates.** A later slice if a consumer asks; the
|
||||
first cut authenticates the server, not the client.
|
||||
- **Updating the doctrine documents.** Retiring "TLS is the proxy's job" means
|
||||
correcting [language 34](../language-runtime-database/34-crypto-builtins.md),
|
||||
[language 38](../language-runtime-database/38-content-platform-capabilities.md)
|
||||
and [porch](../porch/00-story.md) when this lands — a follow-up bookkeeping
|
||||
pass, named here so it is not forgotten, not part of the runtime work.
|
||||
|
||||
## Info
|
||||
|
||||
This is the heaviest iteration in the runtime-v2 track and the only one that
|
||||
forces a doctrine reversal and, most likely, an external-dependency exception —
|
||||
both flagged above rather than buried. It is pure I/O-plane work (a handshake
|
||||
layer over the existing socket verbs); no actors, so it is not exposed to the
|
||||
lang-41 hang. It gates jarvis entirely: until it lands, jarvis cannot reach a
|
||||
model at all.
|
||||
Loading…
Reference in a new issue