writeonce/docs/stories/language-runtime-database/11-fibers.md
shoney.arickathil 1fe808b7a4 docs(stories): add readiness, retire status: refine, sweep all 47 iterations
- `readiness: ready | refine` is a SECOND axis, orthogonal to status.
  `ready` = the brainstorm is complete and the decisions are LOCKED (a spec
  approved, or the forks explicitly confirmed). `refine` = open forks remain
  and it cannot be planned yet
- `status: refine` RETIRED because it carried both meanings at once, so a held
  iteration with an approved spec (language 18, 26) was indistinguishable from
  one nobody had thought about. status is now purely where the WORK is:
  done | in-progress | pending | hold — `pending` was already the board's own
  rendering word, so nothing new was invented
- all 47 iterations classified from EVIDENCE in their own text, not by guess:
  "the four forks are SETTLED" / "spec + plan approved" / "Approved spec:" for
  ready; "Forks the spec must settle" / "no spec exists yet" for refine. Every
  shipped iteration is ready by definition. 19 done, 5 in-progress, 15
  pending, 8 hold; 27 ready, 20 refine
- two iterations moved refine -> in-progress rather than -> pending: language
  31 and 34 are absorbed into 24 and work on them is literally happening, which
  the board already showed as 🔄 while their frontmatter said otherwise. That
  disagreement is now gone
- board legend, board-views' frontmatter contract, and two new Dataview
  queries updated — the useful one being `readiness: ready AND status:
  pending`, the startable set

WHAT THE NEW AXIS IMMEDIATELY SURFACED: of 15 pending iterations, exactly ONE
is startable — databasev2 4, io_uring group-commit, whose forks were confirmed
settled 2026-08-20. Everything else pending needs a brainstorm first. That was
invisible while one key carried both meanings, and it is now on the board.

Also caught by the sweep, unrelated to readiness but found by cross-checking
frontmatter against the board: SIX duplicate rows. Every iteration moved into
databasev2 was still listed in the LANGUAGE pending table under its retired id
(23, 32, 33, 20, 21, 27) as well as its new one. Stale copies removed. And two
databasev2 rows made claims the sweep contradicts — iteration 1 was billed
"startable today" while its forks are open, and 6 still called itself the
ceiling-raiser after 2 took that role.

Docs only. linkcheck 0 broken / 0 anchors.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 16:54:45 +02:00

141 lines
7.2 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

---
iteration: "11"
status: done
readiness: ready
chain: 1
---
# Iteration 11 — fibers (the 8+11 concurrency arc, part 2)
> Format: `product/story-iteration-template`. Part of
> [Story — one language, one runtime, one database, one binary](00-story.md).
>
> **✅ LANDED 2026-08-21** with the arc's stage 3 (fiber substance landed
> stages 1+2; stage 3 added the plane-less reply park `WO_PARK_INBOX` —
> a fiber parked on the DB actor's reply, woken by the envelope drain).
> RE-SCOPED at close, disclosed: the goal's "blocking builtins park"
> covers `net`/`time`; **`fs` still blocks the shard thread** — v1
> semantics, deliberately: program mode is single-fiber by design, and
> no serving workload reads files mid-request yet. fs-park becomes real
> when a workload demands it (candidate rider on 23's ring work). The
> criteria below are met under that re-scope.
>
> **REFINED 2026-08-20** (developer decisions, no code): 8 and 11 ship as
> **one arc** — the DB becomes an actor on an owner shard
> ([iteration 8](08-shard-actor-runtime.md)'s decision), so serving
> shards must PARK on cross-shard replies, which is this iteration.
> The spawn-surface question below is SETTLED: one unified actor-address
> surface (`spawn` returns an address, fiber or remote alike; `send`
> moves ownership; same-heap sends take the cheap path). The arc's
> driving workload is [iteration 24: chat](24-chat-websocket-workload.md)
> — fiber-per-WebSocket-connection is the serving model that retires the
> framework's close-when-idle keep-alive policy.
>
> **STAGE-1 SUBSTANCE LANDED 2026-08-20** (branch `concurrency-arc`):
> reduction-budget fibers (back-edge accounting — the livelock lesson is
> in the plan's deviations), `spawn`/`send`/`actor M`, one-message-at-a-
> time delivery, main-return reap, fiber-trap isolation, and parked
> `net`/`time` builtins on the io_uring-first per-shard I/O plane
> (`WO_IO=uring|epoll`, epoll fallback proven). Demonstrated by
> `docs/examples/fibers` (`just fibers` 8/0).
>
> **STAGE-2 SUBSTANCE LANDED 2026-08-20** (branch `concurrency-arc`,
> T5–T6): pinned thread-per-core shards, envelope sends with ownership
> move, round-robin placement, home-routed frees, WO-E222 on EVERY
> spawn/send (placement makes any actor potentially remote), TID-verified
> shard context. Deviations disclosed in the plan of record
> ([`2026-08-20-shard-fiber-arc.md`](../../superpowers/plans/2026-08-20-shard-fiber-arc.md)):
> the inbox is a mutex-guarded list + eventfd (rings arrive only if
> iteration 22 measures the mutex as a cost), the deterministic corpus
> pins `WO_SHARDS=1`, two TSan races and one teardown SEGV fixed.
>
> **RE-SEQUENCED 2026-08-21**: what remains for the chain is the arc's
> stage 3 (transparent DB actor — [iteration 8](08-shard-actor-runtime.md)),
> then measurement. Order: **stage 3 → 22 → 31 → 24 → 23 → 32**.
## Goals
- Concurrency **inside** a shard the doctrine way: cooperative fibers
scheduled by the VM — a fiber is the interpreter state wovm already
isolates (register windows + frame stack + pc), made per-fiber. One
kernel task per core stays the rule; thousands of fibers run above it
where the kernel tops out at hundreds of threads (CFS O(log N),
~8 MiB + ~20 µs per thread vs an arena-allocated fiber context).
- **Preemption by reduction budget**, the Erlang shape: the dispatch loop
counts down and parks the fiber at zero — no signals, no safepoints, no
stack copying, deterministic replay.
- **Blocking builtins park instead of block** on server shards: the same
`net`/`time`/`fs` calls that block the thread in program mode hand their
fd to the shard's epoll/io_uring loop and resume on completion. One API,
same source text, no async/await, no function coloring — ever.
- Cross-fiber sends follow the iteration 8 rule — ownership moves — on the
cheaper same-heap path.
## Acceptance Criteria
- What to achieve?
- **Given** a shard running thousands of spawned fibers with one hot
compute loop among them,
- **when** the reduction budget expires,
- **then** the hot fiber parks, every other fiber makes progress
(starvation-free corpus fixture), and output is identical across
runs (deterministic scheduling).
- What to achieve?
- **Given** a fiber blocked in a stdlib builtin on a server shard,
- **when** the completion arrives on the shard's event loop,
- **then** the fiber resumes with the result while the shard served
other fibers in the interim — one OS thread, verified by TID.
- What to achieve?
- **Given** a parked fiber at shard shutdown,
- **when** the shard unwinds it,
- **then** every drop map runs (ASan zero leaks) — parked fibers die
as cleanly as trapped ones. (Iteration 26's blue-green drain reuses
exactly this unwind path.)
- What to achieve?
- **Given** TRACED objects (GC-ness inferred since 7b) referenced only
from a parked fiber's frames,
- **when** the per-shard mark-sweep collector scans,
- **then** parked fibers' frames are roots exactly as the live frame
stack is (`vm_gc_roots` grows fiber awareness) — nothing live is
collected, nothing dead survives. (Originally said `@gc`; restated
2026-08-20 in inference terms.)
## Out Of Scope
- Fiber migration across shards (ownership doctrine forbids it; cross-shard
is a message send, iteration 8).
- Priorities, timers, structured-concurrency policy — recipe-box
capabilities for a later `.wo` library, not runtime policy.
- Program mode: stays single-fiber in v1 (blocking legal, log-watcher
needs nothing more).
- async/await keyword — permanently rejected surface, not deferred.
## Info
- Normative contract (added 2026-08-21): stackless coroutine model,
park/resume protocols, the no-`async` rule —
[`docs/plan/oop-vm/03-concurrency-coroutines.md`](../../plan/oop-vm/03-concurrency-coroutines.md).
- Research note: [`docs/plan/exploration/fibers/00-fibers.md`](../../plan/exploration/fibers/00-fibers.md)
— kernel's-eye evidence (task_struct costs, CFS collapse at high task
counts) and the precedent survey (BEAM reductions adopted; Go stack
copying and Tokio coloring rejected; Loom's park-under-blocking-API
matches the stdlib posture).
- Vision origin: [blue-green vision §3](../../plan/exploration/blue-green-vm/00-vision.md);
iteration 8's scheduler is the substrate this extends.
- Open questions — REDUCED AGAIN 2026-08-21: the spawn surface, budget
size/granularity (back-edge accounting — see the plan's livelock
deviation), run-queue fairness (FIFO), and parked-fiber drop semantics
(fiber-trap isolation + main-return reap) are all settled by the landed
implementation. Still open: how a parked fiber's borrow state interacts
with the shard's GC safepoints — tracked for stage 3 / the collector's
next pass. Lifecycle surface (request/response, backpressure,
supervision, timers) is [iteration 31](31-actor-lifecycle.md)'s, not
this story's.
## Proposed Solution
- The plan exists and its fiber stages are landed:
[`2026-08-20-shard-fiber-arc.md`](../../superpowers/plans/2026-08-20-shard-fiber-arc.md)
(stages 1+2 complete 2026-08-20). This story closes when the arc's
stage 3 lands and iteration 22 records the delta; the chat workload
([iteration 24](24-chat-websocket-workload.md)) is the proof.