writeonce/docs/stories/language-runtime-database/refine/11-fibers.md
shoney.arickathil d24c705860 feat: iterations 19 + 17 — Float/Bytes scalars (.wob v5), library kind + internal/
- Float full stack: literals (fraction/exponent; `0..10` still a range), f64
  opcodes 34-41, @table column, WAL bit-exact replay, json fractions in and
  shortest-round-trip out. IEEE-quiet — FDIV never traps where DIV does.
- Bytes: a wo_str with its own class id, so alloc/free/copy are shared but no
  Text builtin accepts one; len/at/slice/eq/concat, base64 both ways, json
  boundary as base64; TEXT_COPY preserves the kind.
- No implicit Int/Float mixing (WO-E201 in the typechecker, not the emitter,
  which picks the opcode from one side and would misread the other).
- One IEEE deviation: float_cmp total order (NaN last, -0.0 == +0.0) for
  indexes and order-by, keys canonicalized to match. `?Float` nil is a
  reserved quiet NaN — the zero word is +0.0, WO_NIL_SCALAR's bits are -2.0.
- Renderer prefers fixed over exponential in 1e-6..1e21: pure shortest makes
  a price of 900.0 read `9e+02`. One renderer for interp/json/float_to_text.
- Fixed en route: lexer double-counted the leading digit; is_scalar_shaped
  took Float/Bytes as Int-shaped; Bytes ownership needed a shared heap-scalar
  predicate or temps never dropped; order-by bit-compared negatives backwards.
- Iteration 17: `kind = "library"` (absent = program; bad value = WO-E109),
  entry-less check mode retiring the `--emit` workaround, Go's `internal/` as
  WO-E108 at the consumer's `use`. Driver-only; VM/.wob/GC untouched.
- Framework reorg: internal/{parse,serve}.wo; http/form.wo split out to keep
  media_type/form_values public (parse.wo had grown public surface).
- Docs: link audit (97 -> 88 broken, conflict markers resolved, 2 duplicate
  stories removed), 00-code-review verified 26/27, iterations re-sequenced.
- Also carries the pre-staged pub(read)/using/#if work from the index.
- Gates: corpus 103/0, test_wal 156/0, web-app 26/0, oop-accept ALL MET.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 19:24:15 +02:00

5.3 KiB

Iteration 11 — fibers (the 8+11 concurrency arc, part 2)

Format: product/story-iteration-template. Part of Story — one language, one runtime, one database, one binary.

REFINED 2026-08-20 (developer decisions, no code): 8 and 11 ship as one arc — the DB becomes an actor on an owner shard (iteration 8's decision), so serving shards must PARK on cross-shard replies, which is this iteration. The spawn-surface question below is SETTLED: one unified actor-address surface (spawn returns an address, fiber or remote alike; send moves ownership; same-heap sends take the cheap path). The arc's driving workload is iteration 24: chat — fiber-per-WebSocket-connection is the serving model that retires the framework's close-when-idle keep-alive policy.

STAGE-1 SUBSTANCE LANDED 2026-08-20 (branch concurrency-arc): reduction-budget fibers (back-edge accounting — the livelock lesson is in the plan's deviations), spawn/send/actor M, one-message-at-a- time delivery, main-return reap, fiber-trap isolation, and parked net/time builtins on the io_uring-first per-shard I/O plane (WO_IO=uring|epoll, epoll fallback proven). Demonstrated by docs/examples/fibers (just fibers 8/0). The shard-context criteria (TID assertions, cross-shard sends, blue-green drain reuse) close with the arc's stage 2.

Goals

  • Concurrency inside a shard the doctrine way: cooperative fibers scheduled by the VM — a fiber is the interpreter state wovm already isolates (register windows + frame stack + pc), made per-fiber. One kernel task per core stays the rule; thousands of fibers run above it where the kernel tops out at hundreds of threads (CFS O(log N), ~8 MiB + ~20 µs per thread vs an arena-allocated fiber context).
  • Preemption by reduction budget, the Erlang shape: the dispatch loop counts down and parks the fiber at zero — no signals, no safepoints, no stack copying, deterministic replay.
  • Blocking builtins park instead of block on server shards: the same net/time/fs calls that block the thread in program mode hand their fd to the shard's epoll/io_uring loop and resume on completion. One API, same source text, no async/await, no function coloring — ever.
  • Cross-fiber sends follow the iteration 8 rule — ownership moves — on the cheaper same-heap path.

Acceptance Criteria

  • What to achieve?
    • Given a shard running thousands of spawned fibers with one hot compute loop among them,
    • when the reduction budget expires,
    • then the hot fiber parks, every other fiber makes progress (starvation-free corpus fixture), and output is identical across runs (deterministic scheduling).
  • What to achieve?
    • Given a fiber blocked in a stdlib builtin on a server shard,
    • when the completion arrives on the shard's event loop,
    • then the fiber resumes with the result while the shard served other fibers in the interim — one OS thread, verified by TID.
  • What to achieve?
    • Given a parked fiber at shard shutdown,
    • when the shard unwinds it,
    • then every drop map runs (ASan zero leaks) — parked fibers die as cleanly as trapped ones. (Iteration 26's blue-green drain reuses exactly this unwind path.)
  • What to achieve?
    • Given TRACED objects (GC-ness inferred since 7b) referenced only from a parked fiber's frames,
    • when the per-shard mark-sweep collector scans,
    • then parked fibers' frames are roots exactly as the live frame stack is (vm_gc_roots grows fiber awareness) — nothing live is collected, nothing dead survives. (Originally said @gc; restated 2026-08-20 in inference terms.)

Out Of Scope

  • Fiber migration across shards (ownership doctrine forbids it; cross-shard is a message send, iteration 8).
  • Priorities, timers, structured-concurrency policy — recipe-box capabilities for a later .wo library, not runtime policy.
  • Program mode: stays single-fiber in v1 (blocking legal, log-watcher needs nothing more).
  • async/await keyword — permanently rejected surface, not deferred.

Info

  • Research note: docs/plan/exploration/fibers/00-fibers.md — kernel's-eye evidence (task_struct costs, CFS collapse at high task counts) and the precedent survey (BEAM reductions adopted; Go stack copying and Tokio coloring rejected; Loom's park-under-blocking-API matches the stdlib posture).
  • Vision origin: blue-green vision §3; iteration 8's scheduler is the substrate this extends.
  • Open questions to settle in the spec — REDUCED 2026-08-20: the spawn surface is settled (the unified actor address, iteration 8 decision 2). Still open for the arc's spec: budget size and check granularity, parked-fiber drop semantics, run-queue fairness (FIFO v1), and how a parked fiber's borrow state interacts with the shard's GC safepoints.

Proposed Solution

  • No implementation plan exists yet — this iteration starts with the brainstorming → spec → writing-plans chain (the superpowers path every prior iteration followed), then executes that plan. The research note above is the brainstorm's entry material.