- runtime ids 91-95: net.read_dl/accept_dl/write_dl (per-call deadline, nil/false = the EXPECTED timeout; ms<=0 = old behavior bit for bit), net.listen_unix (unlink-before-bind, O_NONBLOCK on the listener — probe-found: accept4's flag covers accepted sockets only), net.peer - plane: one-op-per-park stays law — deadlines ride one per-shard TIMEOUT tick (sentinel user_data) + post-CQE expiry sweep + POLL_REMOVE tombstone; epoll's deadline scan grew the fd-park case; fibers POOL instead of freeing mid-run (stale-CQE UAF); plain parks zero park_deadline (no stale sleep deadlines) - probe: all five seams verified on BOTH WO_IO backends (timeout timing exact, peer round-trip, unix rebind) - framework: parse_request grows first_ms/read_ms; serve_conn — the keep-alive loop with deadlines where parked idle conns are LEGAL (close-when-idle RETIRED); App.handle_conn exposes it; plain serve() unchanged for simple apps - web-app: app-owned accept_dl loop + ConnWorker actor per connection (each builds its own App; cross-shard placement rides the DB actor); WA_IDLE_MS knob; gate grows to 41 checks — two slow requests served in PARALLEL, stalled client evicted at the idle deadline, slow-loris torn at the read deadline (400) - docs: story 35 -> done with banner; SQE/CQE design spec LANDED (was the review doc); ledger rows (timeouts/unix/keep-alive/peer), graph (NETSEAM cleared, KEEPAL done), builtin-surface rows, runtime CODE-LOGIC section, board entry - battery 13/13 fresh-built Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
5.6 KiB
5.6 KiB
Iteration 35 — net seams: the SQE/CQE plane design (for review)
Status: LANDED 2026-08-23 (branch
framework-v1b) — implemented as designed; one addition found by the probe:listen_unixmust set O_NONBLOCK on the LISTENER (accept4's flag covers only accepted sockets). Covers story 35's fork 2 (deadline plumbing on the plane) plus the surface decisions taken with it. Normative park-protocol home once approved:../../plan/oop-vm/03-concurrency-coroutines.md.
Decisions taken (the story's four forks)
- Timeout result: nil/false, never a trap — a timeout is an
EXPECTED outcome (
parse_intdoctrine).net.read_dl -> ?Text(nil = deadline, "" = EOF),net.accept_dl -> ?Int,net.write_dl -> Bool(false = torn mid-write, close the fd). - Plane plumbing: shard-tick TIMEOUT + expiry sweep + POLL_REMOVE tombstone (the table below) — NOT per-fiber second ops, NOT linked ops.
- Surface: per-CALL deadline argument (
_dlbuiltin variants, ids 91–95; 89/90 stay reserved for monitor/time.after). No hidden fd state;ms <= 0= the old blocking behavior bit for bit. - Unix sockets:
net.listen_unix(path)unlinks a stale socket file before bind — a restart never needs manual cleanup.net.peer(fd) -> Text:"ip:port"(TCP),"unix",""on error.
The ring today (arc T4, landed) and the additions
| SQE | user_data | CQE consumer action |
|---|---|---|
| POLL_ADD fd, oneshot — net accept/read/write park | fiber pointer | state == PARKED → wake; else ignore (stale, benign) |
TIMEOUT from fb->park_ts — time.sleep (park_done=1: resume continues PAST the builtin) |
fiber pointer | same wake path; the op IS the waker — 1 op, 1 CQE, consumed exactly at wake |
| POLL_ADD wake_efd, oneshot — inbox envelopes arrived | EFD_SENTINEL (1) |
drain eventfd, re-arm, return "adopt-needed" |
NEW TIMEOUT, one per SHARD ("tick"), armed for the NEAREST fd-park deadline (vm->tick_ts) |
TICK_SENTINEL (2) |
tick_armed = 0; the sweep after the CQE batch wakes every expired fd-park |
NEW POLL_REMOVE, addr = the expired park's fiber pointer (matches its POLL's user_data) |
CANCEL_SENTINEL (3) |
nothing — the tombstone's own completion |
Why this shape
- Standing invariant (arc T4): one park = one SQE, and its CQE is consumed precisely when the fiber wakes — no op ever outlives its fiber.
- The problem a deadline'd read creates: two racing wait sources
(fd readiness, timer). Two per-fiber ops would let the LOSER's CQE
land after the fiber is freed — a use-after-free on the
user_datadereference one wait later. - The fix, three parts:
- fd-parks keep exactly ONE op (their POLL_ADD); deadlines ride the shard tick, whose user_data is a sentinel and can never dangle;
- the post-CQE sweep wakes expired fd-parks and submits POLL_REMOVE
to tombstone the orphaned poll (its -ECANCELED CQE arrives later
with the fiber's user_data and is dropped by the
state == PARKEDguard); - dead fibers are POOLED, never freed mid-run (
vm->fib_pool, freed at vm teardown) — even a post-mortemstateread hits live memory; the worst outcome anywhere is a SPURIOUS wake, which the re-execute protocol absorbs (the builtin re-checks EAGAIN and the deadline). Steady-state pool size = peak live fiber count.
- epoll fallback: zero ops — the existing parked-list deadline scan
(previously sleeps only) now also covers fd-parks with
park_deadline > 0; expiry =EPOLL_CTL_DEL+ wake. - Rejected:
IOSQE_IO_LINKPOLL→TIMEOUT chains (kernel cancels the loser) — tighter, but linked-op error semantics are subtle and the ring stays on 5.1-safe ops by doctrine. Also rejected: per-fd deadline setting (net.set_deadline) — hidden fd state, against the no-coloring lean the story records.
The _dl builtin state machine
- FIRST entry stamps the absolute deadline into the fiber
(
fb->dl_active,fb->dl_at— wall ms). The park protocol RE-EXECUTES a parked builtin, and these fields are how the retry remembers the original deadline. - Every entry retries the syscall. Success/EOF/error → clear
dl_active, answer as the plain builtin would. - EAGAIN with the deadline passed → clear
dl_active, answer the timeout result (nil / false). - EAGAIN before the deadline → park with
park_fdANDpark_deadline = dl_atboth set; whichever fires first resumes the builtin, which loops back to "every entry retries". - Existing plain parks (
net.read/accept/write) now explicitly zeropark_deadline— the sweep must never read a stale sleep deadline off a reused fiber.
Consumers this unblocks (the serving slice, same branch)
- framework
serve_conn(fd, dispatcher, read_ms, idle_ms): the keep-alive loop where an idle parked connection is finally LEGAL (idle deadline evicts it — close-when-idle retires); slow-client reads bounded byread_ms. - App-owned acceptor pattern:
accept_dlloop (stop-flag checked per tick) + the app spawns ITS conn-worker actor per connection, each building its own App (interface-typed actor state — spike-proven). client_ipverification (net.peer) and unix-socket deployment become pure framework/app slices.
Proof plan (pending)
- Full battery (byte-identical:
ms <= 0paths and untouched builtins). - New gate: stalled client evicted at the deadline (fds + RSS flat
over a soak), parallel requests complete out of order, unix listener
- peer round-trip — web-app gate additions + both
WO_IObackends.
- peer round-trip — web-app gate additions + both