writeonce/runtime

wo-rt-c — the writeonce runtime environment, in C

A single-file C implementation of the runtime layer the writeonce language runs on — now at phase E: a durable RAM database. Writes follow the dual-write order — RAM apply, framed WAL record (len|crc32|payload|COMMIT) to a per-shard fallocate'd log, one group-commit fdatasync per loop tick, HTTP ack only after the fsync completion. Boot performs the first load, hard drive → RAM: each shard replays its snapshot + WAL tail into its arena slice in parallel before any accept arms; clean shutdown snapshots each slice and truncates the WAL; a meta file pins the shard count so a mismatched WO_THREADS refuses to boot. ./wo-rt wal-check <file> validates a log offline. WO_THREADS pinned threads (default = online cores), each owning its own raw io_uring ring (io_uring_setup + mmap'd SQ/CQ rings + io_uring_enter — no liburing), its own SO_REUSEPORT listener (multishot accept), its own keep-alive connections, and its own slice of the one mlock'd mmap arena — shared-nothing, no locks. Steady state is one io_uring_enter syscall per loop tick. Thread 0 owns the signalfd; shutdown broadcasts through per-thread eventfds, both watched via POLL_ADD SQEs. Zero dependencies beyond libc + kernel uapi headers. The kernel is the runtime.

This is the runtime-layer sibling of prototypes/wo-db/ (the C++ query-layer prototype): a reference card showing, with no abstraction in the way, exactly which kernel primitives the production Rust runtime (crates/rt/) drives through libc. Same role, different layer.

prototypes/wo-db/     C++   what the LANGUAGE executes   (parser, engine, transactions)
runtime/              C     what the RUNTIME stands on   (epoll, signalfd, sockets)
crates/rt/            Rust  the product — both layers, libc only

Moved from prototypes/wo-rt-c/ to root-level runtime/ (monorepo layout, per the OOP compiler + VM design spec). wo-rt.c is untouched — it stays the event-loop reference described below. The wovm bytecode VM now lives under runtime/src/ — see The wovm bytecode VM for what shipped and Debugging for how to step through it.

Build, run, poke

make                 # cc -O2 -Wall -Wextra -std=c11 -pthread — no libraries
./wo-rt              # 127.0.0.1:8085   (WO_PORT=9000 WO_THREADS=4 ./wo-rt to override)

curl localhost:8085/   # {"runtime":"wo-rt-c","loop":"epoll-et","threads":4,
                       #  "shard":2,"shard_requests":[68,36,44,53]}
curl -X POST localhost:8085/api/notes -d '{"title":"hello"}'
                       # {"id":2,"title":"hello","shard":1}   ← ids interleave per shard
curl localhost:8085/api/notes        # the connection's shard only — shared-nothing
# ctrl-C → signalfd on shard 0 → eventfd broadcast → all shards join

Each connection hashes to one shard for life (SO_REUSEPORT 4-tuple): a list may land on a different shard than the create that preceded it. That is the architecture, not a bug — cross-shard reads are a later phase / design decision (see the architecture doc's improvements).

Or from the repo root: just rt-c-demo.

Module map

Every block in wo-rt.c corresponds one-to-one to a module of the Rust runtime, which in turn mirrors Go's netpoller — the same lineage the docs trace:

wo-rt.c block	Rust (crates/rt/src/)	Go (.dev/reference/go/src/runtime/)	Kernel reference card
main event loop (epoll_create1 / epoll_wait, EPOLLET)	runtime/netpoll_epoll.rs	netpoll_epoll.go	linux/01-epoll.md
sig_setup (sigprocmask + signalfd)	runtime/signalfd.rs	signal mask handling	linux/04-signalfd.md
listener_bind (SOCK_NONBLOCK, accept4-to-EAGAIN)	http/listener.rs	net.Listen + accept loop	socket(7)
conn_drive (read-to-EAGAIN, one buffer per fd)	http/connection.rs	conn.Read loop	the edge-triggered contract
notes[] store	engine.rs (BTreeMaps)	—	03-inmemory-engine.md

What it demonstrates

• One thread owns each shard outright. Accept, parse, store, respond — no locks, no worker pool, no connection migration. Scaling past one core is more shards (09-concurrency-scaleout.md), never shared mutable state. The single cross-thread touch is the relaxed-atomic stats counters on / — monotonic, never on the data path.
• Edge-triggered discipline. Every registration sets EPOLLET; every readiness event is drained to EAGAIN (the accept loop and the read loop both). Get this wrong and connections silently hang — the reason the Rust module documents the same contract at the top of netpoll_epoll.rs.
• Signals as fd events. SIGINT/SIGTERM are blocked, then read from a signalfd on the same epoll — no async-signal-unsafe handler, no self-pipe trick.
• RAM is the read path. GET /api/notes touches a C array. The production engine is the same idea with MVCC and a WAL behind it.

Architecture and roadmap

Documentation lives under docs/ (repo convention) — this README stays here as the directory's orientation page only:

• docs/plan/exploration/c-runtime/01-architecture.md — the runtime defined by tracing one memory address through user space, kernel space, and hardware under a million concurrent connections, plus seven improvement proposals (seqlock reads, registered buffers, zero-copy send, SQPOLL, …).
• docs/plan/exploration/c-runtime/00-plan.md — the phase sequence: A → B → C → D → E → F, all ✅ shipped.
• docs/plan/exploration/blue-green-vm/00-vision.md — where the runtime goes next: port-free transports, fibers, source embedded in the binary, agent-managed source over MCP, and the Blue/Green two-VM hot-swap deployment model.

Measured (phase F, 20-core Linux 6.14, tmpfs data dir, just rt-c-bench)

Same C bench client (bench/bench.c, keep-alive, only 2xx counted) against both servers:

Benchmark	wo-rt-c (8 shards, durable WAL, io_uring + group commit)	Go net/http (go1.25.1, 20 cores, no durability)	Rust wo (release, 8 shards, durable WAL, io_uring group commit)¹
GET /healthz	859,033 req/s · p50 71 µs · p99 159 µs	336,444 req/s · p50 70 µs · p99 1,277 µs	746,340 req/s · p50 73 µs · p99 186 µs
GET / (JSON)	671,312 req/s · p99 180 µs	—	692,671 req/s · p99 167 µs
POST write (tmpfs)	618,343 commits/s — fsync-acked · p99 194 µs	320,516 req/s — RAM only, no WAL · p99 1,581 µs	330,285 commits/s — fsync-acked · p99 360 µs
POST write (real ext4/NVMe)	—	—	27,014 commits/s group commit vs 5,765 per-commit (4.7×) · p50 2.2 ms
10,000 idle conns	0 errors	0 errors	0 errors

¹ All numbers measured on a clean box with the same client (earlier parasite-contaminated runs superseded). The Rust column's history is the architecture roadmap, measured: 09a global mutex (74.9k writes/s) → 09b sharded engine (+51%) → 09c per-shard WAL (~1% durability cost) → keep-alive: reads ×3.4 to 770k/s, durable writes ×1.9 to 331k/s, p99 under 350 µs everywhere. Rust now beats Go on both columns while fsyncing every write, and sits within ~10% of the C prototype on reads — converging exactly as the same-architecture argument predicted. The one remaining C advantage is group commit on io_uring (one batched fsync + one syscall per tick vs per-commit fsync over epoll), which is the next port. Bonus finding: with /tmp accidentally full, the C runtime refused to ack non-durable writes under ENOSPC — the durability guarantee holding in an unplanned failure mode.

wo-rt-c on 8 cores outpaces Go on 20 with ~8× tighter p99 (Go's GC shows there) — while fsyncing every write Go doesn't. Honest caveats: net/http does full general-purpose HTTP; our parser is minimal; .NET was not installed on the box. ACID under load: three crash rounds (kill -9 mid-bench at ~2M commits) all showed WAL records ≥ acked; isolation probe: 300 concurrent commits → 300 distinct ids; torn-tail records drop whole by CRC.

The crash-under-load test found and fixed two real bugs the lighter phase-D test missed: an ack-before-fsync race (conn_continue armed the send in the same tick the commit was staged) and an fd-reuse ABA hazard in ack parking (fixed with per-connection generation stamps). That is what phase F is for.

• docs/plan/exploration/c-runtime/02-single-binary.md — the end goal: how the wo build single binary runs on this runtime environment — the runtime kernel is statically linked into every writeonce app (Go model, nothing to install), with the catalog/routes/bytecode payload consumed at boot.

Deliberate simplifications

Single-shot RECV re-armed per request (multishot recv + buffer rings are a phase-F improvement), one outstanding SQE per connection, fixed-size buffers, naive "title" extraction instead of a JSON parser, no timerfd. Requires kernel ≥ 5.19 (multishot accept). This file is for reading; crates/rt is for running writeonce.

The wovm bytecode VM (runtime/src/)

Milestone 1 of the OOP track (spec: docs/superpowers/specs/2026-08-01-oop-compiler-vm-design.md, plan 1: docs/superpowers/plans/2026-08-01-wob-format-and-vm-core.md) — a register VM that executes .wob bytecode (format: docs/plan/oop-vm/00-wob-format.md) with the full milestone-1 memory model. C11, libc only, same doctrine as wo-rt.c. wovm doesn't produce .wob itself — that's the OCaml woc front end's job (compiler/README.md, plan 3); this directory is the VM, not the compiler.

Shipped features:

• Register interpreter — fixed 32-bit instructions, Lua-style window-overlap calls (callee r0 = caller slot A), dual dispatch: computed goto under GNU C, switch under -DWO_ISO_C (both flavors gated in CI so neither rots).
• Owned objects with a runtime borrow word — shared-reader count / exclusive sentinel in every 16-byte header; violations trap T_BORROW. The compiler elides provable sites; the VM enforces the residual ones (hybrid model, spec §4).
• @gc reference counting + budgeted cycle collection — rc at zero frees immediately; possible cycles buffer as candidates (Bacon–Rajan trial deletion), collected in budgeted epochs per shard — no stop-the-world by construction. The wovm CLI pumps the collector to quiescence after the entry method returns (WO_GC_BUDGET steps per call, default 64; WO_GC_TRACE=1 prints one stderr line per step) — scheduler-paced stepping between requests is sub-project 2, not this milestone.
• Deterministic drops — kind-directed drop plans (scalar/owned/gcref/text/multi/map), recursive over class fields and container elements.
• Trap unwinding that never leaks — per-method drop tables (pc → owned/gc register masks); a trap walks every frame and frees what was live; structured error {code, method, line, message} via line tables.
• Validating loader — bounds-checked parse, aligned copies, const-string interning, full static validation (opcodes, registers, indexes, jump targets, terminators, builtin arity, call windows, sorted vtables); what the loader accepts, the interpreter trusts — no UB on any input.
• Structural interfaces — ICALL binary-searches sorted (class, slot, method) vtable triples by receiver class.
• Native containers + builtins — multi/map with element-kind tags; now/print/print_int/words/multi_*/map_*; DB_STUB traps "engine not linked" until the DB engine binds (plan 5).
• CLI contract — wovm app.wob: exit 0 = ran; exit 1 = trap, one stderr line trap CODE in METHOD at line N: MESSAGE; exit 2 = usage/load failure. WO_HEAP_MB overrides the 64 MiB arena. Run with no .wob argument, wovm also checks its own trailer for an appended image (woc build's single-binary output, docs/plan/oop-vm/00-wob-format.md's "single-binary trailer" section) — a recognized-but-corrupt trailer fails clearly on exit 2, never a crash.

File map:

File	What it is
src/wob.h	the .wob v1 contract: opcodes, field kinds, trap codes, builtin ids, limits, the 16-byte object header (machine-readable twin of the format doc)
src/obj.h/.c	arena allocator (16-byte size-class free lists ≤ 1024 B, malloc above), wo_rt context, object creation, strings
src/borrow.h/.c	the borrow word: acquire shared/exclusive, unconditional releases
src/cont.h/.c	native containers: growable multi, linear-scan map (content-compared text keys)
src/gc.h/.c	kind-directed drop dispatcher, rc inc/dec, cycle-candidate buffer, budgeted Bacon–Rajan collector
src/loader.h/.c	.wob parse + full static validation + mmap file path
src/vm.h/.c	the interpreter: dispatch, frames, traps, drop-map unwinding, ICALL
src/builtin.h/.c	builtin dispatcher
src/main.c	the wovm CLI: arg parsing, self-embedded-trailer detection, the post-exit gc pump
test/t.h	20-line assert harness (no framework)
test/wob_build.h/.c	in-memory .wob assembler — the second, independent encoding of the format; builder/loader disagreements fail tests
test/test_*.c	13 suites, one binary each, ASan+UBSan
test/mkwob.c	fixture generator for the CLI smoke
test/cli_smoke.sh	end-to-end exit-code/stderr-shape check

Gates: just wovm-build · just wovm-test (unit suites + ISO flavor + CLI smoke, all ASan-clean) · in runtime/: make test, make test-iso, make wovm, make wovm-asan (sanitized binary for corpus fixtures that need a leak/UB proof, e.g. tests/corpus/gc/). Across both halves: just oop-e2e (the woc + wovm conformance corpus, compiler/README.md) and just oop-accept (milestone 1's full acceptance gate — spec success criteria + both unit suites, one command).

Debugging

VS Code: .vscode/launch.json ships four configs (needs the C/C++ extension, ms-vscode.cpptools):

1. wovm: debug hello.wob — rebuilds wovm at -O0 -g, regenerates fixtures, breaks anywhere in the VM.
2. wovm: debug a .wob file — same, prompts for the image path.
3. wovm: debug unit test — pick one of the 13 ASan test binaries (already -g), step through it.
4. wo-rt: debug server — the event-loop reference at -O0 -g, WO_THREADS=1 so one shard owns everything.

How to work on the VM under a debugger:

• Step the ISO flavor, not the computed-goto one. The goto interpreter jumps label-to-label and single-stepping is disorienting. Test binaries have an ISO twin (build/iso_test_*, plain switch) where next/step behave normally. For wovm itself, build make -B wovm CFLAGS='-O0 -g -std=c11 -DWO_ISO_C'.
• break vm_trap — one breakpoint catches every trap at the moment of failure, with the trapping frame intact (vm->frames[vm->depth-1], pc already rewound to the faulting instruction). vm_unwind is the next frame down if you're chasing a leak-on-trap.
• Other load-bearing breakpoints: wo_load_buf (validation rejects), recv_check (residual field checks), wo_builtin (all builtins), wo_gc_step (cycle collection epochs).
• ASan under gdb: ASAN_OPTIONS=abort_on_error=1 makes the first report SIGABRT so the debugger stops on it with the full stack; without gdb the report alone usually names the exact free you missed.
• CLI knobs: WO_HEAP_MB=1 ./wovm app.wob forces early T_OOM paths; exit codes 0/1/2 are stable for scripting.
• gdb without VS Code: gdb --args ./wovm build/hello.wob, or gdb ./build/iso_test_unwind.