writeonce/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

What this repo is

writeonce is a declarative full-stack programming language. You write .wo files; the runtime compiles them into a single binary that owns the database, serves REST, and (Stage 3+) pushes live subscriptions. Think: Go + Postgres + net/http + Phoenix LiveView folded into one language.

The end goal — zero external deps, kernel primitives only

The runtime's north star — documented in docs/01-problem.md, docs/02-recovery.md, and docs/plan/linux/00-linux.md — is one binary, no external Rust crates, all I/O driven directly by Linux kernel primitives. epoll (or io_uring), inotify, eventfd, timerfd, signalfd, sendfile, mmap — the kernel IS the subscription engine, the async runtime, and the file watcher.

Target end state of crates/rt/Cargo.toml:

[dependencies]
libc = "0.2"   # the unavoidable FFI bridge to syscalls

Stage 2 today carries six transitional deps (anyhow, serde, serde_json, tokio, axum, tower). docs/plan/02 through docs/plan/08 sequence the removal of each one, replaced by hand-rolled modules ported from the v1 crates that already did exactly this (reference/crates/wo-event, wo-http, wo-route, wo-serve, wo-watch). When working on the runtime, default to direct-syscall solutions over reaching for new crates — the docs/plan/ docs name the port source for every module.

Layout

The repo holds three cuts of the same project plus one research reference:

1. crates/rt/ — the active Rust runtime (Stage 2 shipped). Monolithic on purpose for now: lexer, parser, AST, in-memory engine, axum REST server all in one crate. The wo binary lives at crates/rt/src/bin/wo.rs.
2. crates/{ql,value,engine,txn,db,wal,sub,http,gen,policy,logic,service,ui,app}/ — 14 empty sibling crates scaffolded to match the 7-phase design. Each has a Cargo.toml + src/lib.rs with just a doc comment pointing at its phase spec. Real code moves in from rt as each phase activates; do NOT refactor rt to use these today — it would break Stage 2.
3. reference/crates/ — the v1 writeonce blog (13 crates: wo-seg, wo-index, wo-store, wo-htmlx, etc.). This is a nested Cargo workspace, deliberately excluded from the root workspace. The v1 crates keep their wo- prefix; the new runtime crates dropped theirs. cd reference/crates && cargo build builds v1 standalone. See docs/runtime/database/07-wo-seg-migration.md for the plan replacing v1 with the new runtime.
4. reference/linux/ and reference/go/ — symlinks to the Linux kernel source tree and the Go source tree respectively. Not committed (see .gitignore). Research resources:
   • reference/linux/ — grep io_uring/, fs/notify/inotify/, kernel/eventfd.c, include/uapi/linux/*.h when designing the kernel-primitive modules. Paired with per-primitive reference cards at docs/plan/linux/.
   • reference/go/ — grep src/runtime/netpoll_epoll.go, netpoll.go, os_linux*.go, asm_*.s, sys_linux_*.s when designing the runtime layer. The crates/rt/src/runtime/ module mirrors Go's src/runtime/ file-per-flavour naming (netpoll_epoll.rs ↔ netpoll_epoll.go). The docs/plan/assembly/ docs cite this tree.
   • Each contributor sets their own targets via ln -s <path-to-src> reference/{linux,go}.

There is also prototypes/wo-db/ — a ~2k-line C++ prototype of the query-layer engine (SQL + Cypher + document paths, RETURNING aliases, LIVE stub). It keeps its wo-db directory name (C++ project, separate from the Rust crate db). It's the reference implementation the Rust port follows; make test still passes.

And prototypes/wo-rt-c/ — a single-file C reference of the runtime layer (edge-triggered epoll loop, signalfd shutdown, non-blocking listener, in-RAM store over minimal HTTP; libc only). It's the runtime-layer sibling of wo-db: each block maps one-to-one to a crates/rt/src/runtime/ module (table in its README). make builds it; just rt-c-demo exercises it. Its evolution into a multi-threaded io_uring RAM-database runtime (thread-per-core, mmap arena, WAL dual-write, recovery, ACID) is phased A–F in docs/plan/exploration/c-runtime/00-plan.md, with the one-address architecture trace beside it (01-architecture.md) — the proving ground for plans 09–12. Documentation belongs under docs/ — prototype/crate directories keep only their orientation README.

Commands

# Build + test the runtime
cargo build                                              # compiles all 15 crates
cargo test --lib                                         # 14 unit tests (all in rt today)
cargo test --lib parses_inline_struct -- --nocapture     # single named test

# Run the runtime against a sample project
cargo run --bin wo -- run docs/examples/blog       # :8080 — blog sample
cargo run --bin wo -- run docs/examples/ecommerce  # :8080 — ecommerce sample
WO_LISTEN=127.0.0.1:9000 cargo run --bin wo -- run docs/examples/blog   # override port

# v1 blog codebase (nested workspace — must cd first)
cd reference/crates && cargo build && cargo test

# C++ prototype of the query-layer engine
cd prototypes/wo-db && make test     # smoke.wo + checkout.wo
cd prototypes/wo-db && make run      # interactive REPL

# Manual HTTP smoke against a running `wo run ...`
# Open reference/rest/blog.rest or ecommerce.rest in VS Code (with REST Client)
# or JetBrains (built-in HTTP client). Or run curl per reference/rest/README.md.

Architecture — what requires reading multiple files to understand

Naming convention

• New runtime crates are unprefixed. ql, value, engine, txn, db, wal, sub, http, gen, policy, logic, service, ui, app, rt. Internal imports read cleanly: use ql::Parser, use db::Tx, use http::router.
• V1 crates keep the wo- prefix. wo-seg, wo-index, wo-store, wo-htmlx, wo-md, and v1's own wo-rt/wo-http/wo-sub. These live in reference/crates/.
• The C++ prototype directory is prototypes/wo-db/ — unchanged, not a Rust crate.
• The binary is wo — defined in crates/rt/Cargo.toml [[bin]]. Independent of the crate name.

The two-layer .wo language

Covered in docs/runtime/database/02-wo-language.md:

• Schema layer — unified type Name { ... } DSL (fields, embedded structs, ref, multi @edge, multi via, backlink, tagged unions, policy, on <event>, service). This is what developers write day-to-day.
• Query layer — hybrid SQL + Cypher with five "fixed-glue" rules that make the three grammars share semantics: $name parameters everywhere, cross-paradigm RETURNING col AS alias visible to later statements in the same BEGIN … COMMIT, dotted paths identical in SQL/doc/Cypher, one transaction block syntax, one LIVE prefix on subscriptions.

Both layers are .wo files. The schema layer compiles down to query-layer operations — but only when Phase 5 codegen and Phase 6 full-stack blocks need a single authoritative input. Stage 2 ships with the schema layer only.

Concurrency: thread-per-core event loops (09a shipped)

Since plans 09a+09b, wo run boots WO_THREADS pinned worker threads (default = online cores; wo-shard-<t> in ps -T), each running its own epoll EventLoop with its own SO_REUSEPORT listener (crates/rt/src/runtime/scheduler.rs) and its own Engine — there is no Arc<Mutex<Engine>> anywhere. Ids interleave per shard (Engine::for_shard; owner = (id-1) % n); cross-shard operations travel the shard bus (crates/rt/src/shard.rs: mpsc job mailboxes + mail eventfds; creates local, point ops hop once, lists fan out and merge). Deadlock-freedom: jobs never block, waiters pump their own inbox. Routers are thread-local (HandlerFn is not Send/Sync). Signals are blocked before spawn; worker 0 owns the signalfd and broadcasts shutdown via per-worker eventfds. Don't reintroduce shared mutable engine state — the doctrine below is now enforced by ownership.

Single-threaded event loop (original doctrine)

Covered in docs/runtime/database/02-wo-language.md § Concurrency Model and 03-inmemory-engine.md. The runtime is Redis/TigerBeetle-style: one userland thread owns everything — connection accept, parser, engine, subscription registry. The only non-userland thread is the kernel-owned io_uring SQPOLL helper. This is pinned architecturally — group commit still applies (loop drains many commits into one fsync SQE per tick), and scaling past one core is done by sharding independent engine processes, not by adding worker threads. Keep this in mind before proposing Arc<Mutex<...>> anything beyond what's already there.

What's in rt today vs. what the empty crates promise

rt's modules deliberately mirror the future crate names so the extraction is mechanical when each phase activates:

rt module	Will move to	Phase
token.rs + lexer.rs + ast.rs + parser.rs	ql	2
engine.rs (Value + Row helpers)	value	2
engine.rs (Engine + Catalog)	engine	2
compile.rs	engine	2
method.rs (13b method executor)	logic	6
server.rs	http + service	4 / 6
bin/wo.rs	stays in rt (the binary)	—

The sub, wal, txn, policy, logic, ui, app, gen crates have no rt counterpart yet — they land when their phase activates.

Sample projects drive the grammar

docs/examples/blog/ and docs/examples/ecommerce/ are both docs artifacts and the de facto integration tests. The parser survives these because specific features in them (nested {...} object literals inside trigger actions, count(...) / words(...) computed defaults, unions like Pending | Paid | Shipped) forced real fixes. When changing the parser, run the full end-to-end against both samples, not just cargo test.

The ecommerce sample in particular uses features that are deliberately parse-and-discard in Stage 2: fn checkout(...) in txn snapshot, type-attached on update triggers with multi-line do actions, policy read for role .... These are part of the .wo language but Stage 2 does not execute them.

The migration story

docs/runtime/database/07-wo-seg-migration.md specifies phased coexistence: abstract the v1 article store behind an ArticleStore trait, stand up the .wo engine as a second implementation, dual-write, cut over, decommission v1. Phase A (trait abstraction) hasn't started — the plan is on paper, the v1 code is still monolithic in reference/crates/wo-store/. Do not remove anything from reference/crates/ without checking the migration doc.

Stage progress

Stage	Status
1 — wo run <dir> discovers .wo files	✅ shipped
2 — parser + engine + REST CRUD	✅ shipped (cargo run -- run docs/examples/blog)
3 — LIVE subscriptions over WebSocket	pending — /api/<type>/live returns 501 as a stub
4+ — transactional fn, policies, triggers, ##ui, WAL, codegen	design-only (see docs/runtime/database.md)

Stage-3 stubs (501) and policy-shaped 405/404 responses are intentional and documented in reference/rest/*.rest. Don't "fix" them without checking those files first.

Non-obvious gotchas

• rt is monolithic on purpose. Splitting it into the 14 sibling crates is Phase-by-Phase work, not a Stage-2 refactor.
• reference/crates/ is its own workspace. Running cargo build at the root does not build v1. Running it in reference/crates/ does.
• Parser identifiers vs. keywords. subscribe, receive, expect_abort, me, self, and lowercase insert/select are NOT keywords in the lexer — they stay as plain idents (only SQL-layer INSERT/SELECT are keywords). The 13b statement parser matches insert as an ident; the select expression is recognised by the two-token shape select <Ident> {. Adding these to the keyword map breaks service rest "..." expose subscribe and method bodies.
• Type-level annotations. @table(name: "...", index: [a, b]) before a type/class configures storage (it never toggles table-ness — every type IS a table). Unknown keys inside @table(...) are parse errors; unknown annotation names (@foo) skip silently. Engine secondary indexes are maintained ONLY via Engine::row_insert/row_remove — never touch tables directly or indexes drift.
• Parser skip-on-block. Unknown triggers (on update do ...) are parsed-and-discarded by brace-depth-aware skipping. Object literals like { article_id: self.id } inside trigger actions contain } that must not be mistaken for the type's outer close brace — the depth counter exists specifically because of this. Exception since 13b: fn inside a class parses into a real MethodDecl (body statements + expressions, executed by method.rs); fn inside a plain type still skips.
• Newline significance. The lexer emits Kind::Newline tokens and the parser uses them to end policy/trigger lines. Do not filter newlines globally.
• Default-value parsing. = now() is recognised explicitly as DefaultExpr::Now; anything else falls into an opaque-expression path that engine::eval_default then omits from created rows (computed fields display as empty, not as debug-printed tokens).
• Binary variable shadowing. crates/rt/src/bin/wo.rs has let rt = ... (a tokio runtime handle) inside run() that shadows the crate named rt. Inside run() the variable wins; inside serve() (a different function) rt:: refers to the crate. Don't rename the variable without also auditing the crate-path references.

Where to read next

• docs/runtime/wo-language.md — user-facing language overview
• docs/runtime/database.md — 7-phase engineering series index
• docs/plan/linux/00-linux.md — catalogue of kernel primitives the runtime leans on
• docs/plan/02-event-loop-epoll.md through 08-sendfile-static-assets.md — the dependency-removal phase sequence
• docs/plan/done/01-scafolding-crates.md — the completed crate-scaffolding phase
• docs/examples/blog/README.md — the canonical worked example
• prototypes/wo-db/README.md — the C++ prototype that shows the query layer
• reference/rest/README.md — how to exercise the running prototype
• reference/README.md — what's in the v1 archive and why it's preserved
• reference/linux/ (symlink) — the Linux kernel source tree; grep io_uring/, fs/notify/inotify/, include/uapi/linux/*.h when designing kernel-facing modules
• reference/go/ (symlink) — the Go source tree; grep src/runtime/netpoll_*.go, asm_*.s, sys_linux_*.s when porting runtime-layer ideas (writeonce's crates/rt/src/runtime/ mirrors this naming)
• docs/plan/assembly/00-overview.md — role of assembly in a runtime; writeonce policy is "no custom asm, use Rust stdlib"
• crates/README.md — inventory of all 15 crates with phase assignments