Compare commits

...

7 commits

Author SHA1 Message Date
4bcd8bb074 chore: justfile, Cargo.toml, vscode config, crates README, cm.md 2026-08-10 14:11:49 +02:00
80046556af docs: update story iterations (pre-compiler-front)
- 00: story framing
- 01: principles doc
- 02: VM core
- 04: single binary e2e
- 05: language surface
- 08: shard-actor runtime
- 09: database engine
- 10: HTTP service
2026-08-10 14:11:43 +02:00
9baf930c7c docs: update exploration docs (assembly, c-runtime, linux, postgresql, ui) 2026-08-10 14:11:36 +02:00
02714c296f docs: update plan done logs (Phases 1-4)
- 01: crate scaffolding
- 02: event loop epoll
- 03: hand-rolled HTTP
- 04: tokio/axum cutover
2026-08-10 14:11:32 +02:00
138b393bd1 docs: update plan docs (Phases 5-16)
- 05: hand-rolled JSON
- 06: bespoke error type
- 07: inotify content watcher
- 08: sendfile static assets
- 09: concurrency scaleout
- 10: storage foundations
- 11: WAL and recovery
- 12: engine disk cutover
- 14: MVC UI implementation
- 15: MCP streamable HTTP
- 16: Postgres mirror
2026-08-10 14:11:24 +02:00
1d1a637608 docs: update ecommerce/pricing examples and wo-seg migration doc 2026-08-10 14:10:38 +02:00
6b8b41ce06 refactor(crates/rt): Stage 2 runtime improvements
- http: connection, listener, request, response, route updates
- pg: protocol improvements
- runtime: eventfd, netpoll (epoll/io_uring), scheduler, signalfd, timerfd
- shard: cross-shard job improvements
- wal: replay and recovery fixes
2026-08-10 14:10:21 +02:00
80 changed files with 847 additions and 343 deletions

View file

@ -3,6 +3,7 @@
"recommendations": [ "recommendations": [
"bierner.markdown-mermaid", // renders ```mermaid blocks in the markdown preview "bierner.markdown-mermaid", // renders ```mermaid blocks in the markdown preview
"rust-lang.rust-analyzer", "rust-lang.rust-analyzer",
"humao.rest-client" // reference/rest/*.rest files "humao.rest-client", // .dev/reference/rest/*.rest files
"ms-vscode.cpptools" // C debugging (launch.json cppdbg configs, runtime/)
] ]
} }

View file

@ -5,7 +5,7 @@
# docs/plan/done/01-scafolding-crates.md. They are commented out of the # docs/plan/done/01-scafolding-crates.md. They are commented out of the
# workspace until their phase activates — uncomment each one as code lands. # workspace until their phase activates — uncomment each one as code lands.
# #
# The v1 writeonce blog crates at `reference/crates/` are a separate nested # The v1 writeonce blog crates at `.dev/reference/crates/` are a separate nested
# workspace, excluded here so the root build stays focused on the new runtime. # workspace, excluded here so the root build stays focused on the new runtime.
[workspace] [workspace]
@ -32,7 +32,7 @@ members = [
# "crates/app", # phase 06 target — ##app manifest # "crates/app", # phase 06 target — ##app manifest
] ]
exclude = [ exclude = [
"reference/crates", ".dev/reference/crates",
] ]
[workspace.dependencies] [workspace.dependencies]

View file

@ -1,27 +1,27 @@
# `crates/` — the `.wo` runtime # `crates/` — the `.wo` runtime
Fifteen crates make up the new runtime. Only `rt/` carries real code today (Stage 2); the other fourteen are **empty placeholders** scaffolded to match the 7-phase design so each phase's extraction work becomes a mechanical code move into an existing home. Fifteen crates make up the new runtime. Only `rt/` carries real code today (Stage 2); the other fourteen are **planned** per the 7-phase design — scaffolds were deleted 2026-08-08; recreate each crate when its phase activates, so extraction from `rt` stays a mechanical move.
> The crate-name prefix `wo-` was dropped when the active project namespaced itself under `wo` (the binary, the file extension, the language). Internal imports read cleanly: `use ql::Parser`, `use db::Tx`, `use http::router`. The v1 codebase keeps its `wo-*` prefix in [`reference/crates/`](../reference/crates/) to distinguish the generations. > The crate-name prefix `wo-` was dropped when the active project namespaced itself under `wo` (the binary, the file extension, the language). Internal imports read cleanly: `use ql::Parser`, `use db::Tx`, `use http::router`. The v1 codebase keeps its `wo-*` prefix in [`.dev/reference/crates/`](../.dev/reference/crates/) to distinguish the generations.
## Map ## Map
| Phase | Crate | Purpose | Status | | Phase | Crate | Purpose | Status |
| --- | --- | --- | --- | | --- | --- | --- | --- |
| 2 | [`ql`](./ql/) | `.wo` grammar — lexer, parser, AST | placeholder | | 2 | `ql` | `.wo` grammar — lexer, parser, AST | planned |
| 2 | [`value`](./value/) | tagged `Value` + dotted-path helpers | placeholder | | 2 | `value` | tagged `Value` + dotted-path helpers | planned |
| 2 | [`engine`](./engine/) | in-memory executor (rel / doc / graph) + schema catalog | placeholder | | 2 | `engine` | in-memory executor (rel / doc / graph) + schema catalog | planned |
| 2 | [`txn`](./txn/) | transaction coordinator — MVCC, `RETURNING` alias table | placeholder | | 2 | `txn` | transaction coordinator — MVCC, `RETURNING` alias table | planned |
| 2 | [`db`](./db/) | top-level facade — `open()`, `Tx`, `Query`, `Subscribe` | placeholder | | 2 | `db` | top-level facade — `open()`, `Tx`, `Query`, `Subscribe` | planned |
| 3 | [`wal`](./wal/) | write-ahead log — io_uring + fsync + recovery | placeholder | | 3 | `wal` | write-ahead log — io_uring + fsync + recovery | planned |
| 4 | [`sub`](./sub/) | live subscriptions — delta frames on commit | placeholder | | 4 | `sub` | live subscriptions — delta frames on commit | planned |
| 4 | [`http`](./http/) | wire protocol — REST / GraphQL-over-WS / native codec | placeholder | | 4 | `http` | wire protocol — REST / GraphQL-over-WS / native codec | planned |
| 5 | [`gen`](./gen/) | codegen — `.wo type` → Go / TS / Rust / Python clients | placeholder | | 5 | `gen` | codegen — `.wo type` → Go / TS / Rust / Python clients | planned |
| 6 | [`policy`](./policy/) | RBAC + row-level rules compiled into planner rewrites | placeholder | | 6 | `policy` | RBAC + row-level rules compiled into planner rewrites | planned |
| 6 | [`logic`](./logic/) | `on <event>` triggers + `fn ... in txn` interpreter | placeholder | | 6 | `logic` | `on <event>` triggers + `fn ... in txn` interpreter | planned |
| 6 | [`service`](./service/) | `service rest/graphql/native` endpoint dispatch | placeholder | | 6 | `service` | `service rest/graphql/native` endpoint dispatch | planned |
| 6 | [`ui`](./ui/) | `##ui` screens → SSR HTML + client runtime | placeholder | | 6 | `ui` | `##ui` screens → SSR HTML + client runtime | planned |
| 6 | [`app`](./app/) | `##app` route manifest + startup hooks | placeholder | | 6 | `app` | `##app` route manifest + startup hooks | planned |
| — | [`rt`](./rt/) | **active** — Stage-2 monolith + the `wo` binary | **shipped** | | — | [`rt`](./rt/) | **active** — Stage-2 monolith + the `wo` binary | **shipped** |
## Why `rt/` is monolithic right now ## Why `rt/` is monolithic right now
@ -49,6 +49,6 @@ cargo run --bin wo -- run docs/examples/blog # serve the blog sample
## What's outside this directory ## What's outside this directory
- [`../reference/crates/`](../reference/crates/) — the v1 writeonce blog (13 crates, nested workspace). Preserved for reference per [docs/runtime/database/07-wo-seg-migration.md](../docs/runtime/database/07-wo-seg-migration.md). Keeps its `wo-*` prefix. - [`../.dev/reference/crates/`](../.dev/reference/crates/) — the v1 writeonce blog (13 crates, nested workspace). Preserved for reference per [docs/runtime/database/07-wo-seg-migration.md](../docs/runtime/database/07-wo-seg-migration.md). Keeps its `wo-*` prefix.
- [`../prototypes/wo-db/`](../prototypes/wo-db/) — C++ prototype of the query-layer engine (~2k lines). The reference implementation this Rust port follows at the language level. - [`../prototypes/wo-db/`](../prototypes/wo-db/) — C++ prototype of the query-layer engine (~2k lines). The reference implementation this Rust port follows at the language level.
- [`../docs/plan/`](../docs/plan/) — planning documents for in-flight work (the `.md` files directly under `plan/` are upcoming phases; `plan/done/` holds completed ones). [`plan/done/01-scafolding-crates.md`](../docs/plan/done/01-scafolding-crates.md) is the authoritative scope doc for the 14 new placeholders. - [`../docs/plan/`](../docs/plan/) — planning documents for in-flight work (the `.md` files directly under `plan/` are upcoming phases; `plan/done/` holds completed ones). [`plan/done/01-scafolding-crates.md`](../docs/plan/done/01-scafolding-crates.md) is the authoritative scope doc for the 14 new placeholders.

View file

@ -10,7 +10,7 @@
//! any buffered pipelined request); `Connection: close` goes to Done. //! any buffered pipelined request); `Connection: close` goes to Done.
//! Done → loop closes the fd. //! Done → loop closes the fd.
//! //!
//! Adapted from `reference/crates/wo-http/src/connection.rs`. The owning //! Adapted from `.dev/reference/crates/wo-http/src/connection.rs`. The owning
//! [`EventLoop`] supplies `read`/`write` readiness via edge-triggered //! [`EventLoop`] supplies `read`/`write` readiness via edge-triggered
//! `epoll`; this struct is the per-fd part of the state. //! `epoll`; this struct is the per-fd part of the state.
//! //!

View file

@ -1,6 +1,6 @@
//! Non-blocking TCP listener — `socket(2)` + `bind(2)` + `listen(2)` + `accept4(2)`. //! Non-blocking TCP listener — `socket(2)` + `bind(2)` + `listen(2)` + `accept4(2)`.
//! //!
//! Adapted from `reference/crates/wo-http/src/listener.rs`. The v1 hand-rolled //! Adapted from `.dev/reference/crates/wo-http/src/listener.rs`. The v1 hand-rolled
//! IPv4 parser had a byte-order bug for non-localhost addresses; here we //! IPv4 parser had a byte-order bug for non-localhost addresses; here we
//! defer to `std::net::SocketAddr` (stdlib, no extra crate) and convert the //! defer to `std::net::SocketAddr` (stdlib, no extra crate) and convert the
//! resulting octets to a `sockaddr_in` correctly. //! resulting octets to a `sockaddr_in` correctly.

View file

@ -1,6 +1,6 @@
//! Incremental HTTP/1.1 request parser. //! Incremental HTTP/1.1 request parser.
//! //!
//! Adapted from `reference/crates/wo-http/src/request.rs`. v1 only parsed //! Adapted from `.dev/reference/crates/wo-http/src/request.rs`. v1 only parsed
//! request headers (the v1 blog is read-only HTML). The phase-04 cutover //! request headers (the v1 blog is read-only HTML). The phase-04 cutover
//! needs JSON request bodies, so this parser also drains a //! needs JSON request bodies, so this parser also drains a
//! `Content-Length`-delimited body. Chunked transfer encoding is not //! `Content-Length`-delimited body. Chunked transfer encoding is not

View file

@ -1,6 +1,6 @@
//! HTTP/1.1 response builder + serializer. //! HTTP/1.1 response builder + serializer.
//! //!
//! Adapted from `reference/crates/wo-http/src/response.rs`. Adds: //! Adapted from `.dev/reference/crates/wo-http/src/response.rs`. Adds:
//! * `Status` constants for the codes the REST samples assert on //! * `Status` constants for the codes the REST samples assert on
//! (200/201/204/400/404/405/500/501). //! (200/201/204/400/404/405/500/501).
//! * `Response::json(&serde_json::Value)` matching the cutover-handler //! * `Response::json(&serde_json::Value)` matching the cutover-handler

View file

@ -1,6 +1,6 @@
//! Method + URL pattern → handler dispatch. //! Method + URL pattern → handler dispatch.
//! //!
//! Combined adaptation of `reference/crates/wo-route/src/{router,pattern}.rs`. //! Combined adaptation of `.dev/reference/crates/wo-route/src/{router,pattern}.rs`.
//! Handler shape is `Fn(&Request, &RouteParams) -> Response`, captured as a //! Handler shape is `Fn(&Request, &RouteParams) -> Response`, captured as a
//! boxed closure so each route closes over its own state (typically an //! boxed closure so each route closes over its own state (typically an
//! `Arc<Mutex<Engine>>` — see `crates/rt/src/server.rs`). //! `Arc<Mutex<Engine>>` — see `crates/rt/src/server.rs`).

View file

@ -14,7 +14,7 @@
//! * literal/identifier escaping for SQL the mirror generates //! * literal/identifier escaping for SQL the mirror generates
//! //!
//! Protocol reference: PostgreSQL docs “Frontend/Backend Protocol” and //! Protocol reference: PostgreSQL docs “Frontend/Backend Protocol” and
//! `reference/postgresql/src/include/libpq/` (research symlink). //! `.dev/reference/postgresql/src/include/libpq/` (research symlink).
//! //!
//! Blocking I/O is deliberate: the only caller is the dedicated `wo-pg` //! Blocking I/O is deliberate: the only caller is the dedicated `wo-pg`
//! mirror thread (plan 16b) — never a shard worker. //! mirror thread (plan 16b) — never a shard worker.

View file

@ -4,7 +4,7 @@
//! event loop to come back and run something writes a `1` to the eventfd, //! event loop to come back and run something writes a `1` to the eventfd,
//! which becomes readable on the loop's next `wait_once`. //! which becomes readable on the loop's next `wait_once`.
//! //!
//! Ported from `reference/crates/wo-event/src/eventfd.rs` with an added //! Ported from `.dev/reference/crates/wo-event/src/eventfd.rs` with an added
//! `AsRawFd` impl so callers can drop the fd straight into `EventLoop`. //! `AsRawFd` impl so callers can drop the fd straight into `EventLoop`.
use std::io; use std::io;

View file

@ -1,6 +1,6 @@
//! `epoll`-backed event loop. Single-threaded, edge-triggered. //! `epoll`-backed event loop. Single-threaded, edge-triggered.
//! //!
//! Ported from `reference/crates/wo-event/src/epoll.rs`. Differences: //! Ported from `.dev/reference/crates/wo-event/src/epoll.rs`. Differences:
//! * `Token` is a newtype rather than a `u64` alias. //! * `Token` is a newtype rather than a `u64` alias.
//! * `Interest` is a struct exposing `READABLE`, `WRITABLE`, `READ_WRITE` //! * `Interest` is a struct exposing `READABLE`, `WRITABLE`, `READ_WRITE`
//! constants, matching the API in `docs/plan/02-event-loop-epoll.md`. //! constants, matching the API in `docs/plan/02-event-loop-epoll.md`.

View file

@ -1,5 +1,5 @@
//! Raw io_uring — no liburing, kernel ABI structs defined by hand, exactly //! Raw io_uring — no liburing, kernel ABI structs defined by hand, exactly
//! the sequence proven in C (`prototypes/wo-rt-c/wo-rt.c` ring_init/enter; //! the sequence proven in C (`runtime/wo-rt.c` ring_init/enter;
//! card: `docs/plan/exploration/linux/07-io_uring.md`). //! card: `docs/plan/exploration/linux/07-io_uring.md`).
//! //!
//! Scope (this phase): the **storage ring** for per-shard group commit — //! Scope (this phase): the **storage ring** for per-shard group commit —

View file

@ -10,7 +10,7 @@
//! Per plan 09a, **engine state stays globally shared** (`Arc<Mutex<Engine>>` //! Per plan 09a, **engine state stays globally shared** (`Arc<Mutex<Engine>>`
//! inside the per-thread `Router`s) — one thing at a time; the sharded engine //! inside the per-thread `Router`s) — one thing at a time; the sharded engine
//! is 09b. The C proving ground for this exact sequence is //! is 09b. The C proving ground for this exact sequence is
//! `prototypes/wo-rt-c` phase A (see `docs/plan/exploration/c-runtime/`). //! `runtime` phase A (see `docs/plan/exploration/c-runtime/`).
//! //!
//! Shutdown: signals are blocked in `main` before any worker spawns (the //! Shutdown: signals are blocked in `main` before any worker spawns (the
//! mask is inherited), so only worker 0 — which owns the `signalfd` — ever //! mask is inherited), so only worker 0 — which owns the `signalfd` — ever

View file

@ -5,7 +5,7 @@
//! //!
//! Blocks the captured signals in the calling thread's mask, so the //! Blocks the captured signals in the calling thread's mask, so the
//! kernel routes them to the signalfd instead of running default handlers. //! kernel routes them to the signalfd instead of running default handlers.
//! Ported from `reference/crates/wo-event/src/signalfd.rs`. //! Ported from `.dev/reference/crates/wo-event/src/signalfd.rs`.
use std::io; use std::io;
use std::os::unix::io::{AsRawFd, RawFd}; use std::os::unix::io::{AsRawFd, RawFd};

View file

@ -4,7 +4,7 @@
//! fd becomes readable when the timer expires; reading drains the //! fd becomes readable when the timer expires; reading drains the
//! expiration count. //! expiration count.
//! //!
//! Ported from `reference/crates/wo-event/src/timerfd.rs`. Adds `oneshot` //! Ported from `.dev/reference/crates/wo-event/src/timerfd.rs`. Adds `oneshot`
//! and `periodic` constructors that match the API in the phase-02 plan. //! and `periodic` constructors that match the API in the phase-02 plan.
use std::io; use std::io;

View file

@ -16,7 +16,7 @@
//! coordination. Creates are always local (the receiving shard mints from //! coordination. Creates are always local (the receiving shard mints from
//! its own stride); reads/updates/deletes hop at most once; lists fan out //! its own stride); reads/updates/deletes hop at most once; lists fan out
//! to every shard and merge. The C proving ground for the wake mechanism is //! to every shard and merge. The C proving ground for the wake mechanism is
//! `prototypes/wo-rt-c` (eventfd broadcast); the mailbox-per-thread design //! `runtime` (eventfd broadcast); the mailbox-per-thread design
//! is plan 09 decision 2 and 09d's one-message-per-thread fan-out shape. //! is plan 09 decision 2 and 09d's one-message-per-thread fan-out shape.
use std::cell::RefCell; use std::cell::RefCell;

View file

@ -1,6 +1,6 @@
//! Per-shard write-ahead log — plan 09c (`docs/plan/09-concurrency-scaleout.md`) //! Per-shard write-ahead log — plan 09c (`docs/plan/09-concurrency-scaleout.md`)
//! + the durability core of plan 11, ported from the proven C sequence //! + the durability core of plan 11, ported from the proven C sequence
//! (`prototypes/wo-rt-c` phases D/E, `docs/plan/exploration/c-runtime/00-plan.md`). //! (`runtime` phases D/E, `docs/plan/exploration/c-runtime/00-plan.md`).
//! //!
//! One `shard-<t>.rwal` per worker. Frame format (identical shape to the C //! One `shard-<t>.rwal` per worker. Frame format (identical shape to the C
//! prototype): `u32 len | u32 crc32(payload) | payload | u32 COMMIT` — a //! prototype): `u32 len | u32 crc32(payload) | payload | u32 COMMIT` — a

View file

@ -7,11 +7,11 @@ scaffold sibling crates, multi-app ecommerce, REST + concurrency docs
and apps/storefront, with shared/ types/logic/components, per-app and apps/storefront, with shared/ types/logic/components, per-app
app.wo + wo.toml, and reusable .htmlx components (layout, money, app.wo + wo.toml, and reusable .htmlx components (layout, money,
order-row) order-row)
- add reference/rest/{blog,ecommerce}.rest — VS Code/JetBrains HTTP - add .dev/reference/rest/{blog,ecommerce}.rest — VS Code/JetBrains HTTP
request files driving the running prototype, including 501/404/405 request files driving the running prototype, including 501/404/405
expectations for stubbed endpoints expectations for stubbed endpoints
- add docs/plan/09-concurrency-scaleout.md and docs/plan/ui/00-overview.md; - add docs/plan/09-concurrency-scaleout.md and docs/plan/ui/00-overview.md;
refine docs/plan/assembly/02-writeonce-stance.md refine docs/plan/assembly/02-writeonce-stance.md
- refresh templates (about, article, header/footer, home, layout, styles) - refresh templates (about, article, header/footer, home, layout, styles)
and add static favicon/logo and add static favicon/logo
- add infra/sync.sh and tighten .gitignore for reference/ symlinks - add infra/sync.sh and tighten .gitignore for .dev/reference/ symlinks

View file

@ -112,5 +112,5 @@ The [`blog` sample](../blog/) is still a single-app layout (`types/`, `ui/`, `lo
- **Master plan:** [`../../plan/ui/00-overview.md`](../../plan/ui/00-overview.md) - **Master plan:** [`../../plan/ui/00-overview.md`](../../plan/ui/00-overview.md)
- **Language spec the `##ui`/`##app`/`policy` blocks obey:** [`../../runtime/database/06-lowcode-fullstack.md`](../../runtime/database/06-lowcode-fullstack.md) - **Language spec the `##ui`/`##app`/`policy` blocks obey:** [`../../runtime/database/06-lowcode-fullstack.md`](../../runtime/database/06-lowcode-fullstack.md)
- **Wire protocol the app binaries speak to the DB daemon:** [`../../runtime/database/04-client-api.md`](../../runtime/database/04-client-api.md) - **Wire protocol the app binaries speak to the DB daemon:** [`../../runtime/database/04-client-api.md`](../../runtime/database/04-client-api.md)
- **v1 template engine that `.htmlx` compilation will reuse:** [`../../../reference/crates/wo-htmlx/`](../../../reference/crates/wo-htmlx/) - **v1 template engine that `.htmlx` compilation will reuse:** [`../../../.dev/reference/crates/wo-htmlx/`](../../../.dev/reference/crates/wo-htmlx/)
- **Checkout transaction that's the canonical cross-paradigm test:** [`shared/logic/checkout.wo`](shared/logic/checkout.wo) - **Checkout transaction that's the canonical cross-paradigm test:** [`shared/logic/checkout.wo`](shared/logic/checkout.wo)

View file

@ -10,7 +10,7 @@ A `Price` class and a `Product` class with methods — products have prices —
## Layout ## Layout
The UI follows **MVC** ([`docs/plan/exploration/ui/08-mvc-structure.md`](../../plan/exploration/ui/08-mvc-structure.md)), with the same screen anatomy as the v1 Angular app (`reference/writeonce-app/src/app/article/`) collapsed into the single binary: the **model** is the class itself, the **view** is plain `.htmlx` with external `.scss`, and the **controller** is a `.wo` file that binds the model into the view and is the only place UI may call class methods. The UI follows **MVC** ([`docs/plan/exploration/ui/08-mvc-structure.md`](../../plan/exploration/ui/08-mvc-structure.md)), with the same screen anatomy as the v1 Angular app (`.dev/reference/writeonce-app/src/app/article/`) collapsed into the single binary: the **model** is the class itself, the **view** is plain `.htmlx` with external `.scss`, and the **controller** is a `.wo` file that binds the model into the view and is the only place UI may call class methods.
``` ```
pricing/ pricing/

View file

@ -91,7 +91,7 @@ Covers 95% of current `serde_json::json!(...)` uses in the codebase. For the oth
- `emit_stable_key_order` — emitting a `BTreeMap`-backed object produces keys in sorted order (matters for `.rest` expected-body stability). - `emit_stable_key_order` — emitting a `BTreeMap`-backed object produces keys in sorted order (matters for `.rest` expected-body stability).
- `parse_errors` — unterminated string, trailing comma, missing comma, unclosed object all return `ParseError` with line/col. - `parse_errors` — unterminated string, trailing comma, missing comma, unclosed object all return `ParseError` with line/col.
3. **All 14 existing `rt` tests pass** after the swap (the `engine::Engine` and `server::*` tests most affected). 3. **All 14 existing `rt` tests pass** after the swap (the `engine::Engine` and `server::*` tests most affected).
4. **`reference/rest/blog.rest`** — 20 assertions all return the same HTTP status AND the same response body shape (may differ in key ordering if `BTreeMap` ordering differs from `serde_json`'s insertion order — document the shift). 4. **`.dev/reference/rest/blog.rest`** — 20 assertions all return the same HTTP status AND the same response body shape (may differ in key ordering if `BTreeMap` ordering differs from `serde_json`'s insertion order — document the shift).
5. **Dep audit.** `cargo tree -p rt --depth 1` shows zero `serde*` lines. 5. **Dep audit.** `cargo tree -p rt --depth 1` shows zero `serde*` lines.
## Non-scope ## Non-scope
@ -108,7 +108,7 @@ cargo build # three deps
cargo test --lib json # new parser/emitter tests cargo test --lib json # new parser/emitter tests
cargo test --lib # 14 existing tests still green cargo test --lib # 14 existing tests still green
# full .rest smoke — same script as phase 04 exit criterion 3 # full .rest smoke — same script as phase 04 exit criterion 3
cd reference/crates && cargo build && cargo test cd .dev/reference/crates && cargo build && cargo test
``` ```
## After this phase ## After this phase

View file

@ -94,8 +94,8 @@ Find: `rg 'anyhow::|anyhow!|bail!|\.context\(' crates/rt/src | wc -l` — expect
1. **`cargo build`** — compiles with exactly one external dep. 1. **`cargo build`** — compiles with exactly one external dep.
2. **`cargo test --lib`** — all 14 existing `rt` tests pass. A new test in `error.rs` exercises `From<io::Error>`, `with_context`, and `Display` formatting. 2. **`cargo test --lib`** — all 14 existing `rt` tests pass. A new test in `error.rs` exercises `From<io::Error>`, `with_context`, and `Display` formatting.
3. **No `anyhow::` references anywhere in the repo.** `rg 'anyhow' crates/ docs/` returns zero hits (docs updated by this phase too). 3. **No `anyhow::` references anywhere in the repo.** `rg 'anyhow' crates/ docs/` returns zero hits (docs updated by this phase too).
4. **`reference/rest/blog.rest`** — 20 assertions still pass. Error paths (404, 400) still produce the same response body format (plain-text error message from the handler's `.to_string()`). 4. **`.dev/reference/rest/blog.rest`** — 20 assertions still pass. Error paths (404, 400) still produce the same response body format (plain-text error message from the handler's `.to_string()`).
5. **`cd reference/crates && cargo build && cargo test`** unchanged. V1 doesn't use `anyhow` — nothing to touch there. 5. **`cd .dev/reference/crates && cargo build && cargo test`** unchanged. V1 doesn't use `anyhow` — nothing to touch there.
6. **`cargo tree -p rt --depth 1`** lists only `libc` as an external dep (plus transitive ones brought in by libc itself, all of which are kernel-facing). 6. **`cargo tree -p rt --depth 1`** lists only `libc` as an external dep (plus transitive ones brought in by libc itself, all of which are kernel-facing).
## Non-scope ## Non-scope
@ -111,7 +111,7 @@ cargo build # one external dep
cargo test --lib # 14 + error.rs test green cargo test --lib # 14 + error.rs test green
rg 'anyhow' crates/ docs/ # zero hits rg 'anyhow' crates/ docs/ # zero hits
# full .rest smoke — same script as phase 04 # full .rest smoke — same script as phase 04
cd reference/crates && cargo build && cargo test # v1 untouched cd .dev/reference/crates && cargo build && cargo test # v1 untouched
cat crates/rt/Cargo.toml | grep -A 20 '\[dependencies\]' # libc is the only line cat crates/rt/Cargo.toml | grep -A 20 '\[dependencies\]' # libc is the only line
``` ```

View file

@ -26,7 +26,7 @@ Also the first real second consumer of the phase-02 `EventLoop` beyond the HTTP
| File | Responsibility | Port source | | File | Responsibility | Port source |
| --- | --- | --- | | --- | --- | --- |
| `mod.rs` | Re-exports `Watcher`, `WatchEvent` | — | | `mod.rs` | Re-exports `Watcher`, `WatchEvent` | — |
| `inotify.rs` | Raw wrappers: `init()`, `add_watch(path, mask)`, `read_events() -> Vec<RawEvent>`. Registers on the `EventLoop`. | [`reference/crates/wo-watch/src/lib.rs`](../../reference/crates/wo-watch/src/lib.rs) (280 LOC) — v1 already does exactly this | | `inotify.rs` | Raw wrappers: `init()`, `add_watch(path, mask)`, `read_events() -> Vec<RawEvent>`. Registers on the `EventLoop`. | [`.dev/reference/crates/wo-watch/src/lib.rs`](../../.dev/reference/crates/wo-watch/src/lib.rs) (280 LOC) — v1 already does exactly this |
| `recursive.rs` | Walks the project root, calls `add_watch` for every directory matching `types/\|ui/\|logic/\|tests/` or containing `*.wo` | ~80 new LOC | | `recursive.rs` | Walks the project root, calls `add_watch` for every directory matching `types/\|ui/\|logic/\|tests/` or containing `*.wo` | ~80 new LOC |
| `debounce.rs` | Coalesces bursts per-watch-descriptor, fires a `TimerFd` for the 150 ms settle window | ~100 new LOC | | `debounce.rs` | Coalesces bursts per-watch-descriptor, fires a `TimerFd` for the 150 ms settle window | ~100 new LOC |
| `reload.rs` | On debounced fire: re-discover, re-parse, re-compile, `ArcSwap::store(new_catalog)` | ~80 new LOC | | `reload.rs` | On debounced fire: re-discover, re-parse, re-compile, `ArcSwap::store(new_catalog)` | ~80 new LOC |
@ -88,7 +88,7 @@ for event in loop_.wait_once(None)? {
# observe: `curl :8080/api/articles` response shape reflects new field (no restart) # observe: `curl :8080/api/articles` response shape reflects new field (no restart)
``` ```
4. **`[wo]` log lines** match the spec in [00-linux.md](./linux/00-linux.md) — one line per debounced change, showing the relative path and event kind. 4. **`[wo]` log lines** match the spec in [00-linux.md](./linux/00-linux.md) — one line per debounced change, showing the relative path and event kind.
5. **All 14 `rt` unit tests still pass.** `reference/rest/blog.rest` 20-assertion battery still green. 5. **All 14 `rt` unit tests still pass.** `.dev/reference/rest/blog.rest` 20-assertion battery still green.
6. **No fd leak** — `ls -la /proc/$PID/fd` before and after ten consecutive edits shows the same count. 6. **No fd leak** — `ls -la /proc/$PID/fd` before and after ten consecutive edits shows the same count.
## Non-scope ## Non-scope
@ -116,7 +116,7 @@ curl -s http://127.0.0.1:8080/api/articles # server did not restart; catalog
kill $PID kill $PID
git checkout docs/examples/blog/types/article.wo # undo the edit git checkout docs/examples/blog/types/article.wo # undo the edit
cd reference/crates && cargo build && cargo test # v1 untouched cd .dev/reference/crates && cargo build && cargo test # v1 untouched
``` ```
## After this phase ## After this phase

View file

@ -24,9 +24,9 @@ Serve static file bytes — eventually `##ui`-emitted HTML + CSS + JS bundle, to
| File | Responsibility | Port source | | File | Responsibility | Port source |
| --- | --- | --- | | --- | --- | --- |
| `mod.rs` | Re-exports `StaticHandler`, `resolve` | — | | `mod.rs` | Re-exports `StaticHandler`, `resolve` | — |
| `sendfile.rs` | Raw `sendfile(2)` wrapper + non-blocking `send_all` that co-operates with `EPOLLOUT` | [`reference/crates/wo-serve/src/sendfile.rs`](../../reference/crates/wo-serve/src/sendfile.rs) (109 LOC) | | `sendfile.rs` | Raw `sendfile(2)` wrapper + non-blocking `send_all` that co-operates with `EPOLLOUT` | [`.dev/reference/crates/wo-serve/src/sendfile.rs`](../../.dev/reference/crates/wo-serve/src/sendfile.rs) (109 LOC) |
| `resolve.rs` | Path canonicalisation + traversal defence + file existence check | [`reference/crates/wo-serve/src/resolve.rs`](../../reference/crates/wo-serve/src/resolve.rs) (80 LOC) | | `resolve.rs` | Path canonicalisation + traversal defence + file existence check | [`.dev/reference/crates/wo-serve/src/resolve.rs`](../../.dev/reference/crates/wo-serve/src/resolve.rs) (80 LOC) |
| `mime.rs` | Extension → `Content-Type` table | [`reference/crates/wo-serve/src/mime.rs`](../../reference/crates/wo-serve/src/mime.rs) (44 LOC) | | `mime.rs` | Extension → `Content-Type` table | [`.dev/reference/crates/wo-serve/src/mime.rs`](../../.dev/reference/crates/wo-serve/src/mime.rs) (44 LOC) |
| `handler.rs` | `StaticHandler` — integrates the three with phase-03's `Response` builder; returns 404 / 403 / 200 as appropriate | ~120 new LOC | | `handler.rs` | `StaticHandler` — integrates the three with phase-03's `Response` builder; returns 404 / 403 / 200 as appropriate | ~120 new LOC |
Total: ~350 LOC (233 ported + ~120 new). Total: ~350 LOC (233 ported + ~120 new).
@ -74,7 +74,7 @@ The `Response` returned by `handler.serve()` owns the open `File` fd. The phase-
``` ```
4. **Path traversal attempts fail closed.** `curl :8080/static/../Cargo.toml` returns 403. `curl :8080/static/nonexistent.png` returns 404. 4. **Path traversal attempts fail closed.** `curl :8080/static/../Cargo.toml` returns 403. `curl :8080/static/nonexistent.png` returns 404.
5. **`EAGAIN` handling.** A test that rate-limits the socket sendbuf to force a partial write exercises the `EPOLLOUT` re-arm path; the full payload still arrives. 5. **`EAGAIN` handling.** A test that rate-limits the socket sendbuf to force a partial write exercises the `EPOLLOUT` re-arm path; the full payload still arrives.
6. **All 14 `rt` tests** + phase-02/03/04/05/06/07 additions pass. `reference/rest/blog.rest` 20 assertions still green (no regressions on the JSON endpoints). 6. **All 14 `rt` tests** + phase-02/03/04/05/06/07 additions pass. `.dev/reference/rest/blog.rest` 20 assertions still green (no regressions on the JSON endpoints).
## Non-scope ## Non-scope
@ -96,9 +96,9 @@ cargo run --bin wo -- run docs/examples/blog &
# ... (full script from exit criterion 3) # ... (full script from exit criterion 3)
# .rest smoke unchanged # .rest smoke unchanged
# full 20-assertion battery against reference/rest/blog.rest # full 20-assertion battery against .dev/reference/rest/blog.rest
cd reference/crates && cargo build && cargo test # v1 untouched cd .dev/reference/crates && cargo build && cargo test # v1 untouched
``` ```
## After this phase ## After this phase

View file

@ -2,7 +2,7 @@
> **Kanban: 🔄 in progress** — 09a/09b/09c ✅ shipped (+ keep-alive and io_uring group-commit follow-ups, measured in the shipped notes below); 09d/09e/09f ⬜ not started. Board: [00-kanban.md](00-kanban.md) > **Kanban: 🔄 in progress** — 09a/09b/09c ✅ shipped (+ keep-alive and io_uring group-commit follow-ups, measured in the shipped notes below); 09d/09e/09f ⬜ not started. Board: [00-kanban.md](00-kanban.md)
**Context sources:** [`./08-sendfile-static-assets.md`](./08-sendfile-static-assets.md) (last single-threaded phase), [`./assembly/02-writeonce-stance.md`](./assembly/02-writeonce-stance.md) (the "single-threaded" policy we're now refining), [`../runtime/database/02-wo-language.md#concurrency-model`](../runtime/database/02-wo-language.md#concurrency-model) (original concurrency stance), [`docs/examples/ecommerce/`](../examples/ecommerce/) (the target workload), [`./linux/`](./linux/) (kernel primitives), [`reference/go/src/runtime/`](../../reference/go/src/runtime/) (precedent for a runtime that scales across threads). **Context sources:** [`./08-sendfile-static-assets.md`](./08-sendfile-static-assets.md) (last single-threaded phase), [`./assembly/02-writeonce-stance.md`](./assembly/02-writeonce-stance.md) (the "single-threaded" policy we're now refining), [`../runtime/database/02-wo-language.md#concurrency-model`](../runtime/database/02-wo-language.md#concurrency-model) (original concurrency stance), [`docs/examples/ecommerce/`](../examples/ecommerce/) (the target workload), [`./linux/`](./linux/) (kernel primitives), [`.dev/reference/go/src/runtime/`](../../.dev/reference/go/src/runtime/) (precedent for a runtime that scales across threads).
## Context ## Context
@ -28,7 +28,7 @@ Serve the ecommerce sample at 10,000 concurrent websocket subscribers + 1,000 ch
## What we copy from Go, what we don't ## What we copy from Go, what we don't
Read [`reference/go/src/runtime/netpoll_epoll.go`](../../reference/go/src/runtime/netpoll_epoll.go) and [`reference/go/src/runtime/proc.go`](../../reference/go/src/runtime/proc.go) for the shape; copy the **ideas** about fd-to-loop mapping and atomic-counter-based wake-up. Do **not** copy: Read [`.dev/reference/go/src/runtime/netpoll_epoll.go`](../../.dev/reference/go/src/runtime/netpoll_epoll.go) and [`.dev/reference/go/src/runtime/proc.go`](../../.dev/reference/go/src/runtime/proc.go) for the shape; copy the **ideas** about fd-to-loop mapping and atomic-counter-based wake-up. Do **not** copy:
| Go feature | Why writeonce skips it | | Go feature | Why writeonce skips it |
| --- | --- | | --- | --- |
@ -53,10 +53,10 @@ Reference cards already exist for most; this phase adds the ones that are cross-
| Primitive | Use | Reference | | Primitive | Use | Reference |
| --- | --- | --- | | --- | --- | --- |
| `SO_REUSEPORT` | N listener sockets on the same port; kernel load-balances accepts | [`reference/linux/net/core/sock_reuseport.c`](../../reference/linux/net/core/sock_reuseport.c) — worth adding `linux/12-so-reuseport.md` | | `SO_REUSEPORT` | N listener sockets on the same port; kernel load-balances accepts | [`.dev/reference/linux/net/core/sock_reuseport.c`](../../.dev/reference/linux/net/core/sock_reuseport.c) — worth adding `linux/12-so-reuseport.md` |
| `sched_setaffinity` + `cpu_set_t` | Pin each thread to its core | [`reference/linux/kernel/sched/core.c`](../../reference/linux/kernel/sched/core.c) | | `sched_setaffinity` + `cpu_set_t` | Pin each thread to its core | [`.dev/reference/linux/kernel/sched/core.c`](../../.dev/reference/linux/kernel/sched/core.c) |
| `futex(2)` | Fallback cross-thread wait if per-thread eventfd wake-up isn't enough | [`reference/linux/kernel/futex/`](../../reference/linux/kernel/futex/) — worth `linux/13-futex.md` | | `futex(2)` | Fallback cross-thread wait if per-thread eventfd wake-up isn't enough | [`.dev/reference/linux/kernel/futex/`](../../.dev/reference/linux/kernel/futex/) — worth `linux/13-futex.md` |
| `membarrier(2)` | Process-wide memory barrier when a rebalance migrates state between threads | [`reference/linux/kernel/sched/membarrier.c`](../../reference/linux/kernel/sched/membarrier.c) | | `membarrier(2)` | Process-wide memory barrier when a rebalance migrates state between threads | [`.dev/reference/linux/kernel/sched/membarrier.c`](../../.dev/reference/linux/kernel/sched/membarrier.c) |
| `io_uring` with `IORING_SETUP_SINGLE_ISSUER` | One ring per thread, pinned | [`./linux/07-io_uring.md`](./linux/07-io_uring.md) | | `io_uring` with `IORING_SETUP_SINGLE_ISSUER` | One ring per thread, pinned | [`./linux/07-io_uring.md`](./linux/07-io_uring.md) |
| `eventfd` per thread | Cross-thread wake-up — thread A writes to thread B's eventfd to deliver a message | [`./linux/02-eventfd.md`](./linux/02-eventfd.md) | | `eventfd` per thread | Cross-thread wake-up — thread A writes to thread B's eventfd to deliver a message | [`./linux/02-eventfd.md`](./linux/02-eventfd.md) |
| `mmap(MAP_HUGETLB)` | Per-thread arena allocator backed by 2 MB pages for cache locality | [`./linux/08-mmap.md`](./linux/08-mmap.md) | | `mmap(MAP_HUGETLB)` | Per-thread arena allocator backed by 2 MB pages for cache locality | [`./linux/08-mmap.md`](./linux/08-mmap.md) |
@ -132,7 +132,7 @@ If the "single core per process, shard across processes" argument ([Redis Cluste
- [`./08-sendfile-static-assets.md`](./08-sendfile-static-assets.md) — last prerequisite phase; feature-complete single-threaded runtime. - [`./08-sendfile-static-assets.md`](./08-sendfile-static-assets.md) — last prerequisite phase; feature-complete single-threaded runtime.
- [`./assembly/02-writeonce-stance.md`](./assembly/02-writeonce-stance.md) — updated to reference this phase's thread-per-core model; still no asm. - [`./assembly/02-writeonce-stance.md`](./assembly/02-writeonce-stance.md) — updated to reference this phase's thread-per-core model; still no asm.
- [`../runtime/database/02-wo-language.md#concurrency-model`](../runtime/database/02-wo-language.md#concurrency-model) — the stance this plan refines. - [`../runtime/database/02-wo-language.md#concurrency-model`](../runtime/database/02-wo-language.md#concurrency-model) — the stance this plan refines.
- [`reference/go/src/runtime/proc.go`](../../reference/go/src/runtime/proc.go) — Go's scheduler, for contrast. - [`.dev/reference/go/src/runtime/proc.go`](../../.dev/reference/go/src/runtime/proc.go) — Go's scheduler, for contrast.
- [`reference/go/src/runtime/netpoll_epoll.go`](../../reference/go/src/runtime/netpoll_epoll.go) — per-P netpoller, the idea we borrow. - [`.dev/reference/go/src/runtime/netpoll_epoll.go`](../../.dev/reference/go/src/runtime/netpoll_epoll.go) — per-P netpoller, the idea we borrow.
- [`reference/linux/net/core/sock_reuseport.c`](../../reference/linux/net/core/sock_reuseport.c) — kernel load balancer. - [`.dev/reference/linux/net/core/sock_reuseport.c`](../../.dev/reference/linux/net/core/sock_reuseport.c) — kernel load balancer.
- [`reference/linux/kernel/sched/core.c`](../../reference/linux/kernel/sched/core.c) — affinity syscalls. - [`.dev/reference/linux/kernel/sched/core.c`](../../.dev/reference/linux/kernel/sched/core.c) — affinity syscalls.

View file

@ -2,7 +2,7 @@
> **Kanban: ⬜ not started (scope reduced)** — WAL framing/fallocate/CRC landed early via plan 09c; the `@table(name:, index:)` storage-config surface and in-RAM secondary indexes (`Engine::find_by`) landed via the plan-13 follow-up (spec: [`02-wo-language.md § Type-Level Annotations`](../runtime/database/02-wo-language.md)) — this plan inherits the surface and gives indexes their on-disk form. Board: [00-kanban.md](00-kanban.md) > **Kanban: ⬜ not started (scope reduced)** — WAL framing/fallocate/CRC landed early via plan 09c; the `@table(name:, index:)` storage-config surface and in-RAM secondary indexes (`Engine::find_by`) landed via the plan-13 follow-up (spec: [`02-wo-language.md § Type-Level Annotations`](../runtime/database/02-wo-language.md)) — this plan inherits the surface and gives indexes their on-disk form. Board: [00-kanban.md](00-kanban.md)
**Context sources:** [`./done/04-cutover-remove-tokio-axum.md`](./done/04-cutover-remove-tokio-axum.md), [`../runtime/database/03-inmemory-engine.md`](../runtime/database/03-inmemory-engine.md), [`../runtime/database/07-wo-seg-migration.md`](../runtime/database/07-wo-seg-migration.md), [`./exploration/postgresql/smgr-and-md.md`](./exploration/postgresql/smgr-and-md.md), [`./exploration/postgresql/page-format.md`](./exploration/postgresql/page-format.md), [`./exploration/linux/12-pwrite-fsync.md`](./exploration/linux/12-pwrite-fsync.md), [`./exploration/linux/09-fallocate.md`](./exploration/linux/09-fallocate.md), [`reference/crates/wo-seg/src/`](../../reference/crates/wo-seg/src/). **Context sources:** [`./done/04-cutover-remove-tokio-axum.md`](./done/04-cutover-remove-tokio-axum.md), [`../runtime/database/03-inmemory-engine.md`](../runtime/database/03-inmemory-engine.md), [`../runtime/database/07-wo-seg-migration.md`](../runtime/database/07-wo-seg-migration.md), [`./exploration/postgresql/smgr-and-md.md`](./exploration/postgresql/smgr-and-md.md), [`./exploration/postgresql/page-format.md`](./exploration/postgresql/page-format.md), [`./exploration/linux/12-pwrite-fsync.md`](./exploration/linux/12-pwrite-fsync.md), [`./exploration/linux/09-fallocate.md`](./exploration/linux/09-fallocate.md), [`.dev/reference/crates/wo-seg/src/`](../../.dev/reference/crates/wo-seg/src/).
## Goal ## Goal
@ -32,7 +32,7 @@ Lays the codec + filesystem layout that phase 11 (WAL + recovery) and phase 12 (
| `codec.rs` | `trait RowCodec { fn encode(&self, row: &Row, buf: &mut Vec<u8>); fn decode(&self, bytes: &[u8]) -> Result<Row>; }` + `JsonCodec` impl backed by today's `serde_json`. | ~50 | | `codec.rs` | `trait RowCodec { fn encode(&self, row: &Row, buf: &mut Vec<u8>); fn decode(&self, bytes: &[u8]) -> Result<Row>; }` + `JsonCodec` impl backed by today's `serde_json`. | ~50 |
| `seg.rs` | `SegStore { dir: PathBuf, fds: HashMap<String, RawFd>, tails: HashMap<String, u64> }`. `open(dir)`, `append(ty, &Row) -> Result<u64-offset>`, `read(ty, offset) -> Result<Row>` (used by phase 11 recovery, not by the engine yet). | ~250 | | `seg.rs` | `SegStore { dir: PathBuf, fds: HashMap<String, RawFd>, tails: HashMap<String, u64> }`. `open(dir)`, `append(ty, &Row) -> Result<u64-offset>`, `read(ty, offset) -> Result<Row>` (used by phase 11 recovery, not by the engine yet). | ~250 |
Total: ~560 LOC. The framing math + fallocate + pwrite plumbing is ported from [`reference/crates/wo-seg/src/{writer.rs,reader.rs,header.rs}`](../../reference/crates/wo-seg/src/) with the CRC trailer added. Total: ~560 LOC. The framing math + fallocate + pwrite plumbing is ported from [`.dev/reference/crates/wo-seg/src/{writer.rs,reader.rs,header.rs}`](../../.dev/reference/crates/wo-seg/src/) with the CRC trailer added.
### File layout written under `<wo_run_dir>/` ### File layout written under `<wo_run_dir>/`
@ -102,7 +102,7 @@ The root workspace member list also activates: `crates/db` joins `crates/rt` as
- `seg_append_writes_to_disk` — `append` then re-`open` reads the same row back. - `seg_append_writes_to_disk` — `append` then re-`open` reads the same row back.
- `seg_grows_when_full` — appending past the initial 1 MiB triggers a fallocate-grow without losing existing records. - `seg_grows_when_full` — appending past the initial 1 MiB triggers a fallocate-grow without losing existing records.
3. **End-to-end** — `cargo run --bin wo -- run docs/examples/blog`, `curl -X POST /api/articles` with a body, then `xxd docs/examples/blog/data/Article.seg | head -3` — output shows the magic length prefix and the JSON payload. 3. **End-to-end** — `cargo run --bin wo -- run docs/examples/blog`, `curl -X POST /api/articles` with a body, then `xxd docs/examples/blog/data/Article.seg | head -3` — output shows the magic length prefix and the JSON payload.
4. **`reference/rest/blog.rest`** — 20-assertion battery still passes byte-identically. 4. **`.dev/reference/rest/blog.rest`** — 20-assertion battery still passes byte-identically.
5. **Restart leaves the segment on disk but ignores it.** `wo run`, write 5 rows, ctrl-C, `wo run` again, `GET /api/articles` returns `[]`. The segment file still exists. Phase 11 will start replaying it. 5. **Restart leaves the segment on disk but ignores it.** `wo run`, write 5 rows, ctrl-C, `wo run` again, `GET /api/articles` returns `[]`. The segment file still exists. Phase 11 will start replaying it.
## Non-scope ## Non-scope
@ -140,7 +140,7 @@ sleep 1
curl -s http://127.0.0.1:8080/api/articles # expect [] curl -s http://127.0.0.1:8080/api/articles # expect []
kill -INT $PID kill -INT $PID
cd reference/crates && cargo build && cargo test # v1 untouched cd .dev/reference/crates && cargo build && cargo test # v1 untouched
``` ```
## After this phase ## After this phase

View file

@ -203,7 +203,7 @@ kill -INT $PID
strace -e fdatasync,fsync,rename -f -p $(pgrep -f 'target/debug/wo run') 2>&1 | head -20 strace -e fdatasync,fsync,rename -f -p $(pgrep -f 'target/debug/wo run') 2>&1 | head -20
# v1 untouched # v1 untouched
cd reference/crates && cargo build && cargo test cd .dev/reference/crates && cargo build && cargo test
``` ```
## After this phase ## After this phase

View file

@ -165,7 +165,7 @@ curl -s 'http://127.0.0.1:8080/api/articles' | python3 -c 'import json,sys;print
# expect: 10000 # expect: 10000
kill -INT $PID kill -INT $PID
cd reference/crates && cargo build && cargo test # v1 untouched cd .dev/reference/crates && cargo build && cargo test # v1 untouched
``` ```
## After this phase ## After this phase

View file

@ -2,7 +2,7 @@
> **Kanban: ⏸ parked (frontend)** — backend focus first; design stays current. Board: [00-kanban.md](00-kanban.md) > **Kanban: ⏸ parked (frontend)** — backend focus first; design stays current. Board: [00-kanban.md](00-kanban.md)
**Context sources:** [`./exploration/ui/08-mvc-structure.md`](./exploration/ui/08-mvc-structure.md) (the design this plan implements), [`./exploration/ui/01-htmlx-format-spec.md`](./exploration/ui/01-htmlx-format-spec.md) / [`02-ui-compiler.md`](./exploration/ui/02-ui-compiler.md) / [`03-client-runtime.md`](./exploration/ui/03-client-runtime.md) (the three UI-track pieces this plan sequences, each with port sources and LOC budgets), [`./13-class-model-live-pricing.md`](./13-class-model-live-pricing.md) (the class methods controllers call: 13a/13b; the LIVE deltas views consume: 13c), [`../examples/pricing/ui/pricing/`](../examples/pricing/ui/pricing/) (the reference MVC triplet), [`reference/crates/wo-htmlx/`](../../reference/crates/wo-htmlx/) (the v1 template engine, primary port source). **Context sources:** [`./exploration/ui/08-mvc-structure.md`](./exploration/ui/08-mvc-structure.md) (the design this plan implements), [`./exploration/ui/01-htmlx-format-spec.md`](./exploration/ui/01-htmlx-format-spec.md) / [`02-ui-compiler.md`](./exploration/ui/02-ui-compiler.md) / [`03-client-runtime.md`](./exploration/ui/03-client-runtime.md) (the three UI-track pieces this plan sequences, each with port sources and LOC budgets), [`./13-class-model-live-pricing.md`](./13-class-model-live-pricing.md) (the class methods controllers call: 13a/13b; the LIVE deltas views consume: 13c), [`../examples/pricing/ui/pricing/`](../examples/pricing/ui/pricing/) (the reference MVC triplet), [`.dev/reference/crates/wo-htmlx/`](../../.dev/reference/crates/wo-htmlx/) (the v1 template engine, primary port source).
## Context ## Context
@ -29,7 +29,7 @@ Phases 05/06 (hand-rolled JSON / bespoke error) are orthogonal: `crates/ui` adop
### `14a-htmlx-engine.md` — port the view engine into `crates/ui` ### `14a-htmlx-engine.md` — port the view engine into `crates/ui`
Execute [`exploration/ui/01-htmlx-format-spec.md`](./exploration/ui/01-htmlx-format-spec.md) as written: port `reference/crates/wo-htmlx` (585 LOC — `parser.rs`, `ast.rs`, `value.rs`, `registry.rs`, `render.rs` carried over per its table) into `crates/ui/src/htmlx/`, extend with `<wo:live>` structured nodes, `wo:bind` capture, and the `data-wo-manifest` JSON emitter (~250 LOC new). One addition beyond the 01 spec, from the MVC design: `<wo:live source="…">` records whether `source` is a bare name (controller model binding, resolved in 14d) or an inline query — a one-field change to `LiveSubscription`. Execute [`exploration/ui/01-htmlx-format-spec.md`](./exploration/ui/01-htmlx-format-spec.md) as written: port `.dev/reference/crates/wo-htmlx` (585 LOC — `parser.rs`, `ast.rs`, `value.rs`, `registry.rs`, `render.rs` carried over per its table) into `crates/ui/src/htmlx/`, extend with `<wo:live>` structured nodes, `wo:bind` capture, and the `data-wo-manifest` JSON emitter (~250 LOC new). One addition beyond the 01 spec, from the MVC design: `<wo:live source="…">` records whether `source` is a bare name (controller model binding, resolved in 14d) or an inline query — a one-field change to `LiveSubscription`.
**Exit:** the 01 spec's criteria — `cargo build -p ui` green, golden parse+render for every `.htmlx` under `docs/examples/{blog,ecommerce}` **plus** [`pricing/ui/pricing/pricing.htmlx`](../examples/pricing/ui/pricing/pricing.htmlx), manifest matches the 01 schema. **Exit:** the 01 spec's criteria — `cargo build -p ui` green, golden parse+render for every `.htmlx` under `docs/examples/{blog,ecommerce}` **plus** [`pricing/ui/pricing/pricing.htmlx`](../examples/pricing/ui/pricing/pricing.htmlx), manifest matches the 01 schema.
### `14b-scss-subset.md` — the stylesheet compiler ### `14b-scss-subset.md` — the stylesheet compiler
@ -86,5 +86,5 @@ Execute [`exploration/ui/03-client-runtime.md`](./exploration/ui/03-client-runti
- [`./exploration/ui/08-mvc-structure.md`](./exploration/ui/08-mvc-structure.md) — the design; its exit criteria are satisfied by 14c/14b/14f respectively. - [`./exploration/ui/08-mvc-structure.md`](./exploration/ui/08-mvc-structure.md) — the design; its exit criteria are satisfied by 14c/14b/14f respectively.
- [`./13-class-model-live-pricing.md`](./13-class-model-live-pricing.md) — 13a/13b gate 14e; 13c gates 14f; 13d's exit criterion is this plan's end-to-end target. - [`./13-class-model-live-pricing.md`](./13-class-model-live-pricing.md) — 13a/13b gate 14e; 13c gates 14f; 13d's exit criterion is this plan's end-to-end target.
- [`./exploration/ui/00-overview.md`](./exploration/ui/00-overview.md) — the UI track's master frame (per-app binaries, shared DB daemon) that 14d's asset/serving choices stay compatible with. - [`./exploration/ui/00-overview.md`](./exploration/ui/00-overview.md) — the UI track's master frame (per-app binaries, shared DB daemon) that 14d's asset/serving choices stay compatible with.
- [`reference/crates/wo-htmlx/`](../../reference/crates/wo-htmlx/) — primary port source (585 LOC), per ui/01. - [`.dev/reference/crates/wo-htmlx/`](../../.dev/reference/crates/wo-htmlx/) — primary port source (585 LOC), per ui/01.
- [`../examples/pricing/ui/pricing/`](../examples/pricing/ui/pricing/) — the reference triplet every sub-phase tests against. - [`../examples/pricing/ui/pricing/`](../examples/pricing/ui/pricing/) — the reference triplet every sub-phase tests against.

View file

@ -2,7 +2,7 @@
> **Kanban: ⬜ not started (Track 4 — Language & API)** — board: [00-kanban.md](00-kanban.md) > **Kanban: ⬜ not started (Track 4 — Language & API)** — board: [00-kanban.md](00-kanban.md)
**Context sources:** [MCP specification 2025-06-18 — Transports](https://modelcontextprotocol.io/specification/2025-06-18/basic/transports) (the normative Streamable HTTP contract this plan implements, verified 2026-07-12), [`reference/mcp-python-sdk/`](../../reference/README.md) (symlink to the official MCP Python SDK — grep `src/mcp/server/streamable_http.py` + `streamable_http_manager.py` for the reference server behaviour, `src/mcp/client/streamable_http.py` for what a conforming client expects; behaviour is ported, code is not), [`../runtime/database/04-client-api.md`](../runtime/database/04-client-api.md) (the wire-protocol design; its "REST + SSE gateway" row is what this plan makes concrete for agents), [`./13-class-model-live-pricing.md`](./13-class-model-live-pricing.md) (13b methods become MCP tools; 13c's subscription registry carries 15e), [`./09-concurrency-scaleout.md`](./09-concurrency-scaleout.md) (thread-per-core + shard bus the endpoint rides; 09d fan-out gates 15e), `crates/rt/src/server.rs` + `crates/rt/src/http/` (the keep-alive HTTP layer and router this lands in). **Context sources:** [MCP specification 2025-06-18 — Transports](https://modelcontextprotocol.io/specification/2025-06-18/basic/transports) (the normative Streamable HTTP contract this plan implements, verified 2026-07-12), [`.dev/reference/mcp-python-sdk/`](../../.dev/reference/README.md) (symlink to the official MCP Python SDK — grep `src/mcp/server/streamable_http.py` + `streamable_http_manager.py` for the reference server behaviour, `src/mcp/client/streamable_http.py` for what a conforming client expects; behaviour is ported, code is not), [`../runtime/database/04-client-api.md`](../runtime/database/04-client-api.md) (the wire-protocol design; its "REST + SSE gateway" row is what this plan makes concrete for agents), [`./13-class-model-live-pricing.md`](./13-class-model-live-pricing.md) (13b methods become MCP tools; 13c's subscription registry carries 15e), [`./09-concurrency-scaleout.md`](./09-concurrency-scaleout.md) (thread-per-core + shard bus the endpoint rides; 09d fan-out gates 15e), `crates/rt/src/server.rs` + `crates/rt/src/http/` (the keep-alive HTTP layer and router this lands in).
## Context ## Context
@ -62,7 +62,7 @@ The rules the sub-phases implement, condensed from the spec — each MUST below
- **Tool generation**: per exposed type×op → `<type>_list`, `<type>_get`, `<type>_create`, `<type>_update`, `<type>_delete`, with `inputSchema` (JSON Schema) derived from catalog field types (unions → `enum`, embedded structs → nested `object`) — same source of truth as `describe_routes`. - **Tool generation**: per exposed type×op → `<type>_list`, `<type>_get`, `<type>_create`, `<type>_update`, `<type>_delete`, with `inputSchema` (JSON Schema) derived from catalog field types (unions → `enum`, embedded structs → nested `object`) — same source of truth as `describe_routes`.
- **`tools/call` dispatch** through the *same* handler paths REST uses: creates local, point ops `run_on(owner_of(id))`, lists fan out — no second data path. Engine/validation failures return `isError: true` inside the tool *result* (the MCP rule: execution errors are results, protocol errors are JSON-RPC errors). Mutations park on the WAL gate (decision 4). - **`tools/call` dispatch** through the *same* handler paths REST uses: creates local, point ops `run_on(owner_of(id))`, lists fan out — no second data path. Engine/validation failures return `isError: true` inside the tool *result* (the MCP rule: execution errors are results, protocol errors are JSON-RPC errors). Mutations park on the WAL gate (decision 4).
**Exit:** scripted flow (checked in beside [`reference/rest/`](../../reference/rest/README.md)) against the blog sample passes: `initialize` → `202` for `initialized` → `tools/list` enumerates exactly the exposed ops → `article_create` → `article_list` shows the row; runs green with `WO_GROUP_COMMIT` on and off; `GET`→405, `DELETE`→405, bad version→400, disallowed Origin→403; unit tests in the `server.rs` style cover envelope errors and gate parking. **Exit:** scripted flow (checked in beside [`.dev/reference/rest/`](../../.dev/reference/rest/README.md)) against the blog sample passes: `initialize` → `202` for `initialized` → `tools/list` enumerates exactly the exposed ops → `article_create` → `article_list` shows the row; runs green with `WO_GROUP_COMMIT` on and off; `GET`→405, `DELETE`→405, bad version→400, disallowed Origin→403; unit tests in the `server.rs` style cover envelope errors and gate parking.
### `15b-resources.md` — the schema and rows become addressable ### `15b-resources.md` — the schema and rows become addressable
@ -98,7 +98,7 @@ The rules the sub-phases implement, condensed from the spec — each MUST below
| Check | Target | How | | Check | Target | How |
| --- | --- | --- | | --- | --- | --- |
| Spec conformance | T1–T8 matrix green (status codes, headers, content types) | scripted curl flow checked in beside `reference/rest/` | | Spec conformance | T1–T8 matrix green (status codes, headers, content types) | scripted curl flow checked in beside `.dev/reference/rest/` |
| Interop | MCP Inspector connects, lists tools/resources, calls a tool | manual check, noted per release | | Interop | MCP Inspector connects, lists tools/resources, calls a tool | manual check, noted per release |
| Parity | `tools/call <type>_get` ≡ `GET /api/<type>/:id` byte-for-byte on the row payload | unit test | | Parity | `tools/call <type>_get` ≡ `GET /api/<type>/:id` byte-for-byte on the row payload | unit test |
| Durability | mutation results never precede their fsync CQE (`WO_GROUP_COMMIT` on) | gate test in `server.rs` style | | Durability | mutation results never precede their fsync CQE (`WO_GROUP_COMMIT` on) | gate test in `server.rs` style |
@ -122,3 +122,4 @@ The rules the sub-phases implement, condensed from the spec — each MUST below
- [`./05-hand-rolled-json.md`](05-hand-rolled-json.md) — removes this plan's `serde_json` use when it lands. - [`./05-hand-rolled-json.md`](05-hand-rolled-json.md) — removes this plan's `serde_json` use when it lands.
- [`./07-inotify-content-watcher.md`](07-inotify-content-watcher.md) — a future `notifications/tools/list_changed` on hot reload would pair with it (not scheduled). - [`./07-inotify-content-watcher.md`](07-inotify-content-watcher.md) — a future `notifications/tools/list_changed` on hot reload would pair with it (not scheduled).
- [MCP specification 2025-06-18](https://modelcontextprotocol.io/specification/2025-06-18/basic/transports) — the normative transport text summarized in T1–T8. - [MCP specification 2025-06-18](https://modelcontextprotocol.io/specification/2025-06-18/basic/transports) — the normative transport text summarized in T1–T8.
- [`../examples/mcp-think/`](../examples/mcp-think/README.md) — the consumer-side counterpart: a working stdio MCP server (local model via Ollama) that Claude calls today; useful as a live MCP client/server reference while building 15a.

View file

@ -2,7 +2,7 @@
> **Kanban: 🔄 in progress (Track 3 — Storage & durability)** — 16a ✅, 16b ✅ shipped; 16c–16f ⬜. Board: [00-kanban.md](00-kanban.md) > **Kanban: 🔄 in progress (Track 3 — Storage & durability)** — 16a ✅, 16b ✅ shipped; 16c–16f ⬜. Board: [00-kanban.md](00-kanban.md)
**Context sources:** [`README.md` § persistent database](../../README.md) (the product goal this implements: *"reads and writes database to RAM, persist data to postgres SQL"*), [`../runtime/database/03-inmemory-engine.md`](../runtime/database/03-inmemory-engine.md) (RAM-resident doctrine: disk sits behind the read path, never in front), [`./09-concurrency-scaleout.md`](./09-concurrency-scaleout.md) (per-shard WAL + ack-after-fsync this rides behind), [`./13-class-model-live-pricing.md`](./13-class-model-live-pricing.md) (the Product/Price worked example; `@table(name: "prices")` names the mirrored table), [`../runtime/database/07-wo-seg-migration.md`](../runtime/database/07-wo-seg-migration.md) (the dual-write precedent), `reference/postgresql/` (research symlink — `src/include/libpq/` for the wire protocol), PostgreSQL docs *Frontend/Backend Protocol*. **Context sources:** [`README.md` § persistent database](../../README.md) (the product goal this implements: *"reads and writes database to RAM, persist data to postgres SQL"*), [`../runtime/database/03-inmemory-engine.md`](../runtime/database/03-inmemory-engine.md) (RAM-resident doctrine: disk sits behind the read path, never in front), [`./09-concurrency-scaleout.md`](./09-concurrency-scaleout.md) (per-shard WAL + ack-after-fsync this rides behind), [`./13-class-model-live-pricing.md`](./13-class-model-live-pricing.md) (the Product/Price worked example; `@table(name: "prices")` names the mirrored table), [`../runtime/database/07-wo-seg-migration.md`](../runtime/database/07-wo-seg-migration.md) (the dual-write precedent), `.dev/reference/postgresql/` (research symlink — `src/include/libpq/` for the wire protocol), PostgreSQL docs *Frontend/Backend Protocol*.
## Context ## Context

View file

@ -15,8 +15,8 @@ Lay out the full `crates/` directory tree that the 7-phase `.wo` runtime design
1. **Scope: all 7 phases.** 14 new empty library crates covering Phases 2 → 6 land in one pass. `rt` (already shipping Stage 2) is the 15th. 1. **Scope: all 7 phases.** 14 new empty library crates covering Phases 2 → 6 land in one pass. `rt` (already shipping Stage 2) is the 15th.
2. **`rt` stays monolithic.** Today's Stage 2 code — lexer / parser / AST / compile / engine / server — stays inside `crates/rt/` and continues to satisfy the 14 existing unit tests. Code migrates into the new crates as each phase activates, not in this pass. 2. **`rt` stays monolithic.** Today's Stage 2 code — lexer / parser / AST / compile / engine / server — stays inside `crates/rt/` and continues to satisfy the 14 existing unit tests. Code migrates into the new crates as each phase activates, not in this pass.
3. **Contents: `Cargo.toml` + `src/lib.rs` doc-comment only.** Each `lib.rs` is one module-level `//!` doc block pointing at the phase doc, naming the responsibilities, and flagging which modules in `rt` migrate here later. No placeholder types, no stub traits. 3. **Contents: `Cargo.toml` + `src/lib.rs` doc-comment only.** Each `lib.rs` is one module-level `//!` doc block pointing at the phase doc, naming the responsibilities, and flagging which modules in `rt` migrate here later. No placeholder types, no stub traits.
4. **No `wo-` prefix.** New runtime crates are `ql`, `value`, `engine`, etc. — not `wo-ql`, `wo-value`. The prefix is redundant inside the project's own `wo` namespace and noisy in imports (`use ql::Parser` beats `use wo_ql::Parser`). The v1 crates in `reference/crates/` keep their `wo-` prefix — the distinct prefix makes the v1/v2 split visible at a glance. 4. **No `wo-` prefix.** New runtime crates are `ql`, `value`, `engine`, etc. — not `wo-ql`, `wo-value`. The prefix is redundant inside the project's own `wo` namespace and noisy in imports (`use ql::Parser` beats `use wo_ql::Parser`). The v1 crates in `.dev/reference/crates/` keep their `wo-` prefix — the distinct prefix makes the v1/v2 split visible at a glance.
5. **Workspace membership: root `Cargo.toml` lists every new crate as a member.** `reference/crates` stays `exclude`-d (nested workspace, separate v1 code). 5. **Workspace membership: root `Cargo.toml` lists every new crate as a member.** `.dev/reference/crates` stays `exclude`-d (nested workspace, separate v1 code).
Rationale and alternatives considered: see [`../../CLAUDE.md`](../../CLAUDE.md) "What's in `rt` today vs. what the empty crates promise" and the recorded `AskUserQuestion` answers that preceded this plan. Rationale and alternatives considered: see [`../../CLAUDE.md`](../../CLAUDE.md) "What's in `rt` today vs. what the empty crates promise" and the recorded `AskUserQuestion` answers that preceded this plan.
@ -53,7 +53,7 @@ All names are stable — documented in [`../runtime/database/07-wo-seg-migration
| `crates/README.md` with the phase-mapped inventory | ✅ | | `crates/README.md` with the phase-mapped inventory | ✅ |
| `cargo build` at root (compiles 15 crates) | ✅ | | `cargo build` at root (compiles 15 crates) | ✅ |
| `cargo test --lib` at root (14 existing `rt` tests) | ✅ | | `cargo test --lib` at root (14 existing `rt` tests) | ✅ |
| `cd reference/crates && cargo build && cargo test` (v1 still green) | ✅ | | `cd .dev/reference/crates && cargo build && cargo test` (v1 still green) | ✅ |
| `cargo run --bin wo -- run docs/examples/blog` (Stage 2 still serves) | ✅ | | `cargo run --bin wo -- run docs/examples/blog` (Stage 2 still serves) | ✅ |
## Non-scope ## Non-scope

View file

@ -21,17 +21,17 @@ Nothing is removed in this phase. The module sits alongside the tokio-backed axu
| File | Responsibility | Port source | | File | Responsibility | Port source |
| --- | --- | --- | | --- | --- | --- |
| `mod.rs` | Re-exports `EventLoop`, `Event`, `Interest`, `Token`, `EventFd`, `TimerFd`, `SignalFd` | [`reference/crates/wo-event/src/lib.rs`](../../reference/crates/wo-event/src/lib.rs) (9 LOC) | | `mod.rs` | Re-exports `EventLoop`, `Event`, `Interest`, `Token`, `EventFd`, `TimerFd`, `SignalFd` | [`.dev/reference/crates/wo-event/src/lib.rs`](../../.dev/reference/crates/wo-event/src/lib.rs) (9 LOC) |
| `netpoll_epoll.rs` | `EventLoop { fd, events }` — `new()`, `register(raw_fd, interest, token)`, `wait_once(timeout) -> &[Event]`, `deregister(raw_fd)` | [`reference/crates/wo-event/src/epoll.rs`](../../reference/crates/wo-event/src/epoll.rs) (183 LOC); [`reference/go/src/runtime/netpoll_epoll.go`](../../reference/go/src/runtime/netpoll_epoll.go) for idiom | | `netpoll_epoll.rs` | `EventLoop { fd, events }` — `new()`, `register(raw_fd, interest, token)`, `wait_once(timeout) -> &[Event]`, `deregister(raw_fd)` | [`.dev/reference/crates/wo-event/src/epoll.rs`](../../.dev/reference/crates/wo-event/src/epoll.rs) (183 LOC); [`.dev/reference/go/src/runtime/netpoll_epoll.go`](../../.dev/reference/go/src/runtime/netpoll_epoll.go) for idiom |
| `eventfd.rs` | `EventFd { fd }` — counter semaphore for cross-fd wake-up (subscription dispatch, shutdown signal) | [`reference/crates/wo-event/src/eventfd.rs`](../../reference/crates/wo-event/src/eventfd.rs) (66 LOC) | | `eventfd.rs` | `EventFd { fd }` — counter semaphore for cross-fd wake-up (subscription dispatch, shutdown signal) | [`.dev/reference/crates/wo-event/src/eventfd.rs`](../../.dev/reference/crates/wo-event/src/eventfd.rs) (66 LOC) |
| `timerfd.rs` | `TimerFd { fd }` — oneshot + periodic timers as fds for the loop | [`reference/crates/wo-event/src/timerfd.rs`](../../reference/crates/wo-event/src/timerfd.rs) (91 LOC) | | `timerfd.rs` | `TimerFd { fd }` — oneshot + periodic timers as fds for the loop | [`.dev/reference/crates/wo-event/src/timerfd.rs`](../../.dev/reference/crates/wo-event/src/timerfd.rs) (91 LOC) |
| `signalfd.rs` | `SignalFd { fd }` — SIGINT / SIGTERM / SIGHUP delivered as fd reads for graceful shutdown without a tokio signal handler | [`reference/crates/wo-event/src/signalfd.rs`](../../reference/crates/wo-event/src/signalfd.rs) (62 LOC) | | `signalfd.rs` | `SignalFd { fd }` — SIGINT / SIGTERM / SIGHUP delivered as fd reads for graceful shutdown without a tokio signal handler | [`.dev/reference/crates/wo-event/src/signalfd.rs`](../../.dev/reference/crates/wo-event/src/signalfd.rs) (62 LOC) |
Total: ~410 LOC lifted and adapted. The v1 code already compiles standalone in `reference/crates/wo-event/` and has unit tests; the port is near-verbatim plus namespace cleanups. Total: ~410 LOC lifted and adapted. The v1 code already compiles standalone in `.dev/reference/crates/wo-event/` and has unit tests; the port is near-verbatim plus namespace cleanups.
### Why `runtime/` not `event/` ### Why `runtime/` not `event/`
Go's equivalent code lives at [`reference/go/src/runtime/netpoll_epoll.go`](../../reference/go/src/runtime/netpoll_epoll.go) alongside siblings like `netpoll_kqueue.go` (macOS/BSD), `netpoll_io_uring.go` (if/when Go adds it), and the shared `netpoll.go` interface. The directory name "runtime" signals that this is the layer beneath user code — scheduler / netpoll / syscall shims — and the filename prefix `netpoll_<flavour>` makes each implementation alternative visible at a glance. Adopting the same convention in writeonce makes porting ideas bidirectional: a reader who knows Go's layout can find the writeonce equivalent by trimming the `.go` extension and swapping it for `.rs`. When Phase 3's io_uring arrives it'll land as `netpoll_io_uring.rs` next to the epoll one; a cross-platform stub would be `netpoll.rs`. Module boundary and naming both match. See [`docs/plan/assembly/00-overview.md`](./assembly/00-overview.md) for why we stop short of mirroring Go's assembly conventions. Go's equivalent code lives at [`.dev/reference/go/src/runtime/netpoll_epoll.go`](../../.dev/reference/go/src/runtime/netpoll_epoll.go) alongside siblings like `netpoll_kqueue.go` (macOS/BSD), `netpoll_io_uring.go` (if/when Go adds it), and the shared `netpoll.go` interface. The directory name "runtime" signals that this is the layer beneath user code — scheduler / netpoll / syscall shims — and the filename prefix `netpoll_<flavour>` makes each implementation alternative visible at a glance. Adopting the same convention in writeonce makes porting ideas bidirectional: a reader who knows Go's layout can find the writeonce equivalent by trimming the `.go` extension and swapping it for `.rs`. When Phase 3's io_uring arrives it'll land as `netpoll_io_uring.rs` next to the epoll one; a cross-platform stub would be `netpoll.rs`. Module boundary and naming both match. See [`docs/plan/assembly/00-overview.md`](./assembly/00-overview.md) for why we stop short of mirroring Go's assembly conventions.
### `Cargo.toml` change ### `Cargo.toml` change
@ -74,7 +74,7 @@ for event in loop_.wait_once(Some(Duration::from_millis(100)))? {
- `read()` on the eventfd returns `1`. - `read()` on the eventfd returns `1`.
3. A second unit test validates `TimerFd::oneshot(100ms)` fires within a `wait_once(500ms)` window. 3. A second unit test validates `TimerFd::oneshot(100ms)` fires within a `wait_once(500ms)` window.
4. All 14 existing `rt` tests still pass. `cargo run --bin wo -- run docs/examples/blog` still serves (tokio path unchanged). 4. All 14 existing `rt` tests still pass. `cargo run --bin wo -- run docs/examples/blog` still serves (tokio path unchanged).
5. `cd reference/crates && cargo build && cargo test` still green (nothing touched). 5. `cd .dev/reference/crates && cargo build && cargo test` still green (nothing touched).
## Non-scope ## Non-scope
@ -91,7 +91,7 @@ cargo build
cargo test --lib runtime # new tests in crates/rt/src/runtime/ cargo test --lib runtime # new tests in crates/rt/src/runtime/
cargo test --lib # all 14 existing + new epoll/eventfd/timerfd tests green cargo test --lib # all 14 existing + new epoll/eventfd/timerfd tests green
cargo run --bin wo -- run docs/examples/blog # axum path unchanged, still serves cargo run --bin wo -- run docs/examples/blog # axum path unchanged, still serves
cd reference/crates && cargo build && cargo test # v1 untouched cd .dev/reference/crates && cargo build && cargo test # v1 untouched
``` ```
## After this phase ## After this phase

View file

@ -9,10 +9,10 @@ A non-blocking HTTP/1.1 server module that accepts connections, parses requests,
## Design decisions (locked) ## Design decisions (locked)
1. **HTTP/1.1 only, keep-alive supported.** HTTP/2 and HTTP/3 are not on the roadmap for Stage 2 — they need ALPN / TLS support we don't have a plan for yet. HTTP/1.1 covers every endpoint the blog + ecommerce samples exercise. 1. **HTTP/1.1 only, keep-alive supported.** HTTP/2 and HTTP/3 are not on the roadmap for Stage 2 — they need ALPN / TLS support we don't have a plan for yet. HTTP/1.1 covers every endpoint the blog + ecommerce samples exercise.
2. **Per-connection state machine.** Each accepted socket fd is registered on the event loop with its own `Connection { state: Reading | Writing | Idle, parser, pending_response }`. Edge-triggered `EPOLLIN`/`EPOLLOUT` drive state transitions. Matches [v1 wo-http](../../reference/crates/wo-http/src/connection.rs)'s model verbatim. 2. **Per-connection state machine.** Each accepted socket fd is registered on the event loop with its own `Connection { state: Reading | Writing | Idle, parser, pending_response }`. Edge-triggered `EPOLLIN`/`EPOLLOUT` drive state transitions. Matches [v1 wo-http](../../.dev/reference/crates/wo-http/src/connection.rs)'s model verbatim.
3. **Router is pattern-matched at registration.** `Router::new().route("/api/articles/:id", Method::GET, handler)` resolves to a trie at boot. Per-request dispatch is a single trie walk — no axum-style type-erased layers. 3. **Router is pattern-matched at registration.** `Router::new().route("/api/articles/:id", Method::GET, handler)` resolves to a trie at boot. Per-request dispatch is a single trie walk — no axum-style type-erased layers.
4. **Handlers are `fn(&Request, &Engine) -> Response`.** Synchronous. The single-threaded event loop means a handler blocking is a bug; each handler must be a pure transformation over engine state. 4. **Handlers are `fn(&Request, &Engine) -> Response`.** Synchronous. The single-threaded event loop means a handler blocking is a bug; each handler must be a pure transformation over engine state.
5. **Module, not crate (yet).** Lives at `crates/rt/src/http/` with the same "extract when a second consumer shows up" rule as phase 02. The eventual home is the empty [`crates/http/`](../../crates/http/) sibling — but not in this phase. Paired with [phase 02's `crates/rt/src/runtime/`](./02-event-loop-epoll.md) (Go-style naming — `netpoll_epoll.rs`, `eventfd.rs`, …) which this module depends on for the `EventLoop` + raw syscall shims. Go's `src/net/http/` and `src/runtime/` split is the layout precedent; see [`reference/go/src/net/http/`](../../reference/go/src/net/http/). 5. **Module, not crate (yet).** Lives at `crates/rt/src/http/` with the same "extract when a second consumer shows up" rule as phase 02. The eventual home is the empty [`crates/http/`](../../crates/http/) sibling — but not in this phase. Paired with [phase 02's `crates/rt/src/runtime/`](./02-event-loop-epoll.md) (Go-style naming — `netpoll_epoll.rs`, `eventfd.rs`, …) which this module depends on for the `EventLoop` + raw syscall shims. Go's `src/net/http/` and `src/runtime/` split is the layout precedent; see [`.dev/reference/go/src/net/http/`](../../.dev/reference/go/src/net/http/).
## Scope ## Scope
@ -20,12 +20,12 @@ A non-blocking HTTP/1.1 server module that accepts connections, parses requests,
| File | Responsibility | Port source | | File | Responsibility | Port source |
| --- | --- | --- | | --- | --- | --- |
| `mod.rs` | Re-exports `Listener`, `Connection`, `Request`, `Response`, `Router`, `Method`, `Status` | [`reference/crates/wo-http/src/lib.rs`](../../reference/crates/wo-http/src/lib.rs) (4 LOC) | | `mod.rs` | Re-exports `Listener`, `Connection`, `Request`, `Response`, `Router`, `Method`, `Status` | [`.dev/reference/crates/wo-http/src/lib.rs`](../../.dev/reference/crates/wo-http/src/lib.rs) (4 LOC) |
| `listener.rs` | `Listener { fd }` wrapping `socket + bind + listen + accept4(SOCK_NONBLOCK \| SOCK_CLOEXEC)`; integrates with `EventLoop` | [`reference/crates/wo-http/src/listener.rs`](../../reference/crates/wo-http/src/listener.rs) (202 LOC) | | `listener.rs` | `Listener { fd }` wrapping `socket + bind + listen + accept4(SOCK_NONBLOCK \| SOCK_CLOEXEC)`; integrates with `EventLoop` | [`.dev/reference/crates/wo-http/src/listener.rs`](../../.dev/reference/crates/wo-http/src/listener.rs) (202 LOC) |
| `connection.rs` | Per-fd state machine: drain request bytes, parse, dispatch, drain response bytes, keep-alive or close | [`reference/crates/wo-http/src/connection.rs`](../../reference/crates/wo-http/src/connection.rs) (202 LOC) | | `connection.rs` | Per-fd state machine: drain request bytes, parse, dispatch, drain response bytes, keep-alive or close | [`.dev/reference/crates/wo-http/src/connection.rs`](../../.dev/reference/crates/wo-http/src/connection.rs) (202 LOC) |
| `request.rs` | Incremental HTTP/1.1 request parser: request line, headers, optional body. `Content-Length` only (no chunked request bodies in Stage 2 — they don't appear in the samples) | [`reference/crates/wo-http/src/request.rs`](../../reference/crates/wo-http/src/request.rs) (158 LOC) | | `request.rs` | Incremental HTTP/1.1 request parser: request line, headers, optional body. `Content-Length` only (no chunked request bodies in Stage 2 — they don't appear in the samples) | [`.dev/reference/crates/wo-http/src/request.rs`](../../.dev/reference/crates/wo-http/src/request.rs) (158 LOC) |
| `response.rs` | Response builder + writer: status line, headers, body (fixed or chunked) | [`reference/crates/wo-http/src/response.rs`](../../reference/crates/wo-http/src/response.rs) (110 LOC) | | `response.rs` | Response builder + writer: status line, headers, body (fixed or chunked) | [`.dev/reference/crates/wo-http/src/response.rs`](../../.dev/reference/crates/wo-http/src/response.rs) (110 LOC) |
| `route.rs` | Trie-based router: static paths + `:param` segments. `Router::route(method, path, handler) -> Router` | [`reference/crates/wo-route/src/router.rs`](../../reference/crates/wo-route/src/router.rs) (127 LOC) + [`pattern.rs`](../../reference/crates/wo-route/src/pattern.rs) (146 LOC) | | `route.rs` | Trie-based router: static paths + `:param` segments. `Router::route(method, path, handler) -> Router` | [`.dev/reference/crates/wo-route/src/router.rs`](../../.dev/reference/crates/wo-route/src/router.rs) (127 LOC) + [`pattern.rs`](../../.dev/reference/crates/wo-route/src/pattern.rs) (146 LOC) |
Total: ~949 LOC ported. Most of it is mechanical adaptation from v1; the namespace + the `Interest` enum change from phase 02 are the only non-trivial edits. Total: ~949 LOC ported. Most of it is mechanical adaptation from v1; the namespace + the `Interest` enum change from phase 02 are the only non-trivial edits.
@ -76,7 +76,7 @@ loop {
2. An integration test (also in `crates/rt/tests/http_smoke.rs` or similar) spawns the binary, sends three `curl` equivalents using `std::net::TcpStream`, validates status codes and bodies. 2. An integration test (also in `crates/rt/tests/http_smoke.rs` or similar) spawns the binary, sends three `curl` equivalents using `std::net::TcpStream`, validates status codes and bodies.
3. All 14 existing `rt` tests still pass. 3. All 14 existing `rt` tests still pass.
4. `wo run docs/examples/blog` unchanged — axum path still drives the real CLI. 4. `wo run docs/examples/blog` unchanged — axum path still drives the real CLI.
5. `cargo build` at root; `cd reference/crates && cargo build` still green. 5. `cargo build` at root; `cd .dev/reference/crates && cargo build` still green.
## Non-scope ## Non-scope
@ -97,4 +97,4 @@ cargo run --bin wo -- run docs/examples/blog # axum path unchanged
## After this phase ## After this phase
Phase 04 takes the same in-memory `Engine` that the axum router serves and points the phase-03 router at it instead. Removing `tokio`, `axum`, `tower` is a consequence; the behaviour visible to `reference/rest/blog.rest` does not change. Phase 04 takes the same in-memory `Engine` that the axum router serves and points the phase-03 router at it instead. Removing `tokio`, `axum`, `tower` is a consequence; the behaviour visible to `.dev/reference/rest/blog.rest` does not change.

View file

@ -4,7 +4,7 @@
## Goal ## Goal
Flip the `wo` binary off the tokio + axum stack and onto the phase-02 event loop + phase-03 HTTP server. Delete three dependencies from `crates/rt/Cargo.toml`. REST behaviour visible to [`reference/rest/blog.rest`](../../reference/rest/blog.rest) does not change — same status codes, same response bodies, same endpoint paths. Flip the `wo` binary off the tokio + axum stack and onto the phase-02 event loop + phase-03 HTTP server. Delete three dependencies from `crates/rt/Cargo.toml`. REST behaviour visible to [`.dev/reference/rest/blog.rest`](../../.dev/reference/rest/blog.rest) does not change — same status codes, same response bodies, same endpoint paths.
This is the first phase where the dependency count goes *down*. Phases 02 and 03 were additive; this one is the switch. This is the first phase where the dependency count goes *down*. Phases 02 and 03 were additive; this one is the switch.
@ -77,13 +77,13 @@ Twelve handlers total — one pair per `{list, get, create, update, delete}` ×
1. **`cargo build`** at root — compiles with four deps (not seven). 1. **`cargo build`** at root — compiles with four deps (not seven).
2. **`cargo test --lib`** — all 14 existing `rt` unit tests still pass. A new test in `src/server.rs` exercises the router build from a compiled catalog (no HTTP, just static registration). 2. **`cargo test --lib`** — all 14 existing `rt` unit tests still pass. A new test in `src/server.rs` exercises the router build from a compiled catalog (no HTTP, just static registration).
3. **End-to-end REST smoke — the 20-assertion battery from [`reference/rest/blog.rest`](../../reference/rest/blog.rest)** must pass byte-identical to Stage 2 today. Script: 3. **End-to-end REST smoke — the 20-assertion battery from [`.dev/reference/rest/blog.rest`](../../.dev/reference/rest/blog.rest)** must pass byte-identical to Stage 2 today. Script:
```bash ```bash
WO_LISTEN=127.0.0.1:8765 cargo run --bin wo -- run docs/examples/blog & WO_LISTEN=127.0.0.1:8765 cargo run --bin wo -- run docs/examples/blog &
# ... curl each block, check expected status # ... curl each block, check expected status
``` ```
4. **Graceful shutdown.** SIGINT on the process exits cleanly (no panic, no orphan fds). Validate with `strace -f -e signalfd4,close` on shutdown. 4. **Graceful shutdown.** SIGINT on the process exits cleanly (no panic, no orphan fds). Validate with `strace -f -e signalfd4,close` on shutdown.
5. **`cd reference/crates && cargo build && cargo test`** still green. 5. **`cd .dev/reference/crates && cargo build && cargo test`** still green.
6. **Dep audit.** `cargo tree -p rt --depth 1` shows `libc` as the only non-transitive external dep beyond `anyhow`, `serde`, `serde_json`. 6. **Dep audit.** `cargo tree -p rt --depth 1` shows `libc` as the only non-transitive external dep beyond `anyhow`, `serde`, `serde_json`.
## Non-scope ## Non-scope
@ -107,11 +107,11 @@ cargo test --lib # 14 + any new server.rs tes
WO_LISTEN=127.0.0.1:8765 cargo run --bin wo -- run docs/examples/blog & WO_LISTEN=127.0.0.1:8765 cargo run --bin wo -- run docs/examples/blog &
PID=$! PID=$!
sleep 2 sleep 2
# every block in reference/rest/blog.rest, via curl, checking %{http_code} # every block in .dev/reference/rest/blog.rest, via curl, checking %{http_code}
# (copy-paste the 20-assertion script from the Stage 2 turn that verified blog.rest) # (copy-paste the 20-assertion script from the Stage 2 turn that verified blog.rest)
kill $PID kill $PID
cd reference/crates && cargo build && cargo test # v1 untouched cd .dev/reference/crates && cargo build && cargo test # v1 untouched
``` ```
## After this phase ## After this phase

View file

@ -1,14 +1,14 @@
# 00 — The role of assembly in a runtime # 00 — The role of assembly in a runtime
Why does a runtime ship hand-written assembly at all? Three reasons — each one a place where a higher-level language literally cannot express the operation it needs, so the compiler is bypassed and machine instructions are written directly. Go's [`src/runtime/`](../../../reference/go/src/runtime/) is the canonical example; this doc names the three reasons and points at the Go files that embody each. Why does a runtime ship hand-written assembly at all? Three reasons — each one a place where a higher-level language literally cannot express the operation it needs, so the compiler is bypassed and machine instructions are written directly. Go's [`src/runtime/`](../../../.dev/reference/go/src/runtime/) is the canonical example; this doc names the three reasons and points at the Go files that embody each.
## 1 — Operations that violate the language's own calling convention ## 1 — Operations that violate the language's own calling convention
The biggest category. The language's calling convention — how arguments are passed, who saves which registers, how the stack grows — is the contract every compiled function obeys. A few runtime operations *have* to break it because they ARE the mechanism by which control flow enters and exits that contract. The biggest category. The language's calling convention — how arguments are passed, who saves which registers, how the stack grows — is the contract every compiled function obeys. A few runtime operations *have* to break it because they ARE the mechanism by which control flow enters and exits that contract.
**Goroutine stack switching.** When Go's scheduler switches from one goroutine to another, it's literally rewriting the stack pointer mid-function — jumping from one goroutine's stack to another's. The language compiler can't emit this safely because every function assumes its stack is the one it got called on. See [`reference/go/src/runtime/asm_amd64.s`](../../../reference/go/src/runtime/asm_amd64.s) for `TEXT runtime·gogo(SB)`, `TEXT runtime·mcall(SB)`, `TEXT runtime·systemstack(SB)` — all unavoidable. **Goroutine stack switching.** When Go's scheduler switches from one goroutine to another, it's literally rewriting the stack pointer mid-function — jumping from one goroutine's stack to another's. The language compiler can't emit this safely because every function assumes its stack is the one it got called on. See [`.dev/reference/go/src/runtime/asm_amd64.s`](../../../.dev/reference/go/src/runtime/asm_amd64.s) for `TEXT runtime·gogo(SB)`, `TEXT runtime·mcall(SB)`, `TEXT runtime·systemstack(SB)` — all unavoidable.
**Signal-handler entry.** When a signal arrives, the kernel drops the process onto an alternate stack with preserved registers. Returning to normal code means restoring everything the handler touched plus switching stacks back. Go's `runtime·sigtramp` in [`reference/go/src/runtime/sys_linux_amd64.s`](../../../reference/go/src/runtime/sys_linux_amd64.s) handles this. **Signal-handler entry.** When a signal arrives, the kernel drops the process onto an alternate stack with preserved registers. Returning to normal code means restoring everything the handler touched plus switching stacks back. Go's `runtime·sigtramp` in [`.dev/reference/go/src/runtime/sys_linux_amd64.s`](../../../.dev/reference/go/src/runtime/sys_linux_amd64.s) handles this.
**Cgo boundary crossing.** Calling C from Go means switching to the OS thread's "real" stack (C expects contiguous stacks; Go uses segmented). Going back means the inverse. Entirely asm-driven. **Cgo boundary crossing.** Calling C from Go means switching to the OS thread's "real" stack (C expects contiguous stacks; Go uses segmented). Going back means the inverse. Entirely asm-driven.
@ -16,17 +16,17 @@ The biggest category. The language's calling convention — how arguments are pa
Atomics, memory barriers, and some hardware-accelerated primitives need specific instruction sequences. A compiler that sees `a = *b` can't know whether you wanted a relaxed load or an acquire fence without annotation — and the *right* instruction on x86 vs ARM vs RISC-V is different. Atomics, memory barriers, and some hardware-accelerated primitives need specific instruction sequences. A compiler that sees `a = *b` can't know whether you wanted a relaxed load or an acquire fence without annotation — and the *right* instruction on x86 vs ARM vs RISC-V is different.
**Atomic CAS / load-acquire / store-release.** On x86 it's `LOCK CMPXCHG`; on ARM it's `LDXR` / `STXR` with a retry loop; on RISC-V it's `LR.W.AQ` / `SC.W.RL`. Go emits these from [`reference/go/src/runtime/atomic_amd64.s`](../../../reference/go/src/runtime/atomic_amd64.s) (and its per-arch siblings) because a portable compiler can't. **Atomic CAS / load-acquire / store-release.** On x86 it's `LOCK CMPXCHG`; on ARM it's `LDXR` / `STXR` with a retry loop; on RISC-V it's `LR.W.AQ` / `SC.W.RL`. Go emits these from [`.dev/reference/go/src/runtime/atomic_amd64.s`](../../../.dev/reference/go/src/runtime/atomic_amd64.s) (and its per-arch siblings) because a portable compiler can't.
**Memory barriers.** `MFENCE`, `LFENCE`, `SFENCE` on x86; `DMB` / `DSB` / `ISB` on ARM. Used by Go's `publicationBarrier`, `procyield`, and friends. Per-arch asm files carry them. **Memory barriers.** `MFENCE`, `LFENCE`, `SFENCE` on x86; `DMB` / `DSB` / `ISB` on ARM. Used by Go's `publicationBarrier`, `procyield`, and friends. Per-arch asm files carry them.
**Optimised `memmove` / `memequal` / `memclr`.** The compiler knows how to emit `rep movsb`, but a runtime sometimes ships a *better* version than the compiler's — wider vector loads, prefetch hints, alignment-aware loops. Go ships its own in [`asm_amd64.s`](../../../reference/go/src/runtime/asm_amd64.s) using AVX/SSE paths. **Optimised `memmove` / `memequal` / `memclr`.** The compiler knows how to emit `rep movsb`, but a runtime sometimes ships a *better* version than the compiler's — wider vector loads, prefetch hints, alignment-aware loops. Go ships its own in [`asm_amd64.s`](../../../.dev/reference/go/src/runtime/asm_amd64.s) using AVX/SSE paths.
## 3 — Syscall trampolines ## 3 — Syscall trampolines
Every raw syscall to the kernel is an asm stub. The kernel expects arguments in specific registers (on x86_64: `rdi`, `rsi`, `rdx`, `r10`, `r8`, `r9`, with the syscall number in `rax`), a `syscall` instruction, and return-value unpacking from `rax` (including `-errno` convention). A high-level language's calling convention doesn't match that layout — you need a thin asm wrapper per syscall. Every raw syscall to the kernel is an asm stub. The kernel expects arguments in specific registers (on x86_64: `rdi`, `rsi`, `rdx`, `r10`, `r8`, `r9`, with the syscall number in `rax`), a `syscall` instruction, and return-value unpacking from `rax` (including `-errno` convention). A high-level language's calling convention doesn't match that layout — you need a thin asm wrapper per syscall.
See [`reference/go/src/runtime/sys_linux_amd64.s`](../../../reference/go/src/runtime/sys_linux_amd64.s) — 43 `TEXT` functions, one per syscall family: `runtime·write`, `runtime·read`, `runtime·futex`, `runtime·clone`, `runtime·rt_sigaction`, `runtime·rt_sigprocmask`, `runtime·rt_sigreturn`, `runtime·sched_yield`, `runtime·mmap`, `runtime·munmap`, `runtime·madvise`, `runtime·epollcreate1`, `runtime·epollctl`, `runtime·epollwait`, etc. See [`.dev/reference/go/src/runtime/sys_linux_amd64.s`](../../../.dev/reference/go/src/runtime/sys_linux_amd64.s) — 43 `TEXT` functions, one per syscall family: `runtime·write`, `runtime·read`, `runtime·futex`, `runtime·clone`, `runtime·rt_sigaction`, `runtime·rt_sigprocmask`, `runtime·rt_sigreturn`, `runtime·sched_yield`, `runtime·mmap`, `runtime·munmap`, `runtime·madvise`, `runtime·epollcreate1`, `runtime·epollctl`, `runtime·epollwait`, etc.
Go does these in asm because it cannot rely on libc — Go's scheduler needs to enter/exit syscalls at exactly controlled points (`runtime·entersyscall`, `runtime·exitsyscall`) so the M (OS thread) can be parked or reused without losing the goroutine. Going through `libc::write` would sidestep the scheduler's accounting. Go does these in asm because it cannot rely on libc — Go's scheduler needs to enter/exit syscalls at exactly controlled points (`runtime·entersyscall`, `runtime·exitsyscall`) so the M (OS thread) can be parked or reused without losing the goroutine. Going through `libc::write` would sidestep the scheduler's accounting.

View file

@ -2,11 +2,11 @@
The Go runtime ships ~72 `TEXT` functions in `asm_amd64.s` alone, ~43 in `sys_linux_amd64.s`, and per-architecture variants of both for `386`, `arm`, `arm64`, `loong64`, `mips(64)x`, `ppc64x`, `riscv64`, `s390x`, `wasm`. This doc inventories them by purpose so a reader can map each Go asm concern to the writeonce equivalent (spoiler: usually "Rust stdlib does it"). Follow-on reading: [`02-writeonce-stance.md`](./02-writeonce-stance.md). The Go runtime ships ~72 `TEXT` functions in `asm_amd64.s` alone, ~43 in `sys_linux_amd64.s`, and per-architecture variants of both for `386`, `arm`, `arm64`, `loong64`, `mips(64)x`, `ppc64x`, `riscv64`, `s390x`, `wasm`. This doc inventories them by purpose so a reader can map each Go asm concern to the writeonce equivalent (spoiler: usually "Rust stdlib does it"). Follow-on reading: [`02-writeonce-stance.md`](./02-writeonce-stance.md).
All paths are inside [`reference/go/src/runtime/`](../../../reference/go/src/runtime/). All paths are inside [`.dev/reference/go/src/runtime/`](../../../.dev/reference/go/src/runtime/).
## Scheduler & stack switching — `asm_<arch>.s` ## Scheduler & stack switching — `asm_<arch>.s`
One file per arch, everything that has to break Go's calling convention. The x86_64 version lives at [`asm_amd64.s`](../../../reference/go/src/runtime/asm_amd64.s). One file per arch, everything that has to break Go's calling convention. The x86_64 version lives at [`asm_amd64.s`](../../../.dev/reference/go/src/runtime/asm_amd64.s).
| Go symbol | What | | Go symbol | What |
| --- | --- | | --- | --- |
@ -26,7 +26,7 @@ One file per arch, everything that has to break Go's calling convention. The x86
## Atomics & barriers — `internal/runtime/atomic/atomic_<arch>.s` ## Atomics & barriers — `internal/runtime/atomic/atomic_<arch>.s`
Lives at [`internal/runtime/atomic/atomic_amd64.s`](../../../reference/go/src/internal/runtime/atomic/atomic_amd64.s) (and arch variants). Wrappers around arch-specific instructions: Lives at [`internal/runtime/atomic/atomic_amd64.s`](../../../.dev/reference/go/src/internal/runtime/atomic/atomic_amd64.s) (and arch variants). Wrappers around arch-specific instructions:
| Go symbol | x86 instruction | Purpose | | Go symbol | x86 instruction | Purpose |
| --- | --- | --- | | --- | --- | --- |
@ -41,7 +41,7 @@ Lives at [`internal/runtime/atomic/atomic_amd64.s`](../../../reference/go/src/in
## Syscall trampolines — `sys_<os>_<arch>.s` ## Syscall trampolines — `sys_<os>_<arch>.s`
On Linux-x86_64 that's [`sys_linux_amd64.s`](../../../reference/go/src/runtime/sys_linux_amd64.s) — 43 `TEXT` functions. Each is a short wrapper: move args into the kernel's register layout, execute `SYSCALL`, convert `rax` into a Go return value + error. On Linux-x86_64 that's [`sys_linux_amd64.s`](../../../.dev/reference/go/src/runtime/sys_linux_amd64.s) — 43 `TEXT` functions. Each is a short wrapper: move args into the kernel's register layout, execute `SYSCALL`, convert `rax` into a Go return value + error.
| Go symbol | Linux syscall | | Go symbol | Linux syscall |
| --- | --- | | --- | --- |
@ -70,7 +70,7 @@ On Linux-x86_64 that's [`sys_linux_amd64.s`](../../../reference/go/src/runtime/s
## Cgo bridge — `cgo_<os>_<arch>.s` ## Cgo bridge — `cgo_<os>_<arch>.s`
Files like [`cgo/asm_amd64.s`](../../../reference/go/src/runtime/cgo/asm_amd64.s). Machine-code marshalling between Go's register convention and C's SysV AMD64 ABI. Needed because Go's calling convention uses stack slots differently from C's register passing. Files like [`cgo/asm_amd64.s`](../../../.dev/reference/go/src/runtime/cgo/asm_amd64.s). Machine-code marshalling between Go's register convention and C's SysV AMD64 ABI. Needed because Go's calling convention uses stack slots differently from C's register passing.
**Writeonce doesn't cross language boundaries** — Rust is the only language in the binary; `libc` is already in Rust's register convention via `extern "C"`. No cgo bridge needed. **Writeonce doesn't cross language boundaries** — Rust is the only language in the binary; `libc` is already in Rust's register convention via `extern "C"`. No cgo bridge needed.

View file

@ -0,0 +1,180 @@
# Blue/Green VMs — a self-hosting, agent-managed runtime
> **Partially superseded (2026-08-03):** the deployment subsystem (§5–§6 here)
> is now specified in
> [`docs/superpowers/specs/2026-08-03-blue-green-vm-design.md`](../../../superpowers/specs/2026-08-03-blue-green-vm-design.md)
> — developer + `wo` CLI as the management client (agent/MCP becomes a later
> wrapper), schema migration folded into the approval step (additive-only
> auto-diff in v1), fixed slots with alternating activity, HTTP+JSON+SSE.
> §1–§4 (transports, recipe box, fibers, source-in-binary) remain current
> thinking feeding plans 3/4/6.
> Thought-process capture (2026-08-02). Not a phase plan yet — the vision that
> shapes how the wovm runtime grows past milestone 1, recorded before the
> details harden. Related: the OOP spec
> ([`../../../superpowers/specs/2026-08-01-oop-compiler-vm-design.md`](../../../superpowers/specs/2026-08-01-oop-compiler-vm-design.md)),
> plan 4 (shard-actor runtime), plan 15 (MCP streamable HTTP), and the
> single-binary trailer already shipped by `woc build`.
## The idea, in five sentences
The writeonce executable is a **systemd service that never stops**. It embeds
its own **source code**, not just its bytecode. An external **Claude agent**
reads and edits that source through a managed channel; an approved change is
compiled **inside the runtime** and loaded into the idle VM slot. The runtime
holds **exactly two VMs — Blue (active) and Green (previous version)** — and
deployment is an atomic switch between them. Rollback is the same switch in
reverse, because the previous version never left memory.
## 1. A runtime is not a port
The runtime core is the VM pair + engine + scheduler — it must run with zero
listeners. Ports are **transports**, attached at boot like modules: an HTTP
listener, a unix socket, an MCP endpoint, stdio. Consequences:
- The same binary serves as web app, CLI batch runner, or agent-managed
service depending on which transports the deployment attaches — one of the
recipes a custom web framework builds from (§2).
- **systemd socket activation** fits exactly: the unit owns the socket
(`LISTEN_FDS`), the runtime accepts on whatever fds it inherits. The
"always running" property (§6) and the "no port of its own" property come
from the same mechanism.
## 2. The runtime is a recipe box for web frameworks
Everything a custom web framework needs in later phases must exist as a
separable runtime capability, not a monolith: transports (§1), fibers (§3),
routing surface (plan 6), the subscription registry (plan 7), the DB engine
(plan 5), and the deploy/rollback machinery (§5). A "framework" in a later
phase is a `.wo` library that composes these recipes — the runtime itself
stays framework-agnostic.
## 3. Fibers (green threads)
Concurrency inside a shard is **cooperative fibers scheduled by the VM**, not
OS threads — the Erlang shape on the wovm substrate:
- A fiber is exactly the execution state `wo_vm` already isolates: a register
window stack + frame stack + a current pc. Making that state per-fiber
instead of per-VM turns the interpreter into a fiber scheduler almost for
free.
- **Preemption by reduction budget**: the dispatch loop decrements a counter
per instruction (or per call/back-edge); at zero, the fiber parks and the
scheduler picks the next runnable one. No signals, no stack switching
tricks, deterministic and debuggable.
- Fibers **park on I/O**: a blocked read hands the fd to the shard's event
loop (`wo-rt.c`'s epoll/io_uring machinery) and the fiber resumes when the
completion arrives. One OS thread per core (plan 4's shard), thousands of
fibers per shard.
- Fits the ownership model: a fiber is an actor mailbox owner; cross-fiber
sends follow the same ownership-move rule as cross-shard sends.
## 4. The binary contains its source
`woc build` already appends the `.wob` image to a copy of `wovm` with an
offset trailer. The trailer grows one more section: **the `.wo` source tree**
(paths + contents, compressed). Why:
- The deployed artifact is self-describing — no "which commit is prod
running?" class of question. `wovm --dump-source` can always reproduce
exactly what is executing.
- The agent workflow (§5) needs a source of truth that travels with the
binary, not a checkout that can drift from it.
- After a deployment, the runtime rewrites its own source section (write to
temp, fsync, rename) so the artifact on disk always matches the Blue VM.
## 5. Agent-managed source — how Claude fits
The runtime exposes a **management transport** (MCP over streamable HTTP —
plan 15's machinery, localhost + bearer token, the log-watcher posture).
Claude Code connects as an MCP client. Tools the runtime serves:
| Tool | What it does |
| --- | --- |
| `source_list` / `source_read` | browse the embedded source tree of the running (Blue) version |
| `source_propose` | submit a changed file set as a **proposal** — staged, never applied |
| `proposal_diff` | render the pending proposal against Blue's source |
| `proposal_check` | run `woc check` on the proposal inside the runtime — diagnostics come back to the agent |
| `proposal_approve` | **human-only gate** (separate credential or out-of-band confirmation) — approval triggers compile + green-slot load |
| `deploy_switch` | atomic Blue↔Green switch after health checks |
| `deploy_rollback` | the same switch back — Green still holds the previous version |
| `deploy_status` | which version is Blue, which is Green, in-flight drain state |
Properties worth pinning now:
- **The agent proposes; a human approves.** `proposal_approve` is not
reachable with the agent's token. Approval is the compile trigger, not the
edit.
- **Every step is WAL-logged** — proposals, diagnostics, approvals, switches,
rollbacks form an audit trail that survives crashes like any other commit.
- **The compiler lives with the runtime** for this loop to work: either
`woc` embedded in the binary (adds OCaml runtime weight) or shipped beside
it in the service directory (lighter; the systemd unit owns both files).
Open question in §8 — start with "beside it".
## 6. Blue/Green VM lifecycle
Exactly **two VM slots** per runtime, never more:
- **Blue** — the active VM: all new requests/fibers dispatch into it.
- **Green** — the previous version, loaded and warm: the instant-rollback
target. After a successful deploy the roles swap; the old Blue becomes the
new Green.
The critical separation: **VMs own code, the engine owns data.** Tables,
WAL, subscriptions, and the arena slabs live in the engine layer beneath both
VMs; a switch swaps which bytecode handles requests, never the data. That is
what makes the switch cheap and rollback safe — no state migration on the
happy path (and schema changes are exactly the hard part, §8).
Deploy sequence:
1. Approved proposal compiles (`woc emit`) — failure ends the deploy,
Blue untouched.
2. New image loads + validates into the idle slot (loader is the same
validation battery as always — a bad image cannot boot).
3. Health gate: entry smoke / conformance subset runs against the idle VM.
4. **Switch at the dispatch boundary**: new work enters the new Blue;
in-flight fibers on the old VM drain to completion (bounded timeout).
5. Old Blue becomes Green (rollback target); the binary's source section is
rewritten to match (§4).
6. `deploy_rollback` at any later point is step 4 in reverse — no compile,
no load, the code is already resident.
## 7. Always running
The executable maps to a **systemd service**: `Restart=always`, socket
activation for the transports (§1), the hardening posture proven in the
log-watcher units (unprivileged user, read-only system, `StateDirectory`
for WAL/data). Deployment never restarts the unit — that is the whole point
of the VM pair. The unit restarting (crash, host reboot) boots Blue from the
binary's current source/bytecode section and reloads Green only when the
next deploy happens.
## 8. Open questions (deliberately unresolved here)
1. **Schema migrations.** Code switches atomically; data does not. A
proposal that changes a class's fields needs a migration story between
Green-shaped and Blue-shaped rows — the wo-seg migration doc's
dual-write thinking applies inside one process. Hardest problem in this
vision; needs its own exploration.
2. **Live subscriptions across a switch.** Do WebSocket subscribers survive
a deploy (registry lives in the engine layer → yes, by design), and what
do they see mid-drain?
3. **`woc` placement** — beside the binary vs embedded (§5).
4. **Fiber preemption granularity** — per-instruction counter vs
call/back-edge only (cheaper, coarser).
5. **Does Green count against the heap budget** (two arenas resident) or
does Green hibernate (bytecode resident, heap lazily rebuilt on
rollback)?
## 9. Where this lands in the plan sequence
- Fibers (§3): extends **plan 4** (shard-actor runtime) — same scheduler
work, one more scheduling unit.
- Transports-not-ports (§1): shapes **plan 6** (HTTP/service layer) — the
listener becomes one attachable transport among several.
- Management MCP (§5): builds on **plan 15**'s streamable-HTTP machinery.
- Source-in-binary (§4): extends plan 3's `woc build` trailer.
- Blue/Green switch (§6) + agent loop (§5): a new phase after those land —
needs spec + plan of its own once this vision stabilizes.

View file

@ -2,11 +2,11 @@
> **Kanban: ✅ done** — phases A–F all shipped with measured exit evidence below. Board: [../../00-kanban.md](../../00-kanban.md) > **Kanban: ✅ done** — phases A–F all shipped with measured exit evidence below. Board: [../../00-kanban.md](../../00-kanban.md)
**Context sources:** [`prototypes/wo-rt-c/wo-rt.c`](../../../../prototypes/wo-rt-c/wo-rt.c) (phase 0 — the single-threaded epoll baseline), [`../../09-concurrency-scaleout.md`](../../09-concurrency-scaleout.md) (the thread-per-core doctrine every phase here miniaturizes), [`../../10-storage-foundations.md`](../../10-storage-foundations.md) / [`11-wal-and-recovery.md`](../../11-wal-and-recovery.md) / [`12-engine-disk-cutover.md`](../../12-engine-disk-cutover.md) (the storage track), kernel reference cards [`../linux/07-io_uring.md`](../linux/07-io_uring.md), [`08-mmap.md`](../linux/08-mmap.md), [`09-fallocate.md`](../linux/09-fallocate.md), [`12-pwrite-fsync.md`](../linux/12-pwrite-fsync.md), [`02-eventfd.md`](../linux/02-eventfd.md). **Context sources:** [`runtime/wo-rt.c`](../../../../runtime/wo-rt.c) (phase 0 — the single-threaded epoll baseline), [`../../09-concurrency-scaleout.md`](../../09-concurrency-scaleout.md) (the thread-per-core doctrine every phase here miniaturizes), [`../../10-storage-foundations.md`](../../10-storage-foundations.md) / [`11-wal-and-recovery.md`](../../11-wal-and-recovery.md) / [`12-engine-disk-cutover.md`](../../12-engine-disk-cutover.md) (the storage track), kernel reference cards [`../linux/07-io_uring.md`](../linux/07-io_uring.md), [`08-mmap.md`](../linux/08-mmap.md), [`09-fallocate.md`](../linux/09-fallocate.md), [`12-pwrite-fsync.md`](../linux/12-pwrite-fsync.md), [`02-eventfd.md`](../linux/02-eventfd.md).
## Goal ## Goal
Evolve the [`prototypes/wo-rt-c/`](../../../../prototypes/wo-rt-c/) prototype from a single-threaded epoll reference into a **multi-threaded runtime environment for writeonce applications**: thread-per-core io_uring event loops at million-scale read/write concurrency, the whole database resident in RAM (one mmap arena, addressed per shard — no duplication), **ACID** commits that dual-write RAM-first-then-disk, and a boot path that loads the hard drive's state back into RAM before serving. Still one C file's worth of honesty per concern, still **zero dependencies beyond libc** — raw io_uring syscalls, no liburing. Evolve the [`runtime/`](../../../../runtime/) prototype from a single-threaded epoll reference into a **multi-threaded runtime environment for writeonce applications**: thread-per-core io_uring event loops at million-scale read/write concurrency, the whole database resident in RAM (one mmap arena, addressed per shard — no duplication), **ACID** commits that dual-write RAM-first-then-disk, and a boot path that loads the hard drive's state back into RAM before serving. Still one C file's worth of honesty per concern, still **zero dependencies beyond libc** — raw io_uring syscalls, no liburing.
Each phase is the executable proving ground for the matching Rust plan (09–12): get the syscall sequence right here in a few hundred lines, then port with confidence. Each phase is the executable proving ground for the matching Rust plan (09–12): get the syscall sequence right here in a few hundred lines, then port with confidence.
@ -66,9 +66,9 @@ Boot, before any listener opens: each thread replays its own WAL into its arena
### Phase F — million-scale harness + ACID verification — ✅ shipped ### Phase F — million-scale harness + ACID verification — ✅ shipped
*Maps to [plan 09's verification-targets table](../../09-concurrency-scaleout.md).* *Maps to [plan 09's verification-targets table](../../09-concurrency-scaleout.md).*
`setrlimit(RLIMIT_NOFILE)` raised at boot. A small C load client under `prototypes/wo-rt-c/bench/` (keep-alive, pipelined GETs, latency timestamps — `wrk` would be an external dep). Measure honestly on the dev box and commit the numbers to the prototype README: aggregate read req/s across cores (goal order 10⁶/s on 8–16 cores), concurrent open connections (goal order 10⁵–10⁶; ~8 KB/conn + fd limits are the ceiling), commits/s under group fsync, p99 read latency under write load. ACID scripts: torn-WAL injection (atomicity), single-shard interleaving probe (isolation), the phase-D crash test under load (durability). A `just rt-c-bench` recipe runs it all. `setrlimit(RLIMIT_NOFILE)` raised at boot. A small C load client under `runtime/bench/` (keep-alive, pipelined GETs, latency timestamps — `wrk` would be an external dep). Measure honestly on the dev box and commit the numbers to the prototype README: aggregate read req/s across cores (goal order 10⁶/s on 8–16 cores), concurrent open connections (goal order 10⁵–10⁶; ~8 KB/conn + fd limits are the ceiling), commits/s under group fsync, p99 read latency under write load. ACID scripts: torn-WAL injection (atomicity), single-shard interleaving probe (isolation), the phase-D crash test under load (durability). A `just rt-c-bench` recipe runs it all.
**Exit (met):** measured on a 20-core box (table in the [prototype README](../../../../prototypes/wo-rt-c/README.md)): **908,916 reads/s p99 154 µs and 643,250 fsync-acked commits/s p99 177 µs** on 8 shards — vs Go `net/http` on 20 cores at 495k/355k with ~8× worse p99 and no durability (.NET unavailable on the box); 10k idle connections, 0 errors; only 2xx counted (the client tracks status codes). **The crash-under-load test found two real durability bugs the phase-D test missed** — an ack-armed-before-fsync race in `conn_continue` (route parks the response *during* `try_process`; the pre-check missed it) and an fd-reuse ABA hazard in batch ack-parking (fixed with per-connection generation stamps). After both fixes, three `kill -9`-mid-bench rounds at ~1–2M commits each showed **WAL records ≥ acked, every round** (one exact). Isolation: 300 concurrent commits → 300 distinct interleaved ids. Geometry scaling via `-DSLOTS_PER_SHARD` (bitmap region generalized to multi-page); 512 MB arena verified mlocked. **Exit (met):** measured on a 20-core box (table in the [prototype README](../../../../runtime/README.md)): **908,916 reads/s p99 154 µs and 643,250 fsync-acked commits/s p99 177 µs** on 8 shards — vs Go `net/http` on 20 cores at 495k/355k with ~8× worse p99 and no durability (.NET unavailable on the box); 10k idle connections, 0 errors; only 2xx counted (the client tracks status codes). **The crash-under-load test found two real durability bugs the phase-D test missed** — an ack-armed-before-fsync race in `conn_continue` (route parks the response *during* `try_process`; the pre-check missed it) and an fd-reuse ABA hazard in batch ack-parking (fixed with per-connection generation stamps). After both fixes, three `kill -9`-mid-bench rounds at ~1–2M commits each showed **WAL records ≥ acked, every round** (one exact). Isolation: 300 concurrent commits → 300 distinct interleaved ids. Geometry scaling via `-DSLOTS_PER_SHARD` (bitmap region generalized to multi-page); 512 MB arena verified mlocked.
## Non-scope ## Non-scope
@ -82,7 +82,7 @@ Boot, before any listener opens: each thread replays its own WAL into its arena
- [`../../09-concurrency-scaleout.md`](../../09-concurrency-scaleout.md) — the doctrine; this prototype is its executable proving ground (A↔09a, C↔09 decision 4, D↔09c). - [`../../09-concurrency-scaleout.md`](../../09-concurrency-scaleout.md) — the doctrine; this prototype is its executable proving ground (A↔09a, C↔09 decision 4, D↔09c).
- [`../../10-storage-foundations.md`](../../10-storage-foundations.md), [`11-wal-and-recovery.md`](../../11-wal-and-recovery.md), [`12-engine-disk-cutover.md`](../../12-engine-disk-cutover.md) — the storage track phases B/D/E miniaturize. - [`../../10-storage-foundations.md`](../../10-storage-foundations.md), [`11-wal-and-recovery.md`](../../11-wal-and-recovery.md), [`12-engine-disk-cutover.md`](../../12-engine-disk-cutover.md) — the storage track phases B/D/E miniaturize.
- [`../../../../prototypes/wo-rt-c/README.md`](../../../../prototypes/wo-rt-c/README.md) — current state and module map (phase 0). - [`../../../../runtime/README.md`](../../../../runtime/README.md) — current state and module map (phase 0).
- [`./01-architecture.md`](./01-architecture.md) — the target architecture traced through one memory address at million-connection concurrency, plus improvement proposals (seqlock reads, registered buffers, SEND_ZC, SQPOLL) that slot into phases C/F. - [`./01-architecture.md`](./01-architecture.md) — the target architecture traced through one memory address at million-connection concurrency, plus improvement proposals (seqlock reads, registered buffers, SEND_ZC, SQPOLL) that slot into phases C/F.
- [`./02-single-binary.md`](./02-single-binary.md) — the end goal: how the `wo build` single binary runs on this runtime environment (Go model, not JVM — the kernel is statically linked into every app; the embedding contract between compiler payload and runtime kernel). - [`./02-single-binary.md`](./02-single-binary.md) — the end goal: how the `wo build` single binary runs on this runtime environment (Go model, not JVM — the kernel is statically linked into every app; the embedding contract between compiler payload and runtime kernel).
- [`../../../../prototypes/wo-db/`](../../../../prototypes/wo-db/) — the query-layer sibling; one day a phase-G could splice its engine on top of this runtime. - [`../../../../prototypes/wo-db/`](../../../../prototypes/wo-db/) — the query-layer sibling; one day a phase-G could splice its engine on top of this runtime.

View file

@ -1,6 +1,6 @@
# wo-rt-c architecture — one memory address, two spaces, a million connections # wo-rt-c architecture — one memory address, two spaces, a million connections
This document defines the runtime's architecture by following **one memory address** through user space, kernel space, and hardware, under a million connections reading and writing it concurrently — then suggests improvements. Companion docs: [`00-plan.md`](./00-plan.md) (the phases that build this), [`README.md`](../../../../prototypes/wo-rt-c/README.md) (phase-0 module map). This document defines the runtime's architecture by following **one memory address** through user space, kernel space, and hardware, under a million connections reading and writing it concurrently — then suggests improvements. Companion docs: [`00-plan.md`](./00-plan.md) (the phases that build this), [`README.md`](../../../../runtime/README.md) (phase-0 module map).
## The cast: one address ## The cast: one address

View file

@ -0,0 +1,213 @@
# Colibrì — reference analysis, and running Mistral's MoE models locally
Analysis of the vendored reference tree at [`.dev/reference/colibri/`](../../../../.dev/reference/colibri) (Apache-2.0, upstream <https://github.com/JustVugg/colibri>), and a grounded, hands-on answer to the follow-on question: **what does it take to run Mistral's Mixture-of-Experts models (Mixtral 8x22B / 8x7B) locally?** — including a working demonstration of colibrì's "dense resident, stream the experts from disk" idea using the vendored llama.cpp (§7).
> **TL;DR**
> - Colibrì is a **single-file, zero-dependency C inference engine** that runs a **744B-parameter MoE (GLM-5.2)** on a ~25 GB-RAM consumer box by **streaming routed experts from disk** and treating VRAM/RAM/disk as one managed memory hierarchy. It is here as a *runtime-engineering* reference: it does all its I/O with the exact kernel primitives writeonce's north star is built on (`pread`, `posix_fadvise`, `io_uring`, `mmap`, `mlock`, `O_DIRECT`).
> - Colibrì supports **exactly two model architectures today: GLM-5.2 (`c/glm.c`) and OLMoE (`c/olmoe.c`)**. **There is no Mixtral/Mistral code in the tree** (`grep -ri mixtral` → 0 hits).
> - **To run any Mistral MoE locally right now, don't wait on colibrì** — use a runtime that already supports it. The repo now also vendors a full **llama.cpp** checkout at `.dev/reference/llama-cpp` with **verified, first-class Mistral/Mixtral support** (§6): GGUF + `--n-cpu-moe`. Other options: **KTransformers** (CPU/GPU hybrid, the closest philosophical cousin) or **vLLM/SGLang** on a multi-GPU box. See §5–§6.
> - Mixtral 8x22B is actually a **much easier** target for the colibrì streaming trick than GLM-5.2 — 8 coarse experts/layer instead of 256 fine-grained ones, so the whole int4 expert set (~67 GB) fits in commodity RAM and the disk-streaming stops mattering. A `mixtral.c` port modelled on `olmoe.c` is small and plausible (§4), but it does not exist yet.
> - You can **reproduce and observe** colibrì's core mechanism on a small machine with the [`prototypes/llama-moe-stream/`](../../../../prototypes/llama-moe-stream) demo (§7): run an MoE (default **Qwen3-Coder-30B-A3B**) under a `MemoryMax` cap so the small dense part stays resident while the experts stream from disk on demand — the model still answers correctly on far less RAM than its size. **Gotcha found in practice:** in-circulation Mixtral GGUFs use the pre-2024 per-expert layout and **won't load** on current llama.cpp, so the demo uses a modern fused-format MoE.
---
## 1. What colibrì is
**"Tiny engine, immense model."** Colibrì is a lightweight, quality-preserving Mixture-of-Experts *inference runtime* written in pure C with no external libraries (no BLAS, no Python at runtime, no GPU required). Its thesis:
> A 744B MoE activates only ~40B params per token, and only ~11 GB of those (the *routed experts*) change from token to token. So keep the **dense part resident** and **stream the experts from disk on demand.**
Concretely, for GLM-5.2 at int4:
| Component | Size | Placement |
|---|---|---|
| Dense (attention, shared experts, embeddings — ~17B params) | ~9.9 GB | **resident in RAM** at int4 |
| 19,456 routed experts (75 MoE layers × 256 + MTP head, ~19 MB each) | ~370 GB | **on disk**, streamed on demand |
The engine treats **VRAM → RAM → disk as one memory hierarchy** with a per-layer LRU expert cache, an optional pinned hot-store (the hottest experts stay in spare RAM/VRAM), and the OS page cache as a free L2. Insufficient fast memory reduces *speed*, never *precision or router semantics* — the default policy is lossless.
This is not fast (0.05–2 tok/s depending on disk/RAM/CPU — see the community benchmark table in the upstream README), but it runs a **frontier-class 744B model correctly on hardware that costs less than one H100 fan.**
## 2. Why it lives in `.dev/reference/`
writeonce's north star (see root `CLAUDE.md`, `docs/01-problem.md`, `docs/plan/linux/00-linux.md`) is **one binary, zero external crates, all I/O driven directly by Linux kernel primitives.** Colibrì is a working, production-shaped proof of exactly that discipline in a different domain (ML inference rather than a database):
- **One binary, `libc`-only.** The engine is `c/glm.c` (~348 KB) plus small headers. Python appears *only* in the one-time offline weight converter, never at runtime — the same "transitional tooling is allowed, the runtime is not" line writeonce draws.
- **The kernel *is* the async runtime and the storage tier.** Colibrì's expert streaming is built from the same primitives `crates/rt/src/runtime/` is being built on:
| Primitive | Colibrì use | writeonce analogue |
|---|---|---|
| `pread` | read one expert slab at a known offset | WAL / segment reads |
| `posix_fadvise(WILLNEED/DONTNEED)` | async readahead of the next expert block; evict used slabs | page-cache management |
| `io_uring` (`URING=1`, `c/uring.h`) | batched, queued cold expert reads via `IOSQE_ASYNC` | the target event loop (`docs/plan/02`) |
| `O_DIRECT` (`DIRECT=1`) | bypass page cache for sustained NVMe | direct segment I/O |
| `mmap` (`COLI_MMAP=1`) | map weights instead of `read()` into slabs | `sendfile`/mmap static assets (`docs/plan/08`) |
| `mlock` (`MLOCK=1`) | wire the hot expert cache into physical RAM | pinning hot pages |
It even has a portability story writeonce will need: `c/compat.h` maps every POSIX call to the Win32 API (`pread`→`ReadFile`+`OVERLAPPED`, etc.) so the engine source stays platform-clean.
So colibrì is a reference for **how to engineer a disk/RAM/VRAM memory hierarchy on raw syscalls in one C binary** — read `c/uring.h`, `c/tier.h`, `c/st.h` (the safetensors mmap reader), and `c/compat.h` when designing writeonce's I/O layer. It is *not* a database and shares no code; the value is the technique.
## 3. Use case — who runs colibrì, and when
**Use it when:** you want to run a *very large* open-weight MoE (hundreds of billions of params) **locally, offline, at full quality**, on hardware that cannot hold the model in VRAM (or even in RAM), and you can tolerate low-but-usable token rates. Typical: a single workstation or a homelab NVMe box, privacy-sensitive or air-gapped inference, model-behaviour research, or squeezing a frontier model onto a laptop.
**Don't use it when:** you need interactive throughput on a small model (llama.cpp/Ollama are simpler and faster there), or you have enough VRAM to hold your model outright (use vLLM/SGLang/ExLlamaV2).
**Surface area** (all via the `coli` Python CLI, which just sets env vars and launches the C engine):
| `coli <cmd>` | What it does |
|---|---|
| `convert` | offline FP8→int4 converter; downloads the HF checkpoint one ~5 GB shard at a time so the full 756 GB never lands on disk at once (resumable) |
| `plan` | read-only: reports the dense/expert footprint and the planned VRAM/RAM/disk tiers (`--json`) |
| `doctor` | read-only readiness check (model dir, tokenizer, RAM budget, CUDA linkage, GPU devices) |
| `chat` | interactive REPL |
| `run` | one-shot prompt |
| `serve` | OpenAI-compatible HTTP API (`/v1/chat/completions`, SSE streaming) — stdlib-only gateway (`c/openai_server.py`), one model process, FIFO admission queue |
| `web` | serves the React dashboard in `web/` (live token metrics, hardware panel, the "Brain" expert-heat view) |
| `bench` | MMLU/HellaSwag/ARC quality benchmarks |
Also shipped: a **Tauri desktop shell** (`desktop/`) and a **Nix flake** (`flake.nix`, gcc + OpenMP + gmp; Python env for the converter only).
**Runtime environment colibrì itself needs:**
- **OS:** Linux (or WSL2), macOS, or native Windows 11 (MinGW-w64).
- **CPU:** gcc with OpenMP; AVX2 baseline (`x86-64-v3`), with faster paths on AVX-VNNI (Alder Lake+) and ARM NEON/i8mm/SVE2 (Apple Silicon, Grace). `make ARCH=native` enables the best kernel for the host.
- **GPU (optional):** CUDA backend for NVIDIA (resident/pinned expert tier; on Windows a runtime-loaded `coli_cuda.dll`), Metal backend for Apple Silicon. Both are opt-in accelerators — the CPU path is the reference and stays byte-exact.
- **RAM:** ≥16 GB minimum; more RAM = more experts stay hot = higher tok/s (auto-budgeted from `MemAvailable`).
- **Disk:** the int4 model on a **local** NVMe (ext4/NTFS — never a network/9p mount). Random-read bandwidth is the cold-decode ceiling.
Feature depth worth noting (all in `c/glm.c`): MLA attention with a 57×-compressed KV cache, DeepSeek-V3-style sigmoid router, native **MTP speculative decoding** (int8 draft head), grammar-forced drafts (`GRAMMAR=*.gbnf`), int8/int4/int2 packed quant kernels, DSA sparse attention, crash-safe KV-cache persistence, and cache-aware routing. Every knob is an env var — see [`.dev/reference/colibri/docs/ENVIRONMENT.md`](../../../../.dev/reference/colibri/docs/ENVIRONMENT.md).
## 4. The Mixtral gap — and what a port would take
**Colibrì does not support Mixtral / any Mistral model.** The only architectures implemented are:
- **`c/glm.c`** — GLM-5.2 (`glm_moe_dsa`): 744B, 256 experts/layer top-8, MLA, DSA, MTP. The flagship target.
- **`c/olmoe.c`** — OLMoE-1B-7B (`allenai/OLMoE-1B-7B-0125-Instruct`): 7B total / 1B active, 64 experts/layer top-8. Its header states its purpose plainly: *"validate the streaming core before scaling to GLM-5.2."* **This is the template for adding a new architecture.**
Adding Mixtral would mean writing the same two pieces OLMoE has:
1. **`c/mixtral.c`** — a faithful forward pass. Good news: Mixtral is *architecturally simpler* than either existing engine — plain GQA + RoPE attention (no MLA, no DSA, no q/k-norm), RMSNorm, SwiGLU experts, no shared expert, no MTP head. It is closer to `olmoe.c` than to `glm.c`, and smaller.
2. **`c/tools/convert_mixtral.py`** — modelled on `convert_olmoe.py`: keep dense weights as f16/f32, row-wise-quantize the expert matrices to the int8/int4 container. Only the expert-key regex changes — Mixtral names them `model.layers.{L}.block_sparse_moe.experts.{E}.(w1|w2|w3).weight` and the router is `block_sparse_moe.gate`.
**Why Mixtral is an *easier* streaming target than GLM-5.2** (int4, from its config — 56 layers, hidden 6144, intermediate 16384, 8 experts/layer, top-2):
- Each expert = 3 matrices of 6144×16384 ≈ 302M params → **~151 MB at int4** (vs GLM's 19 MB fine-grained experts).
- Total experts = 8 × 56 = **448 experts ≈ 67 GB at int4** (vs GLM's 19,456 experts ≈ 370 GB).
- Cold cost/token = top-2 × 56 = **112 expert-loads ≈ 17 GB/token** — but with only 8 experts/layer, **any 96 GB+ machine caches the entire expert set in RAM**, giving ~100 % hit rate and *zero* disk streaming after warmup. The engine becomes RAM-bandwidth / matmul bound, not disk bound.
In other words, the whole "stream from disk" apparatus that colibrì needs for GLM-5.2 is mostly *unnecessary* for Mixtral 8x22B — the model is small enough (at int4) to just live in RAM. That is exactly why the practical answer below does not require colibrì at all.
## 5. Running Mixtral 8x22B locally — the ready paths
### 5.1 The model
| Config (`Mixtral-8x22B-v0.1`) | Value |
|---|---|
| Total / active params | ~141B / ~39B |
| Layers | 56 |
| hidden_size | 6144 |
| intermediate_size (per expert) | 16384 |
| attention heads / KV heads (GQA) | 48 / 8 (head_dim 128) |
| experts / top-k | 8 / 2 |
| vocab | 32768 |
| rope_theta / context | 1,000,000 / 65,536 |
Approximate on-disk sizes (GGUF): **FP16 ≈ 281 GB · Q8_0 ≈ 149 GB · Q5_K_M ≈ 100 GB · Q4_K_M ≈ 86 GB · Q3_K ≈ 65 GB · Q2_K ≈ 52 GB.** For decent-quality local use, **Q4_K_M (~86 GB) or Q5** is the sweet spot; Q2/Q3 fit smaller boxes with quality loss.
### 5.2 Runtime options, from most-consumer to most-datacenter
| Runtime | How it runs Mixtral 8x22B locally | Hardware reality | Closeness to colibrì |
|---|---|---|---|
| **Ollama** | `ollama run mixtral:8x22b` (wraps llama.cpp, pulls a Q4 GGUF) | ~90 GB RAM for Q4 CPU-only, or GPU+CPU split | Same tiering idea, turnkey |
| **llama.cpp (GGUF)** | Load a Q4/Q5 GGUF; offload expert layers to CPU RAM and keep attention/dense on GPU with **`--n-cpu-moe N`** (or `-ot`/`--override-tensor` regex for per-tensor control) | Runs CPU-only with ~90 GB RAM, *or* a 16–24 GB GPU + system RAM hybrid | **Closest mainstream analog** — same "experts in slow memory, dense on fast" split colibrì automates |
| **KTransformers** | CPU/GPU **hybrid MoE** — attention + shared/hot experts on GPU, the parameter-heavy routed experts in system RAM with AMX/AVX-512 CPU kernels. Explicitly lists **Mixtral 8x7B and 8x22B** as supported. | One consumer GPU + a big-RAM host; higher throughput than llama.cpp on large MoE | **Philosophically closest** — it is colibrì's heterogeneous-tiering idea as a Python/CUDA framework |
| **vLLM / SGLang** | GPU-native, high-throughput serving (AWQ/GPTQ 4-bit or FP16) | Realistically **2× A100-80GB** (4-bit) to 4–8× for FP16 — a local *server*, not a desktop | Different niche (VRAM-resident, batch throughput) |
| **ExLlamaV2 (EXL2)** | 4-bit EXL2 quant, GPU-only | ~4× 24 GB consumer GPUs for a low-bpw quant | GPU-resident, no disk tier |
| **LM Studio / text-generation-webui** | Desktop front-ends over llama.cpp/GGUF | Same as llama.cpp | GUI convenience layer |
### 5.3 Recommendation
- **Single consumer/workstation box (one GPU + 64–128 GB RAM):** **llama.cpp or Ollama** with a **Q4_K_M GGUF** and **`--n-cpu-moe`** to push experts into RAM while attention stays on the GPU. Simplest and proven. If you have AMX/AVX-512 and want more speed on the same hardware, try **KTransformers** — it is the closest thing to "colibrì for Mixtral" that exists today.
- **Local multi-GPU server:** **vLLM or SGLang** with a 4-bit quant for real throughput.
- **If you specifically want the colibrì engine to run it:** that requires writing `c/mixtral.c` + `c/tools/convert_mixtral.py` against the `c/olmoe.c` template (§4). Feasible and not large, but it is net-new work — and because Mixtral's int4 expert set fits in RAM, it would buy little over the paths above except staying inside the pure-C, zero-dep runtime that makes colibrì interesting to writeonce in the first place.
## 6. Verified: `.dev/reference/llama-cpp` already runs Mistral/Mixtral
The repo also vendors a full, recent **llama.cpp** checkout at `.dev/reference/llama-cpp` (a symlink to a local clone; HEAD `635cdd5fc`). Unlike colibrì, it has **first-class Mistral/Mixtral support**, confirmed across the whole stack:
- **Architecture** (`src/llama-arch.{h,cpp}`): Mistral 7B and Mixtral 8x7B/8x22B load under `LLM_ARCH_LLAMA` — llama-arch MoE, driven by the `expert_count` / `expert_used_count` GGUF keys. Dedicated `LLM_ARCH_MISTRAL3` / `LLM_ARCH_MISTRAL4` cover the newer Mistral Small / Mistral 4 families; Pixtral / Mistral-Small-3.1 handle the vision variants.
- **Conversion** (`conversion/` package — the refactored `convert_hf_to_gguf.py`): registers `MistralForCausalLM` / `MixtralForCausalLM` (→ llama arch), plus dedicated `MistralModel`, `MistralMoeModel` (remapped onto DeepSeek-V2), `Mistral3Model`, `Ministral3Model`, `Mistral4Model`, `PixtralModel`.
- **Tokenizer + chat templates**: native `mistral-common` (Tekken / SentencePiece) tokenizers, a `TEKKEN` pre-type, and five built-in templates — `mistral-v1`, `mistral-v3`, `mistral-v3-tekken`, `mistral-v7`, `mistral-v7-tekken` (`src/llama-chat.cpp`).
- **MoE-offload flags** (`common/arg.cpp`): `-cmoe`/`--cpu-moe` and `-ncmoe N`/`--n-cpu-moe N` — the colibrì-style "experts on the slow tier, dense on the fast tier" split, built in (with `--n-cpu-moe-draft` variants for speculative decoding).
So on this repo the runnable path for any Mistral MoE is **llama.cpp**, not colibrì. Of the two vendored inference references: **colibrì = GLM-5.2 + OLMoE only; llama.cpp = full Mistral/Mixtral.**
## 7. Hands-on: understand MoE experts, and stream them from disk
The demo lives at [`prototypes/llama-moe-stream/`](../../../../prototypes/llama-moe-stream) (`run-moe.sh` + a teaching README). It runs an MoE and *forces* the streaming behavior with a RAM cap so the mechanism is observable — the same idea colibrì applies to GLM-5.2.
> **Format-wall gotcha (found the hard way).** The demo originally targeted Mixtral 8x7B, but **every in-circulation Mixtral GGUF (TheBloke Dec-2023, MaziyarPanahi Feb-2024) uses the pre-2024 *per-expert* tensor layout** (`blk.0.ffn_gate.0.weight` … `.7.weight`). Current llama.cpp (HEAD `635cdd5fc`) only loads the **fused** layout (`blk.0.ffn_gate_exps.weight`) and dies with `missing tensor 'blk.0.ffn_down_exps.weight'`. Re-downloading another old quant does not help. So the demo defaults to **Qwen3-Coder-30B-A3B-Instruct** — a modern MoE whose GGUF is fused-format (verified), and which doubles as a capable local coding model. The Mixtral analysis in §1–§6 stands; only the *runnable demo* switched models.
### 7.1 What a "Mixture of Experts" is (the concept)
A **dense** transformer runs every weight for every token. An **MoE** replaces each layer's feed-forward block with **N expert FFNs + a small router**; per token the router routes through only the **top-k** experts, and the rest stay idle. That splits the weights into two classes — and the split is the whole point:
| | what it is | touched per token? | share of the weights |
|---|---|---|---|
| **Dense part** | attention, embeddings, norms, the routers | **always** — every token, every layer | small → keep **resident** |
| **Experts** (routed) | the N expert FFNs in each layer | **only top-k of N** | the bulk → **stream from disk** |
**Qwen3-Coder-30B-A3B** (the demo model): 30B total but only **~3.3B active per token** — the router fires a small top-k of many experts each layer. **Mixtral 8x7B** is the same idea at 46.7B total / ~12.9B active (32 layers, 8 experts/layer, top-2; the name misleads — experts share one attention stack, so it is 46.7B not 56B), and **Mixtral 8x22B** at 141B / ~39B active.
**Why this enables streaming:** the dense part is small and hit constantly → keep it **resident** in fast memory. The experts are the majority of the bytes but each is hit rarely → they can live **on disk** and be pulled in exactly when routed to. A *dense* model of the same size could not do this (all of it every token); an MoE reads only the slice it routes to. That is colibrì's thesis.
### 7.2 Realizing "dense resident, experts streamed" with llama.cpp
- **mmap (on by default)** memory-maps the GGUF; the kernel demand-pages weights and evicts under pressure, backed by the file. This is the streaming engine, for free. **Never `--no-mmap`** on a >RAM model — it forces a full allocation and thrashes.
- **`--cpu-moe`** keeps expert tensors on the CPU/mmap (disk-backed) side. On a **CUDA** build you pair it with `-ngl` to put the dense part in the GPU (resident) while experts stream on the CPU — the textbook split. The vendored build is **CPU-only** (no CUDA backend compiled), so `-ngl`/`--cpu-moe` are GPU no-ops; the split is instead realized by a RAM cap.
- **`MemoryMax` (cgroup v2)** — `systemd-run --user --scope -p MemoryMax=6G` caps the process below the model size. The kernel then keeps the small dense part + hot experts resident and evicts cold expert pages, re-reading them from disk on demand. This turns the OS page cache into colibrì's tiering (hot resident, cold on disk); colibrì just makes it *smart* — per-layer LRU, `fadvise` readahead, pinning the measured-hottest experts.
### 7.3 Run it
```bash
cd prototypes/llama-moe-stream
./run-moe.sh # baseline: 17 GB model fits in RAM → all resident
MEM_CAP=6G ./run-moe.sh # capped: dense stays hot, cold experts stream from disk
```
The proof: under a 6 GB cap the 17 GB model **still answers correctly** — the missing experts are served from disk on demand — and tok/s drops vs the baseline; that gap is the disk-streaming cost. `-hf` downloads + caches the GGUF (`~/.cache/llama.cpp`) on first run.
### 7.4 Privacy — does local inference leak your data?
**No.** llama.cpp inference is fully on-device: it reads a local GGUF, has **no telemetry**, and opens **no outbound connections** while generating — prompts and code never leave the machine (weights are inert data, not code that can "phone home"). The *only* network is the one-time `-hf` weight download (inbound; HuggingFace sees your IP + which file, not your data). To be certain, run air-gapped from the cached file:
```bash
GGUF=$(find ~/.cache/llama.cpp -name 'Qwen3-Coder-30B-A3B-Instruct-Q4_K_M.gguf' | head -1)
MODEL_PATH="$GGUF" OFFLINE=1 MEM_CAP=6G ./run-moe.sh # HF_HUB_OFFLINE=1, no -hf, zero network
```
`ss -tnp` during the run shows **no established connections** from `llama-cli`. The real leak surface is the *client* (an editor plugin misconfigured to a cloud model, or plugin telemetry) — not the engine.
### 7.5 Measured on the dev box
Host: i7-13700H (20 threads, AVX2+VNNI), **31 GB RAM**, RTX 4050 Laptop (6 GB), NVMe; vendored llama.cpp is a **CPU-only** build. Model: `unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF:Q4_K_M` (~17.3 GB, single file, fused-expert layout).
| Run | RAM available to process | Expected behavior | Measured tok/s |
|---|---|---|---|
| baseline (uncapped) | whole 17 GB can stay resident | RAM/matmul-bound after warm-up | _pending run_ |
| `MEM_CAP=6G` + offline | 6 GB — dense + hot experts only | cold experts stream from NVMe; **no network** | _pending run_ |
_Numbers are filled in from the in-progress background run; the qualitative result — correct output under a cap far below model size, with zero outbound connections — is the point regardless of the exact rate._
---
## Sources
- `.dev/reference/colibri/` — vendored source (README, `docs/ENVIRONMENT.md`, `c/glm.c`, `c/olmoe.c`, `c/tools/convert_olmoe.py`, `c/uring.h`, `c/compat.h`, `flake.nix`), upstream <https://github.com/JustVugg/colibri>
- `.dev/reference/llama-cpp/` — vendored llama.cpp checkout (HEAD `635cdd5fc`), Mistral support verified in `src/llama-arch.{h,cpp}`, `conversion/{llama,mistral,mistral3,pixtral}.py`, `src/llama-chat.cpp`, `common/arg.cpp`
- `prototypes/llama-moe-stream/` — the hands-on demo (`run-moe.sh`, README) added by this work
- Model GGUFs: [`unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF`](https://huggingface.co/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF) (fused-format, the demo default); old per-expert-layout examples that **fail** to load on current llama.cpp: [`TheBloke/Mixtral-8x7B-Instruct-v0.1-GGUF`](https://huggingface.co/TheBloke/Mixtral-8x7B-Instruct-v0.1-GGUF), [`MaziyarPanahi/Mixtral-8x22B-Instruct-v0.1-GGUF`](https://huggingface.co/MaziyarPanahi/Mixtral-8x22B-Instruct-v0.1-GGUF)
- [Mixtral 8x22B — Prompt Engineering Guide](https://www.promptingguide.ai/models/mixtral-8x22b) and [Ollama library: mixtral:8x22b](https://ollama.com/library/mixtral:8x22b)
- [Performant local MoE CPU inference with GPU acceleration in llama.cpp](https://huggingface.co/blog/Doctor-Shotgun/llamacpp-moe-offload-guide) (the `--n-cpu-moe` / `--override-tensor` guide)
- [KTransformers](https://github.com/kvcache-ai/ktransformers) — CPU/GPU hybrid MoE inference (lists Mixtral 8x7B/8x22B support)

View file

@ -0,0 +1,96 @@
# Fibers — green threads on the wovm shard scheduler
> Research note (2026-08-08) feeding story iteration 11. Expands
> [the blue-green vision §3](../blue-green-vm/00-vision.md) with the
> kernel's-eye evidence and the precedent survey. Prose only; the spec and
> plan follow the brainstorming → writing-plans path when the iteration
> starts.
## Why the kernel cannot do this for us
Reference: [Threads and the OS kernel's view](https://learn.padho.ai/wiki/threads-and-the-os-kernels-view).
The facts that matter, condensed:
- A Linux thread IS a `task_struct`: own TID, own kernel stack (8–16 KiB),
own scheduling slot; `clone()` flags decide what is shared. There is no
cheaper kernel thread to ask for.
- Costs per thread: ~8 MiB stack VMA, ~9 KiB `task_struct`,
~20 µs creation, 1–3 µs per context switch. Measured against a userspace
runtime spawn (~58 ns): **~342× creation cost, ~70× memory**.
- CFS keys a red-black tree by `vruntime` — O(log N) per pick. At tens of
thousands of runnable tasks the scheduling slice approaches the
context-switch cost and the kernel scheduler becomes the bottleneck
before application code does.
- M:N threading died *in the kernel* (NPTL, 2003) and was reborn in
userspace (goroutines, BEAM processes, Tokio tasks, Loom virtual
threads) — because only the runtime knows its own blocking points and
can keep per-task state tiny.
Conclusion the industry already reached and we adopt: **one kernel task
per core** (plan 09 / iteration 8's pinned shard — already doctrine),
**userspace tasks above it**. The kernel schedules cores; the runtime
schedules work.
## Precedent survey — who has green threads, and how
| Runtime | Task state | Preemption | I/O integration | Lesson for wovm |
| --- | --- | --- | --- | --- |
| **Erlang/BEAM** | interpreter state per process, private heap | **reduction budget** (~2k reductions, checked at calls) | park on scheduler, poll set | the closest shape: interpreter = scheduler; deterministic, signal-free preemption |
| **Go** | native stack, 8 KiB grown by copy | async preemption via signals (since 1.14) + safepoints | netpoller parks goroutines | stack copying needs precise pointer maps — heavy machinery; signals are what the vision explicitly avoids |
| **Java Loom** | continuation frames on heap, unmounted from carrier | cooperative at yield points | blocking calls in the JDK park the virtual thread | "same blocking API, runtime parks underneath" — exactly our stdlib posture |
| **Tokio/Rust** | stackless state machines (`async fn`) | cooperative at `.await` | reactor + waker | rejected surface: writeonce has **no async/await keyword** (systems-track spec Part 2); function coloring is the disease |
| **Lua** | coroutine = own Lua stack (interpreter state) | none (pure cooperative) | up to the host | proof that interpreter-state fibers are nearly free; but no preemption = one hot loop starves the shard |
| **boost.context / libco** | native stack + hand-rolled register switch | none | none | what we do NOT need: wovm executes bytecode, so no native stack switching, no asm, no guard pages |
## The wovm design (vision §3, confirmed by this survey)
A fiber is **the execution state the VM already isolates**: register-window
stack + frame stack + pc. Making that per-fiber instead of per-VM turns the
interpreter into a scheduler almost for free — the BEAM/Lua insight, minus
Lua's starvation problem:
- **Preemption by reduction budget** — the dispatch loop decrements a
counter per instruction (or per call/back-edge); at zero the fiber parks
and the next runnable one is picked. No signals, no safepoint asm, no
stack copying; deterministic and debuggable. (Erlang has run this design
in telecom production for three decades.)
- **Park on I/O** — a blocking stdlib builtin on a server shard hands its
fd to the shard's epoll/io_uring loop and parks the fiber; the completion
resumes it. Same typed builtins, blocking look, no coloring — the
systems-track "one API, two execution disciplines" doctrine gains its
third discipline: program mode blocks the thread, server shards park the
fiber, the source text is identical.
- **Ownership fits** — a fiber owns its objects like a shard owns its heap;
cross-fiber sends move ownership exactly like iteration 8's cross-shard
sends (same rule, cheaper path: same heap, no copy). `@gc` references
stay shard-local either way, so the per-shard cycle collector (staged to
iteration 8) needs fiber stacks as additional roots — the one real
collector interaction to spec.
- **Cost target** — fiber creation is an arena allocation of a small
context (hundreds of bytes, the ~58 ns class), park/resume is a pointer
swap in the dispatch loop; thousands of fibers per shard where the
kernel tops out at hundreds of threads per core.
## What must be specified before implementation (open questions)
1. Surface: `spawn` returns what — a fiber handle, an actor address, or
nothing (fire-and-forget)? Iteration 8's `spawn` and mailbox surface
should be the same word; fibers refine its granularity.
2. Reduction budget size and where it is checked (per instruction vs per
call/back-edge) — measure both in the interpreter before choosing.
3. Parked-fiber lifetime: what drops a fiber blocked forever (shard
shutdown, blue-green drain)? Unwinding a parked fiber must run its drop
maps — same machinery as trap unwind (OOP spec §6).
4. Fairness: run queue is FIFO per shard in v1; priorities/timers are a
later capability (the recipe-box rule — no framework policy in the
runtime).
5. Program mode: stays fiber-free in v1 (one thread, blocking legal) or
gains the same scheduler? Default: fiber-free — log-watcher needs none.
## Doctrine check
Kernel primitives only (epoll/io_uring already ours; no new syscalls
needed) · no signals · no async/await keyword · ownership moves, never
shares · per-shard everything (heap, GC, now run queue) · plain
diagnostics (a starved-fiber warning names the hot function via the line
table). No principle bends.

View file

@ -4,7 +4,7 @@ Kernel primitives that the writeonce binary can leverage, mapped to the architec
### Per-primitive reference cards ### Per-primitive reference cards
Each primitive has its own numbered file with the kernel source path (into [`reference/linux/`](../../../reference/linux/)), Rust FFI signature via `libc`, a minimal direct-syscall example, and the v1 port source. Use these when implementing the phase docs under [`docs/plan/`](../). Each primitive has its own numbered file with the kernel source path (into [`.dev/reference/linux/`](../../../.dev/reference/linux/)), Rust FFI signature via `libc`, a minimal direct-syscall example, and the v1 port source. Use these when implementing the phase docs under [`docs/plan/`](../).
| # | Primitive | Used by | | # | Primitive | Used by |
| --- | --- | --- | | --- | --- | --- |
@ -143,4 +143,4 @@ Each pattern resolves to a set of inotify watch descriptors. When the watched se
## Related: the assembly policy ## Related: the assembly policy
Every primitive above is reached via `libc::<syscall>` or `libc::syscall(SYS_*, ...)` — no custom assembly. The reasoning lives in [`../assembly/`](../assembly/) — three files covering why runtimes use asm at all ([`00-overview.md`](../assembly/00-overview.md)), what Go's [`reference/go/src/runtime/*.s`](../../../reference/go/src/runtime/) actually contains ([`01-go-runtime-asm.md`](../assembly/01-go-runtime-asm.md)), and the writeonce policy that all of it is replaced by Rust stdlib + libc ([`02-writeonce-stance.md`](../assembly/02-writeonce-stance.md)). Every primitive above is reached via `libc::<syscall>` or `libc::syscall(SYS_*, ...)` — no custom assembly. The reasoning lives in [`../assembly/`](../assembly/) — three files covering why runtimes use asm at all ([`00-overview.md`](../assembly/00-overview.md)), what Go's [`.dev/reference/go/src/runtime/*.s`](../../../.dev/reference/go/src/runtime/) actually contains ([`01-go-runtime-asm.md`](../assembly/01-go-runtime-asm.md)), and the writeonce policy that all of it is replaced by Rust stdlib + libc ([`02-writeonce-stance.md`](../assembly/02-writeonce-stance.md)).

View file

@ -6,8 +6,8 @@ Event-driven I/O multiplexing. One `epoll_fd` watches many fds for readiness; `e
| Path | What | | Path | What |
| --- | --- | | --- | --- |
| [`reference/linux/fs/eventpoll.c`](../../../reference/linux/fs/eventpoll.c) | All three syscalls (`epoll_create1`, `epoll_ctl`, `epoll_wait`) live here. Grep for `SYSCALL_DEFINE`. | | [`.dev/reference/linux/fs/eventpoll.c`](../../../.dev/reference/linux/fs/eventpoll.c) | All three syscalls (`epoll_create1`, `epoll_ctl`, `epoll_wait`) live here. Grep for `SYSCALL_DEFINE`. |
| [`reference/linux/include/uapi/linux/eventpoll.h`](../../../reference/linux/include/uapi/linux/eventpoll.h) | `struct epoll_event`, `EPOLL_*` flags, the userspace-facing ABI. | | [`.dev/reference/linux/include/uapi/linux/eventpoll.h`](../../../.dev/reference/linux/include/uapi/linux/eventpoll.h) | `struct epoll_event`, `EPOLL_*` flags, the userspace-facing ABI. |
## Man pages ## Man pages
@ -72,4 +72,4 @@ Every runtime phase that touches I/O: [`02-event-loop-epoll.md`](../02-event-loo
## v1 port source ## v1 port source
[`reference/crates/wo-event/src/epoll.rs`](../../../reference/crates/wo-event/src/epoll.rs) (183 LOC) — already wraps all three syscalls with a safe `EventLoop { register, deregister, wait_once }` facade. [`.dev/reference/crates/wo-event/src/epoll.rs`](../../../.dev/reference/crates/wo-event/src/epoll.rs) (183 LOC) — already wraps all three syscalls with a safe `EventLoop { register, deregister, wait_once }` facade.

View file

@ -6,8 +6,8 @@ Counter as a file descriptor. `write(fd, &n, 8)` adds `n` to the counter; `read(
| Path | What | | Path | What |
| --- | --- | | --- | --- |
| [`reference/linux/fs/eventfd.c`](../../../reference/linux/fs/eventfd.c) | `SYSCALL_DEFINE2(eventfd, ...)`, `struct eventfd_ctx`, read/write handlers. | | [`.dev/reference/linux/fs/eventfd.c`](../../../.dev/reference/linux/fs/eventfd.c) | `SYSCALL_DEFINE2(eventfd, ...)`, `struct eventfd_ctx`, read/write handlers. |
| [`reference/linux/include/uapi/linux/eventfd.h`](../../../reference/linux/include/uapi/linux/eventfd.h) | `EFD_*` flags. | | [`.dev/reference/linux/include/uapi/linux/eventfd.h`](../../../.dev/reference/linux/include/uapi/linux/eventfd.h) | `EFD_*` flags. |
## Man pages ## Man pages
@ -63,4 +63,4 @@ unsafe {
## v1 port source ## v1 port source
[`reference/crates/wo-event/src/eventfd.rs`](../../../reference/crates/wo-event/src/eventfd.rs) (66 LOC) — `EventFd { new, write, read, as_raw_fd }`. [`.dev/reference/crates/wo-event/src/eventfd.rs`](../../../.dev/reference/crates/wo-event/src/eventfd.rs) (66 LOC) — `EventFd { new, write, read, as_raw_fd }`.

View file

@ -6,8 +6,8 @@ Timers as file descriptors. Set an expiry with `timerfd_settime`, `read` the fd
| Path | What | | Path | What |
| --- | --- | | --- | --- |
| [`reference/linux/fs/timerfd.c`](../../../reference/linux/fs/timerfd.c) | All three syscalls (`timerfd_create`, `timerfd_settime`, `timerfd_gettime`). | | [`.dev/reference/linux/fs/timerfd.c`](../../../.dev/reference/linux/fs/timerfd.c) | All three syscalls (`timerfd_create`, `timerfd_settime`, `timerfd_gettime`). |
| [`reference/linux/include/uapi/linux/timerfd.h`](../../../reference/linux/include/uapi/linux/timerfd.h) | `TFD_*` flags. | | [`.dev/reference/linux/include/uapi/linux/timerfd.h`](../../../.dev/reference/linux/include/uapi/linux/timerfd.h) | `TFD_*` flags. |
## Man pages ## Man pages
@ -74,4 +74,4 @@ unsafe {
## v1 port source ## v1 port source
[`reference/crates/wo-event/src/timerfd.rs`](../../../reference/crates/wo-event/src/timerfd.rs) (91 LOC) — `TimerFd { oneshot(dur), periodic(dur), disarm, read_expirations }`. [`.dev/reference/crates/wo-event/src/timerfd.rs`](../../../.dev/reference/crates/wo-event/src/timerfd.rs) (91 LOC) — `TimerFd { oneshot(dur), periodic(dur), disarm, read_expirations }`.

View file

@ -6,9 +6,9 @@ Unix signals as file descriptors. `signalfd(fd, mask)` installs a mask on the pr
| Path | What | | Path | What |
| --- | --- | | --- | --- |
| [`reference/linux/fs/signalfd.c`](../../../reference/linux/fs/signalfd.c) | `SYSCALL_DEFINE4(signalfd4, ...)` + `signalfd_dequeue`. | | [`.dev/reference/linux/fs/signalfd.c`](../../../.dev/reference/linux/fs/signalfd.c) | `SYSCALL_DEFINE4(signalfd4, ...)` + `signalfd_dequeue`. |
| [`reference/linux/include/uapi/linux/signalfd.h`](../../../reference/linux/include/uapi/linux/signalfd.h) | `struct signalfd_siginfo`, `SFD_*` flags. | | [`.dev/reference/linux/include/uapi/linux/signalfd.h`](../../../.dev/reference/linux/include/uapi/linux/signalfd.h) | `struct signalfd_siginfo`, `SFD_*` flags. |
| [`reference/linux/kernel/signal.c`](../../../reference/linux/kernel/signal.c) | Background: `sigprocmask`, pending-signal dequeue. | | [`.dev/reference/linux/kernel/signal.c`](../../../.dev/reference/linux/kernel/signal.c) | Background: `sigprocmask`, pending-signal dequeue. |
## Man pages ## Man pages
@ -74,4 +74,4 @@ unsafe {
## v1 port source ## v1 port source
[`reference/crates/wo-event/src/signalfd.rs`](../../../reference/crates/wo-event/src/signalfd.rs) (62 LOC) — `SignalFd::new(&[SIGINT, SIGTERM]) -> SignalFd` with a safe `read_signo()` helper. [`.dev/reference/crates/wo-event/src/signalfd.rs`](../../../.dev/reference/crates/wo-event/src/signalfd.rs) (62 LOC) — `SignalFd::new(&[SIGINT, SIGTERM]) -> SignalFd` with a safe `read_signo()` helper.

View file

@ -6,9 +6,9 @@ Filesystem event notifications as a file descriptor. `inotify_add_watch(dir, mas
| Path | What | | Path | What |
| --- | --- | | --- | --- |
| [`reference/linux/fs/notify/inotify/inotify_user.c`](../../../reference/linux/fs/notify/inotify/inotify_user.c) | `SYSCALL_DEFINE1(inotify_init1, ...)`, `SYSCALL_DEFINE3(inotify_add_watch, ...)`, `SYSCALL_DEFINE2(inotify_rm_watch, ...)`. | | [`.dev/reference/linux/fs/notify/inotify/inotify_user.c`](../../../.dev/reference/linux/fs/notify/inotify/inotify_user.c) | `SYSCALL_DEFINE1(inotify_init1, ...)`, `SYSCALL_DEFINE3(inotify_add_watch, ...)`, `SYSCALL_DEFINE2(inotify_rm_watch, ...)`. |
| [`reference/linux/fs/notify/inotify/inotify_fsnotify.c`](../../../reference/linux/fs/notify/inotify/inotify_fsnotify.c) | The fsnotify backend that feeds events into the fd. | | [`.dev/reference/linux/fs/notify/inotify/inotify_fsnotify.c`](../../../.dev/reference/linux/fs/notify/inotify/inotify_fsnotify.c) | The fsnotify backend that feeds events into the fd. |
| [`reference/linux/include/uapi/linux/inotify.h`](../../../reference/linux/include/uapi/linux/inotify.h) | `struct inotify_event`, `IN_*` masks. | | [`.dev/reference/linux/include/uapi/linux/inotify.h`](../../../.dev/reference/linux/include/uapi/linux/inotify.h) | `struct inotify_event`, `IN_*` masks. |
## Man pages ## Man pages
@ -85,4 +85,4 @@ unsafe {
## v1 port source ## v1 port source
[`reference/crates/wo-watch/src/lib.rs`](../../../reference/crates/wo-watch/src/lib.rs) (280 LOC) — already does recursive watch setup, event parsing, and path resolution via a `wd → PathBuf` map. [`.dev/reference/crates/wo-watch/src/lib.rs`](../../../.dev/reference/crates/wo-watch/src/lib.rs) (280 LOC) — already does recursive watch setup, event parsing, and path resolution via a `wd → PathBuf` map.

View file

@ -6,8 +6,8 @@ Zero-copy transfer from a file fd to a socket fd. The kernel splices pages direc
| Path | What | | Path | What |
| --- | --- | | --- | --- |
| [`reference/linux/fs/read_write.c`](../../../reference/linux/fs/read_write.c) | `SYSCALL_DEFINE4(sendfile, ...)` and `SYSCALL_DEFINE4(sendfile64, ...)`. Modern glibc aliases the first to the second; the syscalls are distinguished by the offset type. | | [`.dev/reference/linux/fs/read_write.c`](../../../.dev/reference/linux/fs/read_write.c) | `SYSCALL_DEFINE4(sendfile, ...)` and `SYSCALL_DEFINE4(sendfile64, ...)`. Modern glibc aliases the first to the second; the syscalls are distinguished by the offset type. |
| [`reference/linux/fs/splice.c`](../../../reference/linux/fs/splice.c) | Internally `sendfile` delegates to `splice_direct_to_actor`. Related — see [07-splice.md](./07-splice.md) if you ever need the more general fd-to-fd pipe path. | | [`.dev/reference/linux/fs/splice.c`](../../../.dev/reference/linux/fs/splice.c) | Internally `sendfile` delegates to `splice_direct_to_actor`. Related — see [07-splice.md](./07-splice.md) if you ever need the more general fd-to-fd pipe path. |
## Man pages ## Man pages
@ -75,4 +75,4 @@ unsafe {
## v1 port source ## v1 port source
[`reference/crates/wo-serve/src/sendfile.rs`](../../../reference/crates/wo-serve/src/sendfile.rs) (109 LOC) — `send_file(sock, path) -> Result` wrapping the loop + `EAGAIN` handling. [`.dev/reference/crates/wo-serve/src/sendfile.rs`](../../../.dev/reference/crates/wo-serve/src/sendfile.rs) (109 LOC) — `send_file(sock, path) -> Result` wrapping the loop + `EAGAIN` handling.

View file

@ -8,9 +8,9 @@ Ring-buffer based async I/O (Linux 5.1+, mature 5.11+). Two lock-free SPSC rings
| Path | What | | Path | What |
| --- | --- | | --- | --- |
| [`reference/linux/io_uring/`](../../../reference/linux/io_uring/) | Whole subsystem. Start with `io_uring.c` (ring setup + submission/completion) and `fs.c` (fsync op). | | [`.dev/reference/linux/io_uring/`](../../../.dev/reference/linux/io_uring/) | Whole subsystem. Start with `io_uring.c` (ring setup + submission/completion) and `fs.c` (fsync op). |
| [`reference/linux/io_uring/io_uring.c`](../../../reference/linux/io_uring/io_uring.c) | `SYSCALL_DEFINE2(io_uring_setup, ...)`, `SYSCALL_DEFINE6(io_uring_enter, ...)`, `SYSCALL_DEFINE4(io_uring_register, ...)`. | | [`.dev/reference/linux/io_uring/io_uring.c`](../../../.dev/reference/linux/io_uring/io_uring.c) | `SYSCALL_DEFINE2(io_uring_setup, ...)`, `SYSCALL_DEFINE6(io_uring_enter, ...)`, `SYSCALL_DEFINE4(io_uring_register, ...)`. |
| [`reference/linux/include/uapi/linux/io_uring.h`](../../../reference/linux/include/uapi/linux/io_uring.h) | `struct io_uring_sqe`, `io_uring_cqe`, `io_uring_params`, every `IORING_*` flag. | | [`.dev/reference/linux/include/uapi/linux/io_uring.h`](../../../.dev/reference/linux/include/uapi/linux/io_uring.h) | `struct io_uring_sqe`, `io_uring_cqe`, `io_uring_params`, every `IORING_*` flag. |
## Man pages ## Man pages

View file

@ -8,9 +8,9 @@ Central to Phase 3's storage engine: segment files are `mmap`ed read-only for O(
| Path | What | | Path | What |
| --- | --- | | --- | --- |
| [`reference/linux/mm/mmap.c`](../../../reference/linux/mm/mmap.c) | VMA creation, `SYSCALL_DEFINE6(mmap, ...)`, `SYSCALL_DEFINE2(munmap, ...)`. | | [`.dev/reference/linux/mm/mmap.c`](../../../.dev/reference/linux/mm/mmap.c) | VMA creation, `SYSCALL_DEFINE6(mmap, ...)`, `SYSCALL_DEFINE2(munmap, ...)`. |
| [`reference/linux/mm/madvise.c`](../../../reference/linux/mm/madvise.c) | `SYSCALL_DEFINE3(madvise, ...)` + every `MADV_*` handler. | | [`.dev/reference/linux/mm/madvise.c`](../../../.dev/reference/linux/mm/madvise.c) | `SYSCALL_DEFINE3(madvise, ...)` + every `MADV_*` handler. |
| [`reference/linux/include/uapi/linux/mman.h`](../../../reference/linux/include/uapi/linux/mman.h) | `MAP_*` flags, huge-page sizing macros. | | [`.dev/reference/linux/include/uapi/linux/mman.h`](../../../.dev/reference/linux/include/uapi/linux/mman.h) | `MAP_*` flags, huge-page sizing macros. |
| POSIX `<sys/mman.h>` | The other half of the constants (`PROT_*`, `MADV_*`). Usually folded into `linux/mman.h` by libc. | | POSIX `<sys/mman.h>` | The other half of the constants (`PROT_*`, `MADV_*`). Usually folded into `linux/mman.h` by libc. |
## Man pages ## Man pages

View file

@ -8,9 +8,9 @@ Together they form the backbone of the storage engine's on-disk layout: segment
| Path | What | | Path | What |
| --- | --- | | --- | --- |
| [`reference/linux/fs/open.c`](../../../reference/linux/fs/open.c) | `SYSCALL_DEFINE4(fallocate, ...)`. The syscall delegates to `file->f_op->fallocate` — per-filesystem. | | [`.dev/reference/linux/fs/open.c`](../../../.dev/reference/linux/fs/open.c) | `SYSCALL_DEFINE4(fallocate, ...)`. The syscall delegates to `file->f_op->fallocate` — per-filesystem. |
| [`reference/linux/fs/read_write.c`](../../../reference/linux/fs/read_write.c) | `SYSCALL_DEFINE4(pread64, ...)`, `SYSCALL_DEFINE4(pwrite64, ...)`, `SYSCALL_DEFINE6(pwritev2, ...)`. | | [`.dev/reference/linux/fs/read_write.c`](../../../.dev/reference/linux/fs/read_write.c) | `SYSCALL_DEFINE4(pread64, ...)`, `SYSCALL_DEFINE4(pwrite64, ...)`, `SYSCALL_DEFINE6(pwritev2, ...)`. |
| [`reference/linux/include/uapi/linux/falloc.h`](../../../reference/linux/include/uapi/linux/falloc.h) | `FALLOC_FL_*` flags. | | [`.dev/reference/linux/include/uapi/linux/falloc.h`](../../../.dev/reference/linux/include/uapi/linux/falloc.h) | `FALLOC_FL_*` flags. |
## Man pages ## Man pages

View file

@ -8,10 +8,10 @@ Not on the runtime's critical path today; useful when the runtime grows a superv
| Path | What | | Path | What |
| --- | --- | | --- | --- |
| [`reference/linux/kernel/pid.c`](../../../reference/linux/kernel/pid.c) | `SYSCALL_DEFINE2(pidfd_open, ...)`, `pidfd_create`, `pidfd_pid`. | | [`.dev/reference/linux/kernel/pid.c`](../../../.dev/reference/linux/kernel/pid.c) | `SYSCALL_DEFINE2(pidfd_open, ...)`, `pidfd_create`, `pidfd_pid`. |
| [`reference/linux/kernel/signal.c`](../../../reference/linux/kernel/signal.c) | `SYSCALL_DEFINE4(pidfd_send_signal, ...)`. | | [`.dev/reference/linux/kernel/signal.c`](../../../.dev/reference/linux/kernel/signal.c) | `SYSCALL_DEFINE4(pidfd_send_signal, ...)`. |
| [`reference/linux/kernel/fork.c`](../../../reference/linux/kernel/fork.c) | `clone3` — the only way to get a pidfd atomically with spawn. | | [`.dev/reference/linux/kernel/fork.c`](../../../.dev/reference/linux/kernel/fork.c) | `clone3` — the only way to get a pidfd atomically with spawn. |
| [`reference/linux/include/uapi/linux/pidfd.h`](../../../reference/linux/include/uapi/linux/pidfd.h) | `PIDFD_*` flags. | | [`.dev/reference/linux/include/uapi/linux/pidfd.h`](../../../.dev/reference/linux/include/uapi/linux/pidfd.h) | `PIDFD_*` flags. |
## Man pages ## Man pages

View file

@ -8,9 +8,9 @@ Useful for the storage engine's transient work: building an index in memory befo
| Path | What | | Path | What |
| --- | --- | | --- | --- |
| [`reference/linux/mm/memfd.c`](../../../reference/linux/mm/memfd.c) | `SYSCALL_DEFINE2(memfd_create, ...)` + seal ops. | | [`.dev/reference/linux/mm/memfd.c`](../../../.dev/reference/linux/mm/memfd.c) | `SYSCALL_DEFINE2(memfd_create, ...)` + seal ops. |
| [`reference/linux/include/uapi/linux/memfd.h`](../../../reference/linux/include/uapi/linux/memfd.h) | `MFD_*` flags. | | [`.dev/reference/linux/include/uapi/linux/memfd.h`](../../../.dev/reference/linux/include/uapi/linux/memfd.h) | `MFD_*` flags. |
| [`reference/linux/include/uapi/linux/fcntl.h`](../../../reference/linux/include/uapi/linux/fcntl.h) | `F_ADD_SEALS`, `F_GET_SEALS`, `F_SEAL_*` constants. Sealing is a `fcntl(F_ADD_SEALS, ...)` operation on the memfd. | | [`.dev/reference/linux/include/uapi/linux/fcntl.h`](../../../.dev/reference/linux/include/uapi/linux/fcntl.h) | `F_ADD_SEALS`, `F_GET_SEALS`, `F_SEAL_*` constants. Sealing is a `fcntl(F_ADD_SEALS, ...)` operation on the memfd. |
## Man pages ## Man pages

View file

@ -15,10 +15,10 @@ The previous cards cover positional I/O ([`09-fallocate.md`](./09-fallocate.md))
| Postgres call | Wraps | Where | | Postgres call | Wraps | Where |
| --- | --- | --- | | --- | --- | --- |
| `pg_pwrite()` | `pwrite64` | [`storage/file/fd.c`](../../../../reference/postgresql/src/backend/storage/file/fd.c) — every block-aligned write. | | `pg_pwrite()` | `pwrite64` | [`storage/file/fd.c`](../../../../.dev/reference/postgresql/src/backend/storage/file/fd.c) — every block-aligned write. |
| `pg_fsync()` | `fsync` (or platform variant) | [`storage/file/fd.c`](../../../../reference/postgresql/src/backend/storage/file/fd.c) — wraps `wal_sync_method` GUC dispatch. | | `pg_fsync()` | `fsync` (or platform variant) | [`storage/file/fd.c`](../../../../.dev/reference/postgresql/src/backend/storage/file/fd.c) — wraps `wal_sync_method` GUC dispatch. |
| `pg_fdatasync()` | `fdatasync` | Same. Selected when `wal_sync_method = fdatasync`. | | `pg_fdatasync()` | `fdatasync` | Same. Selected when `wal_sync_method = fdatasync`. |
| Async writeback | `sync_file_range` | [`access/transam/xlog.c`](../../../../reference/postgresql/src/backend/access/transam/xlog.c) — `issue_xlog_fsync` calls `sync_file_range(SYNC_FILE_RANGE_WRITE)` to start I/O on the WAL ahead of the durability barrier. | | Async writeback | `sync_file_range` | [`access/transam/xlog.c`](../../../../.dev/reference/postgresql/src/backend/access/transam/xlog.c) — `issue_xlog_fsync` calls `sync_file_range(SYNC_FILE_RANGE_WRITE)` to start I/O on the WAL ahead of the durability barrier. |
The Postgres GUC matrix (`wal_sync_method`) lets the operator pick between `fsync`, `fdatasync`, `open_sync`, `open_datasync`, `fsync_writethrough`. **Writeonce picks one** — `fdatasync` for the WAL, `fsync` for control files and segment rollovers — and ships it. The Postgres GUC matrix (`wal_sync_method`) lets the operator pick between `fsync`, `fdatasync`, `open_sync`, `open_datasync`, `fsync_writethrough`. **Writeonce picks one** — `fdatasync` for the WAL, `fsync` for control files and segment rollovers — and ships it.
@ -26,10 +26,10 @@ The Postgres GUC matrix (`wal_sync_method`) lets the operator pick between `fsyn
| Path | What | | Path | What |
| --- | --- | | --- | --- |
| [`reference/linux/fs/read_write.c`](../../../reference/linux/fs/read_write.c) | `SYSCALL_DEFINE4(pread64, ...)`, `SYSCALL_DEFINE4(pwrite64, ...)`, `SYSCALL_DEFINE6(pwritev2, ...)`. | | [`.dev/reference/linux/fs/read_write.c`](../../../.dev/reference/linux/fs/read_write.c) | `SYSCALL_DEFINE4(pread64, ...)`, `SYSCALL_DEFINE4(pwrite64, ...)`, `SYSCALL_DEFINE6(pwritev2, ...)`. |
| [`reference/linux/fs/sync.c`](../../../reference/linux/fs/sync.c) | `SYSCALL_DEFINE1(fsync, ...)`, `SYSCALL_DEFINE1(fdatasync, ...)`, `SYSCALL_DEFINE4(sync_file_range, ...)`. | | [`.dev/reference/linux/fs/sync.c`](../../../.dev/reference/linux/fs/sync.c) | `SYSCALL_DEFINE1(fsync, ...)`, `SYSCALL_DEFINE1(fdatasync, ...)`, `SYSCALL_DEFINE4(sync_file_range, ...)`. |
| [`reference/linux/include/uapi/asm-generic/fcntl.h`](../../../reference/linux/include/uapi/asm-generic/fcntl.h) | `O_SYNC`, `O_DSYNC`, `O_DIRECT`. | | [`.dev/reference/linux/include/uapi/asm-generic/fcntl.h`](../../../.dev/reference/linux/include/uapi/asm-generic/fcntl.h) | `O_SYNC`, `O_DSYNC`, `O_DIRECT`. |
| [`reference/linux/Documentation/filesystems/ext4/journal.rst`](../../../reference/linux/Documentation/filesystems/ext4/journal.rst) | What ext4's journal commits when `fsync` runs. Worth understanding what the kernel actually does on the durability path. | | [`.dev/reference/linux/Documentation/filesystems/ext4/journal.rst`](../../../.dev/reference/linux/Documentation/filesystems/ext4/journal.rst) | What ext4's journal commits when `fsync` runs. Worth understanding what the kernel actually does on the durability path. |
## Man pages ## Man pages
@ -149,4 +149,4 @@ Pair with [`postgresql/wal.md`](../postgresql/wal.md), [`postgresql/buffer-and-c
## v1 port source ## v1 port source
**Partial.** `reference/crates/wo-seg/src/writer.rs:92` calls `file.sync_all()` (Rust stdlib's `fsync` wrapper). Phase 11 replaces with explicit `libc::fdatasync` for the WAL path; segment files keep `fsync` semantics for rollover events. **Partial.** `.dev/reference/crates/wo-seg/src/writer.rs:92` calls `file.sync_all()` (Rust stdlib's `fsync` wrapper). Phase 11 replaces with explicit `libc::fdatasync` for the WAL path; segment files keep `fsync` semantics for rollover events.

View file

@ -1,14 +1,14 @@
# PostgreSQL — storage subsystem reference # PostgreSQL — storage subsystem reference
These cards exist to make the Postgres backend a useful **library of patterns** for writeonce's persistent-storage phases (10–12) without inviting a multi-process port. Each card pulls one subsystem out of [`reference/postgresql/src/backend/`](../../../../reference/postgresql/src/backend/) — paths into the Postgres tree, the underlying *idea*, and the writeonce translation. These cards exist to make the Postgres backend a useful **library of patterns** for writeonce's persistent-storage phases (10–12) without inviting a multi-process port. Each card pulls one subsystem out of [`.dev/reference/postgresql/src/backend/`](../../../../.dev/reference/postgresql/src/backend/) — paths into the Postgres tree, the underlying *idea*, and the writeonce translation.
The symlink is user-specific: The symlink is user-specific:
```bash ```bash
ln -s /home/shoney/projects/postgresql reference/postgresql ln -s /home/shoney/projects/postgresql .dev/reference/postgresql
``` ```
Gitignored — see [`.gitignore`](../../../../.gitignore). Pair it with [`reference/linux`](../../../../reference/linux) and [`reference/go`](../../../../reference/go) if not already linked. Gitignored — see [`.gitignore`](../../../../.gitignore). Pair it with [`.dev/reference/linux`](../../../../.dev/reference/linux) and [`.dev/reference/go`](../../../../.dev/reference/go) if not already linked.
## Per-subsystem cards ## Per-subsystem cards

View file

@ -13,12 +13,12 @@ No separate process. No shared-buffer pinning. No dynamic-shared-memory coordina
| File | Responsibility | | File | Responsibility |
| --- | --- | | --- | --- |
| [`storage/buffer/bufmgr.c`](../../../../reference/postgresql/src/backend/storage/buffer/bufmgr.c) | Page cache front-door: `ReadBuffer`, `BufferGetPage`, `MarkBufferDirty`, `FlushBuffer`. Tracks dirty bit per buffer; pinning prevents eviction. | | [`storage/buffer/bufmgr.c`](../../../../.dev/reference/postgresql/src/backend/storage/buffer/bufmgr.c) | Page cache front-door: `ReadBuffer`, `BufferGetPage`, `MarkBufferDirty`, `FlushBuffer`. Tracks dirty bit per buffer; pinning prevents eviction. |
| [`storage/buffer/freelist.c`](../../../../reference/postgresql/src/backend/storage/buffer/freelist.c) | Clock-sweep eviction policy. Buffers with `usage_count = 0` and `pin_count = 0` are eviction candidates; usage decremented on every sweep pass, incremented on access. | | [`storage/buffer/freelist.c`](../../../../.dev/reference/postgresql/src/backend/storage/buffer/freelist.c) | Clock-sweep eviction policy. Buffers with `usage_count = 0` and `pin_count = 0` are eviction candidates; usage decremented on every sweep pass, incremented on access. |
| [`storage/buffer/buf_table.c`](../../../../reference/postgresql/src/backend/storage/buffer/buf_table.c) | Hash table from `(file, block)` → buffer slot. The lookup that `ReadBuffer` does. | | [`storage/buffer/buf_table.c`](../../../../.dev/reference/postgresql/src/backend/storage/buffer/buf_table.c) | Hash table from `(file, block)` → buffer slot. The lookup that `ReadBuffer` does. |
| [`postmaster/checkpointer.c`](../../../../reference/postgresql/src/backend/postmaster/checkpointer.c) | The checkpointer process. Triggered by time (`checkpoint_timeout`), WAL volume (`max_wal_size`), or signal. Runs `BufferSync()` to flush dirty buffers, then `CreateCheckPoint()` to update the control file. | | [`postmaster/checkpointer.c`](../../../../.dev/reference/postgresql/src/backend/postmaster/checkpointer.c) | The checkpointer process. Triggered by time (`checkpoint_timeout`), WAL volume (`max_wal_size`), or signal. Runs `BufferSync()` to flush dirty buffers, then `CreateCheckPoint()` to update the control file. |
| [`postmaster/bgwriter.c`](../../../../reference/postgresql/src/backend/postmaster/bgwriter.c) | Continuously trickles dirty pages to disk between checkpoints. Smooths the I/O burst the checkpointer would cause. | | [`postmaster/bgwriter.c`](../../../../.dev/reference/postgresql/src/backend/postmaster/bgwriter.c) | Continuously trickles dirty pages to disk between checkpoints. Smooths the I/O burst the checkpointer would cause. |
| [`storage/buffer/README`](../../../../reference/postgresql/src/backend/storage/buffer/README) | Overview of the pinning, locking, and replacement policy. Worth reading. | | [`storage/buffer/README`](../../../../.dev/reference/postgresql/src/backend/storage/buffer/README) | Overview of the pinning, locking, and replacement policy. Worth reading. |
## The page-cache idea worth porting ## The page-cache idea worth porting

View file

@ -8,11 +8,11 @@ Writeonce's phase 10 starts simpler — variable-length records, no pages. Phase
| File | Responsibility | | File | Responsibility |
| --- | --- | | --- | --- |
| [`storage/page/bufpage.c`](../../../../reference/postgresql/src/backend/storage/page/bufpage.c) | Page initialization (`PageInit`), line-pointer manipulation, free-space accounting. | | [`storage/page/bufpage.c`](../../../../.dev/reference/postgresql/src/backend/storage/page/bufpage.c) | Page initialization (`PageInit`), line-pointer manipulation, free-space accounting. |
| [`storage/page/checksum.c`](../../../../reference/postgresql/src/backend/storage/page/checksum.c) | The page checksum algorithm — CRC32C-style with a Postgres-specific finalization. Optional, enabled at cluster init. | | [`storage/page/checksum.c`](../../../../.dev/reference/postgresql/src/backend/storage/page/checksum.c) | The page checksum algorithm — CRC32C-style with a Postgres-specific finalization. Optional, enabled at cluster init. |
| [`storage/page/itemptr.c`](../../../../reference/postgresql/src/backend/storage/page/itemptr.c) | Item pointer (`ItemPointerData`) — `(block_number, offset_within_page)` 6-byte tuple address. The on-disk equivalent of writeonce's `(TypeName, SegmentOffset)`. | | [`storage/page/itemptr.c`](../../../../.dev/reference/postgresql/src/backend/storage/page/itemptr.c) | Item pointer (`ItemPointerData`) — `(block_number, offset_within_page)` 6-byte tuple address. The on-disk equivalent of writeonce's `(TypeName, SegmentOffset)`. |
| [`include/storage/bufpage.h`](../../../../reference/postgresql/src/include/storage/bufpage.h) | The header-file definition. Read this first — it's the spec. | | [`include/storage/bufpage.h`](../../../../.dev/reference/postgresql/src/include/storage/bufpage.h) | The header-file definition. Read this first — it's the spec. |
| [`storage/page/README`](../../../../reference/postgresql/src/backend/storage/page/README) | One-page overview of the slotted-page model and how checksums interact with WAL. | | [`storage/page/README`](../../../../.dev/reference/postgresql/src/backend/storage/page/README) | One-page overview of the slotted-page model and how checksums interact with WAL. |
## The Postgres page header (24 bytes) ## The Postgres page header (24 bytes)

View file

@ -8,10 +8,10 @@ The writeonce equivalent is **per-type segment files** (`data/<TypeName>.seg`).
| File | Responsibility | | File | Responsibility |
| --- | --- | | --- | --- |
| [`storage/smgr/smgr.c`](../../../../reference/postgresql/src/backend/storage/smgr/smgr.c) | Front-door API. `smgropen`, `smgrread`, `smgrwrite`, `smgrextend`, `smgrdounlink`. Holds the `SMgrRelation` cache. | | [`storage/smgr/smgr.c`](../../../../.dev/reference/postgresql/src/backend/storage/smgr/smgr.c) | Front-door API. `smgropen`, `smgrread`, `smgrwrite`, `smgrextend`, `smgrdounlink`. Holds the `SMgrRelation` cache. |
| [`storage/smgr/md.c`](../../../../reference/postgresql/src/backend/storage/smgr/md.c) | The actual implementation against the kernel. Manages `MdfdVec` (open file descriptor handles per segment number), opens missing segments lazily. | | [`storage/smgr/md.c`](../../../../.dev/reference/postgresql/src/backend/storage/smgr/md.c) | The actual implementation against the kernel. Manages `MdfdVec` (open file descriptor handles per segment number), opens missing segments lazily. |
| [`storage/smgr/bulk_write.c`](../../../../reference/postgresql/src/backend/storage/smgr/bulk_write.c) | Optimized path for bulk-loading: writes directly to `smgrwrite` without going through shared buffers. Useful for `COPY` / `CREATE INDEX` + the recovery path's wal-replay-rebuilds-pages flow. | | [`storage/smgr/bulk_write.c`](../../../../.dev/reference/postgresql/src/backend/storage/smgr/bulk_write.c) | Optimized path for bulk-loading: writes directly to `smgrwrite` without going through shared buffers. Useful for `COPY` / `CREATE INDEX` + the recovery path's wal-replay-rebuilds-pages flow. |
| [`storage/smgr/README`](../../../../reference/postgresql/src/backend/storage/smgr/README) | Brief but worth reading — explains the relfilenode → file naming convention and how `RELSEG_SIZE` interacts with 32-bit-fs-size historical limits. | | [`storage/smgr/README`](../../../../.dev/reference/postgresql/src/backend/storage/smgr/README) | Brief but worth reading — explains the relfilenode → file naming convention and how `RELSEG_SIZE` interacts with 32-bit-fs-size historical limits. |
## What `md.c` actually does ## What `md.c` actually does

View file

@ -8,11 +8,11 @@ Writeonce mirrors the algorithm. The single-thread loop replaces multi-process c
| File | Responsibility | | File | Responsibility |
| --- | --- | | --- | --- |
| [`access/transam/xlog.c`](../../../../reference/postgresql/src/backend/access/transam/xlog.c) | Top-level WAL machinery: insertion locks, segment rollover, flush coordination, control-file rendezvous. | | [`access/transam/xlog.c`](../../../../.dev/reference/postgresql/src/backend/access/transam/xlog.c) | Top-level WAL machinery: insertion locks, segment rollover, flush coordination, control-file rendezvous. |
| [`access/transam/xloginsert.c`](../../../../reference/postgresql/src/backend/access/transam/xloginsert.c) | Build a WAL record (header + payload + backup-block deltas) and place it into the in-memory WAL buffer. | | [`access/transam/xloginsert.c`](../../../../.dev/reference/postgresql/src/backend/access/transam/xloginsert.c) | Build a WAL record (header + payload + backup-block deltas) and place it into the in-memory WAL buffer. |
| [`access/transam/xlogreader.c`](../../../../reference/postgresql/src/backend/access/transam/xlogreader.c) | Decode WAL records during recovery — pure parser, no I/O. Useful as the read-side spec. | | [`access/transam/xlogreader.c`](../../../../.dev/reference/postgresql/src/backend/access/transam/xlogreader.c) | Decode WAL records during recovery — pure parser, no I/O. Useful as the read-side spec. |
| [`access/transam/xlogrecovery.c`](../../../../reference/postgresql/src/backend/access/transam/xlogrecovery.c) | The replay loop. Walks the WAL from the last-checkpoint LSN, replays each record into shared buffers, advances the redo pointer. | | [`access/transam/xlogrecovery.c`](../../../../.dev/reference/postgresql/src/backend/access/transam/xlogrecovery.c) | The replay loop. Walks the WAL from the last-checkpoint LSN, replays each record into shared buffers, advances the redo pointer. |
| [`postmaster/walwriter.c`](../../../../reference/postgresql/src/backend/postmaster/walwriter.c) | Background process that flushes the WAL buffer to disk asynchronously. Writeonce does this **inline in the loop tick**. | | [`postmaster/walwriter.c`](../../../../.dev/reference/postgresql/src/backend/postmaster/walwriter.c) | Background process that flushes the WAL buffer to disk asynchronously. Writeonce does this **inline in the loop tick**. |
## The five Postgres WAL ideas writeonce keeps ## The five Postgres WAL ideas writeonce keeps
@ -57,9 +57,9 @@ Same effect as Postgres' group-commit fence (one `fsync` flushes many commits) w
## Pointers when implementing phase 11 ## Pointers when implementing phase 11
- [`xloginsert.c:XLogInsert()`](../../../../reference/postgresql/src/backend/access/transam/xloginsert.c) — entry point for "insert this record into the WAL." Read the prologue + the LSN-assignment loop, ignore the buffer-juggling. - [`xloginsert.c:XLogInsert()`](../../../../.dev/reference/postgresql/src/backend/access/transam/xloginsert.c) — entry point for "insert this record into the WAL." Read the prologue + the LSN-assignment loop, ignore the buffer-juggling.
- [`xlog.c:XLogFlush()`](../../../../reference/postgresql/src/backend/access/transam/xlog.c) — "make this LSN durable on disk." Read the early-out for "already flushed" and the group-commit waiter logic. - [`xlog.c:XLogFlush()`](../../../../.dev/reference/postgresql/src/backend/access/transam/xlog.c) — "make this LSN durable on disk." Read the early-out for "already flushed" and the group-commit waiter logic.
- [`xlogrecovery.c:PerformWalRecovery()`](../../../../reference/postgresql/src/backend/access/transam/xlogrecovery.c) — the replay loop. Read the redo-pointer advance logic; ignore the multi-process startup signaling. - [`xlogrecovery.c:PerformWalRecovery()`](../../../../.dev/reference/postgresql/src/backend/access/transam/xlogrecovery.c) — the replay loop. Read the redo-pointer advance logic; ignore the multi-process startup signaling.
## Used by ## Used by

View file

@ -1,12 +1,12 @@
# UI track — `.htmlx` live templates + Angular-style monorepo # UI track — `.htmlx` live templates + Angular-style monorepo
**Context sources:** [`docs/examples/ecommerce/ui/`](../../examples/ecommerce/ui/) (current `##ui` screens — storefront, order_tracker, admin_orders), [`docs/examples/ecommerce/types/`](../../examples/ecommerce/types/) + [`docs/examples/ecommerce/logic/`](../../examples/ecommerce/logic/) (the shared-schema + shared-fn anchor), [`reference/crates/wo-htmlx/`](../../../reference/crates/wo-htmlx/) (v1 template engine — `{{path}}`, `{{#each}}`, `{{> partial}}`, `data-bind` attributes), [`templates/`](../../../templates/) (v1 blog's concrete `.htmlx` usage), [`docs/runtime/database/06-lowcode-fullstack.md`](../../runtime/database/06-lowcode-fullstack.md) (Phase 6's `##ui` + `##app` block spec). **Context sources:** [`docs/examples/ecommerce/ui/`](../../examples/ecommerce/ui/) (current `##ui` screens — storefront, order_tracker, admin_orders), [`docs/examples/ecommerce/types/`](../../examples/ecommerce/types/) + [`docs/examples/ecommerce/logic/`](../../examples/ecommerce/logic/) (the shared-schema + shared-fn anchor), [`.dev/reference/crates/wo-htmlx/`](../../../.dev/reference/crates/wo-htmlx/) (v1 template engine — `{{path}}`, `{{#each}}`, `{{> partial}}`, `data-bind` attributes), [`templates/`](../../../templates/) (v1 blog's concrete `.htmlx` usage), [`docs/runtime/database/06-lowcode-fullstack.md`](../../runtime/database/06-lowcode-fullstack.md) (Phase 6's `##ui` + `##app` block spec).
## Context ## Context
Three threads converge into one plan: Three threads converge into one plan:
1. **`##ui` needs a concrete output format.** Phase 6's spec says screens "compile to a render tree" served as SSR HTML with a thin client runtime, but the actual template format isn't named. The v1 `.htmlx` engine at [`reference/crates/wo-htmlx/`](../../../reference/crates/wo-htmlx/) already speaks `{{bindings}}`, `{{#each}}`, `{{> partials}}`, and `data-bind` attributes — it's 90% of what the new runtime needs and already has a working parser + renderer. Adopting it (and extending it with live-subscription semantics) is cheaper than inventing a new format. 1. **`##ui` needs a concrete output format.** Phase 6's spec says screens "compile to a render tree" served as SSR HTML with a thin client runtime, but the actual template format isn't named. The v1 `.htmlx` engine at [`.dev/reference/crates/wo-htmlx/`](../../../.dev/reference/crates/wo-htmlx/) already speaks `{{bindings}}`, `{{#each}}`, `{{> partials}}`, and `data-bind` attributes — it's 90% of what the new runtime needs and already has a working parser + renderer. Adopting it (and extending it with live-subscription semantics) is cheaper than inventing a new format.
2. **The samples want a home that matches how real frontends are organised.** The ecommerce sample today is one flat directory with `types/`, `logic/`, and `ui/` beside each other. A real deployment has *multiple apps* against the same data: a customer storefront, an admin dashboard, a fulfillment console, maybe a read-only analytics viewer. Each has its own routes, its own policies, its own ideal binary shape. Angular (via Nx / Angular CLI workspaces) solved this with `apps/*` + `libs/*` on top of a shared root config — writeonce adopts the same shape. 2. **The samples want a home that matches how real frontends are organised.** The ecommerce sample today is one flat directory with `types/`, `logic/`, and `ui/` beside each other. A real deployment has *multiple apps* against the same data: a customer storefront, an admin dashboard, a fulfillment console, maybe a read-only analytics viewer. Each has its own routes, its own policies, its own ideal binary shape. Angular (via Nx / Angular CLI workspaces) solved this with `apps/*` + `libs/*` on top of a shared root config — writeonce adopts the same shape.
@ -61,7 +61,7 @@ Read before writing each sub-phase:
| Source | Why | | Source | Why |
| --- | --- | | --- | --- |
| [`reference/crates/wo-htmlx/src/parser.rs`](../../../reference/crates/wo-htmlx/src/parser.rs) + [`render.rs`](../../../reference/crates/wo-htmlx/src/render.rs) | The v1 template engine's exact surface — what parses, what renders, what the AST looks like. ~500 LOC total. | | [`.dev/reference/crates/wo-htmlx/src/parser.rs`](../../../.dev/reference/crates/wo-htmlx/src/parser.rs) + [`render.rs`](../../../.dev/reference/crates/wo-htmlx/src/render.rs) | The v1 template engine's exact surface — what parses, what renders, what the AST looks like. ~500 LOC total. |
| [`templates/article.htmlx`](../../../templates/article.htmlx), [`templates/home.htmlx`](../../../templates/home.htmlx) | Concrete usage of the v1 format — how `{{path}}` and `data-bind` actually read in real templates. | | [`templates/article.htmlx`](../../../templates/article.htmlx), [`templates/home.htmlx`](../../../templates/home.htmlx) | Concrete usage of the v1 format — how `{{path}}` and `data-bind` actually read in real templates. |
| [`docs/examples/ecommerce/ui/{storefront,order_tracker,admin_orders}.wo`](../../examples/ecommerce/ui/) | The `##ui` side — what the declarative DSL promises to produce. These screens are the target of the first compiler pass. | | [`docs/examples/ecommerce/ui/{storefront,order_tracker,admin_orders}.wo`](../../examples/ecommerce/ui/) | The `##ui` side — what the declarative DSL promises to produce. These screens are the target of the first compiler pass. |
| [`docs/runtime/database/06-lowcode-fullstack.md`](../../runtime/database/06-lowcode-fullstack.md) | Phase 6's full-stack block spec — `##ui`, `##app`, `##policy`, `##service`, `##logic` — already designed but not yet compiled. | | [`docs/runtime/database/06-lowcode-fullstack.md`](../../runtime/database/06-lowcode-fullstack.md) | Phase 6's full-stack block spec — `##ui`, `##app`, `##policy`, `##service`, `##logic` — already designed but not yet compiled. |
@ -209,5 +209,5 @@ After all seven sub-phases land:
- [`../../runtime/database/06-lowcode-fullstack.md`](../../runtime/database/06-lowcode-fullstack.md) — Phase 6's full-stack block spec that this track implements. - [`../../runtime/database/06-lowcode-fullstack.md`](../../runtime/database/06-lowcode-fullstack.md) — Phase 6's full-stack block spec that this track implements.
- [`../../runtime/database/04-client-api.md`](../../runtime/database/04-client-api.md) — the wire protocol per-app binaries speak to the shared DB over. - [`../../runtime/database/04-client-api.md`](../../runtime/database/04-client-api.md) — the wire protocol per-app binaries speak to the shared DB over.
- [`../../examples/ecommerce/ui/admin_orders.wo`](../../examples/ecommerce/ui/admin_orders.wo) — the motivating workload: a live ops table bound to the order stream. - [`../../examples/ecommerce/ui/admin_orders.wo`](../../examples/ecommerce/ui/admin_orders.wo) — the motivating workload: a live ops table bound to the order stream.
- [`reference/crates/wo-htmlx/`](../../../reference/crates/wo-htmlx/) — the template engine ~90% of this track will reuse. - [`.dev/reference/crates/wo-htmlx/`](../../../.dev/reference/crates/wo-htmlx/) — the template engine ~90% of this track will reuse.
- [`templates/`](../../../templates/) — v1 blog's actual `.htmlx` files; the format this track extends. - [`templates/`](../../../templates/) — v1 blog's actual `.htmlx` files; the format this track extends.

View file

@ -1,6 +1,6 @@
# 01 — `.htmlx` format spec # 01 — `.htmlx` format spec
**Context sources:** [`./00-overview.md`](./00-overview.md) §§ "`.htmlx` with live subscriptions — target format" (L127–166), "Design decisions" (L28–37), [`reference/crates/wo-htmlx/`](../../../reference/crates/wo-htmlx/) (the v1 template engine that 90% of this phase ports), [`templates/article.htmlx`](../../../templates/article.htmlx) and [`templates/home.htmlx`](../../../templates/home.htmlx) (v1 concrete usage), [`docs/examples/ecommerce/shared/components/order-row.htmlx`](../../examples/ecommerce/shared/components/order-row.htmlx) (the live-binding workload this format must serve). **Context sources:** [`./00-overview.md`](./00-overview.md) §§ "`.htmlx` with live subscriptions — target format" (L127–166), "Design decisions" (L28–37), [`.dev/reference/crates/wo-htmlx/`](../../../.dev/reference/crates/wo-htmlx/) (the v1 template engine that 90% of this phase ports), [`templates/article.htmlx`](../../../templates/article.htmlx) and [`templates/home.htmlx`](../../../templates/home.htmlx) (v1 concrete usage), [`docs/examples/ecommerce/shared/components/order-row.htmlx`](../../examples/ecommerce/shared/components/order-row.htmlx) (the live-binding workload this format must serve).
## Goal ## Goal
@ -8,7 +8,7 @@ Lock the exact `.htmlx` grammar — every v1 Mustache construct unchanged plus t
## Design decisions (locked) ## Design decisions (locked)
1. **Mustache constructs carry through unchanged.** `{{path}}`, `{{#each xs as y}}…{{/each}}`, `{{#if cond}}…{{/if}}`, `{{#when cond}}…{{/when}}`, `{{> partial arg=val}}`. The v1 parser already handles all of these; the new parser inherits them verbatim. See [`reference/crates/wo-htmlx/src/parser.rs`](../../../reference/crates/wo-htmlx/src/parser.rs) (173 LOC) and the AST in [`ast.rs`](../../../reference/crates/wo-htmlx/src/ast.rs) (18 LOC). 1. **Mustache constructs carry through unchanged.** `{{path}}`, `{{#each xs as y}}…{{/each}}`, `{{#if cond}}…{{/if}}`, `{{#when cond}}…{{/when}}`, `{{> partial arg=val}}`. The v1 parser already handles all of these; the new parser inherits them verbatim. See [`.dev/reference/crates/wo-htmlx/src/parser.rs`](../../../.dev/reference/crates/wo-htmlx/src/parser.rs) (173 LOC) and the AST in [`ast.rs`](../../../.dev/reference/crates/wo-htmlx/src/ast.rs) (18 LOC).
2. **`<wo:live>` is a parsed structured node, not HTML passthrough.** The parser recognises the `<wo:` prefix, captures attributes (`source`, `key`, optional `sort`, `filter`), and recursively parses the body as a normal `.htmlx` subtree. No nesting in this phase — error at parse if a `<wo:live>` contains another `<wo:live>`. 2. **`<wo:live>` is a parsed structured node, not HTML passthrough.** The parser recognises the `<wo:` prefix, captures attributes (`source`, `key`, optional `sort`, `filter`), and recursively parses the body as a normal `.htmlx` subtree. No nesting in this phase — error at parse if a `<wo:live>` contains another `<wo:live>`.
3. **`wo:bind="field"` is an HTML attribute, parsed but emitted verbatim.** SSR writes the attribute through; the consumer is the client runtime. The parser records each `(element, field)` pair into the manifest; nothing else changes about element rendering. 3. **`wo:bind="field"` is an HTML attribute, parsed but emitted verbatim.** SSR writes the attribute through; the consumer is the client runtime. The parser records each `(element, field)` pair into the manifest; nothing else changes about element rendering.
4. **Helpers are a closed Rust enum.** v1 invocation forms (`{{relative ts}}`, `{{#if (eq for "ops")}}`, `{{> money amount=x}}`) carry through. The registered set is fixed for this phase: `relative`, `eq`, `markdown`, `code`, `money`, `tag-chips`, `pill`, `image`, `stock-badge`, `list`. No author extensibility. 4. **Helpers are a closed Rust enum.** v1 invocation forms (`{{relative ts}}`, `{{#if (eq for "ops")}}`, `{{> money amount=x}}`) carry through. The registered set is fixed for this phase: `relative`, `eq`, `markdown`, `code`, `money`, `tag-chips`, `pill`, `image`, `stock-badge`, `list`. No author extensibility.
@ -20,12 +20,12 @@ Lock the exact `.htmlx` grammar — every v1 Mustache construct unchanged plus t
| File | Responsibility | Port source | | File | Responsibility | Port source |
| --- | --- | --- | | --- | --- | --- |
| `mod.rs` | Re-exports `Template`, `Manifest`, `LiveSubscription`, `BindSite`, `ParseError`, `RenderError` | [`reference/crates/wo-htmlx/src/lib.rs`](../../../reference/crates/wo-htmlx/src/lib.rs) (11 LOC) | | `mod.rs` | Re-exports `Template`, `Manifest`, `LiveSubscription`, `BindSite`, `ParseError`, `RenderError` | [`.dev/reference/crates/wo-htmlx/src/lib.rs`](../../../.dev/reference/crates/wo-htmlx/src/lib.rs) (11 LOC) |
| `ast.rs` | Adds `Node::Live { attrs, body }` and `wo_bind: Option<String>` on element nodes | [`reference/crates/wo-htmlx/src/ast.rs`](../../../reference/crates/wo-htmlx/src/ast.rs) (18 LOC) — extend by ~50 LOC | | `ast.rs` | Adds `Node::Live { attrs, body }` and `wo_bind: Option<String>` on element nodes | [`.dev/reference/crates/wo-htmlx/src/ast.rs`](../../../.dev/reference/crates/wo-htmlx/src/ast.rs) (18 LOC) — extend by ~50 LOC |
| `parser.rs` | Adds `<wo:` prefix recognition + attribute capture; rest unchanged | [`reference/crates/wo-htmlx/src/parser.rs`](../../../reference/crates/wo-htmlx/src/parser.rs) (173 LOC) — extend by ~90 LOC | | `parser.rs` | Adds `<wo:` prefix recognition + attribute capture; rest unchanged | [`.dev/reference/crates/wo-htmlx/src/parser.rs`](../../../.dev/reference/crates/wo-htmlx/src/parser.rs) (173 LOC) — extend by ~90 LOC |
| `value.rs` | Path resolution against a context Value | [`reference/crates/wo-htmlx/src/value.rs`](../../../reference/crates/wo-htmlx/src/value.rs) (122 LOC) — copied verbatim | | `value.rs` | Path resolution against a context Value | [`.dev/reference/crates/wo-htmlx/src/value.rs`](../../../.dev/reference/crates/wo-htmlx/src/value.rs) (122 LOC) — copied verbatim |
| `registry.rs` | Closed helper-fn registry | [`reference/crates/wo-htmlx/src/registry.rs`](../../../reference/crates/wo-htmlx/src/registry.rs) (121 LOC) — extend by ~60 LOC for new helpers | | `registry.rs` | Closed helper-fn registry | [`.dev/reference/crates/wo-htmlx/src/registry.rs`](../../../.dev/reference/crates/wo-htmlx/src/registry.rs) (121 LOC) — extend by ~60 LOC for new helpers |
| `render.rs` | Emits HTML; wraps `<wo:live>` body in `<div data-wo-subscription="…">` for the runtime | [`reference/crates/wo-htmlx/src/render.rs`](../../../reference/crates/wo-htmlx/src/render.rs) (140 LOC) — extend by ~70 LOC | | `render.rs` | Emits HTML; wraps `<wo:live>` body in `<div data-wo-subscription="…">` for the runtime | [`.dev/reference/crates/wo-htmlx/src/render.rs`](../../../.dev/reference/crates/wo-htmlx/src/render.rs) (140 LOC) — extend by ~70 LOC |
| `manifest.rs` | Walks the AST, collects subscriptions + bind sites, serialises JSON | new (~150 LOC) | | `manifest.rs` | Walks the AST, collects subscriptions + bind sites, serialises JSON | new (~150 LOC) |
Total: ~835 LOC (585 ported + ~250 new). Total: ~835 LOC (585 ported + ~250 new).
@ -103,7 +103,7 @@ cargo test -p ui --test manifest # manifest emission
cargo run --bin wo -- run docs/examples/blog & cargo run --bin wo -- run docs/examples/blog &
PID=$!; sleep 1; curl -fsS http://127.0.0.1:8080/ >/dev/null; kill $PID PID=$!; sleep 1; curl -fsS http://127.0.0.1:8080/ >/dev/null; kill $PID
cd reference/crates && cargo build && cargo test cd .dev/reference/crates && cargo build && cargo test
``` ```
## After this phase ## After this phase

View file

@ -60,7 +60,7 @@ for (path, src) in outputs { fs::write(path, src)?; }
3. **Hand-written fallback honoured.** With a hand-written `apps/admin/ui/orders/orders.htmlx` present, the compiler returns its source unchanged but still emits the manifest. 3. **Hand-written fallback honoured.** With a hand-written `apps/admin/ui/orders/orders.htmlx` present, the compiler returns its source unchanged but still emits the manifest.
4. **Manifest cross-check fires.** Renaming `body` to `text` in a hand-written template that the `##ui` block expects under `wo:bind="body"` produces a `CompileError::HandWrittenMissingField` diagnostic. 4. **Manifest cross-check fires.** Renaming `body` to `text` in a hand-written template that the `##ui` block expects under `wo:bind="body"` produces a `CompileError::HandWrittenMissingField` diagnostic.
5. **Parser change is non-breaking.** `crates/rt`'s 14 unit tests still pass; `cargo run --bin wo -- run docs/examples/blog` boots and serves REST as before. 5. **Parser change is non-breaking.** `crates/rt`'s 14 unit tests still pass; `cargo run --bin wo -- run docs/examples/blog` boots and serves REST as before.
6. `cd reference/crates && cargo build && cargo test`. 6. `cd .dev/reference/crates && cargo build && cargo test`.
## Non-scope ## Non-scope
@ -87,7 +87,7 @@ head -1 target/wo/storefront/ui/orders.htmlx # starts with <wo:live source="Or
cargo run --bin wo -- run docs/examples/blog & cargo run --bin wo -- run docs/examples/blog &
PID=$!; sleep 1; curl -fsS http://127.0.0.1:8080/ >/dev/null; kill $PID PID=$!; sleep 1; curl -fsS http://127.0.0.1:8080/ >/dev/null; kill $PID
cd reference/crates && cargo build && cargo test cd .dev/reference/crates && cargo build && cargo test
``` ```
## After this phase ## After this phase

View file

@ -1,6 +1,6 @@
# 03 — Client runtime # 03 — Client runtime
**Context sources:** [`./00-overview.md`](./00-overview.md) §§ "`.htmlx` with live subscriptions — target format" (L127–166) and decisions 1–2, [`./01-htmlx-format-spec.md`](./01-htmlx-format-spec.md) (the manifest schema this runtime consumes), [`reference/crates/wo-sub/src/lib.rs`](../../../reference/crates/wo-sub/src/lib.rs) (the v1 frame model the wire format mirrors), [`docs/examples/ecommerce/shared/components/order-row.htmlx`](../../examples/ecommerce/shared/components/order-row.htmlx) (the live workload the runtime must update without reload). **Context sources:** [`./00-overview.md`](./00-overview.md) §§ "`.htmlx` with live subscriptions — target format" (L127–166) and decisions 1–2, [`./01-htmlx-format-spec.md`](./01-htmlx-format-spec.md) (the manifest schema this runtime consumes), [`.dev/reference/crates/wo-sub/src/lib.rs`](../../../.dev/reference/crates/wo-sub/src/lib.rs) (the v1 frame model the wire format mirrors), [`docs/examples/ecommerce/shared/components/order-row.htmlx`](../../examples/ecommerce/shared/components/order-row.htmlx) (the live workload the runtime must update without reload).
## Goal ## Goal
@ -9,7 +9,7 @@ Ship a ~500-line vanilla-JS client at `crates/ui/assets/wo-runtime.js` that, on
## Design decisions (locked) ## Design decisions (locked)
1. **Vanilla JS, no transpiler.** The file shipped is the file written. Anchored in [`./00-overview.md`](./00-overview.md) L25, L198–199. 1. **Vanilla JS, no transpiler.** The file shipped is the file written. Anchored in [`./00-overview.md`](./00-overview.md) L25, L198–199.
2. **JSON over WebSocket.** Frame schema mirrors `reference/crates/wo-sub` semantics evolved into this phase's predicate-subscription model. `{ subscription_id, kind: "snapshot"|"insert"|"update"|"delete", key, row|fields }`. 2. **JSON over WebSocket.** Frame schema mirrors `.dev/reference/crates/wo-sub` semantics evolved into this phase's predicate-subscription model. `{ subscription_id, kind: "snapshot"|"insert"|"update"|"delete", key, row|fields }`.
3. **Targeted DOM patching, not virtual-DOM.** `update` ⇒ `document.querySelectorAll('[data-wo-subscription="<id>"] [data-key="<k>"] [wo\\:bind="<f>"]')` ⇒ `el.textContent = row[f]`. Matches the Zone-less Angular note in 00-overview decision 9. 3. **Targeted DOM patching, not virtual-DOM.** `update` ⇒ `document.querySelectorAll('[data-wo-subscription="<id>"] [data-key="<k>"] [wo\\:bind="<f>"]')` ⇒ `el.textContent = row[f]`. Matches the Zone-less Angular note in 00-overview decision 9.
4. **Reconnect = full snapshot resync.** On reconnect the runtime re-subscribes and replaces each `<wo:live>` body with the fresh snapshot. No diff, no replay buffer. 4. **Reconnect = full snapshot resync.** On reconnect the runtime re-subscribes and replaces each `<wo:live>` body with the fresh snapshot. No diff, no replay buffer.
5. **Backpressure = drop all but latest update per `data-key`.** A coalescing queue keyed by `(subscription_id, key)` collapses queued `update` frames; the latest wins. New frames of other kinds (`insert`/`delete`) flush the queue. 5. **Backpressure = drop all but latest update per `data-key`.** A coalescing queue keyed by `(subscription_id, key)` collapses queued `update` frames; the latest wins. New frames of other kinds (`insert`/`delete`) flush the queue.
@ -73,7 +73,7 @@ ws.send_text(serde_json::to_string(&frame)?)?;
3. **DOM-patch test (jsdom).** `node crates/ui/runtime-tests/run.mjs` loads a stub HTML containing one `<wo:live>` block and a manifest, fakes a WebSocket emitting `snapshot` → `insert` → `update` → `delete` frames, and asserts each patch hits the right element. 3. **DOM-patch test (jsdom).** `node crates/ui/runtime-tests/run.mjs` loads a stub HTML containing one `<wo:live>` block and a manifest, fakes a WebSocket emitting `snapshot` → `insert` → `update` → `delete` frames, and asserts each patch hits the right element.
4. **Reconnect test.** Killing the fake WS triggers exponential backoff; on resume the runtime re-issues subscriptions and replaces the body with the new snapshot. 4. **Reconnect test.** Killing the fake WS triggers exponential backoff; on resume the runtime re-issues subscriptions and replaces the body with the new snapshot.
5. **Asset served.** Once phase 05 lands, `curl http://127.0.0.1:8080/_wo/runtime.js` returns the file with a stable `ETag` matching `sha256(RUNTIME_JS)`. 5. **Asset served.** Once phase 05 lands, `curl http://127.0.0.1:8080/_wo/runtime.js` returns the file with a stable `ETag` matching `sha256(RUNTIME_JS)`.
6. `cd reference/crates && cargo build && cargo test`. 6. `cd .dev/reference/crates && cargo build && cargo test`.
## Non-scope ## Non-scope
@ -101,7 +101,7 @@ test "$(wc -c < crates/ui/assets/wo-runtime.js)" -le 25600
cargo run --bin wo -- run docs/examples/blog & cargo run --bin wo -- run docs/examples/blog &
PID=$!; sleep 1; curl -fsS http://127.0.0.1:8080/ >/dev/null; kill $PID PID=$!; sleep 1; curl -fsS http://127.0.0.1:8080/ >/dev/null; kill $PID
cd reference/crates && cargo build && cargo test cd .dev/reference/crates && cargo build && cargo test
``` ```
## After this phase ## After this phase

View file

@ -102,12 +102,12 @@ assert_eq!(blog.apps().len(), 1);
3. **Component resolution.** `storefront.resolve_component("money")` returns the path to `shared/components/money.htmlx`. `storefront.resolve_component("nonsense")` errors as `ResolverError::NotFound`. 3. **Component resolution.** `storefront.resolve_component("money")` returns the path to `shared/components/money.htmlx`. `storefront.resolve_component("nonsense")` errors as `ResolverError::NotFound`.
4. **App-local override.** Adding `apps/storefront/ui/components/money.htmlx` makes `resolve_component("money")` return the app-local path; removing it falls back to the shared one. 4. **App-local override.** Adding `apps/storefront/ui/components/money.htmlx` makes `resolve_component("money")` return the app-local path; removing it falls back to the shared one.
5. **Degenerate form.** `Workspace::load(docs/examples/blog)` loads as a one-app workspace; `wo run docs/examples/blog` continues to start unchanged. 5. **Degenerate form.** `Workspace::load(docs/examples/blog)` loads as a one-app workspace; `wo run docs/examples/blog` continues to start unchanged.
6. `cd reference/crates && cargo build && cargo test`. 6. `cd .dev/reference/crates && cargo build && cargo test`.
## Non-scope ## Non-scope
- **No semver, no registry, no lockfile.** Path references only. - **No semver, no registry, no lockfile.** Path references only.
- **No `wo dev` hot-reload.** File watching against `apps/*/ui/` is deferred (would consume `reference/crates/wo-watch/`). - **No `wo dev` hot-reload.** File watching against `apps/*/ui/` is deferred (would consume `.dev/reference/crates/wo-watch/`).
- **No cross-workspace symlinks.** `shared = […]` paths must resolve under the workspace root. - **No cross-workspace symlinks.** `shared = […]` paths must resolve under the workspace root.
- **No build-time enforcement that an app touches only its declared shared dirs.** That's an integrity check for a later hardening phase. - **No build-time enforcement that an app touches only its declared shared dirs.** That's an integrity check for a later hardening phase.
- **No env-var interpolation in `wo.toml`.** `${VAR}` syntax stays out; runtime config comes through env vars at startup, not manifest time. - **No env-var interpolation in `wo.toml`.** `${VAR}` syntax stays out; runtime config comes through env vars at startup, not manifest time.
@ -131,7 +131,7 @@ cargo run --bin wo -- ls-apps docs/examples/blog
cargo run --bin wo -- run docs/examples/blog & cargo run --bin wo -- run docs/examples/blog &
PID=$!; sleep 1; curl -fsS http://127.0.0.1:8080/ >/dev/null; kill $PID PID=$!; sleep 1; curl -fsS http://127.0.0.1:8080/ >/dev/null; kill $PID
cd reference/crates && cargo build && cargo test cd .dev/reference/crates && cargo build && cargo test
``` ```
## After this phase ## After this phase

View file

@ -85,7 +85,7 @@ fn main() -> Result<()> {
3. **Storefront boots.** `WO_DB=wo://127.0.0.1:5555 STOREFRONT_DB_KEY=test ./target/wo/storefront &` then `curl -fsS http://127.0.0.1:8080/healthz` returns `200`. (The DB daemon from phase 06 is mocked or stubbed for this test if 06 hasn't landed yet — refuse-to-start without DB is the contract; the test verifies refuse-to-start when `WO_DB` is unset.) 3. **Storefront boots.** `WO_DB=wo://127.0.0.1:5555 STOREFRONT_DB_KEY=test ./target/wo/storefront &` then `curl -fsS http://127.0.0.1:8080/healthz` returns `200`. (The DB daemon from phase 06 is mocked or stubbed for this test if 06 hasn't landed yet — refuse-to-start without DB is the contract; the test verifies refuse-to-start when `WO_DB` is unset.)
4. **Admin builds separately.** `wo build apps/admin` produces a *different* binary with a disjoint route table. Diffing the two `app_config.rs` files shows different route lists. 4. **Admin builds separately.** `wo build apps/admin` produces a *different* binary with a disjoint route table. Diffing the two `app_config.rs` files shows different route lists.
5. **Refuse-to-start without DB.** `./target/wo/storefront` with no `WO_DB` and no manifest URL exits non-zero with a clear error. 5. **Refuse-to-start without DB.** `./target/wo/storefront` with no `WO_DB` and no manifest URL exits non-zero with a clear error.
6. `cd reference/crates && cargo build && cargo test`. 6. `cd .dev/reference/crates && cargo build && cargo test`.
## Non-scope ## Non-scope
@ -116,7 +116,7 @@ test -x target/wo/admin
cargo run --bin wo -- run docs/examples/blog & cargo run --bin wo -- run docs/examples/blog &
PID=$!; sleep 1; curl -fsS http://127.0.0.1:8080/ >/dev/null; kill $PID PID=$!; sleep 1; curl -fsS http://127.0.0.1:8080/ >/dev/null; kill $PID
cd reference/crates && cargo build && cargo test cd .dev/reference/crates && cargo build && cargo test
``` ```
## After this phase ## After this phase

View file

@ -1,6 +1,6 @@
# 06 — Shared database daemon (`wo db serve`) # 06 — Shared database daemon (`wo db serve`)
**Context sources:** [`./00-overview.md`](./00-overview.md) §§ "Goal" (L23), "Design decisions" 3 (L32), "Non-scope" (L201–203), [`./03-client-runtime.md`](./03-client-runtime.md) (the wire frames this daemon emits), [`./05-per-app-binaries.md`](./05-per-app-binaries.md) (the apps that connect), [`reference/crates/wo-sub/src/lib.rs`](../../../reference/crates/wo-sub/src/lib.rs) (the v1 subscription registry, 470 LOC, that needs generalising past `ByTitle`/`ByTag`/`All`), [`../../runtime/database/04-client-api.md`](../../runtime/database/04-client-api.md) (the wire-protocol owner). **Context sources:** [`./00-overview.md`](./00-overview.md) §§ "Goal" (L23), "Design decisions" 3 (L32), "Non-scope" (L201–203), [`./03-client-runtime.md`](./03-client-runtime.md) (the wire frames this daemon emits), [`./05-per-app-binaries.md`](./05-per-app-binaries.md) (the apps that connect), [`.dev/reference/crates/wo-sub/src/lib.rs`](../../../.dev/reference/crates/wo-sub/src/lib.rs) (the v1 subscription registry, 470 LOC, that needs generalising past `ByTitle`/`ByTag`/`All`), [`../../runtime/database/04-client-api.md`](../../runtime/database/04-client-api.md) (the wire-protocol owner).
## Goal ## Goal
@ -10,7 +10,7 @@ Stand up a headless daemon — `wo db serve` — that runs the engine + WAL + su
1. **Daemon = `crates/db` thin entrypoint + `crates/engine` + the wire acceptor.** No HTTP, no `.htmlx`, no `##ui`. The shared DB process knows nothing about the UI layer. 1. **Daemon = `crates/db` thin entrypoint + `crates/engine` + the wire acceptor.** No HTTP, no `.htmlx`, no `##ui`. The shared DB process knows nothing about the UI layer.
2. **API-key table is in-memory, env-seeded.** On startup the daemon reads `WO_DB_KEY_<APP>=<hex>` for each app declared in the workspace and builds an `AuthTable: HashMap<ApiKey, Principal>`. A `--keys <file>` flag is accepted but treated as a future hook. 2. **API-key table is in-memory, env-seeded.** On startup the daemon reads `WO_DB_KEY_<APP>=<hex>` for each app declared in the workspace and builds an `AuthTable: HashMap<ApiKey, Principal>`. A `--keys <file>` flag is accepted but treated as a future hook.
3. **Generalise `wo-sub`** from `Subscription::ByTitle/ByTag/All` to `Subscription::ByPredicate(TypeRef, Expr, SortKey)`. The v1 variants stay as legacy aliases (`ByTitle(t)` ⇒ `ByPredicate(Article, sys_title == t, _)`) for the blog regression test. Anchored in [`reference/crates/wo-sub/src/lib.rs`](../../../reference/crates/wo-sub/src/lib.rs) L8–17. 3. **Generalise `wo-sub`** from `Subscription::ByTitle/ByTag/All` to `Subscription::ByPredicate(TypeRef, Expr, SortKey)`. The v1 variants stay as legacy aliases (`ByTitle(t)` ⇒ `ByPredicate(Article, sys_title == t, _)`) for the blog regression test. Anchored in [`.dev/reference/crates/wo-sub/src/lib.rs`](../../../.dev/reference/crates/wo-sub/src/lib.rs) L8–17.
4. **Connection scope = `Principal { app, roles }` stored on the connection.** Every query evaluator reads it; phase 07 wires it into policy AND-composition. 4. **Connection scope = `Principal { app, roles }` stored on the connection.** Every query evaluator reads it; phase 07 wires it into policy AND-composition.
5. **One data dir, one engine, many connections.** Snapshot isolation by default (per `[database].isolation = "snapshot"` in the workspace `wo.toml`). 5. **One data dir, one engine, many connections.** Snapshot isolation by default (per `[database].isolation = "snapshot"` in the workspace `wo.toml`).
6. **Foreground-only this phase.** No daemonisation, no PID file, no signal handling beyond `SIGTERM` graceful shutdown. A future ops doc can add `wo db daemonize`. 6. **Foreground-only this phase.** No daemonisation, no PID file, no signal handling beyond `SIGTERM` graceful shutdown. A future ops doc can add `wo db daemonize`.
@ -24,7 +24,7 @@ Stand up a headless daemon — `wo db serve` — that runs the engine + WAL + su
| `crates/db/src/main.rs` | Entrypoint, arg parsing, env-key loading | new (~100 LOC) | | `crates/db/src/main.rs` | Entrypoint, arg parsing, env-key loading | new (~100 LOC) |
| `crates/db/src/server.rs` | Wire-protocol acceptor (TCP listener + per-conn handler) | new (~250 LOC) | | `crates/db/src/server.rs` | Wire-protocol acceptor (TCP listener + per-conn handler) | new (~250 LOC) |
| `crates/db/src/auth.rs` | `AuthTable`, `Principal`, key handshake | new (~120 LOC) | | `crates/db/src/auth.rs` | `AuthTable`, `Principal`, key handshake | new (~120 LOC) |
| `crates/sub/src/lib.rs` | Generalised subscription manager | port [`reference/crates/wo-sub/src/lib.rs`](../../../reference/crates/wo-sub/src/lib.rs) (470 LOC) + ~150 new | | `crates/sub/src/lib.rs` | Generalised subscription manager | port [`.dev/reference/crates/wo-sub/src/lib.rs`](../../../.dev/reference/crates/wo-sub/src/lib.rs) (470 LOC) + ~150 new |
| `crates/sub/src/predicate.rs` | Predicate evaluation against a row (uses `crates/ql` if available, else minimal subset) | new (~150 LOC) | | `crates/sub/src/predicate.rs` | Predicate evaluation against a row (uses `crates/ql` if available, else minimal subset) | new (~150 LOC) |
Total: ~1240 LOC (470 ported + ~770 new). Total: ~1240 LOC (470 ported + ~770 new).
@ -81,7 +81,7 @@ let id = subs.register(conn_fd, sub)?;
3. **Two principals.** Two clients connect, one with each API key; each receives a distinct `Principal` in the `WELCOME` frame. 3. **Two principals.** Two clients connect, one with each API key; each receives a distinct `Principal` in the `WELCOME` frame.
4. **Predicate subscription.** Client registers `Subscription::ByPredicate(Order, "status != Cancelled", "placed_at desc")`; the manager returns a fresh `subscription_id`; on a stub `Order` insert, the matching client receives an `insert` frame. 4. **Predicate subscription.** Client registers `Subscription::ByPredicate(Order, "status != Cancelled", "placed_at desc")`; the manager returns a fresh `subscription_id`; on a stub `Order` insert, the matching client receives an `insert` frame.
5. **v1 regression.** A connection running the legacy `Subscription::ByTitle("hello-world")` against the blog corpus still produces notifications via the legacy alias. 5. **v1 regression.** A connection running the legacy `Subscription::ByTitle("hello-world")` against the blog corpus still produces notifications via the legacy alias.
6. `cd reference/crates && cargo build && cargo test`. 6. `cd .dev/reference/crates && cargo build && cargo test`.
## Non-scope ## Non-scope
@ -112,7 +112,7 @@ kill $DB_PID
# legacy v1 path # legacy v1 path
cargo test -p sub --test legacy_by_title cargo test -p sub --test legacy_by_title
cd reference/crates && cargo build && cargo test cd .dev/reference/crates && cargo build && cargo test
``` ```
## After this phase ## After this phase

View file

@ -65,7 +65,7 @@ assert!(rs.contains(Role::Ops));
3. **Build-time domain check fires.** A test workspace where `apps/storefront/app.wo` declares `role: Anonymous` against a type whose global policy does not define `Anonymous` — `wo build apps/storefront` exits non-zero with `PolicyDomainError`. 3. **Build-time domain check fires.** A test workspace where `apps/storefront/app.wo` declares `role: Anonymous` against a type whose global policy does not define `Anonymous` — `wo build apps/storefront` exits non-zero with `PolicyDomainError`.
4. **Cross-app integration.** Two storefront customers issue the same `GET /api/orders` against the daemon; each sees only their own rows (storefront app-scope narrows global). Admin sees both. Test runs against the phase-06 daemon. 4. **Cross-app integration.** Two storefront customers issue the same `GET /api/orders` against the daemon; each sees only their own rows (storefront app-scope narrows global). Admin sees both. Test runs against the phase-06 daemon.
5. **v1 regression.** `wo run docs/examples/blog` boots; the global `policy read for anyone when published == true` on the blog `Article` type continues to gate anonymous reads as it does today. 5. **v1 regression.** `wo run docs/examples/blog` boots; the global `policy read for anyone when published == true` on the blog `Article` type continues to gate anonymous reads as it does today.
6. `cd reference/crates && cargo build && cargo test`. 6. `cd .dev/reference/crates && cargo build && cargo test`.
## Non-scope ## Non-scope
@ -97,7 +97,7 @@ curl -fsS http://127.0.0.1:8080/api/articles # only published r
test -z "$(curl -fsS http://127.0.0.1:8080/api/articles | grep '"published":false')" test -z "$(curl -fsS http://127.0.0.1:8080/api/articles | grep '"published":false')"
kill $PID kill $PID
cd reference/crates && cargo build && cargo test cd .dev/reference/crates && cargo build && cargo test
``` ```
## After this phase ## After this phase

View file

@ -1,6 +1,6 @@
# 08 — MVC structure: model = class, view = htmlx + scss, controller = .wo # 08 — MVC structure: model = class, view = htmlx + scss, controller = .wo
**Context sources:** [`reference/writeonce-app/src/app/`](../../../../reference/writeonce-app/src/app/) (the v1 Angular app whose component anatomy this formalizes), [`./00-overview.md`](./00-overview.md) ("Angular-component-style layout" — `home/{home.wo, home.htmlx, home.css}`), [`./01-htmlx-format-spec.md`](./01-htmlx-format-spec.md) (the view grammar: Mustache + `<wo:live>` + `wo:bind`), [`./02-ui-compiler.md`](./02-ui-compiler.md), [`./03-client-runtime.md`](./03-client-runtime.md), [`../../13-class-model-live-pricing.md`](../../13-class-model-live-pricing.md) (the class methods controllers call), [`../../../examples/pricing/ui/pricing/`](../../../examples/pricing/ui/pricing/) (the reference screen). **Context sources:** [`.dev/reference/writeonce-app/src/app/`](../../../../.dev/reference/writeonce-app/src/app/) (the v1 Angular app whose component anatomy this formalizes), [`./00-overview.md`](./00-overview.md) ("Angular-component-style layout" — `home/{home.wo, home.htmlx, home.css}`), [`./01-htmlx-format-spec.md`](./01-htmlx-format-spec.md) (the view grammar: Mustache + `<wo:live>` + `wo:bind`), [`./02-ui-compiler.md`](./02-ui-compiler.md), [`./03-client-runtime.md`](./03-client-runtime.md), [`../../13-class-model-live-pricing.md`](../../13-class-model-live-pricing.md) (the class methods controllers call), [`../../../examples/pricing/ui/pricing/`](../../../examples/pricing/ui/pricing/) (the reference screen).
## Goal ## Goal
@ -15,7 +15,7 @@ ui/pricing/
## The mapping, against the v1 Angular app ## The mapping, against the v1 Angular app
| MVC role | v1 Angular (`reference/writeonce-app/src/app/`) | writeonce | | MVC role | v1 Angular (`.dev/reference/writeonce-app/src/app/`) | writeonce |
| --- | --- | --- | | --- | --- | --- |
| **Model** | `models/article.ts` (interface) + `services/article.service.ts` (HTTP fetch) | the `class` / `type` declaration itself (`types/product.wo`). No service layer: the database is in-process, and a model binding **is** a query — `LIVE select` for push, `select` for snapshot | | **Model** | `models/article.ts` (interface) + `services/article.service.ts` (HTTP fetch) | the `class` / `type` declaration itself (`types/product.wo`). No service layer: the database is in-process, and a model binding **is** a query — `LIVE select` for push, `select` for snapshot |
| **View** | `article.component.html` + `article.component.css` | `pricing.htmlx` + `pricing.scss`. Plain markup; the only dynamic constructs are Mustache paths and `<wo:live>` / `wo:bind` from [`01-htmlx-format-spec.md`](./01-htmlx-format-spec.md) | | **View** | `article.component.html` + `article.component.css` | `pricing.htmlx` + `pricing.scss`. Plain markup; the only dynamic constructs are Mustache paths and `<wo:live>` / `wo:bind` from [`01-htmlx-format-spec.md`](./01-htmlx-format-spec.md) |
@ -72,7 +72,7 @@ browser action wo:action="set-price"
## Migration note ## Migration note
The two existing screen specs (`docs/examples/ecommerce/apps/*/ui/*/`, single-file `##ui` shorthand) stay valid under decision 5. New screens — starting with [`docs/examples/pricing/ui/pricing/`](../../../examples/pricing/ui/pricing/) — use the triplet. The v1 Angular app stays archived; its components are the *shape* reference, not a port source (the htmlx port source remains `reference/crates/wo-htmlx`). The two existing screen specs (`docs/examples/ecommerce/apps/*/ui/*/`, single-file `##ui` shorthand) stay valid under decision 5. New screens — starting with [`docs/examples/pricing/ui/pricing/`](../../../examples/pricing/ui/pricing/) — use the triplet. The v1 Angular app stays archived; its components are the *shape* reference, not a port source (the htmlx port source remains `.dev/reference/crates/wo-htmlx`).
## Exit criteria (implementation sequenced in [plan 14](../../14-mvc-ui-implementation.md), landing with plan 13d) ## Exit criteria (implementation sequenced in [plan 14](../../14-mvc-ui-implementation.md), landing with plan 13d)

View file

@ -44,7 +44,7 @@ Each transition is reversible — flip one feature flag or swap one trait object
## Proposed Crate Layout ## Proposed Crate Layout
Port the C++ prototype (`prototypes/wo-db/src/*`) to Rust, split along the natural seams. New-runtime crates are unprefixed; v1 crates keep `wo-` in `reference/crates/`. Port the C++ prototype (`prototypes/wo-db/src/*`) to Rust, split along the natural seams. New-runtime crates are unprefixed; v1 crates keep `wo-` in `.dev/reference/crates/`.
| Crate | Purpose | Prototype source | Phase | | Crate | Purpose | Prototype source | Phase |
| --- | --- | --- | --- | | --- | --- | --- | --- |
@ -60,9 +60,9 @@ Port the C++ prototype (`prototypes/wo-db/src/*`) to Rust, split along the natur
All 15 crates (these 14 plus the existing `rt` binary crate) now exist as empty skeletons in `crates/`. See [`crates/README.md`](../../../crates/README.md) and [`docs/plan/done/01-scafolding-crates.md`](../../plan/done/01-scafolding-crates.md) for the scaffolding plan that landed them. All 15 crates (these 14 plus the existing `rt` binary crate) now exist as empty skeletons in `crates/`. See [`crates/README.md`](../../../crates/README.md) and [`docs/plan/done/01-scafolding-crates.md`](../../plan/done/01-scafolding-crates.md) for the scaffolding plan that landed them.
Today's `reference/crates/wo-seg` and `reference/crates/wo-index` remain in the v1 nested workspace for the entire migration window. They disappear only at the end of Phase D. Today's `.dev/reference/crates/wo-seg` and `.dev/reference/crates/wo-index` remain in the v1 nested workspace for the entire migration window. They disappear only at the end of Phase D.
`reference/crates/wo-store` evolves but survives — it becomes the writeonce-specific glue layer (trait, article domain model, content-directory cold-start) whose backend is swappable. `.dev/reference/crates/wo-store` evolves but survives — it becomes the writeonce-specific glue layer (trait, article domain model, content-directory cold-start) whose backend is swappable.
## Phase A — Abstract the Article Store ## Phase A — Abstract the Article Store

View file

@ -5,7 +5,7 @@
**AS** a developer building and operating my own products end to end **AS** a developer building and operating my own products end to end
**I WANT** a new programming language — with arithmetic, ownership-based memory safety, and garbage collection where I opt in — whose compiler, runtime, and database ship as a single never-stopping Linux binary that can update its own code in place **I WANT** a new statically-typed systems programming language — object-oriented by default, with ownership-based memory safety (borrow semantics) and per-class `@gc` garbage collection where I opt in — whose compiler, runtime, and database ship as a single never-stopping Linux binary that can update its own code in place
**TO** write an application once and run it forever: no external stack to assemble, no database server to operate, and deployments that swap code inside the running process with instant rollback. **TO** write an application once and run it forever: no external stack to assemble, no database server to operate, and deployments that swap code inside the running process with instant rollback.
@ -17,6 +17,15 @@
- Memory safety without a GC tax: Rust-shaped borrowing (single owner, - Memory safety without a GC tax: Rust-shaped borrowing (single owner,
second-class borrows) checked mostly at compile time, with per-class `@gc` second-class borrows) checked mostly at compile time, with per-class `@gc`
opt-in collected per shard — no global pause exists by construction. opt-in collected per shard — no global pause exists by construction.
- Static typing all the way down: every slot's type is known at compile
time, so the VM runs untagged 64-bit registers — no `Dynamic`, no boxing
tax, no hashed field lookups (the Haxe→C++ reference workload pays all
three).
- The database is the object model: a `@table` class is a table, a
`ref`/`multi` field is a relation — the `@`-annotations are the built-in
ORM, resolved at compile time. No external mapping layer, and no
user-defined macros (the verdict table's rejection stands; annotations
are compiler-known).
- Updates are blue-green **inside** the runtime: propose, approve, compile - Updates are blue-green **inside** the runtime: propose, approve, compile
in-process, atomic switch, previous version resident for instant rollback. in-process, atomic switch, previous version resident for instant rollback.
- The runtime is a recipe box: once language + runtime + database exist, a - The runtime is a recipe box: once language + runtime + database exist, a
@ -25,9 +34,10 @@
## Background & Constraints ## Background & Constraints
writeonce today is a declarative Rust-based runtime (Stage 2). This story writeonce began as a declarative Rust-based runtime (Stage 2); that chapter
evolves it into an object-oriented language (`woc` OCaml compiler, `wovm` C closes here. This story pivots it into a statically-typed, object-oriented
VM) per the approved specs: C as the runtime's basis (libc only), no **systems programming language** with its own runtime and database (`woc`
OCaml compiler, `wovm` C VM) per the approved specs: C as the runtime's basis (libc only), no
inheritance ever, mutable value semantics for borrowing, shard-per-core inheritance ever, mutable value semantics for borrowing, shard-per-core
concurrency with ownership-moving messages, RAM-authoritative data under a concurrency with ownership-moving messages, RAM-authoritative data under a
WAL, and the blue-green VM pair for in-runtime deployment. Target OS is WAL, and the blue-green VM pair for in-runtime deployment. Target OS is
@ -36,21 +46,33 @@ parity. Constraints: OCaml stdlib only, C libc only; docs live under
`docs/`; prose-only planning artifacts (no implementation code in stories or `docs/`; prose-only planning artifacts (no implementation code in stories or
iterations); no commits by agents — drafts go to `.dev/commit.md`. iterations); no commits by agents — drafts go to `.dev/commit.md`.
## Iterations (review in this order) ## Iterations (review in this order — the order is chronological)
| # | Iteration | Delivers | | # | Status | Iteration | Delivers |
| --- | --- | --- | | --- | --- | --- | --- |
| 1 | [Principles doc](01-principles-doc.md) | `docs/00-principles.md` — the doctrine page every later slice links back to | | 1 | ✅ | [Principles doc](01-principles-doc.md) | `docs/00-principles.md` — the doctrine page every later slice links back to |
| 2 | [VM core](02-vm-core.md) | `wovm`: `.wob` loader, register interpreter, arena, borrow word, `@gc` collector | | 2 | 🔄 | [VM core](02-vm-core.md) | `wovm`: `.wob` loader, register interpreter, arena, borrow word, `@gc` RC (cycles staged to 8) |
| 3 | [Compiler front](03-compiler-front.md) | `woc`: lexer → parser → typechecker → ownership pass, diagnostics | | 3 | 🔄 | [Compiler front](03-compiler-front.md) | `woc`: lexer → parser → typechecker → ownership pass, diagnostics |
| 4 | [Single binary end-to-end](04-single-binary-e2e.md) | emitter + conformance corpus + `woc build` self-contained binary | | 4 | ⬜ | [Single binary end-to-end](04-single-binary-e2e.md) | emitter + conformance corpus + `woc build` self-contained binary |
| 5 | [Language surface](05-language-surface.md) | Haxe-parity adoptions: switch, records, optionals, try/catch, statics, modules… | | 5 | ⬜ | [Language surface](05-language-surface.md) | Haxe-parity adoptions: switch, records, optionals, try/catch, statics, modules… |
| 6 | [Program mode + stdlib](06-program-mode-stdlib.md) | `fn main`, exit codes, `fs`/`proc`/`net`/`time`/`json` builtins | | 6 | ⬜ | [Program mode + stdlib](06-program-mode-stdlib.md) | `fn main`, exit codes, `fs`/`proc`/`net`/`time`/`json` builtins |
| 7 | [log-watcher proof](07-logwatcher-proof.md) | the driving workload compiled and detecting silent deaths live | | 7 | ⬜ | [log-watcher proof](07-logwatcher-proof.md) | the driving workload compiled and detecting silent deaths live |
| 8 | [Shard-actor runtime](08-shard-actor-runtime.md) | thread-per-core shards, per-shard heaps, ownership-move messaging | | 8 | ⬜ | [Shard-actor runtime](08-shard-actor-runtime.md) | thread-per-core shards, per-shard heaps, ownership-move messaging, `@gc` cycle collector |
| 9 | [Database engine](09-database-engine.md) | class-shaped tables, typed WAL + recovery, `insert`/`select` execute | | 9 | ⬜ | [Database engine](09-database-engine.md) | class-shaped tables, typed WAL + recovery, `insert`/`select` execute |
| 10 | [HTTP service layer](10-http-service.md) | `service` blocks route to VM methods; REST parity with Stage 2 | | 10 | ⬜ | [HTTP service layer](10-http-service.md) | `service` blocks route to VM methods; REST parity with Stage 2 |
| 11 | [Blue-green deploy](11-blue-green-deploy.md) | two VM slots, in-runtime compile, atomic switch, resident rollback | | 11 | ⬜ | [Fibers](11-fibers.md) | green threads on the shard scheduler: reduction-budget preemption, park-on-I/O builtins, ownership-move sends |
| 12 | ⬜ | [Blue-green deploy](12-blue-green-deploy.md) | two VM slots, in-runtime compile, atomic switch, resident rollback |
Chronology notes: 2 and 3 are the one deliberate overlap — the OOP spec's
first sub-project builds both halves concurrently (both in progress:
`runtime/src/` carries the VM's loader/vm/obj/borrow/gc modules,
`compiler/` has front-end tasks 1–4 of 8); they meet in 4. Iterations 1–7
complete the single-shard language; 8–12 scale the runtime (8 is the
substrate for everything after: the DB engine lives in its shards, HTTP
serves from them, fibers refine their scheduler, blue-green drains them).
Fibers (11) land before blue-green so the deploy's drain can unwind parked
fibers as part of its own acceptance; blue-green (12) closes the story —
its plan is authored only after 9–10 ship.
Review protocol: the developer reads one iteration, approves or amends; Review protocol: the developer reads one iteration, approves or amends;
the next starts only after approval. Each iteration is an unsplittable the next starts only after approval. Each iteration is an unsplittable
@ -63,7 +85,7 @@ list, and a pointer to the plan document that already sequences its tasks.
- Systems track spec: `docs/superpowers/specs/2026-08-01-systems-track-design.md` - Systems track spec: `docs/superpowers/specs/2026-08-01-systems-track-design.md`
- Blue-green spec: `docs/superpowers/specs/2026-08-03-blue-green-vm-design.md` - Blue-green spec: `docs/superpowers/specs/2026-08-03-blue-green-vm-design.md`
- Sample+principles spec: `docs/superpowers/specs/2026-08-07-logwatcher-sample-and-principles-design.md` - Sample+principles spec: `docs/superpowers/specs/2026-08-07-logwatcher-sample-and-principles-design.md`
- Plan documents: `docs/superpowers/plans/` (plans 1–10), `compiler/plan/` (2, 3, 8 + architecture) - Plan documents: `docs/superpowers/plans/` (plans 1–10), `docs/plan/compiler/` (2, 3, 8 + architecture)
- Roadmap map: `docs/08-project-structure.md` (build sequence) - Roadmap map: `docs/08-project-structure.md` (build sequence)
- Behavioral reference workload: `~/projects/log-watcher` (Haxe daemon) - Behavioral reference workload: `~/projects/log-watcher` (Haxe daemon)

View file

@ -5,16 +5,18 @@
## Goals ## Goals
- The repo gains `docs/00-principles.md`: one page stating the twelve - The repo gains `docs/00-principles.md`: one page stating the thirteen
writeonce principles — the doctrine every later iteration links back to writeonce principles — the doctrine every later iteration links back to
instead of re-arguing. instead of re-arguing.
- The twelve: one binary is the whole system; zero dependencies (kernel - The thirteen: one binary is the whole system; zero dependencies (kernel
primitives only); memory safety without a GC tax (MVS ownership, opt-in primitives only); memory safety without a GC tax (MVS ownership, opt-in
`@gc`, per-shard collection); no inheritance ever; thread-per-core shards `@gc`, per-shard collection); no inheritance ever; thread-per-core shards
with ownership moves; the runtime never stops (blue-green slots, embedded with ownership moves; the runtime never stops (blue-green slots, embedded
source); RAM authoritative + WAL durable; samples force the grammar; source); RAM authoritative + WAL durable; samples force the grammar;
Linux is the target; capabilities are typed builtins (no FFI); plain Linux is the target; capabilities are typed builtins (no FFI); plain
diagnostics are the product; the runtime is a recipe box. diagnostics are the product; the runtime is a recipe box; statically
typed all the way to the register (no `Dynamic`, untagged VM slots,
annotations as the compile-time ORM).
## Acceptance Criteria ## Acceptance Criteria
@ -44,6 +46,7 @@
## Proposed Solution ## Proposed Solution
- Author the page with the twelve principles in the order listed in the - Author the page with the thirteen principles in the order listed in the
governing spec; verify every link resolves; add the CLAUDE.md pointer governing spec (the thirteenth — static typing — added 2026-08-08 by
story amendment); verify every link resolves; add the CLAUDE.md pointer
line; record the commit draft in `.dev/commit.md`. line; record the commit draft in `.dev/commit.md`.

View file

@ -8,7 +8,8 @@
- A C, libc-only virtual machine that loads a `.wob` bytecode module and - A C, libc-only virtual machine that loads a `.wob` bytecode module and
executes method calls with the language's memory model enforced: owned executes method calls with the language's memory model enforced: owned
objects with deterministic drops, runtime borrow checks at residual objects with deterministic drops, runtime borrow checks at residual
sites, and `@gc` classes collected without stop-the-world pauses. sites, and `@gc` classes reference-counted — **RC only in this
iteration**; the cycle collector is staged to iteration 8 (see Info).
- Arithmetic and text operations execute correctly — the language's first - Arithmetic and text operations execute correctly — the language's first
observable behavior. observable behavior.
@ -27,21 +28,35 @@
every drop, and ASan/Valgrind report zero leaks on both success and every drop, and ASan/Valgrind report zero leaks on both success and
trap paths. trap paths.
- What to achieve? - What to achieve?
- **Given** a cyclic `@gc` object graph that becomes garbage, - **Given** `@gc` objects whose aliases are created and dropped,
- **when** the collector's budgeted ticks run, - **when** the last reference drops (`rc == 0`),
- **then** the cycle is freed within the configured budget and no pause - **then** the object frees immediately, elided RC pairs stay elided,
exceeds the configured slice. and ASan/Valgrind report zero leaks on every acyclic fixture.
- What to achieve?
- **Given** a cyclic `@gc` graph that becomes garbage,
- **when** the corpus runs,
- **then** the leak is *expected and asserted* by a must-leak fixture —
the recorded debt iteration 8's cycle collector retires.
## Out Of Scope ## Out Of Scope
- The OCaml compiler (iteration 3) — fixtures here are assembled by the - The OCaml compiler (iteration 3) — fixtures here are assembled by the
test tool, not compiled. test tool, not compiled.
- Threads, shards, mailboxes (iteration 8); any DB or HTTP capability. - Threads, shards, mailboxes (iteration 8); any DB or HTTP capability.
- The Bacon–Rajan cycle collector — staged to iteration 8, where the
shard's event loop (its per-tick budget host) first exists. Until then a
cyclic `@gc` graph leaks, documented and fixture-asserted.
## Info ## Info
- The 16-byte object header reserves a shard id now so iteration 8 needs no - The 16-byte object header reserves a shard id now so iteration 8 needs no
relayout. relayout.
- Staging rationale (decided 2026-08-08): trial deletion under mutation is
the subtlest piece of this iteration, while every sample to date needs
zero cyclic `@gc` (log-watcher: none; pricing: one acyclic cache). Swift
ships RC-without-cycles at mass scale. The header's `IN_CYCLE_BUF` flag
bit and the possible-cycle buffer hook stay reserved, so iteration 8
adds the scan without relayout or opcode changes.
- Format contract: `docs/plan/oop-vm/00-wob-format.md` twinned with - Format contract: `docs/plan/oop-vm/00-wob-format.md` twinned with
`runtime/src/wob.h`; ~40-op register instruction set, computed-goto `runtime/src/wob.h`; ~40-op register instruction set, computed-goto
dispatch. dispatch.
@ -49,6 +64,7 @@
## Proposed Solution ## Proposed Solution
- Execute the existing plan: `docs/superpowers/plans/2026-08-01-wob-format-and-vm-core.md` - Execute the existing plan: `docs/superpowers/plans/2026-08-01-wob-format-and-vm-core.md`
(16 TDD tasks: arena, object model, borrow word, RC + cycle collector, (16 TDD tasks: arena, object model, borrow word, RC, test assembler,
test assembler, validating loader, interpreter, drop-map unwinding, validating loader, interpreter, drop-map unwinding, builtins, CLI +
builtins, CLI + `just` gate). `just` gate) — with its cycle-collector task deferred: that task moves
to iteration 8's plan, replaced here by the must-leak cycle fixture.

View file

@ -35,7 +35,7 @@
## Out Of Scope ## Out Of Scope
- Anything beyond the milestone grammar (iteration 5 grows the surface). - Anything beyond the milestone grammar (iteration 5 grows the surface).
- Hot reload / deployment mechanics (iteration 11 — but the self-exec - Hot reload / deployment mechanics (iteration 12 — but the self-exec
trailer this iteration ships is its foundation). trailer this iteration ships is its foundation).
## Info ## Info
@ -48,7 +48,7 @@
## Proposed Solution ## Proposed Solution
- Execute the existing plan: `compiler/plan/2026-08-01-wob-emit-e2e-single-binary.md` - Execute the existing plan: `docs/plan/compiler/2026-08-01-wob-emit-e2e-single-binary.md`
(emitter with ownership lowering + drop maps + vtables, corpus harness, (emitter with ownership lowering + drop maps + vtables, corpus harness,
pricing corpus, ownership/trap corpora, gc pump e2e, `woc build` trailer, pricing corpus, ownership/trap corpora, gc pump e2e, `woc build` trailer,
`just oop-accept` gate over the five spec success criteria). `just oop-accept` gate over the five spec success criteria).

View file

@ -48,6 +48,6 @@
## Proposed Solution ## Proposed Solution
- Execute the existing plan: `compiler/plan/2026-08-01-haxe-parity-language.md` - Execute the existing plan: `docs/plan/compiler/2026-08-01-haxe-parity-language.md`
(nine tasks, each shipping its fixtures and error-catalog entries in the (nine tasks, each shipping its fixtures and error-catalog entries in the
same task). same task).

View file

@ -11,6 +11,10 @@
state never exists. state never exists.
- The language grows `spawn` and message send; garbage collection stays - The language grows `spawn` and message send; garbage collection stays
per-shard, so no global pause appears at any core count. per-shard, so no global pause appears at any core count.
- The `@gc` story completes here: the Bacon–Rajan cycle collector (staged
out of iteration 2) lands on the shard's own event loop — the per-tick
budget host it was always specified to run on — retiring iteration 2's
documented cycle leak.
## Acceptance Criteria ## Acceptance Criteria
@ -29,11 +33,17 @@
- **when** code attempts to send it cross-shard, - **when** code attempts to send it cross-shard,
- **then** the compiler rejects it — aliased references cannot cross - **then** the compiler rejects it — aliased references cannot cross
heap boundaries. heap boundaries.
- What to achieve?
- **Given** a cyclic `@gc` object graph that becomes garbage,
- **when** the shard's budgeted collection ticks run,
- **then** the cycle is freed within the configured budget, no pause
exceeds the configured slice, and iteration 2's must-leak fixture
flips to must-collect.
## Out Of Scope ## Out Of Scope
- Fibers/green threads (recorded in the blue-green vision §3; extends this - Fibers/green threads — iteration 11 extends this scheduler; research at
scheduler later). `docs/plan/exploration/fibers/00-fibers.md`.
- Cross-shard transactions (the database iteration's 2PC concern, later). - Cross-shard transactions (the database iteration's 2PC concern, later).
## Info ## Info
@ -48,4 +58,6 @@
- Execute the existing plan: `docs/superpowers/plans/2026-08-01-shard-actor-vm-runtime.md` - Execute the existing plan: `docs/superpowers/plans/2026-08-01-shard-actor-vm-runtime.md`
(pinned-worker scheduler, shard-stamped heaps, MPSC mailbox rings + mail (pinned-worker scheduler, shard-stamped heaps, MPSC mailbox rings + mail
eventfds, send-as-move with home-routed frees, gc pacing per tick, eventfds, send-as-move with home-routed frees, gc pacing per tick,
spawn/send surface, actor corpus). spawn/send surface, actor corpus) — plus the cycle-collector task
adopted from iteration 2's plan: possible-cycle buffer on RC decrement,
trial-deletion scan under the per-tick budget.

View file

@ -6,8 +6,10 @@
## Goals ## Goals
- The language's oldest promise executes on the new runtime: every class is - The language's oldest promise executes on the new runtime: every class is
a table. `insert` and `select` stop trapping (`DB_STUB` retires) and run a table, every `ref`/`multi` field a relation — the `@`-annotations
against class-shaped row storage inside the VM's shards. (`@table`, `@unique`) are the built-in ORM, mapped at compile time with no
external layer. `insert` and `select` stop trapping (`DB_STUB` retires)
and run against class-shaped row storage inside the VM's shards.
- Data survives anything: a typed write-ahead log with ack-after-fsync, - Data survives anything: a typed write-ahead log with ack-after-fsync,
parallel boot replay, and a crash battery proving no committed row is parallel boot replay, and a crash battery proving no committed row is
ever lost and no half-applied transaction ever visible. ever lost and no half-applied transaction ever visible.
@ -45,6 +47,11 @@
- Doctrine: RAM is authoritative; the WAL makes it durable; indexes drift - Doctrine: RAM is authoritative; the WAL makes it durable; indexes drift
unless writes go through the row API — the Rust runtime learned this unless writes go through the row API — the Rust runtime learned this
lesson, the C engine enforces it. lesson, the C engine enforces it.
- Annotations, not macros: `@table`/`@unique` are compiler-known and
resolved at compile time; user-defined macros stay rejected (systems-track
verdict table). A `ref T` field compiles to a typed row id (FK); `multi T`
to the owning-side collection edge — the relation model the examples
(pricing's `Product`/`Price`) already write.
- The wo-db overlap manifest keeps the C++ prototype and this engine - The wo-db overlap manifest keeps the C++ prototype and this engine
answer-compatible where features overlap. answer-compatible where features overlap.

View file

@ -16,7 +16,7 @@
- What to achieve? - What to achieve?
- **Given** the blog sample's `service` declarations, - **Given** the blog sample's `service` declarations,
- **when** the compiled binary boots and `reference/rest/blog.rest` - **when** the compiled binary boots and `.dev/reference/rest/blog.rest`
runs against it, runs against it,
- **then** every request in the smoke file answers as documented — - **then** every request in the smoke file answers as documented —
including the intentional 501/405/404 responses. including the intentional 501/405/404 responses.
@ -36,7 +36,7 @@
## Out Of Scope ## Out Of Scope
- WebSocket/live subscriptions and UI (the parked `##ui` story). - WebSocket/live subscriptions and UI (the parked `##ui` story).
- The management plane endpoints (iteration 11 builds them on this - The management plane endpoints (iteration 12 builds them on this
machinery). machinery).
## Info ## Info

View file

@ -1,65 +0,0 @@
# Iteration 11 — blue-green in-runtime deployment
> Format: `product/story-iteration-template`. Part of
> [Story — one language, one runtime, one database, one binary](00-story.md).
## Goals
- The story's closing promise: the running binary updates its own code.
Two fixed VM slots (activity alternating); a proposal pipeline —
propose → approve → in-runtime compile → additive schema migration →
load → health → atomic switch — with the previous version staying
resident as the instant rollback target.
- The developer drives it remotely: `wo remote pull / propose / diff /
approve / rollback / status` over a loopback management transport,
every stage streaming live and WAL-audited.
## Acceptance Criteria
- What to achieve?
- **Given** a running fixture app and an additive code+schema change,
- **when** the developer proposes and approves it,
- **then** the deploy completes with zero dropped requests (in-flight
work drains on the old slot), and the binary's embedded source
trailer matches the new active version afterward.
- What to achieve?
- **Given** a failure at any pipeline stage (compile diagnostic,
destructive-change rejection, load failure, health failure, drain
timeout),
- **when** it occurs,
- **then** the active slot keeps serving untouched, the failure is
visible in the SSE stream and the WAL trail, and a destructive
change was rejected at propose time naming the offending
declaration.
- What to achieve?
- **Given** a completed deploy,
- **when** `wo remote rollback` runs,
- **then** the previous version serves again in under one second with
no compile and no data change — additive-only migration guarantees
old code runs correctly against the migrated schema.
- What to achieve?
- **Given** kill -9 during COMPILING / MIGRATING / SWITCHING /
trailer-rewrite,
- **when** the unit restarts,
- **then** it serves one consistent version and the WAL shows whole
migrations only.
## Out Of Scope
- Script-based/destructive migrations, in-runtime editing workspace,
MCP/agent wrapper over the management plane — all recorded follow-ups.
- Fibers (the vision's §3; a scheduler concern, not a deploy concern).
## Info
- Approved spec: `docs/superpowers/specs/2026-08-03-blue-green-vm-design.md`;
its implementation plan is deliberately authored only after iterations
9–10 ship (prerequisites: a catalog to diff, HTTP machinery to build on).
- VMs own code; the engine owns data — the separation that makes the
switch cheap and rollback unconditional.
## Proposed Solution
- Author the implementation plan from the approved spec once iterations
9–10 land, then execute it (slots, deploy state machine, additive
differ, management surface + SSE, `wo remote` verbs, crash battery).

View file

@ -4,16 +4,16 @@
hello: hello:
cargo run --bin wo -- run docs/examples/hello cargo run --bin wo -- run docs/examples/hello
# C runtime reference (prototypes/wo-rt-c): build, serve, CRUD round-trip, shut down # C runtime reference (runtime/): build, serve, CRUD round-trip, shut down
# Phase A: thread-per-core — each connection hashes to one shard (SO_REUSEPORT), # Phase A: thread-per-core — each connection hashes to one shard (SO_REUSEPORT),
# so a list may land on a different shard than the create. The counters on / # so a list may land on a different shard than the create. The counters on /
# show the spread. WO_THREADS=4 keeps the demo output readable. # show the spread. WO_THREADS=4 keeps the demo output readable.
rt-c-demo port="8085" threads="4": rt-c-demo port="8085" threads="4":
#!/usr/bin/env bash #!/usr/bin/env bash
set -euo pipefail set -euo pipefail
make -C prototypes/wo-rt-c make -C runtime
data=$(mktemp -d /tmp/wo-demo-XXXXXX) data=$(mktemp -d /tmp/wo-demo-XXXXXX)
WO_PORT={{port}} WO_THREADS={{threads}} WO_DATA=$data ./prototypes/wo-rt-c/wo-rt & WO_PORT={{port}} WO_THREADS={{threads}} WO_DATA=$data ./runtime/wo-rt &
server=$! server=$!
trap 'kill $server 2>/dev/null; sleep 0.3; rm -rf $data' EXIT trap 'kill $server 2>/dev/null; sleep 0.3; rm -rf $data' EXIT
base=http://127.0.0.1:{{port}} base=http://127.0.0.1:{{port}}
@ -26,18 +26,36 @@ rt-c-demo port="8085" threads="4":
curl -s "$base/api/notes"; echo curl -s "$base/api/notes"; echo
echo "--- spread:"; curl -s "$base/"; echo echo "--- spread:"; curl -s "$base/"; echo
# wovm VM core (runtime/src): build the binary
wovm-build:
make -C runtime wovm
# wovm full gate: unit suite (ASan+UBSan), ISO dispatch flavor, CLI smoke
wovm-test:
make -C runtime test
make -C runtime test-iso
bash runtime/test/cli_smoke.sh
# woc compiler front (compiler/): build the executable
woc-build:
dune build --root compiler
# woc gate: unit tests (test_diag) + golden suite (runner, WOC_BLESS=1 to update)
woc-test:
dune runtest --root compiler
# phase-F benchmark: reads, durable writes, 10k idle conns (scaled geometry) # phase-F benchmark: reads, durable writes, 10k idle conns (scaled geometry)
rt-c-bench port="8085" threads="8" conns="64": rt-c-bench port="8085" threads="8" conns="64":
#!/usr/bin/env bash #!/usr/bin/env bash
set -euo pipefail set -euo pipefail
make -C prototypes/wo-rt-c clean >/dev/null make -C runtime clean >/dev/null
make -C prototypes/wo-rt-c CFLAGS="-O2 -Wall -Wextra -std=c11 -DSLOTS_PER_SHARD=262144" wo-rt bench >/dev/null make -C runtime CFLAGS="-O2 -Wall -Wextra -std=c11 -DSLOTS_PER_SHARD=262144" wo-rt bench >/dev/null
data=$(mktemp -d /tmp/wo-bench-XXXXXX) data=$(mktemp -d /tmp/wo-bench-XXXXXX)
WO_PORT={{port}} WO_THREADS={{threads}} WO_DATA=$data ./prototypes/wo-rt-c/wo-rt >/dev/null 2>&1 & WO_PORT={{port}} WO_THREADS={{threads}} WO_DATA=$data ./runtime/wo-rt >/dev/null 2>&1 &
server=$! server=$!
trap 'kill $server 2>/dev/null; sleep 0.3; rm -rf $data; make -C prototypes/wo-rt-c clean >/dev/null; make -C prototypes/wo-rt-c wo-rt bench >/dev/null' EXIT trap 'kill $server 2>/dev/null; sleep 0.3; rm -rf $data; make -C runtime clean >/dev/null; make -C runtime wo-rt bench >/dev/null' EXIT
base=127.0.0.1; for _ in $(seq 1 40); do curl -s "http://$base:{{port}}/healthz" >/dev/null && break; sleep 0.25; done base=127.0.0.1; for _ in $(seq 1 40); do curl -s "http://$base:{{port}}/healthz" >/dev/null && break; sleep 0.25; done
B=./prototypes/wo-rt-c/bench/bench B=./runtime/bench/bench
echo "wo-rt-c ({{threads}} shards, durable WAL):" echo "wo-rt-c ({{threads}} shards, durable WAL):"
$B $base {{port}} {{conns}} 5 /healthz $B $base {{port}} {{conns}} 5 /healthz
$B $base {{port}} {{conns}} 5 / $B $base {{port}} {{conns}} 5 /