writeonce/docs/00-status.md
shoney.arickathil 79605cbb4f feat: milestone 1 complete — .wob emitter, conformance corpus, single binary; GC redesign specced
- `woc` now emits `.wob` that `wovm` runs: emit.ml lowers the typed,
  owner-annotated AST (scope-stack registers with a >64 WO-E401 diagnostic,
  Lua-style call windows, ICALL by slot, dedup const pool, drop maps, line
  tables, implicit terminators); disasm.ml backs `--dump-bc` goldens.
- Ownership lowering consumes the four owner tables verbatim; RESIDUAL is the
  only source of borrow ops, coalesced per operand. Review caught the emitter
  consuming only 2 of owner.ml's 4 residual producers — an assignment-anchored
  aliasing violation ran to exit 0 instead of trapping; fixed, plus a backstop
  raising WO-E404 for any residual region left unconsumed.
- Conformance harness `scripts/oop-e2e.sh` (`just oop-e2e`): four fixture
  kinds with exact outcomes — byte-exact stdout, one WO-E### anchored on
  `error CODE:`, numeric trap code, gc trace. 25 fixtures incl. pricing-demo
  logic, the ownership suite, and DB_STUB's parse-but-trap. `tests/` un-ignored
  so the corpus is actually tracked.
- `woc build` produces a self-contained binary: wovm copy + appended image +
  20-byte trailer, self-exec via /proc/self/exe. Verified relocated outside
  the repo, argless, and against adversarial trailer corruption.
- Milestone 1's five spec criteria all MET (`just oop-accept`). Criterion 3
  closed by WO-E405 — the entry must return `Int`, since program mode already
  says its return value is the exit code — which deletes the leak class
  without adding return-type metadata to the format. `gc/held-cycle` retired:
  an externally-held cycle is not expressible in a post-exit pump.
- New spec: inferred GC + incremental per-shard tri-color mark-sweep, retiring
  `@gc` and reference counting. Story gains iterations 7b (that work) and 9b
  (`@table`, relations, compiler-checked query); `.dev/reference` gains a
  sparse System.Linq checkout. Priority: 5→6→7 (log-watcher) then 7b, 8, 9, 9b.
2026-08-11 19:31:26 +02:00

17 KiB
Raw Blame History

Status board — what is done, what is next

The single place to learn where this project stands. Organised in six buckets: stories (the narrative arc), in progress, done, pending, discarded, learnings. Every phase doc carries a matching status banner; this board is the index.

Update this board in the same change that finishes work — move the item to done with what actually landed, set the next in-progress item, and record any rejection in discarded.md with its reason.

Statuses: ✅ done · 🔄 in progress · ⬜ pending · ⏸ parked


▶ NEXT PLAN

Story iteration 5 — language surface (Haxe-parity adoptions). Plan: plan/compiler/2026-08-01-haxe-parity-language.md · Story slice: docs/stories/language-runtime-database/05-language-surface.md

The language grows from milestone grammar to a daily-driver surface: every adopt row of the systems-track verdict table (switch expressions, typedef records, ?T optionals, enum payloads, try/catch, statics, using, modules, is, pub(read), #if) lands with a golden + must-fail fixture pair; every reject row refuses with a doctrine-citing diagnostic. First task: ?T forced handling (plan 8 Task 6) — plumbed since iteration 3 but unenforced (WO-E211–E213 dead), and the log-watcher port (iterations 6–7) uses optionals throughout in place of the Haxe original's sentinel values, so nothing else in this plan can land ahead of it.

Two tracks run in this repo. The critical path is the language track: iterations 3 → 4 → 5 → 6 → 7, ending at compile and run log-watcher. The Rust-runtime track is shipped-and-maintained, not advancing.


Stories

docs/stories/language-runtime-database/ — one language, one runtime, one database, one binary. Twelve iterations, each an unsplittable slice with Given/When/Then acceptance and a pointer to the plan that sequences its tasks. Read one, approve, then the next starts.

# Iteration State
1 Principles doc ✅
2 VM core (wovm) ✅
3 Compiler front (woc) ✅ (known gaps below)
4 Single binary end-to-end ✅ (known gaps below)
5 Language surface 🔄 next
6 Program mode + stdlib ⬜
7 log-watcher proof ⬜ acceptance
7b Inferred GC + mark-sweep ⬜ closes iteration 4's gate
8 Shard-actor runtime ⬜
9 Database engine ⬜
9b @table, relations, query ⬜ needs a spec first
10 HTTP service layer ⬜
11 Fibers ⬜
12 Blue-green deploy ⬜

In progress

Track Item Where
Language Iteration 5 — Haxe-parity language surface, ?T forced handling first plan 8

Nothing else should be started until iteration 5 lands. Off-critical-path work is parked by explicit scope directive (2026-08-08).


Done

Language track — compiler + VM (OOP track)

Status Item Doc What actually landed
✅ Principles ../00-principles.md 13 principles, each with a why and a link to the doc that enforces it
✅ wovm VM core plan 1 .wob v1 loader with full static validation, register interpreter (computed-goto + ISO-C fallback), arena with size-class free lists, borrow word, RC + budgeted Bacon–Rajan cycle collector, drop-map trap unwinding, containers, builtins, ICALL, CLI. 13 suites × 2 dispatch flavors + CLI smoke, ASan/UBSan clean
✅ .wob format contract oop-vm/00-wob-format.md Normative; twinned with runtime/src/wob.h
✅ woc compiler front plan 2 Tasks 1–8: dune scaffold, diag (WO-E codes, two-site related errors, ordered dedup), newline-significant lexer at rt parity, declaration + statement/expression parser with skip-on-block and multi-error recovery, typechecker (field kinds, ?T plumbing, W201, E225, E214), MVS ownership pass with the four emitter tables, driver with directory discovery + cross-file programs. 14 + 264 checks
✅ Error catalog oop-vm/01-error-catalog.md 14 emitted codes + 10 reserved, each with the reason it is not yet emitted
✅ log-watcher .wo sample ../examples/log-watcher/ Eight-file port authored docs-first with its .hx mapping table; compiles for real in iteration 7
✅ Scalar cleanup discarded.md Money/SKU/Float and the abstract allowlist removed; abstract flipped adopt → reject
✅ woc emitter, corpus, single binary plan 3 Tasks 1–6 + 8 (Task 7, a parity harness against the Rust runtime, deferred by explicit user decision — the two stacks diverge by design). Bytecode emitter (emit.ml) + disassembler (disasm.ml, --dump-bc); three-kind conformance harness (scripts/oop-e2e.sh, just oop-e2e) over tests/corpus/{run,compile-fail,trap,gc}; pricing-demo + ownership/trap corpora (19 fixtures); @gc cycle collector's post-exit pump (WO_GC_BUDGET/WO_GC_TRACE) + 2 gc fixtures (gc/held-cycle retired — see criterion-3 closure below); woc build single-binary output + relocation/corrupt-trailer smoke; WO-E405 closing criterion 3's ASan leak (entry must return Int); just oop-accept wiring all five spec criteria + both unit gates into one command. 14 + 399 compiler checks; oop-e2e 25/25 against the release wovm. Milestone-1 acceptance gate is fully green — all five criteria met (see the dated acceptance note in docs/superpowers/specs/2026-08-01-oop-compiler-vm-design.md)

Known gaps carried out of iteration 3 — recorded, not silently owed:

  • ?T is plumbed but unenforced. Lexer/token/AST/parser/dump all handle ?T; the semantics do not exist (WO-E211/E212/E213 declared, never emitted — a probe returning ?Int as Int exits 0). Owned by iteration 5, plan 8 Task 6, which is that iteration's first task because it blocks the log-watcher port. See compiler/nullable-types-implementation.md.
  • Structural interface satisfaction is not checked (WO-E205 dead), along with type mismatch, bad arity, and unknown-fn (E201/E203/E204) — all named in plan 2 Task 6's own must-fail list. Gaps in shipped work, catalogued as reserved.
  • Six further narrowings (W201 heuristic, E225 reach, dead code after return, unresolved-callee drops, RC table ordering, residual b-side role) are listed in the plan-2 SDD ledger and in the affected files' own comments.

Known gaps carried out of iteration 4 — recorded, not silently owed:

  • WO-E205 (unsatisfied interface) is reachable but unenforced — a real hybrid-boundary inversion, not just a dead code path. A class that does not structurally satisfy an interface it's passed as compiles clean (exit 0, zero diagnostics) even though the violation is statically provable, and the mismatched call reaches wovm as an ICALL with no matching vtable entry, trapping WO_T_BOUNDS (6) at runtime instead of failing at compile time. Pinned by tests/corpus/trap/unsatisfied-interface/; when WO-E205 is wired, that fixture must move to compile-fail/ in the same change.
  • set(m, k, v)'s @gc retention gap on map keys/values is open — the twin of the push bug Task 5 fixed for multi. set has no equivalent special case in owner.ml's analyze_call, so a @gc key or value handed to set is under-counted and the collector can free it while the map still points at it. Nothing in the corpus exercises this yet. See oop-vm/08-builtin-surface.md.
  • E201/E203 and seven other WO-E2xx codes remain declared but unemitted — see oop-vm/01-error-catalog.md.
  • CLOSED — milestone-1's ASan gate (just oop-accept) failing on gc/held-cycle. Root cause (Task 8's finding, restated): main.c's entry-method return value (uint64_t ret, src/main.c:158) is stored but never released, so gc/held-cycle's "permanent external hold" was actually a permanent refcount inflation — LeakSanitizer's "definite leak" (1184 bytes / 3 allocations) was correctly reporting exactly that, not a false positive. Fixing it by releasing ret was rejected: the .wob method table carries no return-type/kind metadata, so main.c has no way to know ret is a pointer rather than a scalar, and adding that metadata is a format change out of scope here. Fixed instead at the source: the systems-track spec already requires the entry to return Int (its return value is the process exit code), so a class-returning main was never legal — WO-E405 (compiler/src/emit.ml, 01-error-catalog.md) now rejects it at compile time, and gc/held-cycle is retired because its premise (an externally-held cycle survives a post-exit pump) is no longer expressible — see oop-vm/02-corpus.md's "Retired" note for why, and for where the scenario it meant to cover is actually proven (runtime/test/test_cycle.c, plus a proper in-flight fixture scheduled for story iteration 7b). Spec success criterion 3 is now MET; just oop-accept passes all five criteria.

Rust runtime track — Stage 2 shipped, maintained

Status Phase Doc Notes
✅ 01 crate scaffolding done/01 15 crates
✅ 02 epoll event loop done/02 runtime/netpoll_epoll.rs
✅ 03 hand-rolled HTTP done/03 + keep-alive & pipelining
✅ 04 tokio/axum cutover done/04 deps now: anyhow, serde, serde_json, libc
✅ 09a thread-per-core 09 scheduler.rs, SO_REUSEPORT, pinned wo-shard-<t> workers
✅ 09b sharded engine 09 shard.rs bus; Arc<Mutex<Engine>> deleted; interleaved ids
✅ 09c per-shard WAL 09 ack-after-fsync; boot replay; meta shard guard
✅ — keep-alive follow-up 09 reads ×3.4 → 770k/s
✅ — io_uring group commit 09 raw ring; 4.7× durable writes on real disk
✅ 16a PG wire client 16 hand-rolled protocol v3, zero crates
✅ 16b PG backup mirror 16 async JSONB upserts behind the WAL ack; RAM authoritative
✅ 13a class surface 13 class parses, CRUD serves
✅ 13b method execution 13 row-scoped txn per call; abort → 409 rollback
✅ — @table + indexed DML 13 secondary indexes, find_by, select Type{…}, REST filters
✅ C proving ground A–F exploration/c-runtime/00-plan.md 859k reads/s, 618k durable commits/s; found the ack-ordering + fd-ABA bugs the Rust port avoided

Ecommerce sample (verified 2026-06-13): api.rest 17/17 expected statuses pass.


Pending

Language track — sequenced, on the critical path

# Item Plan
5 Haxe-parity language surface — ?T forced handling first, then switch expressions, records, enum payloads, try/catch, statics, using, modules, is, pub(read), #if plan 8
6 Program mode + systems stdlib — fn main, exit codes, fs/proc/net/time/json plan 9
7 log-watcher proof — the sample compiles and detects a silent death live plan 10
7b Inferred GC + incremental mark-sweep — @gc removed, GC-ness inferred, RC retired spec — plan to be written
8 Shard-actor runtime plan 4
9 Database engine binding plan 5
9b @table + relations + language-integrated query no spec yet — three open forks recorded in the iteration; brainstorm before planning
10 HTTP service layer plan 6
11 Fibers vision §3, blue-green exploration
12 Blue-green deploy spec — plan authored after iterations 9–10

Language track — parked until after iteration 12

Recorded 2026-08-08 by scope directive; nothing here lands before the log-watcher proof.

  • WO-W201 @gc-suggestion refinement beyond the self-reference heuristic
  • WO-E225 broadened to ref/multi/map element types and fn signatures
  • ADT container roster adoption (Stack, Queue, Set, Tree, Graph, …) — see the roster in compiler/nullable-types-implementation.md
  • Web framework as a .wo library; UI (##ui SSR + live patches); script-based destructive migrations; MCP/agent wrapper over the management plane
  • throw (explicit raise) — cut 2026-08-10, 0 uses in the driving workload (log-watcher); catch frames ship without it
  • time.mono — cut 2026-08-10, 0 uses in the driving workload; returns when a workload needs monotonic math
  • is — cut 2026-08-10, 0 uses in the driving workload; emptied plan 8's old Task 7, which is deleted rather than deferred

Rust runtime track — not advancing while the language track runs

Status Phase Doc Notes
⬜ 05 hand-rolled JSON 05 removes serde/serde_json
⬜ 06 bespoke error type 06 removes anyhow
⬜ 07 inotify content watcher 07 wo dev hot reload
⬜ 08 sendfile static assets 08 needed by the parked UI track
⬜ 09d cross-shard subscriptions 09 LIVE fan-out; pairs with Stage 3
⬜ 09e cross-shard transactions (2PC) 09 needed by fn checkout spanning shards
⬜ 09f observability & reshard 09 per-shard metrics, WO_RESHARD
⬜ 10–12 storage completion 10, 11, 12 snapshots, compaction, WAL rotation, mmap arena engine
⬜ 13c LIVE pricing push · Stage 3 wire layer · 13e at scale 13 replaces the 501 stub
⬜ 15a–15e MCP over streamable HTTP 15 15e needs 13c + 09d
⬜ 16c–16f typed columns, lossless resync, restore, SCRAM 16

Frontend — parked

Status Phase Doc
⏸ 13d pricing UI 13
⏸ 14 MVC UI implementation (14a–f) 14
⏸ UI exploration track exploration/ui/00-overview.md

Discarded

Settled rejections with their reasons live in discarded.md — inheritance, abstract newtypes, Money/SKU/Float, Dynamic/cast/ macro/extern, AOT-to-C, Menhir, shared mutable engine state, external deployer daemon, destructive migrations in v1, and more. Argue against the recorded reason rather than re-opening an entry as new.

Learnings

What attempts taught, shipped or not, in learnings.md — plumbed-is-not-enforced, vacuously-passing goldens, exit-0-with-wrong-output, the malloc-path ASan trick, deferred checks that never reach the runtime, validate-once-at-the-boundary, and reference-implement-in-C-first.