shoney/writeonce
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
writeonce/docs/00-status.md
shoney.arickathil 641a41e25c feat: milestone 1 complete — .wob emitter, conformance corpus, single binary; GC redesign specced 
- `woc` now emits `.wob` that `wovm` runs: emit.ml lowers the typed,
  owner-annotated AST (scope-stack registers with a >64 WO-E401 diagnostic,
  Lua-style call windows, ICALL by slot, dedup const pool, drop maps, line
  tables, implicit terminators); disasm.ml backs `--dump-bc` goldens.
- Ownership lowering consumes the four owner tables verbatim; RESIDUAL is the
  only source of borrow ops, coalesced per operand. Review caught the emitter
  consuming only 2 of owner.ml's 4 residual producers — an assignment-anchored
  aliasing violation ran to exit 0 instead of trapping; fixed, plus a backstop
  raising WO-E404 for any residual region left unconsumed.
- Conformance harness `scripts/oop-e2e.sh` (`just oop-e2e`): four fixture
  kinds with exact outcomes — byte-exact stdout, one WO-E### anchored on
  `error CODE:`, numeric trap code, gc trace. 25 fixtures incl. pricing-demo
  logic, the ownership suite, and DB_STUB's parse-but-trap. `tests/` un-ignored
  so the corpus is actually tracked.
- `woc build` produces a self-contained binary: wovm copy + appended image +
  20-byte trailer, self-exec via /proc/self/exe. Verified relocated outside
  the repo, argless, and against adversarial trailer corruption.
- Milestone 1's five spec criteria all MET (`just oop-accept`). Criterion 3
  closed by WO-E405 — the entry must return `Int`, since program mode already
  says its return value is the exit code — which deletes the leak class
  without adding return-type metadata to the format. `gc/held-cycle` retired:
  an externally-held cycle is not expressible in a post-exit pump.
- New spec: inferred GC + incremental per-shard tri-color mark-sweep, retiring
  `@gc` and reference counting. Story gains iterations 7b (that work) and 9b
  (`@table`, relations, compiler-checked query); `.dev/reference` gains a
  sparse System.Linq checkout. Priority: 5→6→7 (log-watcher) then 7b, 8, 9, 9b.
2026-08-11 19:31:26 +02:00

243 lines
17 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Status board — what is done, what is next
The single place to learn where this project stands. Organised in six buckets:
**stories** (the narrative arc), **in progress**, **done**, **pending**,
**discarded**, **learnings**. Every phase doc carries a matching status banner;
this board is the index.
Update this board in the same change that finishes work — move the item to done
with *what actually landed*, set the next in-progress item, and record any
rejection in [`discarded.md`](plan/discarded.md) with its reason.
Statuses: ✅ **done** · 🔄 **in progress** · ⬜ **pending** · ⏸ **parked**
---
## ▶ NEXT PLAN
**Story iteration 5 — language surface (Haxe-parity adoptions).**
Plan: [`plan/compiler/2026-08-01-haxe-parity-language.md`](plan/compiler/2026-08-01-haxe-parity-language.md) ·
Story slice: [`docs/stories/language-runtime-database/05-language-surface.md`](stories/language-runtime-database/05-language-surface.md)
The language grows from milestone grammar to a daily-driver surface: every
**adopt** row of the systems-track verdict table (switch expressions, typedef
records, `?T` optionals, enum payloads, try/catch, statics, `using`, modules,
`is`, `pub(read)`, `#if`) lands with a golden + must-fail fixture pair; every
**reject** row refuses with a doctrine-citing diagnostic. **First task: `?T`
forced handling (plan 8 Task 6)** — plumbed since iteration 3 but unenforced
(`WO-E211`–`E213` dead), and the log-watcher port (iterations 6–7) uses
optionals throughout in place of the Haxe original's sentinel values, so
nothing else in this plan can land ahead of it.
Two tracks run in this repo. The critical path is the **language track**:
iterations 3 → 4 → 5 → 6 → 7, ending at *compile and run log-watcher*. The
Rust-runtime track is shipped-and-maintained, not advancing.
---
## Stories
[`docs/stories/language-runtime-database/`](stories/language-runtime-database/00-story.md)
— one language, one runtime, one database, one binary. Twelve iterations, each
an unsplittable slice with Given/When/Then acceptance and a pointer to the plan
that sequences its tasks. Read one, approve, then the next starts.
| # | Iteration | State |
| --- | --- | --- |
| 1 | [Principles doc](stories/language-runtime-database/01-principles-doc.md) | ✅ |
| 2 | [VM core (`wovm`)](stories/language-runtime-database/02-vm-core.md) | ✅ |
| 3 | [Compiler front (`woc`)](stories/language-runtime-database/03-compiler-front.md) | ✅ (known gaps below) |
| 4 | [Single binary end-to-end](stories/language-runtime-database/04-single-binary-e2e.md) | ✅ (known gaps below) |
| 5 | [Language surface](stories/language-runtime-database/05-language-surface.md) | 🔄 **next** |
| 6 | [Program mode + stdlib](stories/language-runtime-database/06-program-mode-stdlib.md) | ⬜ |
| 7 | [log-watcher proof](stories/language-runtime-database/07-logwatcher-proof.md) | ⬜ acceptance |
| 7b | [Inferred GC + mark-sweep](stories/language-runtime-database/07b-inferred-gc-mark-sweep.md) | ⬜ closes iteration 4's gate |
| 8 | [Shard-actor runtime](stories/language-runtime-database/08-shard-actor-runtime.md) | ⬜ |
| 9 | [Database engine](stories/language-runtime-database/09-database-engine.md) | ⬜ |
| 9b | [`@table`, relations, query](stories/language-runtime-database/09b-table-relations-query.md) | ⬜ needs a spec first |
| 10 | [HTTP service layer](stories/language-runtime-database/10-http-service.md) | ⬜ |
| 11 | [Fibers](stories/language-runtime-database/11-fibers.md) | ⬜ |
| 12 | [Blue-green deploy](stories/language-runtime-database/12-blue-green-deploy.md) | ⬜ |
---
## In progress
| Track | Item | Where |
| --- | --- | --- |
| Language | Iteration 5 — Haxe-parity language surface, `?T` forced handling first | [plan 8](plan/compiler/2026-08-01-haxe-parity-language.md) |
Nothing else should be started until iteration 5 lands. Off-critical-path work
is parked by explicit scope directive (2026-08-08).
---
## Done
### Language track — compiler + VM (OOP track)
| Status | Item | Doc | What actually landed |
| --- | --- | --- | --- |
| ✅ | Principles | [`../00-principles.md`](00-principles.md) | 13 principles, each with a why and a link to the doc that enforces it |
| ✅ | `wovm` VM core | [plan 1](superpowers/plans/2026-08-01-wob-format-and-vm-core.md) | `.wob` v1 loader with full static validation, register interpreter (computed-goto + ISO-C fallback), arena with size-class free lists, borrow word, RC + budgeted Bacon–Rajan cycle collector, drop-map trap unwinding, containers, builtins, ICALL, CLI. 13 suites × 2 dispatch flavors + CLI smoke, ASan/UBSan clean |
| ✅ | `.wob` format contract | [`oop-vm/00-wob-format.md`](plan/oop-vm/00-wob-format.md) | Normative; twinned with `runtime/src/wob.h` |
| ✅ | `woc` compiler front | [plan 2](plan/compiler/2026-08-01-woc-compiler-front.md) | Tasks 1–8: dune scaffold, `diag` (WO-E codes, two-site related errors, ordered dedup), newline-significant lexer at rt parity, declaration + statement/expression parser with skip-on-block and multi-error recovery, typechecker (field kinds, `?T` plumbing, W201, E225, E214), MVS ownership pass with the four emitter tables, driver with directory discovery + cross-file programs. 14 + 264 checks |
| ✅ | Error catalog | [`oop-vm/01-error-catalog.md`](plan/oop-vm/01-error-catalog.md) | 14 emitted codes + 10 reserved, each with the reason it is not yet emitted |
| ✅ | log-watcher `.wo` sample | [`../examples/log-watcher/`](examples/log-watcher/README.md) | Eight-file port authored docs-first with its `.hx` mapping table; compiles for real in iteration 7 |
| ✅ | Scalar cleanup | [`discarded.md`](plan/discarded.md) | `Money`/`SKU`/`Float` and the abstract allowlist removed; `abstract` flipped adopt → reject |
| ✅ | `woc` emitter, corpus, single binary | [plan 3](plan/compiler/2026-08-01-wob-emit-e2e-single-binary.md) | Tasks 1–6 + 8 (Task 7, a parity harness against the Rust runtime, **deferred by explicit user decision** — the two stacks diverge by design). Bytecode emitter (`emit.ml`) + disassembler (`disasm.ml`, `--dump-bc`); three-kind conformance harness (`scripts/oop-e2e.sh`, `just oop-e2e`) over `tests/corpus/{run,compile-fail,trap,gc}`; pricing-demo + ownership/trap corpora (19 fixtures); `@gc` cycle collector's post-exit pump (`WO_GC_BUDGET`/`WO_GC_TRACE`) + 2 gc fixtures (`gc/held-cycle` retired — see criterion-3 closure below); `woc build` single-binary output + relocation/corrupt-trailer smoke; `WO-E405` closing criterion 3's ASan leak (entry must return `Int`); `just oop-accept` wiring all five spec criteria + both unit gates into one command. 14 + 399 compiler checks; `oop-e2e` 25/25 against the release `wovm`. **Milestone-1 acceptance gate is fully green — all five criteria met** (see the dated acceptance note in `docs/superpowers/specs/2026-08-01-oop-compiler-vm-design.md`) |
**Known gaps carried out of iteration 3** — recorded, not silently owed:
- **`?T` is plumbed but unenforced.** Lexer/token/AST/parser/dump all handle
`?T`; the semantics do not exist (`WO-E211`/`E212`/`E213` declared, never
emitted — a probe returning `?Int` as `Int` exits 0). Owned by iteration 5,
plan 8 Task 6, which is that iteration's first task because it blocks the
log-watcher port. See [`compiler/nullable-types-implementation.md`](plan/compiler/nullable-types-implementation.md).
- **Structural interface satisfaction is not checked** (`WO-E205` dead), along
with type mismatch, bad arity, and unknown-fn (`E201`/`E203`/`E204`) — all
named in plan 2 Task 6's own must-fail list. Gaps in shipped work, catalogued
as reserved.
- Six further narrowings (W201 heuristic, E225 reach, dead code after `return`,
unresolved-callee drops, RC table ordering, residual b-side role) are listed
in the plan-2 SDD ledger and in the affected files' own comments.
**Known gaps carried out of iteration 4** — recorded, not silently owed:
- **`WO-E205` (unsatisfied interface) is reachable but unenforced — a real
hybrid-boundary inversion, not just a dead code path.** A class that does
not structurally satisfy an interface it's passed as compiles clean (exit
0, zero diagnostics) even though the violation is statically provable, and
the mismatched call reaches `wovm` as an `ICALL` with no matching vtable
entry, trapping `WO_T_BOUNDS` (6) at runtime instead of failing at compile
time. Pinned by `tests/corpus/trap/unsatisfied-interface/`; when `WO-E205`
is wired, that fixture must move to `compile-fail/` in the same change.
- **`set(m, k, v)`'s `@gc` retention gap on map keys/values is open** — the
twin of the `push` bug Task 5 fixed for `multi`. `set` has no equivalent
special case in `owner.ml`'s `analyze_call`, so a `@gc` key or value handed
to `set` is under-counted and the collector can free it while the map still
points at it. Nothing in the corpus exercises this yet. See
[`oop-vm/08-builtin-surface.md`](plan/oop-vm/08-builtin-surface.md).
- **E201/E203 and seven other `WO-E2xx` codes remain declared but unemitted**
— see [`oop-vm/01-error-catalog.md`](plan/oop-vm/01-error-catalog.md).
- **CLOSED — milestone-1's ASan gate (`just oop-accept`) failing on
`gc/held-cycle`.** Root cause (Task 8's finding, restated): `main.c`'s
entry-method return value (`uint64_t ret`, `src/main.c:158`) is stored
but never released, so `gc/held-cycle`'s "permanent external hold" was
actually a permanent refcount inflation — LeakSanitizer's "definite
leak" (1184 bytes / 3 allocations) was correctly reporting exactly
that, not a false positive. Fixing it by releasing `ret` was rejected:
the `.wob` method table carries no return-type/kind metadata, so
`main.c` has no way to know `ret` is a pointer rather than a scalar,
and adding that metadata is a format change out of scope here. Fixed
instead at the source: the systems-track spec already requires the
entry to return `Int` (its return value is the process exit code), so
a class-returning `main` was never legal — `WO-E405`
(`compiler/src/emit.ml`, `01-error-catalog.md`) now rejects it at
compile time, and `gc/held-cycle` is retired because its premise (an
externally-held cycle survives a *post-exit* pump) is no longer
expressible — see `oop-vm/02-corpus.md`'s "Retired" note for why, and
for where the scenario it meant to cover is actually proven
(`runtime/test/test_cycle.c`, plus a proper in-flight fixture scheduled
for story iteration 7b). Spec success criterion 3 is now **MET**;
`just oop-accept` passes all five criteria.
### Rust runtime track — Stage 2 shipped, maintained
| Status | Phase | Doc | Notes |
| --- | --- | --- | --- |
| ✅ | 01 crate scaffolding | [done/01](plan/done/01-scafolding-crates.md) | 15 crates |
| ✅ | 02 epoll event loop | [done/02](plan/done/02-event-loop-epoll.md) | `runtime/netpoll_epoll.rs` |
| ✅ | 03 hand-rolled HTTP | [done/03](plan/done/03-hand-rolled-http.md) | + keep-alive & pipelining |
| ✅ | 04 tokio/axum cutover | [done/04](plan/done/04-cutover-remove-tokio-axum.md) | deps now: anyhow, serde, serde_json, libc |
| ✅ | 09a thread-per-core | [09](plan/09-concurrency-scaleout.md) | `scheduler.rs`, `SO_REUSEPORT`, pinned `wo-shard-<t>` workers |
| ✅ | 09b sharded engine | [09](plan/09-concurrency-scaleout.md) | `shard.rs` bus; `Arc<Mutex<Engine>>` deleted; interleaved ids |
| ✅ | 09c per-shard WAL | [09](plan/09-concurrency-scaleout.md) | ack-after-fsync; boot replay; `meta` shard guard |
| ✅ | — keep-alive follow-up | [09](plan/09-concurrency-scaleout.md) | reads ×3.4 → 770k/s |
| ✅ | — io_uring group commit | [09](plan/09-concurrency-scaleout.md) | raw ring; 4.7× durable writes on real disk |
| ✅ | 16a PG wire client | [16](plan/16-postgres-mirror.md) | hand-rolled protocol v3, zero crates |
| ✅ | 16b PG backup mirror | [16](plan/16-postgres-mirror.md) | async JSONB upserts behind the WAL ack; RAM authoritative |
| ✅ | 13a class surface | [13](plan/13-class-model-live-pricing.md) | `class` parses, CRUD serves |
| ✅ | 13b method execution | [13](plan/13-class-model-live-pricing.md) | row-scoped txn per call; abort → 409 rollback |
| ✅ | — `@table` + indexed DML | [13](plan/13-class-model-live-pricing.md) | secondary indexes, `find_by`, `select Type{…}`, REST filters |
| ✅ | C proving ground A–F | [exploration/c-runtime/00-plan.md](plan/exploration/c-runtime/00-plan.md) | 859k reads/s, 618k durable commits/s; found the ack-ordering + fd-ABA bugs the Rust port avoided |
Ecommerce sample (verified 2026-06-13): `api.rest` 17/17 expected statuses pass.
---
## Pending
### Language track — sequenced, on the critical path
| # | Item | Plan |
| --- | --- | --- |
| 5 | Haxe-parity language surface — **`?T` forced handling first**, then switch expressions, records, enum payloads, try/catch, statics, `using`, modules, `is`, `pub(read)`, `#if` | [plan 8](plan/compiler/2026-08-01-haxe-parity-language.md) |
| 6 | Program mode + systems stdlib — `fn main`, exit codes, `fs`/`proc`/`net`/`time`/`json` | [plan 9](superpowers/plans/2026-08-01-program-mode-stdlib.md) |
| 7 | log-watcher proof — the sample compiles and detects a silent death live | [plan 10](superpowers/plans/2026-08-01-log-watcher-sample.md) |
| 7b | Inferred GC + incremental mark-sweep — `@gc` removed, GC-ness inferred, RC retired | [spec](superpowers/specs/2026-08-11-inferred-gc-mark-sweep-design.md) — plan to be written |
| 8 | Shard-actor runtime | [plan 4](superpowers/plans/2026-08-01-shard-actor-vm-runtime.md) |
| 9 | Database engine binding | [plan 5](superpowers/plans/2026-08-01-db-engine-binding.md) |
| 9b | `@table` + relations + language-integrated query | **no spec yet** — three open forks recorded in the iteration; brainstorm before planning |
| 10 | HTTP service layer | [plan 6](superpowers/plans/2026-08-01-http-service-layer.md) |
| 11 | Fibers | vision §3, [blue-green exploration](plan/exploration/blue-green-vm/00-vision.md) |
| 12 | Blue-green deploy | [spec](superpowers/specs/2026-08-03-blue-green-vm-design.md) — plan authored after iterations 9–10 |
### Language track — parked until after iteration 12
Recorded 2026-08-08 by scope directive; nothing here lands before the
log-watcher proof.
- `WO-W201` `@gc`-suggestion refinement beyond the self-reference heuristic
- `WO-E225` broadened to `ref`/`multi`/`map` element types and fn signatures
- ADT container roster adoption (Stack, Queue, Set, Tree, Graph, …) — see the
roster in [`compiler/nullable-types-implementation.md`](plan/compiler/nullable-types-implementation.md)
- Web framework as a `.wo` library; UI (`##ui` SSR + live patches);
script-based destructive migrations; MCP/agent wrapper over the management plane
- `throw` (explicit raise) — cut 2026-08-10, 0 uses in the driving workload
(log-watcher); catch frames ship without it
- `time.mono` — cut 2026-08-10, 0 uses in the driving workload; returns when a
workload needs monotonic math
- `is` — cut 2026-08-10, 0 uses in the driving workload; emptied plan 8's old
Task 7, which is deleted rather than deferred
### Rust runtime track — not advancing while the language track runs
| Status | Phase | Doc | Notes |
| --- | --- | --- | --- |
| ⬜ | 05 hand-rolled JSON | [05](plan/05-hand-rolled-json.md) | removes serde/serde_json |
| ⬜ | 06 bespoke error type | [06](plan/06-bespoke-error.md) | removes anyhow |
| ⬜ | 07 inotify content watcher | [07](plan/07-inotify-content-watcher.md) | `wo dev` hot reload |
| ⬜ | 08 sendfile static assets | [08](plan/08-sendfile-static-assets.md) | needed by the parked UI track |
| ⬜ | 09d cross-shard subscriptions | [09](plan/09-concurrency-scaleout.md) | LIVE fan-out; pairs with Stage 3 |
| ⬜ | 09e cross-shard transactions (2PC) | [09](plan/09-concurrency-scaleout.md) | needed by `fn checkout` spanning shards |
| ⬜ | 09f observability & reshard | [09](plan/09-concurrency-scaleout.md) | per-shard metrics, `WO_RESHARD` |
| ⬜ | 10–12 storage completion | [10](plan/10-storage-foundations.md), [11](plan/11-wal-and-recovery.md), [12](plan/12-engine-disk-cutover.md) | snapshots, compaction, WAL rotation, mmap arena engine |
| ⬜ | 13c LIVE pricing push · Stage 3 wire layer · 13e at scale | [13](plan/13-class-model-live-pricing.md) | replaces the 501 stub |
| ⬜ | 15a–15e MCP over streamable HTTP | [15](plan/15-mcp-streamable-http.md) | 15e needs 13c + 09d |
| ⬜ | 16c–16f typed columns, lossless resync, restore, SCRAM | [16](plan/16-postgres-mirror.md) | |
### Frontend — parked
| Status | Phase | Doc |
| --- | --- | --- |
| ⏸ | 13d pricing UI | [13](plan/13-class-model-live-pricing.md) |
| ⏸ | 14 MVC UI implementation (14a–f) | [14](plan/14-mvc-ui-implementation.md) |
| ⏸ | UI exploration track | [exploration/ui/00-overview.md](plan/exploration/ui/00-overview.md) |
---
## Discarded
Settled rejections with their reasons live in [`discarded.md`](plan/discarded.md) —
inheritance, `abstract` newtypes, `Money`/`SKU`/`Float`, `Dynamic`/`cast`/
`macro`/`extern`, AOT-to-C, Menhir, shared mutable engine state, external
deployer daemon, destructive migrations in v1, and more. Argue against the
recorded reason rather than re-opening an entry as new.
## Learnings
What attempts taught, shipped or not, in [`learnings.md`](plan/learnings.md) —
plumbed-is-not-enforced, vacuously-passing goldens, exit-0-with-wrong-output,
the malloc-path ASan trick, deferred checks that never reach the runtime,
validate-once-at-the-boundary, and reference-implement-in-C-first.
Reference in a new issue View git blame Copy permalink