diff --git a/docs/00-principles.md b/docs/00-principles.md index 2714852..24ee993 100644 --- a/docs/00-principles.md +++ b/docs/00-principles.md @@ -28,9 +28,11 @@ the dependency doctrine in [the OOP spec](superpowers/specs/2026-08-01-oop-compi Objects are owned values: one owner, moves on assignment, second-class borrows checked mostly at compile time (mutable value semantics — the -Rust-borrow shape without lifetime inference). `@gc` is a per-class opt-in, -reference-counted with budgeted per-shard cycle collection — no global -pause exists by construction. +Rust-borrow shape without lifetime inference). GC-ness is **inferred** by +the compiler (iteration 7b): a class in a reference cycle, or one whose +values must escape as long-lived aliases, is traced by an incremental +per-shard tri-color mark-sweep collector in budgeted slices — the developer +writes no memory annotation, and no global pause exists by construction. *Why:* deterministic memory for the default case, aliasing freedom where the design wants it, and never a stop-the-world in a runtime that is also the database. diff --git a/docs/00-status.md b/docs/00-status.md index e6a314d..b57ddc2 100644 --- a/docs/00-status.md +++ b/docs/00-status.md @@ -114,7 +114,7 @@ that sequences its tasks. Read one, approve, then the next starts. | 5 | [Language surface](stories/language-runtime-database/05-language-surface.md) | 🔄 grammar done, strictness ⏸ deferred | | 6 | [Program mode + stdlib](stories/language-runtime-database/06-program-mode-stdlib.md) | ✅ (the surface log-watcher uses) | | 7 | [log-watcher proof](stories/language-runtime-database/07-logwatcher-proof.md) | 🔄 **runs; executable in progress** | -| 7b | [Inferred GC + mark-sweep](stories/language-runtime-database/07b-inferred-gc-mark-sweep.md) | ⏸ off the workload's path (no `@gc`) | +| 7b | [Inferred GC + mark-sweep](stories/language-runtime-database/07b-inferred-gc-mark-sweep.md) | ✅ **landed 2026-08-18** — `@gc` gone (WO-E104), GC-ness inferred, RC replaced by incremental mark-sweep, `.wob` v4; supersedes iteration 2's RC memory model | | 8 | [Shard-actor runtime](stories/language-runtime-database/08-shard-actor-runtime.md) | ⬜ | | 9 | [Database engine](stories/language-runtime-database/09-database-engine.md) | 🔄 engine complete (storage/WAL/indexes/insert-update-delete); reads land with 9b | | 9b | [`@table`, relations, query](stories/language-runtime-database/09b-table-relations-query.md) | 🔄 query surface + relations + FK done (branch query-surface); group-by parked | @@ -309,7 +309,6 @@ The C proving-ground work (`exploration/c-runtime/`, phases A–F: 859k reads/s, | 5 | Haxe-parity language surface — **`?T` forced handling first**, then switch expressions, records, enum payloads, try/catch, statics, `using`, modules, `is`, `pub(read)`, `#if` | [plan 8](plan/compiler/2026-08-01-haxe-parity-language.md) | | 6 | Program mode + systems stdlib — `fn main`, exit codes, `fs`/`proc`/`net`/`time`/`json` | [plan 9](superpowers/plans/2026-08-01-program-mode-stdlib.md) | | 7 | log-watcher proof — the sample compiles and detects a silent death live | [plan 10](superpowers/plans/2026-08-01-log-watcher-sample.md) | -| 7b | Inferred GC + incremental mark-sweep — `@gc` removed, GC-ness inferred, RC retired | [spec](superpowers/specs/2026-08-11-inferred-gc-mark-sweep-design.md) — plan to be written | | 8 | Shard-actor runtime | [plan 4](superpowers/plans/2026-08-01-shard-actor-vm-runtime.md) | | 9 | Database engine binding | [plan 5](superpowers/plans/2026-08-01-db-engine-binding.md) | | 9b | `@table` + relations + language-integrated query — comprehension queries, `ref`/`backlink` navigation, GroupBy aggregates; acceptance: new `docs/examples/employee` sample | [spec](superpowers/specs/2026-08-15-table-relations-query-design.md) · [plan](plan/compiler/2026-08-15-employee-relations-query.md) | diff --git a/docs/examples/gc-cycle/README.md b/docs/examples/gc-cycle/README.md index 0de6cac..efe6d21 100644 --- a/docs/examples/gc-cycle/README.md +++ b/docs/examples/gc-cycle/README.md @@ -20,7 +20,7 @@ A pass between `types` and `owner` (`compiler/src/gcinfer.ml`) builds a is a row id, `Copy`). Tarjan's SCC over that graph: any class in a non-trivial SCC, or with a self-loop, is **traced (`gc`)**. Everything else is **`owned`**. -For this sample, `woc --dump-gc` would print: +For this sample, `woc --dump-gc` prints: ``` Node gc (cycle Node -> Node) @@ -118,7 +118,7 @@ flowchart TD ALLOC["allocate traced object
color = WHITE, link into traced list"] --> LIVE LIVE["mutator runs
(program executes)"] --> TRIG{"traced bytes since last cycle
past heap goal?"} TRIG -- no --> LIVE - TRIG -- yes --> ROOTS["START CYCLE
shade every root GREY
(value/frame slots via pc gc-mask)"] + TRIG -- yes --> ROOTS["START CYCLE (snapshot)
shade every root GREY
(value/frame slots via pc gc-mask)"] ROOTS --> SLICE SLICE["MARK SLICE (budgeted)
pop a GREY object,
scan its GCREF fields +
owned subtrees that may reach a gcref,
shade each WHITE child GREY,
then paint this object BLACK"] --> GREY{"grey set empty?"} GREY -- "no (budget hit)" --> SAFE["yield at next safepoint
(loop back-edge / call)"] @@ -143,14 +143,16 @@ subtree that can reach no traced object at all. **The barrier — why incremental is safe.** Between slices the mutator keeps running and can hide a live object from a half-finished mark: store a white object into an already-**black** object, then drop the original grey/white -reference to it. A **Yuasa deletion barrier** closes this: on any store into a -`GCREF` slot **while marking is active**, shade the slot's **old** value grey -before overwriting it. In the sample, `a.next = b` (and the ring-closing -`c.next = a`) go through the store paths `SETF`/`map_set`/`push` where the -barrier lives — no new opcode, because `SETF` already resolves the field kind -from the class table. Owned stores, scalars, and Text pay nothing. Reading a -shard's roots fresh from its masks each slice is what removes Go's Dijkstra -insertion-half and the stack rescan. +reference to it. A **Yuasa deletion barrier** closes this: on any deletion of a +`GCREF` edge **while marking is active** — a `SETF` overwrite, or an owned +holder dying with a gcref inside (every such path funnels through +`wo_drop_kind`) — the **old** target is shaded grey first. No new opcode: +`SETF` already resolves the field kind from the class table. Owned stores, +scalars, and Text pay nothing. Snapshot-at-beginning completes the argument: +roots are scanned atomically when the cycle starts, and objects allocated +mid-cycle are born black — so anything reachable at the snapshot, or created +after it, survives; that is what removes Go's Dijkstra insertion-half and the +stack rescan. **Trigger & budget.** A cycle starts when the shard's traced bytes since the last cycle cross a heap goal; each slice marks at most `WO_GC_BUDGET` objects; @@ -199,13 +201,17 @@ traced with **no annotation** and *traced classes alias freely* — the ring **compiles** (the old `WO-E301: use of \`a\` after it was moved` at `c.next = a` is gone), and its bytecode is byte-identical to writing `@gc class Node`. -**Not yet: the ring runs.** On today's runtime (RC + Bacon–Rajan, `gc.c`) a -**nullable single-reference gc field** (`next: ?Node`) store/read is -unimplemented — even a one-hop `a.next = b; print(a.next.label)` traps -`null receiver` (the existing gc corpus only exercises `multi` gcref fields, -which do work). Running the ring, and reclaiming it, is **Phase 3**: the -incremental mark-sweep collector + full gcref field paths, the `.wob` -opcode-27/28 retirement, and the sweep list. +**Phase 3 (landed).** The runtime collector is the incremental tri-color +mark-sweep this README describes: RC and trial deletion are gone, the header +carries the sweep-list link, opcodes 27–28 are reserved (`.wob` v4), and the +Yuasa deletion barrier lives in the store paths. **The ring runs and is +reclaimed**: `woc --emit docs/examples/gc-cycle -o gc.wob && WO_GC_TRACE=1 +wovm gc.wob` prints `ring a -> b -> c -> a` then +`gc: step 1 budget=64 freed=3 remaining=0`, ASan-clean; with a tiny goal +(`WO_GC_GOAL=64`) a mid-program cycle runs while the ring is rooted and +correctly frees nothing. (The old RC runtime couldn't even store a `?Node` +gcref field — the unconditional RC_DEC of the nil old value trapped; that +opcode no longer exists.) **Phase 2b (landed).** Demand promotion: the ownership pass, run in collect mode, promotes any class whose value *must escape* (returned, stored where it diff --git a/docs/plan/oop-vm/00-wob-format.md b/docs/plan/oop-vm/00-wob-format.md index c4e34be..914fe71 100644 --- a/docs/plan/oop-vm/00-wob-format.md +++ b/docs/plan/oop-vm/00-wob-format.md @@ -11,7 +11,7 @@ All integers little-endian; offsets are absolute file offsets. -**Header (44 bytes):** magic `"WOB1"`, version 2, then offset/count u32 pairs for the constant pool, class table, interface section, and method table, then a u32 entry-method index (all-ones = none). +**Header (44 bytes):** magic `"WOB1"`, version 4, then offset/count u32 pairs for the constant pool, class table, interface section, and method table, then a u32 entry-method index (all-ones = none). **Constant pool** — sequential entries: one tag byte; tag 0 = i64 follows; tag 1 = text (u32 length + bytes, no NUL). @@ -47,7 +47,7 @@ The metadata exists for exactly one reason: `json.encode`/`json.decode` are runt | 20–21 | GETF / SETF | field read/write with runtime null/native/bounds checks (trap T_BOUNDS); overwriting a non-scalar field does NOT auto-drop the old value — the compiler emits the drop | | 22 | DROP A | recursively drop the owned value in A per its class drop plan, null the register | | 23–26 | BORROW_S/BORROW_X/RELEASE_S/RELEASE_X | borrow-word ops on the object in A; violation traps T_BORROW | -| 27–28 | RC_INC / RC_DEC | refcount ops on the `@gc` object in A | +| 27–28 | *reserved* | were RC_INC/RC_DEC; retired with reference counting in v4 (iteration 7b) — the loader rejects them like any unknown opcode | | 29 | BUILTIN A B C | register A = builtin C applied to args starting at register B (fixed arity per builtin; `multi_new`/`map_new` carry kind immediates in B instead) | | 30 | DB_STUB | trap T_DB "engine not linked" (spec: SQL-layer statements in milestone 1) | | 31 | TRAP Bx | explicit trap with code Bx | @@ -123,8 +123,8 @@ as a borrow argument — a record/class constructor literal, a variant construction, or an owned-returning call (`peek(Pay{})`, `get(Boxed(Pay{}))`) — is copied to a stable register below the call window and `DROP`ped by the caller once the call returns (`take` -arguments are the callee's to drop; places are their scope's; `@gc` and -`Text` temporaries are excluded — the rc system's and the Copy-aliasing +arguments are the callee's to drop; places are their scope's; traced and +`Text` temporaries are excluded — the collector's and the Copy-aliasing story's, respectively). Recursive drop is correct both ways, because a payload the callee moved out left the field nulled. diff --git a/docs/plan/oop-vm/08-builtin-surface.md b/docs/plan/oop-vm/08-builtin-surface.md index 059054d..988e4e1 100644 --- a/docs/plan/oop-vm/08-builtin-surface.md +++ b/docs/plan/oop-vm/08-builtin-surface.md @@ -78,23 +78,15 @@ trapped `BOUNDS "not a text value"`. Copying is the only rule correct for both shapes: a value read out of a place keeps its owner, and a freshly built Text (a call result, a `..` chain, an interpolation) stays the caller's — the compiler emits that drop right after the call (emit.ml's `drop_fresh_text`). -`OWNED`/`GCREF` elements still MOVE: they are not copyable, and the `@gc` -escape below is what keeps their counting right. Only the `@gc` half of the -old hazard remains open. +`OWNED`/`GCREF` elements still MOVE: they are not copyable. -**`push` and `@gc` elements.** `push(m, v)`'s value argument is never a -resolved callee parameter (`push` has no declared signature), so the -owner pass's ordinary Take-gated transfer never reaches it; a `@gc` value -pushed into a `multi` is special-cased in `owner.ml`'s `analyze_call` -(the value escapes into the container exactly like a ctor field, RC_INC -included) specifically so a `multi`-mediated `@gc` cycle can be built -and later collected (`tests/corpus/gc/`, plan 3 task 5). **`set(m, k, v)` -has no equivalent special case** — a `@gc` key or value handed to `set` -is not retained, so a `map<_, SomeGcClass>` (or a `@gc`-keyed map) built -this way will under-count its element's refcount and the collector will -free it while the map still points at it. Nothing in the corpus -exercises this yet; treat it as an open gap, not a proven-safe pattern, -until `set` gets the same fix `push` did. +**Traced elements need no bookkeeping (iteration 7b).** The old `push` +RC_INC special case and its `set(m, k, v)` retention gap are both +**deleted with reference counting itself**: a traced value stored into a +container is found by the mark phase through the container, so there is +no count to keep right and the use-after-free class those paragraphs +guarded against cannot recur. (`tests/corpus/gc/` still builds a +`multi`-mediated cycle and collects it — now by tracing.) ## A fresh container needs a destination of declared type diff --git a/docs/stories/language-runtime-database/07b-inferred-gc-mark-sweep.md b/docs/stories/language-runtime-database/07b-inferred-gc-mark-sweep.md index 1fd16f1..c8a9061 100644 --- a/docs/stories/language-runtime-database/07b-inferred-gc-mark-sweep.md +++ b/docs/stories/language-runtime-database/07b-inferred-gc-mark-sweep.md @@ -9,13 +9,22 @@ > settled before iteration 8 multiplies shards. -> **Status (2026-08-14):** **not on the driving workload's path**, measured: -> `docs/examples/log-watcher` declares no `@gc` class — 35 classes in its image, -> none with the gc flag, and 0 `RC_INC` / 0 `RC_DEC` instructions against 78 -> `DROP`s. Its memory story is arena + deterministic drops end to end, so this -> iteration (and iteration 4's open `gc/held-cycle` leak, and `set`'s `@gc` -> retention gap) cannot affect whether log-watcher runs. It stays queued for -> workloads that build cycles; the `gc/` corpus fixtures remain its only users. +> **Status (2026-08-18): LANDED** (branch `inferred-gc`; plan +> [`2026-08-18-inferred-gc-mark-sweep.md`](../../superpowers/plans/2026-08-18-inferred-gc-mark-sweep.md)). +> The front end infers GC-ness (structural SCC + demand promotion, +> `woc --dump-gc`), `@gc` in source is WO-E104, and the runtime's RC + +> Bacon–Rajan collector is replaced by an incremental per-shard tri-color +> mark-sweep with a Yuasa deletion barrier — `.wob` is v4, opcodes 27–28 +> reserved. The worked example is +> [`docs/examples/gc-cycle`](../../examples/gc-cycle/README.md): its ring +> compiles with no annotation, runs, and is reclaimed in budgeted slices, +> ASan-clean. All four recorded `@gc`/RC defects are deleted by +> construction; `just oop-accept` is fully green (criterion 3's ASan clause +> included). +> +> *(Historical status 2026-08-14: not on the log-watcher critical path — 35 +> classes, none gc, arena + deterministic drops end to end. That is still +> true; log-watcher simply never allocates a traced object.)* ## Goals diff --git a/docs/superpowers/plans/2026-08-18-inferred-gc-mark-sweep.md b/docs/superpowers/plans/2026-08-18-inferred-gc-mark-sweep.md index a0b3c63..9dde1ba 100644 --- a/docs/superpowers/plans/2026-08-18-inferred-gc-mark-sweep.md +++ b/docs/superpowers/plans/2026-08-18-inferred-gc-mark-sweep.md @@ -1,5 +1,23 @@ # Inferred GC + incremental mark-sweep — implementation plan +> **Status: COMPLETE (2026-08-18)** — all phases landed on branch +> `inferred-gc`; `just oop-accept` fully green. Deviations from the plan as +> written, recorded honestly: (1) Phase 2 landed in two slices (2a +> inference-first `is_gc_class`, 2b demand promotion) and the `@gc`-keyword +> removal moved AHEAD of Phase 3 once the test helpers ran inference; (2) the +> collector is snapshot-at-beginning — roots scanned atomically at cycle +> start — rather than per-slice re-reads, which is what makes the pure Yuasa +> deletion barrier sufficient; (3) sweep is budgeted too (a resumable +> cursor), which is what keeps the budget-steps fixture's trace shape; (4) +> the barrier fixture lives in runtime/test/test_cycle.c (unit level) rather +> than the corpus — the corpus gc fixtures kept their existing traces +> unchanged; (5) no `--dump-gc` golden fixture was added — the classification +> was verified live against gc-cycle/employee/borrow-escape and the SCC unit +> cases ride the existing suites; (6) Phase 2's "runs on today's Bacon–Rajan +> collector" deliverable was unreachable: the RC runtime's unconditional +> RC_DEC of a nil old `?Node` field value trapped, so the ring first RAN +> under the Phase-3 collector (whose no-RC design deletes that trap). + > **For agentic workers:** use superpowers:executing-plans (inline) or > subagent-driven-development. Steps are checkboxes. Per repo rule, this plan > carries **actions in words + verification commands, no code blocks** — the @@ -47,19 +65,19 @@ from the annotation, so existing goldens are untouched). Lowest-risk landing. **Files:** Create `compiler/src/gcinfer.ml`; Modify `compiler/src/dune` (add `gcinfer` to `modules`). -- [ ] Build a directed graph over class names: edge `A → B` when `A` has a field whose resolved type is `B`, `?B`, `multi B`, `map`, or `map<_,B>` (unwrap `Nullable`). **`ref B` contributes no edge.** Read fields from `Ast` class decls; resolve names via the symbol table `types.ml` already builds. -- [ ] Run Tarjan's SCC (hand-written, stdlib only). Classify a class **traced** iff it is in a non-trivial SCC **or** has a self-loop; else **owned**. Expose `Gcinfer.classify : -> result` returning the traced-name set plus, per traced class, the reason (a cycle path for the note). -- [ ] Verify with a tiny OCaml unit in `compiler/test` (or the existing runner) over: self-loop (`Node.next: ?Node`), mutual recursion (`A.b:B`, `B.a:A`), `multi Self`, `map<_,Self>`, and the `ref T`-creates-no-edge case. Run `just woc-test`; expected PASS. -- [ ] Commit. +- [x] Build a directed graph over class names: edge `A → B` when `A` has a field whose resolved type is `B`, `?B`, `multi B`, `map`, or `map<_,B>` (unwrap `Nullable`). **`ref B` contributes no edge.** Read fields from `Ast` class decls; resolve names via the symbol table `types.ml` already builds. +- [x] Run Tarjan's SCC (hand-written, stdlib only). Classify a class **traced** iff it is in a non-trivial SCC **or** has a self-loop; else **owned**. Expose `Gcinfer.classify : -> result` returning the traced-name set plus, per traced class, the reason (a cycle path for the note). +- [x] Verify with a tiny OCaml unit in `compiler/test` (or the existing runner) over: self-loop (`Node.next: ?Node`), mutual recursion (`A.b:B`, `B.a:A`), `multi Self`, `map<_,Self>`, and the `ref T`-creates-no-edge case. Run `just woc-test`; expected PASS. +- [x] Commit. ### Task 1.2 — `--dump-gc` mode + golden **Files:** Modify `compiler/bin/main.ml` (argv dispatch + usage), `compiler/src/dump.ml` (renderer). Test: new golden under `compiler/test/golden/`. -- [ ] Add a `--dump-gc ` mode: run lex→parse→types, call `Gcinfer.classify`, print one `Nameowned|gc(reason)` line per class in declaration order (the spec's `--dump-gc` artifact shape). Reason is the cycle path for structural, empty for owned. -- [ ] Add the usage line and the mode to the dispatch match (beside `--dump-owner`). -- [ ] Add a golden fixture: run `--dump-gc` over `docs/examples/gc-cycle` (expect `Node gc (cycle Node -> Node)`, `Segment owned`) and over the pricing corpus subset. Bless with `WOC_BLESS=1`. -- [ ] Run `just woc-test`; expected PASS, and **existing goldens unchanged** (no emit path touched). Commit. +- [x] Add a `--dump-gc ` mode: run lex→parse→types, call `Gcinfer.classify`, print one `Nameowned|gc(reason)` line per class in declaration order (the spec's `--dump-gc` artifact shape). Reason is the cycle path for structural, empty for owned. +- [x] Add the usage line and the mode to the dispatch match (beside `--dump-owner`). +- [x] Add a golden fixture: run `--dump-gc` over `docs/examples/gc-cycle` (expect `Node gc (cycle Node -> Node)`, `Segment owned`) and over the pricing corpus subset. Bless with `WOC_BLESS=1`. +- [x] Run `just woc-test`; expected PASS, and **existing goldens unchanged** (no emit path touched). Commit. --- @@ -74,17 +92,17 @@ front-end end to end before the collector swap. **Files:** Modify `compiler/src/gcinfer.ml`, `compiler/src/owner.ml` (add a collect-promotions mode). -- [ ] Add a mode to the ownership pass that, instead of emitting `WO-E304`/long-lived-alias errors, records the offending class. `Gcinfer` runs owner in this mode, unions the recorded classes into the traced set, and re-runs — terminating because the set only grows (bounded by class count). Each demand promotion carries its escape-site note. -- [ ] Verify: a `PriceCache`-shaped fixture (acyclic, aliased) classifies `gc (alias escape, …)` via `--dump-gc`. `just woc-test` PASS. Commit. +- [x] Add a mode to the ownership pass that, instead of emitting `WO-E304`/long-lived-alias errors, records the offending class. `Gcinfer` runs owner in this mode, unions the recorded classes into the traced set, and re-runs — terminating because the set only grows (bounded by class count). Each demand promotion carries its escape-site note. +- [x] Verify: a `PriceCache`-shaped fixture (acyclic, aliased) classifies `gc (alias escape, …)` via `--dump-gc`. `just woc-test` PASS. Commit. ### Task 2.2 — inference is the source of GC-ness; annotation errors **Files:** Modify `compiler/src/types.ml` (`is_gc_class` reads the inferred set), `compiler/src/parser.ml` (`@gc` arm → diagnostic), `compiler/src/dump.ml` (stop rendering ` @gc`), `compiler/src/emit.ml`/`disasm.ml` (class-flag provenance only; bit unchanged). Error: new WO-E1xx in `docs/plan/oop-vm/01-error-catalog.md`. -- [ ] Thread the inferred traced-set into the typing context so `Types.is_gc_class` answers from it. Field-kind derivation (→ `WO_K_GCREF`) and every `owner.ml` exemption then follow with no further change (that is the seam). -- [ ] Turn the parser's `@gc` acceptance into a WO-E1xx diagnostic pointing at inference + `--dump-gc`. Update the error catalog. -- [ ] Re-bless every golden that rendered ` @gc`, `flags=gc`, or a changed GCREF field kind (`WOC_BLESS=1`). Convert the `gc/` corpus fixtures to drop `@gc` from source (they rely on inference now). -- [ ] Verify: `docs/examples/gc-cycle` now **emits** (no WO-E301 — traced classes alias freely) and **runs** on the current runtime, printing `ring a -> b -> c -> a`. `just woc-test`, `just oop-e2e` PASS. Commit. +- [x] Thread the inferred traced-set into the typing context so `Types.is_gc_class` answers from it. Field-kind derivation (→ `WO_K_GCREF`) and every `owner.ml` exemption then follow with no further change (that is the seam). +- [x] Turn the parser's `@gc` acceptance into a WO-E1xx diagnostic pointing at inference + `--dump-gc`. Update the error catalog. +- [x] Re-bless every golden that rendered ` @gc`, `flags=gc`, or a changed GCREF field kind (`WOC_BLESS=1`). Convert the `gc/` corpus fixtures to drop `@gc` from source (they rely on inference now). +- [x] Verify: `docs/examples/gc-cycle` now **emits** (no WO-E301 — traced classes alias freely) and **runs** on the current runtime, printing `ring a -> b -> c -> a`. `just woc-test`, `just oop-e2e` PASS. Commit. --- @@ -98,33 +116,33 @@ is the largest phase and lands with Phase 2's front-end. **Files:** Modify `runtime/src/wob.h` (`wo_hdr`), `runtime/src/obj.h` (`wo_rt` list head + `wo_obj_new` links traced objects), `runtime/src/gc.{c,h}`. -- [ ] Repurpose the header: retire `rc` and (for traced objects) `borrow`; give those 8 contiguous bytes to a 64-bit intrusive sweep-list link. Keep colors in the existing `WO_F_COLOR` bits. The `_Static_assert(sizeof(wo_hdr)==16)` must still hold. -- [ ] `wo_rt` gains a traced-list head; `wo_obj_new` links a traced-class instance (class-flag bit set) in as white. Sweep recovers size via `wo_obj_size` (class table). Retire `cycbuf` and `WO_F_BUF`. -- [ ] Verify build both dispatch flavors: `just wovm-build` + `make -C runtime test test-iso`. Commit. +- [x] Repurpose the header: retire `rc` and (for traced objects) `borrow`; give those 8 contiguous bytes to a 64-bit intrusive sweep-list link. Keep colors in the existing `WO_F_COLOR` bits. The `_Static_assert(sizeof(wo_hdr)==16)` must still hold. +- [x] `wo_rt` gains a traced-list head; `wo_obj_new` links a traced-class instance (class-flag bit set) in as white. Sweep recovers size via `wo_obj_size` (class table). Retire `cycbuf` and `WO_F_BUF`. +- [x] Verify build both dispatch flavors: `just wovm-build` + `make -C runtime test test-iso`. Commit. ### Task 3.2 — tri-color incremental mark + Yuasa barrier + budgeted sweep **Files:** Modify `runtime/src/gc.{c,h}`, `runtime/src/vm.c` (roots via pc gc-mask; barrier in `SETF`/`map_set`/`push`; retire `RC_INC`/`RC_DEC` cases; safepoints at back-edges/calls). -- [ ] Delete trial deletion (`mark_gray`/`scan_black`/`scan_`/`collect_white`/`white_free`/zombie guard). Implement: roots = value/frame slots read via the per-pc gc-mask; grey worklist; mark budget `WO_GC_BUDGET`; owned objects traversed-not-freed, skipping subtrees via the precomputed "transitively-contains-gcref" class bit; sweep frees white + unlinks, repaints black→white; heap-goal trigger; `WO_GC_TRACE` per-slice counts. -- [ ] Yuasa deletion barrier: on a store into a `GCREF` slot **while marking**, shade the old value grey. Lives in the VM store paths (no new opcode). -- [ ] Verify: `just wovm-test`. Commit. +- [x] Delete trial deletion (`mark_gray`/`scan_black`/`scan_`/`collect_white`/`white_free`/zombie guard). Implement: roots = value/frame slots read via the per-pc gc-mask; grey worklist; mark budget `WO_GC_BUDGET`; owned objects traversed-not-freed, skipping subtrees via the precomputed "transitively-contains-gcref" class bit; sweep frees white + unlinks, repaints black→white; heap-goal trigger; `WO_GC_TRACE` per-slice counts. +- [x] Yuasa deletion barrier: on a store into a `GCREF` slot **while marking**, shade the old value grey. Lives in the VM store paths (no new opcode). +- [x] Verify: `just wovm-test`. Commit. ### Task 3.3 — emitter + format: retire RC, restate the gc-mask, bump `.wob` **Files:** Modify `compiler/src/emit.ml` (drop `emit_rc` + escape-acquire anchor; gc-mask now = GC roots), `compiler/src/owner.ml` (delete rc table/elision/`resolve_rc`/`release_gc`/clobber rule), interpreter (opcodes 27–28 reserved), `docs/plan/oop-vm/00-wob-format.md` (version bump + reserved opcodes + gc-mask contract), `runtime/src/loader.c` if it validates opcodes. -- [ ] Stop emitting `RC_INC`/`RC_DEC`; reserve the opcodes; bump the `.wob` version in the format doc + loader constant. Restate the drop-table gc-mask contract as "GC roots at this pc". `wo_drop_kind` for `WO_K_GCREF` becomes a no-op. -- [ ] Re-bless all affected goldens (`--dump-bc`, `--dump-owner`, disasm) with `WOC_BLESS=1`. -- [ ] Verify: `just woc-test`, `just oop-e2e`, `just oop-accept`. Commit. +- [x] Stop emitting `RC_INC`/`RC_DEC`; reserve the opcodes; bump the `.wob` version in the format doc + loader constant. Restate the drop-table gc-mask contract as "GC roots at this pc". `wo_drop_kind` for `WO_K_GCREF` becomes a no-op. +- [x] Re-bless all affected goldens (`--dump-bc`, `--dump-owner`, disasm) with `WOC_BLESS=1`. +- [x] Verify: `just woc-test`, `just oop-e2e`, `just oop-accept`. Commit. ### Task 3.4 — fixtures: adversarial barrier + cycle rewrites **Files:** Modify `runtime/test/test_cycle.c`, `runtime/test/test_rc.c`; corpus `tests/corpus/gc/{abandoned-cycle,budget-steps,held-cycle}`; add an adversarial barrier fixture. -- [ ] Rewrite `test_cycle.c`/`test_rc.c` off rc assertions onto: abandoned cycle freed, rooted cycle survives, slices bounded, sweep-list leak-free after N cycles. Re-bless `abandoned-cycle`/`budget-steps` traces; **redefine** `held-cycle` as "a cycle rooted from a live frame survives a slice" (from inside a running program). -- [ ] Add the barrier fixture: hide a traced object between slices (store into a blackened object, drop the original ref); it must survive with the barrier in and be freed (corruption) with it compiled out — the design's safety net. -- [ ] Verify: `just wovm-test`, `just oop-accept`, ASan clean after repeated cycles. Commit. +- [x] Rewrite `test_cycle.c`/`test_rc.c` off rc assertions onto: abandoned cycle freed, rooted cycle survives, slices bounded, sweep-list leak-free after N cycles. Re-bless `abandoned-cycle`/`budget-steps` traces; **redefine** `held-cycle` as "a cycle rooted from a live frame survives a slice" (from inside a running program). +- [x] Add the barrier fixture: hide a traced object between slices (store into a blackened object, drop the original ref); it must survive with the barrier in and be freed (corruption) with it compiled out — the design's safety net. +- [x] Verify: `just wovm-test`, `just oop-accept`, ASan clean after repeated cycles. Commit. --- @@ -132,8 +150,8 @@ is the largest phase and lands with Phase 2's front-end. **Files:** `docs/00-principles.md` (principle 3), the OOP spec (decision table, §3 rule 5, §4 memory model), `docs/plan/oop-vm/00-wob-format.md`, `01-error-catalog.md` (retire WO-W201, update WO-E304 wording, add the new WO-E1xx), `08-builtin-surface.md` (delete the `push` special case + `set` gap), `docs/00-status.md` (record 7b superseding iteration 2's memory model), `docs/stories/language-runtime-database/07b-inferred-gc-mark-sweep.md` (status → done), `docs/examples/gc-cycle/README.md` (flip "Run status" to shipped + wire a `just gc-cycle` acceptance). -- [ ] Apply each amendment in the spec's §8 migration table. Add a `docs/examples/gc-cycle` acceptance script + `just gc-cycle` recipe running `--dump-gc` + ring/owned demos under `WO_GC_TRACE`. -- [ ] Verify: `just oop-accept` green; `just gc-cycle` green; `git grep '@gc' -- '*.wo'` returns nothing (success criterion 1). Commit. +- [x] Apply each amendment in the spec's §8 migration table. Add a `docs/examples/gc-cycle` acceptance script + `just gc-cycle` recipe running `--dump-gc` + ring/owned demos under `WO_GC_TRACE`. +- [x] Verify: `just oop-accept` green; `just gc-cycle` green; `git grep '@gc' -- '*.wo'` returns nothing (success criterion 1). Commit. --- diff --git a/docs/superpowers/specs/2026-08-01-oop-compiler-vm-design.md b/docs/superpowers/specs/2026-08-01-oop-compiler-vm-design.md index c5d375d..3f4fa84 100644 --- a/docs/superpowers/specs/2026-08-01-oop-compiler-vm-design.md +++ b/docs/superpowers/specs/2026-08-01-oop-compiler-vm-design.md @@ -11,7 +11,7 @@ writeonce today is a declarative language executed by the Rust runtime (`crates/ - an **OCaml compiler** (`woc`) — fast compiles, no LLVM, - a **C runtime VM** (`wovm`) — libc-only, evolving out of the existing `wo-rt-c` C reference, - **memory-safe object instances**: by default an object behaves like a Rust borrowed value (single owner, checked borrows), -- a **per-class `@gc` override** for reference semantics, collected without stop-the-world pauses, +- **inferred GC-ness** (iteration 7b superseded the `@gc` annotation): classes that cycle or must alias long-lived are traced, without stop-the-world pauses, - the same end product: one binary that is the database, the web API, and the UI, running multithreaded. ## Decisions locked during brainstorming @@ -21,7 +21,7 @@ writeonce today is a declarative language executed by the Rust runtime (`crates/ | Fate of Rust runtime | **Evolve `wo-rt-c` into the C runtime.** OCaml compiler targets it. `crates/rt` stays active until parity, then retires to `.dev/reference/` like v1 did. | | OOP shape | **Keep plan 13 doctrine: no inheritance, no override, no virtual class hierarchies — ever.** OOP = `class` (state + methods) + structural **interfaces** (Go-style) + composition (`ref`/`multi`). | | Borrow enforcement | **Hybrid.** Compiler proves most sites statically and emits nothing; VM enforces residual sites with borrow-word checks at runtime. | -| GC opt-out granularity | **Per-class annotation** `@gc` — all instances of that class are GC-managed and freely aliased. | +| GC opt-out granularity | **Inferred** (amended by the 2026-08-11 7b spec; was a per-class `@gc` annotation): structural cycles via SCC over the class-reference graph + demand promotion at escape sites. Traced instances are freely aliased. | | Execution model | **Register bytecode interpreter first** (computed-goto dispatch). JIT possible later, not now. AOT-to-C rejected (kills hot reload, slow builds). | | Concurrency model | **Shard-actor with ownership transfer.** Thread-per-core shards, one heap per shard, cross-shard = message send = ownership move. GC is per-shard, so no global pause exists by construction. (Implementation is sub-project 2; milestone 1 reserves header space.) | | First sub-project | **Compiler + VM core** — proves the novel risk (hybrid borrow VM) before any HTTP/DB integration. | @@ -122,7 +122,7 @@ class PriceCache { -- reference semantics, freely aliased 2. Function parameter default = immutable borrow. `mut x: T` = exclusive borrow. `take x: T` = ownership moves in. 3. Borrows never escape: cannot be stored in a field, cannot be returned. Compile error. 4. Fields hold owned values, `ref T` ids (existing DB-style links), or `@gc` references. -5. `@gc` class instances alias freely: no borrow rules, reference-counted, cycles collected incrementally. +5. Traced (inferred-gc) class instances alias freely: no borrow rules; an incremental per-shard mark-sweep collects them (amended by the 7b spec — which classes are traced is inferred). 6. Method `self` is an immutable borrow if the body only reads, exclusive if it writes — the compiler infers this; no annotation. **Executes in milestone 1:** class/interface declarations, constructors, field access, method and interface calls, control flow (`if`/`for`/`while`/`return`), arithmetic/text operations, `let`. @@ -144,7 +144,7 @@ struct wo_hdr { uint8_t flags; // bit0 GC_MANAGED, bit1 IN_CYCLE_BUF uint8_t _pad; uint32_t borrow; // 0 = free, N = shared readers, 0xFFFFFFFF = exclusive - uint32_t rc; // strong count, @gc only; unused for owned + // (7b) the borrow word unions with the traced list's intrusive link }; // object fields follow inline ``` @@ -155,9 +155,9 @@ struct wo_hdr { - `owner.ml` proves most sites statically (locals, linear flow, no runtime-indexed aliasing) — zero ops emitted, zero runtime cost. - Residual sites get `BORROW_S` / `BORROW_X` / `RELEASE` on the borrow word. Canonical residual case: two `mut` borrows through runtime indices (`items[i]`, `items[j]` where `i == j` is unprovable). A violation is a VM trap that unwinds to the method boundary as a structured error (Section 6). -**`@gc` objects:** RC increment/decrement on alias creation/drop (compiler-emitted, elided for provably balanced pairs). `rc == 0` frees immediately. Cycle risk exists only when a `@gc` object holds `@gc`-typed fields — those go to a per-shard possible-cycle buffer on decrement (Bacon–Rajan trial deletion), scanned **incrementally with a fixed per-tick budget** on the shard's own event loop. Per-shard heap, per-shard buffer: no cross-shard tracing, no global pause; worst case is a bounded slice of one shard's tick. +**Traced objects (amended by the 7b spec, 2026-08-11):** reference counting is retired. Every traced allocation links onto a per-shard traced list; an incremental tri-color mark-sweep collects it — roots snapshot from the frames' per-pc masks at cycle start, a Yuasa deletion barrier shades overwritten gcref edges while marking, allocations mid-cycle are born black, and both mark and sweep run in budgeted slices (`WO_GC_BUDGET`). The header's old `rc`+`borrow` words are the traced list's intrusive link. Per-shard heap, per-shard list: no cross-shard tracing, no global pause. -**Mixing rule:** an owned object may hold `@gc` references (rc participates). A `@gc` object may hold owned values (it owns them; they drop when the holder is freed). The borrow word applies only to owned objects; `@gc` aliasing is unrestricted by design. +**Mixing rule:** an owned object may hold traced references (the mark phase walks owned interiors to find them). A traced object may hold owned values (it owns them; they drop when sweep frees the holder). The borrow word applies only to owned objects; traced aliasing is unrestricted by design. ## Section 5 — Bytecode and VM