writeonce/docs/examples/gc-cycle/README.md
shoney.arickathil c0b0dbb846 docs: audit all markdown against the code, fix findings, flatten status folders
- README: shipped concurrency/HTTP/WebSockets sat in the roadmap as "not yet
  available"; "no package manager" contradicted [deps]; the deps example
  would not have compiled (the key IS the module name)
- runtime/README: leads with wovm, wo-rt.c demoted to a historical section;
  dropped 2 nonexistent recipes, crates/rt, @gc refcounting, 13 suites -> 18
- employee + log-watcher READMEs claimed "does not compile"; both are gates
- error catalog: +10 emitted codes incl WO-E250, the only diagnostic the
  shipped query surface raises; recorded why the sweep rotted
- language-surface: group-by parses, then the typechecker refuses it
- 00-code-review + 00-link-audit re-run; history kept, not rewritten
- 48 dead Rust-era exploration links de-linked rather than re-pointed (their
  prose names the retired plan by number); successor map -> discarded.md
- 08-project-structure: compiler/plan/ never existed; corpus has 9 dirs, 5 empty
- releasing.md: dropped a --draft step the workflow never had
- new docs/00-doc-audit.md: findings + disposition, incl one row where the
  audit was wrong and the doc it accused was right
- status folders removed: 34 stories flat, status only in frontmatter; 252
  links recomputed from resolved paths; board/board-views/structure retaught
- story 24 -> in-progress, since frontmatter is now the only truth
- new iteration 38: fs mutation verbs + net.connect, the two capability
  families no iteration owned
- new iteration 39: gofiber/fiber v3.5.0 parity study. The ledger called
  CSRF/sessions unblocked by iteration 34's HMAC, but the runtime has no
  source of randomness at all
- linkcheck skips .dev/.superpowers: 0 broken paths, 0 bad anchors

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 19:20:22 +02:00

244 lines
12 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# gc-cycle — inferred GC + incremental mark-sweep, by example
The smallest program that needs a tracing collector (`ring_demo`) and the
smallest that does not (`owned_demo`). This README defines **how pointers/
addresses flow through the heap** and **the collector's mark-sweep logic**, the
way iteration 7b specs it
([`2026-08-11-inferred-gc-mark-sweep-design.md`](../../superpowers/specs/2026-08-11-inferred-gc-mark-sweep-design.md)).
The one-line model: **ownership frees everything it can; the collector traces
only the residue ownership cannot free — cycles and long-lived aliases — and
the compiler infers which types those are.**
---
## 1. Inference — the compiler decides `owned` vs `gc`
A pass between `types` and `owner` (`compiler/src/gcinfer.ml`) builds a
**class-reference graph** — an edge `A → B` whenever `A` has a field of type
`B`, `?B`, `multi B`, `map<B,_>`, or `map<_,B>`. **`ref B` creates no edge** (it
is a row id, `Copy`). Tarjan's SCC over that graph: any class in a non-trivial
SCC, or with a self-loop, is **traced (`gc`)**. Everything else is **`owned`**.
For this sample, `woc --dump-gc` prints:
```
Node gc (cycle Node -> Node)
Segment owned
```
`Node.next: ?Node` is the self-loop → `Node` is traced, and its `next` field
gets kind `WO_K_GCREF`. `Segment` has no class-typed field → no edge → `owned`,
freed deterministically. A second, **demand** half promotes a type when the
ownership pass would otherwise report a long-lived-alias escape (the
`PriceCache` shape) — not exercised here; every promotion is reported, never
silent.
---
## 2. Address flow — where a pointer lives, and the object's shape
Every heap object is a 16-byte `wo_hdr` followed by `field_cnt` 8-byte slots
(`wo_fields(o)`; `wo_obj_size = 16 + field_cnt*8`). Allocation is the arena:
bump + 16-byte size-class free lists (16…1024), larger falls to `malloc`. **The
arena keeps no size headers and cannot enumerate objects** — so traced objects
thread an intrusive **sweep list**.
The header is where iteration 7b pays for the collector with **zero growth**:
```
today (RC) under 7b (tracing)
┌──────────────────────────┐ ┌──────────────────────────┐
│ class_id (4 bytes) │ │ class_id (4 bytes) │
│ flags (1) + pad │ │ flags (1) ── 2 color bits (white/grey/black), + pad
│ borrow (4 bytes) │ │ sweep-list link (8 bytes)│ ← reclaimed from
│ rc (4 bytes) │ │ (next traced object) │ borrow + rc
└──────────────────────────┘ └──────────────────────────┘
16 bytes 16 bytes (unchanged)
```
`rc` is retired (no reference counting); traced objects are exempt from borrow
rules so `borrow` is dead too — the two adjacent 4-byte words become one 64-bit
list link. Color lives in 2 existing flag bits (`WO_F_COLOR`).
A traced pointer (the address of a `wo_hdr`) is only ever held in one of three
places, and these are exactly what the collector reads:
| Holder | How the collector sees it |
| --- | --- |
| a **VM value-stack / frame slot** (a local like `a`) | a **root**, via the per-pc **gc-mask** the emitter already emits |
| a **`GCREF` field slot** of another object (`a.next`) | followed during **mark** |
| a **container** item (`multi`/`map`) whose element kind is `GCREF` | followed during mark |
Here is the sample's heap after `ring_demo` builds the ring, while `a`/`b`/`c`
are still live roots:
```mermaid
flowchart LR
subgraph STACK["VM value stack (roots this pc, from the gc-mask)"]
A["a"]:::root
B["b"]:::root
C["c"]:::root
end
subgraph HEAP["arena (one mmap region)"]
NA["Node a<br/>hdr, label(TEXT), next(GCREF)"]
NB["Node b<br/>hdr, label(TEXT), next(GCREF)"]
NC["Node c<br/>hdr, label(TEXT), next(GCREF)"]
end
A --> NA
B --> NB
C --> NC
NA -->|next| NB
NB -->|next| NC
NC -->|next, closes cycle| NA
NA -.->|sweep link| NB
NB -.->|sweep link| NC
NC -.->|sweep link| NULL(("nil"))
classDef root fill:#2b6,stroke:#083,color:#fff;
```
Solid arrows are `GCREF` pointers the mark phase follows; the dotted chain is
the per-shard **traced list** the sweep phase walks (independent of
reachability). When `a`/`b`/`c` leave scope, the three solid *root* arrows
vanish — the ring still points to itself, but nothing points *in*, so it is
unreachable yet un-freed. Ownership cannot help: `b` cannot be owned by both
`let b` and `a.next`. That is the collector's entire job.
---
## 3. Mark-sweep logic — tri-color, incremental, with a deletion barrier
Marking runs in **budgeted slices** (`WO_GC_BUDGET` objects per slice), so the
pause is bounded regardless of heap size. Colors: **white** = unproven (candidate
to free), **grey** = reachable but children not yet scanned, **black** = reachable
and scanned.
```mermaid
flowchart TD
ALLOC["allocate traced object<br/>color = WHITE, link into traced list"] --> LIVE
LIVE["mutator runs<br/>(program executes)"] --> TRIG{"traced bytes since last cycle<br/>past heap goal?"}
TRIG -- no --> LIVE
TRIG -- yes --> ROOTS["START CYCLE (snapshot)<br/>shade every root GREY<br/>(value/frame slots via pc gc-mask)"]
ROOTS --> SLICE
SLICE["MARK SLICE (budgeted)<br/>pop a GREY object,<br/>scan its GCREF fields +<br/>owned subtrees that may reach a gcref,<br/>shade each WHITE child GREY,<br/>then paint this object BLACK"] --> GREY{"grey set empty?"}
GREY -- "no (budget hit)" --> SAFE["yield at next safepoint<br/>(loop back-edge / call)"]
SAFE --> LIVE2["mutator resumes<br/>(barrier active)"]
LIVE2 --> SLICE
GREY -- yes --> SWEEP["SWEEP: walk traced list —<br/>WHITE: free (wo_obj_size) and unlink;<br/>BLACK: repaint WHITE, keep"]
SWEEP --> DONE["cycle done"] --> LIVE
```
**Roots.** The value stack and frame stack, read through the per-pc gc-mask the
emitter already produces (the drop-table's gc bits, reinterpreted from "rc_dec
these on unwind" to "these registers are GC roots at this pc"). No stack
scanning, no native frames — a bytecode VM with an explicit value stack has the
map for free.
**Owned objects are traversed, never freed.** An `owned` object can hold a
`GCREF` field, so mark must walk through owned subtrees to reach traced
objects — but it never frees an owned object (drop owns those). A precomputed
class-table bit, **"transitively contains a gcref"**, lets mark skip any owned
subtree that can reach no traced object at all.
**The barrier — why incremental is safe.** Between slices the mutator keeps
running and can hide a live object from a half-finished mark: store a white
object into an already-**black** object, then drop the original grey/white
reference to it. A **Yuasa deletion barrier** closes this: on any deletion of a
`GCREF` edge **while marking is active** — a `SETF` overwrite, or an owned
holder dying with a gcref inside (every such path funnels through
`wo_drop_kind`) — the **old** target is shaded grey first. No new opcode:
`SETF` already resolves the field kind from the class table. Owned stores,
scalars, and Text pay nothing. Snapshot-at-beginning completes the argument:
roots are scanned atomically when the cycle starts, and objects allocated
mid-cycle are born black — so anything reachable at the snapshot, or created
after it, survives; that is what removes Go's Dijkstra insertion-half and the
stack rescan.
**Trigger & budget.** A cycle starts when the shard's traced bytes since the
last cycle cross a heap goal; each slice marks at most `WO_GC_BUDGET` objects;
`WO_GC_TRACE` prints freed/marked counts per slice. Collection is **per-shard**
— ownership moves mean no traced object spans shards, so there is no global
stop-the-world and no cross-shard tracing.
---
## 4. The owned path, for contrast (no collector at all)
`owned_demo` builds a `Segment`. At the closing `}` the drop table lists its
register in the **owned mask**; the VM frees the object and its `Text` field
deterministically and immediately. No color, no list link, no barrier, no
slice. This is the common case, and log-watcher proves it scales: 35 classes,
**zero** traced, arena + deterministic drops end to end.
```mermaid
flowchart LR
NEW["let s = Segment (from, len)"] --> USE["use s"] --> SCOPE["scope end"]
SCOPE --> DROP["DROP (owned mask):<br/>free s.from (Text), free s"]
DROP --> GONE["reclaimed — collector never involved"]
```
So a `.wo` program has **two** reclamation systems working together: ownership
(deterministic, free, the 99%) and tracing (only the cyclic/aliased residue,
inferred). The developer writes no memory annotations for either.
---
## Run status
Iteration 7b landed 2026-08-18 (plan:
[`../../superpowers/plans/2026-08-18-inferred-gc-mark-sweep.md`](../../superpowers/plans/2026-08-18-inferred-gc-mark-sweep.md)).
This sample compiles and runs on today's toolchain — `woc
docs/examples/gc-cycle/` then `./docs/examples/gc-cycle/target/gc-cycle`.
> **It has no `just` recipe.** The plan's phase 4 checked off a
> `just gc-cycle` acceptance that never landed; the sample is the one
> compiling example in the repo with no gate behind it. Run it by hand,
> and see the plan's 2026-08-26 disclosure note.
**Phase 1 (landed).** The inference pass classifies each class; `woc --dump-gc
docs/examples/gc-cycle` prints:
```
Node gc (cycle Node -> Node)
Segment owned
```
**Phase 2a (landed).** `Types.is_gc_class` is now inference-first, so `Node` is
traced with **no annotation** and *traced classes alias freely* — the ring
**compiles** (the old `WO-E301: use of \`a\` after it was moved` at `c.next = a`
is gone), and its bytecode is byte-identical to writing `@gc class Node`.
**Phase 3 (landed).** The runtime collector is the incremental tri-color
mark-sweep this README describes: RC and trial deletion are gone, the header
carries the sweep-list link, opcodes 27–28 are reserved (`.wob` v4), and the
Yuasa deletion barrier lives in the store paths. **The ring runs and is
reclaimed**: `woc --emit docs/examples/gc-cycle -o gc.wob && WO_GC_TRACE=1
wovm gc.wob` prints `ring a -> b -> c -> a` then
`gc: step 1 budget=64 freed=3 remaining=0`, ASan-clean; with a tiny goal
(`WO_GC_GOAL=64`) a mid-program cycle runs while the ring is rooted and
correctly frees nothing. (The old RC runtime couldn't even store a `?Node`
gcref field — the unconditional RC_DEC of the nil old value trapped; that
opcode no longer exists.)
**Phase 2b (landed).** Demand promotion: the ownership pass, run in collect
mode, promotes any class whose value *must escape* (returned, stored where it
outlives its scope) — the acyclic-but-aliased case inference by structure cannot
see, targeting the escaping projection's type precisely. So GC-ness is fully
inferred: cycles by structure + aliasing by demand.
**`@gc` removed from the language (landed).** The keyword is now a hard error
(`WO-E104`): a developer never writes or mentions it; `woc --dump-gc` shows what
inference decided. WO-W201 (which suggested `@gc`) is retired. No `.wo` in the
repo carries `@gc`. This is the front-end half of iteration 7b, complete.
When 7b lands, the acceptance is:
- `woc --dump-gc docs/examples/gc-cycle` classifies `Node gc (cycle …)` /
`Segment owned`.
- `ring_demo` prints `ring a -> b -> c -> a`, and after the roots die the ring
is collected within budgeted slices (observable via `WO_GC_TRACE`), ASan-clean
after repeated cycles.
- The adversarial barrier fixture (hide a node between slices) frees the node
when the barrier is compiled out and keeps it when it is in.
Files: [`types.wo`](types.wo) (the two shapes), [`main.wo`](main.wo) (the two
demos), [`wo.toml`](wo.toml).