writeonce/docs/stories/language-runtime-database/07b-inferred-gc-mark-sweep.md
shoney.arickathil 1fe808b7a4 docs(stories): add readiness, retire status: refine, sweep all 47 iterations
- `readiness: ready | refine` is a SECOND axis, orthogonal to status.
  `ready` = the brainstorm is complete and the decisions are LOCKED (a spec
  approved, or the forks explicitly confirmed). `refine` = open forks remain
  and it cannot be planned yet
- `status: refine` RETIRED because it carried both meanings at once, so a held
  iteration with an approved spec (language 18, 26) was indistinguishable from
  one nobody had thought about. status is now purely where the WORK is:
  done | in-progress | pending | hold — `pending` was already the board's own
  rendering word, so nothing new was invented
- all 47 iterations classified from EVIDENCE in their own text, not by guess:
  "the four forks are SETTLED" / "spec + plan approved" / "Approved spec:" for
  ready; "Forks the spec must settle" / "no spec exists yet" for refine. Every
  shipped iteration is ready by definition. 19 done, 5 in-progress, 15
  pending, 8 hold; 27 ready, 20 refine
- two iterations moved refine -> in-progress rather than -> pending: language
  31 and 34 are absorbed into 24 and work on them is literally happening, which
  the board already showed as 🔄 while their frontmatter said otherwise. That
  disagreement is now gone
- board legend, board-views' frontmatter contract, and two new Dataview
  queries updated — the useful one being `readiness: ready AND status:
  pending`, the startable set

WHAT THE NEW AXIS IMMEDIATELY SURFACED: of 15 pending iterations, exactly ONE
is startable — databasev2 4, io_uring group-commit, whose forks were confirmed
settled 2026-08-20. Everything else pending needs a brainstorm first. That was
invisible while one key carried both meanings, and it is now on the board.

Also caught by the sweep, unrelated to readiness but found by cross-checking
frontmatter against the board: SIX duplicate rows. Every iteration moved into
databasev2 was still listed in the LANGUAGE pending table under its retired id
(23, 32, 33, 20, 21, 27) as well as its new one. Stale copies removed. And two
databasev2 rows made claims the sweep contradicts — iteration 1 was billed
"startable today" while its forks are open, and 6 still called itself the
ceiling-raiser after 2 took that role.

Docs only. linkcheck 0 broken / 0 anchors.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 16:54:45 +02:00

7.5 KiB
Raw Blame History

iteration status readiness
7b done ready

Iteration 7b — inferred GC + incremental mark-sweep

Format: product/story-iteration-template. Part of Story — one language, one runtime, one database, one binary.

Inserted 2026-08-11, after the plan was first drawn — hence 7b rather than a renumber. It sits here because the log-watcher critical path (iterations 3–7) must not be delayed, and because the collector should be settled before iteration 8 multiplies shards.

Status (2026-08-18): LANDED (branch inferred-gc; plan 2026-08-18-inferred-gc-mark-sweep.md). The front end infers GC-ness (structural SCC + demand promotion, woc --dump-gc), @gc in source is WO-E104, and the runtime's RC + Bacon–Rajan collector is replaced by an incremental per-shard tri-color mark-sweep with a Yuasa deletion barrier — .wob is v4, opcodes 27–28 reserved. The worked example is docs/examples/gc-cycle: its ring compiles with no annotation, runs, and is reclaimed in budgeted slices, ASan-clean. All four recorded @gc/RC defects are deleted by construction; just oop-accept is fully green (criterion 3's ASan clause included).

(Historical status 2026-08-14: not on the log-watcher critical path — 35 classes, none gc, arena + deterministic drops end to end. That is still true; log-watcher simply never allocates a traced object.)

Goals

  • The developer stops deciding which types are garbage collected. @gc disappears from the language; the compiler infers GC-ness and reports every decision with its reason.
  • Reference counting is replaced by an incremental per-shard tri-color mark-sweep collector, so the compiler no longer has to emit a balanced acquire/release at every alias site — the source of every recorded @gc defect.
  • Milestone 1's acceptance criterion 3 (ASan-clean across the corpus) closes, because the leak blocking it is one of the defects this deletes.

Acceptance Criteria

  • What to achieve?
    • Given a class whose declaration can form a reference cycle, and a separate class that is only ever shared through a long-lived alias,
    • when the program is compiled,
    • then both are classified GC-managed without any annotation, and --dump-gc names the reason for each — a cycle path for the first, the escaping alias site for the second.
  • What to achieve?
    • Given any .wo source containing @gc,
    • when it is compiled,
    • then it is a diagnostic pointing at inference and --dump-gc, not a silently accepted no-op.
  • What to achieve?
    • Given a program that hides a traced object from the collector — storing it into an already-blackened object between marking slices and dropping the original reference,
    • when the collector completes,
    • then the object is still alive; and the same fixture fails loudly if the write barrier is compiled out.
  • What to achieve?
    • Given an abandoned cycle and a cycle still rooted from a live frame,
    • when collection runs,
    • then the abandoned one is freed within budgeted slices with no slice exceeding the configured budget, and the rooted one survives.
  • What to achieve?
    • Given the whole conformance corpus, run repeatedly so several collection cycles occur,
    • when it runs under ASan,
    • then zero leaks and zero errors — the clause that currently fails.
  • What to achieve?
    • Given any emitted .wob image,
    • when it is disassembled,
    • then no RC_INC or RC_DEC appears, and the format doc records opcodes 27–28 as reserved behind a version bump.

Out Of Scope

  • Cross-shard tracing — ownership moves mean no traced object spans shards.
  • Generational collection and compaction. Non-moving is load-bearing: no forwarding pointers, no read barrier. Go's collector is not generational either.
  • Scheduler-integrated pacing beyond the heap-goal trigger; that stays iteration 8's concern, which is part of why this lands first.
  • ref T semantics, unchanged — it is an id, not a pointer, and creates no edge in the inference graph.

Info

  • Governing spec: docs/superpowers/specs/2026-08-11-inferred-gc-mark-sweep-design.md.
  • Gated by the benchmark (2026-08-15): this is the "implement garbage collection" lever of the performance arc — tri-color mark-sweep replacing RC changes the write path's tail latency, so landing it means re-running iteration 22 and recording the delta (does tracing help or hurt p99 under write load?).
  • Constraint added by the database track (2026-08-15): a GC-managed value in a @table field is a compile error (the engine/heap bulkhead — 9b design, section 6). Once GC-ness is inferred rather than annotated, the inference pass must classify every class before table-field validation, and the diagnostic must name the inference reason ("class X is garbage-collected via Y and cannot be stored in a table field") — otherwise the error becomes unactionable exactly when it stops being self-evident.
  • Why the annotation was insufficient, not merely inconvenient: the OOP spec's own example, @gc class PriceCache { entries: map<SKU, Money> }, is acyclic. It needs GC because it is shared, and second-class borrows cannot be stored or returned. So the developer was being asked to reason about type shape and whole-program aliasing at once — hence the hybrid rule (structural SCC plus reported demand promotion).
  • Why tracing rather than better reference counting: all four recorded @gc defects are RC bookkeeping failures — push missing an increment, set still missing one, the mut-@gc clobber, and the held-cycle leak. Inferring GC-ness would widen that population and so widen that bug class. Tracing emits no per-alias bookkeeping at all.
  • Most of what tracing needs already exists. The emitter already produces precise per-pc pointer masks (the drop table's gc mask) and the class table already carries per-field kinds — the two pieces Go gets from stack maps and type maps. Go's dependence on OS threads is incidental; the algorithm needs only per-frame PC→map lookup and the ability to suspend one stack.
  • The one real runtime addition: the arena cannot enumerate objects — bump allocation plus size-class free lists, with large objects on bare malloc and no size headers anywhere. Sweep needs its own list. Retiring rc, plus the borrow word that traced objects never use, frees exactly eight contiguous bytes for an intrusive link, so the 16-byte header survives.
  • This iteration supersedes part of iteration 2's memory model (§4 of the OOP spec) and closes iteration 4's open gate clause. Neither is renumbered; both carry pointers here.

Proposed Solution

  • Write the implementation plan from the approved spec, then execute it: the gcinfer.ml pass (Tarjan SCC over the class-reference graph, then demand promotion to a fixpoint, with a note per decision), the @gc removal and its diagnostic, retiring RC_INC/RC_DEC and the rc machinery from owner.ml/emit.ml, the per-shard traced list and sweep, incremental tri-color marking with roots read from the existing pc masks, the Yuasa deletion barrier inside the VM's store paths, and the doc/golden migration the spec's §8 table enumerates.