docs: update story iterations (pre-compiler-front)
- 00: story framing - 01: principles doc - 02: VM core - 04: single binary e2e - 05: language surface - 08: shard-actor runtime - 09: database engine - 10: HTTP service
This commit is contained in:
parent
9baf930c7c
commit
80046556af
9 changed files with 102 additions and 107 deletions
|
|
@ -5,7 +5,7 @@
|
|||
|
||||
**AS** a developer building and operating my own products end to end
|
||||
|
||||
**I WANT** a new programming language — with arithmetic, ownership-based memory safety, and garbage collection where I opt in — whose compiler, runtime, and database ship as a single never-stopping Linux binary that can update its own code in place
|
||||
**I WANT** a new statically-typed systems programming language — object-oriented by default, with ownership-based memory safety (borrow semantics) and per-class `@gc` garbage collection where I opt in — whose compiler, runtime, and database ship as a single never-stopping Linux binary that can update its own code in place
|
||||
|
||||
**TO** write an application once and run it forever: no external stack to assemble, no database server to operate, and deployments that swap code inside the running process with instant rollback.
|
||||
|
||||
|
|
@ -17,6 +17,15 @@
|
|||
- Memory safety without a GC tax: Rust-shaped borrowing (single owner,
|
||||
second-class borrows) checked mostly at compile time, with per-class `@gc`
|
||||
opt-in collected per shard — no global pause exists by construction.
|
||||
- Static typing all the way down: every slot's type is known at compile
|
||||
time, so the VM runs untagged 64-bit registers — no `Dynamic`, no boxing
|
||||
tax, no hashed field lookups (the Haxe→C++ reference workload pays all
|
||||
three).
|
||||
- The database is the object model: a `@table` class is a table, a
|
||||
`ref`/`multi` field is a relation — the `@`-annotations are the built-in
|
||||
ORM, resolved at compile time. No external mapping layer, and no
|
||||
user-defined macros (the verdict table's rejection stands; annotations
|
||||
are compiler-known).
|
||||
- Updates are blue-green **inside** the runtime: propose, approve, compile
|
||||
in-process, atomic switch, previous version resident for instant rollback.
|
||||
- The runtime is a recipe box: once language + runtime + database exist, a
|
||||
|
|
@ -25,9 +34,10 @@
|
|||
|
||||
## Background & Constraints
|
||||
|
||||
writeonce today is a declarative Rust-based runtime (Stage 2). This story
|
||||
evolves it into an object-oriented language (`woc` OCaml compiler, `wovm` C
|
||||
VM) per the approved specs: C as the runtime's basis (libc only), no
|
||||
writeonce began as a declarative Rust-based runtime (Stage 2); that chapter
|
||||
closes here. This story pivots it into a statically-typed, object-oriented
|
||||
**systems programming language** with its own runtime and database (`woc`
|
||||
OCaml compiler, `wovm` C VM) per the approved specs: C as the runtime's basis (libc only), no
|
||||
inheritance ever, mutable value semantics for borrowing, shard-per-core
|
||||
concurrency with ownership-moving messages, RAM-authoritative data under a
|
||||
WAL, and the blue-green VM pair for in-runtime deployment. Target OS is
|
||||
|
|
@ -36,21 +46,33 @@ parity. Constraints: OCaml stdlib only, C libc only; docs live under
|
|||
`docs/`; prose-only planning artifacts (no implementation code in stories or
|
||||
iterations); no commits by agents — drafts go to `.dev/commit.md`.
|
||||
|
||||
## Iterations (review in this order)
|
||||
## Iterations (review in this order — the order is chronological)
|
||||
|
||||
| # | Iteration | Delivers |
|
||||
| --- | --- | --- |
|
||||
| 1 | [Principles doc](01-principles-doc.md) | `docs/00-principles.md` — the doctrine page every later slice links back to |
|
||||
| 2 | [VM core](02-vm-core.md) | `wovm`: `.wob` loader, register interpreter, arena, borrow word, `@gc` collector |
|
||||
| 3 | [Compiler front](03-compiler-front.md) | `woc`: lexer → parser → typechecker → ownership pass, diagnostics |
|
||||
| 4 | [Single binary end-to-end](04-single-binary-e2e.md) | emitter + conformance corpus + `woc build` self-contained binary |
|
||||
| 5 | [Language surface](05-language-surface.md) | Haxe-parity adoptions: switch, records, optionals, try/catch, statics, modules… |
|
||||
| 6 | [Program mode + stdlib](06-program-mode-stdlib.md) | `fn main`, exit codes, `fs`/`proc`/`net`/`time`/`json` builtins |
|
||||
| 7 | [log-watcher proof](07-logwatcher-proof.md) | the driving workload compiled and detecting silent deaths live |
|
||||
| 8 | [Shard-actor runtime](08-shard-actor-runtime.md) | thread-per-core shards, per-shard heaps, ownership-move messaging |
|
||||
| 9 | [Database engine](09-database-engine.md) | class-shaped tables, typed WAL + recovery, `insert`/`select` execute |
|
||||
| 10 | [HTTP service layer](10-http-service.md) | `service` blocks route to VM methods; REST parity with Stage 2 |
|
||||
| 11 | [Blue-green deploy](11-blue-green-deploy.md) | two VM slots, in-runtime compile, atomic switch, resident rollback |
|
||||
| # | Status | Iteration | Delivers |
|
||||
| --- | --- | --- | --- |
|
||||
| 1 | ✅ | [Principles doc](01-principles-doc.md) | `docs/00-principles.md` — the doctrine page every later slice links back to |
|
||||
| 2 | 🔄 | [VM core](02-vm-core.md) | `wovm`: `.wob` loader, register interpreter, arena, borrow word, `@gc` RC (cycles staged to 8) |
|
||||
| 3 | 🔄 | [Compiler front](03-compiler-front.md) | `woc`: lexer → parser → typechecker → ownership pass, diagnostics |
|
||||
| 4 | ⬜ | [Single binary end-to-end](04-single-binary-e2e.md) | emitter + conformance corpus + `woc build` self-contained binary |
|
||||
| 5 | ⬜ | [Language surface](05-language-surface.md) | Haxe-parity adoptions: switch, records, optionals, try/catch, statics, modules… |
|
||||
| 6 | ⬜ | [Program mode + stdlib](06-program-mode-stdlib.md) | `fn main`, exit codes, `fs`/`proc`/`net`/`time`/`json` builtins |
|
||||
| 7 | ⬜ | [log-watcher proof](07-logwatcher-proof.md) | the driving workload compiled and detecting silent deaths live |
|
||||
| 8 | ⬜ | [Shard-actor runtime](08-shard-actor-runtime.md) | thread-per-core shards, per-shard heaps, ownership-move messaging, `@gc` cycle collector |
|
||||
| 9 | ⬜ | [Database engine](09-database-engine.md) | class-shaped tables, typed WAL + recovery, `insert`/`select` execute |
|
||||
| 10 | ⬜ | [HTTP service layer](10-http-service.md) | `service` blocks route to VM methods; REST parity with Stage 2 |
|
||||
| 11 | ⬜ | [Fibers](11-fibers.md) | green threads on the shard scheduler: reduction-budget preemption, park-on-I/O builtins, ownership-move sends |
|
||||
| 12 | ⬜ | [Blue-green deploy](12-blue-green-deploy.md) | two VM slots, in-runtime compile, atomic switch, resident rollback |
|
||||
|
||||
Chronology notes: 2 and 3 are the one deliberate overlap — the OOP spec's
|
||||
first sub-project builds both halves concurrently (both in progress:
|
||||
`runtime/src/` carries the VM's loader/vm/obj/borrow/gc modules,
|
||||
`compiler/` has front-end tasks 1–4 of 8); they meet in 4. Iterations 1–7
|
||||
complete the single-shard language; 8–12 scale the runtime (8 is the
|
||||
substrate for everything after: the DB engine lives in its shards, HTTP
|
||||
serves from them, fibers refine their scheduler, blue-green drains them).
|
||||
Fibers (11) land before blue-green so the deploy's drain can unwind parked
|
||||
fibers as part of its own acceptance; blue-green (12) closes the story —
|
||||
its plan is authored only after 9–10 ship.
|
||||
|
||||
Review protocol: the developer reads one iteration, approves or amends;
|
||||
the next starts only after approval. Each iteration is an unsplittable
|
||||
|
|
@ -63,7 +85,7 @@ list, and a pointer to the plan document that already sequences its tasks.
|
|||
- Systems track spec: `docs/superpowers/specs/2026-08-01-systems-track-design.md`
|
||||
- Blue-green spec: `docs/superpowers/specs/2026-08-03-blue-green-vm-design.md`
|
||||
- Sample+principles spec: `docs/superpowers/specs/2026-08-07-logwatcher-sample-and-principles-design.md`
|
||||
- Plan documents: `docs/superpowers/plans/` (plans 1–10), `compiler/plan/` (2, 3, 8 + architecture)
|
||||
- Plan documents: `docs/superpowers/plans/` (plans 1–10), `docs/plan/compiler/` (2, 3, 8 + architecture)
|
||||
- Roadmap map: `docs/08-project-structure.md` (build sequence)
|
||||
- Behavioral reference workload: `~/projects/log-watcher` (Haxe daemon)
|
||||
|
||||
|
|
|
|||
|
|
@ -5,16 +5,18 @@
|
|||
|
||||
## Goals
|
||||
|
||||
- The repo gains `docs/00-principles.md`: one page stating the twelve
|
||||
- The repo gains `docs/00-principles.md`: one page stating the thirteen
|
||||
writeonce principles — the doctrine every later iteration links back to
|
||||
instead of re-arguing.
|
||||
- The twelve: one binary is the whole system; zero dependencies (kernel
|
||||
- The thirteen: one binary is the whole system; zero dependencies (kernel
|
||||
primitives only); memory safety without a GC tax (MVS ownership, opt-in
|
||||
`@gc`, per-shard collection); no inheritance ever; thread-per-core shards
|
||||
with ownership moves; the runtime never stops (blue-green slots, embedded
|
||||
source); RAM authoritative + WAL durable; samples force the grammar;
|
||||
Linux is the target; capabilities are typed builtins (no FFI); plain
|
||||
diagnostics are the product; the runtime is a recipe box.
|
||||
diagnostics are the product; the runtime is a recipe box; statically
|
||||
typed all the way to the register (no `Dynamic`, untagged VM slots,
|
||||
annotations as the compile-time ORM).
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
|
|
@ -44,6 +46,7 @@
|
|||
|
||||
## Proposed Solution
|
||||
|
||||
- Author the page with the twelve principles in the order listed in the
|
||||
governing spec; verify every link resolves; add the CLAUDE.md pointer
|
||||
- Author the page with the thirteen principles in the order listed in the
|
||||
governing spec (the thirteenth — static typing — added 2026-08-08 by
|
||||
story amendment); verify every link resolves; add the CLAUDE.md pointer
|
||||
line; record the commit draft in `.dev/commit.md`.
|
||||
|
|
|
|||
|
|
@ -8,7 +8,8 @@
|
|||
- A C, libc-only virtual machine that loads a `.wob` bytecode module and
|
||||
executes method calls with the language's memory model enforced: owned
|
||||
objects with deterministic drops, runtime borrow checks at residual
|
||||
sites, and `@gc` classes collected without stop-the-world pauses.
|
||||
sites, and `@gc` classes reference-counted — **RC only in this
|
||||
iteration**; the cycle collector is staged to iteration 8 (see Info).
|
||||
- Arithmetic and text operations execute correctly — the language's first
|
||||
observable behavior.
|
||||
|
||||
|
|
@ -27,21 +28,35 @@
|
|||
every drop, and ASan/Valgrind report zero leaks on both success and
|
||||
trap paths.
|
||||
- What to achieve?
|
||||
- **Given** a cyclic `@gc` object graph that becomes garbage,
|
||||
- **when** the collector's budgeted ticks run,
|
||||
- **then** the cycle is freed within the configured budget and no pause
|
||||
exceeds the configured slice.
|
||||
- **Given** `@gc` objects whose aliases are created and dropped,
|
||||
- **when** the last reference drops (`rc == 0`),
|
||||
- **then** the object frees immediately, elided RC pairs stay elided,
|
||||
and ASan/Valgrind report zero leaks on every acyclic fixture.
|
||||
- What to achieve?
|
||||
- **Given** a cyclic `@gc` graph that becomes garbage,
|
||||
- **when** the corpus runs,
|
||||
- **then** the leak is *expected and asserted* by a must-leak fixture —
|
||||
the recorded debt iteration 8's cycle collector retires.
|
||||
|
||||
## Out Of Scope
|
||||
|
||||
- The OCaml compiler (iteration 3) — fixtures here are assembled by the
|
||||
test tool, not compiled.
|
||||
- Threads, shards, mailboxes (iteration 8); any DB or HTTP capability.
|
||||
- The Bacon–Rajan cycle collector — staged to iteration 8, where the
|
||||
shard's event loop (its per-tick budget host) first exists. Until then a
|
||||
cyclic `@gc` graph leaks, documented and fixture-asserted.
|
||||
|
||||
## Info
|
||||
|
||||
- The 16-byte object header reserves a shard id now so iteration 8 needs no
|
||||
relayout.
|
||||
- Staging rationale (decided 2026-08-08): trial deletion under mutation is
|
||||
the subtlest piece of this iteration, while every sample to date needs
|
||||
zero cyclic `@gc` (log-watcher: none; pricing: one acyclic cache). Swift
|
||||
ships RC-without-cycles at mass scale. The header's `IN_CYCLE_BUF` flag
|
||||
bit and the possible-cycle buffer hook stay reserved, so iteration 8
|
||||
adds the scan without relayout or opcode changes.
|
||||
- Format contract: `docs/plan/oop-vm/00-wob-format.md` twinned with
|
||||
`runtime/src/wob.h`; ~40-op register instruction set, computed-goto
|
||||
dispatch.
|
||||
|
|
@ -49,6 +64,7 @@
|
|||
## Proposed Solution
|
||||
|
||||
- Execute the existing plan: `docs/superpowers/plans/2026-08-01-wob-format-and-vm-core.md`
|
||||
(16 TDD tasks: arena, object model, borrow word, RC + cycle collector,
|
||||
test assembler, validating loader, interpreter, drop-map unwinding,
|
||||
builtins, CLI + `just` gate).
|
||||
(16 TDD tasks: arena, object model, borrow word, RC, test assembler,
|
||||
validating loader, interpreter, drop-map unwinding, builtins, CLI +
|
||||
`just` gate) — with its cycle-collector task deferred: that task moves
|
||||
to iteration 8's plan, replaced here by the must-leak cycle fixture.
|
||||
|
|
|
|||
|
|
@ -35,7 +35,7 @@
|
|||
## Out Of Scope
|
||||
|
||||
- Anything beyond the milestone grammar (iteration 5 grows the surface).
|
||||
- Hot reload / deployment mechanics (iteration 11 — but the self-exec
|
||||
- Hot reload / deployment mechanics (iteration 12 — but the self-exec
|
||||
trailer this iteration ships is its foundation).
|
||||
|
||||
## Info
|
||||
|
|
@ -48,7 +48,7 @@
|
|||
|
||||
## Proposed Solution
|
||||
|
||||
- Execute the existing plan: `compiler/plan/2026-08-01-wob-emit-e2e-single-binary.md`
|
||||
- Execute the existing plan: `docs/plan/compiler/2026-08-01-wob-emit-e2e-single-binary.md`
|
||||
(emitter with ownership lowering + drop maps + vtables, corpus harness,
|
||||
pricing corpus, ownership/trap corpora, gc pump e2e, `woc build` trailer,
|
||||
`just oop-accept` gate over the five spec success criteria).
|
||||
|
|
|
|||
|
|
@ -48,6 +48,6 @@
|
|||
|
||||
## Proposed Solution
|
||||
|
||||
- Execute the existing plan: `compiler/plan/2026-08-01-haxe-parity-language.md`
|
||||
- Execute the existing plan: `docs/plan/compiler/2026-08-01-haxe-parity-language.md`
|
||||
(nine tasks, each shipping its fixtures and error-catalog entries in the
|
||||
same task).
|
||||
|
|
|
|||
|
|
@ -11,6 +11,10 @@
|
|||
state never exists.
|
||||
- The language grows `spawn` and message send; garbage collection stays
|
||||
per-shard, so no global pause appears at any core count.
|
||||
- The `@gc` story completes here: the Bacon–Rajan cycle collector (staged
|
||||
out of iteration 2) lands on the shard's own event loop — the per-tick
|
||||
budget host it was always specified to run on — retiring iteration 2's
|
||||
documented cycle leak.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
|
|
@ -29,11 +33,17 @@
|
|||
- **when** code attempts to send it cross-shard,
|
||||
- **then** the compiler rejects it — aliased references cannot cross
|
||||
heap boundaries.
|
||||
- What to achieve?
|
||||
- **Given** a cyclic `@gc` object graph that becomes garbage,
|
||||
- **when** the shard's budgeted collection ticks run,
|
||||
- **then** the cycle is freed within the configured budget, no pause
|
||||
exceeds the configured slice, and iteration 2's must-leak fixture
|
||||
flips to must-collect.
|
||||
|
||||
## Out Of Scope
|
||||
|
||||
- Fibers/green threads (recorded in the blue-green vision §3; extends this
|
||||
scheduler later).
|
||||
- Fibers/green threads — iteration 11 extends this scheduler; research at
|
||||
`docs/plan/exploration/fibers/00-fibers.md`.
|
||||
- Cross-shard transactions (the database iteration's 2PC concern, later).
|
||||
|
||||
## Info
|
||||
|
|
@ -48,4 +58,6 @@
|
|||
- Execute the existing plan: `docs/superpowers/plans/2026-08-01-shard-actor-vm-runtime.md`
|
||||
(pinned-worker scheduler, shard-stamped heaps, MPSC mailbox rings + mail
|
||||
eventfds, send-as-move with home-routed frees, gc pacing per tick,
|
||||
spawn/send surface, actor corpus).
|
||||
spawn/send surface, actor corpus) — plus the cycle-collector task
|
||||
adopted from iteration 2's plan: possible-cycle buffer on RC decrement,
|
||||
trial-deletion scan under the per-tick budget.
|
||||
|
|
|
|||
|
|
@ -6,8 +6,10 @@
|
|||
## Goals
|
||||
|
||||
- The language's oldest promise executes on the new runtime: every class is
|
||||
a table. `insert` and `select` stop trapping (`DB_STUB` retires) and run
|
||||
against class-shaped row storage inside the VM's shards.
|
||||
a table, every `ref`/`multi` field a relation — the `@`-annotations
|
||||
(`@table`, `@unique`) are the built-in ORM, mapped at compile time with no
|
||||
external layer. `insert` and `select` stop trapping (`DB_STUB` retires)
|
||||
and run against class-shaped row storage inside the VM's shards.
|
||||
- Data survives anything: a typed write-ahead log with ack-after-fsync,
|
||||
parallel boot replay, and a crash battery proving no committed row is
|
||||
ever lost and no half-applied transaction ever visible.
|
||||
|
|
@ -45,6 +47,11 @@
|
|||
- Doctrine: RAM is authoritative; the WAL makes it durable; indexes drift
|
||||
unless writes go through the row API — the Rust runtime learned this
|
||||
lesson, the C engine enforces it.
|
||||
- Annotations, not macros: `@table`/`@unique` are compiler-known and
|
||||
resolved at compile time; user-defined macros stay rejected (systems-track
|
||||
verdict table). A `ref T` field compiles to a typed row id (FK); `multi T`
|
||||
to the owning-side collection edge — the relation model the examples
|
||||
(pricing's `Product`/`Price`) already write.
|
||||
- The wo-db overlap manifest keeps the C++ prototype and this engine
|
||||
answer-compatible where features overlap.
|
||||
|
||||
|
|
|
|||
|
|
@ -16,7 +16,7 @@
|
|||
|
||||
- What to achieve?
|
||||
- **Given** the blog sample's `service` declarations,
|
||||
- **when** the compiled binary boots and `reference/rest/blog.rest`
|
||||
- **when** the compiled binary boots and `.dev/reference/rest/blog.rest`
|
||||
runs against it,
|
||||
- **then** every request in the smoke file answers as documented —
|
||||
including the intentional 501/405/404 responses.
|
||||
|
|
@ -36,7 +36,7 @@
|
|||
## Out Of Scope
|
||||
|
||||
- WebSocket/live subscriptions and UI (the parked `##ui` story).
|
||||
- The management plane endpoints (iteration 11 builds them on this
|
||||
- The management plane endpoints (iteration 12 builds them on this
|
||||
machinery).
|
||||
|
||||
## Info
|
||||
|
|
|
|||
|
|
@ -1,65 +0,0 @@
|
|||
# Iteration 11 — blue-green in-runtime deployment
|
||||
|
||||
> Format: `product/story-iteration-template`. Part of
|
||||
> [Story — one language, one runtime, one database, one binary](00-story.md).
|
||||
|
||||
## Goals
|
||||
|
||||
- The story's closing promise: the running binary updates its own code.
|
||||
Two fixed VM slots (activity alternating); a proposal pipeline —
|
||||
propose → approve → in-runtime compile → additive schema migration →
|
||||
load → health → atomic switch — with the previous version staying
|
||||
resident as the instant rollback target.
|
||||
- The developer drives it remotely: `wo remote pull / propose / diff /
|
||||
approve / rollback / status` over a loopback management transport,
|
||||
every stage streaming live and WAL-audited.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- What to achieve?
|
||||
- **Given** a running fixture app and an additive code+schema change,
|
||||
- **when** the developer proposes and approves it,
|
||||
- **then** the deploy completes with zero dropped requests (in-flight
|
||||
work drains on the old slot), and the binary's embedded source
|
||||
trailer matches the new active version afterward.
|
||||
- What to achieve?
|
||||
- **Given** a failure at any pipeline stage (compile diagnostic,
|
||||
destructive-change rejection, load failure, health failure, drain
|
||||
timeout),
|
||||
- **when** it occurs,
|
||||
- **then** the active slot keeps serving untouched, the failure is
|
||||
visible in the SSE stream and the WAL trail, and a destructive
|
||||
change was rejected at propose time naming the offending
|
||||
declaration.
|
||||
- What to achieve?
|
||||
- **Given** a completed deploy,
|
||||
- **when** `wo remote rollback` runs,
|
||||
- **then** the previous version serves again in under one second with
|
||||
no compile and no data change — additive-only migration guarantees
|
||||
old code runs correctly against the migrated schema.
|
||||
- What to achieve?
|
||||
- **Given** kill -9 during COMPILING / MIGRATING / SWITCHING /
|
||||
trailer-rewrite,
|
||||
- **when** the unit restarts,
|
||||
- **then** it serves one consistent version and the WAL shows whole
|
||||
migrations only.
|
||||
|
||||
## Out Of Scope
|
||||
|
||||
- Script-based/destructive migrations, in-runtime editing workspace,
|
||||
MCP/agent wrapper over the management plane — all recorded follow-ups.
|
||||
- Fibers (the vision's §3; a scheduler concern, not a deploy concern).
|
||||
|
||||
## Info
|
||||
|
||||
- Approved spec: `docs/superpowers/specs/2026-08-03-blue-green-vm-design.md`;
|
||||
its implementation plan is deliberately authored only after iterations
|
||||
9–10 ship (prerequisites: a catalog to diff, HTTP machinery to build on).
|
||||
- VMs own code; the engine owns data — the separation that makes the
|
||||
switch cheap and rollback unconditional.
|
||||
|
||||
## Proposed Solution
|
||||
|
||||
- Author the implementation plan from the approved spec once iterations
|
||||
9–10 land, then execute it (slots, deploy state machine, additive
|
||||
differ, management surface + SSE, `wo remote` verbs, crash battery).
|
||||
Loading…
Reference in a new issue