diff --git a/docs/00-status.md b/docs/00-status.md index 0cfd7ae..76e9319 100644 --- a/docs/00-status.md +++ b/docs/00-status.md @@ -124,6 +124,7 @@ that sequences its tasks. Read one, approve, then the next starts. | 10 | [HTTP service layer](stories/language-runtime-database/10-http-service.md) | ⬜ | Hold | | 11 | [Fibers](stories/language-runtime-database/11-fibers.md) | ⬜ | Hold | | 12 | [Blue-green deploy](stories/language-runtime-database/12-blue-green-deploy.md) | ⬜ | Hold | +| 13 | [Compile-time metaprogramming](stories/language-runtime-database/13-compile-time-metaprogramming.md) | ⬜ needs a spec first | --- diff --git a/docs/stories/language-runtime-database/00-story.md b/docs/stories/language-runtime-database/00-story.md index 745c027..3e9648e 100644 --- a/docs/stories/language-runtime-database/00-story.md +++ b/docs/stories/language-runtime-database/00-story.md @@ -60,6 +60,7 @@ iterations); no commits by agents — drafts go to `.dev/commit.md`. | 10 | [HTTP service layer](10-http-service.md) | `service` blocks route to VM methods; REST parity with Stage 2 | | 11 | [Fibers](11-fibers.md) | green threads on the shard scheduler: reduction-budget preemption, park on I/O | | 12 | [Blue-green deploy](12-blue-green-deploy.md) | two VM slots, in-runtime compile, atomic switch, resident rollback | +| 13 | [Compile-time metaprogramming](13-compile-time-metaprogramming.md) | `@derive(Json/Csv/Eq/Hash/Show)` — the compiler generates per-type code from the class-table metadata; generic capabilities within principle 13, no reflection | Review protocol: the developer reads one iteration, approves or amends; the next starts only after approval. Each iteration is an unsplittable diff --git a/docs/stories/language-runtime-database/13-compile-time-metaprogramming.md b/docs/stories/language-runtime-database/13-compile-time-metaprogramming.md new file mode 100644 index 0000000..0ed1e3d --- /dev/null +++ b/docs/stories/language-runtime-database/13-compile-time-metaprogramming.md @@ -0,0 +1,167 @@ +# Iteration 13 — compile-time metaprogramming (derive from the class table) + +> Format: `product/story-iteration-template`. Part of +> [Story — one language, one runtime, one database, one binary](00-story.md). +> +> **Inserted 2026-08-16.** A language-capability iteration, deliberately +> numbered to echo the principle it lives inside: +> [principle 13, "statically typed, all the way to the register"](../../00-principles.md). +> It comes late because it earns its keep only once there are enough +> types worth deriving over (the `@table` classes of iteration 9/9b, the +> records the query surface projects), and it must never be the excuse that +> re-opens a dynamic hole. +> +> **No spec exists yet.** The forks in *Info* are genuine decisions. + +## Why this iteration exists + +Principle 13 forbids runtime reflection: no `Dynamic`, no runtime type tags, +no walking an unknown value's fields at run time. That ban is correct — the +untagged VM, the borrow checker, and the ORM all stand on it. But it leaves a +real gap: a *generic* capability like "serialize any type to CSV" cannot be a +user-written function, because such a function would need to enumerate a +value's fields at run time, which is exactly what is forbidden. Today the only +escape is a **hand-written function per type**, or a **single compiler +builtin** (`json.encode`) that already does the right thing — it is lowered by +the compiler and walks the class-table metadata (`field_names` / `field_class` +/ `field_elem`, `.wob` v2/v3), never a runtime type tag. + +Rust faced the identical ban (it has no runtime field reflection either) and +answered with **compile-time metaprogramming**: `#[derive(Serialize)]` reads a +type's fields *at compile time* and emits per-type field-naming code, so +`serde` serializes any deriving type with zero reflection. `json.encode` is, +in effect, a single hand-built instance of exactly that mechanism. This +iteration **generalizes `json.encode`'s mechanism into a reusable derive +facility**: the compiler generates per-type code from the class-table metadata +it already emits, so generic-feeling capabilities exist *within* principle 13 +rather than against it. + +## Goals + +- **A closed, compiler-known set of derivable capabilities** requestable on a + class — the first set: `Json` (retrofitting the existing `json.encode`), + `Csv`, structural `Eq`, `Hash`, and `Show` (a debug rendering). Each is + generated by the compiler from the class's field names and kinds; none is a + runtime reflective loop. +- **Generation preserves principle 13 exactly.** The emitted code is ordinary + bytecode over statically-known offsets and kinds — monomorphic per type, no + `Dynamic`, no runtime type tag, no dynamic dispatch. Disassembly must show + a plain per-type routine, not a reflection opcode. +- **`json.encode` becomes the `Json` derive**, reimplemented on the framework + so the framework is proven by rebuilding the thing that already works — + byte-identical output, or the change is wrong. +- **The query-result serialization gap closes**: a `multi Employee` whose + `Employee` derives `Csv` can be serialized whole, which is precisely the + `toCSV(from e in Employee where … select e)` case that has no expression + today (a generic serializer can neither be user-written under principle 13 + nor attached as a method to a native `multi`). + +## Acceptance Criteria + +- What to achieve? + - **Given** a class annotated to derive `Csv` (surface per the spec), + - **when** the program is compiled and a value (or a `multi` of values) is + encoded, + - **then** the output is the expected CSV, the encoder is generated from + the class table, and **disassembly shows ordinary bytecode with no + reflection and no dynamic dispatch** — principle 13 provable, not + asserted. +- What to achieve? + - **Given** `json.encode` reimplemented as the `Json` derive, + - **when** the existing db, log-watcher, and json corpus run, + - **then** every output is byte-identical to today — the framework + generalizes the mechanism without changing its result. +- What to achieve? + - **Given** a derive requested on a class one of whose fields the derive + cannot handle (a `@gc` field for a value derive, a kind with no CSV + rendering), + - **when** it is compiled, + - **then** it is a **compile error naming the field and the reason** — no + silent partial output, no runtime failure. A derive's applicability is + decided entirely at compile time. +- What to achieve? + - **Given** two classes deriving `Eq` where one embeds the other, + - **when** structural equality is generated, + - **then** it recurses through the embedded type's own derived `Eq` — the + framework composes across types the way the field kinds nest. + +## Out Of Scope + +- **A full trait / typeclass system** — bounds like `fn f(x: T)`, + generic functions, and the inference they need. That is a large, separate + language iteration; this one ships a **closed, compiler-known derivable + set**, not open generics. The derive facility is the pragmatic 80% without + the type-system weight. +- **User-defined / procedural macros.** Rust lets users write `proc_macro` + derives; writeonce does not, and this iteration keeps it that way — only the + compiler-builtin derive set. A user-macro system is a much larger surface + and likely never wanted (KISS). +- **Monomorphized generics as a general feature.** Per-type generation here is + specific to the derive set, not a general generics engine. +- **Deriving across the attach channel** — a client generating an encoder over + the owner's types (iterations 9c/9d). Composes later; the class-table + metadata already crosses the channel's schema handshake, so the pieces are + in place, but it is not this iteration's problem. +- **Reopening principle 13 in any form.** If a derive appears to need runtime + reflection, the derive is wrong, not the principle — that is a defect report + against this iteration. + +## Info + +Prior art in the tree: + +- **`json.encode`/`decode` (`runtime/src/json.c`) is already this mechanism**, + built once by hand: metadata-driven, compiler-lowered with the class id, + no reflection. This iteration lifts its shape into a reusable framework. +- **The class-table metadata** (`.wob` v2's `field_names`/`field_class`/ + `field_elem`, v3's index metadata) is the substrate every derive reads. It + already exists and is already what `json.encode`'s lowering walks. +- **Principle 13 is both the constraint and the enabler**: because every + type is known at compile time, per-type generation needs no runtime + dispatch, so the generated code is as fast and as untagged as hand-written. + +Forks the spec must settle: + +**1. The request surface.** Options: an annotation in the existing ORM style +(`@derive(Csv, Json, Eq)` on the class, matching `@table`/`@unique`); a +`derive` keyword; or trait-style `impl`-blocks. Leaning: the `@derive(...)` +annotation — smallest surface, consistent with the annotation-driven design +the language already has, and it keeps derives a closed compiler-known set +rather than implying an open trait system. + +**2. How a derived capability is invoked.** With no UFCS and no methods on +native containers, `value.to_csv()` cannot be a method on a `multi`. Options: +a compiler-recognized builtin per capability (`csv.encode(x)`, exactly like +`json.encode(x)` is lowered today), or generated free functions named by +convention (`Employee_to_csv`). Leaning: compiler-recognized builtins +(`json.encode`/`csv.encode`/…), so the invocation is uniform and the +collection case (`csv.encode(a_multi)`) is handled by the same lowering that +already special-cases a value's static kind. + +**3. Whether `Eq`/`Hash` change what the VM already does.** Structural +equality and hashing over stored/embedded types touch the same metadata the +engine's indexes use — the spec should decide whether derived `Eq`/`Hash` +share code with the engine's key comparison (`database/src/table.c`'s +`idx_cols_equal`/`idx_hash`) or generate independent routines. Leaning: share +where the shapes match (one definition of "these two values are equal"), so a +derived `Eq` and an index's uniqueness check can never disagree. + +**4. Applicability checking.** A derive must reject at compile time any field +it cannot handle (a `@gc` field in a by-value derive, a kind with no rendering +for the target format). The spec pins the rule per capability — and this is +the mechanism by which the facility stays inside principle 13: applicability +is a static question with a static answer, never a runtime probe. + +## Proposed Solution + +- **Brainstorm the spec** settling the four forks, then a plan whose first + task is retrofitting `json.encode` onto the framework — the proof that the + generalization changes nothing observable — before adding `Csv`/`Eq`/`Hash`/ + `Show`. +- Expected shape: a `@derive(...)` annotation parsed like `@table`; a + compiler pass that, per derived capability per type, generates a routine + from the class-table metadata (the same metadata `json.encode` walks); + compiler-recognized encode builtins that lower to those routines; and + applicability diagnostics in a new WO-E range. The runtime gains no new + reflective machinery — only, at most, small shared helpers the generated + code calls.