# `@table`, Relations, and Language-Integrated Query — Design > **Status: proposed** (story iteration 9b). Settles the three forks recorded in > [`09b-table-relations-query.md`](../../stories/language-runtime-database/09b-table-relations-query.md). > Plan: [`docs/plan/compiler/2026-08-15-employee-relations-query.md`](../../plan/compiler/2026-08-15-employee-relations-query.md). > Depends on iteration 9's engine plan > ([`2026-08-01-db-engine-binding.md`](../plans/2026-08-01-db-engine-binding.md)) > for storage, WAL, insert and the select subset. **One sentence:** queries are written in the language, checked by the compiler, and lowered to bytecode loops over a handful of engine cursor builtins — no SQL text exists anywhere in a compiled program — and the acceptance workload is a new `docs/examples/employee` sample whose report mode is one `GROUP BY` after another. --- ## 1. The three forks, settled ### Fork 1 — the SQL/Cypher layer is superseded as the program surface `docs/runtime/database/02-wo-language.md` specifies a query layer of literal SQL and Cypher with five fixed-glue rules. That layer is **not built on the C stack**. The language-integrated surface below is the only way a `.wo` program queries its tables. Reasons, in order of weight: - The iteration's own acceptance forbids the alternative: a query must lower to engine operations "not a string handed to a parser at runtime — provable by disassembly." A resident SQL parser and a compile-checked surface would be two grammars for one meaning, and the second grammar would re-introduce at runtime every error class the first one eliminated at compile time. - The language's identity is compile-time checking (the WO-E diagnostic catalogue). "A typo in a field name is a compile error" cannot be delivered by a text layer. - A binary that ships a SQL parser it uses only for its own programs pays image size and attack surface for nothing. What survives: `02-wo-language.md` stays as design history and as the specification the `prototypes/wo-db` C++ prototype implements; that prototype remains the **engine-semantics** reference (what an index probe returns, what unique violation means), not a syntax reference. SQL text remains a candidate *export/interop* format for much later (external tools speaking to a writeonce service), explicitly not part of this iteration. ### Fork 2 — comprehension syntax, desugared at compile time writeonce has no function values, so LINQ's method-chains-taking-lambdas are unwritable. The surface is a **query expression** — a comprehension the compiler desugars — where every predicate and projection is ordinary expression syntax with the range variable in scope: ```wo let seniors = from e in Employee where e.salary > 90000 order by e.salary desc select e; let by_dept = from e in Employee group e by e.dept into g select { dept: g.key.name, headcount: count(g), avg_salary: avg(g.salary) }; ``` (Illustrative; the grammar is normative in prose, section 3.) Why comprehension and not chaining: a chained `.where(e.salary > 100)` has no binding site for `e` — the comprehension's `from e in` clause is what introduces the variable, which is exactly the property that makes every later clause an ordinary typed expression the existing typechecker can check. C# had to add query expressions *on top of* lambdas for the same readability reason; we get to skip the lambda layer entirely. The predicate is compile-time syntax, never a runtime closure — which is also what lets the whole query lower to plain bytecode. ### Fork 3 — what each reference contributes **System.Linq** (surveyed 2026-08-15) contributes the operator vocabulary and its semantic edge cases, not machinery: - The lowering target for `group … select aggregate` is the shape of `AggregateBy`/`CountBy` and the `GroupBy(key, resultSelector)` overload: **group-and-reduce as one node, no intermediate group object materialized** (`Grouping.cs:63`, `AggregateBy.cs`). The `IGrouping`-returning overloads exist to hand groups around as values; we have no delegates to hand them to, so groups surface only as the `into g` binding inside the query itself. - Empty-source rules per aggregate (section 4's table) adapt LINQ's — where C# throws `InvalidOperationException`, writeonce answers `nil` through a `?T` result, because an empty group is data, not a fault. - Ordering is **stable**, guaranteed — LINQ enforces it with an original-index tiebreak (`OrderedEnumerable.cs:432`); we adopt the same guarantee and the same escape hatch (an unstable sort is legal when the key is a scalar and rows are not identity-bearing, `OrderBy.cs:144-163` precedent). - Join semantics: build a hash on the inner side, probe with the outer, and **nil never joins** — LINQ achieves SQL's NULL-never-matches by refusing to insert null keys into the build side (`Lookup.cs:119-122`); we do the same. In grouping, by contrast, nil **is** a legitimate key (`Lookup.cs:207`). **PostgreSQL** (surveyed 2026-08-15) contributes execution vocabulary for the engine side: - Aggregate execution is the transition/finalize split from `nodeAgg.c`: a per-group transition value advanced once per row, a finalize step converting state to result (`avg` carries sum+count without the executor knowing). The `noTransValue` vs `transValueIsNull` distinction — "no row seen yet" is not "the value is nil" — is adopted verbatim; it is what makes nil-skipping aggregates correct without special-casing the first row. - Grouping strategy: hash aggregation (`AGG_HASHED`) is the only strategy this iteration builds; sorted grouping is an optimization for later. Project the input to only the columns the aggregate needs before hashing (`find_hash_columns` precedent). - Referential integrity semantics from `ri_triggers.c`, mechanism discarded: an FK check is a **direct probe of the referenced table's primary index** (never query text); a nil `ref` passes the check (MATCH SIMPLE rule); an update that does not change the key skips the check (`RI_FKey_fk_upd_check_required` precedent). Enforcement action is **restrict only** — deleting a Department that still has Employees traps; cascade/set-nil are out of scope. - Everything MVCC, buffer-manager, lock-manager and cost-planner shaped is explicitly non-transferable: single-writer-per-shard RAM-authoritative storage designed those problems away (iteration 9's doctrine). --- ## 2. Relations become typed navigation The declaration vocabulary already exists in the samples and partially in the compiler; this iteration makes it mean something: - `dept: ref Department` — a foreign key. Stored as the target's row id (a scalar column, already the compiler's classification). **Navigating** it in an expression (`e.dept.name`) typechecks with `Department`'s field set in scope and lowers to a primary-index point read. - `staff: backlink Employee.dept` — the declared inverse. Not a stored column; reading it (`d.staff`) is a secondary-index scan of `Employee`'s `dept` column and yields `multi Employee`. Declaring a `backlink` whose target field is not a `ref` to this class is a compile error. - `?ref Department` — an optional relation; nil stores as id 0, never probes, never joins. Integrity, enforced at the engine's row choke points (iteration 9's index doctrine — nothing touches storage except the row API): - Insert/update of a non-nil `ref`: probe the referenced primary index; a miss traps with a foreign-key violation code (sibling of iteration 9's unique-violation trap). - Delete of a row that a non-nil `ref` still points at: trap (restrict). The check is a probe of the secondary index that the `backlink` already requires, so restrict costs one lookup and no new structure. --- ## 3. The query surface (normative, in prose) A **query expression** is an expression. Its clauses, in the only order they may appear: 1. `from in ` — required, first. Source is a table (class name), a `backlink` navigation, or a `multi` value. Introduces the range variable. 2. `where ` — optional, repeatable. Ordinary boolean expression over range variables; each `where` is a filter. 3. `join in on == ` — optional. Equi-join only; one side references only the outer variable, the other only the joined one. Most joins in practice are spelled as `ref` navigation instead; explicit `join` exists for joining on non-relation columns. 4. `group by into ` — optional. Ends the scope of the range variables and opens the scope of `g`. `g.key` is the key's value. For any field `f` of the grouped element, `g.f` names the **column of members** — legal only inside an aggregate call. Nil is a legitimate key. 5. `order by [desc]` — optional, comma-repeatable keys. Stable. 6. `take ` / `skip ` — optional. 7. `select ` — required, last. The result element: a whole row, a single field, or a projection literal `{ name: expr, … }` whose type the compiler synthesizes as an anonymous record. Using a column the projection dropped is a compile error thereafter (the 9b acceptance's projection criterion). A query's value is `multi `; a query wrapped directly in a whole- query aggregate (`count(from …)`) is that aggregate's scalar. Execution is eager at the point the expression is evaluated — no deferred queries, no query values passed around (that would be a function value in disguise). Aggregates are compiler-recognized **clause functions**, legal over a group binding or a whole query: `count(g)`, `count(from …)`, `sum(g.f)`, `avg(g.f)`, `min(g.f)`, `max(g.f)`. They are not general functions; naming one outside a query is the existing unknown-identifier error. Diagnostics this surface owns (new WO-E5xx range): unknown table, unknown column (naming the class), relation navigated through a non-`ref` field, aggregate outside a query, group member column used bare outside an aggregate, join sides not separable, projection field name collision. --- ## 4. Aggregate semantics (normative table) | Aggregate | Input | Result type | Empty source | Nil elements (`?T` column) | |---|---|---|---|---| | `count(g)` | group/query | `Int` | `0` | counted (row exists) | | `sum(g.f)` | `Int` column | `Int` | `0` | skipped | | `avg(g.f)` | `Int` column | `?Int` | `nil` | skipped; all-nil ⇒ `nil` | | `min(g.f)` / `max(g.f)` | `Int` or `Text` column | `?T` | `nil` | skipped; all-nil ⇒ `nil` | Decisions behind the table: - **Empty is data, not a fault.** LINQ throws on empty `Average`/`Min`/`Max` over non-nullables; writeonce has `?T` and a nil-forcing style already, so the nullable-column LINQ behavior (`null` on empty, skip nils, all-nil ⇒ `null` — `Average.cs:165`, `Min.cs:47-56`) is the behavior for everyone. Traps stay reserved for integrity violations. - **`sum` wraps.** The VM's ADD wraps two's-complement; `sum` is a loop of ADDs and inherits that. Documented, consistent with language arithmetic, and the alternative (checked overflow, LINQ's `Sum.cs:40`) would make `sum` the only trapping arithmetic in the language. - **`avg` truncates** toward zero (integer division), same as `/`. - Execution is the transition/finalize ABI (section 1, fork 3): one opaque transition slot per (group, aggregate), advanced per row, finalized per group when the hash table drains. --- ## 5. Lowering and execution model A query compiles to an **ordinary bytecode loop** — the same opcodes every `for` loop uses — over a small set of new engine cursor builtins. There is no plan-tree interpreter in the VM: the compiler *is* the planner, at the only scale this iteration promises (index selection, no cost model). - **Scan**: table cursor builtins (open by class id, advance, read row into a register as a borrowed row view). `where` clauses are ordinary compiled boolean expressions guarding the loop body — the flattened-expression lesson of PostgreSQL's executor (`execExpr.c`) is in our case simply "the bytecode we already generate." - **Index selection**: if the leading `where` conjuncts equality-match a declared index's prefix (longest-prefix rule, iteration 9's `find_by`), the compiler emits an index-probe cursor instead of a scan, and the acceptance demonstrates the difference (a probe counter the engine exposes in debug builds — "demonstrated, not assumed"). - **`ref` navigation** lowers to a primary-index point-read builtin. **`backlink`** lowers to a secondary-index cursor over the ref column. - **`group … into g … select`** lowers to the two-phase hash-aggregation shape: phase one iterates the source, keying a group hash table and advancing transition slots (builtins: group-table create / upsert-advance); phase two drains the table, runs finalize, evaluates the `select` projection per group. One node, no intermediate group objects — the `AggregateBy` shape. - **`join`** lowers to build-inner/probe-outer against a transient hash keyed on the join column; nil keys are never inserted and never probe. - **`order by`** materializes the result `multi` and stable-sorts it; `take`/ `skip` slice afterwards (a partial-sort fast path is recorded as a later optimization, LINQ `SpeedOpt` precedent). - **Ownership**: rows read from a table are engine-owned; anything a query *returns* is copied out at the select boundary under Task-2's established rule — a Text crossing an ownership boundary is copied, a projection record is freshly built and caller-owned. Queries introduce no new ownership classes. Format consequences: new builtin ids appended to the format doc's table (the cursor/group/probe set), no new opcodes, no version bump beyond iteration 9's. --- ## 6. Acceptance workload — `docs/examples/employee` A new sample, structured like `log-watcher` (wo.toml manifest, program mode, `just` module, acceptance script), small enough to read in one sitting and shaped so every 9b feature is load-bearing: - **Types**: `Department` (`@table(name: "departments", index: [name])`, `name: Text @unique`) and `Employee` (`@table(name: "employees", index: [dept], index: [dept, salary])`, `name: Text`, `salary: Int` cents, `hired: Int` ms, `dept: ref Department`); `Department.staff: backlink Employee.dept`. - **Modes** (CLI, exit codes per the program-mode contract): - `seed` — inserts departments and employees; proves insert + WAL + unique trap (second `seed` run reports the duplicate-department trap caught). - `report` — the GroupBy showcase: headcount by department, average / min / max salary by department, total payroll, departments ordered by average salary — each line's expected output is byte-exact in the acceptance script. - `staff ` — relation navigation both directions: finds the department by unique name (index probe), lists its `staff` backlink ordered by salary; also prints each employee's `e.dept.name` to prove forward navigation. - `raise ` — update through a query result; re-running `report` shows the moved averages; a `raise` for a missing department exercises the empty-query path (`avg` ⇒ nil). - **Integrity demo**: deleting a department with staff traps (restrict); the acceptance asserts the trap code. - **Crash step**: kill -9 between `seed` and `report`, re-run `report` — iteration 9's WAL replay proven on this workload too. The sample is 9b's acceptance the way log-watcher was iterations 1–7's: no new corpus fixtures beyond the db corpus iteration 9 already plans; the sample is the test. ## 7. Out of scope (inherited and new) - Everything 9b's story already excludes: cross-shard queries and distributed joins, `LIVE` subscriptions, migrations, cost-based planning. - The `IGrouping`-as-value surface (groups escaping their query), `Distinct`/ set operators, outer joins (`LeftJoin` family), subqueries in `where`, and `group by` composite keys beyond a single expression — each waits for a workload that demands it. - FK actions other than restrict (cascade, set-nil), deferred constraint checking (the after-trigger queue pattern is recorded for when transactions span statements). - SQL text in any runtime role.