writeonce/docs/plan/compiler/2026-08-15-employee-relations-query.md

@table, Relations, Query — Implementation Plan (employee sample)

Status: ⬜ pending (story iteration 9b) — blocked on iteration 9's engine plan (2026-08-01-db-engine-binding.md): Tasks 3–6 below consume its row storage, WAL, indexes and select subset. Board: 00-status.md

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Style rule (user convention): concept, reason, and required behavior in words only; the executor writes the code.

Spec: docs/superpowers/specs/2026-08-15-table-relations-query-design.md (normative: the settled forks, the clause grammar, the aggregate semantics table, the lowering model), amended by the systems-track spec's program-mode contract for the sample's CLI.

Goal: @table classes queried in the language: comprehension queries with where/join/group/order/take/select, typed ref/backlink navigation with restrict integrity, and the LINQ aggregate vocabulary (count/sum/avg/min/max, GroupBy as group-and-reduce) — all lowered to bytecode loops over engine cursor builtins, proven by a new docs/examples/employee sample whose acceptance script is the gate.

Architecture: compiler front (compiler/src/{lexer,parser,ast,types,owner,emit}.ml) for the query surface; the database engine (database/src/ — its own top-level directory per the 2026-08-15 decision, statically linked into wovm; table.c/db.c from iteration 9, plus a new query.c for cursors and group hashing). The compiler is the planner: index selection happens at lowering, the VM never sees a plan tree. Reference semantics: System.Linq for operator meaning, PostgreSQL's nodeAgg/ri_triggers for execution and integrity vocabulary (both surveyed in the spec, fork 3).

Tech Stack: OCaml stdlib (compiler), C11 libc (runtime). No new opcodes; new builtin ids appended to the format doc.

Global Constraints

• The sample is the test. docs/examples/employee plus its acceptance script is 9b's gate; the only corpus additions are the db-corpus fixtures iteration 9 already plans. just oop-e2e, just woc-test, just wovm-test stay green as regression after every task.
• Index doctrine (iteration 9, verbatim): secondary indexes are maintained only through the engine's row choke points; FK checks and backlink reads are index probes, never storage walks.
• No SQL text anywhere — the 9b acceptance's disassembly criterion.
• No function values. Every predicate/projection is expression syntax; a query value is never deferred or passed around.
• Plans/specs in words; commits local only, never push; docs under docs/; CODE-LOGIC.md updated beside changed code.

File Structure

compiler/src/lexer.ml parser.ml ast.ml   query-expression grammar, clause AST (Task 1)
compiler/src/types.ml                    range/group scopes, navigation, projection synthesis (Task 2)
compiler/src/owner.ml emit.ml            query ownership + lowering to cursor builtins (Task 5)
database/src/query.c query.h             cursors, group hash, transition/finalize aggregates (Tasks 3, 4)
database/src/db.c table.c                FK restrict checks at the row choke points (Task 3)
docs/examples/employee/                  the acceptance workload (Task 6)
scripts/employee-accept.sh               the gate (Task 6)
docs/plan/oop-vm/00-wob-format.md        appended builtin ids (Tasks 3-5)

---

Task 1: Query expressions parse

Concept & reason: the comprehension grammar from spec section 3 — from/where/join/group…into/order by/take/skip/select, clause order fixed, aggregates as clause functions — becomes lexer keywords (contextual, so from/group/order stay legal identifiers outside a query), a clause AST, and parse diagnostics in a new WO-E5xx range (clause out of order, missing select, aggregate named outside a query). Fixed clause order is a parse rule, not a type rule, so the error lands on the exact token.

• Grammar and AST for every clause; contextual keywords verified against the existing samples (no .wo file in the tree breaks).
• WO-E5xx diagnostics with golden coverage in the existing woc-test suite (dump-ast goldens for well-formed queries, diagnostic goldens for each malformed shape).
• Gates green; commit locally.

Task 2: Queries typecheck

Concept & reason: the surface's whole promise is compile-time checking. from e in Employee opens a scope where e's fields are the class's; ref navigation substitutes the target class's field set (e.dept.name); backlink reads type as multi of the source class; group … into g closes the range scope and opens the group scope, where g.key has the key's type and g.f is legal only inside an aggregate call; select { … } synthesizes an anonymous record type so later use of a dropped column is a compile error. Aggregate result types follow the spec's table exactly — count is Int, sum Int, avg/min/max are ?T because an empty group is data. Unknown table, unknown column (naming the class), navigation through a non-ref, bare group-member column, join sides not separable: each is its own WO-E5xx with a golden.

• Scope machinery for range/group variables; navigation typing both directions; projection record synthesis interned like other class shapes.
• Aggregate typing per the spec table; ?T results force the existing nil-handling style at use sites.
• Diagnostic goldens for every error class named above; gates green; commit locally.

Task 3: Cursors and integrity in the engine

Concept & reason: the runtime side queries need, built on iteration 9's storage: a table-scan cursor (open by class id, advance, borrowed row view), a primary-index point read (ref navigation), a secondary-index cursor with longest-prefix probe (backlink reads, indexed where), all as builtins appended to the format doc. Integrity lands at the row choke points where the indexes already live: inserting/updating a non-nil ref probes the referenced primary index and traps on a miss (foreign-key violation, sibling of the unique trap); deleting a row still referenced traps (restrict) via the same secondary index a backlink requires — one probe, no new structure. Nil ref never probes (the MATCH SIMPLE rule); an update that leaves the key unchanged skips the check (both from the PostgreSQL RI survey, mechanism discarded — a check is an index probe, never query text).

• Cursor builtins + borrowed-row-view lifetime rules written into the binding doc (a row view never escapes the loop that opened the cursor — the ownership pass enforces it, mirror of the container-read borrow). Rows have no borrow word (they share the VM's field encoding, not its header), so unlike every VM-heap borrow there is no runtime trap behind this rule — the compile-time check is load-bearing alone, which is why it gets its own diagnostic and goldens rather than riding on E30x.
• Cursor stability per spec section 6: scans materialize their id list before the body runs and point-read per iteration; updates through the row view stay legal (exclusive row borrow, index maintenance at the row API); insert/delete targeting a table with an open cursor is a new WO-E5xx (the ownership pass carries the open-cursor table set through the loop body); read-only nested queries over the same table stay legal. The raise mode — updating an indexed column mid-scan — is the fixture that proves the materialized-id semantics.
• FK trap + restrict trap wired through the row API; a debug-build probe counter exposed for Task 5's index-selection proof.
• The delete statement (point delete of a row value) lands here too: iteration 9's subset is insert/select/update-point, and restrict has nothing to restrict without it — same typed-AST-plus-builtin shape as insert, WAL remove record already specified by iteration 9's Task 2.
• db-corpus fixtures from iteration 9's plan extended with one FK-violation and one restrict fixture (trap-code exact); ASan green; commit locally.

Task 4: Group hash + aggregate execution

Concept & reason: the AggregateBy shape from the spec — group-and-reduce in one pass, no group objects. A group hash table builtin set: create (keyed by the group key's kind, nil legal), upsert-advance (locate-or-create the group, advance each aggregate's transition slot), drain (iterate groups, finalize, hand key + finals to the compiled projection loop). Transition/ finalize is PostgreSQL's split: avg carries sum+count in its transition state and divides at finalize; the "no row seen yet" state is distinct from "transition value is nil" so nil-skipping aggregates need no first-row special case. Aggregate semantics are the spec's table: empty ⇒ nil (or 0 for count/sum), nil elements skipped, sum wraps, avg truncates.

• Builtins implemented; the input projected to only the columns the aggregates read before hashing (the nodeAgg memory lesson).
• Fixture-level verification through iteration 9's db corpus (one grouped query, exact output) plus the sample in Task 6; ASan green; commit locally.

Task 5: Lowering — the compiler is the planner

Concept & reason: a query desugars to the bytecode loops the language already has. Scan or probe chosen at compile time: leading where equality conjuncts matched against declared indexes longest-prefix-first, probe emitted on a hit, scan otherwise — and the choice is demonstrated via Task 3's probe counter in the acceptance, not assumed. join builds a transient hash on the inner side and probes with the outer (nil never inserted, never probes). group emits the two-phase hash-aggregation loop from Task 4. order by materializes and stable-sorts (original-index tiebreak — the LINQ stability guarantee); take/skip slice the result. Ownership: row views stay loop-bound borrows; everything select emits is copied/built at the boundary under the established Text-copy and fresh-value rules — queries add no new ownership classes, and the ownership pass's existing drop machinery covers the query's temporaries because the lowering IS ordinary loops.

• Desugar + lowering for every clause; disassembly of the sample's report mode shows loops and builtins, no plan tree, no text. The select boundary is the ownership bulkhead (spec section 6): everything a query returns is copied or freshly built, so no value anywhere points into a row slab after the query ends — asserted under ASan by mutating rows after a query and re-reading the query's results.
• Index selection proven: the acceptance asserts probe-counter deltas for the indexed staff <department> path versus a full-scan query.
• oop-e2e, woc-test, ASan-corpus gates green; commit locally.

Task 6: The employee sample is the acceptance

Concept & reason: spec section 7 verbatim — docs/examples/employee with Department/Employee (@table, @unique name, composite [dept, salary] index, ref/backlink pair), wo.toml manifest so woc . builds it, a just module beside it (the log-watcher convention), and scripts/employee-accept.sh as the gate: seed (insert + WAL + duplicate-department trap on the second run), report (headcount/avg/min/max by department, total payroll, ordered by average salary — byte-exact lines), staff (both navigation directions + index-probe proof), raise (update through a query; missing department ⇒ the empty-query nil path), the restrict-trap demo, and a kill -9 between seed and report proving replay on this workload.

• Sample source pre-authored 2026-08-15 (docs/examples/employee/ — the target workload, sample-first like log-watcher was); this task makes woc docs/examples/employee compile it with zero diagnostics and the binary's modes run. The sample is authoritative: divergence between it and the spec is resolved in the spec's favor and committed.
• scripts/employee-accept.sh + just employee module: every mode checked with exact expectations, trap codes asserted, crash step included; soak-style RSS/fd sampling reused from the log-watcher script's pattern for the report loop.
• ASan run of the full acceptance: zero leaks (the log-watcher bar).
• Docs: README beside the sample, CODE-LOGIC.md updates beside changed compiler/runtime code, format-doc builtin table final, board + story rows updated with measured numbers; commit locally.

Out of scope — deferred by name

• Everything spec section 8 lists: set operators, outer joins, subqueries, composite group keys, groups as values, FK cascade/set-nil, deferred checks, SQL text in any role, cross-shard queries, LIVE, migrations, cost-based planning.
• Sorted-grouping and partial-sort optimizations (recorded LINQ/postgres precedents; hash + full stable sort are this plan's only strategies).
• The ecommerce sample's query rewrite (9b's fifth acceptance criterion) — it lands as its own follow-up once the employee gate is green, so this plan's blast radius stays one new sample.