Commit graph

16 commits

Author SHA1 Message Date
edd6091579 Merge branch 'query-surface' into database-engine 2026-08-16 19:01:16 +02:00
2677c1457e feat: FK restrict on delete + employee sample runs; group-by parked (9b)
- FK restrict: deleting a row a non-nullable `ref` still points at traps
  WO_T_FK (11), catchable. The compiler now records a `ref` field's
  target class in the class-table field_class metadata; the engine
  (wo_row_has_referrers) scans referencing scalar columns before a
  delete. Correctness-first full scan; the backlink-index optimization
  is recorded for later
- docs/examples/employee now COMPILES AND RUNS all six modes against a
  WAL-durable database: seed (+@unique trap across restart), report
  (per-dept aggregates + payroll), staff (unique probe + backlink +
  ref nav), raise (update-through-row), drop (FK restrict), and
  persistence via replay
- group-by SYNTAX parked to a future iteration (user decision): the
  report mode is hand-rolled from the shipped primitives meanwhile
  (same numbers). "table relations and FK" is complete
- scripts/employee-accept.sh (8 checks) + a `just employee` module;
  manifest parser tolerates iteration 9c's [share]/[[share.clients]]
  sections so `woc .` builds the sample on this branch
- fixtures trap/db-fk-restrict (code 11) + run/db-fk-restrict-catch;
  oop-e2e 79/0, woc-test 566/0, 15 runtime suites, log-watcher 7/0,
  employee-accept 8/0
- 9b story + status board updated

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 18:37:23 +02:00
ad8cfb456e docs: iteration 13 story -- compile-time metaprogramming (derive)
- captures the toCSV/reflection thread: principle 13 forbids runtime
  reflection, so a generic serializer can't be a user-written function;
  Rust answers with derive macros (compile-time codegen), and
  json.encode is already a single hand-built instance of exactly that
- iteration generalizes json.encode's mechanism into a reusable derive
  facility: @derive(Json/Csv/Eq/Hash/Show) -> the compiler generates
  per-type routines from the class-table metadata it already emits,
  monomorphic, no runtime type tag, no dynamic dispatch
- closes the query-result-serialization gap
  (csv.encode(from e in Employee ... select e)) that has no expression
  today; acceptance requires json.encode retrofitted onto the framework
  with byte-identical output, and disassembly proving no reflection
- four forks: request surface (lean @derive annotation), invocation
  (lean compiler-recognized encode builtins, no UFCS/methods), Eq/Hash
  sharing the engine's key comparison, static applicability checking
- out of scope: full trait/typeclass system, user proc-macros, general
  generics, cross-channel derive -- a CLOSED compiler-known derivable
  set, the pragmatic 80% without the type-system weight
- numbered 13 to echo the principle it lives inside; roadmap + board
  rows added

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 04:57:00 +02:00
71d3985e81 docs: performance arc as story iterations (9e measure, 9f io_uring)
- 9e durability/throughput/scale: the measurement backbone -- run the
  employee program, restart to prove persistence, benchmark read/write
  through compiled .wo, ~1M-row mixed load with throughput floor + p99
  ceiling + flat RSS; the gate every optimization signs (before/after
  delta required, no measured delta = not accepted)
- 9f io_uring group-commit: replace fsync-per-commit with batched
  io_uring durability overlapped on shard threads; same ack-after-
  durable contract, crash battery unchanged, automatic fsync fallback
  on kernels without it; deliberately LAST (needs 8's threads to
  overlap and 9e's baseline to beat)
- wired the existing levers into the arc: iteration 8 (thread-per-core)
  = "optimize multithreading", 7b (mark-sweep) = "implement GC" --
  each now gated by re-running 9e and recording the delta
- explicit sequence recorded in 9e: 9b lands -> 9e baseline -> 7b
  re-bench -> 8 re-bench -> 9f re-bench
- roadmap + board rows for 9e/9f; four forks each for the specs
  (load generator, absolute vs relative budgets, what "1M" means,
  durable vs RAM headline; ring model, liburing vs raw, batch
  boundary, fallback testing)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-15 22:12:48 +02:00
2e1041daed docs: employee-list sample -- program B's manifest pair (9c/9d target)
- docs/examples/employee-list: attaches to the running employee
  program; modes list / report (byte-identical to A's own) /
  staff <dept> / probe-write (registered read-only, insert must trap
  access-denied, exit 4, A unchanged)
- the manifests ARE the design, written as a pair: A's [share] gains
  listen = unix socket beside WO_DATA plus [[share.clients]] naming
  B's public-key fingerprint with rights = "read"; B's
  [connect.employee] carries A's ipc string, A's PINNED fingerprint,
  and project = ../employee for compile-time shapes -- the connect
  section's name is the code's namespace (employee.Employee)
- fingerprints are PASTE-HERE placeholders by design: keys generate
  into WO_DATA at first boot (9d), tomls carry fingerprints only,
  printed by --identity
- sample-first: compiles after 9/9b/9c/9d; README maps each mode to
  the acceptance line it exists for; 9c/9d stories now name this
  sample as their workload

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-15 12:34:19 +02:00
fe56ba0944 docs: iteration 9d story -- keypair authentication for attach
- promotes 9c's identity fork to its own iteration: program identity
  is a keypair (first-boot generated into WO_DATA, 0600, printable
  fingerprint); A's [share] grants name PUBLIC KEYS, B pins A's key
  in [connect.a]; mutual challenge-response, fresh nonces, transcript-
  hash signing (protocol tag + fingerprints + nonces + channel)
- acceptance criteria: registered-key attach carries 9c rights
  unchanged; unregistered key refused pre-statement with fingerprint
  logged; same-uid-wrong-key refused (uid SUPERSEDED, not
  supplemented); impostor on A's socket path aborted by B's pinned-key
  check; handshake replay refused; rotation = manifest change
- four forks recorded: crypto provenance (lean: vendored compact
  Ed25519 as the one sanctioned vendored component), keygen home
  (lean: first-boot into WO_DATA), signed-transcript layout, uid
  survival (lean: keys only, peer-cred demoted to log enrichment)
- out of scope: transport encryption, CA machinery, key escrow,
  root-attacker protection
- plan folds into 9c's when specced (neither ships alone); 9c fork 3
  marked superseded-as-end-state; roadmap + board rows added

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-15 12:26:49 +02:00
b92357b6b9 docs: iteration 9c story -- cross-program tables (attach over IPC)
- program B attaches to running program A's persistent database via an
  IPC string in B's wo.toml [connect.<name>]; A registers clients by
  name with read / read+write rights in its [share] manifest section;
  unregistered = refused at connect, under-privileged = catchable trap
- doctrine preserved: A stays the single writer -- B's statements
  execute inside A through the same choke-point row API, B never
  touches A's WAL or slabs; typed statements checked by B's compiler
  against A's table shapes, schema handshake at attach
- four forks recorded for the spec: channel carrier (lean: unix
  socket + SO_PEERCRED), how B's compiler learns A's shapes (lean:
  project reference + live handshake), grant granularity (lean:
  whole-db rights, name+uid identity), blocking semantics (lean:
  blocking round-trip, stop-flag rule applies)
- acceptance sketch: employee sample as A, a thin employee-report
  client as B (read-only GroupBy over the wire) + audit-log writer
  exercising the rights matrix
- slots after 9b (shares its typed surface), before 10 (HTTP is the
  external face; this is the writeonce-native one); prior art:
  04-client-api.md wire protocol + the WAL's value encoding
- roadmap + status board rows added

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-15 12:13:38 +02:00
1749440351 docs: borrow/GC analysis for the database engine, into the iterations
- 9b spec gains section 6 "Ownership, borrows, and GC across the
  engine boundary": two one-way copy gates (no VM pointer enters a
  row, everything a select returns is copied out), so the collector
  never traces engine memory and the engine never touches refcounts
- row views are borrows WITHOUT a runtime net: rows share the VM's
  field encoding but not its header, so no borrow word backs them --
  the compile-time escape rule is load-bearing alone
- cursor stability settled: scans materialize their id list before
  the body, row updates through the view stay legal (raise mode
  updates an indexed column mid-scan and is the proving fixture),
  insert/delete on a table with an open cursor is a new WO-E5xx
- GC-pause interaction recorded: collector runs between statements,
  a long scan delays slices -- accepted, documented
- iteration-7b ordering constraint: GC inference must classify before
  table-field validation, diagnostic names the inference reason --
  noted in 7b story, iteration-9 plan constraints, 9b plan tasks
- stories 09/09b Info sections point at the analysis; 9b plan Tasks
  3/5 carry the enforceable checkboxes (ASan boundary assertion)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-15 09:57:53 +02:00
5943bf6890 docs: iteration 9b spec + plan — @table, relations, query (employee)
- spec settles 9b's three forks: SQL/Cypher layer superseded as the
  program surface (design history + wo-db engine-semantics reference);
  comprehension syntax desugared at compile time (no function values);
  System.Linq = operator vocabulary + edge cases, PostgreSQL = execution
  + integrity vocabulary (both references surveyed 2026-08-15)
- aggregate semantics normative: count/sum total 0 on empty, avg/min/max
  are ?T with nil (empty is data, not a fault); nil skipped; sum wraps
  like language arithmetic; GroupBy lowers as group-and-reduce
  (AggregateBy shape), transition/finalize ABI from nodeAgg
- relations: ref = FK with direct-index-probe check (nil passes,
  unchanged-key skips), backlink = secondary-index scan, delete is
  restrict-only; nil never joins, nil is a legal group key
- lowering: the compiler is the planner — queries become bytecode loops
  over cursor/group builtins, longest-prefix index selection, no plan
  tree, no SQL text in the image (disassembly-provable)
- plan: 6 tasks gated by a new docs/examples/employee sample
  (Department/Employee, @unique, composite index, ref/backlink,
  GroupBy report mode) with its own acceptance script + crash step;
  blocked on iteration 9's engine plan
- story 09b + status board updated; 02-wo-language.md carries the
  supersession note

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-15 09:10:08 +02:00
7c10df67a3 docs: reprioritise to log-watcher-executable only; update stories and plans
Every remaining item is now traced to a measurement on the sample; anything the
sample does not exercise is deferred by name with the measurement that says so.

- new plan docs/plan/compiler/2026-08-14-logwatcher-executable.md — six tasks
  between "it runs" and "you can leave it running": the ownership pass learning
  stdlib return types (>1 MB leaked in 8s of `run` mode, one fs.read_all
  result), dropping a projected temporary (`for e in parse_dir(d).entries`
  leaks the shell per rescan), the runtime's own argv container (128 B every
  run), honouring the stop signal in blocking calls (a server in accept ignores
  SIGTERM), closing accepted connections (net.close exists, unused), and a soak
  that would have caught all of it. Opens with the measured starting point and
  closes with an explicit out-of-scope list
- story 7 (log-watcher proof): status banner separating the met compile-and-run
  half from the executable half, plus a new Given/When/Then — clean SIGTERM
  exit, zero leaks, flat RSS and descriptors across a soak
- story 5: grammar half landed, strictness half deliberately deferred
- story 6: landed for the surface the workload uses, with the two lifetime
  defects it exposed pointed at the new plan
- story 7b: recorded as off this workload's path, measured — the sample has no
  @gc class, 0 RC_INC/RC_DEC against 78 DROPs
- 00-status.md: NEXT PLAN is the executable list; story table and in-progress
  row point at the new plan; deferrals carry their evidence
- plan 8 (haxe-parity): banner now says on hold behind the executable plan, and
  its task states are corrected — Task 5 shipped, Tasks 6 and 7 are half done

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-14 22:16:14 +02:00
79605cbb4f feat: milestone 1 complete — .wob emitter, conformance corpus, single binary; GC redesign specced
- `woc` now emits `.wob` that `wovm` runs: emit.ml lowers the typed,
  owner-annotated AST (scope-stack registers with a >64 WO-E401 diagnostic,
  Lua-style call windows, ICALL by slot, dedup const pool, drop maps, line
  tables, implicit terminators); disasm.ml backs `--dump-bc` goldens.
- Ownership lowering consumes the four owner tables verbatim; RESIDUAL is the
  only source of borrow ops, coalesced per operand. Review caught the emitter
  consuming only 2 of owner.ml's 4 residual producers — an assignment-anchored
  aliasing violation ran to exit 0 instead of trapping; fixed, plus a backstop
  raising WO-E404 for any residual region left unconsumed.
- Conformance harness `scripts/oop-e2e.sh` (`just oop-e2e`): four fixture
  kinds with exact outcomes — byte-exact stdout, one WO-E### anchored on
  `error CODE:`, numeric trap code, gc trace. 25 fixtures incl. pricing-demo
  logic, the ownership suite, and DB_STUB's parse-but-trap. `tests/` un-ignored
  so the corpus is actually tracked.
- `woc build` produces a self-contained binary: wovm copy + appended image +
  20-byte trailer, self-exec via /proc/self/exe. Verified relocated outside
  the repo, argless, and against adversarial trailer corruption.
- Milestone 1's five spec criteria all MET (`just oop-accept`). Criterion 3
  closed by WO-E405 — the entry must return `Int`, since program mode already
  says its return value is the exit code — which deletes the leak class
  without adding return-type metadata to the format. `gc/held-cycle` retired:
  an externally-held cycle is not expressible in a post-exit pump.
- New spec: inferred GC + incremental per-shard tri-color mark-sweep, retiring
  `@gc` and reference counting. Story gains iterations 7b (that work) and 9b
  (`@table`, relations, compiler-checked query); `.dev/reference` gains a
  sparse System.Linq checkout. Priority: 5→6→7 (log-watcher) then 7b, 8, 9, 9b.
2026-08-11 19:31:26 +02:00
49872a4b11 docs: six-bucket status board; recover lost vision doc; stop ignoring docs/
- 00-kanban.md rebuilt: ▶ NEXT PLAN pointer (iteration 4 — emitter, corpus,
  `woc build`) then six buckets — stories, in progress, done, pending,
  discarded, learnings. It tracked only the Rust runtime before, so the whole
  OOP track (wovm shipped, woc Tasks 1-8 shipped) was invisible.
- Board now records iteration 3's known gaps instead of silently owing them:
  `?T` plumbed but unenforced; E205/E201/E203/E204 dead, so structural
  interface satisfaction is unchecked.
- New discarded.md — settled rejections with reasons so they are not
  re-proposed: inheritance, `abstract` newtypes, Money/SKU/Float, Dynamic/cast/
  macro/extern, AOT-to-C, Menhir, shared engine state, external deployer.
- RECOVERED docs/plan/exploration/blue-green-vm/00-vision.md — gone from disk,
  never committed (gitignored path), cited by five docs incl. principle 12.
- Root cause was broader: all seven forward-roadmap plans in
  docs/superpowers/plans/ were untracked and ignored, on one disk only. Rules
  were half-fiction — 33 of 34 exploration files were already tracked, so they
  swallowed only *new* files.
- Dropped the docs ignore rules (exploration, oop-vm, superpowers/plans,
  examples/agent-loop, examples/mcp-think) with a do-not-re-add comment; added
  __pycache__/*.pyc. `tests/` stays ignored but warns that the next plan lands
  the corpus there.
2026-08-10 21:52:59 +02:00
2de724df11 feat(compiler): MVS ownership pass + woc driver; drop Money/SKU/Float, reject abstract
Completes plan 2 Tasks 7-8. owner.ml: mutable-value-semantics flow analysis
producing the four plan-3 emitter tables (moves, drops incl. LIVE-MASK for trap
unwinding, rc with elision, residual borrow sites) plus WO-E301-304 two-site
diagnostics. Alias questions run over canonicalized places, so a double-mut
reached through let-bound aliases lands in the residual table like the direct
form; dump.ml's contract notes the emitter must coalesce guards per operand.
main.ml: directory discovery, cross-file programs (symbols merge before bodies
check), diagnostics ordered by (file,line,col), new WO-E214 for a name declared
in two files. New docs/plan/oop-vm/01-error-catalog.md (14 emitted + 10 reserved
codes), un-ignored so both plan tracks can cite it; justfile regains woc-*.

builtin_scalars is now the five that work: Int, Bool, Text, Timestamp, Id.
Money/SKU/Float and the abstract_types allowlist are gone — `abstract` never
lexed, and Float had no literal syntax and no wob kind, so no value could exist.
Fixtures and samples retype Money->Int, SKU->Text. The abstract newtype feature
is rejected outright (verdict row adopt->reject); haxe-parity Task 7 keeps `is`.

nullable-types-implementation.md corrected: ?T is plumbed but UNENFORCED
(E211-213 declared, never emitted; probe exits 0), handed to haxe-parity Task 6
as next work item. Records all 10 dead codes incl. E205 — interface satisfaction
is unchecked. crates/rt keeps its Money/SKU fixtures (opaque strings, Stage 2).

Gate: build warning-clean, 14 + 264 checks 0 failures, pricing golden exit 0,
docs/examples histograms unchanged (13/70, zero WO-E225).
2026-08-10 21:24:50 +02:00
8fb10530ce docs: story iterations 11-12 (fibers, blue-green deploy) 2026-08-10 09:58:52 +02:00
5a43210448 docs: compiler front-end specifications and plans (Tasks 2-6)
- Plan docs: architecture, woc front (Tasks 2-7), emit+e2e (Plan 3), Haxe parity (Plan 8)
- nullable-types implementation plan
- Iteration 3: compiler front story
- Specs: systems track, blue-green VM, log-watcher sample, OOP compiler/VM
- Principles + project structure
- Kanban updated with compiler front-end progress
2026-08-10 09:11:04 +02:00
2986c3c906 language-runtime-database iterations 2026-08-07 19:36:49 +02:00