diff --git a/docs/00-status.md b/docs/00-status.md index f4906f9..d5c1ef1 100644 --- a/docs/00-status.md +++ b/docs/00-status.md @@ -117,6 +117,7 @@ that sequences its tasks. Read one, approve, then the next starts. | 8 | [Shard-actor runtime](stories/language-runtime-database/08-shard-actor-runtime.md) | ⬜ | | 9 | [Database engine](stories/language-runtime-database/09-database-engine.md) | ⬜ | | 9b | [`@table`, relations, query](stories/language-runtime-database/09b-table-relations-query.md) | ⬜ spec + plan ready (2026-08-15) | +| 9c | [Cross-program tables](stories/language-runtime-database/09c-cross-program-tables.md) | ⬜ needs a spec first | | 10 | [HTTP service layer](stories/language-runtime-database/10-http-service.md) | ⬜ | Hold | | 11 | [Fibers](stories/language-runtime-database/11-fibers.md) | ⬜ | Hold | | 12 | [Blue-green deploy](stories/language-runtime-database/12-blue-green-deploy.md) | ⬜ | Hold | @@ -323,6 +324,7 @@ Ecommerce sample (verified 2026-06-13): `api.rest` 17/17 expected statuses pass. | 8 | Shard-actor runtime | [plan 4](superpowers/plans/2026-08-01-shard-actor-vm-runtime.md) | | 9 | Database engine binding | [plan 5](superpowers/plans/2026-08-01-db-engine-binding.md) | | 9b | `@table` + relations + language-integrated query — comprehension queries, `ref`/`backlink` navigation, GroupBy aggregates; acceptance: new `docs/examples/employee` sample | [spec](superpowers/specs/2026-08-15-table-relations-query-design.md) · [plan](plan/compiler/2026-08-15-employee-relations-query.md) | +| 9c | Cross-program tables — attach to a running program's database (IPC string in wo.toml, manifest-granted rights, owner stays the single writer) | **no spec yet** — four open forks recorded in the iteration; brainstorm before planning | | 10 | HTTP service layer | [plan 6](superpowers/plans/2026-08-01-http-service-layer.md) | | 11 | Fibers | vision §3, [blue-green exploration](plan/exploration/blue-green-vm/00-vision.md) | | 12 | Blue-green deploy | [spec](superpowers/specs/2026-08-03-blue-green-vm-design.md) — plan authored after iterations 9–10 | diff --git a/docs/stories/language-runtime-database/00-story.md b/docs/stories/language-runtime-database/00-story.md index 9c2de21..8a2b3a2 100644 --- a/docs/stories/language-runtime-database/00-story.md +++ b/docs/stories/language-runtime-database/00-story.md @@ -53,6 +53,7 @@ iterations); no commits by agents — drafts go to `.dev/commit.md`. | 8 | [Shard-actor runtime](08-shard-actor-runtime.md) | thread-per-core shards, per-shard heaps, ownership-move messaging | | 9 | [Database engine](09-database-engine.md) | class-shaped tables, typed WAL + recovery, `insert`/`select` execute | | 9b | [`@table`, relations, query](09b-table-relations-query.md) | `@table` becomes real storage; typed `ref`/`backlink`/`multi` relations; compiler-checked LINQ-shaped queries lowered to engine ops | +| 9c | [Cross-program tables](09c-cross-program-tables.md) | attach to a running program's database over a local IPC channel: manifest-granted read/write rights, typed statements checked against the owner's shapes, owner stays the single writer | | 10 | [HTTP service layer](10-http-service.md) | `service` blocks route to VM methods; REST parity with Stage 2 | | 11 | [Fibers](11-fibers.md) | green threads on the shard scheduler: reduction-budget preemption, park on I/O | | 12 | [Blue-green deploy](12-blue-green-deploy.md) | two VM slots, in-runtime compile, atomic switch, resident rollback | diff --git a/docs/stories/language-runtime-database/09c-cross-program-tables.md b/docs/stories/language-runtime-database/09c-cross-program-tables.md new file mode 100644 index 0000000..4639820 --- /dev/null +++ b/docs/stories/language-runtime-database/09c-cross-program-tables.md @@ -0,0 +1,168 @@ +# Iteration 9c — cross-program tables: attach to a running program's database + +> Format: `product/story-iteration-template`. Part of +> [Story — one language, one runtime, one database, one binary](00-story.md). +> +> **Inserted 2026-08-15**, hence `9c`. It follows 9b because a program +> attaching to another's tables wants the same typed statements and queries +> the owner has — a surface that must exist before it can be shared — and +> precedes iteration 10 because HTTP is the *external* face of a program; +> this iteration is the *writeonce-native* face, program to program on the +> same machine. +> +> **No spec exists yet.** This iteration frames the outcome and records the +> open forks; the design must be brainstormed before a plan is written. The +> forks in *Info* are genuine decisions, not details. + +## Goals + +- A running program **A** with a persistent database (`WO_DATA`, iteration 9) + can be **attached** by a second writeonce program **B**: B names A's + IPC connection string in its own `wo.toml`, and from then on reads and + writes `A.Table` rows with the same typed statements it uses on its own + tables — checked by B's compiler against A's declared table shapes. +- **A stays the single writer.** B never opens A's WAL, never maps A's + slabs: every statement B issues travels the IPC channel and executes + inside A's engine, through the same choke-point row API A's own + statements use. The ownership doctrine survives contact with a second + process because the second process never touches the memory. +- **Access is granted, never assumed.** A's manifest *registers* B by name + with explicit rights (read, or read+write); an unregistered client is + refused at connect, an under-privileged statement is refused at execute + with a trap B can catch. No registration, no access — including on the + same uid. + +## Acceptance Criteria + +- What to achieve? + - **Given** A running with `[share]` registering client "b" as + read+write, and B's `wo.toml` carrying `[connect.a]` with A's IPC + string, + - **when** B executes `insert a.AuditLog { … }` and a query over + `a.AuditLog`, + - **then** the row exists in A (visible to A's own queries, WAL-logged + before B's insert acknowledges), and B's query returns it — with B's + compiler having checked every field name against A's declared shape. +- What to achieve? + - **Given** A registers client "c" as read-only, + - **when** C executes a query it succeeds, and when C attempts an + insert, + - **then** the insert traps with the access-denied code inside C + (catchable), and A's log records the refusal; nothing was applied, + nothing was WAL-logged. +- What to achieve? + - **Given** a program with no registration in A's manifest, + - **when** it presents A's IPC string and attempts to attach, + - **then** the connect itself is refused — rights are checked at the + door, not per statement only. +- What to achieve? + - **Given** B attached and mid-statement, + - **when** A shuts down cleanly (SIGTERM) or crashes, + - **then** B's in-flight statement traps with a connection error B can + catch (never a hang), and B can re-attach after A reboots and + replays — with every previously acknowledged write still present. +- What to achieve? + - **Given** the employee sample running as A with its departments and + employees tables, + - **when** a second sample program (a thin reporting client) attaches + read-only and runs the GroupBy report over `a.Employee`, + - **then** it prints the same report the owner prints — the + demonstration that attach + query compose. + +## Out Of Scope + +- **Remote machines.** The IPC string names a local channel; cross-host + access is the HTTP/service layer's job (iteration 10) or a much later + network protocol. Same-machine is what "attach" means here. +- **B caching A's rows.** Every read crosses the channel; a client-side + cache (and its invalidation) is a later performance iteration, if ever. +- **Cross-program transactions.** A statement is atomic inside A exactly as + A's own statements are; B cannot open a transaction spanning its own + tables and A's. That is 2PC territory, recorded with the database track's + deferred items. +- **`LIVE` subscriptions over the channel** — composes with the + subscription registry later (the client-api phase doc already sketches + the wire shape). +- **Schema migration while attached** — a blue-green swap in A while B + holds an attachment is iteration 12's compatibility problem; this + iteration may simply drop attachments on swap. + +## Info + +Prior art in the tree: `docs/runtime/database/04-client-api.md` already +designs a native binary wire protocol for external clients (length-prefixed, +typed, subscription-ready) — this iteration's channel should be its +same-machine profile, not a new invention. The WAL's typed value encoding +(`database/src/wal.c`, iteration 9 Task 2) is a working engine-value wire +format today: statements and rows can ride the same encoding the log already +uses. The `wo.toml` manifest exists and is compiler-read (`woc `), so +both ends' declarations have a natural home. + +Forks the spec must settle: + +**1. What carries the channel — and what does the IPC string name?** +A unix domain socket is the obvious carrier (peer credentials for free, +`net`-stdlib adjacency); the string would be `unix:/path/a.sock` in B's +`[connect.a]` and A would listen beside its `WO_DATA` directory. The +alternatives — a FIFO pair, shared memory + doorbell — buy latency at the +cost of the credential story and the crash-detection story (a dead socket +peer is unambiguous; a dead shm peer is a protocol). Leaning: unix socket, +one connection per attached client, A serving requests on its event loop +(iteration 8's shard-actor loop when it lands; a dedicated accept loop +until then — which is also the fork's dependency question: how much of +iteration 8 does this need?). + +**2. How does B's compiler know A's table shapes?** B typechecks +`a.Employee { … }` against A's declarations, so B needs them at compile +time. Options: B's `[connect.a]` names A's **project directory** and `woc` +reads A's types straight from A's source (simple, but couples B's build to +A's checkout); A **exports a schema file** (a `.wob`-adjacent digest of its +class table) that B's manifest points at (decoupled, but a new artifact +with a staleness story); or shared type definitions in a common module both +import (cleanest language story, needs the module system to span projects). +A runtime schema handshake must exist regardless — B's compiled expectation +of `a.Employee`'s shape is verified against A's live class table at attach, +and a mismatch refuses the attachment with both sides' shapes named. +Leaning: project-directory reference for the milestone plus the mandatory +handshake; the export artifact when the staleness story matters. + +**3. What exactly does A's registration grant?** The request's shape is +per-client rights: `[share] clients = [{ name = "b", rights = "rw" }]` or +per-table refinement (`tables = ["AuditLog"]`). Identity: the client NAME +must be bound to something a peer cannot fake — unix peer credentials +(uid), a token A mints, or both. Leaning: name + uid via `SO_PEERCRED` for +the milestone (same-machine, same-trust-domain), rights whole-database +read or read+write (per-table refinement deferred until a workload needs +it), and the registration is A's manifest so a grant is a config change + +restart, not an API. + +**4. What does B's statement actually block on?** B's insert crosses the +channel, executes in A (RAM + WAL + fsync), and acknowledges back — a +blocking round-trip on B's thread, exactly like B's own `WO_DATA` inserts +block on their own fsync. Queries stream results back whole (materialized; +no cursors over the wire this iteration). The alternative — async +statements with completion callbacks — has no language surface to stand on +(no function values) and waits for fibers (iteration 11). Leaning: +blocking, with the stop-flag rule from the log-watcher work applying (a +SIGTERM'd B parked on a channel read exits cleanly). + +## Proposed Solution + +- **Brainstorm the spec first**, settling the four forks; then a plan. + Expected shape: A-side — a listener beside the engine, a request + dispatcher that executes through the same statement executors iteration + 9 built (`database/src/db.c`), the registration check at accept and per + statement; B-side — `[connect.]` manifest surface, compiler + namespace `.Table` binding table statements/queries to channel + stubs instead of local engine builtins; both — the client-api phase + doc's wire protocol, profiled for unix sockets, values in the WAL's + encoding. +- **The acceptance workload extends the employee sample**: A = the employee + program with `[share]`; B = a new thin `docs/examples/employee-report` + client attaching read-only for the GroupBy report, plus a read+write + audit-log writer path exercising the rights matrix and the refusal + traps. The sample stays the test. +- Depends on iterations 9 (engine, WAL — done through Task 3 as of + 2026-08-15) and 9b (typed statements and queries worth sharing); wants + iteration 8's event loop for A's serving side but can prototype on a + dedicated accept loop the way the MCP sample serves today.