docs: iteration 9c story -- cross-program tables (attach over IPC)
- program B attaches to running program A's persistent database via an IPC string in B's wo.toml [connect.<name>]; A registers clients by name with read / read+write rights in its [share] manifest section; unregistered = refused at connect, under-privileged = catchable trap - doctrine preserved: A stays the single writer -- B's statements execute inside A through the same choke-point row API, B never touches A's WAL or slabs; typed statements checked by B's compiler against A's table shapes, schema handshake at attach - four forks recorded for the spec: channel carrier (lean: unix socket + SO_PEERCRED), how B's compiler learns A's shapes (lean: project reference + live handshake), grant granularity (lean: whole-db rights, name+uid identity), blocking semantics (lean: blocking round-trip, stop-flag rule applies) - acceptance sketch: employee sample as A, a thin employee-report client as B (read-only GroupBy over the wire) + audit-log writer exercising the rights matrix - slots after 9b (shares its typed surface), before 10 (HTTP is the external face; this is the writeonce-native one); prior art: 04-client-api.md wire protocol + the WAL's value encoding - roadmap + status board rows added Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
7374d4d807
commit
59162f7aca
3 changed files with 171 additions and 0 deletions
|
|
@ -117,6 +117,7 @@ that sequences its tasks. Read one, approve, then the next starts.
|
|||
| 8 | [Shard-actor runtime](stories/language-runtime-database/08-shard-actor-runtime.md) | ⬜ |
|
||||
| 9 | [Database engine](stories/language-runtime-database/09-database-engine.md) | ⬜ |
|
||||
| 9b | [`@table`, relations, query](stories/language-runtime-database/09b-table-relations-query.md) | ⬜ spec + plan ready (2026-08-15) |
|
||||
| 9c | [Cross-program tables](stories/language-runtime-database/09c-cross-program-tables.md) | ⬜ needs a spec first |
|
||||
| 10 | [HTTP service layer](stories/language-runtime-database/10-http-service.md) | ⬜ | Hold |
|
||||
| 11 | [Fibers](stories/language-runtime-database/11-fibers.md) | ⬜ | Hold |
|
||||
| 12 | [Blue-green deploy](stories/language-runtime-database/12-blue-green-deploy.md) | ⬜ | Hold |
|
||||
|
|
@ -323,6 +324,7 @@ Ecommerce sample (verified 2026-06-13): `api.rest` 17/17 expected statuses pass.
|
|||
| 8 | Shard-actor runtime | [plan 4](superpowers/plans/2026-08-01-shard-actor-vm-runtime.md) |
|
||||
| 9 | Database engine binding | [plan 5](superpowers/plans/2026-08-01-db-engine-binding.md) |
|
||||
| 9b | `@table` + relations + language-integrated query — comprehension queries, `ref`/`backlink` navigation, GroupBy aggregates; acceptance: new `docs/examples/employee` sample | [spec](superpowers/specs/2026-08-15-table-relations-query-design.md) · [plan](plan/compiler/2026-08-15-employee-relations-query.md) |
|
||||
| 9c | Cross-program tables — attach to a running program's database (IPC string in wo.toml, manifest-granted rights, owner stays the single writer) | **no spec yet** — four open forks recorded in the iteration; brainstorm before planning |
|
||||
| 10 | HTTP service layer | [plan 6](superpowers/plans/2026-08-01-http-service-layer.md) |
|
||||
| 11 | Fibers | vision §3, [blue-green exploration](plan/exploration/blue-green-vm/00-vision.md) |
|
||||
| 12 | Blue-green deploy | [spec](superpowers/specs/2026-08-03-blue-green-vm-design.md) — plan authored after iterations 9–10 |
|
||||
|
|
|
|||
|
|
@ -53,6 +53,7 @@ iterations); no commits by agents — drafts go to `.dev/commit.md`.
|
|||
| 8 | [Shard-actor runtime](08-shard-actor-runtime.md) | thread-per-core shards, per-shard heaps, ownership-move messaging |
|
||||
| 9 | [Database engine](09-database-engine.md) | class-shaped tables, typed WAL + recovery, `insert`/`select` execute |
|
||||
| 9b | [`@table`, relations, query](09b-table-relations-query.md) | `@table` becomes real storage; typed `ref`/`backlink`/`multi` relations; compiler-checked LINQ-shaped queries lowered to engine ops |
|
||||
| 9c | [Cross-program tables](09c-cross-program-tables.md) | attach to a running program's database over a local IPC channel: manifest-granted read/write rights, typed statements checked against the owner's shapes, owner stays the single writer |
|
||||
| 10 | [HTTP service layer](10-http-service.md) | `service` blocks route to VM methods; REST parity with Stage 2 |
|
||||
| 11 | [Fibers](11-fibers.md) | green threads on the shard scheduler: reduction-budget preemption, park on I/O |
|
||||
| 12 | [Blue-green deploy](12-blue-green-deploy.md) | two VM slots, in-runtime compile, atomic switch, resident rollback |
|
||||
|
|
|
|||
|
|
@ -0,0 +1,168 @@
|
|||
# Iteration 9c — cross-program tables: attach to a running program's database
|
||||
|
||||
> Format: fiberloom `product/story-iteration-template`. Part of
|
||||
> [Story — one language, one runtime, one database, one binary](00-story.md).
|
||||
>
|
||||
> **Inserted 2026-08-15**, hence `9c`. It follows 9b because a program
|
||||
> attaching to another's tables wants the same typed statements and queries
|
||||
> the owner has — a surface that must exist before it can be shared — and
|
||||
> precedes iteration 10 because HTTP is the *external* face of a program;
|
||||
> this iteration is the *writeonce-native* face, program to program on the
|
||||
> same machine.
|
||||
>
|
||||
> **No spec exists yet.** This iteration frames the outcome and records the
|
||||
> open forks; the design must be brainstormed before a plan is written. The
|
||||
> forks in *Info* are genuine decisions, not details.
|
||||
|
||||
## Goals
|
||||
|
||||
- A running program **A** with a persistent database (`WO_DATA`, iteration 9)
|
||||
can be **attached** by a second writeonce program **B**: B names A's
|
||||
IPC connection string in its own `wo.toml`, and from then on reads and
|
||||
writes `A.Table` rows with the same typed statements it uses on its own
|
||||
tables — checked by B's compiler against A's declared table shapes.
|
||||
- **A stays the single writer.** B never opens A's WAL, never maps A's
|
||||
slabs: every statement B issues travels the IPC channel and executes
|
||||
inside A's engine, through the same choke-point row API A's own
|
||||
statements use. The ownership doctrine survives contact with a second
|
||||
process because the second process never touches the memory.
|
||||
- **Access is granted, never assumed.** A's manifest *registers* B by name
|
||||
with explicit rights (read, or read+write); an unregistered client is
|
||||
refused at connect, an under-privileged statement is refused at execute
|
||||
with a trap B can catch. No registration, no access — including on the
|
||||
same uid.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- What to achieve?
|
||||
- **Given** A running with `[share]` registering client "b" as
|
||||
read+write, and B's `wo.toml` carrying `[connect.a]` with A's IPC
|
||||
string,
|
||||
- **when** B executes `insert a.AuditLog { … }` and a query over
|
||||
`a.AuditLog`,
|
||||
- **then** the row exists in A (visible to A's own queries, WAL-logged
|
||||
before B's insert acknowledges), and B's query returns it — with B's
|
||||
compiler having checked every field name against A's declared shape.
|
||||
- What to achieve?
|
||||
- **Given** A registers client "c" as read-only,
|
||||
- **when** C executes a query it succeeds, and when C attempts an
|
||||
insert,
|
||||
- **then** the insert traps with the access-denied code inside C
|
||||
(catchable), and A's log records the refusal; nothing was applied,
|
||||
nothing was WAL-logged.
|
||||
- What to achieve?
|
||||
- **Given** a program with no registration in A's manifest,
|
||||
- **when** it presents A's IPC string and attempts to attach,
|
||||
- **then** the connect itself is refused — rights are checked at the
|
||||
door, not per statement only.
|
||||
- What to achieve?
|
||||
- **Given** B attached and mid-statement,
|
||||
- **when** A shuts down cleanly (SIGTERM) or crashes,
|
||||
- **then** B's in-flight statement traps with a connection error B can
|
||||
catch (never a hang), and B can re-attach after A reboots and
|
||||
replays — with every previously acknowledged write still present.
|
||||
- What to achieve?
|
||||
- **Given** the employee sample running as A with its departments and
|
||||
employees tables,
|
||||
- **when** a second sample program (a thin reporting client) attaches
|
||||
read-only and runs the GroupBy report over `a.Employee`,
|
||||
- **then** it prints the same report the owner prints — the
|
||||
demonstration that attach + query compose.
|
||||
|
||||
## Out Of Scope
|
||||
|
||||
- **Remote machines.** The IPC string names a local channel; cross-host
|
||||
access is the HTTP/service layer's job (iteration 10) or a much later
|
||||
network protocol. Same-machine is what "attach" means here.
|
||||
- **B caching A's rows.** Every read crosses the channel; a client-side
|
||||
cache (and its invalidation) is a later performance iteration, if ever.
|
||||
- **Cross-program transactions.** A statement is atomic inside A exactly as
|
||||
A's own statements are; B cannot open a transaction spanning its own
|
||||
tables and A's. That is 2PC territory, recorded with the database track's
|
||||
deferred items.
|
||||
- **`LIVE` subscriptions over the channel** — composes with the
|
||||
subscription registry later (the client-api phase doc already sketches
|
||||
the wire shape).
|
||||
- **Schema migration while attached** — a blue-green swap in A while B
|
||||
holds an attachment is iteration 12's compatibility problem; this
|
||||
iteration may simply drop attachments on swap.
|
||||
|
||||
## Info
|
||||
|
||||
Prior art in the tree: `docs/runtime/database/04-client-api.md` already
|
||||
designs a native binary wire protocol for external clients (length-prefixed,
|
||||
typed, subscription-ready) — this iteration's channel should be its
|
||||
same-machine profile, not a new invention. The WAL's typed value encoding
|
||||
(`database/src/wal.c`, iteration 9 Task 2) is a working engine-value wire
|
||||
format today: statements and rows can ride the same encoding the log already
|
||||
uses. The `wo.toml` manifest exists and is compiler-read (`woc <dir>`), so
|
||||
both ends' declarations have a natural home.
|
||||
|
||||
Forks the spec must settle:
|
||||
|
||||
**1. What carries the channel — and what does the IPC string name?**
|
||||
A unix domain socket is the obvious carrier (peer credentials for free,
|
||||
`net`-stdlib adjacency); the string would be `unix:/path/a.sock` in B's
|
||||
`[connect.a]` and A would listen beside its `WO_DATA` directory. The
|
||||
alternatives — a FIFO pair, shared memory + doorbell — buy latency at the
|
||||
cost of the credential story and the crash-detection story (a dead socket
|
||||
peer is unambiguous; a dead shm peer is a protocol). Leaning: unix socket,
|
||||
one connection per attached client, A serving requests on its event loop
|
||||
(iteration 8's shard-actor loop when it lands; a dedicated accept loop
|
||||
until then — which is also the fork's dependency question: how much of
|
||||
iteration 8 does this need?).
|
||||
|
||||
**2. How does B's compiler know A's table shapes?** B typechecks
|
||||
`a.Employee { … }` against A's declarations, so B needs them at compile
|
||||
time. Options: B's `[connect.a]` names A's **project directory** and `woc`
|
||||
reads A's types straight from A's source (simple, but couples B's build to
|
||||
A's checkout); A **exports a schema file** (a `.wob`-adjacent digest of its
|
||||
class table) that B's manifest points at (decoupled, but a new artifact
|
||||
with a staleness story); or shared type definitions in a common module both
|
||||
import (cleanest language story, needs the module system to span projects).
|
||||
A runtime schema handshake must exist regardless — B's compiled expectation
|
||||
of `a.Employee`'s shape is verified against A's live class table at attach,
|
||||
and a mismatch refuses the attachment with both sides' shapes named.
|
||||
Leaning: project-directory reference for the milestone plus the mandatory
|
||||
handshake; the export artifact when the staleness story matters.
|
||||
|
||||
**3. What exactly does A's registration grant?** The request's shape is
|
||||
per-client rights: `[share] clients = [{ name = "b", rights = "rw" }]` or
|
||||
per-table refinement (`tables = ["AuditLog"]`). Identity: the client NAME
|
||||
must be bound to something a peer cannot fake — unix peer credentials
|
||||
(uid), a token A mints, or both. Leaning: name + uid via `SO_PEERCRED` for
|
||||
the milestone (same-machine, same-trust-domain), rights whole-database
|
||||
read or read+write (per-table refinement deferred until a workload needs
|
||||
it), and the registration is A's manifest so a grant is a config change +
|
||||
restart, not an API.
|
||||
|
||||
**4. What does B's statement actually block on?** B's insert crosses the
|
||||
channel, executes in A (RAM + WAL + fsync), and acknowledges back — a
|
||||
blocking round-trip on B's thread, exactly like B's own `WO_DATA` inserts
|
||||
block on their own fsync. Queries stream results back whole (materialized;
|
||||
no cursors over the wire this iteration). The alternative — async
|
||||
statements with completion callbacks — has no language surface to stand on
|
||||
(no function values) and waits for fibers (iteration 11). Leaning:
|
||||
blocking, with the stop-flag rule from the log-watcher work applying (a
|
||||
SIGTERM'd B parked on a channel read exits cleanly).
|
||||
|
||||
## Proposed Solution
|
||||
|
||||
- **Brainstorm the spec first**, settling the four forks; then a plan.
|
||||
Expected shape: A-side — a listener beside the engine, a request
|
||||
dispatcher that executes through the same statement executors iteration
|
||||
9 built (`database/src/db.c`), the registration check at accept and per
|
||||
statement; B-side — `[connect.<name>]` manifest surface, compiler
|
||||
namespace `<name>.Table` binding table statements/queries to channel
|
||||
stubs instead of local engine builtins; both — the client-api phase
|
||||
doc's wire protocol, profiled for unix sockets, values in the WAL's
|
||||
encoding.
|
||||
- **The acceptance workload extends the employee sample**: A = the employee
|
||||
program with `[share]`; B = a new thin `docs/examples/employee-report`
|
||||
client attaching read-only for the GroupBy report, plus a read+write
|
||||
audit-log writer path exercising the rights matrix and the refusal
|
||||
traps. The sample stays the test.
|
||||
- Depends on iterations 9 (engine, WAL — done through Task 3 as of
|
||||
2026-08-15) and 9b (typed statements and queries worth sharing); wants
|
||||
iteration 8's event loop for A's serving side but can prototype on a
|
||||
dedicated accept loop the way the MCP sample serves today.
|
||||
Loading…
Reference in a new issue