diff --git a/docs/00-status.md b/docs/00-status.md index 3c96b59..92420f7 100644 --- a/docs/00-status.md +++ b/docs/00-status.md @@ -130,6 +130,8 @@ that sequences its tasks. Read one, approve, then the next starts. | 12 | [Blue-green deploy](stories/language-runtime-database/12-blue-green-deploy.md) | ⬜ | Hold | | 13 | [Compile-time metaprogramming](stories/language-runtime-database/13-compile-time-metaprogramming.md) | ⬜ needs a spec first | | 14 | [skillhost host workload](stories/language-runtime-database/14-skillhost-host-workload.md) | ⬜ gaps recorded (branch query-grammar found skillhost needs no new query grammar); each gap a candidate iteration | +| 15 | [deps: `wo.toml [deps]`](stories/language-runtime-database/15-deps-package-manager.md) | ⬜ spec'd 2026-08-18 (web-framework spec §A); the enabler for 16 | +| 16 | [web framework](stories/language-runtime-database/16-web-framework.md) | ⬜ spec'd 2026-08-18 (§B; h2c parked §C behind 8/9f/11); TLS proxy-terminated by decision | --- @@ -318,6 +320,8 @@ The C proving-ground work (`exploration/c-runtime/`, phases A–F: 859k reads/s, | 9f | io_uring group-commit write path — batched durability overlapped on shard threads, fsync fallback | **no spec yet** — brainstorm after iterations 8 + 9e | | 9g | Query grammar from real embedded-DB corpora — whole-query count + correlated exists, driven by the skillhost SQL catalogue; add only what a corpus uses | **no spec yet** — three forks; may collapse to "confirm len(query) + add exists" | | 14 | skillhost host workload — port skillhost (MCP host + confined script runner) to writeonce; drives the missing host capabilities into the open (bounded subprocess, stdin/stdout transport, fs metadata, FFI-vs-out-of-process) | **no spec yet** — gaps recorded in the iteration; each gap brainstormed on demand, bounded-subprocess first | +| 15 | deps — `wo.toml [deps]` exact-rev git fetch, `wo.lock`, `.wo-deps` cache, `use ` resolution; flat-only v1 | [spec §A](superpowers/specs/2026-08-18-web-framework-design.md) — plan after board approval | +| 16 | web framework — `.wo` library (HTTP/1.1 keep-alive behind a TLS-terminating proxy), Handler/Middleware interfaces, @table data layer; web-app sample consumes via [deps] | [spec §B](superpowers/specs/2026-08-18-web-framework-design.md) — depends on 15; h2c parked (§C) behind 8/9f/11 | | 10 | HTTP service layer | [plan 6](superpowers/plans/2026-08-01-http-service-layer.md) | | 11 | Fibers | vision §3, [blue-green exploration](plan/exploration/blue-green-vm/00-vision.md) | | 12 | Blue-green deploy | [spec](superpowers/specs/2026-08-03-blue-green-vm-design.md) — plan authored after iterations 9–10 | diff --git a/docs/stories/language-runtime-database/00-story.md b/docs/stories/language-runtime-database/00-story.md index 7382c24..b447196 100644 --- a/docs/stories/language-runtime-database/00-story.md +++ b/docs/stories/language-runtime-database/00-story.md @@ -63,6 +63,8 @@ iterations); no commits by agents — drafts go to `.dev/commit.md`. | 12 | [Blue-green deploy](12-blue-green-deploy.md) | two VM slots, in-runtime compile, atomic switch, resident rollback | | 13 | [Compile-time metaprogramming](13-compile-time-metaprogramming.md) | `@derive(Json/Csv/Eq/Hash/Show)` — the compiler generates per-type code from the class-table metadata; generic capabilities within principle 13, no reflection | | 14 | [skillhost host workload](14-skillhost-host-workload.md) | a host-shaped driving workload (writeonce port of skillhost) that names the runtime gaps it exposes: bounded/killable subprocess, stdin/stdout transport, fs metadata, and FFI-vs-out-of-process model driver — each a candidate iteration | +| 15 | [deps: `wo.toml [deps]`](15-deps-package-manager.md) | exact-rev git dependencies + `wo.lock` + `.wo-deps` cache; `use ` resolves a fetched project as a module root; flat-only, network-free when locked | +| 16 | [web framework](16-web-framework.md) | a `.wo`-library framework (HTTP/1.1 keep-alive behind a TLS-terminating proxy): router, `Handler`/`Middleware` structural interfaces, `@table` data layer; `docs/examples/web-app` consumes it via `[deps]`; h2c parked behind 8/9f/11 | Review protocol: the developer reads one iteration, approves or amends; the next starts only after approval. Each iteration is an unsplittable diff --git a/docs/stories/language-runtime-database/15-deps-package-manager.md b/docs/stories/language-runtime-database/15-deps-package-manager.md new file mode 100644 index 0000000..cf0d538 --- /dev/null +++ b/docs/stories/language-runtime-database/15-deps-package-manager.md @@ -0,0 +1,49 @@ +# Iteration 15 — dependencies: `wo.toml [deps]`, git fetch, `wo.lock` + +> Format: fiberloom `product/story-iteration-template`. Part of +> [Story — one language, one runtime, one database, one binary](00-story.md). +> +> **Inserted 2026-08-18.** The enabler for code shared between writeonce +> repositories — the web framework (iteration 16) is the driving consumer. +> +> **Spec exists:** [`2026-08-18-web-framework-design.md`](../../superpowers/specs/2026-08-18-web-framework-design.md) +> section A is normative for this iteration. + +## Goals + +- A project declares exact-rev git dependencies in `wo.toml [deps]` + (`name = { git = "...", rev = "..." }`); `woc` fetches them (via the `git` + binary — no network code in the compiler) into `.wo-deps//` and + resolves `use ` / `use /sub` into the dep's module tree. +- `wo.lock` pins resolved SHAs: builds are reproducible, a moved tag is + reported rather than silently followed, and a lock-satisfied build never + touches the network. +- Honest edges: transitive `[deps]` refused with a diagnostic (flat-only v1), + dep/local module-name collisions diagnosed, a dep's `fn main` ignored, + `pub` applies across the boundary exactly as across modules. + +## Acceptance Criteria + +- **Given** an app whose `[deps]` names a framework in a local `file://` git + repo, **when** `woc ` runs twice, **then** the first run fetches + + writes `wo.lock`, the second builds offline from `.wo-deps`, and the built + binary runs. +- **Given** the dep's tag moved after `wo.lock` was written, **when** the app + builds, **then** the lock wins and the drift is reported; + `woc --update-deps` refreshes it. +- **Given** a dep whose own `wo.toml` has `[deps]`, or a dep name colliding + with a local module, **when** the app builds, **then** each is a named + diagnostic, never silent misresolution. + +## Out Of Scope + +Registries, version ranges/semver solving, transitive dependencies (the +successor's first fork), private-remote auth handling beyond what ambient +`git` config provides, vendoring commands. + +## Proposed Solution + +Manifest parsing extends `compiler/bin/main.ml`'s existing wo.toml reader; +fetch = `Sys.command` over the `git` binary; resolution plugs the dep root +into the existing directory-as-module discovery. Fixtures under `tests/` use +local `file://` remotes so the suite stays network-free. diff --git a/docs/stories/language-runtime-database/16-web-framework.md b/docs/stories/language-runtime-database/16-web-framework.md new file mode 100644 index 0000000..d47b804 --- /dev/null +++ b/docs/stories/language-runtime-database/16-web-framework.md @@ -0,0 +1,67 @@ +# Iteration 16 — the web framework: a `.wo` library, HTTP/1.1 behind a proxy + +> Format: fiberloom `product/story-iteration-template`. Part of +> [Story — one language, one runtime, one database, one binary](00-story.md). +> +> **Inserted 2026-08-18.** The framework is written IN writeonce and imported +> like any dependency (iteration 15 is the prerequisite). TLS terminates at a +> reverse proxy — browsers get TLS+ALPN+h2 from nginx/caddy while the +> framework speaks HTTP/1.1 keep-alive behind it, so no TLS exists anywhere +> in the toolchain. h2c is the parked successor (after iterations 8/9f/11, +> when multiplexing has a scheduler to pay off on). +> +> **Spec exists:** [`2026-08-18-web-framework-design.md`](../../superpowers/specs/2026-08-18-web-framework-design.md) +> sections B (normative) and C (the parked h2c successor). + +## Goals + +- **The framework**, incubated at `docs/examples/writeonce-framework/`: an + HTTP/1.1 keep-alive server core (extracted from the soak-proven log-watcher + `mcp` pattern), a method+path router with `:param` captures, `Req`/`Resp` + records with builder helpers, and the no-function-values handler model — + `Handler` / `Middleware` structural interfaces dispatched by ICALL, with + WO-E205 making a non-conforming handler a compile error and `?Resp` + middleware short-circuiting on the shipped `?T` narrowing. +- **The consuming app**, `docs/examples/web-app/`: a small storefront + (`Product`/`Order` as `@table` classes, list/show/create routes, one auth + middleware, JSON responses) that imports the framework **through + `[deps]`** — the sample exercises the whole chain: fetch → lock → build → + serve → durable data across restart. +- **Honest limits stated where users read them**: single-threaded blocking + (concurrency arrives underneath via iterations 8/11), no chunked encoding, + no WebSockets, JSON-first (no templates — the removed UI track stays + removed). +- **Relationship to iteration 10 recorded in both**: `service` blocks later + *lower onto this library* — compiler sugar over the same router, never a + rival stack. + +## Acceptance Criteria + +- **Given** the web-app sample, **when** `just web-app` runs, **then** deps + fetch, the app builds, serves list/show/create with `@table` persistence + across a process restart, answers 404/400 correctly, a deliberately + trapping handler returns 500 while the server survives, and SIGTERM stops + it cleanly — fd- and resident-flat under the opt-in soak. +- **Given** a handler class missing `handle`, **when** the app compiles, + **then** WO-E205 names the class and interface. +- **Given** nginx in front (documented config), **when** a browser hits it, + **then** the browser negotiates h2 with the proxy while the backend link is + HTTP/1.1 — proving the TLS/h2 story without TLS in writeonce. +- **Given** the framework extracted to its own repository, **when** the app's + `[deps]` URL is updated, **then** nothing else changes (success criterion 4 + of the spec). + +## Out Of Scope + +TLS in the toolchain (proxy-terminated by decision); HTTP/2 + the +bytes/buffer type (parked to the h2c successor, after 8/9f/11); chunked +transfer encoding; WebSockets/SSE; templates/SSR; multipart uploads; +performance work beyond the soak's flatness gate (benchmarks belong to 9e's +measurement backbone). + +## Proposed Solution + +Framework modules `http/`, `router/`, `app.wo` as spec §B lays out; the +web-app acceptance script is the gate (curl matrix + restart + stop + soak); +`just web-app` wires fetch-build-serve-verify into one command. Plan follows +after this iteration is approved on the board. diff --git a/docs/superpowers/specs/2026-08-18-web-framework-design.md b/docs/superpowers/specs/2026-08-18-web-framework-design.md new file mode 100644 index 0000000..5c7226d --- /dev/null +++ b/docs/superpowers/specs/2026-08-18-web-framework-design.md @@ -0,0 +1,184 @@ +# Web framework + dependency system — design spec + +**Date:** 2026-08-18 +**Status:** approved design, pre-implementation +**Scope:** (A) a minimal git-backed dependency system (`wo.toml [deps]` + +`wo.lock`), (B) a web framework written in writeonce as an importable `.wo` +library, HTTP/1.1 behind a TLS-terminating reverse proxy, and (C) the parked +HTTP/2 path. Three sub-projects; A and B are the fundable ones, C is a +recorded successor. +**Relates to:** iteration 10 (`service` blocks — this framework becomes their +lowering target, not a rival), iterations 8/9f/11 (the concurrency work h2c +waits for), `docs/plan/discarded.md` (FFI reject row — load-bearing here). + +## Decisions locked during brainstorming + +| Question | Decision | +| --- | --- | +| TLS | **Proxy-terminated** (nginx/caddy). Browsers get TLS+ALPN+h2 from the proxy; the framework speaks HTTP/1.1 (later h2c) behind it. Zero TLS in the language or runtime. Direct-serving TLS is a separate future iteration and would be judged against the FFI/libc-only doctrine then, not now. Homegrown TLS is refused outright: a decade of side-channel and certificate-validation subtleties makes it a security liability, not a milestone. | +| Dependencies | **`wo.toml [deps]` + git fetch.** A real (mini) package manager: exact-rev git dependencies, a lockfile, a per-project cache. No registry, no semver solving. | +| HTTP/2 | **v1 is HTTP/1.1 keep-alive.** h2's payoff is multiplexing, which a single blocking thread cannot exploit; h2c lands as its own iteration after shards (8) / io_uring (9f) / fibers (11). Behind the proxy, browsers see h2 from day one regardless. | +| Handler model | **Structural interfaces, not closures.** The language has no function values by doctrine; a route handler is a class satisfying a `Handler` interface, dispatched by ICALL — which works on today's runtime and is checked by WO-E205. | +| Incubation | Framework is born at `docs/examples/writeonce-framework/`; the consuming app at `docs/examples/web-app/` imports it **through the `[deps]` mechanism** (a local git URL), so the whole import chain is exercised by the sample. Extraction to `github.com/shoneyj/` later is a `git subtree split`, not a redesign. | + +Rejected: TLS in the runtime via rustls/OpenSSL (breaks libc-only for a +benefit the proxy already provides; revisit only if direct serving becomes a +requirement); nghttp2 binding (same doctrine cost, and the scheduler cannot +use multiplexing yet); vendor-directory imports (the user chose the real +dependency mechanism); native h2 in v1 (HPACK synchronization and +flow-control deadlocks are where the time goes — the user's own estimate). + +## A. Dependency system (`wo.toml [deps]`, `wo.lock`) + +**Manifest.** `wo.toml` gains one section: + +``` +[deps] +niceframework = { git = "https://github.com/shoneyj/niceframework", rev = "v0.1.0" } +``` + +`rev` is mandatory and exact (a tag or SHA). No version ranges, no registry, +no resolution algorithm — those are future work a corpus of real deps must +justify first. + +**Fetch.** `woc` shells out to the `git` binary (`git clone --depth 1`, +`git checkout `) — no network code inside the compiler, OCaml-stdlib +doctrine intact; `git` joins `cc` in the "external tools the toolchain may +invoke" set. Dependencies land in `.wo-deps//` beside `wo.toml` +(gitignored). A missing `git` binary or unreachable remote is a plain +diagnostic, never a hang without a message. + +**Lockfile.** `wo.lock` records `name -> resolved SHA` for every fetch. +Present lockfile wins over the manifest's rev label (a moved tag is detected +and reported, not silently followed). `woc --update-deps` refreshes the lock; +plain builds never touch the network when `.wo-deps` already satisfies the +lock. + +**Resolution.** A dependency is an ordinary writeonce project (its own +`wo.toml` with `name`). `use ` resolves the dep's root directory as +a module root, exactly like a project-internal module; `use /sub` +reaches its subdirectories. Collisions between a dep name and a local module +diagnose (the existing WO-E collision family). + +**Transitive deps are refused in v1.** A fetched dep whose own `wo.toml` +carries `[deps]` is a diagnostic naming the dep — flat-only keeps the +resolver small and the failure honest. Recorded as the successor iteration's +first fork. + +**Entry-point rule.** A dep's `fn main` (if any) is ignored — only the +consuming app owns the entry. `pub` visibility applies across the dep +boundary exactly as across modules. + +## B. The framework (v1, HTTP/1.1, library-only) + +**Repo shape.** `docs/examples/writeonce-framework/` is a complete writeonce +project (`wo.toml name = "niceframework"` — final name decided at extraction) +whose modules are the framework: + +- `http/` — request parsing, response serialization, keep-alive loop. + Extracted from the proven log-watcher `mcp` pattern (601k requests soaked, + fd-clean, SIGTERM-clean). +- `router/` — method+path table with `:param` captures. +- `app.wo` — the assembly: register routes/middleware, `serve(port)`. + +**Types (records).** + +- `Req`: `method: Text`, `path: Text`, `params: map` (the + `:param` captures), `query: map`, `headers: map`, + `body: Text`. +- `Resp`: `status: Int`, `headers: map`, `body: Text`, plus + builder helpers (`ok_json`, `ok_text`, `not_found`, `redirect`, …). + +**Handlers.** One structural interface: + +``` +interface Handler { + fn handle(req: Req) -> Resp +} +``` + +A route is any class satisfying it (state lives in the class's own fields — +the closure substitute). Registration passes the handler *value*: +`app.get("/products/:id", ProductShow { ... })`. Dispatch is ICALL; +WO-E205 makes a non-conforming handler a compile error. + +**Middleware.** Its own one-method interface (distinct from `Handler`, +because the signatures differ): + +``` +interface Middleware { + fn before(req: Req) -> ?Resp +} +``` + +The app holds a `multi` of middleware run in order before routing (auth, +logging, content-type defaults). Returning a `Resp` short-circuits; nil means +"continue" — `?Resp` narrowing is exactly what the 2026-08-18 `?T` +enforcement shipped, used as a design tool. + +**Server core.** `net.listen/accept` blocking loop, HTTP/1.1 with keep-alive +and `Content-Length` bodies (no chunked encoding in v1 — disclosed), fd +closed on every exit path, `env.stopping()` honored. **Single-threaded, +blocking, one request at a time** — stated in the framework README, with the +proxy config (nginx upstream keepalive) as the deployment story. Concurrency +arrives via iterations 8/11 underneath the same library surface. + +**Database.** Nothing to build: handlers use `@table` + the query surface +directly. This is the differentiator — a durable, compiler-checked data layer +with no ORM and no separate database process, in the same binary. + +**The consuming app.** `docs/examples/web-app/` — a small storefront: +`Product`/`Order` as `@table` classes, list/show/create routes, one auth +middleware, JSON responses. Its `wo.toml [deps]` points at the framework by +git URL (`file://` in CI, the GitHub URL after extraction). Its acceptance +script is the whole feature's gate: fetch deps, build, serve, curl the +routes, restart-persistence check, SIGTERM. + +## C. HTTP/2 (h2c) — parked successor + +After iterations 8 (shards) / 9f (io_uring) / 11 (fibers): h2c framing + +HPACK, either natively (needs a bytes/buffer type with cheap slicing — that +type rides with this iteration, not v1) or via nghttp2-in-runtime (a doctrine +decision to re-argue then, with the TweetNaCl precedent and the libc-only +rule both on the table). Behind the proxy, h2c's win is backend multiplexing; +browsers already had h2 since v1. No TLS obligation even here. + +## Error handling + +Dep failures (missing git, bad rev, dirty cache, transitive deps) are +compiler diagnostics with the WO-E1xx driver family. Framework runtime +failures follow house rules: malformed requests get a 400 and a closed +connection, handler traps are caught at the serve loop (`try`) and answered +with a 500 — a bad request must never kill the server; the soak asserts +resident + fd flatness exactly like log-watcher's. + +## Testing + +- Dep system: fixture projects under `tests/` — fetch-and-build from a local + git repo, lockfile drift detection, transitive-dep refusal, collision + diagnostic. Network-free (local `file://` remotes). +- Framework: unit-ish `.wo` fixtures for the router and parser; the web-app + acceptance script (curl matrix incl. keep-alive reuse, 404/400/500 paths, + restart persistence, stop) is the gate; `LW_SOAK`-style opt-in soak. +- The whole chain (deps fetch -> build -> serve) runs in `just web-app`. + +## Success criteria + +1. `docs/examples/web-app` builds by fetching `writeonce-framework` through + `[deps]` + `wo.lock` with no path references, and `woc` never touches the + network when the lock is satisfied. +2. The storefront serves list/show/create with `@table` persistence across a + restart, behind nginx with browser-visible h2 (proxy-terminated), while + the backend speaks HTTP/1.1. +3. A non-conforming handler class is WO-E205 at compile time; a trapping + handler answers 500 and the server survives (soak-proven). +4. Framework extraction to a standalone repo requires changing only the + app's `[deps]` URL. + +## Out of scope (v1) + +TLS anywhere in the toolchain; HTTP/2 and the bytes/buffer type (parked to +C); chunked transfer encoding; WebSockets/SSE (needs the push story); +transitive dependencies, version ranges, registries; templates/SSR (the +removed UI track is not resurrected here — JSON APIs first); multipart +uploads.