docs: web framework + deps design — spec + iterations 15/16

Brainstorm outcome (forks locked with the developer):
- TLS: proxy-terminated (nginx/caddy gives browsers TLS+ALPN+h2; the
  framework speaks HTTP/1.1 behind it) — zero TLS in the toolchain, no
  doctrine fight; homegrown TLS refused outright.
- Dependencies: a real mini package manager — wo.toml [deps] with exact-rev
  git deps, wo.lock, .wo-deps cache, `use <dep>` as a module root; fetch by
  shelling to the git binary (no network code in woc); flat-only v1.
- HTTP/2: v1 is HTTP/1.1 keep-alive; h2c is the parked successor behind
  iterations 8/9f/11 (multiplexing needs a scheduler to pay off); the
  bytes/buffer type rides with it, not v1.
- Handler model: no function values by doctrine, so Handler/Middleware are
  structural interfaces (ICALL dispatch, WO-E205-checked); middleware returns
  ?Resp and rides the shipped ?T narrowing.
- Incubation: framework at docs/examples/writeonce-framework/, consuming
  storefront at docs/examples/web-app/ importing it THROUGH [deps] — the
  sample exercises fetch -> lock -> build -> serve -> durable-restart.
- Iteration 10 relationship: service blocks later LOWER ONTO this library.

Files: specs/2026-08-18-web-framework-design.md (A deps normative, B
framework normative, C h2c parked); stories 15-deps-package-manager.md +
16-web-framework.md; roadmap + board rows.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
shoney.arickathil 2026-08-19 19:07:16 +02:00
parent f282451867
commit 976ccda225
5 changed files with 306 additions and 0 deletions

View file

@ -130,6 +130,8 @@ that sequences its tasks. Read one, approve, then the next starts.
| 12 | [Blue-green deploy](stories/language-runtime-database/12-blue-green-deploy.md) | ⬜ | Hold |
| 13 | [Compile-time metaprogramming](stories/language-runtime-database/13-compile-time-metaprogramming.md) | ⬜ needs a spec first |
| 14 | [skillhost host workload](stories/language-runtime-database/14-skillhost-host-workload.md) | ⬜ gaps recorded (branch query-grammar found skillhost needs no new query grammar); each gap a candidate iteration |
| 15 | [deps: `wo.toml [deps]`](stories/language-runtime-database/15-deps-package-manager.md) | ⬜ spec'd 2026-08-18 (web-framework spec §A); the enabler for 16 |
| 16 | [web framework](stories/language-runtime-database/16-web-framework.md) | ⬜ spec'd 2026-08-18 (§B; h2c parked §C behind 8/9f/11); TLS proxy-terminated by decision |
---
@ -318,6 +320,8 @@ The C proving-ground work (`exploration/c-runtime/`, phases A–F: 859k reads/s,
| 9f | io_uring group-commit write path — batched durability overlapped on shard threads, fsync fallback | **no spec yet** — brainstorm after iterations 8 + 9e |
| 9g | Query grammar from real embedded-DB corpora — whole-query count + correlated exists, driven by the skillhost SQL catalogue; add only what a corpus uses | **no spec yet** — three forks; may collapse to "confirm len(query) + add exists" |
| 14 | skillhost host workload — port skillhost (MCP host + confined script runner) to writeonce; drives the missing host capabilities into the open (bounded subprocess, stdin/stdout transport, fs metadata, FFI-vs-out-of-process) | **no spec yet** — gaps recorded in the iteration; each gap brainstormed on demand, bounded-subprocess first |
| 15 | deps — `wo.toml [deps]` exact-rev git fetch, `wo.lock`, `.wo-deps` cache, `use <dep>` resolution; flat-only v1 | [spec §A](superpowers/specs/2026-08-18-web-framework-design.md) — plan after board approval |
| 16 | web framework — `.wo` library (HTTP/1.1 keep-alive behind a TLS-terminating proxy), Handler/Middleware interfaces, @table data layer; web-app sample consumes via [deps] | [spec §B](superpowers/specs/2026-08-18-web-framework-design.md) — depends on 15; h2c parked (§C) behind 8/9f/11 |
| 10 | HTTP service layer | [plan 6](superpowers/plans/2026-08-01-http-service-layer.md) |
| 11 | Fibers | vision §3, [blue-green exploration](plan/exploration/blue-green-vm/00-vision.md) |
| 12 | Blue-green deploy | [spec](superpowers/specs/2026-08-03-blue-green-vm-design.md) — plan authored after iterations 9–10 |

View file

@ -63,6 +63,8 @@ iterations); no commits by agents — drafts go to `.dev/commit.md`.
| 12 | [Blue-green deploy](12-blue-green-deploy.md) | two VM slots, in-runtime compile, atomic switch, resident rollback |
| 13 | [Compile-time metaprogramming](13-compile-time-metaprogramming.md) | `@derive(Json/Csv/Eq/Hash/Show)` — the compiler generates per-type code from the class-table metadata; generic capabilities within principle 13, no reflection |
| 14 | [skillhost host workload](14-skillhost-host-workload.md) | a host-shaped driving workload (writeonce port of skillhost) that names the runtime gaps it exposes: bounded/killable subprocess, stdin/stdout transport, fs metadata, and FFI-vs-out-of-process model driver — each a candidate iteration |
| 15 | [deps: `wo.toml [deps]`](15-deps-package-manager.md) | exact-rev git dependencies + `wo.lock` + `.wo-deps` cache; `use <dep>` resolves a fetched project as a module root; flat-only, network-free when locked |
| 16 | [web framework](16-web-framework.md) | a `.wo`-library framework (HTTP/1.1 keep-alive behind a TLS-terminating proxy): router, `Handler`/`Middleware` structural interfaces, `@table` data layer; `docs/examples/web-app` consumes it via `[deps]`; h2c parked behind 8/9f/11 |
Review protocol: the developer reads one iteration, approves or amends;
the next starts only after approval. Each iteration is an unsplittable

View file

@ -0,0 +1,49 @@
# Iteration 15 — dependencies: `wo.toml [deps]`, git fetch, `wo.lock`
> Format: `product/story-iteration-template`. Part of
> [Story — one language, one runtime, one database, one binary](00-story.md).
>
> **Inserted 2026-08-18.** The enabler for code shared between writeonce
> repositories — the web framework (iteration 16) is the driving consumer.
>
> **Spec exists:** [`2026-08-18-web-framework-design.md`](../../superpowers/specs/2026-08-18-web-framework-design.md)
> section A is normative for this iteration.
## Goals
- A project declares exact-rev git dependencies in `wo.toml [deps]`
(`name = { git = "...", rev = "..." }`); `woc` fetches them (via the `git`
binary — no network code in the compiler) into `.wo-deps/<name>/` and
resolves `use <name>` / `use <name>/sub` into the dep's module tree.
- `wo.lock` pins resolved SHAs: builds are reproducible, a moved tag is
reported rather than silently followed, and a lock-satisfied build never
touches the network.
- Honest edges: transitive `[deps]` refused with a diagnostic (flat-only v1),
dep/local module-name collisions diagnosed, a dep's `fn main` ignored,
`pub` applies across the boundary exactly as across modules.
## Acceptance Criteria
- **Given** an app whose `[deps]` names a framework in a local `file://` git
repo, **when** `woc <app>` runs twice, **then** the first run fetches +
writes `wo.lock`, the second builds offline from `.wo-deps`, and the built
binary runs.
- **Given** the dep's tag moved after `wo.lock` was written, **when** the app
builds, **then** the lock wins and the drift is reported;
`woc --update-deps` refreshes it.
- **Given** a dep whose own `wo.toml` has `[deps]`, or a dep name colliding
with a local module, **when** the app builds, **then** each is a named
diagnostic, never silent misresolution.
## Out Of Scope
Registries, version ranges/semver solving, transitive dependencies (the
successor's first fork), private-remote auth handling beyond what ambient
`git` config provides, vendoring commands.
## Proposed Solution
Manifest parsing extends `compiler/bin/main.ml`'s existing wo.toml reader;
fetch = `Sys.command` over the `git` binary; resolution plugs the dep root
into the existing directory-as-module discovery. Fixtures under `tests/` use
local `file://` remotes so the suite stays network-free.

View file

@ -0,0 +1,67 @@
# Iteration 16 — the web framework: a `.wo` library, HTTP/1.1 behind a proxy
> Format: `product/story-iteration-template`. Part of
> [Story — one language, one runtime, one database, one binary](00-story.md).
>
> **Inserted 2026-08-18.** The framework is written IN writeonce and imported
> like any dependency (iteration 15 is the prerequisite). TLS terminates at a
> reverse proxy — browsers get TLS+ALPN+h2 from nginx/caddy while the
> framework speaks HTTP/1.1 keep-alive behind it, so no TLS exists anywhere
> in the toolchain. h2c is the parked successor (after iterations 8/9f/11,
> when multiplexing has a scheduler to pay off on).
>
> **Spec exists:** [`2026-08-18-web-framework-design.md`](../../superpowers/specs/2026-08-18-web-framework-design.md)
> sections B (normative) and C (the parked h2c successor).
## Goals
- **The framework**, incubated at `docs/examples/writeonce-framework/`: an
HTTP/1.1 keep-alive server core (extracted from the soak-proven log-watcher
`mcp` pattern), a method+path router with `:param` captures, `Req`/`Resp`
records with builder helpers, and the no-function-values handler model —
`Handler` / `Middleware` structural interfaces dispatched by ICALL, with
WO-E205 making a non-conforming handler a compile error and `?Resp`
middleware short-circuiting on the shipped `?T` narrowing.
- **The consuming app**, `docs/examples/web-app/`: a small storefront
(`Product`/`Order` as `@table` classes, list/show/create routes, one auth
middleware, JSON responses) that imports the framework **through
`[deps]`** — the sample exercises the whole chain: fetch → lock → build →
serve → durable data across restart.
- **Honest limits stated where users read them**: single-threaded blocking
(concurrency arrives underneath via iterations 8/11), no chunked encoding,
no WebSockets, JSON-first (no templates — the removed UI track stays
removed).
- **Relationship to iteration 10 recorded in both**: `service` blocks later
*lower onto this library* — compiler sugar over the same router, never a
rival stack.
## Acceptance Criteria
- **Given** the web-app sample, **when** `just web-app` runs, **then** deps
fetch, the app builds, serves list/show/create with `@table` persistence
across a process restart, answers 404/400 correctly, a deliberately
trapping handler returns 500 while the server survives, and SIGTERM stops
it cleanly — fd- and resident-flat under the opt-in soak.
- **Given** a handler class missing `handle`, **when** the app compiles,
**then** WO-E205 names the class and interface.
- **Given** nginx in front (documented config), **when** a browser hits it,
**then** the browser negotiates h2 with the proxy while the backend link is
HTTP/1.1 — proving the TLS/h2 story without TLS in writeonce.
- **Given** the framework extracted to its own repository, **when** the app's
`[deps]` URL is updated, **then** nothing else changes (success criterion 4
of the spec).
## Out Of Scope
TLS in the toolchain (proxy-terminated by decision); HTTP/2 + the
bytes/buffer type (parked to the h2c successor, after 8/9f/11); chunked
transfer encoding; WebSockets/SSE; templates/SSR; multipart uploads;
performance work beyond the soak's flatness gate (benchmarks belong to 9e's
measurement backbone).
## Proposed Solution
Framework modules `http/`, `router/`, `app.wo` as spec §B lays out; the
web-app acceptance script is the gate (curl matrix + restart + stop + soak);
`just web-app` wires fetch-build-serve-verify into one command. Plan follows
after this iteration is approved on the board.

View file

@ -0,0 +1,184 @@
# Web framework + dependency system — design spec
**Date:** 2026-08-18
**Status:** approved design, pre-implementation
**Scope:** (A) a minimal git-backed dependency system (`wo.toml [deps]` +
`wo.lock`), (B) a web framework written in writeonce as an importable `.wo`
library, HTTP/1.1 behind a TLS-terminating reverse proxy, and (C) the parked
HTTP/2 path. Three sub-projects; A and B are the fundable ones, C is a
recorded successor.
**Relates to:** iteration 10 (`service` blocks — this framework becomes their
lowering target, not a rival), iterations 8/9f/11 (the concurrency work h2c
waits for), `docs/plan/discarded.md` (FFI reject row — load-bearing here).
## Decisions locked during brainstorming
| Question | Decision |
| --- | --- |
| TLS | **Proxy-terminated** (nginx/caddy). Browsers get TLS+ALPN+h2 from the proxy; the framework speaks HTTP/1.1 (later h2c) behind it. Zero TLS in the language or runtime. Direct-serving TLS is a separate future iteration and would be judged against the FFI/libc-only doctrine then, not now. Homegrown TLS is refused outright: a decade of side-channel and certificate-validation subtleties makes it a security liability, not a milestone. |
| Dependencies | **`wo.toml [deps]` + git fetch.** A real (mini) package manager: exact-rev git dependencies, a lockfile, a per-project cache. No registry, no semver solving. |
| HTTP/2 | **v1 is HTTP/1.1 keep-alive.** h2's payoff is multiplexing, which a single blocking thread cannot exploit; h2c lands as its own iteration after shards (8) / io_uring (9f) / fibers (11). Behind the proxy, browsers see h2 from day one regardless. |
| Handler model | **Structural interfaces, not closures.** The language has no function values by doctrine; a route handler is a class satisfying a `Handler` interface, dispatched by ICALL — which works on today's runtime and is checked by WO-E205. |
| Incubation | Framework is born at `docs/examples/writeonce-framework/`; the consuming app at `docs/examples/web-app/` imports it **through the `[deps]` mechanism** (a local git URL), so the whole import chain is exercised by the sample. Extraction to `github.com/shoneyj/<name>` later is a `git subtree split`, not a redesign. |
Rejected: TLS in the runtime via rustls/OpenSSL (breaks libc-only for a
benefit the proxy already provides; revisit only if direct serving becomes a
requirement); nghttp2 binding (same doctrine cost, and the scheduler cannot
use multiplexing yet); vendor-directory imports (the user chose the real
dependency mechanism); native h2 in v1 (HPACK synchronization and
flow-control deadlocks are where the time goes — the user's own estimate).
## A. Dependency system (`wo.toml [deps]`, `wo.lock`)
**Manifest.** `wo.toml` gains one section:
```
[deps]
niceframework = { git = "https://github.com/shoneyj/niceframework", rev = "v0.1.0" }
```
`rev` is mandatory and exact (a tag or SHA). No version ranges, no registry,
no resolution algorithm — those are future work a corpus of real deps must
justify first.
**Fetch.** `woc` shells out to the `git` binary (`git clone --depth 1`,
`git checkout <rev>`) — no network code inside the compiler, OCaml-stdlib
doctrine intact; `git` joins `cc` in the "external tools the toolchain may
invoke" set. Dependencies land in `.wo-deps/<name>/` beside `wo.toml`
(gitignored). A missing `git` binary or unreachable remote is a plain
diagnostic, never a hang without a message.
**Lockfile.** `wo.lock` records `name -> resolved SHA` for every fetch.
Present lockfile wins over the manifest's rev label (a moved tag is detected
and reported, not silently followed). `woc --update-deps` refreshes the lock;
plain builds never touch the network when `.wo-deps` already satisfies the
lock.
**Resolution.** A dependency is an ordinary writeonce project (its own
`wo.toml` with `name`). `use <depname>` resolves the dep's root directory as
a module root, exactly like a project-internal module; `use <depname>/sub`
reaches its subdirectories. Collisions between a dep name and a local module
diagnose (the existing WO-E collision family).
**Transitive deps are refused in v1.** A fetched dep whose own `wo.toml`
carries `[deps]` is a diagnostic naming the dep — flat-only keeps the
resolver small and the failure honest. Recorded as the successor iteration's
first fork.
**Entry-point rule.** A dep's `fn main` (if any) is ignored — only the
consuming app owns the entry. `pub` visibility applies across the dep
boundary exactly as across modules.
## B. The framework (v1, HTTP/1.1, library-only)
**Repo shape.** `docs/examples/writeonce-framework/` is a complete writeonce
project (`wo.toml name = "niceframework"` — final name decided at extraction)
whose modules are the framework:
- `http/` — request parsing, response serialization, keep-alive loop.
Extracted from the proven log-watcher `mcp` pattern (601k requests soaked,
fd-clean, SIGTERM-clean).
- `router/` — method+path table with `:param` captures.
- `app.wo` — the assembly: register routes/middleware, `serve(port)`.
**Types (records).**
- `Req`: `method: Text`, `path: Text`, `params: map<Text, Text>` (the
`:param` captures), `query: map<Text, Text>`, `headers: map<Text, Text>`,
`body: Text`.
- `Resp`: `status: Int`, `headers: map<Text, Text>`, `body: Text`, plus
builder helpers (`ok_json`, `ok_text`, `not_found`, `redirect`, …).
**Handlers.** One structural interface:
```
interface Handler {
fn handle(req: Req) -> Resp
}
```
A route is any class satisfying it (state lives in the class's own fields —
the closure substitute). Registration passes the handler *value*:
`app.get("/products/:id", ProductShow { ... })`. Dispatch is ICALL;
WO-E205 makes a non-conforming handler a compile error.
**Middleware.** Its own one-method interface (distinct from `Handler`,
because the signatures differ):
```
interface Middleware {
fn before(req: Req) -> ?Resp
}
```
The app holds a `multi` of middleware run in order before routing (auth,
logging, content-type defaults). Returning a `Resp` short-circuits; nil means
"continue" — `?Resp` narrowing is exactly what the 2026-08-18 `?T`
enforcement shipped, used as a design tool.
**Server core.** `net.listen/accept` blocking loop, HTTP/1.1 with keep-alive
and `Content-Length` bodies (no chunked encoding in v1 — disclosed), fd
closed on every exit path, `env.stopping()` honored. **Single-threaded,
blocking, one request at a time** — stated in the framework README, with the
proxy config (nginx upstream keepalive) as the deployment story. Concurrency
arrives via iterations 8/11 underneath the same library surface.
**Database.** Nothing to build: handlers use `@table` + the query surface
directly. This is the differentiator — a durable, compiler-checked data layer
with no ORM and no separate database process, in the same binary.
**The consuming app.** `docs/examples/web-app/` — a small storefront:
`Product`/`Order` as `@table` classes, list/show/create routes, one auth
middleware, JSON responses. Its `wo.toml [deps]` points at the framework by
git URL (`file://` in CI, the GitHub URL after extraction). Its acceptance
script is the whole feature's gate: fetch deps, build, serve, curl the
routes, restart-persistence check, SIGTERM.
## C. HTTP/2 (h2c) — parked successor
After iterations 8 (shards) / 9f (io_uring) / 11 (fibers): h2c framing +
HPACK, either natively (needs a bytes/buffer type with cheap slicing — that
type rides with this iteration, not v1) or via nghttp2-in-runtime (a doctrine
decision to re-argue then, with the TweetNaCl precedent and the libc-only
rule both on the table). Behind the proxy, h2c's win is backend multiplexing;
browsers already had h2 since v1. No TLS obligation even here.
## Error handling
Dep failures (missing git, bad rev, dirty cache, transitive deps) are
compiler diagnostics with the WO-E1xx driver family. Framework runtime
failures follow house rules: malformed requests get a 400 and a closed
connection, handler traps are caught at the serve loop (`try`) and answered
with a 500 — a bad request must never kill the server; the soak asserts
resident + fd flatness exactly like log-watcher's.
## Testing
- Dep system: fixture projects under `tests/` — fetch-and-build from a local
git repo, lockfile drift detection, transitive-dep refusal, collision
diagnostic. Network-free (local `file://` remotes).
- Framework: unit-ish `.wo` fixtures for the router and parser; the web-app
acceptance script (curl matrix incl. keep-alive reuse, 404/400/500 paths,
restart persistence, stop) is the gate; `LW_SOAK`-style opt-in soak.
- The whole chain (deps fetch -> build -> serve) runs in `just web-app`.
## Success criteria
1. `docs/examples/web-app` builds by fetching `writeonce-framework` through
`[deps]` + `wo.lock` with no path references, and `woc` never touches the
network when the lock is satisfied.
2. The storefront serves list/show/create with `@table` persistence across a
restart, behind nginx with browser-visible h2 (proxy-terminated), while
the backend speaks HTTP/1.1.
3. A non-conforming handler class is WO-E205 at compile time; a trapping
handler answers 500 and the server survives (soak-proven).
4. Framework extraction to a standalone repo requires changing only the
app's `[deps]` URL.
## Out of scope (v1)
TLS anywhere in the toolchain; HTTP/2 and the bytes/buffer type (parked to
C); chunked transfer encoding; WebSockets/SSE (needs the push story);
transitive dependencies, version ranges, registries; templates/SSR (the
removed UI track is not resurrected here — JSON APIs first); multipart
uploads.