From 01df75245f9fa472ebf7420a85495d7912f9e213 Mon Sep 17 00:00:00 2001 From: "shoney.arickathil" Date: Wed, 26 Aug 2026 20:17:33 +0200 Subject: [PATCH] docs(porch): give the framework its own story track, iterations 1-8 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - docs/stories/porch/ — a TRACK folder, not a status folder: status still lives only in frontmatter. Adds `track: porch` so a query over docs/stories/ can tell a porch 3 from a language 3 - 00-story.md carries the sequence, the dependency graph, and a table of what the track explicitly does NOT own (binding -> 29, cache -> 18, proxy -> 38, metrics -> 30, TLS/templates -> doctrine) - eight iterations, each with phases, per-phase tasks, Given/When/Then criteria, out-of-scope and the forks a spec must settle: 1 store-backed middleware (limiter + idempotency — needs nothing new, first on purpose so the store pattern is proven cheaply) 2 randomness + cookies (phase A is language-track: a CSPRNG builtin; `Resp.headers` being a map cannot emit two Set-Cookie lines) 3 sessions 4 CSRF 5 routing/response ergonomics (independent) 6 streaming core (the seam 7 and 8 wait on; chunked-request refusal must survive) 7 SSE + compression 8 static + lifecycle hooks - language iteration 39 -> status: hold, retitled superseded, with a row mapping each of its goals to the porch iteration that took it. Kept, not deleted: the Fiber study cites it and its randomness argument is what this track is built on - board gains a porch section; board-views gains porch and both-track Dataview queries; porch README and the Fiber study §7 point at the track - no code blocks in any story (plans carry concept and actions in words); linkcheck 0 broken / 0 anchors Co-Authored-By: Claude Opus 5 (1M context) --- docs/examples/porch/README.md | 4 +- .../plan/exploration/fiber/00-fiber-parity.md | 11 +- docs/stories/00-status.md | 30 +++- docs/stories/board-views.md | 31 +++- .../39-web-framework-parity.md | 26 ++- docs/stories/porch/00-story.md | 88 +++++++++ .../porch/01-store-backed-middleware.md | 144 +++++++++++++++ .../porch/02-randomness-and-cookies.md | 170 ++++++++++++++++++ docs/stories/porch/03-sessions.md | 129 +++++++++++++ docs/stories/porch/04-csrf.md | 130 ++++++++++++++ .../porch/05-routing-response-ergonomics.md | 142 +++++++++++++++ docs/stories/porch/06-streaming-core.md | 155 ++++++++++++++++ docs/stories/porch/07-sse-and-compression.md | 144 +++++++++++++++ docs/stories/porch/08-static-and-lifecycle.md | 153 ++++++++++++++++ 14 files changed, 1347 insertions(+), 10 deletions(-) create mode 100644 docs/stories/porch/00-story.md create mode 100644 docs/stories/porch/01-store-backed-middleware.md create mode 100644 docs/stories/porch/02-randomness-and-cookies.md create mode 100644 docs/stories/porch/03-sessions.md create mode 100644 docs/stories/porch/04-csrf.md create mode 100644 docs/stories/porch/05-routing-response-ergonomics.md create mode 100644 docs/stories/porch/06-streaming-core.md create mode 100644 docs/stories/porch/07-sse-and-compression.md create mode 100644 docs/stories/porch/08-static-and-lifecycle.md diff --git a/docs/examples/porch/README.md b/docs/examples/porch/README.md index b824521..8fad15d 100644 --- a/docs/examples/porch/README.md +++ b/docs/examples/porch/README.md @@ -12,7 +12,9 @@ > handler body mentions it. A web framework **written in writeonce**, consumed as a `[deps]` dependency -(iteration 15). Spec: `docs/superpowers/specs/2026-08-18-web-framework-design.md` §B. +(iteration 15). Roadmap: [`docs/stories/porch/`](../../stories/porch/00-story.md) +— its own track, eight iterations, numbered from 1, derived from +[the Fiber parity study](../../plan/exploration/fiber/00-fiber-parity.md). Spec: `docs/superpowers/specs/2026-08-18-web-framework-design.md` §B. ```toml [deps] diff --git a/docs/plan/exploration/fiber/00-fiber-parity.md b/docs/plan/exploration/fiber/00-fiber-parity.md index 650782d..f37fc7f 100644 --- a/docs/plan/exploration/fiber/00-fiber-parity.md +++ b/docs/plan/exploration/fiber/00-fiber-parity.md @@ -190,13 +190,16 @@ Not gaps. Each was decided and the reasoning is on file. ## 7. What this study feeds -[**Iteration 39 — web framework parity**](../../../stories/language-runtime-database/39-web-framework-parity.md) -takes §0–§2 and the cheap half of §5, in that order, because §0 gates §2 and §1 -gates most of it. +The **[`porch` track](../../../stories/porch/00-story.md)** — eight iterations +numbered from 1, which superseded language iteration 39 on the day this study +was written. §0–§2 and the cheap half of §5 became porch 1–5, in dependency +order because §0 gates §2 and §1 gates most of it; §3's streaming seam and what +falls out of it became porch 6–8, so nothing in this study is now unscheduled +except what the table below hands to someone else. Explicitly *not* iteration 39's, with owners: -- streaming, SSE, compression, byte ranges (§3) — the parked streaming slice +- streaming, SSE, compression, byte ranges (§3) — [porch 6](../../../stories/porch/06-streaming-core.md)–[8](../../../stories/porch/08-static-and-lifecycle.md), which unparked them - typed binding (§4) — [iteration 29](../../../stories/language-runtime-database/29-compile-time-metaprogramming.md) - TTL cache middleware — [iteration 18](../../../stories/language-runtime-database/18-memory-db-features.md) - `proxy` — [iteration 38](../../../stories/language-runtime-database/38-content-platform-capabilities.md) diff --git a/docs/stories/00-status.md b/docs/stories/00-status.md index c326c8c..f5809f9 100644 --- a/docs/stories/00-status.md +++ b/docs/stories/00-status.md @@ -10,9 +10,14 @@ The single place to learn where this project stands. Organised in six buckets: folders** — a doc stays where it was authored, and only its frontmatter, its banner and this board change. +**Two tracks** (2026-08-26): [`language-runtime-database/`](language-runtime-database/00-story.md) +— the language, runtime and database — and [`porch/`](porch/00-story.md), the web +framework written in it. Each numbers its iterations from 1, so a porch 3 is not +a language 3; porch stories carry `track: porch` in frontmatter to keep queries +honest. Track folders are fine; **status** folders are not. + **Status lives in frontmatter, nowhere else** (directive 2026-08-26). Every -story iteration file sits flat in -[`language-runtime-database/`](language-runtime-database/00-story.md) and +story iteration file sits flat in its track folder and carries `status:` in its YAML header; the active slice's marker doc sits flat in `docs/`. **No directory anywhere encodes state.** This replaces the 2026-08-20/21 convention under which files moved between `done/`, `refine/`, @@ -594,6 +599,27 @@ precedence notes for resumption. **30** — observability, CI, fuzz: named 2026-08-20, still row-only (no story file); slots in when scheduled — nothing in the chain depends on it. +### ▸ porch — the web framework track + +New 2026-08-26, from [the Fiber v3.5.0 parity study](../plan/exploration/fiber/00-fiber-parity.md). +Supersedes language iteration 39, now a pointer. All eight are ⬜ `refine` — +none has an approved spec yet. Ordered by dependency; the first slice is +deliberately the cheapest so the store pattern and gate shape are proven before +the runtime and `Resp` are touched. + +| # | Iteration | State | +| --- | --- | --- | +| 1 | [Store-backed middleware](porch/01-store-backed-middleware.md) | ⬜ **startable today** — rate limiter + idempotency over a `@table`; needs no new primitive, only `time.ticks`. Durable counters are the differentiator over Fiber's in-memory default, so the gate includes a restart | +| 2 | [Randomness and cookies](porch/02-randomness-and-cookies.md) | ⬜ the foundation. Phase A is **language-track work**: a CSPRNG builtin (id 96+; 89/90 are iteration 31's reserved holes). Then repeated response headers — `Resp.headers` is a `map` and structurally cannot emit two `Set-Cookie` lines — then `Cookie:` parsing and signed cookies | +| 3 | [Sessions](porch/03-sessions.md) | ⬜ after 2. Server-side rows keyed by a random id, idle **and** absolute timeout, id rotation on login, revoke-all-for-principal, durable across restart | +| 4 | [CSRF](porch/04-csrf.md) | ⬜ after 2 + 3. Session-bound tokens, trusted origins as the second layer, opt-in single use, and refusal classes that are distinguishable in logs | +| 5 | [Routing + response ergonomics](porch/05-routing-response-ergonomics.md) | ⬜ **independent, any time** — `patch`/`options`/`head`/`all`, named routes + URL building, per-route body limit (today `BODY_MAX` is one compile-time number), request ids, `Location`/`Vary`/`Attachment`, and q-value ranking (retires a standing 🔶) | +| 6 | [Streaming core](porch/06-streaming-core.md) | ⬜ the riskiest and highest-leverage slice: incremental writes + chunked framing + an explicit commit point. `serialize()` always emits `Content-Length` today. Chunked REQUEST bodies are deliberately refused (request smuggling) and that refusal must survive | +| 7 | [SSE + compression](porch/07-sse-and-compression.md) | ⬜ after 6. SSE fits the actor/fiber model unusually well; compression carries a real fork — pure-`.wo` DEFLATE (now expressible after iteration 36's bit operators) vs a C builtin. CRC32 finally gets its consumer | +| 8 | [Static files + lifecycle](porch/08-static-and-lifecycle.md) | ⬜ static half after 6. Byte ranges, `Last-Modified`/`Cache-Control`, index resolution, listing off-by-default, shutdown hooks (the ledger's "no user teardown hooks yet"), plus healthcheck/favicon/redirect/rewrite/skip | + +--- + ⏸ **Held** (2026-08-21, developer decision): 18, 20, 21, 25, 26, 27, 28, 29 — every story carrying `status: hold` in its frontmatter (25's story file removed; its diff --git a/docs/stories/board-views.md b/docs/stories/board-views.md index 1d8d166..0a3369f 100644 --- a/docs/stories/board-views.md +++ b/docs/stories/board-views.md @@ -5,7 +5,8 @@ is the source of truth**: ```yaml --- -iteration: "8" # immutable id (string: "7b", "9b" exist) +track: porch # OMITTED on language-runtime-database stories +iteration: "8" # immutable id, LOCAL TO ITS TRACK (string: "7b", "9b" exist) status: in-progress # done | in-progress | refine | hold — the ONLY place status lives chain: 1 # concurrency-chain position, chain stories only (1–6) --- @@ -27,6 +28,13 @@ standup narrative; these queries are the live views over the same facts. Adjust the `FROM` path to your vault root (queries below assume the vault opens at the repo root). +Two tracks now carry iterations, each numbered from 1: +`language-runtime-database/` (the language, runtime and database) and `porch/` +(the web framework, added 2026-08-26). Iteration ids therefore repeat across +tracks — a porch 3 is not a language 3 — so every query below is scoped by +`FROM` path, and porch stories carry `track: porch` so a combined query can +still tell them apart. + ## Everything not done, chain order first ```dataview @@ -70,3 +78,24 @@ second copy of status. To keep frontmatter the single source of truth: the place status is edited.** A status change is one edit to one `status:` key; a Kanban card drag that only rewrites the Kanban file is a lie the next query won't see. + +## The porch track + +```dataview +TABLE iteration, status +FROM "docs/stories/porch" +WHERE status != "done" +SORT iteration ASC +``` + +## Both tracks at once, grouped + +Relies on `track:` being present on porch stories and absent on language ones, +so the language track shows up under an empty group. + +```dataview +TABLE rows.file.link AS story, rows.iteration AS iteration, rows.status AS status +FROM "docs/stories" +WHERE iteration AND status != "done" +GROUP BY track +``` diff --git a/docs/stories/language-runtime-database/39-web-framework-parity.md b/docs/stories/language-runtime-database/39-web-framework-parity.md index 4013c89..f4c2396 100644 --- a/docs/stories/language-runtime-database/39-web-framework-parity.md +++ b/docs/stories/language-runtime-database/39-web-framework-parity.md @@ -1,9 +1,31 @@ --- iteration: "39" -status: refine +status: hold --- -# Iteration 39 — web framework parity: randomness, cookies, and the store-backed middleware chain +# Iteration 39 — web framework parity *(superseded by the porch track)* + +> **⏸ SUPERSEDED 2026-08-26, the same day it was written.** Framework work now +> lives in its own track: [`docs/stories/porch/`](../porch/00-story.md), numbered +> from 1. This iteration's content was split across **porch 1–5** and is not +> planned from here — the sequencing below survives, but as that track's +> dependency order. +> +> | This iteration's goal | Now | +> | --- | --- | +> | limiter + idempotency (the cheap first slice) | [porch 1](../porch/01-store-backed-middleware.md) | +> | random-bytes builtin, cookies, `Resp` repeated headers | [porch 2](../porch/02-randomness-and-cookies.md) | +> | sessions | [porch 3](../porch/03-sessions.md) | +> | CSRF | [porch 4](../porch/04-csrf.md) | +> | method helpers, named routes, body limit, request id, response helpers | [porch 5](../porch/05-routing-response-ergonomics.md) | +> | *(deferred here, now scheduled)* streaming, SSE, compression, byte ranges | [porch 6](../porch/06-streaming-core.md)–[8](../porch/08-static-and-lifecycle.md) | +> +> Kept rather than deleted because the [Fiber study](../../plan/exploration/fiber/00-fiber-parity.md) +> cites it and because the reasoning below — especially why the randomness +> blocker comes first — is what the porch track is built on. Original text +> follows. + +## Original scope > Format: `product/story-iteration-template`. Part of > [Story — one language, one runtime, one database, one binary](00-story.md). diff --git a/docs/stories/porch/00-story.md b/docs/stories/porch/00-story.md new file mode 100644 index 0000000..5e7a4d6 --- /dev/null +++ b/docs/stories/porch/00-story.md @@ -0,0 +1,88 @@ +# Story — `porch`, the writeonce web framework + +The second track. Where +[`language-runtime-database/`](../language-runtime-database/00-story.md) grows +the *language*, this track grows the one library written **in** it: +[`porch`](../../examples/porch/README.md), consumed by every serving sample +through `wo.toml [deps]`. + +Numbering restarts at 1 and is local to this track. Frontmatter carries +`track: porch` so a query over `docs/stories/` can tell a porch iteration 3 from +a language iteration 3. Status rules are the repo's, unchanged: `status:` in +frontmatter is the only place state lives, no directory encodes it. + +## Why a separate track + +Three reasons, all practical: + +1. **Different substrate, different gates.** porch is `.wo` source. Its + iterations are proven by `just web-app` and `just site`, never by the + conformance corpus or `oop-accept`. Mixing them into the language track's + sequence made both harder to read. +2. **Different cadence.** A porch slice is days; a language slice that touches + `wob.h` and the VM is longer and riskier. Interleaving them in one numbering + forced false ordering decisions. +3. **The framework is now the product surface.** `writeonce.de` is served by + porch. Its gaps are what a visitor hits first, so they deserve a roadmap that + is not buried behind runtime work. + +The language track stays upstream: when a porch iteration needs a new builtin, +that half is called out explicitly and the language track owns it. + +## Where the sequence came from + +The [Fiber v3.5.0 parity study](../../plan/exploration/fiber/00-fiber-parity.md) +— gofiber/fiber read end to end against porch's actual `.wo` source: its routing +surface, `Req`/`Res` API, binder, lifecycle hooks and the `Config` of all 32 of +its `middleware/` packages. Nine of those 32 already have a working porch +counterpart, so this is a breadth roadmap, not a rescue. + +That study replaced language-track +[iteration 39](../language-runtime-database/39-web-framework-parity.md), which is +now a pointer here. + +## The sequence + +Ordered by dependency, not by importance — and the first slice is deliberately +the *cheapest*, so the store pattern and the gate shape are proven before the +risky work starts. + +| # | Iteration | Delivers | Needs | +| --- | --- | --- | --- | +| 1 | [Store-backed middleware](01-store-backed-middleware.md) | rate limiting + idempotency over a `@table` store | nothing new — starts today | +| 2 | [Randomness and cookies](02-randomness-and-cookies.md) | a `random_bytes` runtime builtin, repeated response headers, `Cookie:` parsing, signed cookies | a language-track builtin (phase A) | +| 3 | [Sessions](03-sessions.md) | server-side sessions, idle + absolute timeout, revocation | 2 | +| 4 | [CSRF](04-csrf.md) | token mint/verify, trusted origins, single-use tokens | 2, 3 | +| 5 | [Routing and response ergonomics](05-routing-response-ergonomics.md) | the remaining method helpers, named routes, per-route body limit, request ids, the missing response helpers | nothing — parallel to 2–4 | +| 6 | [Streaming core](06-streaming-core.md) | incremental response writes and chunked framing — the seam three iterations wait on | nothing new, but it changes `Resp` | +| 7 | [SSE and compression](07-sse-and-compression.md) | server-sent events, gzip/deflate | 6 | +| 8 | [Static files and lifecycle](08-static-and-lifecycle.md) | byte ranges, cache headers, directory listing, lifecycle hooks, the small middleware everyone ships | 6 | + +``` +1 ─ independent, start here +5 ─ independent, any time +2 ──▶ 3 ──▶ 4 +6 ──▶ 7 + └──▶ 8 +``` + +## What this track does NOT own + +| Not porch's | Owner | +| --- | --- | +| typed binding of query/params/form into a class | language: [`@derive`](../language-runtime-database/29-compile-time-metaprogramming.md) — reflection is forbidden by principle 13 | +| TTL cache, `transaction { }`, durable job queue | language: [iteration 18](../language-runtime-database/18-memory-db-features.md) | +| a `proxy` middleware | language: [iteration 38](../language-runtime-database/38-content-platform-capabilities.md) — needs `net.connect`, which does not exist | +| metrics, profiling, per-change CI, fuzzing | language iteration 30 (no story file yet) | +| TLS, HTTP/2 | nobody — proxy-terminated by doctrine | +| a runtime template engine | nobody — rejected; markup is a compile-time literal (`writeonce-view`) | +| a radix-tree router | nobody yet — waiting on a *measurement*, not a decision | + +## Review protocol + +Same as the language track: the developer reads one iteration, approves or +amends; the next starts only after approval. Each iteration is an unsplittable +value slice with phases, per-phase tasks, Given/When/Then acceptance criteria, +and an out-of-scope list. Every phase ends with both serving gates green — +`just web-app` and `just site` — because porch has two consumers and a change +that only satisfies one is not done. diff --git a/docs/stories/porch/01-store-backed-middleware.md b/docs/stories/porch/01-store-backed-middleware.md new file mode 100644 index 0000000..c743a31 --- /dev/null +++ b/docs/stories/porch/01-store-backed-middleware.md @@ -0,0 +1,144 @@ +--- +track: porch +iteration: "1" +status: refine +--- + +# porch 1 — store-backed middleware: rate limiting and idempotency + +> Part of [Story — `porch`, the writeonce web framework](00-story.md). +> Source: [the Fiber parity study](../../plan/exploration/fiber/00-fiber-parity.md) §2. +> +> **First deliberately because it is the cheapest.** Both features need only a +> `@table` and `time.ticks`, both of which already exist — no new builtin, no +> cookie, no change to `Resp`. It exists to prove the store pattern and the gate +> shape on low-risk work before iterations 2–4 touch the runtime and the public +> response type. + +## Goals + +- **A rate limiter that survives a restart.** Fixed-window counting keyed by + client, answering 429 with the conventional headers when the window is spent. + Fiber's `limiter` keeps counters in memory by default and expects Redis for + anything real; porch's live in a `@table`, so they are WAL-durable and + crash-recoverable for free. That is the difference worth demonstrating, and it + is why the acceptance criteria include a restart. +- **Idempotent replay of unsafe requests.** A client resending a POST with the + same idempotency key gets the stored response and the handler does not run + twice. This is the correctness feature the storefront sample has silently + needed since it grew a checkout. +- **Establish the store convention for iterations 2–4.** Sessions and CSRF will + want the same shape. Decide it once, here, on the cheap slice. + +## Phases + +### Phase A — the store convention + +- Decide the store shape (see Info fork 1) and write it down before any + middleware exists, because three later iterations inherit it. +- Add the `@table` classes for a counter row and a stored-response row, with + the secondary indexes their lookups need — the read path is an equality + probe, which is the only index shape the engine has. +- Decide and document the expiry discipline: rows are pruned lazily on access, + not by a background sweeper, because porch has no timer and iteration 30 owns + scheduled work. +- Verify: `woc docs/examples/porch/` typechecks entry-less as a library; the + new tables appear in the WAL and replay across a restart. + +### Phase B — the rate limiter + +- A `Limiter` middleware class with a `before` that counts and either passes or + short-circuits with 429 — the `?Resp` short-circuit the chain already has. +- Key selection: reuse `client_ip(req)` and the existing trusted-proxy + judgement rather than inventing a second one. A keyed-by-principal variant + falls out for free once `req.principal` is set by an auth middleware. +- The response headers on both paths, and a `Retry-After` on the refusal. +- Window arithmetic on `time.ticks` (µs monotonic), not `time.now` — a + wall-clock jump must not hand out a free window. +- Verify: a burst crosses the threshold at exactly N, the window rolls, the + counters survive `SIGTERM` + restart. + +### Phase C — idempotency + +- An `Idempotent` middleware pair: `before` looks the key up and replays a hit; + `after` stores the response for a miss. This is the first real user of the + `after` chain for something other than headers, which is worth noting. +- Decide what is part of the identity: the key header alone, or key plus a + digest of method+path+body (`sha256` exists). Replaying a stored response for + a *different* body under a reused key is the failure mode that matters. +- In-flight collision handling: a second request arriving while the first is + still running. Fiber takes a lock; porch's shard model means the honest + answer is probably to refuse with 409 rather than to block. +- Verify: replay returns the stored response, the handler's side effect happens + exactly once, a reused key with a different body is refused, concurrent + duplicates do not both execute. + +### Phase D — the gate and the ledger + +- Extend `scripts/web-app-accept.sh` with the checks above, including the + restart leg — a durability claim that no gate exercises is not a claim. +- Update porch's README ledger rows for both features, and record in the + status board what landed versus what was planned. +- Verify: `just web-app` and `just site` both green; `just linkcheck` clean. + +## Acceptance Criteria + +- **Given** a limiter of N requests per window, **when** a client sends N+1, + **then** the first N succeed and the last is 429 with `Retry-After` set. +- **Given** counters at their limit, **when** the process is SIGTERMed and + restarted, **then** the client is still limited — the counters replayed from + the WAL rather than resetting to zero. +- **Given** a window that has fully elapsed, **when** the same client returns, + **then** it is served, and the expired row is pruned on that access. +- **Given** the system clock jumping backwards, **when** the window is + evaluated, **then** no extra allowance is granted (`time.ticks` is monotonic). +- **Given** a POST with an idempotency key that has been seen, **when** it is + replayed, **then** the stored response is returned byte-identically and the + handler's side effect count is unchanged — proven by a row count, not by a + log line. +- **Given** a reused idempotency key with a different request body, **when** it + arrives, **then** it is refused rather than answered with the other request's + response. +- **Given** two identical keyed requests in flight at once, **when** both are + dispatched, **then** exactly one executes and the other gets the decided + answer (replay or 409), never a partial write. + +## Out Of Scope + +- **A pluggable `Storage` interface.** Fiber abstracts it so one middleware runs + on memory or Redis. porch has one store, and interfaces here are structural — + an abstraction with exactly one implementor is decoration, which iteration 37 + learned the hard way about `Component`. Concrete `@table` until a second + backend actually exists. +- **Sliding-window or token-bucket algorithms.** Fixed window is what the + sample needs; a better algorithm is a later slice with a measurement behind + it. +- **Distributed limiting across processes.** One program owns its database; + cross-program state is language + [iteration 20](../language-runtime-database/20-cross-program-tables.md). +- **A background expiry sweeper.** No timer exists (`time.after` is still a + reserved builtin id in `wob.h`). Lazy pruning on access, deliberately. +- **The TTL cache middleware** — language + [iteration 18](../language-runtime-database/18-memory-db-features.md) owns it, + spec already approved. Do not build a second cache here. + +## Info + +Forks the spec must settle: + +1. **One store or two?** A single generic key/value/expiry table serving both + features, or a purpose-shaped table each. Leaning two: the columns genuinely + differ (a counter is an Int, a stored response is status + headers + body), + and a generic table would force everything through `Text`, which is how the + framework's cache ended up storing JSON strings. +2. **What is the limiter's key when there is no auth?** `client_ip(req)` reads + `X-Forwarded-For`, which a direct client can forge. Behind the mandated TLS + proxy that is fine; on an open port it is not. `net.peer(fd)` gives the real + peer — decide which is authoritative and reuse whatever the trusted-proxy + slice concludes rather than deciding twice. +3. **Does idempotency store headers?** Fiber has `KeepResponseHeaders` because + replaying `Set-Cookie` or a fresh `Date` is usually wrong. porch has no + cookies yet (iteration 2), so this is cheap to decide now and expensive to + retrofit later. + +Nothing here needs a new runtime primitive, which is the point of going first. diff --git a/docs/stories/porch/02-randomness-and-cookies.md b/docs/stories/porch/02-randomness-and-cookies.md new file mode 100644 index 0000000..be0dac8 --- /dev/null +++ b/docs/stories/porch/02-randomness-and-cookies.md @@ -0,0 +1,170 @@ +--- +track: porch +iteration: "2" +status: refine +--- + +# porch 2 — randomness and cookies: the foundation three iterations stand on + +> Part of [Story — `porch`, the writeonce web framework](00-story.md). +> Source: [the Fiber parity study](../../plan/exploration/fiber/00-fiber-parity.md) §0–§1. +> +> **The study's sharpest finding lives here.** porch's own README claimed signed +> cookies, CSRF and session integrity were "UNBLOCKED — the primitives exist +> since iteration 34". For CSRF and sessions that is **wrong**: SHA-256 and HMAC +> let a program *authenticate* a token, they cannot *mint* one, and writeonce has +> no source of randomness anywhere. An HMAC over a guessable session id is a +> signed guess. Phase A closes that before anything depends on it. + +## Goals + +- **A CSPRNG builtin in the runtime.** No `getrandom`, no `/dev/urandom` read, + no CSPRNG builtin exists today — grep the runtime and nothing comes back. This + is the one phase of this track that is **language-track work** (C, a new + builtin id, a corpus fixture); it is here because porch is what needs it and + splitting it across two tracks would hide the dependency. +- **Repeated response headers, which `Resp` structurally cannot express.** + `Resp.headers` is a `map`. A login response setting a session + cookie *and* a flash cookie needs two `Set-Cookie` lines and the map can hold + one. This is a change to porch's most public type, and deciding its shape is + the real work of this iteration — the cookie formatting is the easy half. +- **Cookies in both directions.** Parse a `Cookie:` request header into + something typed; build a `Set-Cookie` with the attributes that matter for + security — `HttpOnly`, `Secure`, `SameSite`, `Max-Age`, `Path`, `Domain`. +- **Signed cookies.** HMAC-SHA256 over the value with constant-time comparison + on the way back (`hmac_sha256` and `ct_eq` both already exist). This is the + half that genuinely *was* unblocked by iteration 34, and it is what makes a + cookie tamper-evident without a server-side lookup. +- **Correct the ledger row that started this.** The README's crypto line, and + anything else claiming the sessions/CSRF path was already open. + +## Phases + +### Phase A — the runtime primitive (language-track work) + +- Add a random-bytes builtin at the next free id (89 and 90 are reserved holes + for language iteration 31's `monitor` and `time.after`, so this starts at 96). +- Source it from the kernel. Decide the failure mode explicitly: if the source + is unavailable the builtin **refuses**, loudly. A CSPRNG that quietly degrades + to something weaker is worse than none, because every layer above it will + assume it worked. +- Corpus fixtures: the builtin's arity and type contract, and the refusal path. + Statistical quality is not a corpus concern — the kernel's guarantee is the + guarantee. +- Document it in the builtin-surface contract and the error catalog if it adds a + diagnostic, in the same change. Both of those went stale once before by not + doing this. +- Verify: `oop-e2e` green; the value differs across separate processes and is + not derivable from the clock. + +### Phase B — repeated response headers + +- Settle fork 1 and change `Resp`. Every response builder + (`ok_text`/`ok_json`/`ok_html`/`created_json`/`not_found`/…) and + `serialize()` in `internal/serve.wo` move with it. +- Keep the single-value path ergonomic: the overwhelmingly common case is one + value per header, and it must not get worse to write. +- Verify: `serialize()` emits two distinct `Set-Cookie` lines for one response; + every existing header behaviour is byte-identical, proven by both serving + gates passing unchanged. + +### Phase C — reading request cookies + +- Parse the `Cookie:` header: multiple pairs, quoted values, stray whitespace, + and duplicate names. +- Decide where parsed cookies live — a lazily-parsed field on `Req`, or a + helper the handler calls. `Req` already carries a `ctx` bag and a `params` + map, so the precedent exists either way. +- A malformed header is a 400, not a silent partial parse. The parser's + existing discipline around duplicate `Content-Length` is the model. +- Verify: values recovered exactly across the awkward cases above; malformed + input refused. + +### Phase D — writing and signing cookies + +- A `Set-Cookie` builder covering the attribute set, with `HttpOnly` and + `SameSite` defaulted to the safe choice rather than the permissive one — a + cookie API whose defaults are insecure is a footgun that ships. +- Sign with `hmac_sha256`, verify with `ct_eq`. Decide the encoding (`base64` is + already available) and the payload framing so a value containing the delimiter + cannot forge a signature. +- Clear-cookie support, which is its own case: the attributes must match or the + browser keeps the old one. +- Verify: a one-bit change to value or signature is rejected; a valid cookie + round-trips; a cleared cookie is actually gone. + +### Phase E — prove it and correct the record + +- A login/logout flow in a sample exercising two cookies on one response, using + the signed-cookie path only — no sessions yet, that is iteration 3. +- Correct porch's README crypto row and re-point the ledger rows this iteration + touched. +- Verify: `just web-app`, `just site`, `oop-accept`, `just linkcheck` all green. + +## Acceptance Criteria + +- **Given** the random builtin, **when** many values are drawn across separate + processes, **then** none repeats and none is derivable from the clock; **and** + when the kernel source is unavailable, the builtin refuses loudly rather than + returning weak bytes. +- **Given** a handler setting a session cookie and a flash cookie on one + response, **when** it is serialized, **then** **two** distinct `Set-Cookie` + headers reach the wire. This is the criterion today's `map` + provably cannot satisfy. +- **Given** every pre-existing response shape, **when** both serving gates run, + **then** output is byte-identical to before phase B — the `Resp` change is + additive or it is wrong. +- **Given** a `Cookie:` header with several pairs, quoted values and stray + whitespace, **when** it is parsed, **then** each value is recovered exactly; + **and** a malformed header yields 400, never a partial parse. +- **Given** a signed cookie altered by one bit in either the value or the + signature, **when** it is verified, **then** it is rejected in constant time. +- **Given** a signed value that itself contains the payload delimiter, **when** + it round-trips, **then** it cannot be re-framed to forge a valid signature. +- **Given** cookie defaults, **when** a cookie is created without explicit + attributes, **then** `HttpOnly` is on and `SameSite` is not `None`. + +## Out Of Scope + +- **Encrypted cookies.** Fiber's `encryptcookie` needs a symmetric cipher, and + the runtime has digests only. Signed-and-readable is honest and sufficient for + a session id; encrypting a payload is a separate ask with a separate primitive + behind it. +- **Server-side session state** — iteration [3](03-sessions.md). This iteration + stops at a signed cookie carrying a value the app chose. +- **CSRF** — iteration [4](04-csrf.md), which needs both this and 3. +- **JWT.** Verification is already possible with `hmac_sha256`; issuing needs + phase A. Either way it is a library slice with a hard stop at HS256 — no + RS256, no JOSE — and not this iteration. +- **Cookie-based *cache* keys.** Fiber's cache middleware has `KeyCookies`; the + cache belongs to language + [iteration 18](../language-runtime-database/18-memory-db-features.md). + +## Info + +Forks the spec must settle, in order of how much they move: + +1. **What replaces `Resp.headers: map`?** Three candidate shapes: a + `multi Text` of raw extra header lines beside the existing map; a dedicated + typed `cookies` field on `Resp` that `serialize()` renders; or a general + repeated-header list replacing the map. The first is the smallest change, the + second the most typed, the third the most honest about HTTP — and the third + touches every builder and both consumers. This fork decides the size of the + whole iteration, so settle it first. +2. **What shape is the builtin?** A bytes-returning primitive composes with + everything iterations 19 and 34 added (`Bytes`, `base64_encode`, + `hmac_sha256`), which argues for exactly one function and no convenience + wrappers. Decide whether a hex/id helper rides along or whether + `base64_encode` is enough. +3. **Where do parsed cookies live?** A field on `Req` parsed eagerly costs every + request that has no cookies; a helper parsed on demand costs nothing but is + easy to call twice. `Req` is a `typedef` record, so adding a field is cheap — + the cost is the eager parse, not the shape. +4. **Signed-cookie framing.** Value-then-signature with a delimiter is the + obvious encoding and the obvious place to get it wrong. Decide the framing so + that a value containing the delimiter cannot shift the boundary. + +Phase A is the only part of this track that is language-track work. It is +sequenced here rather than filed as a language iteration because nothing else +wants it yet, and a primitive with no consumer is how the `Component` interface +became decoration. diff --git a/docs/stories/porch/03-sessions.md b/docs/stories/porch/03-sessions.md new file mode 100644 index 0000000..59d4b61 --- /dev/null +++ b/docs/stories/porch/03-sessions.md @@ -0,0 +1,129 @@ +--- +track: porch +iteration: "3" +status: refine +--- + +# porch 3 — sessions: server-side state, revocable, durable + +> Part of [Story — `porch`, the writeonce web framework](00-story.md). +> Source: [the Fiber parity study](../../plan/exploration/fiber/00-fiber-parity.md) §2. +> Needs [iteration 2](02-randomness-and-cookies.md) (a random session id and a +> signed cookie to carry it) and inherits the store convention from +> [iteration 1](01-store-backed-middleware.md). + +## Goals + +- **A session is a `@table` row keyed by a random id; the cookie carries only + the id.** The alternative — a signed cookie carrying the whole payload — needs + no store but cannot be revoked, and revocation is not optional for a real + login. This is the fork iteration 1's store convention exists to answer. +- **Both timeouts, because they answer different questions.** Idle timeout + bounds "how long since you did anything"; absolute timeout bounds "how long + since you authenticated". Fiber ships both (`IdleTimeout`, `AbsoluteTimeout`) + and a session with only the first never expires for an active attacker. +- **Durability is the differentiator.** Fiber's default store is in-memory: a + restart logs everyone out. porch's sessions ride the WAL, so they survive — + and the gate proves it, because an unexercised durability claim is not a + claim. +- **Revocation that works.** Logout, and "log out everywhere for this + principal" — the second is what a password change needs, and it is nearly free + once sessions are rows with an owner column. + +## Phases + +### Phase A — the session table and its lifecycle + +- The `@table` class: random id, principal, created-at, last-seen, and whatever + payload shape fork 1 settles. Secondary index on principal, because + "revoke all for this user" is an equality probe and that is the only index + shape the engine has. +- Create, touch, expire and delete, with expiry evaluated on access (the lazy + discipline iteration 1 established — there is still no timer). +- Verify: rows replay across a restart; an expired row is pruned when touched. + +### Phase B — the middleware + +- A `Session` middleware whose `before` reads the signed cookie, loads the row, + checks both timeouts, and attaches identity. Decide whether it sets + `req.principal` or writes into `req.ctx` — `principal` is the field the + framework already means for identity, so it should be that unless the session + carries more than identity. +- Rotate the id on privilege change (login especially). Session fixation is the + attack that a login which reuses the pre-login id walks straight into. +- A missing, expired, unknown or tampered cookie all end at the same place — + unauthenticated — but for distinguishable reasons, because debugging a login + loop without that distinction is miserable. +- Verify: each of those four cases behaves; the id changes across login. + +### Phase C — login, logout, revoke-all + +- The three flows end to end in a sample, using porch's existing auth pieces + (`bearer_token`, `basic_credentials`, `ct_eq`) for the credential check — + sessions are about *keeping* identity, not establishing it. +- Logout deletes the row and clears the cookie with matching attributes. +- Revoke-all deletes by principal and is proven with two concurrent sessions. +- Verify: after logout the old cookie is inert even though it is still + well-signed — the point of server-side state. + +### Phase D — the gate and the ledger + +- Extend `scripts/site-accept.sh` rather than only the storefront: the site has + an admin route that currently checks a bearer token per request, which is + exactly the thing sessions replace. +- Ledger rows, status board entry, and the standup questions answered. +- Verify: `just web-app`, `just site`, `just linkcheck` green. + +## Acceptance Criteria + +- **Given** a valid session cookie, **when** a request arrives, **then** + identity is attached and the row's last-seen advances. +- **Given** a session whose signature is valid but whose row was deleted, + **when** it is presented, **then** the request is unauthenticated — a + well-formed cookie is not authority. +- **Given** an idle timeout of T, **when** a session is unused for longer than + T, **then** it is rejected and its row pruned on that access. +- **Given** an absolute timeout of A, **when** a session is *continuously + active* past A, **then** it is still rejected — the criterion an idle-only + implementation fails. +- **Given** an anonymous session id, **when** the user logs in, **then** the id + is rotated and the pre-login id is inert. +- **Given** active sessions and a SIGTERM plus restart, **when** the same + cookies return, **then** the users are still logged in, replayed from the WAL. +- **Given** two sessions for one principal, **when** revoke-all runs, **then** + both are inert and other principals are untouched. +- **Given** logout, **when** the cleared cookie is compared to the one that set + it, **then** the attributes match, so the browser actually drops it. + +## Out Of Scope + +- **Flash messages and a general session bag.** A typed session with a known + shape first; an untyped `map` payload is a decision to defer, not + a default to adopt. +- **OAuth, OIDC, SSO, "log in with X".** Every one needs an outbound socket, + which does not exist — language + [iteration 38](../language-runtime-database/38-content-platform-capabilities.md). +- **Remember-me tokens** — a second, longer-lived credential class with its own + rotation story. Its own slice. +- **CSRF** — iteration [4](04-csrf.md). Sessions make CSRF *possible* to do + properly; they do not provide it. +- **Sliding-window renewal of the absolute timeout.** That is not what absolute + means. + +## Info + +Forks the spec must settle: + +1. **What does the session row carry beyond identity?** Just principal, or a + payload? A payload wants a shape, and without generics the shape is either a + declared class per app or `Text`. The framework's cache already stores `Text` + and that is the compromise this repo dislikes most — so leaning: identity + plus declared columns, and apps that want more keep their own table keyed by + session id. +2. **Idle-timeout writes on every request.** Touching last-seen means a WAL + write per request, which turns every read into a durable write. Options: + write at a coarser granularity, or accept the cost and say so with a number + from `just db-bench`. This is a real performance fork, not a detail. +3. **Does `Session` set `req.principal` or `req.ctx`?** `principal` is the + framework's declared home for identity and auth middleware already writes it, + so two writers of one field need a documented precedence. diff --git a/docs/stories/porch/04-csrf.md b/docs/stories/porch/04-csrf.md new file mode 100644 index 0000000..d7f19ae --- /dev/null +++ b/docs/stories/porch/04-csrf.md @@ -0,0 +1,130 @@ +--- +track: porch +iteration: "4" +status: refine +--- + +# porch 4 — CSRF: tokens that are unguessable, bound, and spendable once + +> Part of [Story — `porch`, the writeonce web framework](00-story.md). +> Source: [the Fiber parity study](../../plan/exploration/fiber/00-fiber-parity.md) §2. +> Needs [2](02-randomness-and-cookies.md) for randomness and cookies, and +> [3](03-sessions.md) for something to bind a token to. + +## Goals + +- **Tokens that cannot be guessed or forged.** Minted from the iteration-2 + builtin, verified with `ct_eq`. This is the iteration that could not have been + written honestly before phase A of 2 existed, which is the whole reason the + track is ordered this way. +- **Bound to a session, not floating.** An unbound token is a token an attacker + can fetch for themselves and replay against a victim. Binding is what makes + the defence real, and it is why this follows sessions rather than preceding + them. +- **Origin checking as the cheap second layer.** Fiber ships `TrustedOrigins` + alongside the token. Same-site cookies plus an origin check stop most of what + tokens stop, for almost no cost — and the two layers fail differently, which + is the argument for having both. +- **Single-use where it matters.** Fiber's `SingleUseToken` exists because a + long-lived token in a browser history or a referrer header is a credential + left lying around. Decide which routes get it rather than making everything + pay. +- **Refusals that are distinguishable.** Missing, stale, foreign-origin and + already-spent must be told apart in the logs, or nobody can debug a form that + stopped working. + +## Phases + +### Phase A — mint and verify + +- Token generation, storage keyed by session, and constant-time verification. +- Decide the transport: a dedicated cookie plus a form field (double-submit), or + session-stored plus a form field. The second needs no second cookie and is the + stronger of the two once sessions exist. +- Extraction from where forms and fetch clients actually put it: a form field, a + header, and decide whether a query parameter is ever allowed (it should not be + — it leaks into logs and referrers). +- Verify: a valid token passes; altered, absent and foreign tokens each fail + distinctly. + +### Phase B — the middleware and safe-method policy + +- A `Csrf` middleware gating unsafe methods only. `GET`/`HEAD`/`OPTIONS` must + pass untouched or every link on the site breaks. +- Origin and `Referer` checking against a configured trusted set, including the + awkward cases: absent origin, `null` origin, and a same-origin request that + arrives without the header. +- Failure is a distinct status with a body that does not leak whether the token + was wrong or merely stale. +- Verify: the site's admin edit flow works through the middleware; unsafe + methods without a token are refused; safe methods are unaffected. + +### Phase C — single use and rotation + +- Mark-spent-on-use for the routes that opt in, and decide what happens to a + double-submitted form (the user double-clicking is not an attack, and treating + it as one is a support ticket). +- Rotate on privilege change, matching the session-id rotation from iteration 3. +- Verify: a spent token is refused; a double-click produces a comprehensible + outcome rather than a raw 403. + +### Phase D — the gate and the ledger + +- Both serving gates: the site's admin edit is the natural CSRF subject, the + storefront's checkout the natural single-use subject. +- Ledger and status board. +- Verify: `just web-app`, `just site`, `just linkcheck` green. + +## Acceptance Criteria + +- **Given** an unsafe request with no token, **when** it is dispatched, **then** + it is refused and the handler never runs. +- **Given** a token minted for session A, **when** it is presented with session + B's cookie, **then** it is refused — binding is enforced, not decorative. +- **Given** a token altered by one bit, **when** it is verified, **then** it is + refused in constant time. +- **Given** a request from an untrusted origin carrying an otherwise valid + token, **when** it arrives, **then** it is refused — the layers are + independent. +- **Given** a safe method (`GET`, `HEAD`, `OPTIONS`), **when** it arrives with + no token, **then** it passes untouched. +- **Given** a single-use token, **when** it is submitted twice, **then** the + second attempt is refused and the outcome is distinguishable from a forged + token in the logs. +- **Given** login, **when** the session id rotates, **then** the outstanding + token for the old session is no longer valid. +- **Given** each refusal class, **when** the logs are read, **then** missing, + stale, foreign-origin and spent are told apart — while the response body + tells the client none of it. + +## Out Of Scope + +- **CORS.** Already shipped (`Cors` before/after middleware). It is a different + problem — CORS decides who may *read* a response, CSRF stops a forged write. + Conflating them is the most common way both get misconfigured. +- **Same-site cookie attributes as the whole answer.** Iteration 2 sets a safe + `SameSite` default and it does real work, but it is a browser behaviour, not a + server guarantee, and older clients ignore it. +- **Captcha, rate-limited forms, bot defence.** Rate limiting shipped in + iteration [1](01-store-backed-middleware.md); the rest is not porch's. +- **Encrypted or stateless tokens.** No symmetric cipher exists, and a + stateless token cannot be revoked or spent once. + +## Info + +Forks the spec must settle: + +1. **Double-submit cookie, or session-stored token?** Double-submit needs no + store and works without sessions; session-stored needs no second cookie and + is strictly stronger. Since iteration 3 lands first, leaning session-stored — + and if so, say plainly that porch has no CSRF story for sessionless apps + rather than shipping the weaker one silently. +2. **Which routes default to single-use?** All of them is safest and the most + annoying; opt-in is pleasant and easy to forget on the one route that + mattered. Leaning opt-in with the checkout as the worked example, because a + default nobody can live with gets disabled wholesale. +3. **What happens when a form is submitted twice by a human?** This is the fork + that decides whether the feature is usable. Iteration + [1](01-store-backed-middleware.md)'s idempotency machinery may be the honest + answer — the same key, replayed, gets the same response — which would make + these two features compose rather than collide. diff --git a/docs/stories/porch/05-routing-response-ergonomics.md b/docs/stories/porch/05-routing-response-ergonomics.md new file mode 100644 index 0000000..6535e2f --- /dev/null +++ b/docs/stories/porch/05-routing-response-ergonomics.md @@ -0,0 +1,142 @@ +--- +track: porch +iteration: "5" +status: refine +--- + +# porch 5 — routing and response ergonomics: the parity that is merely missing + +> Part of [Story — `porch`, the writeonce web framework](00-story.md). +> Source: [the Fiber parity study](../../plan/exploration/fiber/00-fiber-parity.md) §5. +> +> Independent of iterations 2–4 and of the streaming seam — startable at any +> time, and a reasonable slice to interleave when the risky work needs a break. +> Nothing here is hard; all of it is felt. + +## Goals + +- **The rest of the method helpers.** `App` has `get`/`post`/`put`/`delete_`. + A `Route { method: "PATCH" }` literal already works, so this is registration + ergonomics rather than capability — but writing the literal by hand for + `PATCH` while `get` exists is the kind of asymmetry that makes a framework + feel unfinished. Add `patch`, `options`, `head`, and an `all`. +- **Named routes and reverse routing.** Fiber has `Name()` and `GetRouteURL()`. + porch has neither, so every link in the site is a hand-written string that no + compiler checks — and the site is exactly the app where a renamed path breaks + a page silently. +- **A per-route body limit.** `BODY_MAX = 1048576` is one compile-time constant + for the whole server. An upload route and a JSON route want different numbers, + and the JSON route wants a much smaller one than the upload route can live + with. +- **Request ids.** `req.ctx` already exists to carry one; there is no generator + and no middleware. With iteration 2's builtin available this is a few lines, + and it is the difference between logs you can correlate and logs you cannot. +- **The response helpers written by hand today.** `Location`, `Vary`, + `Attachment`/`Download`, and a content-negotiated `format` dispatch on top of + the existing `accepts()`. Also q-value *ranking*, which the ledger has carried + as a known 🔶 since the negotiation slice landed. + +## Phases + +### Phase A — method helpers and route introspection + +- The missing registration helpers, including `all`, and decide whether `head` + auto-registers alongside `get` (Fiber has `DisableHeadAutoRegister`, which + tells you the default is auto and that people want it off). +- Route introspection — list the table — because it is nearly free once routes + are already a `multi Route`, and it is what makes a startup banner or a + route-dump flag possible. +- Verify: each method dispatches; `405` still carries a correct `Allow` built + from the real table; existing routes unchanged. + +### Phase B — named routes and URL building + +- A name on `Route`, a lookup, and a builder that fills `:param` captures. +- Decide the failure mode for a missing or extra parameter. A silently wrong URL + is worse than a trap, and this is a compile-time-checkable shape only once + language iteration 29's `@derive` exists — so for now it is a runtime check + and should say so. +- Migrate the site's internal links onto it, which is the proof it is usable. +- Verify: every site link resolves through the builder; a wrong parameter set is + refused loudly. + +### Phase C — per-route body limits and request ids + +- Move the limit from a module constant to route-level configuration with the + current value as the default, so no existing app changes behaviour. +- A request-id middleware writing into `req.ctx`, and settle whether an inbound + header is trusted (fork 2). +- Thread the id into the logging middleware's output, since a request id nothing + logs is decoration. +- Verify: an oversized body is refused per-route; the id appears in logs and is + stable across a request's lifetime. + +### Phase D — response helpers and negotiation ranking + +- `Location`, `Vary`, `Attachment`/`Download`, and a `format`-style dispatch + choosing a builder from `accepts()`. +- Rank q-values properly instead of stripping them, retiring the ledger's 🔶. +- Verify: `Vary` accumulates rather than overwrites (which the iteration-2 + repeated-header work makes possible); negotiation picks the highest-q match, + not the first. + +### Phase E — the gate and the ledger + +- Both serving gates, the ledger rows, the board entry. +- Verify: `just web-app`, `just site`, `just linkcheck` green. + +## Acceptance Criteria + +- **Given** a route registered with each new helper, **when** the matching + method arrives, **then** it dispatches; **and** an unmatched method still + yields `405` with an `Allow` listing exactly the registered methods. +- **Given** `head` auto-registration, **when** a `HEAD` request hits a `GET` + route, **then** the response is headers-only with the `Content-Length` a `GET` + would have sent — the behaviour `serialize()` already implements, now + reachable by registration. +- **Given** a named route with `:param` captures, **when** a URL is built with + the right parameters, **then** it matches that route's pattern exactly; + **and** a wrong or missing parameter is refused rather than producing a + plausible-looking wrong URL. +- **Given** two routes with different body limits, **when** a body exceeding the + smaller arrives at each, **then** it is refused at the small route and + accepted at the large one. +- **Given** no per-route limit, **when** a request arrives, **then** the + previous global limit applies unchanged. +- **Given** a request-id middleware, **when** a request is handled, **then** the + same id appears in every log line for that request and in the response header. +- **Given** an `Accept` header with q-values out of order, **when** negotiation + runs, **then** the highest-q acceptable type wins — not the first listed. +- **Given** two `Vary` contributions from different middleware, **when** the + response leaves, **then** both appear. + +## Out Of Scope + +- **A radix-tree router.** Path matching is a linear scan and the ledger marks + it 🔶 pending a *measurement*. Language iteration 22 built the benchmark + harness but pointed it at the database. Until someone benches the router, this + is an optimisation without evidence. +- **Case-insensitive or strict-slash routing.** Fiber exposes both as config. + porch is case-sensitive and lenient; changing that is a behaviour change for + existing apps and wants its own decision. +- **Compile-time-checked URL building.** The typed version needs language + iteration [29](../language-runtime-database/29-compile-time-metaprogramming.md). + Runtime-checked now, upgraded later. +- **Streaming responses, `SendFile`, byte ranges** — iterations + [6](06-streaming-core.md) and [8](08-static-and-lifecycle.md). +- **Typed binding of params into a class** — language iteration 29 again. + +## Info + +Forks the spec must settle: + +1. **Does `head` auto-register?** Fiber's default is yes with an opt-out. Auto is + friendlier; explicit is more predictable and never surprises someone + debugging why a route they did not register is answering. +2. **Is an inbound request-id header trusted?** Behind the mandated proxy, + trusting it is what makes tracing work across hops. On an open port it lets a + client forge correlation ids and poison logs. `client_ip` and `net.peer` + already exist for exactly this trust decision — reuse that conclusion. +3. **Where does a route's body limit live?** A field on `Route` is the obvious + home but widens a record that the conformance corpus pins the ownership shape + of. Check that fixture before choosing. diff --git a/docs/stories/porch/06-streaming-core.md b/docs/stories/porch/06-streaming-core.md new file mode 100644 index 0000000..1002619 --- /dev/null +++ b/docs/stories/porch/06-streaming-core.md @@ -0,0 +1,155 @@ +--- +track: porch +iteration: "6" +status: refine +--- + +# porch 6 — streaming core: the seam three iterations wait on + +> Part of [Story — `porch`, the writeonce web framework](00-story.md). +> Source: [the Fiber parity study](../../plan/exploration/fiber/00-fiber-parity.md) §3. +> +> The largest and riskiest slice in this track, and the one with the most +> downstream value: iterations [7](07-sse-and-compression.md) and +> [8](08-static-and-lifecycle.md) are both blocked on it, and porch's README has +> carried "lazy body streaming · streaming responses · explicit commit point" +> as parked since framework v1. + +## Goals + +- **A response that can be written incrementally.** Today `internal/serve.wo` + builds the whole response as one `Text` and hands it to a single `net.write`, + and `serialize()` always emits `Content-Length`. Nothing can produce output it + cannot first hold entirely in memory — which rules out large downloads, + server-sent events, and any response whose length is unknown when the first + byte is ready. +- **Chunked transfer-encoding on the way out**, correctly framed and correctly + terminated, because a truncated chunked response is indistinguishable from a + network failure to the client and corrupts keep-alive for the connection. +- **Chunked request bodies on the way in — carefully.** + `internal/parse.wo` **deliberately refuses** them today, with a correct note + that silently treating a chunked request as body-less is request smuggling. + That refusal is good engineering. It may only be lifted by an implementation + that handles the smuggling cases explicitly, and the refusal must remain the + behaviour for anything the parser is not certain about. +- **An explicit commit point.** Once the first byte is written, the status and + headers are gone and no `after` middleware can change them. That is a real + semantic change to the middleware contract and it has to be stated, not + discovered — the `after` chain currently runs on *every* response and + security headers depend on it. + +## Phases + +### Phase A — the writer seam + +- Decide the shape (fork 1) and introduce a way for a handler to emit body + bytes progressively instead of returning a complete `Resp`. Handlers are + classes, so this is a second interface beside `Handler`, not a callback. +- Keep the existing whole-response path as the default and unchanged: the + overwhelming majority of responses are small and should not pay for this. +- Verify: the two paths coexist; every existing response is byte-identical. + +### Phase B — chunked responses + +- Chunk framing, the terminating zero-length chunk, and the interaction with + keep-alive — a connection whose chunked response was truncated must be closed, + not reused. +- `Content-Length` and chunked are mutually exclusive; `serialize()` must pick + one and never emit both. +- `HEAD` on a streaming route: headers only, and decide what `Content-Length` + claims when the length is unknown. +- Verify: a chunked response reassembles byte-exactly; a mid-stream trap closes + the connection rather than leaving a half-frame; `HEAD` is coherent. + +### Phase C — the commit point and the middleware contract + +- Define and enforce when headers are locked. An `after` middleware that tries + to mutate a committed response must fail loudly in development rather than + silently doing nothing. +- Decide what happens to `SecurityHeaders` and `Cors` — both are `after` + middleware and both must still apply to streamed responses, which means they + have to run *before* the commit for those routes. +- Verify: security headers and CORS are present on a streamed response; a + post-commit mutation attempt is reported. + +### Phase D — chunked request bodies + +- Only if phase C is clean. Parse chunked request bodies with the smuggling + cases enumerated and tested: both `Content-Length` and `Transfer-Encoding` + present, duplicated `Transfer-Encoding`, unknown transfer codings, and + oversized or malformed chunk sizes. +- Every ambiguous case stays a 400-and-close, matching the existing duplicate + `Content-Length` discipline. +- Verify: a well-formed chunked upload arrives intact; every enumerated + smuggling shape is refused. + +### Phase E — the gate and the ledger + +- A streaming route in a sample, gated on both consumers, plus a large-body leg + proving memory does not scale with response size. +- Retire the README's parked streaming rows; record the commit-point semantics + where a handler author will find them. +- Verify: `just web-app`, `just site`, `just linkcheck` green; ASan clean. + +## Acceptance Criteria + +- **Given** a streaming handler emitting N chunks, **when** a client reads the + response, **then** the reassembled body is byte-exact and the framing is + well-formed. +- **Given** a response far larger than the arena, **when** it is streamed, + **then** it completes and peak memory does not grow with the body — the + criterion that distinguishes streaming from buffering. +- **Given** a handler that traps mid-stream, **when** the failure occurs, + **then** the connection is closed rather than reused, no half-frame is left + behind, and the server survives. +- **Given** a streamed response, **when** it leaves, **then** the security and + CORS headers the `after` chain contributes are still present. +- **Given** an `after` middleware attempting to change a committed response, + **when** it runs, **then** the attempt is reported rather than silently + dropped. +- **Given** a `HEAD` request to a streaming route, **when** it is answered, + **then** the headers are coherent and no body is sent. +- **Given** a request with both `Content-Length` and `Transfer-Encoding`, + **when** it is parsed, **then** it is refused with 400 and the connection is + closed — smuggling is refused, never guessed at. +- **Given** every pre-existing non-streaming response, **when** both serving + gates run, **then** output is byte-identical to before this iteration. + +## Out Of Scope + +- **SSE** — iteration [7](07-sse-and-compression.md), the first consumer. +- **Compression** — also [7](07-sse-and-compression.md); it composes with + chunking and should not be entangled with building it. +- **Byte ranges and `SendFile`** — iteration + [8](08-static-and-lifecycle.md). +- **Request-body backpressure as a general mechanism.** Reading a body slowly to + push back on a producer wants cancellation, which porch does not have and + which language iteration 31's actor lifecycle owns. This iteration streams + *out* and parses chunked *in*; it does not add flow control. +- **WebSockets.** Already shipped (`ws_accept`, `wsframe`) and deliberately a + hijack that bypasses `serialize()` — that path must keep working untouched, + which is worth an explicit regression check. +- **HTTP/2.** Proxy-terminated by doctrine, and parked behind language + iteration 23 regardless. + +## Info + +Forks the spec must settle: + +1. **What is the writer?** Candidates: a second interface whose method is + called repeatedly until it signals done; a `Resp` variant carrying a producer + object instead of a `Text` body; or a handler that receives the connection and + writes directly (which is what `ws_accept` already does via the 101 hijack + sentinel). The third is the least new machinery and the most footgun. The + first fits the no-closures doctrine best, since a producer is just another + class with fields. +2. **Does the `after` chain still run for streamed responses?** It must, or + security headers regress. But it cannot run *after* the body. So either + `after` runs at commit time for streaming routes, or streaming routes declare + they opt out and the framework refuses to combine them with header-mutating + middleware. Silent partial application is the one unacceptable answer. +3. **Is chunked request parsing in this iteration at all?** It is separable and + it is the riskiest security surface in the framework. Splitting phase D into + its own iteration is a legitimate outcome of the brainstorm — the study is + explicit that the current refusal is *correct*, so there is no pressure to + rush it. diff --git a/docs/stories/porch/07-sse-and-compression.md b/docs/stories/porch/07-sse-and-compression.md new file mode 100644 index 0000000..8518694 --- /dev/null +++ b/docs/stories/porch/07-sse-and-compression.md @@ -0,0 +1,144 @@ +--- +track: porch +iteration: "7" +status: refine +--- + +# porch 7 — server-sent events and compression + +> Part of [Story — `porch`, the writeonce web framework](00-story.md). +> Source: [the Fiber parity study](../../plan/exploration/fiber/00-fiber-parity.md) §3. +> Both halves need [iteration 6](06-streaming-core.md); neither is possible +> before it. + +## Goals + +- **SSE, which fits this runtime unusually well.** A room actor already has the + fan-out shape a live feed needs, and a parked fiber per subscriber costs + almost nothing on the shard model — so the awkward part of SSE in most + frameworks (holding thousands of idle connections) is the part writeonce + already solved with iteration 35's idle deadlines and fiber-per-connection. + Fiber ships `Retry`, `HeartbeatInterval` and `OnClose`; all three matter, + because a proxy will silently drop an idle event stream. +- **gzip/deflate, decided honestly.** Iteration 36 landed the bitwise operators, + so a pure-`.wo` DEFLATE is now *expressible* — the question is whether it + should be. A C builtin is faster and smaller to write; a `.wo` implementation + keeps the runtime doctrine intact and proves the language can do real + bit-level work. Fork 2 decides, and the answer should turn on whether anything + else will ever want zlib. +- **Negotiate, never assume.** Compress only when the client said it accepts the + coding, only above a size threshold, and never for content that is already + compressed — a gzipped JPEG is bigger than the JPEG. + +## Phases + +### Phase A — SSE framing and lifecycle + +- The event framing (`data:`, `event:`, `id:`, `retry:`), the double-newline + terminator, and the `text/event-stream` content type with caching disabled. +- Heartbeats, because an idle stream through a proxy dies quietly. Interacts + directly with iteration 35's `idle_ms` — a heartbeat interval longer than the + idle deadline evicts the client the heartbeat exists to keep. +- Disconnect detection and cleanup: a write to a gone client must free the + fiber, the actor subscription and the fd, on every path. +- Verify: a client receives ordered events; a heartbeat keeps an otherwise idle + stream alive past `idle_ms`; a disconnect releases everything (fd count flat). + +### Phase B — an SSE workload worth gating + +- A live feed in a sample fed by an actor, so the fan-out path is real rather + than a loop in one handler. +- `Last-Event-ID` resumption, or an explicit statement that it is not supported + — silently ignoring it means clients think they resumed when they did not. +- Verify: N concurrent subscribers all receive an event published once; fds and + memory flat across a churn of connect/disconnect. + +### Phase C — the compression codec + +- Implement or bind the codec per fork 2, with the framing gzip requires + (header, deflate stream, CRC32 and length trailer). Note CRC32 does not exist + yet — the crypto row lists it as waiting for a consumer, and this is that + consumer. +- Correctness against a reference decompressor is the acceptance bar, on + awkward input: empty, highly repetitive, incompressible, and larger than any + internal buffer. +- Verify: `gzip -d` reproduces the input byte-exactly for every case. + +### Phase D — the compression middleware + +- `Accept-Encoding` negotiation reusing the q-value ranking iteration + [5](05-routing-response-ergonomics.md) added, a minimum-size threshold, and a + content-type skip list. +- `Vary: Accept-Encoding` on anything compressed, or every cache in front will + serve gzipped bytes to a client that cannot read them. This needs iteration + 2's repeated-header work to accumulate correctly with other `Vary` + contributions. +- Composition with chunked streaming: compress then chunk, and the ETag question + — an ETag computed over compressed bytes is a different entity than the same + resource uncompressed. +- Verify: a compressed response decompresses to the original; `Vary` is present; + no double-compression; an already-compressed content type is skipped. + +### Phase E — the gate and the ledger + +- Both serving gates plus an ASan leg for the codec, which is the part most + likely to leak or over-read. +- Retire the ledger's SSE and compression rows. +- Verify: `just web-app`, `just site`, `just linkcheck` green; ASan clean. + +## Acceptance Criteria + +- **Given** an SSE endpoint and a subscribed client, **when** events are + published, **then** the client receives them in order with correct framing. +- **Given** an idle SSE stream and a heartbeat interval shorter than `idle_ms`, + **when** it idles past the deadline, **then** it stays open. +- **Given** a heartbeat interval *longer* than `idle_ms`, **when** the stream + idles, **then** the misconfiguration is evident rather than mysterious — the + interaction is documented and, ideally, refused at construction. +- **Given** a client disconnecting mid-stream, **when** the next publish + occurs, **then** the write failure frees the fiber, the subscription and the + fd; fd count returns to baseline. +- **Given** N concurrent subscribers, **when** one event is published, **then** + all N receive it and memory does not grow per event. +- **Given** any input including empty, repetitive and incompressible, **when** + it is compressed, **then** a reference `gzip -d` reproduces it byte-exactly. +- **Given** a client that did not send `Accept-Encoding`, **when** it requests a + compressible resource, **then** the response is uncompressed. +- **Given** a compressed response, **when** it leaves, **then** `Vary: + Accept-Encoding` is set and coexists with any other `Vary` contribution. +- **Given** an already-compressed content type, **when** it is served, **then** + it is not compressed again. + +## Out Of Scope + +- **Brotli and zstd.** One codec, proven, before a second. gzip is what every + client accepts. +- **Request-body decompression.** A compressed *upload* is a separate surface + with its own decompression-bomb risk, and no workload asks yet. +- **WebSockets as an SSE alternative.** Already shipped and a different tool; + SSE is the one-way, proxy-friendly, reconnect-by-default option. +- **Compressing static files at rest.** Fiber's static has `Compress`; + precompressed-file serving belongs with iteration + [8](08-static-and-lifecycle.md). +- **A general-purpose zlib library surface.** Whatever lands is what the + middleware needs. If a second consumer appears, it can argue for a library. + +## Info + +Forks the spec must settle: + +1. **Heartbeat versus idle deadline.** These two mechanisms can silently fight, + and the failure looks like a flaky network. Decide whether the framework + refuses an incoherent pair at construction — leaning yes, because a + configuration that cannot work should not be constructible. +2. **`.wo` DEFLATE or a C builtin?** The honest tiebreaker is whether anything + else ever wants zlib. If compression is the only consumer forever, a `.wo` + implementation keeps the runtime small and is a genuine demonstration that + iteration 36's bit operators earned their place. If a second consumer is + plausible (precompressed assets, a WAL codec, an archive format), the builtin + wins. CRC32 comes along either way. +3. **ETag over compressed or uncompressed bytes?** `etag_for` exists and is used + with `with_etag` for 304s. Compressing after the ETag is computed keeps the + entity identity stable across encodings, which is almost certainly right — + but it must be decided, because getting it wrong serves the wrong body for a + conditional request. diff --git a/docs/stories/porch/08-static-and-lifecycle.md b/docs/stories/porch/08-static-and-lifecycle.md new file mode 100644 index 0000000..40e8e7b --- /dev/null +++ b/docs/stories/porch/08-static-and-lifecycle.md @@ -0,0 +1,153 @@ +--- +track: porch +iteration: "8" +status: refine +--- + +# porch 8 — static files, lifecycle hooks, and the small middleware everyone ships + +> Part of [Story — `porch`, the writeonce web framework](00-story.md). +> Source: [the Fiber parity study](../../plan/exploration/fiber/00-fiber-parity.md) §3, §5. +> The static half needs [iteration 6](06-streaming-core.md); the rest does not. +> +> The clean-up iteration. Individually every item is small; together they are +> most of what makes a framework feel finished rather than adequate. + +## Goals + +- **Static files that can serve something large.** `StaticFiles` exists and is + already careful — traversal is refused rather than normalised, `max_bytes` is a + hard ceiling, and the site's `/dl` downloads run through it. What it cannot do + is serve a file it cannot hold in memory, resume a partial download, or let a + browser cache correctly. Fiber's static ships `ByteRange`, `MaxAge`, + `CacheDuration`, `IndexNames`, `Browse` and `Download`; the range support is + what makes video and large downloads work at all. +- **Cache headers that let a client skip the request.** `etag_for`/`with_etag` + give conditional GETs; `Cache-Control`, `Last-Modified` and `If-Modified-Since` + are the other half, and `fs.stat` already returns the mtime they need. +- **Lifecycle hooks.** The ledger records "no user teardown hooks yet" as a known + gap. Fiber has eleven hook families; porch needs a small handful — on-listen, + on-shutdown, and on-route-registered — and the shutdown one is the one that + matters, because an app with its own resources currently has nowhere to close + them. +- **The small middleware every framework ships**: healthcheck, favicon, + redirect, rewrite, and a `skip` combinator. Each is a handful of lines and + their absence is felt immediately by anyone starting a new app. + +## Phases + +### Phase A — ranges and cache headers + +- `Range` request parsing (single range first; multi-range is a multipart + response and can wait), `206 Partial Content`, `Content-Range`, and + `Accept-Ranges`. An unsatisfiable range is `416`, not a truncated `200`. +- `Last-Modified` from `fs.stat`'s mtime, `If-Modified-Since` handling, and + `Cache-Control` with a configurable max-age. +- Serve the body through iteration 6's writer so file size stops bounding what + can be served. +- Verify: a ranged request returns exactly the requested bytes; a file much + larger than the arena serves; a conditional request returns 304 with no body. + +### Phase B — directory behaviour + +- Index-file resolution (`index.html` and friends) and an optional directory + listing, defaulting **off** — a listing that is on by default is an + information leak the first time someone points it at the wrong directory. +- `Download`/`Attachment` disposition, reusing the helper iteration + [5](05-routing-response-ergonomics.md) added. +- Re-verify the traversal refusal against every new path (index resolution and + listing both construct paths, which is exactly where traversal creeps back + in). +- Verify: index resolution works; listing is off unless asked for; traversal is + still refused on all new paths. + +### Phase C — lifecycle hooks + +- Decide the minimal set (fork 1) and the interface — hooks are classes, like + everything else here. +- Wire the shutdown hook into the existing `env.stopping()` path so an app can + flush and close before the process exits, and guarantee it runs exactly once + even on a trapping path. +- Verify: the shutdown hook fires on SIGTERM before the listener closes, once; + a trapping hook does not prevent shutdown. + +### Phase D — the small middleware set + +- Healthcheck (liveness and readiness are different questions — say which), + favicon, redirect (permanent and temporary), rewrite (internal, no round + trip), and `skip` wrapping another middleware with a predicate. +- Verify: each behaves; `skip` composes with the existing chain in registration + order. + +### Phase E — the gate and the ledger + +- Both serving gates. The site is the natural subject: it already serves + `/favicon.svg`, `/health`, and `/dl` downloads through `StaticFiles`, so these + features have a real consumer rather than a synthetic one. +- Close out the porch track's ledger rows and record what the whole track + actually landed versus what the Fiber study predicted. +- Verify: `just web-app`, `just site`, `just linkcheck` green. + +## Acceptance Criteria + +- **Given** a `Range: bytes=a-b` request, **when** it is served, **then** the + response is `206` with exactly those bytes and a correct `Content-Range`. +- **Given** an unsatisfiable range, **when** it is served, **then** the response + is `416`, never a truncated `200`. +- **Given** a file larger than the heap, **when** it is requested, **then** it + serves completely and peak memory does not track file size. +- **Given** `If-Modified-Since` matching the file's mtime, **when** the request + arrives, **then** the response is `304` with no body. +- **Given** a directory with an index file, **when** the directory is requested, + **then** the index is served; **and** with no index and listing disabled, the + response is `404`, not a listing. +- **Given** a traversal attempt through the index-resolution and listing paths, + **when** it is served, **then** it is refused — the existing guarantee, re-proven + against the new code paths. +- **Given** a registered shutdown hook, **when** the process receives SIGTERM, + **then** the hook runs exactly once before the listener closes, and in-flight + requests still complete. +- **Given** a hook that traps, **when** shutdown runs, **then** shutdown still + completes. +- **Given** `skip` wrapping a middleware with a predicate, **when** the + predicate matches, **then** the wrapped middleware does not run and the chain + continues in order. + +## Out Of Scope + +- **Multi-range requests.** A multipart byte-range response is a separate + format; single ranges cover downloads and media seeking, which is what the + workload needs. +- **Precompressed asset serving** (`file.gz` beside `file`). Composes with + iteration [7](07-sse-and-compression.md); worth doing once, later, when both + exist. +- **A file-watching or hot-reload story.** Assets are read from disk per + request; a cache with invalidation is a different feature and language + iteration [18](../language-runtime-database/18-memory-db-features.md) owns + caching. +- **`pprof`, `expvar`, metrics endpoints.** Language iteration 30 (no story file + yet). A healthcheck is not observability — it is one bit. +- **Fiber's fork, mount and prefork hooks.** The shard runtime owns placement; + there is no worker-pool to hook. +- **`SendFile` with kernel `sendfile(2)`.** No such builtin exists and no + iteration owns adding one; the streaming writer is the portable answer here. + +## Info + +Forks the spec must settle: + +1. **Which hooks, exactly?** Fiber has eleven families and porch needs the + fewest that are load-bearing. On-shutdown is clearly one — an app with open + resources has nowhere to close them today. On-listen is convenient for a + startup banner. On-route-registered is only useful for introspection, which + iteration [5](05-routing-response-ergonomics.md) may already cover. Fewer is + better; each hook is a contract forever. +2. **Liveness or readiness for healthcheck?** They answer different questions + and conflating them is why deployments flap: liveness says "do not kill me", + readiness says "do not send me traffic yet". A framework shipping one + endpoint called `/health` should say which it is, and probably ships both. +3. **Is directory listing available at all?** Off-by-default is not the same as + present-but-off. Shipping it at all means it will eventually be switched on + somewhere it should not be. Leaning: ship it, off, with the doc saying + plainly what it exposes — the alternative is every app hand-rolling a worse + one.