shoney/writeonce
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

refactor(porch-store): re-scope porch 1 to the limiter, revert idempotency

Browse source 
- idempotency built, reviewed, then reverted WHOLE to the tag
  archive/porch-idempotency. Not a design failure: it passed its gates.
  It provokes a C-runtime SIGSEGV in wo_arena_alloc/wo_str_new under
  concurrent call()-parked callers
- the evidence for that attribution: over ten gate runs every failure
  was an idempotency leg and none was the limiter's, which drives the
  same pool through the same call/park machinery. The begin arm has 5x
  the allocation sites inside receive and moves a whole Req plus a
  Handler through the mailbox
- before the split the suite reported 0 to 6 failures run to run; after
  it, five consecutive runs at 56 checks, 0 failures
- PoolMsg loses digest/req/handler, and NullHandler/dummy_req/fresh_req
  go with them — every rate-limit count used to allocate a throwaway
  Req it never read
- IdempotencyKey is KEPT and commented: the schema is settled and the
  digest-as-column decision cost a review round to get right
- the limiter's saturation 503 has no leg of its own now (§19 drove
  Idempotent). Stated in the README rather than papered over — a
  deterministic leg needs a slow actor, and only the reverted arm was
- new: porch 9 (idempotency, on hold) and language 41 (the arena crash,
  with the reproduction harness and the evidence that localises it)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 79e6da4465133dc555e913c960d544ef1c7bedd8)
This commit is contained in:
shoney.arickathil 2026-08-30 04:56:16 +02:00
parent f3215c2f43
commit aa13b2125f
6 changed files with 71 additions and 935 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
docs/examples/porch/README.md
View file

@ -156,8 +156,8 @@ first (pure `.wo` cannot express it yet).
| Item | State |
| --- | --- |
| Rate limiting (fixed window, durable) | ✅ counting serializes through a per-key actor pool (`middleware/keypool.wo`, `limiter.wo`) — no handler-fiber read-modify-write left to lose an increment. Gate-proven: exact count under genuine concurrency (30 parallel requests, no lost increments), WAL-durable restart still limiting, and `trust_proxy`'s peer fallback — [porch 1](../../stories/porch/01-store-backed-middleware.md). Saturation failing closed (503) shares `limiter.wo`'s own `try/catch` code path with idempotency's, but is gate-proven only via `Idempotent`/`pool_begin`'s own saturation leg in `scripts/web-app-accept.sh` — no leg drives `Limiter`'s own 503 arm directly |
| Idempotent replay of unsafe requests | ✅ the pool actor runs the route's `Handler` itself (`middleware/idempotent.wo`), so a duplicate blocks in the actor's mailbox until the owner's row commits — no in-flight heuristic, no window where a duplicate can see "nothing yet". Gate-proven: byte-identical replay, digest-mismatch refusal (422), concurrent duplicates never double-executing, a transient 5xx never replayed (solo or concurrent), ephemeral rows not leaking, and pool saturation failing closed (503) — [porch 1](../../stories/porch/01-store-backed-middleware.md) |
| Rate limiting (fixed window, durable) | ✅ `Limiter` middleware over a sharded actor pool — exact counting under 30 genuinely-parallel clients, WAL-durable across a SIGTERM restart, `trust_proxy` off by default with a `net.peer` fallback. **One gap, stated rather than hidden:** the fail-closed 503 on a saturated pool is correct by construction (same `try`/`catch` as the arm that was gate-proven) but has no leg of its own — saturating the count arm deterministically needs a slow actor, and only the reverted idempotency arm was slow. The proof lives in `archive/porch-idempotency` |
| Idempotent replay of unsafe requests | ⏸ **built, reviewed, then reverted 2026-08-30.** Not a design failure: the pool actor ran the route handler inside its own `receive` so a duplicate waited in the mailbox, and it passed its gates. It provoked a C-runtime SIGSEGV in `wo_arena_alloc`/`wo_str_new` under concurrent `call()`-parked callers. Whole in `archive/porch-idempotency`, which doubles as the reproduction harness. Blocked on the runtime fix |
| Transaction-per-request middleware (commit on 2xx, roll back otherwise) | ⏸ **v2** — needs iteration 18's `transaction { }` |
| Cancellation → rollback | ⏸ arc landed; still needs v2's `transaction { }` (iteration 18) |
| Migration generation + review workflow | ⬜ recorded future story (script-based destructive migrations) |

121
docs/examples/porch/middleware/idempotent.wo
View file

@ -1,121 +0,0 @@
-- porch/middleware/idempotent.wo — idempotency, actor-run.
-- Iteration 1 of the porch track, porch-store task 4.
--
-- Rebuilt, not patched: the old before/after shape ran the handler and
-- stored the response afterward, so two simultaneous duplicates both
-- missed and both executed — before() has nothing to find until after()
-- runs, which is too late for the request that is racing it. The fix is
-- that the ACTOR runs the handler: Idempotent wraps the route's own
-- Handler and hands both the request and that handler to the pool. A
-- duplicate for the same key then simply waits in the actor's mailbox
-- (one message at a time) and is dequeued once the owner's receive has
-- already committed the row — no in-flight heuristic needed, because
-- there is no window where a duplicate can see "nothing yet".
--
-- `call`'s reply is a copyable scalar only (WO-E226): keypool.wo's kind-2
-- arm answers with the SAME pool_pack(count, remaining_ms) encoding kind-1
-- uses, never the response itself. The response travels through the
-- @table instead — the actor stores it, this file reads the same row
-- back and builds the Resp from it, so owner and duplicate answer with
-- byte-identical bytes structurally, not by careful bookkeeping.
use http
use json
-- Idempotent wraps a route's Handler. Registration: Idempotent { key_header:
-- "Idempotency-Key", pool: p, inner: CreateOrder {} } in place of the bare
-- handler in app.post(...) -- key_header is matched case-insensitively
-- (req.headers keys are lowercased on read, so any case here works). Only
-- the response allowlists content-type, location, etag, cache-control for
-- replay (never Set-Cookie, never Date) — cookies arrive in porch 2.
pub class Idempotent {
key_header: Text -- e.g., "Idempotency-Key" (matched case-insensitively)
pool: Pool
inner: Handler -- the real route handler; the actor runs this
include_body: Bool = true -- digest method+path+body, refuse a key reused with a different one
ttl: Int = 86_400_000_000 -- 24h in µs, lazy-expired on access
fn handle(req: Req) -> Resp {
-- req.headers keys are lowercased on read (internal/parse.wo); normalise
-- key_header the same way, or a documented `key_header: "Idempotency-Key"`
-- (capitalised, as app authors are told to write it) NEVER matches and
-- this middleware silently falls through to self.inner.handle(req) below
-- on every request -- idempotency disabled, no 503, no log.
let name = to_lower(self.key_header);
let header_val = req.headers[name];
if header_val == nil { return self.inner.handle(req); }
let key = "idem:${name}:${header_val}";
-- method+path scope the digest unconditionally: with include_body false
-- a bare "" digest would match ANY other request under this same key,
-- letting a different route/method replay this one's stored response.
let digest_input = "${req.method}|${req.path}";
if self.include_body { digest_input = "${digest_input}|${req.body}"; }
let digest = base64_encode(bytes_slice(sha256(bytes_of_text(digest_input)), 0, 16));
let raw = try pool_begin(self.pool, key, digest, self.ttl, req, self.inner) catch (e) {
print_err("idempotent: pool_begin trapped: ${e.msg}");
nil
};
if raw == nil {
let r = Resp { status: 503, headers: {}, body: "{\"error\":\"idempotency store saturated\"}" };
set_header(r, "content-type", "application/json");
set_header(r, "retry-after", "1");
return r;
}
let outcome = raw / 1_000_000_000;
if outcome == 2 {
-- Same key, a different request: refuse rather than serve the
-- other request's response.
let r = Resp { status: 422, headers: {}, body: "{\"error\":\"idempotency key reused with a different request\"}" };
set_header(r, "content-type", "application/json");
return r;
}
-- Outcome 1: the bare key names a durable (2xx/3xx) replay target --
-- this file never deletes it; it is the whole point of a durable row.
-- Outcome 3: this call's own 4xx/5xx answer lives under a row keyed
-- by a nonce (the reply's low digits) that nobody else's message
-- for this same bare key ever writes to -- rebuild that exact key
-- rather than reading the bare one, so a race with a LATER message
-- for this key (which never touches this row) can't hand back the
-- wrong response.
let lookup_key = key;
if outcome == 3 { lookup_key = "${key}#eph:${raw % 1_000_000_000}"; }
let hits = from k in IdempotencyKey where k.key == lookup_key take 1 select k;
if len(hits) == 0 { return server_error(); }
let row = hits[0];
let stored = json.decode(row.response) as IdempotentStoredResp;
if stored == nil { return server_error(); }
-- `.. ""` forces a fresh, independently-owned Text for every key/value
-- copied out of the decoded record: json.decode's Text values do not
-- survive being handed onward as-is once the decoded record itself
-- goes out of scope (a stale row read back corrupted mid-response
-- otherwise) — concat is documented to always allocate new owned text.
let hdrs: map<Text, Text> = {};
for k, v in stored.headers { hdrs[k .. ""] = v .. ""; }
let result = Resp { status: stored.status, headers: hdrs, body: stored.body .. "" };
if outcome == 3 {
-- Safe to delete here, unlike the shared bare-key row rounds 1/2
-- removed: the nonce that names this row was never handed to
-- anyone but this one call() reply, so no other request -- a
-- duplicate, a retry, anything -- can ever construct this exact
-- key to read it. Deleting it removes the leak AND the
-- nonce-wraparound collision (time.ticks() % 1_000_000_000 repeats
-- every ~1000s; a lingering row from an earlier failed attempt
-- landing on the same nonce would otherwise be there to collide
-- with, or worse, poison the actor's OWN unguarded insert on the
-- next failure for this key).
delete row;
}
return result;
}
}
-- JSON shape of IdempotencyKey.response. Allowlisted headers only:
-- content-type, location, etag, cache-control, never Set-Cookie or Date.
typedef IdempotentStoredResp = {
status: Int,
headers: map<Text, Text>,
body: Text
}

203
docs/examples/porch/middleware/keypool.wo
View file

@ -17,22 +17,21 @@
-- type, so a second encoding is not an option.
use time
use http
use json
-- To a pool actor. kind 1 = count (this file); kind 2 = begin (Task 4
-- fills in the arm — the fields below are already shaped for it: the
-- bare idempotency key travels in `key`, the body digest in `digest`,
-- and the actor runs `handler` against `req` itself so a duplicate waits
-- in the mailbox rather than needing a held reply).
-- To a pool actor. `kind` is kept even though only one kind exists today:
-- idempotency's `kind: 2` arm was built, reviewed and then REVERTED (see
-- archive/porch-idempotency), and it will come back. Adding a second kind is
-- a field and an `if`, not a redesign.
class PoolMsg {
kind: Int
key: Text
limit: Int -- count: max requests per window
window: Int -- count: window size, µs
digest: Text -- begin: sha256(method|path|body), Task 4
req: Req -- begin: the request, Task 4
handler: Handler -- begin: the route's handler, invoked inside receive, Task 4
}
-- What the limiter reads back from a count. `allowed` and `limit` are
@ -46,45 +45,23 @@ class Verdict {
}
-- PoolMsg requires `req`/`handler` on every construction (an actor
-- message's fields are all required, like RoomMsg's `writer` in
-- docs/examples/chat/main.wo). A count message has no request to run, so
-- it fills those two with an inert placeholder — same shape as chat's
-- dummy_writer() for RoomMsg's shutdown message.
class NullHandler {
fn handle(req: Req) -> Resp {
return Resp { status: 500, headers: {}, body: "" };
}
}
-- NOTE: this file used to carry NullHandler, dummy_req() and fresh_req().
-- They existed ONLY because PoolMsg had to carry a Req and a Handler for
-- idempotency's kind-2 arm, which meant every rate-limit count allocated a
-- throwaway Req (four maps) it never read. With that arm reverted the
-- placeholders go too, and counting stops paying for a feature it never
-- used. They are preserved with the arm in archive/porch-idempotency.
fn dummy_req() -> Req {
return Req {
method: "", path: "", params: {}, query: {}, headers: {},
body: "", principal: "", ctx: {}, conn: 0 - 1
};
}
-- A live Req arriving at a Handler is a borrow (Handler.handle's signature
-- fixes that, in router.wo — not this file's to change): its map fields
-- are references that cannot outlive the caller's scope, so forwarding
-- them as-is into an actor message is refused (WO-E222 — the same
-- aliasing rule Pool's own doc comment above describes). Copying each map
-- field into a brand-new map, then building a brand-new Req from that plus
-- the plain scalars, produces a value with no other referrer — the same
-- shape dummy_req() already sends, just carrying the real request.
fn fresh_req(r: Req) -> Req {
let params: map<Text, Text> = {};
for k, v in r.params { params[k] = v; }
let query: map<Text, Text> = {};
for k, v in r.query { query[k] = v; }
let headers: map<Text, Text> = {};
for k, v in r.headers { headers[k] = v; }
let ctx: map<Text, Text> = {};
for k, v in r.ctx { ctx[k] = v; }
return Req {
method: r.method, path: r.path, params: params, query: query,
headers: headers, body: r.body, principal: r.principal, ctx: ctx,
conn: r.conn
};
-- Packs (count, remaining-ms-in-window) into one Int: count * 1e9 +
-- remaining_ms, remaining_ms clamped to stay under 1e9 (~11.5 days —
-- far past any realistic rate-limit window). That clamp only blurs the
-- advisory reset header on an absurdly long window; it never touches the
-- count, which is the correctness-critical half.
fn pool_pack(count: Int, remaining_ms: Int) -> Int {
let r = remaining_ms;
if r < 0 { r = 0; }
if r >= 1_000_000_000 { r = 999_999_999; }
return count * 1_000_000_000 + r;
}
-- One actor per shard. Reads the row for the key, decides, and writes the
@ -94,94 +71,16 @@ fn fresh_req(r: Req) -> Req {
-- row for the new window — the stale row is retired, not mutated).
class KeyActor {
fn receive(msg: PoolMsg) -> Int {
if msg.kind == 2 {
-- Task 4: idempotency begin. msg.window carries the TTL here (both
-- are µs durations; kind 1 has no use for a TTL and kind 2 has no
-- use for a window, so the one field serves both). A miss runs
-- msg.handler right here, inside receive, so a duplicate already
-- queued behind this message dequeues to a settled row instead of
-- a race.
let now = time.ticks();
let hits = from k in IdempotencyKey where k.key == msg.key take 1 select k;
if len(hits) > 0 {
let stored = hits[0];
if now - stored.created_at > msg.window {
-- Expired: lazy delete (no sweeper exists), fall through to miss.
delete stored;
} else if stored.digest == msg.digest {
-- A row lives under the BARE key only when it is durable (see
-- below) -- an ephemeral one never does -- so any hit here is
-- already a stable, valid replay target.
return pool_pack(1, 0);
} else {
-- Same key, a different request: refuse rather than serve the
-- other request's response.
return pool_pack(2, 0);
}
}
-- Miss: fresh key, an expired row just deleted, or the prior
-- attempt (if any) was ephemeral and so is invisible to the
-- bare-key lookup above -- all three run the handler fresh.
let resp = msg.handler.handle(msg.req);
-- Allowlist, not denylist: an allowlist fails safe when Resp grows a
-- new header later (excluded from replay until reviewed, never
-- replayed by accident from day one). content-type alone dropped
-- Location off every redirect() and any Etag/Cache-Control a handler
-- set; matched case-insensitively since set_header writes the name
-- verbatim (a handler using "Content-Type" was silently losing it).
let hdrs: map<Text, Text> = {};
for hk, hv in resp.headers {
let lhk = to_lower(hk);
if lhk == "content-type" or lhk == "location" or lhk == "etag" or lhk == "cache-control" {
hdrs[lhk] = hv;
}
}
let stored_json = json.encode(IdempotentStoredResp {
status: resp.status, headers: hdrs, body: resp.body
});
if resp.status >= 200 and resp.status < 400 {
-- 2xx/3xx: a real answer worth replaying for the TTL, stored
-- under the bare key -- one stable row per key, unguarded same
-- as kind 1's own insert (this actor is the only writer for
-- this key; a @unique violation here would mean something is
-- genuinely wrong, not a race to paper over -- a swallowed
-- failure would answer "stored" for a response never written).
insert IdempotencyKey {
key: msg.key, response: stored_json, created_at: now, digest: msg.digest
};
return pool_pack(1, 0);
}
-- 4xx/5xx: never a replay target, so it does NOT go under the bare
-- key -- a bare-key row is memoryless (this file's own doc above),
-- but a SHARED, mutable row is not: a second message racing the
-- first could delete-and-replace it before the first caller's own
-- middleware-side read (necessarily outside receive -- WO-E226,
-- a Resp cannot ride the mailbox) ever runs, so the FIRST caller
-- could read back the SECOND caller's answer. Every miss instead
-- gets its own row, keyed by a nonce carried back in the scalar's
-- low digits (the same slot pool_pack's remaining_ms uses for
-- kind 1) so idempotent.wo can reconstruct the exact same key and
-- read only ever what THIS call produced -- immune to any other
-- message touching this bare key, ever. That same unguessability
-- (the nonce is never handed to anyone but this one call() reply)
-- is also why idempotent.wo deletes this row right after reading
-- it: nothing else can ever construct this exact key, so nothing
-- else is deleted out from under. Without that delete, the row
-- would linger forever (no sweeper exists) AND time.ticks() % 1e9
-- wraps every ~1000s, so a later failed attempt for the SAME
-- bare key landing on the same nonce would collide with it --
-- reintroducing a stale-replay risk on wraparound, or poisoning
-- this actor's own unguarded insert above. Deleting it removes
-- both, not just the storage growth.
let nonce = now % 1_000_000_000;
insert IdempotencyKey {
key: "${msg.key}#eph:${nonce}", response: stored_json, created_at: now, digest: msg.digest
};
return pool_pack(3, nonce);
}
-- Only kind 1 exists today. The `kind: 2` arm — idempotency, where the
-- actor ran the route handler inside this receive so a duplicate waited
-- in the mailbox — was built, reviewed and then REVERTED. It is whole in
-- the tag archive/porch-idempotency, which doubles as the reproduction
-- harness for the C-runtime crash that caused the revert: a SIGSEGV in
-- wo_arena_alloc / wo_str_new under concurrent call()-parked callers.
-- That arm allocated 5x what this one does inside receive and moved a
-- whole Req plus a Handler through the mailbox; over ten gate runs every
-- failure was one of its legs, and none were this one's.
-- kind 1: count.
let now = time.ticks();
let hits = from c in RateLimitCounter where c.key == msg.key take 1 select c;
@ -209,22 +108,6 @@ class KeyActor {
}
}
-- Packs (count, remaining-ms-in-window) into one Int: count * 1e9 +
-- remaining_ms, remaining_ms clamped to stay under 1e9 (~11.5 days —
-- far past any realistic rate-limit window). That clamp only blurs the
-- advisory reset header on an absurdly long window; it never touches the
-- count, which is the correctness-critical half.
fn pool_pack(count: Int, remaining_ms: Int) -> Int {
let r = remaining_ms;
if r < 0 { r = 0; }
if r >= 1_000_000_000 { r = 999_999_999; }
return count * 1_000_000_000 + r;
}
-- One actor address per slot. `multi actor PoolMsg` does not parse (a
-- `multi`'s element type is one token) — chat/main.wo's RoomRef wraps an
-- actor handle in a one-field class for exactly this reason, mirrored
-- here as PoolSlot.
class PoolSlot {
a: actor PoolMsg
}
@ -308,10 +191,7 @@ pub fn pool_select(pool: Pool, key: Text) -> actor PoolMsg {
-- actor's scalar reply into the Verdict the limiter reads.
pub fn pool_count(pool: Pool, key: Text, limit: Int, window: Int) -> Verdict {
let a = pool_select(pool, key);
let raw = call(a, PoolMsg {
kind: 1, key: key, limit: limit, window: window,
digest: "", req: dummy_req(), handler: NullHandler {}
});
let raw = call(a, PoolMsg { kind: 1, key: key, limit: limit, window: window });
let count = raw / 1_000_000_000;
let remaining_ms = raw % 1_000_000_000;
return Verdict {
@ -322,23 +202,6 @@ pub fn pool_count(pool: Pool, key: Text, limit: Int, window: Int) -> Verdict {
};
}
-- The begin accessor idempotent.wo calls: hides pool_select/call the same
-- way pool_count does. Returns the raw packed outcome — 1 means the
-- BARE key names a durable (2xx/3xx) replay target; 2 means a digest
-- mismatch (422, nothing to read); 3 means this call's own 4xx/5xx
-- answer lives under "${key}#eph:${raw % 1_000_000_000}" instead --
-- the low digits of the reply are that row's own nonce, not a window
-- size (kind 1's use of the same slot), so idempotent.wo can rebuild
-- the exact key and read only ever what THIS call produced. Never 0:
-- idempotent.wo's own catch-and-log-nil handler cannot tell a literal 0
-- reply apart from a trapped call, so the encoding avoids it on
-- purpose. A trapped call (a saturated mailbox) propagates to the
-- caller uncaught, same as pool_count -- the middleware's own
-- try/catch answers 503.
pub fn pool_begin(pool: Pool, key: Text, digest: Text, ttl: Int, req: Req, handler: Handler) -> Int {
let a = pool_select(pool, key);
return call(a, PoolMsg {
kind: 2, key: key, limit: 0, window: ttl,
digest: digest, req: fresh_req(req), handler: handler
});
}
-- NOTE: pool_begin() lived here — the accessor idempotent.wo called to run a
-- request through the actor. Reverted with the kind-2 arm; whole in the tag
-- archive/porch-idempotency.

21
docs/examples/porch/middleware/store.wo
View file

@ -12,6 +12,27 @@ class RateLimitCounter {
window: Int
}
-- ============================================================================
-- KEPT DELIBERATELY, UNUSED TODAY.
--
-- Nothing in porch reads or writes this table right now. The idempotency
-- middleware that did was reverted on 2026-08-30 — not because the design was
-- wrong (it was built, reviewed and works) but because it provoked a C-runtime
-- crash: a SIGSEGV in wo_arena_alloc / wo_str_new under concurrent
-- call()-parked callers allocating heavily inside an actor's receive. Over ten
-- gate runs every failure belonged to an idempotency leg and none to the rate
-- limiter's, which drives the same pool through the same machinery but
-- allocates a fifth as much.
--
-- The table stays because the schema is settled and re-adding it would be
-- churn, not design: `digest` as its own column (never folded into the key, or
-- "same key, different body" becomes undetectable) is the one decision that
-- cost a review round to get right. The middleware, its actor arm and its gate
-- legs are whole in the tag `archive/porch-idempotency`, which is also the
-- reproduction harness for the runtime bug.
--
-- If the runtime bug is fixed and idempotency is NOT resumed, delete this.
-- ============================================================================
-- Idempotency: stored response for replay.
-- Key format: "idem:keyheader" or "idem:keyheader:sha256(method|path|body)"
-- Response = JSON-encoded Resp {status, headers, body} (headers allowlist:

11
docs/superpowers/specs/2026-08-29-porch-store-backed-middleware-design.md
View file

@ -1,5 +1,16 @@
# porch 1 — store-backed middleware: rate limiting and idempotency
> **Status 2026-08-30 — half of this spec shipped, half is on hold.**
> The rate limiter landed and is stable. Idempotency was implemented against
> this design, passed review, and was then reverted whole to the tag
> `archive/porch-idempotency` because it provokes a C-runtime SIGSEGV in
> `wo_arena_alloc`/`wo_str_new`
> ([language 41](../../stories/language-runtime-database/41-actor-arena-crash.md)).
> **Nothing below is retracted** — the design is sound and was proven by a
> working implementation. It is waiting on the runtime, and it is tracked as
> [porch 9](../../stories/porch/09-idempotent-replay.md).
Design settled 2026-08-29. Implements
[porch 1](../../stories/porch/01-store-backed-middleware.md).

638
scripts/web-app-accept.sh Executable file → Normal file
View file

@ -736,644 +736,6 @@ else
bad "limiter-compile" "$(printf '%s' "$lp_out" | head -1)"
fi
# ---- 18. porch-store task 4: idempotency runs the handler inside the actor --
# Same flattening trick as the keypool/limiter legs. Idempotent wraps a
# SLOW route handler (500ms) so two genuinely-parallel duplicates actually
# overlap in the pool's mailbox. The handler's side effect (ExecMark) is
# a real @table row count read back over GET /execs -- never a log line,
# per the brief. One server serves all three legs with distinct keys, so
# the exec count accumulates 1 -> 2 -> 3 across them.
IP="$W/idempotent-check"
cp -r "$ROOT/docs/examples/porch" "$IP"
rm -f "$IP/wo.toml"
rm -rf "$IP/target"
cat >"$IP/idempotent_check_main.wo" <<'WOEOF'
use net
use env
use http
use router
use middleware
use time
@table(name: "exec_marks")
class ExecMark {
n: Int
}
-- a deliberately slow handler: the concurrency leg's workload. Records
-- one row per REAL execution so a duplicate that wrongly ran it too
-- shows up as a row-count of 2, never as a log line.
class SlowHandler {
fn handle(req: Req) -> Resp {
insert ExecMark { n: 1 };
let n = len(from e in ExecMark select e);
time.sleep(500);
return ok_json("{\"echo\":\"${req.body}\",\"exec\":${n}}");
}
}
class ExecCount {
fn handle(req: Req) -> Resp {
let n = len(from e in ExecMark select e);
return ok_json("{\"count\":${n}}");
}
}
@table(name: "flaky_marks")
class FlakyMark {
n: Int
}
-- reviewer finding, task 4 follow-up: a transient 5xx must not be cached
-- for the TTL -- fails on the first call, succeeds on every call after.
class FlakyHandler {
fn handle(req: Req) -> Resp {
let n = len(from f in FlakyMark select f);
insert FlakyMark { n: 1 };
if n == 0 { return server_error(); }
return ok_json("{\"ok\":true}");
}
}
-- second reviewer follow-up: the SAME fails-once handler and table,
-- separate from FlakyMark/FlakyHandler above so the sequential leg
-- (18d) can't consume the one-time failure this leg (18e) needs -- but
-- this time hit by two GENUINELY concurrent duplicates, to pin that
-- neither ever receives a replayed 5xx from the other's ephemeral row.
@table(name: "flaky_marks2")
class FlakyMark2 {
n: Int
}
class FlakyHandler2 {
fn handle(req: Req) -> Resp {
let n = len(from f in FlakyMark2 select f);
insert FlakyMark2 { n: 1 };
if n == 0 { return server_error(); }
return ok_json("{\"ok\":true}");
}
}
class FlakyCount2 {
fn handle(req: Req) -> Resp {
let n = len(from f in FlakyMark2 select f);
return ok_json("{\"count\":${n}}");
}
}
-- coordinator follow-up: ephemeral (4xx/5xx) rows must not accumulate.
-- Always fails, so every attempt against the SAME idempotency key is
-- its own ephemeral miss -- never durable, never a hit for the next one.
class AlwaysFailHandler {
fn handle(req: Req) -> Resp {
return server_error();
}
}
class IdemKeyCount {
fn handle(req: Req) -> Resp {
let n = len(from k in IdempotencyKey select k);
return ok_json("{\"count\":${n}}");
}
}
-- reviewer finding 1: the README's own documented registration
-- (`key_header: "Idempotency-Key"`, capitalised) against a wire header a
-- real client also sends capitalised. Its own table/handler so the exec
-- count is never confused with SlowHandler's.
@table(name: "casecheck_marks")
class CaseCheckMark {
n: Int
}
class CaseCheckHandler {
fn handle(req: Req) -> Resp {
insert CaseCheckMark { n: 1 };
let n = len(from c in CaseCheckMark select c);
return ok_json("{\"exec\":${n}}");
}
}
class CaseCheckCount {
fn handle(req: Req) -> Resp {
let n = len(from c in CaseCheckMark select c);
return ok_json("{\"count\":${n}}");
}
}
-- reviewer finding 3: include_body:false must still scope the digest by
-- method+path -- two distinct routes sharing one Idempotency-Key must
-- never let the second replay the first's response.
class PathAHandler {
fn handle(req: Req) -> Resp {
return ok_json("{\"route\":\"a\"}");
}
}
class PathBHandler {
fn handle(req: Req) -> Resp {
return ok_json("{\"route\":\"b\"}");
}
}
fn build_app(slot: actor PoolMsg) -> App {
let app = App { middleware: [], routes: [] };
let p = Pool { actors: [PoolSlot { a: slot }] };
app.post("/create", Idempotent { key_header: "idempotency-key", pool: p, inner: SlowHandler {} });
app.post("/flaky", Idempotent { key_header: "idempotency-key", pool: p, inner: FlakyHandler {} });
app.post("/flaky2", Idempotent { key_header: "idempotency-key", pool: p, inner: FlakyHandler2 {} });
app.post("/alwaysfail", Idempotent { key_header: "idempotency-key", pool: p, inner: AlwaysFailHandler {} });
app.post("/casecheck", Idempotent { key_header: "Idempotency-Key", pool: p, inner: CaseCheckHandler {} });
app.post("/patha", Idempotent { key_header: "idempotency-key", pool: p, inner: PathAHandler {}, include_body: false });
app.post("/pathb", Idempotent { key_header: "idempotency-key", pool: p, inner: PathBHandler {}, include_body: false });
app.get("/execs", ExecCount {});
app.get("/flaky2count", FlakyCount2 {});
app.get("/idemkeycount", IdemKeyCount {});
app.get("/casecheckcount", CaseCheckCount {});
return app;
}
class Conn { fd: net.Conn }
class ConnWorker {
slot: actor PoolMsg
fn receive(msg: Conn) {
let app = build_app(self.slot);
app.handle_conn(msg.fd, 5000, 5000);
}
}
fn main(args: multi Text) -> Int {
if len(args) < 1 {
print_err("usage: idempotent_check <port>");
return 2;
}
let port = parse_int(args[0]);
if port == nil { print_err("bad port"); return 2; }
let ka: actor PoolMsg = spawn KeyActor {};
let srv = net.listen("127.0.0.1", port);
print("listening on 127.0.0.1:${port}");
while true {
if env.stopping() { net.close(srv); return 0; }
let c = net.accept_dl(srv, 250);
if c != nil {
let w: actor Conn = spawn ConnWorker { slot: ka };
send(w, Conn { fd: c });
}
}
}
WOEOF
if ip_out="$("$WOC" --emit "$IP" -o "$IP/idempotent_check.wob" 2>&1)"; then
ok "idempotent: compiles (actor-run handler, digest, pool_begin)"
IPORT=$((PORT + 2))
IDATA="$W/idempotent-data"; mkdir -p "$IDATA"
printf '\n===== idempotent check — port %s =====\n' "$IPORT" >>"$SRVLOG"
LEGFROM=$(( $(wc -l < "$SRVLOG") + 1 ))
WO_DATA="$IDATA" "$WOVM" "$IP/idempotent_check.wob" "$IPORT" >>"$SRVLOG" 2>&1 &
SRV=$!
iwait_listen() {
for _ in $(seq 1 40); do
tail -n "+$LEGFROM" "$SRVLOG" 2>/dev/null | grep -q listening && return
sleep 0.1
done
}
iwait_listen
ipost() { # key body outfile -> prints STATUS, leaves the body in outfile
curl -s -o "$3" -w '%{http_code}' --max-time 5 -X POST \
-H "Host: a" -H "Idempotency-Key: $1" -H "Content-Type: text/plain" \
--data-binary "$2" "http://127.0.0.1:$IPORT/create"
}
iexecs() { # -> the ExecMark row count
curl -s --max-time 5 -H "Host: a" "http://127.0.0.1:$IPORT/execs" \
| grep -o '"count":[0-9]*' | cut -d: -f2
}
# ---- 18a. gate leg: replay is exact (brief step 8) ----------------------
s1="$(ipost leg8-key hello "$W/i8a.body")"
s2="$(ipost leg8-key hello "$W/i8b.body")"
[[ "$s1" == "200" && "$s2" == "200" ]] \
&& ok "idempotent: same key + same body both answer 200" \
|| bad "idempotent-replay-status" "s1=$s1 s2=$s2"
if cmp -s "$W/i8a.body" "$W/i8b.body"; then
ok "idempotent: replay is byte-identical"
else
bad "idempotent-replay-bytes" "$(cat "$W/i8a.body") != $(cat "$W/i8b.body")"
fi
ec="$(iexecs)"
[[ "$ec" == "1" ]] \
&& ok "idempotent: handler ran exactly once (ExecMark row count = 1)" \
|| bad "idempotent-replay-execs" "ExecMark count=$ec want 1"
# ---- 18b. gate leg: digest mismatch is 422 (brief step 9) ---------------
m1="$(ipost leg9-key bodyA "$W/i9a.body")"
m2="$(ipost leg9-key bodyB "$W/i9b.body")"
[[ "$m1" == "200" && "$m2" == "422" ]] \
&& ok "idempotent: same key + different body is 422, not 200" \
|| bad "idempotent-mismatch-status" "m1=$m1 m2=$m2"
if ! cmp -s "$W/i9a.body" "$W/i9b.body"; then
ok "idempotent: 422 body is the refusal, not the other request's response"
else
bad "idempotent-mismatch-bytes" "422 body equals the first request's stored response"
fi
ec="$(iexecs)"
[[ "$ec" == "2" ]] \
&& ok "idempotent: the refused request never ran the handler (ExecMark row count = 2)" \
|| bad "idempotent-mismatch-execs" "ExecMark count=$ec want 2"
# ---- 18c. gate leg: concurrent duplicates (brief step 10) ---------------
# Two backgrounded curl clients, launched together, hitting the SAME
# slow (500ms) handler through the SAME key -- a sequential version of
# this passes against the old before/after flow too and proves nothing.
t0=$(date +%s%3N)
( s="$(ipost leg10-key samebody "$W/i10a.body")"; echo "$s" >"$W/i10a.status" ) &
cc1=$!
( s="$(ipost leg10-key samebody "$W/i10b.body")"; echo "$s" >"$W/i10b.status" ) &
cc2=$!
wait "$cc1" "$cc2"
t1=$(date +%s%3N)
elapsed=$((t1 - t0))
cs1="$(cat "$W/i10a.status")"; cs2="$(cat "$W/i10b.status")"
[[ "$cs1" == "200" && "$cs2" == "200" ]] \
&& ok "idempotent concurrency: both parallel duplicates answer 200" \
|| bad "idempotent-cc-status" "cs1=$cs1 cs2=$cs2"
if cmp -s "$W/i10a.body" "$W/i10b.body"; then
ok "idempotent concurrency: both clients got the same response body"
else
bad "idempotent-cc-bytes" "$(cat "$W/i10a.body") != $(cat "$W/i10b.body")"
fi
[[ "$elapsed" -lt 900 ]] \
&& ok "idempotent concurrency: genuinely overlapped (${elapsed}ms, serial would be ~1000ms+)" \
|| bad "idempotent-cc-elapsed" "${elapsed}ms"
ec="$(iexecs)"
[[ "$ec" == "3" ]] \
&& ok "idempotent concurrency: exactly one execution despite 2 parallel duplicates (ExecMark row count = 3)" \
|| bad "idempotent-cc-execs" "ExecMark count=$ec want 3"
# ---- 18d. gate leg: a transient 5xx is never replayed (reviewer finding) --
# The miss path must persist only a 2xx/3xx response. FlakyHandler fails
# on its first-ever call and succeeds on every call after; hit twice with
# the SAME idempotency key, the answer must be 500 then 200 -- caching the
# 500 would make every retry fail for the rest of the TTL (default 24h),
# a worse outcome than no idempotency at all.
f1="$(curl -s -o "$W/i11a.body" -w '%{http_code}' --max-time 5 -X POST \
-H "Host: a" -H "Idempotency-Key: leg11-key" -H "Content-Type: text/plain" \
--data-binary "x" "http://127.0.0.1:$IPORT/flaky")"
f2="$(curl -s -o "$W/i11b.body" -w '%{http_code}' --max-time 5 -X POST \
-H "Host: a" -H "Idempotency-Key: leg11-key" -H "Content-Type: text/plain" \
--data-binary "x" "http://127.0.0.1:$IPORT/flaky")"
[[ "$f1" == "500" && "$f2" == "200" ]] \
&& ok "idempotent: a transient 5xx is not replayed -- retry re-executes (500 then 200)" \
|| bad "idempotent-5xx-not-cached" "first=$f1 second=$f2 want 500 then 200"
# ---- 18e. gate leg: a 5xx is never replayed to a CONCURRENT duplicate ---
# (coordinator follow-up on 18d's residual). Two backgrounded clients,
# launched together, same key, against a handler that fails only its
# first-ever invocation. Whichever message the actor's mailbox happens
# to process first gets that real failure; the second message must find
# the row ephemeral and re-run the handler itself -- never read back a
# replayed 500. Which of the two clients goes first is a race this test
# cannot pin, so it asserts the UNORDERED outcome instead: the statuses
# are exactly one 500 and one 200 (both requests genuinely executed --
# FlakyMark2 count = 2). The pre-fix behavior (middleware-side delete,
# racing the actor) would show 500 and 500 with count = 1 whenever the
# duplicate is dequeued before the owner's delete lands -- deterministic
# either way, no ordering assumption needed.
( s="$(curl -s -o "$W/i12a.body" -w '%{http_code}' --max-time 5 -X POST \
-H "Host: a" -H "Idempotency-Key: leg12-key" -H "Content-Type: text/plain" \
--data-binary "x" "http://127.0.0.1:$IPORT/flaky2")"; echo "$s" >"$W/i12a.status" ) &
cf1=$!
( s="$(curl -s -o "$W/i12b.body" -w '%{http_code}' --max-time 5 -X POST \
-H "Host: a" -H "Idempotency-Key: leg12-key" -H "Content-Type: text/plain" \
--data-binary "x" "http://127.0.0.1:$IPORT/flaky2")"; echo "$s" >"$W/i12b.status" ) &
cf2=$!
wait "$cf1" "$cf2"
g1="$(cat "$W/i12a.status")"; g2="$(cat "$W/i12b.status")"
gsorted="$(printf '%s\n%s\n' "$g1" "$g2" | sort | tr '\n' ' ')"
[[ "$gsorted" == "200 500 " ]] \
&& ok "idempotent: concurrent duplicates never replay a 5xx (one 500, one 200)" \
|| bad "idempotent-5xx-concurrent" "g1=$g1 g2=$g2 want one 500 and one 200"
fc2="$(curl -s --max-time 5 -H "Host: a" "http://127.0.0.1:$IPORT/flaky2count" \
| grep -o '"count":[0-9]*' | cut -d: -f2)"
[[ "$fc2" == "2" ]] \
&& ok "idempotent: both concurrent attempts genuinely executed (FlakyMark2 count = 2)" \
|| bad "idempotent-5xx-concurrent-execs" "FlakyMark2 count=$fc2 want 2"
# ---- 18f. gate leg: ephemeral rows do not accumulate (coordinator follow-up) --
# AlwaysFailHandler fails every time, so N attempts against the SAME
# idempotency key are N separate ephemeral misses -- never a durable
# row, never a hit for the next one. Before the fix each attempt left
# its own permanent, nonce-keyed row behind; after it, idempotent.wo
# deletes that row the instant it reads it back (safe: the nonce is
# never handed to anyone else, so nothing else could ever address that
# row anyway). The IdempotencyKey row count must return to its
# baseline after all N attempts, not grow by N.
idemkeycount() {
curl -s --max-time 5 -H "Host: a" "http://127.0.0.1:$IPORT/idemkeycount" \
| grep -o '"count":[0-9]*' | cut -d: -f2
}
ik_baseline="$(idemkeycount)"
IK_N=3
for i in $(seq 1 $IK_N); do
curl -s -o /dev/null --max-time 5 -X POST -H "Host: a" \
-H "Idempotency-Key: leg13-key" -H "Content-Type: text/plain" \
--data-binary "x" "http://127.0.0.1:$IPORT/alwaysfail"
done
ik_after="$(idemkeycount)"
[[ "$ik_after" == "$ik_baseline" ]] \
&& ok "idempotent: $IK_N ephemeral attempts leave no rows behind (IdempotencyKey count stays $ik_baseline)" \
|| bad "idempotent-ephemeral-leak" "baseline=$ik_baseline after $IK_N attempts=$ik_after"
kill -TERM "$SRV" 2>/dev/null
istopped=1
for _ in $(seq 1 30); do kill -0 "$SRV" 2>/dev/null || { istopped=0; break; }; sleep 0.1; done
[[ $istopped -eq 0 ]] && ok "idempotent: SIGTERM stops the server" || bad "idempotent-stop" "still running"
if [[ $istopped -eq 1 ]]; then
# §14/§17b's own pattern clears SRV here unconditionally, which is
# exactly how an orphan survives past this leg: the EXIT trap only
# kills a non-empty $SRV, so a still-running process that this loop
# gave up on would otherwise keep the port bound for the NEXT run of
# this whole script. Force it dead right here instead of trusting the
# trap -- the bad-verdict line above already told the reader SIGTERM
# alone did not work.
kill -9 "$SRV" 2>/dev/null
for _ in $(seq 1 20); do kill -0 "$SRV" 2>/dev/null || break; sleep 0.1; done
fi
SRV=""
# ---- 18g/18h: reviewer findings 1 and 3, on a FRESH restart -------------
# Same compiled binary, fresh WO_DATA, a new port -- not piled onto 18a-18f's
# already-loaded server. 18a-18f alone already spawn a dozen-plus per-
# connection actors on that one process; adding these two legs there measurably
# raised how often this run hit the pre-existing, out-of-scope C-runtime
# arena-allocator race (a SIGSEGV inside wo_str_new, confirmed by gdb
# backtrace -- unrelated to this file's own .wo logic; see the story's
# Outstanding notes). A fresh process for these two legs keeps them just as
# rigorous while giving that race far less to chew on.
IPORT2=$((IPORT + 50))
IDATA2="$W/idempotent-data-2"; mkdir -p "$IDATA2"
printf '\n===== idempotent check — reviewer findings 1/3, port %s =====\n' "$IPORT2" >>"$SRVLOG"
LEGFROM=$(( $(wc -l < "$SRVLOG") + 1 ))
WO_DATA="$IDATA2" "$WOVM" "$IP/idempotent_check.wob" "$IPORT2" >>"$SRVLOG" 2>&1 &
SRV=$!
iwait_listen2() {
for _ in $(seq 1 40); do
tail -n "+$LEGFROM" "$SRVLOG" 2>/dev/null | grep -q listening && return
sleep 0.1
done
}
iwait_listen2
# ---- 18g. gate leg: README's own documented registration, over the wire
# a real client actually sends (reviewer finding 1) ----------------------
# key_header: "Idempotency-Key" (capitalised, exactly as the README told
# app authors to write it) against a wire header ALSO sent capitalised.
# parse.wo lowercases every header name on read, so a lookup on the
# unnormalised self.key_header always misses -- pre-fix this falls
# through to self.inner.handle(req) on EVERY call, so the second
# request is never deduplicated: the exec count is the load-bearing
# assertion, not the status (both calls answer 200 either way).
cc1="$(curl -s -o "$W/i14a.body" -w '%{http_code}' --max-time 5 -X POST \
-H "Host: a" -H "Idempotency-Key: leg14-key" -H "Content-Type: text/plain" \
--data-binary "x" "http://127.0.0.1:$IPORT2/casecheck")"
cc2="$(curl -s -o "$W/i14b.body" -w '%{http_code}' --max-time 5 -X POST \
-H "Host: a" -H "Idempotency-Key: leg14-key" -H "Content-Type: text/plain" \
--data-binary "x" "http://127.0.0.1:$IPORT2/casecheck")"
[[ "$cc1" == "200" && "$cc2" == "200" ]] \
&& ok "idempotent: capitalised key_header + capitalised wire header both answer 200" \
|| bad "idempotent-header-case-status" "cc1=$cc1 cc2=$cc2"
casecheckcount="$(curl -s --max-time 5 -H "Host: a" "http://127.0.0.1:$IPORT2/casecheckcount" \
| grep -o '"count":[0-9]*' | cut -d: -f2)"
[[ "$casecheckcount" == "1" ]] \
&& ok "idempotent: README's documented capitalised key_header still dedupes (CaseCheckMark count = 1)" \
|| bad "idempotent-header-case-execs" "CaseCheckMark count=$casecheckcount want 1"
# ---- 18h. gate leg: include_body:false must still scope by method+path
# (reviewer finding 3) ----------------------------------------------------
# Same Idempotency-Key against two DIFFERENT routes, include_body:false on
# both. Pre-fix the digest is "" unconditionally when include_body is
# false, so the second route's request matches the first route's stored
# bare-key row on digest alone and replays route a's body under route b.
# Post-fix, method+path are always part of the digest, so this is the
# SAME "different request, same key" case leg 18b already pins for a body
# mismatch -- route a succeeds (200), route b is refused (422), and
# route b's body must be the refusal, never route a's replayed body.
d1="$(curl -s -o "$W/i15a.body" -w '%{http_code}' --max-time 5 -X POST \
-H "Host: a" -H "Idempotency-Key: leg15-key" -H "Content-Type: text/plain" \
--data-binary "x" "http://127.0.0.1:$IPORT2/patha")"
d2="$(curl -s -o "$W/i15b.body" -w '%{http_code}' --max-time 5 -X POST \
-H "Host: a" -H "Idempotency-Key: leg15-key" -H "Content-Type: text/plain" \
--data-binary "x" "http://127.0.0.1:$IPORT2/pathb")"
[[ "$d1" == "200" && "$d2" == "422" ]] \
&& ok "idempotent: include_body:false, same key on two different routes: first 200, second refused (422)" \
|| bad "idempotent-diffpath-status" "d1=$d1 d2=$d2 want 200 then 422"
if grep -q '"route":"a"' "$W/i15b.body"; then
bad "idempotent-diffpath-replay" "pathb's response replayed patha's body: $(cat "$W/i15b.body")"
else
ok "idempotent: include_body:false does not replay a different route's response across paths"
fi
kill -TERM "$SRV" 2>/dev/null
istopped2=1
for _ in $(seq 1 30); do kill -0 "$SRV" 2>/dev/null || { istopped2=0; break; }; sleep 0.1; done
[[ $istopped2 -eq 0 ]] && ok "idempotent (findings 1/3): SIGTERM stops the server" || bad "idempotent-stop-2" "still running"
if [[ $istopped2 -eq 1 ]]; then
kill -9 "$SRV" 2>/dev/null
for _ in $(seq 1 20); do kill -0 "$SRV" 2>/dev/null || break; sleep 0.1; done
fi
SRV=""
else
bad "idempotent-compile" "$(printf '%s' "$ip_out" | head -1)"
fi
# ---- 19. porch-store task 5: pool saturation fails closed (503, no bypass) --
# Same flattening trick as the earlier legs. WO_MAILBOX (runtime/src/vm.c,
# wo_mailbox_cap, default 1024) shrinks the runtime's per-actor mailbox cap
# so a handful of concurrent requests can actually exhaust it. A pool of
# ONE actor -- the only address a one-slot Pool's pool_select can ever
# return -- fed a handler that blocks it for SP_SLEEP_MS turns every
# genuinely-concurrent request into a race for that one mailbox's slots.
# Distinct idempotency keys per request rule out replay masking a request
# that never actually ran the handler.
#
# The runtime frees a reserved slot the instant a message is POPPED for
# delivery, not when its receive returns (wo_mbox_reserve/release), so
# with cap C exactly the first C+1 concurrent calls to the one busy actor
# ever get a slot -- one executing, C queued behind it -- and every later
# concurrent call finds the mailbox full and traps (WO_T_ACTOR), which
# idempotent.wo's own try/catch turns into 503. SP_SLEEP_MS only has to
# outlast the time it takes SP_N curl clients to all reach their `call`,
# comfortably true on localhost.
SP="$W/saturation-check"
cp -r "$ROOT/docs/examples/porch" "$SP"
rm -f "$SP/wo.toml"
rm -rf "$SP/target"
cat >"$SP/saturation_check_main.wo" <<'WOEOF'
use net
use env
use http
use router
use middleware
use time
@table(name: "sat_execs")
class SatMark {
n: Int
}
-- Blocks the pool's one actor for a few seconds on every genuine
-- (non-replay) execution -- the same shape as idempotent-check's
-- SlowHandler (section 18), its own table so the two legs' counts can
-- never be confused.
class SlowSatHandler {
fn handle(req: Req) -> Resp {
insert SatMark { n: 1 };
time.sleep(3000);
return ok_json("{\"ok\":true}");
}
}
class SatExecCount {
fn handle(req: Req) -> Resp {
let n = len(from e in SatMark select e);
return ok_json("{\"count\":${n}}");
}
}
fn build_app(slot: actor PoolMsg) -> App {
let app = App { middleware: [], routes: [] };
let p = Pool { actors: [PoolSlot { a: slot }] };
app.post("/slow", Idempotent { key_header: "idempotency-key", pool: p, inner: SlowSatHandler {} });
app.get("/execs", SatExecCount {});
return app;
}
class Conn { fd: net.Conn }
class ConnWorker {
slot: actor PoolMsg
fn receive(msg: Conn) {
let app = build_app(self.slot);
app.handle_conn(msg.fd, 8000, 8000);
}
}
fn main(args: multi Text) -> Int {
if len(args) < 1 {
print_err("usage: saturation_check <port>");
return 2;
}
let port = parse_int(args[0]);
if port == nil { print_err("bad port"); return 2; }
let ka: actor PoolMsg = spawn KeyActor {};
let srv = net.listen("127.0.0.1", port);
print("listening on 127.0.0.1:${port}");
while true {
if env.stopping() { net.close(srv); return 0; }
let c = net.accept_dl(srv, 250);
if c != nil {
let w: actor Conn = spawn ConnWorker { slot: ka };
send(w, Conn { fd: c });
}
}
}
WOEOF
if sp_out="$("$WOC" --emit "$SP" -o "$SP/saturation_check.wob" 2>&1)"; then
ok "saturation: compiles (one-actor pool, slow in-actor handler)"
SPORT=$((PORT + 3))
SPDATA="$W/saturation-data"; mkdir -p "$SPDATA"
SPSTATUS="$W/saturation-status"; mkdir -p "$SPSTATUS"
SP_CAP=2
SP_N=15
printf '\n===== saturation check — port %s (WO_MAILBOX=%s) =====\n' "$SPORT" "$SP_CAP" >>"$SRVLOG"
LEGFROM=$(( $(wc -l < "$SRVLOG") + 1 ))
WO_DATA="$SPDATA" WO_MAILBOX="$SP_CAP" "$WOVM" "$SP/saturation_check.wob" "$SPORT" >>"$SRVLOG" 2>&1 &
SRV=$!
spwait_listen() {
for _ in $(seq 1 40); do
tail -n "+$LEGFROM" "$SRVLOG" 2>/dev/null | grep -q listening && return
sleep 0.1
done
}
spwait_listen
# SP_N genuinely-parallel duplicates, each its own idempotency key, all
# against the SAME (one-actor) pool -- a sequential version proves nothing,
# same reasoning as every other concurrency leg in this file.
sp_pids=()
for i in $(seq 1 $SP_N); do
( st="$(curl -s -D "$SPSTATUS/$i.hdr" -o "$SPSTATUS/$i.body" -w '%{http_code}' --max-time 15 -X POST \
-H "Host: a" -H "Idempotency-Key: sat-key-$i" -H "Content-Type: text/plain" \
--data-binary "x" "http://127.0.0.1:$SPORT/slow")"
echo "$st" >"$SPSTATUS/$i.status" ) &
sp_pids+=("$!")
done
for p in "${sp_pids[@]}"; do wait "$p"; done
sp_200=0
sp_503=0
sp_other=0
sp_one503=""
for i in $(seq 1 $SP_N); do
st="$(cat "$SPSTATUS/$i.status" 2>/dev/null)"
case "$st" in
200) sp_200=$((sp_200 + 1)) ;;
503) sp_503=$((sp_503 + 1)); sp_one503="$i" ;;
*) sp_other=$((sp_other + 1)) ;;
esac
done
sp_want_ok=$((SP_CAP + 1))
sp_want_bad=$((SP_N - sp_want_ok))
[[ "$sp_other" -eq 0 ]] \
&& ok "saturation: every one of $SP_N requests answered 200 or 503, nothing else" \
|| bad "saturation-codes" "$sp_other requests answered neither (200=$sp_200 503=$sp_503)"
[[ "$sp_200" -eq "$sp_want_ok" && "$sp_503" -eq "$sp_want_bad" ]] \
&& ok "saturation: exactly $sp_want_ok served (1 running + $SP_CAP queued), $sp_want_bad overflow answer 503" \
|| bad "saturation-threshold" "200=$sp_200 503=$sp_503 want 200=$sp_want_ok 503=$sp_want_bad"
sp_execs="$(curl -s --max-time 5 -H "Host: a" "http://127.0.0.1:$SPORT/execs" \
| grep -o '"count":[0-9]*' | cut -d: -f2)"
[[ "$sp_execs" == "$sp_200" ]] \
&& ok "saturation: handler ran exactly once per 200 (SatMark count=$sp_execs) -- no overflow request slipped through uncounted" \
|| bad "saturation-execs" "SatMark count=$sp_execs want $sp_200 (== the 200 count)"
if [[ -n "$sp_one503" ]]; then
grep -qi '^retry-after:' "$SPSTATUS/$sp_one503.hdr" \
&& ok "saturation 503 carries Retry-After" \
|| bad "saturation-503-retry-after" "$(head -1 "$SPSTATUS/$sp_one503.hdr")"
grep -q 'idempotency store saturated' "$SPSTATUS/$sp_one503.body" \
&& ok "saturation 503 names the real cause (idempotency store saturated), not a generic failure" \
|| bad "saturation-503-body" "$(cat "$SPSTATUS/$sp_one503.body")"
else
bad "saturation-503-missing" "no 503 observed among $SP_N requests -- cannot verify overflow shape"
fi
# Teardown is deliberately NOT asserted pass/fail here (unlike the earlier
# legs' own SIGTERM checks): this leg's subject is saturation, not graceful
# shutdown -- already proven in §14 and (usually) §17b/§18. This exact
# workload -- many concurrent call()-parked callers against one busy actor
# doing real per-request table I/O -- is the sharpest known trigger for a
# pre-existing runtime defect (see the story's Outstanding notes): main()
# can return cleanly while the OS process itself hangs. Failing this leg
# over that already-documented, out-of-scope defect would be exactly the
# kind of flaky check that erodes trust in every other leg in this file, so
# it force-kills instead of asserting graceful-vs-forced.
kill -TERM "$SRV" 2>/dev/null
spstopped=1
for _ in $(seq 1 30); do kill -0 "$SRV" 2>/dev/null || { spstopped=0; break; }; sleep 0.1; done
if [[ $spstopped -eq 1 ]]; then
kill -9 "$SRV" 2>/dev/null
for _ in $(seq 1 20); do kill -0 "$SRV" 2>/dev/null || break; sleep 0.1; done
fi
note "saturation: server torn down (graceful SIGTERM, or kill -9 on the known actor-pool hang)"
SRV=""
else
bad "saturation-compile" "$(printf '%s' "$sp_out" | head -1)"
fi
echo
printf 'web-app-accept: %d checks, %d failures\n' "$((pass + fail))" "$fail"