- Root cause of the residual: a 4xx/5xx row lived under the bare key, so a second message could delete-and-replace it before the FIRST caller's own middleware-side read (necessarily outside receive, WO-E226) ever ran -- the owner itself could read back a LATER message's answer, not just a duplicate reading a stale one - Fix: a 4xx/5xx miss is never stored under the bare key at all. Each such attempt gets its own row, keyed by a nonce carried back in the scalar reply's low digits, so no other message for the same bare key ever touches it -- the decision AND the row's identity are both fixed inside the one serialized receive call - Disclosed trade-off: that row is never revisited by a bare-key lookup, so it is never TTL-pruned either -- permanent per failed attempt, the same no-sweeper trade-off this codebase already makes elsewhere, not a new one - Gate leg 18e: reran 20x in isolation against the fix with zero 500-500 or 200-200 outcomes (was reproducible before) - §18's SIGTERM-stop check now force-kills on timeout before clearing $SRV, instead of matching §14/§17b's own gap where a still-running process escapes the exit trap too -- an orphan no longer survives past this leg regardless of the assertion's own outcome Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> (cherry picked from commit 464147a9ddf3a53cb1946637c3f517ba615ee358)
93 lines
4.5 KiB
Text
93 lines
4.5 KiB
Text
-- porch/middleware/idempotent.wo — idempotency, actor-run.
|
|
-- Iteration 1 of the porch track, porch-store task 4.
|
|
--
|
|
-- Rebuilt, not patched: the old before/after shape ran the handler and
|
|
-- stored the response afterward, so two simultaneous duplicates both
|
|
-- missed and both executed — before() has nothing to find until after()
|
|
-- runs, which is too late for the request that is racing it. The fix is
|
|
-- that the ACTOR runs the handler: Idempotent wraps the route's own
|
|
-- Handler and hands both the request and that handler to the pool. A
|
|
-- duplicate for the same key then simply waits in the actor's mailbox
|
|
-- (one message at a time) and is dequeued once the owner's receive has
|
|
-- already committed the row — no in-flight heuristic needed, because
|
|
-- there is no window where a duplicate can see "nothing yet".
|
|
--
|
|
-- `call`'s reply is a copyable scalar only (WO-E226): keypool.wo's kind-2
|
|
-- arm answers with the SAME pool_pack(count, remaining_ms) encoding kind-1
|
|
-- uses, never the response itself. The response travels through the
|
|
-- @table instead — the actor stores it, this file reads the same row
|
|
-- back and builds the Resp from it, so owner and duplicate answer with
|
|
-- byte-identical bytes structurally, not by careful bookkeeping.
|
|
use http
|
|
use json
|
|
|
|
-- Idempotent wraps a route's Handler. Registration: Idempotent { key_header:
|
|
-- "Idempotency-Key", pool: p, inner: CreateOrder {} } in place of the bare
|
|
-- handler in app.post(...). Only the digest allowlists content-type for
|
|
-- replay (never Set-Cookie, never Date) — cookies arrive in porch 2.
|
|
pub class Idempotent {
|
|
key_header: Text -- e.g., "Idempotency-Key"
|
|
pool: Pool
|
|
inner: Handler -- the real route handler; the actor runs this
|
|
include_body: Bool = true -- digest method+path+body, refuse a key reused with a different one
|
|
ttl: Int = 86_400_000_000 -- 24h in µs, lazy-expired on access
|
|
|
|
fn handle(req: Req) -> Resp {
|
|
let header_val = req.headers[self.key_header];
|
|
if header_val == nil { return self.inner.handle(req); }
|
|
|
|
let key = "idem:${self.key_header}:${header_val}";
|
|
let digest = "";
|
|
if self.include_body {
|
|
let digest_input = "${req.method}|${req.path}|${req.body}";
|
|
digest = base64_encode(bytes_slice(sha256(bytes_of_text(digest_input)), 0, 16));
|
|
}
|
|
|
|
let raw = try pool_begin(self.pool, key, digest, self.ttl, req, self.inner) catch (e) nil;
|
|
if raw == nil {
|
|
let r = Resp { status: 503, headers: {}, body: "{\"error\":\"idempotency store saturated\"}" };
|
|
set_header(r, "content-type", "application/json");
|
|
set_header(r, "retry-after", "1");
|
|
return r;
|
|
}
|
|
|
|
let outcome = raw / 1_000_000_000;
|
|
if outcome == 2 {
|
|
-- Same key, a different request: refuse rather than serve the
|
|
-- other request's response.
|
|
let r = Resp { status: 422, headers: {}, body: "{\"error\":\"idempotency key reused with a different request\"}" };
|
|
set_header(r, "content-type", "application/json");
|
|
return r;
|
|
}
|
|
|
|
-- Outcome 1: the bare key names a durable (2xx/3xx) replay target.
|
|
-- Outcome 3: this call's own 4xx/5xx answer lives under a row keyed
|
|
-- by a nonce (the reply's low digits) that nobody else's message
|
|
-- for this same bare key ever writes to -- rebuild that exact key
|
|
-- rather than reading the bare one, so a race with a LATER message
|
|
-- for this key (which never touches this row) can't hand back the
|
|
-- wrong response. This file never deletes either kind of row.
|
|
let lookup_key = key;
|
|
if outcome == 3 { lookup_key = "${key}#eph:${raw % 1_000_000_000}"; }
|
|
let hits = from k in IdempotencyKey where k.key == lookup_key take 1 select k;
|
|
if len(hits) == 0 { return server_error(); }
|
|
let stored = json.decode(hits[0].response) as IdempotentStoredResp;
|
|
if stored == nil { return server_error(); }
|
|
-- `.. ""` forces a fresh, independently-owned Text for every key/value
|
|
-- copied out of the decoded record: json.decode's Text values do not
|
|
-- survive being handed onward as-is once the decoded record itself
|
|
-- goes out of scope (a stale row read back corrupted mid-response
|
|
-- otherwise) — concat is documented to always allocate new owned text.
|
|
let hdrs: map<Text, Text> = {};
|
|
for k, v in stored.headers { hdrs[k .. ""] = v .. ""; }
|
|
return Resp { status: stored.status, headers: hdrs, body: stored.body .. "" };
|
|
}
|
|
}
|
|
|
|
-- JSON shape of IdempotencyKey.response. Allowlisted headers only:
|
|
-- content-type, never Set-Cookie or Date.
|
|
typedef IdempotentStoredResp = {
|
|
status: Int,
|
|
headers: map<Text, Text>,
|
|
body: Text
|
|
}
|