- normalise self.key_header via to_lower before the req.headers lookup
- req.headers keys are already lowercased on read (internal/parse.wo);
the documented key_header: "Idempotency-Key" never matched, silently
disabling idempotency (falls through to inner.handle) on every request
- key/digest lookups use the normalised name consistently
- digest now always includes method+path, body appended only when
include_body is set -- a bare "" digest under include_body:false
previously matched any other request reusing the same key
- log a genuine pool_begin trap instead of silently folding it into 503
- update the two doc comments describing the old, unsafe shape
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 91099cfbb2fb9a2d351894e444fed306b25557d5)
- Add saturation leg (scripts/web-app-accept.sh): one-actor pool,
WO_MAILBOX=2, 15 concurrent requests, exactly 3 served + 12 answer
503; execution count matches the 200 count, retry-after + real
cause verified on the 503s
- Guard make_pool(n<1) by clamping in make_pool itself, not
pool_select's division -- that trap runs inside the middleware's
own try/catch and would be swallowed as ordinary saturation forever
- README: rate limiting + idempotency ledger rows moved to done,
scoped to what the gate proves; documented Handler-decorator
shape, Pool aliasing (WO-E222), call's scalar-only reply (WO-E226),
pool size as a capacity decision
- Story: Progress table filled with real hashes, 7/9 acceptance
criteria marked verified with citations, 2 marked verified by
construction (never gated even in the original plan), status: done
- Status board: standup entry, porch 1 pending row updated
- Recorded a pre-existing runtime hang (main() returns cleanly, OS
process sometimes hangs under concurrent call()-parked callers)
that also reaches the new leg's teardown; contained with kill -9
rather than asserted, so it can't flake the leg's actual subject
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 21934b18910070a3f24b8bd4367fcb9d397dc1fb)
- The nonce naming an ephemeral (4xx/5xx) row is handed to exactly one
call() reply and nowhere else -- no other message can ever construct
that key, so idempotent.wo deleting it right after building the Resp
is safe by construction (unlike the earlier shared bare-key row,
which a second message COULD reach and made deleting it racy)
- Closes the leak AND a real correctness edge: the nonce is
time.ticks() % 1_000_000_000, wrapping every ~1000s -- with rows kept
forever, a later failed attempt on the same key could land on the
same nonce and either collide with the unguarded insert or resurface
a stale replay, exactly what rounds 1/2 removed
- Gate leg 18f: N ephemeral attempts against the same key must return
IdempotencyKey's row count to baseline, not grow it by N -- confirmed
failing (baseline+N) against the pre-fix code, passing after
- N picked at 3: the pre-existing runtime hang/segfault (out of scope,
being tracked separately) reproduces more often at higher sequential
insert+delete volume against the same key; 3 stayed clean across
many runs while still proving the property precisely
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 9ad594748e01a665ccaf29733a48c2b83a2749da)
- Root cause of the residual: a 4xx/5xx row lived under the bare key,
so a second message could delete-and-replace it before the FIRST
caller's own middleware-side read (necessarily outside receive,
WO-E226) ever ran -- the owner itself could read back a LATER
message's answer, not just a duplicate reading a stale one
- Fix: a 4xx/5xx miss is never stored under the bare key at all. Each
such attempt gets its own row, keyed by a nonce carried back in the
scalar reply's low digits, so no other message for the same bare key
ever touches it -- the decision AND the row's identity are both
fixed inside the one serialized receive call
- Disclosed trade-off: that row is never revisited by a bare-key
lookup, so it is never TTL-pruned either -- permanent per failed
attempt, the same no-sweeper trade-off this codebase already makes
elsewhere, not a new one
- Gate leg 18e: reran 20x in isolation against the fix with zero
500-500 or 200-200 outcomes (was reproducible before)
- §18's SIGTERM-stop check now force-kills on timeout before clearing
$SRV, instead of matching §14/§17b's own gap where a still-running
process escapes the exit trap too -- an orphan no longer survives
past this leg regardless of the assertion's own outcome
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 464147a9ddf3a53cb1946637c3f517ba615ee358)
- Miss path only marks a response a durable replay target (outcome 1)
when status is 2xx/3xx; a 4xx/5xx gets outcome 3 instead
- Outcome 3's row is a one-shot relay: the scalar reply still can't carry
a Resp (WO-E226), so the row exists only to hand the exact response
back once, then idempotent.wo deletes it -- a retry with the same key
is a genuine miss and re-executes, instead of caching a 500 for the
24h default TTL
- Reviewer finding: caching any status meant a transient failure was
replayed verbatim until TTL expiry, worse than no idempotency at all
- Gate leg 18d: FlakyHandler fails once then succeeds; same key twice
must answer 500 then 200 -- confirmed failing (500, 500) before the
fix, passing after
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit e61015f2065a7c6aec6f5c2439e78b53f796bab3)
- Delete before/after flow: it stored on a miss, so two duplicates both
missed and both ran; its 10s "in flight" check fired on fast legit
replays and never on a real collision
- Idempotent now wraps the route's Handler and hands request + handler to
the pool; a duplicate waits in the actor's mailbox, not a held reply
- keypool.wo kind-2 arm: digest match replays, mismatch refuses (422), a
miss runs the handler inside receive and stores status/body/
content-type, all via the same pool_pack(count, remaining_ms) scalar
kind-1 uses (WO-E226 forces one return type)
- Outcome codes start at 1, never 0: idempotent.wo's try/catch cannot
tell a literal 0 reply apart from a trapped call
- fresh_req() copies a borrowed Req's map fields into a new Req before it
crosses the actor boundary (WO-E222: aliased graphs can't cross heaps)
- Reading a stored row back forces fresh Text via `.. ""` on every field
copied out of json.decode's result -- decoded Text does not survive
being handed onward once the decoded record goes out of scope
- insert is unguarded (kind 1's own convention): a swallowed failure
would answer "stored" for a response never written
- web-app-accept.sh: leg 18a/b/c -- byte-identical replay off an ExecMark
row count, digest mismatch is 422, genuinely parallel duplicates run
the handler exactly once
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit eae1b06cdc38e4766d4e27a66b02264f47164e99)
- limiter_key: an empty client_ip(req) under trust_proxy no longer keys
on the literal "ip:" -- falls through to net.peer(req.conn) instead,
same as the untrusted-default path
- the bug: every client omitting X-Forwarded-For shared ONE bucket,
so one could exhaust it and deny/hide the rest
- curl availability check added alongside the existing woc/wovm check
(the limiter gate legs drive the server with it)
- new gate leg: LIMIT+1 sequential no-XFF requests must all be 200
(own key per connection, via a fresh ephemeral port each time) --
confirmed it fails against the pre-fix code (6th comes back 429)
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 831e9d8e6b1ef39cd938783dbc47c1abe6d53211)
- delete all RateLimitCounter access from limiter.wo: query, increment,
delete-then-insert, and the swallowing catch (e) nil -- the pool is now
the only writer, so its serialization guarantee actually holds
- Limiter gains pool/limit/trust_proxy fields; before() calls pool_count
and acts on the Verdict; make_limiter takes a pool
- key selection: req.principal first, else trust_proxy ? client_ip(req)
: net.peer(req.conn); delete the dead req.ctx["verified_proxy"] branch
- 429 on a spent window (Retry-After, X-RateLimit-*); 503 + Retry-After
on a caught actor trap (saturated pool), request never let through
- add Limiter.after(), registered alongside before() as both Mw and Aw
(Cors's own shape) so the allowed path's X-RateLimit-* headers reach
the response, not just req.ctx
- scripts/web-app-accept.sh: three new gate legs -- threshold (N pass,
N+1th 429), SIGTERM+restart (still limited from the WAL), and N
genuinely-parallel curl clients on one key with an exact-count assertion
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit a653dd0aa64711c43461126d32b4632cc8f64c7a)
- keypool.wo:78,89 passed msg.window (µs) straight into pool_pack's
remaining_ms (ms) parameter on the first-hit and post-prune-reset
paths; the third call site already divided by 1000 and was correct
- fix: pool_pack(1, msg.window / 1000) at both sites — a 60s window
no longer reports reset_at ~16.7h away
- count/allowed were unaffected (computed independently); this only
hit the client-visible reset instant, on the two most common cases
(new key, window rollover)
- extended gate leg 16 to assert reset_at falls within a 5s band of
time.now() + window_ms, not just on count — verified the assertion
itself by reverting the fix, confirming leg 16 failed with the
exact defect shape, then restoring it and confirming green
- woc docs/examples/porch/ exits 0; web-app-accept.sh: 47 checks,
0 failures
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 153fd295d37905dd083823536c6781843699e455)
- WO-E226: call's reply must be a copyable scalar, and every receive
program-wide must declare the same return type. Verified by fixture:
"call's reply type `Out` is not a copyable scalar"
- the spec had the actor return the response object, which cannot cross
the mailbox. Corrected: the actor stores the response and returns an
outcome code; the middleware reads the row and builds the Resp
- owner and duplicate now read the SAME durable row, so byte-identical
replay is structural rather than careful copying
- blocking, exactly-once execution and the mailbox queue are unchanged
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 77e06c1690b92d456a9bc53503695fdaa2b4b44e)
- add docs/examples/porch/middleware/keypool.wo: one PoolMsg (kind 1 =
count, kind 2 = begin placeholder for task 4), a Verdict class, a
fixed-size actor pool with a byte-sum-mod-N selector
- KeyActor.receive implements kind 1: reads the row, writes the new
count via field assignment (writes through, never delete+insert),
prunes a fully-elapsed window's row instead of resetting it
- window arithmetic on time.ticks(); reset instant sent back is built
from time.now() only
- call's reply must be a copyable scalar (WO-E226), so the count and
remaining window time are packed into one Int by the actor and
unpacked into Verdict by pool_count — the packing stays inside this
file, callers only ever see Verdict
- gate leg in scripts/web-app-accept.sh: a flat copy of porch (manifest
stripped) with a driver dropped beside keypool.wo asserts two
sequential counts return 1 then 2; verified failing (E403, make_pool
undeclared) before this file existed, passing after
- woc docs/examples/porch/ exits 0; full web-app-accept.sh: 47 checks,
0 failures
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 676e651d808ef2c3619f88c3808adc372cd0be6c)
- Add digest field to store sha256(method|path|body) separately from key
- Enables detection of "same key, different body" in future tasks
- Update idempotent.wo insert to compute and store digest value
- Typechecker passes: exit 0
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 3a9bddcd1e3c11f5b371ce54cafc373685ca08b6)
- one message class with a kind discriminator, not a receive per
message type: an actor handle is typed to one message class, so a
second receive compiles but is unreachable. chat/main.wo is the
precedent. Verified by fixture before amending
- Task 4's Files list omitted keypool.wo, which its step 4 edits
- clarified that the delete-then-insert ban targets using that pair as
an UPDATE; pruning an expired row is a plain delete and is required
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit a96ebe20e92e2dc6dfecc59a955d4c6e75d74689)
- the spec's blocking design was unimplementable: call's reply IS the
return value of receive, so an actor cannot hold a waiter. Holding
means never returning, and an actor that never returns cannot process
the completion it waits for — deadlock
- corrected shape: the actor RUNS the handler inside its own receive, so
a duplicate waits in the mailbox and is served after the owner. The
queue blocking needs is the mailbox; nothing is held
- verified before adopting it, not after: an actor can receive a message
carrying an interface-typed value and invoke it, so the route's
Handler passes through the mailbox
- spec History records the reasoning error — "the primitives landed" was
taken as "blocking needs no new surface", which does not follow
- plan: 5 tasks. Counting and replay live in one new keypool.wo; both
middlewares become thin key-choosers, so porch 2 and 3 inherit one
serialization convention instead of re-implementing it
- self-review added two legs it was missing: exact counting under real
concurrency (the criterion the pool exists for), and pruning an
elapsed limiter row rather than resetting it, which otherwise leaks a
row per IP ever seen
- plan is code-free per house convention; the writing-plans skill wants
code blocks and the project rule overrides it
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit f079455755a189a86bb12e07cc11549ed7a78b91)
- docs/examples/porch/ did not typecheck: WO-E403 "cannot resolve the
receiver's type for the call to `encode`" on json.encode
- every other example that calls json.encode/decode imports it; this
file did not, so the whole porch library was uncompilable on dev
- woc docs/examples/porch/ now exits 0
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 5c3544d524fa98dbb7a363600cd2eeb6dd1badac)
- supersedes Phases B and C as built: a store-after-completion
middleware cannot satisfy three of the story's seven criteria
- in-flight collision is undetectable (the row is written after the
handler ran, so concurrent duplicates both miss and both execute)
- the 10s in-flight heuristic is inverted: created_at is stamped at
store time, so it fires on legitimate fast replays and never on a
genuinely concurrent request
- "reused key, different body is refused" is unreachable while the
digest is folded into the key — nothing looks the bare key up
- design: sharded actor pool serializes per key, @table persists;
actors own volatile state, tables own durability. Inherited by
porch 2 and 3
- limiter joins the pool for exact counting, writes through instead of
delete+insert, keys on net.peer unless trust_proxy is declared, and
uses monotonic ticks for arithmetic but wall clock for the header
- idempotency blocks rather than answering 409: call parks the
duplicate until the owner reports. Digest becomes a column
- saturation fails closed with 503 for both: saturating the pool must
not become the limiter bypass
- records that the story's "time.after is still reserved" is stale;
spawn/send/call/monitor/time.after all landed
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit fc09e94373db65837ff5eb620fec67bab02c1931)
- Idempotent middleware (aee7926) added to the porch-store row
- records that Phase C is unverified: no `use json` import despite
calling json.decode and json.encode
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit aa8abfb4b7a1dabaa0c68afa0ee7fdfccf1b4689)
- replays a stored response for a repeated Idempotency-Key: before()
checks the key, after() stores status/body/content-type on a 2xx/3xx
- key is "idem:<header>:<value>", optionally plus a sha256 digest of
method|path|body when include_body is set
- 409 while a key is in flight (stored within the last 10s), lazy TTL
expiry on access, default 24h
- replay allowlists content-type only — never Set-Cookie or Date
- backed by IdempotencyKey from Phase A (519d411)
Written by a parallel session and committed here as-is because its
branch was consolidated away. NOT verified: it calls json.decode and
json.encode without a `use json` import, which every other example that
uses json has. Left unedited rather than fixed blind.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit aee79264095eea3b2c38c92789c44b31f1c9ef8a)
- porch-store and query-corpus prefixes registered; both replayed onto
dev, so dev is a superset of porch-store-middleware
- three branches could not be replayed and are preserved as annotated
tags rather than merged or discarded:
- cleanup/pre-existing-changes carries crates/ + Cargo.toml, the Rust
runtime master deleted; replaying it would resurrect it
- ipc-attach refactors wo_row_insert/wo_row_update_field into
encoded cores, which db2-keys rewrote for keys-residency — two
overlapping refactors of one function
- keypair-auth builds on ipc-attach, blocked by the same overlap
- names the specific hazard: 9c transfers ownership of vals on failure,
dev's keys-resident arm returns early without freeing, so a merge
that compiles and passes could still leak or double-free
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit bc8fec01d565d0ad54696fb3d2f85a9d1e0531a1)
- resolved 9g's forks EMPIRICALLY against the running compiler:
- count(<query>) and len(<query>) already work (fork 2 collapses to
zero code)
- skillhost's correlated NOT EXISTS is a backlink emptiness in
writeonce (`where len(x.children) == 0`), using only 9b machinery
(fork 1) — verified on a self-referential ?ref/backlink table
=> corpus #1 forces NO new grammar; per the method ("add only what a
corpus uses"), exists/not-exists was NOT built
- docs/examples/skill-catalog: mirrors skillhost's `skills` table
(name @unique, description/location/root, parent ?ref Skill, children
backlink) and translates all five of its SQL statements 1:1
(insert+dup-trap, get-by-name, list, roots via backlink-emptiness,
count); scripts/skill-catalog-accept.sh 7/0, WAL-durable, dup trap
persists across restart
- fixture run/db-query-corpus (count(query) + backlink NOT EXISTS);
just skill-catalog module; target/ gitignored
- general exists/not-exists left unbuilt and recorded as "enters when a
corpus forces a non-relation correlation"
- gates: oop-e2e 80/0, woc-test 566/0, skill-catalog 7/0; story + board
record the finding
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 4c82461634d45f11eca1a252702031c03d999c7f)
- story frontmatter status: done, Progress section records what landed
vs the spec (everything, same day as the brainstorm)
- board: NEXT PLAN entry with the six standup answers (deadlock proven
real: 5 s hang, 8192-byte truncation; 15 ms after; ping 2 ms during a
parked child; 1000 spawns fd-flat; SIGTERM leaves no child); pending
row flipped to DONE
- graph: node 42 class done, same change as the board row
- runtime/src/CODE-LOGIC.md: the bounded-subprocess section (bundle
park, slot registry, ownership sweeps, raw pidfd syscalls)
- full belt at close: 19 runtime suites 0 fail (test_proc 128/0),
woc-test 557/0, subprocess-accept 12/0, site-accept 23/0
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- docs/examples/subprocess: line-oriented TCP service, one Handler actor
per request — ping/run/slow/deadline/cap/long exercise the whole
bounded surface from .wo, traps caught with try/catch in the language
- scripts/subprocess-accept.sh + `just subprocess`: 12 checks, 0 failures
first run — deadline and cap messages verbatim, ping answered in 2 ms
while a sleep-2 child was parked, SIGTERM exit 0 with the sleep-30
child verifiably gone (pid checked from outside)
- service logs to /tmp/subprocess.log, banner-separated per run
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- seven commits dev to master, zero conflicts: the six db2-migrate
commits plus site-deploy, which the close-out edits and which had
been dev-only
- verified on master after the pick: 36 suites 0 fail (test_wal
5966/0), woc-test clean, residency-accept 14/0, site-accept 23/0
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- crash-before-rename test: a COMPLETE valid migrated temp beside the
untouched original is discarded and the boot re-migrates — the
sharpest point on the crash timeline, deterministic, no fault
injection needed
- story: all six tasks done with commit hashes, all eight criteria met
with the test that proves each, plus the three deviations from the
plan and why (transcode over replay, lazy head, poison forces
transcode)
- CODE-LOGIC: migration section; also corrected limitation 3, which
still claimed unbounded hot-row chains — iteration 11 closed that
- status board row 12; deploy guide's rollback section gets its real
answer (rolling back across a migration is a migration backwards:
expect the refusal, restore the .bak)
- test_wal 5966 pass, 0 fail
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 4bb6ece2531e2123eb958c91d9bef4a6528eab3b)
- brainstorm settled: declarative and automatic at boot; v1 verbs are
add and delete only; data/seed migrations deferred to v2
- added fields zero-fill by kind: the grammar has no field-default
syntax and v1 refuses to grow compiler surface for it
- same-kind delete+add refuses as a disguised rename; retype and
vanished classes refuse by name
- schema lives in the log itself: WO_WAL_SCHEMA head record, written by
fresh-log open and compaction; name-keyed diff also closes the
silent cid-renumbering hole
- story is iteration 12, board row added, db2-migrate prefix claimed
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 072e007b144ff6689b6ff920ea10665900c1a2ef)
- new docs/guides/deploying-site.md: build, content refresh, systemd
unit, post-deploy verification, rollback, and the gaps behind each
workaround
- leads with the trap that costs the most: shipping a binary does NOT
update chapters. seed_if_empty only fills an EMPTY table and
AdminEdit answers not_found for an unknown slug, so a host with an
existing WO_DATA shows the old chapter list with no error anywhere
- that claim is measured, not argued: a 9-chapter build seeded a data
dir, then the 10-chapter binary against it still 404'd /ch/storage
and rendered 9 nav entries; wiping WO_DATA gave 200 and 10
- records two more blockers found while writing it: both site deps
(porch, writeonce-view) 404 on GitHub and wo.lock is untracked, so
the site submodule cannot build standalone; and the embedded wovm
sets the glibc floor (this machine: 2.38, above Ubuntu 22.04's 2.35)
- build recipe run verbatim before publishing; releasing.md points here
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 930a715c4a3d847529e3e341edc71c65a7e11d1c)
- 4b56348 -> 7d9d526, 4eead89 -> 653a91c
- the registry rows for lang41, porch-store and query-corpus came with
the pick and are kept: the registry is a global claim ledger, so a
copy that silently omits three claimed prefixes is worse than one
that names them and says they are dev-only
- site-submodule row corrected on the way in — it said "Not picked to
master", which this pick is precisely what falsifies
- verified on master after the pick: site-accept 23 checks, 0 failures
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- records that docs/examples/site is now a submodule on dev only
- states the consequence plainly: master still carries the site inline,
so the branches differ structurally at that path until this is
cherry-picked
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 4eead8938c7292a130723aeabe7b74bdd1ba60f6)
- extracted to github.com/shoneyJ/writeonce-site with `git subtree
split`, so the site keeps its own 9 commits of history rather than
landing there as a flattened snapshot
- .gitmodules gains the third entry, alongside reference/writeonce-app
and reference/writeonce-api; path is unchanged, so every doc and
script that names docs/examples/site still resolves
- site-accept.sh fails early and says `git submodule update --init`
when the directory is empty. Without it a clone lacking submodules
copies an empty app and fails later as a build error naming nothing
- releasing.md: the steps that edit install/view.wo now say that edit
is a commit in the site repo plus a pointer bump here — editing and
committing only in this repo would record nothing
- gate re-run against the submodule: site-accept 23 checks, 0 failures
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 4b5634801d5379890bad24c8129cfb23ab6b98df)
- first cherry-pick under this convention: 37 commits, dev to master,
mapped one-to-one with titles
- iteration 11 could not travel alone — its commits touch
wo_wal_fold_row_at, keys_fold_into and row_apply_field_keys, none of
which existed on master, so the whole db2-keys/db2-delta stack came
- porch-store (26 commits) deliberately left on dev: porch 1 was
re-scoped mid-flight, which is what "ready, not merely green" is for
- records the three docs conflicts and how each was resolved, including
keeping only the databasev2 half of a status entry that would
otherwise have had master claiming porch 1 was done
- records what is still outstanding: task 6's byte-budget refusal, a
missing guard rather than an unhonoured annotation
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- records the workflow: develop on dev, feature prefix as the
conventional-commit scope, cherry-pick onto master when ready
- prefix registry so two features cannot claim the same prefix; the
prefix is claimed before the feature's first commit
- cherry-pick log maps dev hashes to the master hashes they produced —
they differ, and that mapping is what makes a feature traceable or
revertible as a unit after dev moves on
- notes the db2-keys seam: written pre-convention on
porch-store-middleware, replayed onto dev, replay verified identical
- work before 2026-08-29 landed by merge; git log --merges covers it
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 41923eb1f9510ab53804ca8dc6fe30a9a7eac849)
- new chapter 7, "Storage modes: durable and resident", covering what
master gains with the databasev2 cherry-pick: durable: false for a
RAM-only table, resident: keys for a table that outgrows RAM
- states the parts a reader would otherwise hit as surprises: a
keys-resident table is REFUSED at startup without WO_DATA, an update
appends a delta rather than rewriting the row, and the chain is
bounded at 16 links so a hot row does not degrade reads or replay
- quotes the measured 2.55x smaller resident set, not an estimate
- actors/deps/serving shift to ord 8/9/10; seeding is ord-driven so an
existing WO_DATA keeps its rows and only a fresh boot reseeds
- home card says a table can be RAM-only or outgrow RAM
- two gate legs pin the new chapter; site-accept 23 checks, 0 failures
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 3b503c0db50aa737dd06ef6d17151c654a42c59b)
- status: done in the story frontmatter (was the non-conventional
"complete"; the board's axis uses done/in-progress/pending/hold)
- board row 11: what landed, the WO_WAL_UPDATE correction, the ceiling
removed as unreachable, and the one criterion still weaker than
written (expected value, not a resident: all oracle)
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit de39a88e81e97bf6f8b75e9f15331aae20f7ab29)
- flattened row image is WO_WAL_UPDATE, not WO_WAL_INSERT: the row's
original INSERT is already in a live log, so a second one for the same
id is a duplicate replay refuses as corruption. INSERT is right only
for compaction, which builds a fresh log
- remove WO_CKPT_MAX_GARBAGE: with the absolute term at 64 MiB, garbage
large enough to reach a 256 MiB ceiling has already tripped it, so the
branch was unreachable. Postgres needs both constants because it
thresholds on tuples with its pair at opposite ends; this thresholds
on bytes, where one constant does both jobs
- test_delta_chain_flattens_at_k: chain depth stays <= WO_DELTA_MAX_HOPS
across 2K+2 updates, and a reset is observed
- test_delta_chain_flatten_replays: a flattened chain replays correctly
- test_keys_resident_indexed_across_flatten: a delta on an indexed
column composes with flattening, checked at every step across the
bound and after restart. Found no product defect
- test_should_compact_absolute_and_ceiling: pins the absolute term, the
boundary just under it, and the small-log case the ratio still governs
- test_wal 5700 pass / 0 fail; wovm-test and woc-test green
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit f93b5d9db753305c297e868d977670e6d703684c)
TESTS DELIBERATELY HELD at the developer's instruction — logic only.
The existing suite passes (36 suites, 0 fail) but exercises NEITHER new
behaviour: nothing builds a 16-deep chain, and no checkpoint test uses a
log near 64 MiB. Green here means "did not break what existed".
- tier 1: wo_wal_fold_row_at gains hops_out. The walk already visits
every hop, so the depth is free — this is the design's pd_prune_xid,
a cheap "is work worth doing" hint taken from work already happening
- the update path branches on it: past WO_DELTA_MAX_HOPS (16) it writes
a full-row image instead of a delta, terminating the chain. `r`
already holds the complete post-update row because index maintenance
required folding it, so flattening costs bytes, not an extra read
- wo_wal_append_row_image encodes from a caller-held row, as
WO_WAL_INSERT: a chain's base must replay into a database where
nothing precedes it, so replay/compaction/fold need no change
- tier 2: should_compact gains a TRIGGERING absolute term and a ceiling.
Our `floor` SUPPRESSES on a small log — the opposite of postgres's
vac_base_thresh, which triggers on a small absolute problem the
proportion hides. We had the proportion and the suppressor and
neither real guard
- verified by construction, not test: both update entry points converge
on row_apply_field_keys; db.c captures next_offset BEFORE calling in,
so the re-point is transparent to which record type was written
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 1b808abd5942de81c3a6416714d1302384103040)
- new `residency` leg in db-bench.py driving docs/examples/residency-bench:
two tables identical except the annotation, control cap + binding cap
- ITS OWN PROGRAM, not a db-bench mode: declaring a resident: keys table
is a WHOLE-PROGRAM constraint, so the no-WO_DATA refusal fires for
every mode in the module. Putting those classes in db-bench's shared
types made growth/ceiling/randread — which run without WO_DATA —
refuse to start. Caught by running the leg, not by reading it
- gates the RATIOS, waives the absolutes: ops/sec under a cap is swap
and disk I/O and belongs to the box. Same split randread makes
- rss_ratio 2.55 floor 2.0 tol 10% (structural, like bytes_per_row);
overcap_vs_swap_x 1.53 floor 1.0; in_ram_cost_x 4.23 ceiling 8.0;
all_collapse_x 105.4 floor 2.0
- all_collapse_x exists because the leg's FIRST run silently measured
nothing: at QUICK's 40k rows a 48 MiB cap binds neither mode, so the
"over-cap" half was not over cap. The cap now scales with N and the
leg asserts it binds
- verified the gate bites: rss_ratio 1.4, overcap_vs_swap_x 0.6 and
in_ram_cost_x 12.0 are all rejected
- task 7 closed: both criteria moved to Met with how each was verified
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit a310496664982c372f51b43113465eb8ad9e9fb5)
- hreadall/hreadkeys: the same resident A/B as wread_*, but ~2 KB per
row so a GB of data is reachable in a few hundred thousand inserts
- the insert path is fsync-bound at roughly 2 000 rows/s, so row COUNT
is the expensive axis and row SIZE is nearly free — 20k rows already
produce 38 MB
- NOT RUN: the GB-scale measurement was called off. These modes are
committed working and typechecking so the leg can be run later
without rebuilding it, not because a result exists
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit abc276ac39dd45a1052b6ae24d25aedb9eea3e1e)
- two tables identical except the annotation, 200k rows, 40k reads in
one key order, WAL on ext4 (not /tmp, which is tmpfs here and would
have put the log in RAM), rootless cgroup v2 cap
- WIDE shape, 2.55x smaller resident set: 34.4 MB vs 87.5 MB. That is
the real win and the thing the mode was built for
- under a 48 MB cap (between the two resident sets): keys 19635 ops/s
vs all 12854 — only 1.53x faster than letting the kernel swap
- degradation is far gentler though: all collapses 105x from its own
uncapped throughput, keys 16x
- costs 4.2x read throughput when memory is not tight, and writes are
markedly slower — the keys fill did not finish in 2 min where the
resident fill plus 40k reads did. No design doc had costed writes
- THE UNANTICIPATED FINDING: cgroup limits charge the PAGE CACHE, so
moving rows to a file does not escape a container memory limit. WAL
37 MB + RSS 34 MB cannot both live under a 48 MB cap, so every pread
reaches disk. The premise "the page cache will hold the hot rows"
fails in exactly the deployment this targets
- first attempt used Int-only rows and showed parity; recorded, because
drop_payload frees a field's VALUE and an Int's value is its inline
slot word, so that shape cannot benefit and would have condemned the
feature for the wrong reason
- verdict: keep it, to fit ~2.5x more data in given RAM — not to make
an over-capacity table fast
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 7cba9b1174b0bf581314b3147e25cc49e6f49464)
- fixes a limitation iteration 2 shipped: compaction was supposed to
bound chain length, but wo_wal_should_compact triggers on a whole-log
byte ratio and cannot see one hot row's chain
- tier 1, flatten on update: the update path ALREADY folds the row for
index maintenance and the fold already walks hop by hop, so it reports
depth for free. Past a fixed K it writes a full row instead of a
delta. Read <= K+1 reads, replay O(K^2) per row. No format change, no
per-row RAM, no new trigger
- tier 2: our compaction policy has a proportional term and a
SUPPRESSOR misleadingly called a floor; postgres's floor TRIGGERS on
small absolute garbage. Add that term and a ceiling
- design read from .dev/reference/postgresql, not recalled:
heap_page_prune_opt gates on an O(1) on-page hint then page fullness
against Max(fillfactor, BLCKSZ/10); autovacuum uses base + scale *
reltuples clamped by a max (50, 0.2, 1e8). Neither thresholds on
new-bytes-versus-old-bytes
- K deliberately does NOT scale with table size: postgres scales a
table-level aggregate with proportional harm, ours is per-row with
additive cost, so scaling up would make big databases boot worst
- the story says plainly it should NOT be next: task 7 has still never
measured whether resident: keys beats the kernel's own paging
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit f667cad2cfbe187b5973440ab1af015b2df288f8)
- explains replay, the row chain, how a checkpoint flattens it, and why
replay of a long chain is quadratic
- worked SKU example with the actual record layout and back-pointers,
and a trace of the fold showing first-seen-wins
- states plainly why the checkpoint does not bound the hot-row case:
both triggers are ratios over the whole log and nothing counts
per-row chain length
- records the bounded-memory vs linear-time conflict behind the O(N^2)
replay rather than presenting it as an oversight
- closes with the reviewing lesson, since this shape survived several
rounds: complexity bugs hide in the caller's loop, not in the linear
helper being read
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit e6434403d566b5d72a24e9c0dcad1f25a2c16320)
- "no storage behind it" / "nothing yet stores a table that way" was
false — CRUD, checkpoint survival and updates all landed; replaced
with an accurate summary naming task 6/7 as what remains
- the three checked delete/delete-replay/update criteria sat in
Outstanding despite being done; moved to Met, leaving Outstanding
holding only genuine task 6/7 work
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit b575678ee95fa3125fa4e8145320a9cdca10ba38)
- loader.c: delete the INCOMPLETE-update BAIL; durable:false +
resident:keys stays refused (nowhere to read from)
- table.c: root-cause fix for the Text-index gap — a keys-resident
borrow now holds ENGINE values, matching wo_row_ptr's contract
(table.h's "no VM pointer" doctrine), not a VM-decoded row. Fixes
idx_hash/idx_cols_equal/wo_idx_probe AND db.c's GET_FIELD/PROBE
arms with one change; reproduced pre-fix as an ASan
heap-buffer-overflow
- docs/examples/residency: Product is genuinely resident:keys;
residency-accept.sh's refusal leg replaced by proving the program
runs and stock survives a restart (11/0)
- test_wal.c: oracle test drives resident:all and resident:keys
through the same update sequence and asserts identical rows;
Text-indexed-update test catches the representation bug; five
pre-existing tests corrected to the fixed contract (4746/0)
- story, README, status board, CODE-LOGIC.md updated; three known
limitations documented: mid-drain stale reads, O(N^2) replay in
chain length, compaction blind to per-row chain length
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit b87c68f950f01aa5e572fbb86a0f374adc83d813)
- the plan placed the fold near wo_wal_read_row_at at "line ~600";
it is at line 794
- every other citation verified: wal.h:46, table.c:452/487,
db.c:88/289, wal.c:738, wal.c:861
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit c6cd48679e21e510696391b340c735aa0ad098f9)
- 6 tasks: the record kind, the fold, updates + indexes, the request
path and group commit, replay + compaction, then lifting the loader
refusal and proving it end to end
- the fold is written ONCE and called from three places; the tests are
arranged to prove each caller separately, and the plan says that
wanting a second fold "just for this caller" means the design failed
- Task 2 includes a same-field ordering test specifically, because a
fold walking the chain backwards the wrong way returns plausible data
and is otherwise invisible
- Task 6 step 1 audits the db.c request arms BEFORE lifting anything —
they were never audited for keys-residency the way the inline path
was, and the last audit of that kind found delete corrupting memory
- self-review found the spec's crash criterion had no task: added a step
that commits a delta, skips the re-point, and replays, which is the
state a crash between barrier and flush leaves behind
- every symbol the plan names verified to exist in database/src
- code-free per house convention; the writing-plans skill wants code
blocks and the project rule overrides it
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit abb8fc9454bdca9d8d703c8cfeb224f89a93d14d)
- updates append a DELTA (class, id, field, value, back-pointer), not a
full row. The workload decides it: a product catalogue changes one
narrow field of a wide row on every order, so a full-row append would
rewrite every field to move one integer on a shop's hottest path
- the back-pointer keeps the id map at one slot per row, which is the
mode's whole premise; a map growing per update would defeat it
- ONE fold function, three callers (read, replay, compaction). Three
implementations of one rule is how they drift, and a fold that differs
between reading and replaying is a database that changes its mind at
boot. Named as the design's principal risk
- indexed columns MAY change: price is exactly what a catalogue indexes,
so forbidding it would be a restriction users meet immediately
- no chain cap, deliberately. Compaction already rewrites live rows, so
every checkpoint resets every chain, and deltas grow the log which
pulls the next checkpoint forward — the workload that lengthens chains
triggers the fold that shortens them
- the risk that accepts: one hot SKU under an otherwise quiet write
rate. Task 7's benchmark must include it
- supersedes the 2026-08-26 spec's one-line full-row Update sketch,
marked in place rather than left as a second design in the tree
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit c9a88b05c4c62a5df13253686c990aaccc3f9cbb)
- the motivating workload was an audit log, which is append-only and so
argues for nothing. A catalogue is the real case: stable ids, and
stock moving on every order while name/price/sku sit still
- Product (durable, resident: all) and Cart (durable: false), with
place_order decrementing stock through a write-through field assign
- run `order` twice and stock goes 10 -> 7 -> 4: a level below the
seeded 10 can only mean an earlier order's UPDATE replayed. That is
the stronger claim — not just that inserts survive, but that a field
change does
- caught by running it three times: my first assertion required
before == 10, which only holds on a fresh seed and failed on the
third run even though the data was correct
- gate gains a leg for the update-replay claim; residency 12/0
- the commented resident: keys block now argues the DESIGN too: only
stock changes per sale, so appending the whole row would rewrite
every field to move one integer on a shop's hottest write path
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit d4104dcc5e3a460459dbf2d24043ce25b113bcdf)
