- crypto.c: x509_find_ext (generic extension walker) + wo_x509_basic_constraints
(cA / pathLenConstraint, absent => not a CA) + wo_x509_eku_serverauth_ok
(EKU absent, serverAuth, or anyEKU => usable; else not)
- wo_tls_verify_chain enforces decision 6: the leaf must be server-usable
(EKU), every server-sent issuer and the signing anchor must be a CA
(basicConstraints CA:TRUE) with a pathLenConstraint covering the
intermediates below it — stops a leaf masquerading as a CA
- gen_x509.py extended (folds in the wildcard leaf, adds EKU clientAuth-only,
EKU serverAuth, a non-CA intermediate + a leaf issued under it); vectors
regenerated
- KATs: extractors (test_crypto 104) + chain enforcement (test_tls 103) —
EKU serverAuth accepted, clientAuth-only rejected, leaf-under-non-CA
rejected though every signature verifies; existing chains still pass.
ASan/UBSan clean
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 3811418014c2c8d9bc3a9464256f52104261b1b9)
- wo_tls_verify_chain: leaf-first DER chain — each cert signed by the
next, the top trusted (equal to, or signed by, a trust anchor), the leaf
SAN matching host, every cert temporally valid. Any failure rejects;
no partial trust. Pure over the phase-D/E verifiers, so offline-testable
- KAT with the phase-E RSA + EC chains: leaf trusted via its issuing CA
anchor; wrong-anchor / wrong-host / expired / broken-link / no-anchor
all rejected; two-cert chain with a byte-equal root anchor; NULL host
skips the SAN check. test_tls 100 pass, ASan/UBSan clean
- remaining F3c-net (live-gated): CA-bundle PEM loader, random ephemeral,
the net.connect_tls builtin driving the sans-io driver over a real fd
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 9d40055108d301be32df0e1d4140c23446baa7c6)
- wo_x509_check_host: match a hostname against the cert subjectAltName
dNSNames (RFC 6125) — case-insensitive, single left-most wildcard that
covers exactly one label; no SAN => refused; no legacy CN fallback.
Completes the phase-E deferred hostname check (walks the [3] extensions)
- wo_tls_client_set_host + driver enforcement: with a host set, a leaf
whose SAN does not match is refused at the Certificate step (MITM
defense); unset skips the check (offline testing only, documented unsafe)
- KAT: exact/case-insensitive/mismatch, no-SAN refused, wildcard one-label
(not zero, not sub-label) via a wildcard-SAN cert; driver refuses the
RFC 8448 leaf (no SAN) once a host is set. test_crypto 95, test_tls 91,
ASan/UBSan clean
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 319ce8bfcf608030f958b36ab2c8f62fd76e1740)
- wo_tls_client: a pure state machine (no sockets). Caller frames
records; driver runs ClientHello->ServerHello->flight->Finished and
hands back bytes to send. Keeps all I/O out of the security-critical FSM
- start_with (inject CH + ephemeral priv), push_record, take_output,
encrypt/decrypt (application traffic keys). Handshake-message reassembly
across records; per-message transcript timing (CertVerify signs CH..Cert,
Finished MACs CH..CertVerify); constant-time Finished compare; every
failure lands in FAILED (no warn-and-continue)
- verifies server CertificateVerify (phase E+D) + server Finished, emits
the client Finished, switches to application keys
- KAT: whole handshake driven offline against the RFC 8448 record trace —
client Finished record byte-for-byte, first client app record
byte-for-byte, NewSessionTicket + server app data decrypt to plaintext,
tampered flight -> FAILED. test_tls 90 pass, ASan/UBSan clean
- SECURITY TODO before live use (documented in tls.h + story): chain walk
to a trust anchor + SAN/hostname match; random ephemeral for production
start; the net.connect_tls socket glue
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 74c332d7efdb8bbfdbe90bd2fcc3defa2fe8da00)
- wo_tls_verify_cert_verify: verifies a server CertificateVerify
(RFC 8446 §4.4.3) — builds the 64-space || context || 0x00 ||
transcript-hash content, parses the leaf SPKI (phase E) and dispatches
to phase-D RSA-PSS / RSA-PKCS1 / ECDSA-P256; the scheme must match the
leaf key type. ECDSA sig r/s pulled from its DER SEQ
- reuses wo_tls_finished_verify (phase F2) for server + client Finished
- KAT: the whole handshake crypto driven offline from the RFC 8448 §3
recorded messages — CertificateVerify (RSA-PSS) VALID, wrong-transcript
/ tampered-sig / mismatched-scheme rejected, server Finished byte-exact,
and the client Finished we would send byte-exact. test_tls 78 pass,
ASan/UBSan clean
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit afd9f23508c648322712df91329aa117c975ccab)
- new tls.c/tls.h on the crypto ladder: wo_tls_record_seal/open
(RFC 8446 §5.2) — TLSInnerPlaintext (content||type, no padding),
5-byte header as AEAD additional-data, per-record nonce = iv XOR
seq big-endian (§5.3)
- suite dispatch: TLS_AES_128_GCM_SHA256 (mandatory) +
TLS_CHACHA20_POLY1305_SHA256 (AES-NI-less fallback), over phase-A AEAD
- open() strips trailing zero padding to recover the inner content type;
rejects a length-field lie before the AEAD, and auth failure after
- KAT vs python AEAD oracle (test/gen_tls_record.py): sealed record
byte-for-byte both suites, open() recovers it, 5-seq round-trip,
tamper + wrong-seq + bad-suite rejected. test_tls 51 pass, ASan/UBSan
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 5021a99f8359f78a642bb0cc2b28ab66f6b624e2)
- defensive ASN.1/DER reader: every length/bound checked; malformation
is rejection, never over-read (truncated input KAT-gated)
- x509_parse: tbsCertificate span, sig-alg OID, signature,
SubjectPublicKeyInfo (RSA n/e or EC P-256 x/y), validity dates
- wo_x509_verify_one: one chain link's signature, dispatching to
phase-D RSA-PKCS1/PSS + ECDSA-P256 by the issuer key type
- wo_x509_parse_spki + wo_x509_check_validity (caller supplies time)
- KAT against real python-generated chains (test/gen_x509.py):
RSA CA+leaf (SHA256withRSA), EC P-256 CA+leaf (ecdsa-with-SHA256);
leaf-vs-CA, self-signed CA, wrong-issuer/tampered/truncated reject,
validity window, SPKI extraction. test_crypto 84 pass, ASan/UBSan clean
- deferred to phase F: SAN/hostname match + multi-cert chain walk to a
system CA bundle (both need the target host / trust store, known at
handshake time)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 4ec1c75f889df3612e31b9a1a77c4c927fb546c1)
- wo_ecdsa_p256_sha256_verify: NIST P-256 signature verify for EC-cert chains
and TLS 1.3 CertificateVerify
- Jacobian point arithmetic (double a=-3, general add with the H==0 special
cases), double-and-add scalar mult; field/scalar arithmetic reuses the
bignum Montgomery multiply and modexp (Fermat inverses mod p and mod n)
- validates r,s in [1,n-1] and that Q is on the curve (invalid-curve guard)
- verify-only public data -> not constant-time by design
- renamed the P-256 field mul to fpmul to avoid the clash with X25519's fmul
- VERIFIED against a python ECDSA-P256 vector; tamper + wrong-hash rejected;
test_crypto 69/0; ASan/UBSan clean; battery green
- phase D COMPLETE (RSA PKCS1+PSS + ECDSA-P256). Next E: ASN.1/X.509
(cherry picked from commit 92c996ba96d785d098c173d4ac6ace9052ef6a2f)
- wo_rsa_pkcs1_sha256_verify + wo_rsa_pss_sha256_verify (SHA-256), for the
server cert chain and TLS 1.3 CertificateVerify
- bignum: Montgomery multiply (CIOS, 64-bit limbs, __int128), modexp with the
public exponent (R^2 via 128k modular doublings, no division); MGF1-SHA256
- verification is public data only -> NOT constant-time by design (correct and
much simpler than a private-key op)
- assumes a full-length modulus for PSS emBits (standard RSA-2048/3072/4096)
- VERIFIED against python cryptography RSA-2048 vectors (PKCS#1 v1.5 + PSS,
salt 32); tamper + wrong-hash rejected; test_crypto 66/0; ASan/UBSan clean;
battery green
- internal C, no builtin/compiler change. Remaining in D: ECDSA-P256 (D2)
(cherry picked from commit 9118177fbfd03eff9757defea6931afdd68830c4)
- wo_x25519: constant-time Montgomery ladder + mask-based conditional swap,
radix-2^51 field arithmetic with __int128 products (curve25519-donna-c64,
public domain); scalar clamped, u-coord high bit masked per RFC 7748
- internal C (consumer is the TLS ECDHE handshake); no builtin/compiler change
- KAT-gated in test_crypto: RFC 7748 §5.2 both direct vectors AND the
1000-iteration base-point test; test_crypto 61/0; ASan/UBSan clean; battery green
- fixed one transcription bug found via the KAT: crecip needs 5 final squarings
(p-2 = 2^255-21 = (2^250-1)*2^5 + 11), not 3
- rv2 9 ladder: A (AEAD) + B (HKDF) + C (X25519) done; next D signatures/RSA
(cherry picked from commit f41b1c5f56caa841d1904382830baff0f75525d9)
- wo_hkdf_sha256_extract/expand (RFC 5869) + expand_label (RFC 8446 §7.1),
internal C over the existing hmac_sha256; SHA-256 (mandatory-suite hash;
SHA-384 a later add for the AES-256 suite)
- no builtin, no compiler change -- no .wo consumer yet (the TLS handshake
is the consumer); exposed for the C unit test
- KAT-gated in test_crypto: RFC 5869 Test Case 1 (PRK + 42-byte OKM) and
three HKDF-Expand-Label vectors (key/iv/derived-secret shape); 57/0,
ASan/UBSan clean; runtime battery green
- rv2 9 ladder: A (AEAD, = rv2 8) and B (HKDF) now done; next C X25519
(cherry picked from commit c8d27b6c89a80cd97a996ff7d8b64ff4895e4b26)
- no-intrinsics AES: S-box = GF(2^8) inverse via a fixed-exponent power ladder
(constant-time in the input, no tables), constant-time gf8_mul, byte-oriented
ShiftRows/MixColumns/key-expansion (AES-128 and AES-256)
- constant-time GHASH: bit-by-bit GF(2^128) multiply (mask-driven, no tables)
- aes_gcm_seal/open now dispatch: AES-NI path when present (and not forced
software), else this portable fallback -> AES-GCM works on ANY CPU, so the
phase-B no-AES-NI trap is retired
- wo_aes_force_software test hook; both hw and sw paths verified against NIST
SP 800-38D cases 4 (AES-128) and 16 (AES-256) byte-for-byte; test_crypto 48/0;
ASan/UBSan clean; full runtime battery green
- ARMv8 crypto-extension hardware path deferred (untestable on x86-64 host)
(cherry picked from commit dccf650899798401a9adac8489f34c85ed9304af)
- new builtin net.connect(host, port) -> Int: the outbound-socket gap
language 38 named and jarvis surfaced; the client half of the net verbs
- getaddrinfo for DNS (v4/v6, numeric or hostname), blocking connect with
the same EINTR/stop handling as net.connect_unix, then O_NONBLOCK for the
park plane; returns the same fd-scalar accept yields
- wob.h enum + WO_B_MAX 110; types.ml registration; loader.c arity;
builtin.c sysio dispatch range extended to WO_B_NET_CONNECT; sysio.c impl
- verified: numeric IP + hostname (DNS) connect to a local listener, closed
port traps cleanly; ASan-clean; runtime battery 0 fail
- deferred (next slice): net.connect_dl deadline/park variant (no shard
stall during handshake), on the accept_dl pattern
(cherry picked from commit 13c6f124428243b4956fbb4eceb1e7d0206d45f2)
- wo_schema_diff poisons a class (fmap=NULL, new_cid=NONE) when a
referenced/nested type changed or a field type is incompatible — the
field-level map does not apply and wo_wal_migrate transcodes it instead.
- main.c's pre-migration "migrating `X`: +/-fields" print loop dereferenced
fmap unconditionally, so a poisoned-but-field-added class (e.g. a wmux
Window whose nested Vte gained fields) was a NULL read → SIGSEGV at boot,
before the migrate call could refuse or transcode.
- guard the field detail on fmap != NULL; for a poisoned class print
"(a referenced type changed — cannot migrate in place)" and let
wo_wal_migrate proceed. It then transcodes cleanly when no record blocks
it — so an additive nested change (Vte +oscbuf +title) now migrates and
the session replays, instead of crashing serve.
- test_wal 5966/0; verified against the real WAL that crashed (recovers
session `main`); wmux gate 52/0.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 35efa214d20dd7b055910ae66e4bfcc6201821c9)
- five stories status: done; 00-story records the one-run landing
- spec History: three implementation amendments (Signal record not
scalar, caller-owned stdio fds, handler-latch instead of signalfd)
- board NEXT PLAN entry with measured findings (zero transport code
added; the tty-across-the-socket handover proven; the double-raw
refusal restoring the terminal — the "bug" that was the design
working); section rows flipped; graph nodes green
- CODE-LOGIC.md: the runtime-v2 section
- full belt quoted on the board: suites 0 fail both flavors (test_proc
193/0, test_term 60/0), woc 557/0, subprocess 12/0, site 23/0
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
(cherry picked from commit bc1b4f070693eb755ad6a9fd0c853fb3e2bda347)
- sendmsg/recvmsg with one SCM_RIGHTS fd and a sentinel byte; EAGAIN
parks in the net mould; the received fd arrives nonblocking as a plain
Int every fd verb accepts
- SO_DOMAIN gate: send_fd on anything but a unix socket refuses by name;
plain bytes deliver nil from recv_fd
- net.connect_unix carried here (iteration 38 still pending)
- legs (single-fiber: unix connect completes while the listener holds
the handshake): a pipe's read end crosses and still reads "ping"; a
tty crosses, term.raw works on the RECEIVED copy and destroy restores
it; refusal and nil legs verbatim. test_term 60/0
- full belt: all suites 0 fail both flavors, woc 557/0,
subprocess-accept 12/0, site-accept 23/0
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
(cherry picked from commit 1d689027920d6814f87b97c216b0cb42f7eba3e9)
- two verbs on any tty fd; saved termios in a per-shard 8-entry table;
double-raw and restore-without-save refuse by name
- restore is a RUNTIME obligation: vm_unwind at depth 0 (uncaught trap,
fiber reap) restores the dying fiber's entries newest-first, and
wo_vm_destroy sweeps the rest — proven twice in the legs: a DIV0
while raw restores, and even the double-raw REFUSAL (itself a trap)
restores the first raw
- test_term 39/0 against a real PTY pair made by the test
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
(cherry picked from commit b439387dbf4f653e034c721bc3f08b83616c5e24)
- mechanics amendment to the spec (recorded at close-out): no signalfd —
the stop-latch pattern generalized. An async-signal-safe handler
latches the number, bumps a sequence and pokes shard 0's wake eventfd;
wo_io_wait's loop head drains latches into fresh Signal{sig} records
delivered via runtime_notify (exported as wo_actor_notify)
- payloads must be heap objects (vm.c drops them unconditionally) — the
Signal record exists exactly for that; class id rides the call as the
appended record operand (sm_record drives it even with no return)
- offerable: WINCH/CHLD/HUP/USR1/USR2; SIGTERM/SIGINT refused naming the
stop latch; shard-0-only registration; coalescing disclosed
- stdlib_modules gains `signal` (and `term`, next task)
- test_term: a real child kills the test process with USR1; the actor's
multi holds one coalesced delivery; refusal leg verbatim. 14/0
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
(cherry picked from commit 14e03a6a4a343b97c9b47fab1a5e3c4bb69d8201)
- posix_openpt/grantpt/unlockpt/ptsname_r (plain libc, no -lutil); child
setsid + opens the slave as its controlling terminal, initial
TIOCSWINSZ from the call
- Child.stdin == Child.stdout = the master (caller's copy); the slot
keeps a private dup so resize survives the caller closing theirs
- proc.resize -> TIOCSWINSZ; refuses by name on a pipe child
- legs: test -t proves a real tty; stty size reads "24 80" then "40 120"
after a mid-sleep resize; refusal asserted; test_proc 193/0 ASan clean
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
(cherry picked from commit 9836c9cd5197153517c054b243cab3b453d130a0)
- a child is fds: Child {id, stdin, stdout, stderr}, driven by the
existing net verbs (echo leg proves cat round-trip through write_dl/
read_dl); caller owns the fds, the runtime owns pid + pidfd
- wait_dl parks on the pidfd: code on exit, nil at the deadline with the
child untouched; one waiter per id, a second refuses by name; stale
ids refused via a generation counter in the handle
- proc.signal through pidfd_send_signal; actor_die kills the streaming
children the dying actor owns; dead fibers cannot linger as waiters
- ids 97-107 registered wholesale (wob.h, loader arities, dispatch
bound); Child + Signal predeclared records in types.ml; unimplemented
ids trap at the default case until their task lands
- test_proc 168/0 (echo, wait trio, one-waiter refusal, 200-round churn
fd-flat), suite ASan clean, woc-test green
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
(cherry picked from commit 9be87f159f1bf9cdd509ceed160e7ea518fde46c)
- root cause: a worker's runtime is initialised lazily on first fiber
adoption, and rt.shard_id is stamped only there — but INBOX_READY[i]
is set at thread creation. A shard that never adopts is still settled
at shutdown, carrying rt.shard_id 0 from the memset
- it then impersonated shard 0: wo_drop_obj saw 0 == 0 for anything the
primary allocated, took the "we are home" branch instead of routing,
and called class_free against rt->classes, which lazy init never
filled. &rt->classes[class_id] off a NULL base is the faulting read
- fix: stamp the runtime's real identity at thread creation. An
uninitialised shard owns nothing, so its true id makes every payload
correctly foreign and routes it to an owner that can free it
- ASan could not name this: the arena is one hand-managed malloc block,
so intra-arena reuse is invisible and it surfaces as a bare SEGV
- pinned by tests/regress/lang-41, driven from db-actor-accept. Needs
multiple shards (the corpus runner pins WO_SHARDS=1) and the ASan
build. SEGVs twice per run unfixed, clean fixed
- the HANG is a separate defect and is NOT fixed: with this in place the
harness stops losing whole sections, but idempotent-stop-2 still
fires ~1 run in 6. The story records where to look
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 9dca0b4b4727b976d326b29cb4c6522b62d48a73)
- story frontmatter status: done, Progress section records what landed
vs the spec (everything, same day as the brainstorm)
- board: NEXT PLAN entry with the six standup answers (deadlock proven
real: 5 s hang, 8192-byte truncation; 15 ms after; ping 2 ms during a
parked child; 1000 spawns fd-flat; SIGTERM leaves no child); pending
row flipped to DONE
- graph: node 42 class done, same change as the board row
- runtime/src/CODE-LOGIC.md: the bounded-subprocess section (bundle
park, slot registry, ownership sweeps, raw pidfd syscalls)
- full belt at close: 19 runtime suites 0 fail (test_proc 128/0),
woc-test 557/0, subprocess-accept 12/0, site-accept 23/0
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- ceiling: 32 fibers hold live sleepers; the 33rd spawn traps WO_T_IO
naming the ceiling; destroy sweeps all 32 (waitpid -1 = ECHILD after)
- unwind: a fiber parked on a live child is reaped at main's return and
the child dies with it (nchildren 0 straight after the call)
- churn: one thousand sequential `true` runs through a bytecode loop —
fd count flat, every slot released
- stop: SIGTERM from a helper 200 ms into a sleep-10 child answers rc 1
(STOPPED) with no surviving child
- test_proc 128 pass 0 fail in 2.6 s, suite ASan clean
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- proc.run_dl reachable: dispatch range extended to id 96 (builtin.c) and
the loader arity table gains [WO_B_PROC_RUN_DL] = 6 — without both, the
builtin answered "unknown stdlib builtin" (WO_T_EXPLICIT)
- deadline leg: sleep 10 vs 100 ms deadline traps WO_T_IO naming the
deadline in ~120 ms; the pid is gone (waitpid -1 = ECHILD) and the fd
count is flat; a worker fiber completes WHILE main is parked — the
shard was never blocked
- cap legs: stdout and stderr caps trap naming "cap 1000", child dead
- argv multi carries a drop entry at the run pc: a trapping run frees it
(LeakSanitizer caught the miss)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- deadlock proven first: chatty child (200 KB stdout, stderr held open)
hung the old sequential drain 5.0 s into the alarm, code -1, stdout
truncated at 8192; the leg demands completion under 4 s
- rework: nonblocking pipe read ends + pidfd_open behind one epoll fd the
fiber parks on (the _dl retry mould); both pipes drain on readiness, so
the deadlock is gone structurally — leg passes in 15 ms
- wo_child slot table in wo_vm (32/shard) carries cross-park state; caps
refuse by name (kill + WO_T_IO), deadline armed via dl_active/dl_at,
defaults 30 s / 1 MiB / 64 KiB
- WO_B_PROC_RUN_DL = 96 shares the case (per-call deadline_ms/out_cap/
err_cap; compiler row lands in a later task)
- fib_reap kills a reaped fiber's child; wo_vm_destroy sweeps the table
- raw syscalls for pidfd_open/pidfd_send_signal: glibc 2.35 build floor
has no wrappers
- all 19 suites green under ASan+UBSan
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- new suite runtime/test/test_proc.c (auto-globbed by the Makefile)
- three legs against today's behavior: echo exits 0 with exact stdout,
false exits 1, a missing command answers 127 (the execvp convention)
- record fields copied out before the vm dies; ASan clean
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- crash-before-rename test: a COMPLETE valid migrated temp beside the
untouched original is discarded and the boot re-migrates — the
sharpest point on the crash timeline, deterministic, no fault
injection needed
- story: all six tasks done with commit hashes, all eight criteria met
with the test that proves each, plus the three deviations from the
plan and why (transcode over replay, lazy head, poison forces
transcode)
- CODE-LOGIC: migration section; also corrected limitation 3, which
still claimed unbounded hot-row chains — iteration 11 closed that
- status board row 12; deploy guide's rollback section gets its real
answer (rolling back across a migration is a migration backwards:
expect the refusal, restore the .bak)
- test_wal 5966 pass, 0 fail
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 4bb6ece2531e2123eb958c91d9bef4a6528eab3b)
- main.c builds the compiled schema (names out of the constant pool,
which the database layer never sees), peeks the log head before
replay, and diffs: match replays as-is, add/delete migrates through
the transcode, poisons refuse naming class, field and what to do
- fixed en route: a poisoned class SKIPPED the identity check, so no
transcode ran and replay greeted the shape mismatch with the generic
"corruption" — the exact message this iteration exists to replace.
A poison now forces the transcode, where it either bites with its
text or passes harmlessly when the class has no records
- the schema head is written LAZILY, ahead of the first real record:
an eager head broke the documented "durable: false writes ZERO
bytes" contract by 75 bytes and the residency gate caught it
- end-to-end at the language level: fresh boot seeds, identity
replays, +field migrates with "migrating `Note`: +flag" and reads 0,
retype refuses naming `val`, and the refused log still boots the
previous binary untouched
- gates: wovm-test all green (test_wal 5951/0), woc-test clean,
residency-accept 14/0, site-accept 23/0
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit b21943aa91152ebdcfc72bd4c9fba38730ab1c2f)
- wo_wal_migrate rewrites the log without touching db state: no id
maps, no indexes, no keys-resident logic — the new log replays
through the machinery that already exists and is already tested
- cids remap by name, INCLUDING the ones embedded inside stored owned
values (an owned value carries a cid on the wire); the embed closure
guarantees every nested class is shape-unchanged, so only numbers
move
- surviving fields go to their new slot, deleted fields' values are
freed, added fields take the kind's zero value straight from
enc_val(0)
- a delta on a deleted field is SPLICED out: an offset map (old record
start -> new) rewrites every back pointer, and the dropped delta maps
to its own target so later deltas step over it
- temp + fsync + rename, compaction's own crash discipline; a stale
temp is discarded at start; a torn tail bounds the intact prefix
exactly as replay does
- fixed en route: early `goto corrupt` jumped over initializers, so the
handler freed uninitialized memory — declarations hoisted above the
first jump
- six end-to-end tests: add, delete (ASan watches the freed Text),
reorder with owned fixup, delta splice on a keys-resident chain,
poison-bites-only-with-records, corrupt input
- test_wal 5951 pass, 0 fail
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit b69092a99206e1dcdf6f2dcdb02146939b0564c6)
- wo_schema_diff matches classes and fields by NAME, so declaration
reordering is identity apart from the cid map — the silent
cid-renumbering hole closes as a side effect
- owned-field references (fclass) compare by the NAME the number
resolves to, never the number: a raw compare would false-poison
retype on every pure reorder
- refusals are per-class POISONS carried in the plan, not diff errors:
a poison bites only when a record of the class is met, so a retyped
class with no stored rows never blocks a boot
- poison set: retype, same-shape delete+add (a disguised rename, one
reading destroys a column), vanished class, storage-flag change, and
the embed closure — any class whose old records carry values of a
class whose shape changed, iterated to a fixpoint
- identity plans skip the rewrite entirely; a NEW class in the binary
does not break identity (no records; the head refreshes at the next
compaction)
- ten verdict tests; test_wal 5778 pass, 0 fail
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 63a063b822af381a10c2e599c58cf3455d9c5bf7)
- new record kind 5: class and field NAMES, kinds and the two
encoding-relevant metadata words (field_class, field_elem), CRC-framed
like every record. Index layout deliberately absent: indexes rebuild
from rows at boot and never touch record bytes
- names are byte pointers, not constant-pool indices — the database
layer never sees the module's consts, so the runtime resolves them
once; a decoded schema owns a private copy of its bytes
- wo_wal_set_schema adopts the compiled schema; wo_wal_ensure_schema
writes it as a fresh log's first record; compaction writes it at the
head of every replacement, which is how a legacy log becomes
self-describing without a migration step of its own
- apply_record skips it BEFORE reading cid/id (its class count would be
misread as a cid and bounds-refused); replay does not count it
- schema unset = byte-for-byte today's behaviour: all 5700 prior
assertions pass untouched; four new tests cover roundtrip, fresh-log
head, legacy adoption via compaction, and absent/empty files
- test_wal 5743 pass, 0 fail
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit ba8519fa5e39c6ed6a3504d39e5a372f8decd045)
- flattened row image is WO_WAL_UPDATE, not WO_WAL_INSERT: the row's
original INSERT is already in a live log, so a second one for the same
id is a duplicate replay refuses as corruption. INSERT is right only
for compaction, which builds a fresh log
- remove WO_CKPT_MAX_GARBAGE: with the absolute term at 64 MiB, garbage
large enough to reach a 256 MiB ceiling has already tripped it, so the
branch was unreachable. Postgres needs both constants because it
thresholds on tuples with its pair at opposite ends; this thresholds
on bytes, where one constant does both jobs
- test_delta_chain_flattens_at_k: chain depth stays <= WO_DELTA_MAX_HOPS
across 2K+2 updates, and a reset is observed
- test_delta_chain_flatten_replays: a flattened chain replays correctly
- test_keys_resident_indexed_across_flatten: a delta on an indexed
column composes with flattening, checked at every step across the
bound and after restart. Found no product defect
- test_should_compact_absolute_and_ceiling: pins the absolute term, the
boundary just under it, and the small-log case the ratio still governs
- test_wal 5700 pass / 0 fail; wovm-test and woc-test green
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit f93b5d9db753305c297e868d977670e6d703684c)
- wo_row_borrow's keys arm folded at hget()'s DURABLE offset even
when an earlier update in the same drain had only a PENDING
re-point
- idx_remove_row then hashed the pre-first-update value, found no
matching bucket entry (already moved by the earlier update), and
idx_add_row added a second one — N same-drain updates leaked N-1
entries, unbounded, nothing reclaims them but a restart
- now prefers wo_wal_repoint_offset1() over the durable offset, same
as back_off already does, closing it for every borrow
- new test: 5 updates to one row in one drain, assert exactly one
index entry — fails (5) before the fix, passes (1) after
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit d4b12d1908e574419c7af2411e52d02623f7e5b7)
- loader stopped refusing durable:true+resident:keys once UPDATE
landed; nothing replaced it at runtime
- rows for such a table live only in the WAL, so every read failed
with a misleading "no such row" instead of naming the problem
- main.c now refuses at startup, names the class, exit(2)
- residency-accept.sh gains a leg: refuses without WO_DATA, still
runs with it — verified failing before the fix, passing after
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 3ea6d6452f260d45f92045c0f300fa49faa1d810)
- loader.c: delete the INCOMPLETE-update BAIL; durable:false +
resident:keys stays refused (nowhere to read from)
- table.c: root-cause fix for the Text-index gap — a keys-resident
borrow now holds ENGINE values, matching wo_row_ptr's contract
(table.h's "no VM pointer" doctrine), not a VM-decoded row. Fixes
idx_hash/idx_cols_equal/wo_idx_probe AND db.c's GET_FIELD/PROBE
arms with one change; reproduced pre-fix as an ASan
heap-buffer-overflow
- docs/examples/residency: Product is genuinely resident:keys;
residency-accept.sh's refusal leg replaced by proving the program
runs and stock survives a restart (11/0)
- test_wal.c: oracle test drives resident:all and resident:keys
through the same update sequence and asserts identical rows;
Text-indexed-update test catches the representation bug; five
pre-existing tests corrected to the fixed contract (4746/0)
- story, README, status board, CODE-LOGIC.md updated; three known
limitations documented: mid-drain stale reads, O(N^2) replay in
chain length, compaction blind to per-row chain length
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit b87c68f950f01aa5e572fbb86a0f374adc83d813)
- apply_delta: DELTA replay arm — fold pre-delta state via back_off,
overlay the field, remove-then-recreate so indexes stay correct
- apply_record/replay loop: dispatch DELTA to apply_delta, drop its
payload back to the log same as INSERT/UPDATE
- stage_flattened_row: compaction's delta-chain path — fold + re-encode
as one fresh INSERT instead of copying the chain
- wo_wal_compact: peek the row's current record kind, flatten deltas,
keep the byte-for-byte copy for chains already at length zero
- test_wal: three new tests — chain-of-three replay incl. secondary
index, compaction flattens to chain length zero (asserts the record
is a full row, not a delta), and the commit-before-repoint crash
window replays the update without ever re-pointing the map
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 7e4ae70d7beb2a2e396a002184bc06f41253decb)
- table.c: unique shadow-check's candidate lookup now checks
wo_wal_repoint_offset1 before the durable wo_row_offset1, same as
back_off — a candidate updated earlier in the SAME uncommitted drain
was folded from its stale pre-update offset, letting a real @unique
clash through and committing a duplicate
- the offset-only substitution alone was NOT enough (verified): the
candidate must be FOLDED to compare values, and folding a pending
offset via pread saw "no record" (bytes still only in the staging
buffer), so the clash was still missed, just for a different reason
- wal.c: wo_wal_fold_row_at now reads a hop inside the currently-staged
region from `w->buf` (new scan_record_staged, scan_record's framing
over memory) instead of pread; every durable hop, and every existing
caller, is unchanged
- test_wal.c: two updates in one drain where the second collides with
the first's new unique value; must be refused. Verified failing
against the prior commit, and still failing with only the offset
substitution, before the fold fix; passing with both in place
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit c049ab92570cfba4d12a018884a20b25a8916727)
- db.c: guard both WO_B_DB_UPDATE_FIELD arms on keys-resident tables —
wo_wal_append_update read a NULL wo_row_ptr there; a live crash, fixed
- ruling override on Task 3: row_apply_field_keys no longer commits or
moves the id map — stages the delta, does the index swap (RAM apply,
unconditional past the shadow-check; a stage failure past that point
is now fatal, like insert). Commit/re-point move to the caller,
mirroring insert. table.c's WAL commit removal is this ruling, not a
regression
- offset passed back via caller-side wo_wal_next_offset(), insert's
koff pattern
- inline arm commits then re-points; request arm records
wo_wal_pend_repoint (own list/name — a drop and a re-point differ),
flushed by wo_db_flush_drops after the barrier
- back_off checks the pending re-point before the durable offset, else
a second update in one drain skips the first delta; verified failing
this way, passing after
- wal.c: fixed a stale comment — keys-resident updates CAN reach
wo_wal_append_update's caller now, they just never call it
- test_wal.c: 2 tests updated for the new contract; new test drives 2
same-row updates via wo_row_update_field_slot in one uncommitted
"drain", checks the value and delta 2's on-disk back-pointer
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 4d13bcebfe51936ba8c608dd9e791c784e5b8983)
- row_apply_field_keys's shadow-check borrowed candidates via
wo_row_borrow, which shares ONE per-table scratch with the row already
borrowed for the update — every candidate borrow returned NULL, clash
was always false, `@unique` silently accepted duplicates on update
- idx_add_row's own internal check has the identical defect at the same
call site; discarding its result is now actually safe, since the fixed
shadow-check clears uniqueness before it ever runs
- fix: extracted keys_fold_into (fold+decode) out of wo_row_borrow so it
can target a throwaway per-call buffer instead of t->scratch; the
shadow-check probes candidates into that buffer — r is never
released-and-reborrowed (r IS t->scratch; that would overwrite it)
- wo_row_borrow itself is behavior-preserving: same checks, same order,
same messages, just factored
- test_wal.c: new test — genuine @unique index, update collides with an
existing row, asserts refusal (DB_ERR_UNIQUE) and both rows untouched;
verified failing (update wrongly succeeded) pre-fix, passing after
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 409186da51fec0042d8d1e3dcf9f69f704937823)
- table.c: wo_row_update_field/_slot no longer refuse `resident: keys`,
both converge on one new static row_apply_field_keys
- borrows (folds), shadow-checks uniqueness, appends the delta with the
row's current offset as back-pointer, commits, THEN idx_remove_row +
idx_add_row + wo_row_set_offset — failure through commit leaves the
row's offset and index untouched
- nv decoded to a VM value before touching the materialised copy, since
wo_row_release drops every slot through the runtime, not db_val_free
- borrow released on every exit, including every failure arm
- resident: all path (row_apply_field_slot) byte-for-byte unchanged;
wal.c untouched — Tasks 1/2 already expose everything needed
- test_wal.c: plain field update read back, and an indexed scalar
column updated then found via wo_idx_probe by its new value, gone
from its old — both verified failing pre-implementation, passing after
- concern: idx_hash/idx_cols_equal/wo_idx_probe cast Text slots to
db_text* unconditionally; a keys-resident borrow decodes Text to a VM
wo_str* (different layout) — pre-existing, left untouched; tests use
a scalar index to sidestep it
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 89c56a13ed9f4abd082bfcabb46f46bb53d39faa)
- wal.c: wo_wal_fold_row_at now refuses any delta back-pointer that
does not point strictly earlier than the record naming it
(back_off >= cur), instead of capping total hops at off/13+1
- this is the real invariant, not a proxy for it: a step-count bound
lets a forward-pointing back-pointer through in one hop whenever it
happens to land on a genuine record, returning a plausible-but-wrong
row instead of refusing it
- removes the 13-byte-record magic number entirely; no arithmetic
tied to record framing remains in the guard
- wal.h: docblock updated to describe the direction invariant
- test_wal.c: two new tests — self-pointing back-pointer (boundary
case, back_off == cur) and forward-pointing back-pointer to a real
future record for the same row (the actual gap: verified failing
against the old step-count guard, passing after the fix)
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 173dbf28a42d45a8c7f9fe430355f61f70c81935)
- wal.h/wal.c: wo_wal_fold_row_at — THE fold. Walks BACKWARD from an
offset through WO_WAL_DELTA records, remembering the first value
seen per field index (newest wins, since newest is seen first),
stops at the first INSERT/UPDATE, decodes it, overlays resolved
fields. Returns ENGINE-owned values so reads, replay, and
compaction (Tasks 3/5) can all build on the same output.
- Cycle guard: caps the walk at what the log up to the starting
offset could possibly hold (13 = scan_record's own record-size
floor), so a corrupt or malicious back-pointer fails loudly
instead of spinning.
- table.c: wo_row_borrow's keys arm now calls the fold instead of
wo_wal_read_row_at directly, then VM-decodes the result — same
two-stage pattern wo_wal_read_row_at used internally. Per-table
scratch, scratch_busy nested-borrow refusal, and the cid/id
identity check all preserved unchanged.
- resident: all path (wo_row_ptr) untouched.
- test_wal.c: two new tests — deltas on two different fields (changed
fields take the new value, the untouched field keeps its original)
and two deltas on the SAME field (the newer wins, pinning direction
— a reversed fold would pass with the older value instead).
Verified failing pre-implementation (wo_row_borrow returned NULL
since a delta record isn't INSERT/UPDATE) and passing after.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit a60231cde1d49d74743cedbd134d2da11158b70b)
- review finding: field_idx=0 and back_off=0 (fresh WAL, offset 0) meant
a u32/u64 swap of these two values wrote identical zero bytes either
way — undetectable by the prior assertions
- test_delta_record: new dedicated 3-scalar-field class (not shared
KEYS_CLASSES) so field_idx can be a nonzero, fixed-8-byte value without
a Text field's variable-length encoding complicating the fixed body
size assertion
- stage+commit a filler row first so the target row's insert record (the
delta's back-pointer) lands at a nonzero offset, not the WAL's initial 0
- delta now targets field_idx=2 with back_off=base_off, both nonzero and
distinct from each other and from class_id=0
- class_id stays 0: this fixture registers exactly one class, so there is
no other value to give it without an unused second class purely to
shift an index
- verified live: temporarily swapped the field_idx/back_off wput calls in
wal.c, confirmed test_wal now fails (fidx==49 want 2, back==2 want 49),
then reverted — wal.c diff is a no-op, only the test changed
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 20ba0965e15e200641b8b63893a2f238a0279fba)
- enum: add WO_WAL_DELTA = 4, existing 1/2/3 untouched (on-disk logs)
- wal.h: document kind 4's payload shape in the format docblock
- wal.h: declare wo_wal_append_delta(w, db, class_id, id, field_idx,
back_off, value) — back-pointer taken as a parameter, not looked up,
keeping the encoder ignorant of table/map state
- wal.c: implement it, modeled on wo_wal_append_insert's shape —
wput_u8/u32/u64 the header fields, enc_val the one field, stage()
- test_wal.c: new test_delta_record — stages a delta after an insert,
commits, then preads the raw record and asserts kind/class/id/
field_idx/back-pointer/value all round-trip; registered in main()
- nothing reads deltas back yet — decode/apply is a later task
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 9c6f832c0534a59e644c53b7cd2850581da12159)
- wo_row_remove's keys arm borrows the row from the log to find its
index entries, and a borrow reads through db->rt->wal. At boot that
pointer is not wired yet: main.c replays first (main.c:226) and
assigns rt.wal afterwards (main.c:268)
- so the borrow found no log, the remove failed, and replay reported a
valid tombstone as CORRUPTION. An UPDATE record would have failed the
same way, since replay applies it as remove-then-recreate
- replay now lends the runtime a read-only view over the fd it already
has open, for the replay's duration only, and restores what was there
- broken by the delete fix in 76b8fd9 — deletes worked in-process but
their tombstones broke the next boot. Unreachable in production only
because the loader still refuses the annotation
- pinned by test_keys_resident_delete_then_replay, verified failing
against the unfixed code (2 failures) and clean with it
- found by asking whether the read-modify-append plan was ready, not by
a gate
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit dc25462461b9f79d70c803f7174adc90fa16c90e)
