- wo_pkey_parse: PKCS#8 PrivateKeyInfo (wrapping PKCS#1/SEC1), bare PKCS#1
RSAPrivateKey, and bare SEC1 ECPrivateKey -> RSA (n,d) or the EC P-256
32-byte scalar. Reuses the X.509 DER reader; spans point into the buffer
- KAT: all three formats parse, and the extracted key signs a hash our
verify accepts (RSA-PSS + ECDSA); garbage rejected. test_crypto 130,
ASan/UBSan clean
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 819d67226a94c4cd5a765ec403e57466c64f1e65)
- wo_tls_server: the mirror of the client driver. parse ClientHello (pick
suite, extract x25519 share, echo session id; reject no-x25519/no-1.3),
build ServerHello, derive the role-symmetric keys, emit the encrypted
flight (EncryptedExtensions + Certificate + a signed CertificateVerify +
Finished), verify the client Finished, switch to application keys
- server_sign_cv signs the CertificateVerify with the phase-G1 primitives
(RSA-PSS or ECDSA-P256 + a minimal DER SEQ{r,s} encoder); parse_client_hello
+ build helpers reuse the file's wire reader/writer
- wo_tls_server_start builds the Certificate message from a cert chain +
private key (RSA n/d or EC scalar) + ephemeral; encrypt/decrypt over the
application keys
- KAT: loopback — our client driver against our server driver, EC then RSA
server identity, reaching ESTABLISHED with an app round-trip both ways.
test_tls 123, ASan/UBSan clean
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 34d2b8f87cebe536cd2b1b33e6948251ec11684f)
- wo_ecdsa_p256_sha256_sign: deterministic nonce (RFC 6979 HMAC-DRBG over
the key + message — no RNG, no nonce-reuse/bias risk), then r = (k*G).x
mod n and s = k^-1 (z + r*d) mod n
- constant-time in the secret: jmul_ct (double-and-add-always + point
cmov) for k*G, and bn_modexp_ct for k^-1 mod n and the affine inversion.
Known residual (documented): the ladder leaks k's leading-zero count (a
bit-length hint, not the key) — a complete-formula/Montgomery-ladder
upgrade is the named follow-up
- KAT: byte-for-byte vs the RFC 6979 A.2.5 P-256/SHA-256 vectors ("sample"
+ "test"), our sign verifies with our verify, determinism checked.
test_crypto 115, ASan/UBSan clean
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 1bc6d04f9e18180dc7d0a6e5e7021dbd588e499a)
- bn_modexp_ct: constant-time modexp for the SECRET exponent — squares and
multiplies every bit, selects the product with a mask (bn_cmov), so the
op sequence is independent of d (the existing bn_modexp branches on the
bit, fine only for the public e)
- wo_rsa_pss_sha256_sign: EMSA-PSS-ENCODE (RFC 8017 §9.1.1) + modexp with d;
caller supplies the salt (fresh in production; fixed makes the KAT
deterministic). Private key (n,d)
- KAT: deterministic sign vs a python from-spec oracle byte-for-byte
(fixed salt), our sign round-trips through our verify, tamper rejected.
test_crypto 108, ASan/UBSan clean
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit cf8fdfcc47b2b076b63b9c63bf56411615b0d6f9)
- wo_tls_pem_to_ders: scan a PEM bundle for CERTIFICATE blocks, base64-decode
each into a caller arena, record DER spans as trust anchors for
wo_tls_verify_chain. Pure (caller reads the file + owns the arena) so it is
offline-testable; the file read + per-shard cache land with the builtin
- b64_decode helper (standard alphabet, skips whitespace/newlines)
- KAT: decode the real /etc/ssl/certs/ca-certificates.crt (>100 anchors,
each parses, first is a CA), garbage PEM -> 0 with no over-read,
skip-if-absent for CI. test_tls 107 pass, ASan/UBSan clean
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 6445d55aa83dbed831d84fd3cca3a74fe4609a00)
- crypto.c: x509_find_ext (generic extension walker) + wo_x509_basic_constraints
(cA / pathLenConstraint, absent => not a CA) + wo_x509_eku_serverauth_ok
(EKU absent, serverAuth, or anyEKU => usable; else not)
- wo_tls_verify_chain enforces decision 6: the leaf must be server-usable
(EKU), every server-sent issuer and the signing anchor must be a CA
(basicConstraints CA:TRUE) with a pathLenConstraint covering the
intermediates below it — stops a leaf masquerading as a CA
- gen_x509.py extended (folds in the wildcard leaf, adds EKU clientAuth-only,
EKU serverAuth, a non-CA intermediate + a leaf issued under it); vectors
regenerated
- KATs: extractors (test_crypto 104) + chain enforcement (test_tls 103) —
EKU serverAuth accepted, clientAuth-only rejected, leaf-under-non-CA
rejected though every signature verifies; existing chains still pass.
ASan/UBSan clean
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 3811418014c2c8d9bc3a9464256f52104261b1b9)
- wo_tls_verify_chain: leaf-first DER chain — each cert signed by the
next, the top trusted (equal to, or signed by, a trust anchor), the leaf
SAN matching host, every cert temporally valid. Any failure rejects;
no partial trust. Pure over the phase-D/E verifiers, so offline-testable
- KAT with the phase-E RSA + EC chains: leaf trusted via its issuing CA
anchor; wrong-anchor / wrong-host / expired / broken-link / no-anchor
all rejected; two-cert chain with a byte-equal root anchor; NULL host
skips the SAN check. test_tls 100 pass, ASan/UBSan clean
- remaining F3c-net (live-gated): CA-bundle PEM loader, random ephemeral,
the net.connect_tls builtin driving the sans-io driver over a real fd
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 9d40055108d301be32df0e1d4140c23446baa7c6)
- wo_x509_check_host: match a hostname against the cert subjectAltName
dNSNames (RFC 6125) — case-insensitive, single left-most wildcard that
covers exactly one label; no SAN => refused; no legacy CN fallback.
Completes the phase-E deferred hostname check (walks the [3] extensions)
- wo_tls_client_set_host + driver enforcement: with a host set, a leaf
whose SAN does not match is refused at the Certificate step (MITM
defense); unset skips the check (offline testing only, documented unsafe)
- KAT: exact/case-insensitive/mismatch, no-SAN refused, wildcard one-label
(not zero, not sub-label) via a wildcard-SAN cert; driver refuses the
RFC 8448 leaf (no SAN) once a host is set. test_crypto 95, test_tls 91,
ASan/UBSan clean
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 319ce8bfcf608030f958b36ab2c8f62fd76e1740)
- wo_tls_client: a pure state machine (no sockets). Caller frames
records; driver runs ClientHello->ServerHello->flight->Finished and
hands back bytes to send. Keeps all I/O out of the security-critical FSM
- start_with (inject CH + ephemeral priv), push_record, take_output,
encrypt/decrypt (application traffic keys). Handshake-message reassembly
across records; per-message transcript timing (CertVerify signs CH..Cert,
Finished MACs CH..CertVerify); constant-time Finished compare; every
failure lands in FAILED (no warn-and-continue)
- verifies server CertificateVerify (phase E+D) + server Finished, emits
the client Finished, switches to application keys
- KAT: whole handshake driven offline against the RFC 8448 record trace —
client Finished record byte-for-byte, first client app record
byte-for-byte, NewSessionTicket + server app data decrypt to plaintext,
tampered flight -> FAILED. test_tls 90 pass, ASan/UBSan clean
- SECURITY TODO before live use (documented in tls.h + story): chain walk
to a trust anchor + SAN/hostname match; random ephemeral for production
start; the net.connect_tls socket glue
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 74c332d7efdb8bbfdbe90bd2fcc3defa2fe8da00)
- wo_tls_verify_cert_verify: verifies a server CertificateVerify
(RFC 8446 §4.4.3) — builds the 64-space || context || 0x00 ||
transcript-hash content, parses the leaf SPKI (phase E) and dispatches
to phase-D RSA-PSS / RSA-PKCS1 / ECDSA-P256; the scheme must match the
leaf key type. ECDSA sig r/s pulled from its DER SEQ
- reuses wo_tls_finished_verify (phase F2) for server + client Finished
- KAT: the whole handshake crypto driven offline from the RFC 8448 §3
recorded messages — CertificateVerify (RSA-PSS) VALID, wrong-transcript
/ tampered-sig / mismatched-scheme rejected, server Finished byte-exact,
and the client Finished we would send byte-exact. test_tls 78 pass,
ASan/UBSan clean
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit afd9f23508c648322712df91329aa117c975ccab)
- new tls.c/tls.h on the crypto ladder: wo_tls_record_seal/open
(RFC 8446 §5.2) — TLSInnerPlaintext (content||type, no padding),
5-byte header as AEAD additional-data, per-record nonce = iv XOR
seq big-endian (§5.3)
- suite dispatch: TLS_AES_128_GCM_SHA256 (mandatory) +
TLS_CHACHA20_POLY1305_SHA256 (AES-NI-less fallback), over phase-A AEAD
- open() strips trailing zero padding to recover the inner content type;
rejects a length-field lie before the AEAD, and auth failure after
- KAT vs python AEAD oracle (test/gen_tls_record.py): sealed record
byte-for-byte both suites, open() recovers it, 5-seq round-trip,
tamper + wrong-seq + bad-suite rejected. test_tls 51 pass, ASan/UBSan
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 5021a99f8359f78a642bb0cc2b28ab66f6b624e2)
- defensive ASN.1/DER reader: every length/bound checked; malformation
is rejection, never over-read (truncated input KAT-gated)
- x509_parse: tbsCertificate span, sig-alg OID, signature,
SubjectPublicKeyInfo (RSA n/e or EC P-256 x/y), validity dates
- wo_x509_verify_one: one chain link's signature, dispatching to
phase-D RSA-PKCS1/PSS + ECDSA-P256 by the issuer key type
- wo_x509_parse_spki + wo_x509_check_validity (caller supplies time)
- KAT against real python-generated chains (test/gen_x509.py):
RSA CA+leaf (SHA256withRSA), EC P-256 CA+leaf (ecdsa-with-SHA256);
leaf-vs-CA, self-signed CA, wrong-issuer/tampered/truncated reject,
validity window, SPKI extraction. test_crypto 84 pass, ASan/UBSan clean
- deferred to phase F: SAN/hostname match + multi-cert chain walk to a
system CA bundle (both need the target host / trust store, known at
handshake time)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 4ec1c75f889df3612e31b9a1a77c4c927fb546c1)
- wo_ecdsa_p256_sha256_verify: NIST P-256 signature verify for EC-cert chains
and TLS 1.3 CertificateVerify
- Jacobian point arithmetic (double a=-3, general add with the H==0 special
cases), double-and-add scalar mult; field/scalar arithmetic reuses the
bignum Montgomery multiply and modexp (Fermat inverses mod p and mod n)
- validates r,s in [1,n-1] and that Q is on the curve (invalid-curve guard)
- verify-only public data -> not constant-time by design
- renamed the P-256 field mul to fpmul to avoid the clash with X25519's fmul
- VERIFIED against a python ECDSA-P256 vector; tamper + wrong-hash rejected;
test_crypto 69/0; ASan/UBSan clean; battery green
- phase D COMPLETE (RSA PKCS1+PSS + ECDSA-P256). Next E: ASN.1/X.509
(cherry picked from commit 92c996ba96d785d098c173d4ac6ace9052ef6a2f)
- wo_rsa_pkcs1_sha256_verify + wo_rsa_pss_sha256_verify (SHA-256), for the
server cert chain and TLS 1.3 CertificateVerify
- bignum: Montgomery multiply (CIOS, 64-bit limbs, __int128), modexp with the
public exponent (R^2 via 128k modular doublings, no division); MGF1-SHA256
- verification is public data only -> NOT constant-time by design (correct and
much simpler than a private-key op)
- assumes a full-length modulus for PSS emBits (standard RSA-2048/3072/4096)
- VERIFIED against python cryptography RSA-2048 vectors (PKCS#1 v1.5 + PSS,
salt 32); tamper + wrong-hash rejected; test_crypto 66/0; ASan/UBSan clean;
battery green
- internal C, no builtin/compiler change. Remaining in D: ECDSA-P256 (D2)
(cherry picked from commit 9118177fbfd03eff9757defea6931afdd68830c4)
- wo_x25519: constant-time Montgomery ladder + mask-based conditional swap,
radix-2^51 field arithmetic with __int128 products (curve25519-donna-c64,
public domain); scalar clamped, u-coord high bit masked per RFC 7748
- internal C (consumer is the TLS ECDHE handshake); no builtin/compiler change
- KAT-gated in test_crypto: RFC 7748 §5.2 both direct vectors AND the
1000-iteration base-point test; test_crypto 61/0; ASan/UBSan clean; battery green
- fixed one transcription bug found via the KAT: crecip needs 5 final squarings
(p-2 = 2^255-21 = (2^250-1)*2^5 + 11), not 3
- rv2 9 ladder: A (AEAD) + B (HKDF) + C (X25519) done; next D signatures/RSA
(cherry picked from commit f41b1c5f56caa841d1904382830baff0f75525d9)
- wo_hkdf_sha256_extract/expand (RFC 5869) + expand_label (RFC 8446 §7.1),
internal C over the existing hmac_sha256; SHA-256 (mandatory-suite hash;
SHA-384 a later add for the AES-256 suite)
- no builtin, no compiler change -- no .wo consumer yet (the TLS handshake
is the consumer); exposed for the C unit test
- KAT-gated in test_crypto: RFC 5869 Test Case 1 (PRK + 42-byte OKM) and
three HKDF-Expand-Label vectors (key/iv/derived-secret shape); 57/0,
ASan/UBSan clean; runtime battery green
- rv2 9 ladder: A (AEAD, = rv2 8) and B (HKDF) now done; next C X25519
(cherry picked from commit c8d27b6c89a80cd97a996ff7d8b64ff4895e4b26)
- no-intrinsics AES: S-box = GF(2^8) inverse via a fixed-exponent power ladder
(constant-time in the input, no tables), constant-time gf8_mul, byte-oriented
ShiftRows/MixColumns/key-expansion (AES-128 and AES-256)
- constant-time GHASH: bit-by-bit GF(2^128) multiply (mask-driven, no tables)
- aes_gcm_seal/open now dispatch: AES-NI path when present (and not forced
software), else this portable fallback -> AES-GCM works on ANY CPU, so the
phase-B no-AES-NI trap is retired
- wo_aes_force_software test hook; both hw and sw paths verified against NIST
SP 800-38D cases 4 (AES-128) and 16 (AES-256) byte-for-byte; test_crypto 48/0;
ASan/UBSan clean; full runtime battery green
- ARMv8 crypto-extension hardware path deferred (untestable on x86-64 host)
(cherry picked from commit dccf650899798401a9adac8489f34c85ed9304af)
- sendmsg/recvmsg with one SCM_RIGHTS fd and a sentinel byte; EAGAIN
parks in the net mould; the received fd arrives nonblocking as a plain
Int every fd verb accepts
- SO_DOMAIN gate: send_fd on anything but a unix socket refuses by name;
plain bytes deliver nil from recv_fd
- net.connect_unix carried here (iteration 38 still pending)
- legs (single-fiber: unix connect completes while the listener holds
the handshake): a pipe's read end crosses and still reads "ping"; a
tty crosses, term.raw works on the RECEIVED copy and destroy restores
it; refusal and nil legs verbatim. test_term 60/0
- full belt: all suites 0 fail both flavors, woc 557/0,
subprocess-accept 12/0, site-accept 23/0
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
(cherry picked from commit 1d689027920d6814f87b97c216b0cb42f7eba3e9)
- two verbs on any tty fd; saved termios in a per-shard 8-entry table;
double-raw and restore-without-save refuse by name
- restore is a RUNTIME obligation: vm_unwind at depth 0 (uncaught trap,
fiber reap) restores the dying fiber's entries newest-first, and
wo_vm_destroy sweeps the rest — proven twice in the legs: a DIV0
while raw restores, and even the double-raw REFUSAL (itself a trap)
restores the first raw
- test_term 39/0 against a real PTY pair made by the test
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
(cherry picked from commit b439387dbf4f653e034c721bc3f08b83616c5e24)
- mechanics amendment to the spec (recorded at close-out): no signalfd —
the stop-latch pattern generalized. An async-signal-safe handler
latches the number, bumps a sequence and pokes shard 0's wake eventfd;
wo_io_wait's loop head drains latches into fresh Signal{sig} records
delivered via runtime_notify (exported as wo_actor_notify)
- payloads must be heap objects (vm.c drops them unconditionally) — the
Signal record exists exactly for that; class id rides the call as the
appended record operand (sm_record drives it even with no return)
- offerable: WINCH/CHLD/HUP/USR1/USR2; SIGTERM/SIGINT refused naming the
stop latch; shard-0-only registration; coalescing disclosed
- stdlib_modules gains `signal` (and `term`, next task)
- test_term: a real child kills the test process with USR1; the actor's
multi holds one coalesced delivery; refusal leg verbatim. 14/0
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
(cherry picked from commit 14e03a6a4a343b97c9b47fab1a5e3c4bb69d8201)
- posix_openpt/grantpt/unlockpt/ptsname_r (plain libc, no -lutil); child
setsid + opens the slave as its controlling terminal, initial
TIOCSWINSZ from the call
- Child.stdin == Child.stdout = the master (caller's copy); the slot
keeps a private dup so resize survives the caller closing theirs
- proc.resize -> TIOCSWINSZ; refuses by name on a pipe child
- legs: test -t proves a real tty; stty size reads "24 80" then "40 120"
after a mid-sleep resize; refusal asserted; test_proc 193/0 ASan clean
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
(cherry picked from commit 9836c9cd5197153517c054b243cab3b453d130a0)
- a child is fds: Child {id, stdin, stdout, stderr}, driven by the
existing net verbs (echo leg proves cat round-trip through write_dl/
read_dl); caller owns the fds, the runtime owns pid + pidfd
- wait_dl parks on the pidfd: code on exit, nil at the deadline with the
child untouched; one waiter per id, a second refuses by name; stale
ids refused via a generation counter in the handle
- proc.signal through pidfd_send_signal; actor_die kills the streaming
children the dying actor owns; dead fibers cannot linger as waiters
- ids 97-107 registered wholesale (wob.h, loader arities, dispatch
bound); Child + Signal predeclared records in types.ml; unimplemented
ids trap at the default case until their task lands
- test_proc 168/0 (echo, wait trio, one-waiter refusal, 200-round churn
fd-flat), suite ASan clean, woc-test green
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
(cherry picked from commit 9be87f159f1bf9cdd509ceed160e7ea518fde46c)
- ceiling: 32 fibers hold live sleepers; the 33rd spawn traps WO_T_IO
naming the ceiling; destroy sweeps all 32 (waitpid -1 = ECHILD after)
- unwind: a fiber parked on a live child is reaped at main's return and
the child dies with it (nchildren 0 straight after the call)
- churn: one thousand sequential `true` runs through a bytecode loop —
fd count flat, every slot released
- stop: SIGTERM from a helper 200 ms into a sleep-10 child answers rc 1
(STOPPED) with no surviving child
- test_proc 128 pass 0 fail in 2.6 s, suite ASan clean
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- proc.run_dl reachable: dispatch range extended to id 96 (builtin.c) and
the loader arity table gains [WO_B_PROC_RUN_DL] = 6 — without both, the
builtin answered "unknown stdlib builtin" (WO_T_EXPLICIT)
- deadline leg: sleep 10 vs 100 ms deadline traps WO_T_IO naming the
deadline in ~120 ms; the pid is gone (waitpid -1 = ECHILD) and the fd
count is flat; a worker fiber completes WHILE main is parked — the
shard was never blocked
- cap legs: stdout and stderr caps trap naming "cap 1000", child dead
- argv multi carries a drop entry at the run pc: a trapping run frees it
(LeakSanitizer caught the miss)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- deadlock proven first: chatty child (200 KB stdout, stderr held open)
hung the old sequential drain 5.0 s into the alarm, code -1, stdout
truncated at 8192; the leg demands completion under 4 s
- rework: nonblocking pipe read ends + pidfd_open behind one epoll fd the
fiber parks on (the _dl retry mould); both pipes drain on readiness, so
the deadlock is gone structurally — leg passes in 15 ms
- wo_child slot table in wo_vm (32/shard) carries cross-park state; caps
refuse by name (kill + WO_T_IO), deadline armed via dl_active/dl_at,
defaults 30 s / 1 MiB / 64 KiB
- WO_B_PROC_RUN_DL = 96 shares the case (per-call deadline_ms/out_cap/
err_cap; compiler row lands in a later task)
- fib_reap kills a reaped fiber's child; wo_vm_destroy sweeps the table
- raw syscalls for pidfd_open/pidfd_send_signal: glibc 2.35 build floor
has no wrappers
- all 19 suites green under ASan+UBSan
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- new suite runtime/test/test_proc.c (auto-globbed by the Makefile)
- three legs against today's behavior: echo exits 0 with exact stdout,
false exits 1, a missing command answers 127 (the execvp convention)
- record fields copied out before the vm dies; ASan clean
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- crash-before-rename test: a COMPLETE valid migrated temp beside the
untouched original is discarded and the boot re-migrates — the
sharpest point on the crash timeline, deterministic, no fault
injection needed
- story: all six tasks done with commit hashes, all eight criteria met
with the test that proves each, plus the three deviations from the
plan and why (transcode over replay, lazy head, poison forces
transcode)
- CODE-LOGIC: migration section; also corrected limitation 3, which
still claimed unbounded hot-row chains — iteration 11 closed that
- status board row 12; deploy guide's rollback section gets its real
answer (rolling back across a migration is a migration backwards:
expect the refusal, restore the .bak)
- test_wal 5966 pass, 0 fail
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 4bb6ece2531e2123eb958c91d9bef4a6528eab3b)
- wo_wal_migrate rewrites the log without touching db state: no id
maps, no indexes, no keys-resident logic — the new log replays
through the machinery that already exists and is already tested
- cids remap by name, INCLUDING the ones embedded inside stored owned
values (an owned value carries a cid on the wire); the embed closure
guarantees every nested class is shape-unchanged, so only numbers
move
- surviving fields go to their new slot, deleted fields' values are
freed, added fields take the kind's zero value straight from
enc_val(0)
- a delta on a deleted field is SPLICED out: an offset map (old record
start -> new) rewrites every back pointer, and the dropped delta maps
to its own target so later deltas step over it
- temp + fsync + rename, compaction's own crash discipline; a stale
temp is discarded at start; a torn tail bounds the intact prefix
exactly as replay does
- fixed en route: early `goto corrupt` jumped over initializers, so the
handler freed uninitialized memory — declarations hoisted above the
first jump
- six end-to-end tests: add, delete (ASan watches the freed Text),
reorder with owned fixup, delta splice on a keys-resident chain,
poison-bites-only-with-records, corrupt input
- test_wal 5951 pass, 0 fail
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit b69092a99206e1dcdf6f2dcdb02146939b0564c6)
- wo_schema_diff matches classes and fields by NAME, so declaration
reordering is identity apart from the cid map — the silent
cid-renumbering hole closes as a side effect
- owned-field references (fclass) compare by the NAME the number
resolves to, never the number: a raw compare would false-poison
retype on every pure reorder
- refusals are per-class POISONS carried in the plan, not diff errors:
a poison bites only when a record of the class is met, so a retyped
class with no stored rows never blocks a boot
- poison set: retype, same-shape delete+add (a disguised rename, one
reading destroys a column), vanished class, storage-flag change, and
the embed closure — any class whose old records carry values of a
class whose shape changed, iterated to a fixpoint
- identity plans skip the rewrite entirely; a NEW class in the binary
does not break identity (no records; the head refreshes at the next
compaction)
- ten verdict tests; test_wal 5778 pass, 0 fail
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 63a063b822af381a10c2e599c58cf3455d9c5bf7)
- new record kind 5: class and field NAMES, kinds and the two
encoding-relevant metadata words (field_class, field_elem), CRC-framed
like every record. Index layout deliberately absent: indexes rebuild
from rows at boot and never touch record bytes
- names are byte pointers, not constant-pool indices — the database
layer never sees the module's consts, so the runtime resolves them
once; a decoded schema owns a private copy of its bytes
- wo_wal_set_schema adopts the compiled schema; wo_wal_ensure_schema
writes it as a fresh log's first record; compaction writes it at the
head of every replacement, which is how a legacy log becomes
self-describing without a migration step of its own
- apply_record skips it BEFORE reading cid/id (its class count would be
misread as a cid and bounds-refused); replay does not count it
- schema unset = byte-for-byte today's behaviour: all 5700 prior
assertions pass untouched; four new tests cover roundtrip, fresh-log
head, legacy adoption via compaction, and absent/empty files
- test_wal 5743 pass, 0 fail
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit ba8519fa5e39c6ed6a3504d39e5a372f8decd045)
- flattened row image is WO_WAL_UPDATE, not WO_WAL_INSERT: the row's
original INSERT is already in a live log, so a second one for the same
id is a duplicate replay refuses as corruption. INSERT is right only
for compaction, which builds a fresh log
- remove WO_CKPT_MAX_GARBAGE: with the absolute term at 64 MiB, garbage
large enough to reach a 256 MiB ceiling has already tripped it, so the
branch was unreachable. Postgres needs both constants because it
thresholds on tuples with its pair at opposite ends; this thresholds
on bytes, where one constant does both jobs
- test_delta_chain_flattens_at_k: chain depth stays <= WO_DELTA_MAX_HOPS
across 2K+2 updates, and a reset is observed
- test_delta_chain_flatten_replays: a flattened chain replays correctly
- test_keys_resident_indexed_across_flatten: a delta on an indexed
column composes with flattening, checked at every step across the
bound and after restart. Found no product defect
- test_should_compact_absolute_and_ceiling: pins the absolute term, the
boundary just under it, and the small-log case the ratio still governs
- test_wal 5700 pass / 0 fail; wovm-test and woc-test green
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit f93b5d9db753305c297e868d977670e6d703684c)
- wo_row_borrow's keys arm folded at hget()'s DURABLE offset even
when an earlier update in the same drain had only a PENDING
re-point
- idx_remove_row then hashed the pre-first-update value, found no
matching bucket entry (already moved by the earlier update), and
idx_add_row added a second one — N same-drain updates leaked N-1
entries, unbounded, nothing reclaims them but a restart
- now prefers wo_wal_repoint_offset1() over the durable offset, same
as back_off already does, closing it for every borrow
- new test: 5 updates to one row in one drain, assert exactly one
index entry — fails (5) before the fix, passes (1) after
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit d4b12d1908e574419c7af2411e52d02623f7e5b7)
- loader.c: delete the INCOMPLETE-update BAIL; durable:false +
resident:keys stays refused (nowhere to read from)
- table.c: root-cause fix for the Text-index gap — a keys-resident
borrow now holds ENGINE values, matching wo_row_ptr's contract
(table.h's "no VM pointer" doctrine), not a VM-decoded row. Fixes
idx_hash/idx_cols_equal/wo_idx_probe AND db.c's GET_FIELD/PROBE
arms with one change; reproduced pre-fix as an ASan
heap-buffer-overflow
- docs/examples/residency: Product is genuinely resident:keys;
residency-accept.sh's refusal leg replaced by proving the program
runs and stock survives a restart (11/0)
- test_wal.c: oracle test drives resident:all and resident:keys
through the same update sequence and asserts identical rows;
Text-indexed-update test catches the representation bug; five
pre-existing tests corrected to the fixed contract (4746/0)
- story, README, status board, CODE-LOGIC.md updated; three known
limitations documented: mid-drain stale reads, O(N^2) replay in
chain length, compaction blind to per-row chain length
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit b87c68f950f01aa5e572fbb86a0f374adc83d813)
- apply_delta: DELTA replay arm — fold pre-delta state via back_off,
overlay the field, remove-then-recreate so indexes stay correct
- apply_record/replay loop: dispatch DELTA to apply_delta, drop its
payload back to the log same as INSERT/UPDATE
- stage_flattened_row: compaction's delta-chain path — fold + re-encode
as one fresh INSERT instead of copying the chain
- wo_wal_compact: peek the row's current record kind, flatten deltas,
keep the byte-for-byte copy for chains already at length zero
- test_wal: three new tests — chain-of-three replay incl. secondary
index, compaction flattens to chain length zero (asserts the record
is a full row, not a delta), and the commit-before-repoint crash
window replays the update without ever re-pointing the map
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 7e4ae70d7beb2a2e396a002184bc06f41253decb)
- table.c: unique shadow-check's candidate lookup now checks
wo_wal_repoint_offset1 before the durable wo_row_offset1, same as
back_off — a candidate updated earlier in the SAME uncommitted drain
was folded from its stale pre-update offset, letting a real @unique
clash through and committing a duplicate
- the offset-only substitution alone was NOT enough (verified): the
candidate must be FOLDED to compare values, and folding a pending
offset via pread saw "no record" (bytes still only in the staging
buffer), so the clash was still missed, just for a different reason
- wal.c: wo_wal_fold_row_at now reads a hop inside the currently-staged
region from `w->buf` (new scan_record_staged, scan_record's framing
over memory) instead of pread; every durable hop, and every existing
caller, is unchanged
- test_wal.c: two updates in one drain where the second collides with
the first's new unique value; must be refused. Verified failing
against the prior commit, and still failing with only the offset
substitution, before the fold fix; passing with both in place
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit c049ab92570cfba4d12a018884a20b25a8916727)
- db.c: guard both WO_B_DB_UPDATE_FIELD arms on keys-resident tables —
wo_wal_append_update read a NULL wo_row_ptr there; a live crash, fixed
- ruling override on Task 3: row_apply_field_keys no longer commits or
moves the id map — stages the delta, does the index swap (RAM apply,
unconditional past the shadow-check; a stage failure past that point
is now fatal, like insert). Commit/re-point move to the caller,
mirroring insert. table.c's WAL commit removal is this ruling, not a
regression
- offset passed back via caller-side wo_wal_next_offset(), insert's
koff pattern
- inline arm commits then re-points; request arm records
wo_wal_pend_repoint (own list/name — a drop and a re-point differ),
flushed by wo_db_flush_drops after the barrier
- back_off checks the pending re-point before the durable offset, else
a second update in one drain skips the first delta; verified failing
this way, passing after
- wal.c: fixed a stale comment — keys-resident updates CAN reach
wo_wal_append_update's caller now, they just never call it
- test_wal.c: 2 tests updated for the new contract; new test drives 2
same-row updates via wo_row_update_field_slot in one uncommitted
"drain", checks the value and delta 2's on-disk back-pointer
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 4d13bcebfe51936ba8c608dd9e791c784e5b8983)
- row_apply_field_keys's shadow-check borrowed candidates via
wo_row_borrow, which shares ONE per-table scratch with the row already
borrowed for the update — every candidate borrow returned NULL, clash
was always false, `@unique` silently accepted duplicates on update
- idx_add_row's own internal check has the identical defect at the same
call site; discarding its result is now actually safe, since the fixed
shadow-check clears uniqueness before it ever runs
- fix: extracted keys_fold_into (fold+decode) out of wo_row_borrow so it
can target a throwaway per-call buffer instead of t->scratch; the
shadow-check probes candidates into that buffer — r is never
released-and-reborrowed (r IS t->scratch; that would overwrite it)
- wo_row_borrow itself is behavior-preserving: same checks, same order,
same messages, just factored
- test_wal.c: new test — genuine @unique index, update collides with an
existing row, asserts refusal (DB_ERR_UNIQUE) and both rows untouched;
verified failing (update wrongly succeeded) pre-fix, passing after
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 409186da51fec0042d8d1e3dcf9f69f704937823)
- table.c: wo_row_update_field/_slot no longer refuse `resident: keys`,
both converge on one new static row_apply_field_keys
- borrows (folds), shadow-checks uniqueness, appends the delta with the
row's current offset as back-pointer, commits, THEN idx_remove_row +
idx_add_row + wo_row_set_offset — failure through commit leaves the
row's offset and index untouched
- nv decoded to a VM value before touching the materialised copy, since
wo_row_release drops every slot through the runtime, not db_val_free
- borrow released on every exit, including every failure arm
- resident: all path (row_apply_field_slot) byte-for-byte unchanged;
wal.c untouched — Tasks 1/2 already expose everything needed
- test_wal.c: plain field update read back, and an indexed scalar
column updated then found via wo_idx_probe by its new value, gone
from its old — both verified failing pre-implementation, passing after
- concern: idx_hash/idx_cols_equal/wo_idx_probe cast Text slots to
db_text* unconditionally; a keys-resident borrow decodes Text to a VM
wo_str* (different layout) — pre-existing, left untouched; tests use
a scalar index to sidestep it
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 89c56a13ed9f4abd082bfcabb46f46bb53d39faa)
- wal.c: wo_wal_fold_row_at now refuses any delta back-pointer that
does not point strictly earlier than the record naming it
(back_off >= cur), instead of capping total hops at off/13+1
- this is the real invariant, not a proxy for it: a step-count bound
lets a forward-pointing back-pointer through in one hop whenever it
happens to land on a genuine record, returning a plausible-but-wrong
row instead of refusing it
- removes the 13-byte-record magic number entirely; no arithmetic
tied to record framing remains in the guard
- wal.h: docblock updated to describe the direction invariant
- test_wal.c: two new tests — self-pointing back-pointer (boundary
case, back_off == cur) and forward-pointing back-pointer to a real
future record for the same row (the actual gap: verified failing
against the old step-count guard, passing after the fix)
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 173dbf28a42d45a8c7f9fe430355f61f70c81935)
- wal.h/wal.c: wo_wal_fold_row_at — THE fold. Walks BACKWARD from an
offset through WO_WAL_DELTA records, remembering the first value
seen per field index (newest wins, since newest is seen first),
stops at the first INSERT/UPDATE, decodes it, overlays resolved
fields. Returns ENGINE-owned values so reads, replay, and
compaction (Tasks 3/5) can all build on the same output.
- Cycle guard: caps the walk at what the log up to the starting
offset could possibly hold (13 = scan_record's own record-size
floor), so a corrupt or malicious back-pointer fails loudly
instead of spinning.
- table.c: wo_row_borrow's keys arm now calls the fold instead of
wo_wal_read_row_at directly, then VM-decodes the result — same
two-stage pattern wo_wal_read_row_at used internally. Per-table
scratch, scratch_busy nested-borrow refusal, and the cid/id
identity check all preserved unchanged.
- resident: all path (wo_row_ptr) untouched.
- test_wal.c: two new tests — deltas on two different fields (changed
fields take the new value, the untouched field keeps its original)
and two deltas on the SAME field (the newer wins, pinning direction
— a reversed fold would pass with the older value instead).
Verified failing pre-implementation (wo_row_borrow returned NULL
since a delta record isn't INSERT/UPDATE) and passing after.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit a60231cde1d49d74743cedbd134d2da11158b70b)
- review finding: field_idx=0 and back_off=0 (fresh WAL, offset 0) meant
a u32/u64 swap of these two values wrote identical zero bytes either
way — undetectable by the prior assertions
- test_delta_record: new dedicated 3-scalar-field class (not shared
KEYS_CLASSES) so field_idx can be a nonzero, fixed-8-byte value without
a Text field's variable-length encoding complicating the fixed body
size assertion
- stage+commit a filler row first so the target row's insert record (the
delta's back-pointer) lands at a nonzero offset, not the WAL's initial 0
- delta now targets field_idx=2 with back_off=base_off, both nonzero and
distinct from each other and from class_id=0
- class_id stays 0: this fixture registers exactly one class, so there is
no other value to give it without an unused second class purely to
shift an index
- verified live: temporarily swapped the field_idx/back_off wput calls in
wal.c, confirmed test_wal now fails (fidx==49 want 2, back==2 want 49),
then reverted — wal.c diff is a no-op, only the test changed
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 20ba0965e15e200641b8b63893a2f238a0279fba)
- enum: add WO_WAL_DELTA = 4, existing 1/2/3 untouched (on-disk logs)
- wal.h: document kind 4's payload shape in the format docblock
- wal.h: declare wo_wal_append_delta(w, db, class_id, id, field_idx,
back_off, value) — back-pointer taken as a parameter, not looked up,
keeping the encoder ignorant of table/map state
- wal.c: implement it, modeled on wo_wal_append_insert's shape —
wput_u8/u32/u64 the header fields, enc_val the one field, stage()
- test_wal.c: new test_delta_record — stages a delta after an insert,
commits, then preads the raw record and asserts kind/class/id/
field_idx/back-pointer/value all round-trip; registered in main()
- nothing reads deltas back yet — decode/apply is a later task
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 9c6f832c0534a59e644c53b7cd2850581da12159)
- wo_row_remove's keys arm borrows the row from the log to find its
index entries, and a borrow reads through db->rt->wal. At boot that
pointer is not wired yet: main.c replays first (main.c:226) and
assigns rt.wal afterwards (main.c:268)
- so the borrow found no log, the remove failed, and replay reported a
valid tombstone as CORRUPTION. An UPDATE record would have failed the
same way, since replay applies it as remove-then-recreate
- replay now lends the runtime a read-only view over the fd it already
has open, for the replay's duration only, and restores what was there
- broken by the delete fix in 76b8fd9 — deletes worked in-process but
their tombstones broke the next boot. Unreachable in production only
because the loader still refuses the annotation
- pinned by test_keys_resident_delete_then_replay, verified failing
against the unfixed code (2 failures) and clean with it
- found by asking whether the read-modify-append plan was ready, not by
a gate
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit dc25462461b9f79d70c803f7174adc90fa16c90e)
- wo_row_remove read the id map's value as a slot, but on a keys table
that value is a LOG OFFSET (hput(t, id, wal_off + 1)). slot_row does
no bounds check, so a delete indexed t->slabs[] with a byte offset and
then called db_val_free on whatever it landed on — arbitrary frees,
not a wrong answer
- keys tables now take their own arm: no slab slot, no bitmap bit, no
free-list entry to return. The index hook needs the row's values, so
the row is borrowed from the log for exactly that long
- wo_row_ptr carried the same trap and is public. It cannot refuse keys
tables outright (insert legitimately calls it while the map still
holds a slot), so it now detects the offset case — index past the
slabs, or bitmap bit clear — and returns NULL. Callers all handle NULL
- test_keys_resident_delete pins it; it SEGVs against the old code,
verified by reverting the fix rather than assumed
- found while auditing every hget() reader before narrowing the loader
refusal to allow benchmarking. The refusal was justified in the docs
by "updates are unimplemented" while actually standing in front of
this too: a guard whose stated reason is narrower than its real one
gets removed by someone who believes the stated reason
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 76b8fd944af9ed062467bdf9ab93c2e96dd198cf)
- wo_row_read and the @unique shadow probe go through borrow/release;
release runs before every exit, including wo_row_read's early return
- updates on a keys table refused explicitly in wo_row_update_field and
the slot variant: no slab slot to mutate, and writing the borrow's
scratch would discard the write silently. Needs read-modify-append
- compaction walked the bitmap, which a keys row has no bit in — every
such row would have been dropped from the new log. Now walks
wo_row_next_id and re-points each row to where it lands
- moves records byte-for-byte (copy_record) rather than decoding: a
borrowed row holds VM values, enc_val expects engine values, and ASan
caught that mismatch as a 4294967292-byte memcpy
- wo_row_set_offset updates a value in place and never rehashes, so a
wo_row_next_id cursor stays valid while compaction re-points
- a compaction that fails after moving rows is fatal: the map would name
an unlinked temp file, and the intact log replays correctly
- test_keys_resident_survives_compaction pins both failure modes; rows
rewrite in hash order so offsets really move
- loader still refuses resident: keys — updates are not implemented
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit f606fc9b76d983cac2b348f03f7d4f01433cd905)
