shoney.arickathil 02b4b13a52 Merge master into db-residency-doctrine — and close the two half-exposed features

The branch was 17 ahead / 25 behind with 11 conflicting files, and drifting
further: db.c had been rewritten twice on master since (group commit, then
compaction). Resolved rather than rebased so both histories stay legible.

Conflicts, and how each was settled:

- db.c: BOTH semantics kept. Master's fatal path and compaction check now sit
  behind the branch's `table_is_durable` predicate, in all three inline arms —
  a volatile table reaches neither the barrier nor the compaction check
- db-bench sample: every mode from both sides (growth, growth-verify, randread,
  replayseed, wmix) and ONE `boot` mode, which both sides had added
  independently
- db-bench.py: all six legs kept. Both sides had also grown the same
  WAL-size helper under different names; collapsed into one
- perf-targets: the branch's §5 (RAM ceiling) then master's §6/§7 — master's
  numbering had already assumed a §5 it did not have
- story frontmatter: master's `status` (the landing truth) plus the branch's
  `readiness` axis. 03 would have read `done` + `refine`, which is a
  contradiction — it was brainstormed and landed on master, so `ready`
- board: both standup blocks newest-first; master's chain rows (a superset);
  the branch's databasev2 1-2 rows with master's 3-4. Fixed a stray `|` in
  master's row 3
- baseline: master's, then REGENERATED from a full campaign — 143 metrics,
  132 checks, 0 failures with both sides' legs present

TWO HALF-EXPOSED FEATURES FIXED, because the merge rule is that master gets
no feature that is honoured in name only:

- `resident: keys` PARSED, set a .wob flag, and did nothing: rows stayed fully
  resident. A developer could declare a 120 GB table keys-resident, watch it
  compile, and be OOM-killed. The loader now REFUSES it with a message naming
  what to write instead, until tasks 5c/5d land. The compiler still parses it
  and its AST golden still passes, so the grammar work stays tested
- `durable: false` was honoured ONLY on the inline path. wo_db_exec_req had no
  guard at all, so a volatile table written from an actor on a worker shard
  would still be logged — precisely porch's session-table case, and precisely
  what iteration 2 exists to provide. All three request-path arms now carry the
  same predicate. Found by reading the merged code, not by a test: the obvious
  probe runs main() on the primary and therefore only exercises the inline path

Verified on the merged tree: wovm-test 0, woc-test 0, oop-e2e 122/0,
residency-accept 8/0, db-bench 132/0, linkcheck clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

2026-08-29 10:14:25 +02:00

5.6 KiB

Raw Blame History

iteration	status	readiness
34	done	ready

Iteration 34 — crypto builtins: digests and HMAC in the runtime

Format: product/story-iteration-template. Part of Story — one language, one runtime, one database, one binary.

Inserted 2026-08-22 — the framework ledger's oldest unowned gap gets an owner. PREMISE UPDATE (same day, post-merge): iteration 36 landed bitwise & | ^ << >> + hex literals, so digests ARE now expressible in pure .wo — the original "no bitwise" impossibility is gone. The fork is now a real choice for this story's brainstorm: hand-rolled C builtins (bounded, fast, libc-only doctrine permits) vs pure-.wo (no runtime surface growth; interpreter-speed hashing). Off the concurrency chain but gates chain position 4: iteration 24's WebSocket handshake needs SHA-1 before chat can land.

✅ LANDED 2026-08-27 — inside 24 as its task 1. The fork resolved to C builtins: sha1 (85), sha256 (86), hmac_sha256 (87), each over one buffer returning a fresh Bytes. Pinned to the published vectors — RFC 3174, the SHA-256 vectors, RFC 4231 — in runtime/test/test_crypto.c, 18 checks, plus a corpus fixture hashing "abc" from .wo. This unblocked chain position 4: the WebSocket handshake needs SHA-1, and just chat verifies the accept-key independently.

The gap it did NOT close: there is still no RNG in the runtime. HMAC authenticates a token and cannot mint one, so CSRF and sessions stay blocked — which is why 39 leads with a random-bytes builtin rather than treating them as unblocked.

Why this iteration exists

Four consumers already wait on it, none able to proceed: iteration 24's upgrade handshake (Sec-WebSocket-Accept = base64(SHA-1(key + GUID)) — SHA-1 specifically, not a choice); the framework's ETag/conditional-request row (wants a content hash); HMAC-signed tokens the auth core can grow; and held databasev2 10, whose challenge–response needs primitives that "do not exist" (its demotion note). Bytes and base64 landed with iteration 19 — the carriers exist, only the digests are missing.

Goals

Digest builtins over Bytes: SHA-1 (the WS handshake's hard requirement), SHA-256 (the modern default for ETag/HMAC), each Bytes -> Bytes, streaming not required (whole-value, like every existing builtin).
HMAC-SHA256 (key: Bytes, msg: Bytes -> Bytes) — the one composition real services need (signed tokens, webhook signatures); expressible in .wo since iteration 36's bitwise set — C-builtin vs pure-.wo is this story's brainstorm call.
Test vectors are the acceptance: FIPS 180 / RFC 2202 / RFC 4231 vectors in a corpus fixture — a digest that "looks right" is worth nothing.
The contract doc row: names, arities, Bytes-in/Bytes-out, and the explicit note that SHA-1 exists for protocol compatibility (WS), not for new designs.

Acceptance Criteria (draft — the spec refines)

Given the published test vectors for SHA-1, SHA-256, and HMAC-SHA256, when the corpus fixture runs them through the builtins, then every output matches byte-for-byte (via the existing base64/Bytes surface).
Given the WS handshake's worked example from RFC 6455 (dGhlIHNhbXBsZSBub25jZQ== → s3pPLMBiTxaQ9kYGzzhZRbK+xOo=), when composed in pure .wo from sha1 + base64_encode, then the exact accept token comes out — iteration 24's handshake is provably one expression away.
Given the full battery, when it runs, then nothing regresses — new builtin ids only, no opcode, no .wob version bump (the iteration-19/time.ticks precedent).

Out Of Scope

Asymmetric crypto (ed25519 signatures/keypairs) — held iteration 21's spec decides what it needs when it unholds; this iteration lays the digest floor it will stand on.
TLS — permanently the proxy's job (framework doctrine).
CRC32 — the ledger lists it, but no consumer is blocked on it; it joins only if 24's spec finds a real need (rejecting speculative surface).
A password-hashing story (bcrypt/argon2) — no workload asks yet.
Bitwise operators in the language — a separate, bigger surface decision this iteration deliberately routes around.

Info

Forks the spec must settle:

The namespace. The reserved stdlib namespaces are exactly fs/proc/net/time/json/env (compiler-enforced list) — a new crypto namespace touches that list plus the typechecker table, or the functions ride an existing namespace. Leaning: a real crypto namespace (the list exists to be grown deliberately; this is deliberate).
Surface shape: crypto.sha1(b: Bytes) -> Bytes, crypto.sha256(b: Bytes) -> Bytes, crypto.hmac_sha256(key: Bytes, msg: Bytes) -> Bytes — Text convenience overloads rejected (the caller has bytes_of_text).
Implementation source: hand-rolled C from the FIPS pseudocode (~200 lines for both digests; well-trodden, vector-verified) — the doctrine's shape. No linking against OpenSSL, ever.
Where the code lives: runtime/src/crypto.c beside sysio, or inside builtin.c — file layout, the executor decides.

Proposed Solution

Brainstorm → (small) spec settling the four forks → implement with the time.ticks slice's shape (ids, one table row per compiler surface, contract-doc rows, vector fixtures). Lands any time before iteration 24's spec; independent of 31/22/23.

5.6 KiB Raw Blame History Unescape Escape