databasev2 3, task 1.
- walks each class's live rows via the bitmap-over-slabs pattern db.c
already uses in three places, appending one INSERT per live row through
the EXISTING append path. No second encoder, no new format, and ids are
preserved exactly because wo_wal_append_insert takes the id and reads
the row from the store
- FLUSHES EVERY 256 RECORDS rather than staging the whole store: stage()
grows the staging buffer by doubling and never shrinks it, so a
one-buffer dump would hold the entire store in RAM on top of the store
— the unbounded growth databasev2 1 identified as how this engine dies
- the switch, in order: fsync the temp file, rename over the live path,
fsync the PARENT DIRECTORY (rename's atomicity is in-kernel; the
directory entry is not durable until the parent is synced — Postgres
does the same for the same reason), then reopen the descriptor, because
the old one refers to an unlinked inode
- REFUSES when anything is staged: those records would land in a file
about to be replaced. The caller-side guard is task 3; this is the
backstop
- a failure leaves the ORIGINAL log intact and usable and returns -1. A
failed checkpoint is a missed optimisation, not a durability event, so
it deliberately does NOT take databasev2 4's fatal path
- records the bytes written, so task 3's trigger can compare against a
measured denominator instead of estimating the live set (which would
mean estimating Text)
Recovery is untouched — the result is an ordinary log in the ordinary
grammar, replayed from byte 0. Crash safety comes from rename, not from
code of ours.
Test asserts BOTH halves, on purpose:
- the log shrinks: 43 records (3 inserts + 40 updates of the SAME row, so
history grows while the live set does not) -> 3 records, fewer bytes
- AND a fresh replay reproduces the store: every id present, and row 0
carries the 40th update's value rather than its original. "It got
shorter" is also true of a truncating bug, so the replay comparison is
what actually proves it
- and the WAL stays usable after the swap: a further append lands after
the compacted records, giving 4 on the next check
Verified: just wovm-test — 36 suites 0 fail, test_wal 315 pass (was 165),
cli_smoke OK.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
databasev2 4 part A, task 1.
- wo_wal gains `path`: the abort diagnostic is worthless without naming
the file it could not write. strdup'd in open, freed in close; NULL is
tolerated so the message degrades rather than crashes
- wo_wal_commit now reports WHICH half failed — WO_WAL_ERR_WRITE for
pwrite, WO_WAL_ERR_SYNC for fdatasync. A short write and a device
refusing the flush are different operational problems and the operator
needs the right one named
- wo_wal_commit_fatal(w, nrec): commits, or prints one diagnostic naming
the operation, path, errno and record count, then exits
WO_EXIT_DURABILITY (3 — 1 is a trap, 2 is a refusal, so this takes a
third of its own)
- retrying is not offered, deliberately: on Linux a failed fsync may have
already discarded the dirty pages, so a second call can report success
having written nothing. Replay is the recovery that works
- test_wal: a failed commit is DETECTED, reports the write error
specifically, keeps the batch staged (a failed commit consumes
nothing), and the WAL knows its own path. 165 pass (was 156)
DISCLOSED GAP: the exit path itself is not exercised. Forcing a real
fdatasync failure needs a full or read-only filesystem, which the gate
cannot arrange without mount privileges. No fault-injection switch was
added — shipping a binary that can be told to kill itself is the worse
trade, and the spec rejected it.
Verified: just wovm-test — 36 suites (18 x both dispatch flavors) 0 fail,
cli_smoke OK.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- Float full stack: literals (fraction/exponent; `0..10` still a range), f64
opcodes 34-41, @table column, WAL bit-exact replay, json fractions in and
shortest-round-trip out. IEEE-quiet — FDIV never traps where DIV does.
- Bytes: a wo_str with its own class id, so alloc/free/copy are shared but no
Text builtin accepts one; len/at/slice/eq/concat, base64 both ways, json
boundary as base64; TEXT_COPY preserves the kind.
- No implicit Int/Float mixing (WO-E201 in the typechecker, not the emitter,
which picks the opcode from one side and would misread the other).
- One IEEE deviation: float_cmp total order (NaN last, -0.0 == +0.0) for
indexes and order-by, keys canonicalized to match. `?Float` nil is a
reserved quiet NaN — the zero word is +0.0, WO_NIL_SCALAR's bits are -2.0.
- Renderer prefers fixed over exponential in 1e-6..1e21: pure shortest makes
a price of 900.0 read `9e+02`. One renderer for interp/json/float_to_text.
- Fixed en route: lexer double-counted the leading digit; is_scalar_shaped
took Float/Bytes as Int-shaped; Bytes ownership needed a shared heap-scalar
predicate or temps never dropped; order-by bit-compared negatives backwards.
- Iteration 17: `kind = "library"` (absent = program; bad value = WO-E109),
entry-less check mode retiring the `--emit` workaround, Go's `internal/` as
WO-E108 at the consumer's `use`. Driver-only; VM/.wob/GC untouched.
- Framework reorg: internal/{parse,serve}.wo; http/form.wo split out to keep
media_type/form_values public (parse.wo had grown public surface).
- Docs: link audit (97 -> 88 broken, conflict markers resolved, 2 duplicate
stories removed), 00-code-review verified 26/27, iterations re-sequenced.
- Also carries the pre-staged pub(read)/using/#if work from the index.
- Gates: corpus 103/0, test_wal 156/0, web-app 26/0, oop-accept ALL MET.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- wo_row_update_field: encode new value, unique re-check against a
shadow BEFORE any mutation (violating update leaves the row
untouched, DB_ERR_UNIQUE), index entries moved old-hash -> new-hash,
old engine value freed; proven by test_table (unique refusal keeps
the row, released key becomes insertable)
- WAL UPDATE record: full-row re-log, replay = replace (remove +
re-create same id); prefix/suffix delta recorded as later
optimization; test_wal replays insert+update to the updated state
- builtins 62 DB_UPDATE_FIELD (cid,id,field,value) and 63 DB_DELETE
(cid,id), commit-before-ack like insert, WO_T_UNIQUE/WO_T_DB/WO_T_IO
mapping; dispatch range 61..63; loader arities; runner mirror
- plan Task 5 marked superseded-in-part with the recorded deviation:
the language surface (reads, queries, row views, delete statement)
is 9b's, where the comprehension design put it -- no interim brace-
select grammar to retire later
- gates: test_table 839/0, test_wal 102/0, 15 suites, oop-e2e 73/0
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- .wob v3: class records carry an index tail (flags bit0 = unique,
col_cnt, columns) -- @table(index:[a,b]) entries plus one unique
single-column entry per @unique field; loader validates columns in
range and scalar/Text-kinded; emitter validates the declarations
(unknown column, un-indexable kind => diagnostic)
- engine: db_index hash multimap per table, built from the class
table at first touch, maintained ONLY inside wo_row_insert/
wo_row_remove; unique checks re-compare actual column values (a
hash is a hint); replay re-indexes via wo_row_raw_commit AFTER
slots are filled, so recovered tables carry their indexes
- WO_T_UNIQUE = 10; a violating insert is un-applied whole (bitmap,
hash, count, and the never-observable id reclaimed) and traps
catchably -- the employee SEED-DUP pattern
- wo_row_insert gains err_kind so db.c maps UNIQUE/OOM/other to the
right trap; test images and the runner's loader mirror speak v3
- fixtures: trap/db-unique-violation (code 10 exact) and
run/db-unique-catch (catchable dup, composite index accepts
duplicates, next id dense after a refusal)
- gates: oop-e2e 73/0, all 15 runtime suites, woc-test green,
log-watcher 7/0
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- database/src/wal.{c,h}: framed records len|crc32|payload|mark
("WOL1" written last -- no mark, no record), typed-row payloads
walking the class-table kinds (nested records, containers, nil
encodings), little-endian like the loader
- commit order verbatim from the shipped phase-D pattern: RAM apply,
stage, ONE pwrite + ONE fdatasync for the batch, ack after -- group
commit is everything staged riding one sync
- replay decodes straight into engine-owned values (no VM at boot)
and re-enters rows through the choke-point row API, so Task 4's
indexes will rebuild for free; next_id advances past replayed ids
this shard owns (wo_row_create_raw)
- torn tail = short/CRC-fail/no-mark/zero-len: intact prefix applies,
tear dropped whole, wo_wal_open positions AT the tear so the next
commit overwrites it; CRC-valid-but-undecodable = corruption, loud
- wo_wal_check: offline oracle, no engine needed -- the crash
battery's verifier
- test_wal 90/0 ASan+UBSan incl. five crash-battery rounds (fork,
insert/commit/ack-over-pipe, SIGKILL mid-stream: zero acked-but-
missing, zero acked-but-wrong); all runtime suites green, oop-e2e
71/0; binding doc WAL section + CODE-LOGIC + plan Task 2 checked
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
