databasev2 3, task 1.
- walks each class's live rows via the bitmap-over-slabs pattern db.c
already uses in three places, appending one INSERT per live row through
the EXISTING append path. No second encoder, no new format, and ids are
preserved exactly because wo_wal_append_insert takes the id and reads
the row from the store
- FLUSHES EVERY 256 RECORDS rather than staging the whole store: stage()
grows the staging buffer by doubling and never shrinks it, so a
one-buffer dump would hold the entire store in RAM on top of the store
— the unbounded growth databasev2 1 identified as how this engine dies
- the switch, in order: fsync the temp file, rename over the live path,
fsync the PARENT DIRECTORY (rename's atomicity is in-kernel; the
directory entry is not durable until the parent is synced — Postgres
does the same for the same reason), then reopen the descriptor, because
the old one refers to an unlinked inode
- REFUSES when anything is staged: those records would land in a file
about to be replaced. The caller-side guard is task 3; this is the
backstop
- a failure leaves the ORIGINAL log intact and usable and returns -1. A
failed checkpoint is a missed optimisation, not a durability event, so
it deliberately does NOT take databasev2 4's fatal path
- records the bytes written, so task 3's trigger can compare against a
measured denominator instead of estimating the live set (which would
mean estimating Text)
Recovery is untouched — the result is an ordinary log in the ordinary
grammar, replayed from byte 0. Crash safety comes from rename, not from
code of ours.
Test asserts BOTH halves, on purpose:
- the log shrinks: 43 records (3 inserts + 40 updates of the SAME row, so
history grows while the live set does not) -> 3 records, fewer bytes
- AND a fresh replay reproduces the store: every id present, and row 0
carries the 40th update's value rather than its original. "It got
shorter" is also true of a truncating bug, so the replay comparison is
what actually proves it
- and the WAL stays usable after the swap: a further append lands after
the compacted records, giving 4 on the next check
Verified: just wovm-test — 36 suites 0 fail, test_wal 315 pass (was 165),
cli_smoke OK.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
databasev2 4 part A, task 1.
- wo_wal gains `path`: the abort diagnostic is worthless without naming
the file it could not write. strdup'd in open, freed in close; NULL is
tolerated so the message degrades rather than crashes
- wo_wal_commit now reports WHICH half failed — WO_WAL_ERR_WRITE for
pwrite, WO_WAL_ERR_SYNC for fdatasync. A short write and a device
refusing the flush are different operational problems and the operator
needs the right one named
- wo_wal_commit_fatal(w, nrec): commits, or prints one diagnostic naming
the operation, path, errno and record count, then exits
WO_EXIT_DURABILITY (3 — 1 is a trap, 2 is a refusal, so this takes a
third of its own)
- retrying is not offered, deliberately: on Linux a failed fsync may have
already discarded the dirty pages, so a second call can report success
having written nothing. Replay is the recovery that works
- test_wal: a failed commit is DETECTED, reports the write error
specifically, keeps the batch staged (a failed commit consumes
nothing), and the WAL knows its own path. 165 pass (was 156)
DISCLOSED GAP: the exit path itself is not exercised. Forcing a real
fdatasync failure needs a full or read-only filesystem, which the gate
cannot arrange without mount privileges. No fault-injection switch was
added — shipping a binary that can be told to kill itself is the worse
trade, and the spec rejected it.
Verified: just wovm-test — 36 suites (18 x both dispatch flavors) 0 fail,
cli_smoke OK.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- cap 1024 (WO_MAILBOX override at boot): sender-side atomic
reserve/release on every path — same-shard, cross-shard envelope,
OOM rollbacks; full mailbox traps the SENDER catchably; delivery
pop releases; overshoot bounded by in-flight sends (disclosed)
- test_mailbox 12/0: exact cap single-threaded, two racing senders win
exactly cap slots, drain/refill clean
- corpus run/mailbox-full-trap: parked sleeper, send loop catches
"actor mailbox full" after >= 1024 sends
- pre-existing compiler bug found + fixed: a try ARM yielding a Text
PLACE (bare e.msg, try box.field) aliased a register the arm's scope
end freed — ASan use-after-free, SEGV on the next unwind's
double-walk; emit_try now applies copy_place_text to both arm
results; pinned by corpus run/catch-msg-place
- db-bench driver: msgrate keeps iteration 22's unbounded-flood
contract via WO_MAILBOX=MSG_N (the cap is 24's policy, not 22's)
- battery 12/12 fresh-built
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- engine: wo_idx_probe answers single-column equality from the index
hash buckets (idx_hash_key1 reproduces idx_hash bit for bit; verify
compares exactly as the slab walk did, so results identical);
composite indexes keep the walk; both executors wired (local + DB
actor RPC)
- compiler: probe_key_of_where lowers "var.col == key" on an indexed
column to DB_PROBE; all where guards still run (guard stays the
final arbiter); keys = ident/int-literal only; Float/Bytes excluded
(engine raw-eq narrower than VM float-eq)
- measured: reads 1.3k -> 1.3M ops/s, p50 600us -> 1us (~x850);
query x830; mixread 1.3k -> 89k s1, 21 -> ~1.9k sN
- gate policy moved into the driver (tolerance_for: refresh-proof);
latency floors max(4x,100us); quick mode skips poll-bound mix
floors; both tolerance classes proven to bite
- proof: test_table wo_idx_probe suite (RED first), corpus
query-index-probe 105/0, full battery green, TSan clean, two
campaigns pass the refreshed baseline
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Float full stack: literals (fraction/exponent; `0..10` still a range), f64
opcodes 34-41, @table column, WAL bit-exact replay, json fractions in and
shortest-round-trip out. IEEE-quiet — FDIV never traps where DIV does.
- Bytes: a wo_str with its own class id, so alloc/free/copy are shared but no
Text builtin accepts one; len/at/slice/eq/concat, base64 both ways, json
boundary as base64; TEXT_COPY preserves the kind.
- No implicit Int/Float mixing (WO-E201 in the typechecker, not the emitter,
which picks the opcode from one side and would misread the other).
- One IEEE deviation: float_cmp total order (NaN last, -0.0 == +0.0) for
indexes and order-by, keys canonicalized to match. `?Float` nil is a
reserved quiet NaN — the zero word is +0.0, WO_NIL_SCALAR's bits are -2.0.
- Renderer prefers fixed over exponential in 1e-6..1e21: pure shortest makes
a price of 900.0 read `9e+02`. One renderer for interp/json/float_to_text.
- Fixed en route: lexer double-counted the leading digit; is_scalar_shaped
took Float/Bytes as Int-shaped; Bytes ownership needed a shared heap-scalar
predicate or temps never dropped; order-by bit-compared negatives backwards.
- Iteration 17: `kind = "library"` (absent = program; bad value = WO-E109),
entry-less check mode retiring the `--emit` workaround, Go's `internal/` as
WO-E108 at the consumer's `use`. Driver-only; VM/.wob/GC untouched.
- Framework reorg: internal/{parse,serve}.wo; http/form.wo split out to keep
media_type/form_values public (parse.wo had grown public surface).
- Docs: link audit (97 -> 88 broken, conflict markers resolved, 2 duplicate
stories removed), 00-code-review verified 26/27, iterations re-sequenced.
- Also carries the pre-staged pub(read)/using/#if work from the index.
- Gates: corpus 103/0, test_wal 156/0, web-app 26/0, oop-accept ALL MET.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- fiber states (RUNNABLE/PARKED/DONE), intrusive FIFO run queue,
wo_vm_spawn_fiber (calloc'd context, frame 0 set up like wo_vm_call)
- reduction budget: WO_REDUCTIONS (default 4000), checked at loop
BACK-EDGES AFTER the jump lands so the saved pc is the loop head —
a pre-instruction save at budget 1 re-executes the jump into the
same decrement and livelocks (found by reasoning, pinned by the
budget-1 test; deviation from the spec's three-site wording,
recorded in the yield macro's comment)
- FIBER_DONE: main returning ends the program and reaps every
remaining fiber through vm_unwind (drop maps run); a spawned fiber
ending frees silently; its return value is discarded by contract
- TRAPF: an uncaught trap in a spawned fiber kills that fiber ALONE
(stderr report, program lives); in main it stays the program's death
- WO_SYS_STOPPED reaps all fibers wherever it lands (main unlinked
from the queue and unwound if a spawned fiber caught the stop)
- vm_gc_roots walks the live fiber plus every queued one
- test_fiber (45 checks, ASan): EXACT round-robin interleave at budget
1 across three fibers pushing tags into one shared multi;
main-return reaps a spinning fiber holding an owned Big (ASan proves
the free); a DIV0 fiber dies alone, main answers 0
- full battery green: wovm-test, oop-e2e 89/0, woc-test, log-watcher
7/0, employee 8/0, web-app 21/0, deps-accept 8/0 (scheduler dormant
= one branch per back-edge)
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The compiler no longer emits reference-counting ops anywhere, and the format
reserves them. With Phase 3a's collector this completes the runtime half of
iteration 7b: spec success criteria 3 (no RC ops in any image, opcodes
reserved) and 6 (corpus ASan-clean) are met — `just oop-accept` is fully green.
- owner.ml: the rc machinery is deleted outright — rc_site/rc_op types, the
rcs table, fn_rcs/rc_groups/rc_escaped, record_rc, release_gc, gc_escape,
resolve_rc, and the clobber rule (its only consumer was elision). The
`push`-of-a-gc-value RC_INC special case is gone (the bug class cannot recur
without RC). Drop tables (owned + LGc kinds) are untouched — the gc mask is
what feeds the collector's root maps.
- emit.ml: emit_rc, the v_rc view, the escape-acquire anchor, and every
caller deleted; assignment displacing a traced value emits nothing (the VM's
store barrier owns it); scope-ended LGc handles clear their gc-mask bit so
root maps stay precise.
- .wob v4: WOB_VERSION 3 -> 4 in wob.h + emit.ml + disasm.ml + the runner's
loader battery; opcodes 27-28 removed from the enum/jump table/interpreter
and REJECTED by the loader like any unknown opcode.
- dump.ml: the == RC == owner-dump section is gone; 6 goldens re-blessed
(owner dumps lose the section, elision.wo's bc dump loses its RC ops).
- runner.ml: rc-table/ELIDED assertions deleted; the elision test now asserts
the WHOLE image contains no RC op; the table-contract sweep asserts rc ops
never appear.
- test_unwind.c: the rc-opcodes test becomes two — the loader rejects reserved
opcode 27, and an abandoned traced instance is freed by rt_destroy
(ASan-proven).
Verified: woc-test 540/0 + test_diag 14/0; runtime test + test-iso all suites
ASan/UBSan (test_unwind 12/0); cli_smoke; oop-e2e 79/0 (v4 images end to end);
employee 8/0; log-watcher 7/0; ring runs + reclaimed (freed=3) with zero RC
ops in its image; `just oop-accept` ALL CRITERIA MET.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Reference counting and Bacon-Rajan trial deletion are gone from the runtime.
Traced (inferred-gc) objects now die only by the collector; owned values keep
deterministic drops exactly as before.
- wo_hdr: rc retired; borrow and the freed 4 bytes become a union — non-traced
values keep the borrow word, traced objects use the 8 bytes as the intrusive
sweep-list link. Header stays exactly 16 bytes. WHITE is now the all-zero
color (allocations born white by memset); WO_F_BUF retired.
- gc.c rewritten: snapshot-at-beginning tri-color mark-sweep. Roots (frames'
gc+owned masks) shaded atomically at cycle start; Yuasa deletion barrier
shades the OLD target of every gcref edge deleted while marking (SETF
overwrites + every owned-death path, which all funnel through wo_drop_kind's
GCREF case); allocations mid-cycle born black. Mark AND sweep budgeted
(WO_GC_BUDGET objects/slice), sweep resumes via a cursor; gray-worklist OOM
degrades to a blacken-all cycle (frees nothing, never wrong). Owned interiors
walked eagerly (single-owner trees), pruned by a per-class may-gcref bit
computed at rt_init (fixpoint over kinds + v2 field_class/field_elem;
conservative when metadata is absent).
- vm.c: safepoints at NEW (the heap-goal trigger), CALL, and backward JMP;
root scan follows vm_unwind's governing-pc convention. Unwind's gc-mask
branch just nulls the register. RC_INC/RC_DEC are accepted as no-ops until
the emitter stops producing them (next commit) — which also deletes the old
RC_DEC-on-nil trap that broke `?Node` gcref field stores.
- main.c pump: post-exit, a rootless cycle frees everything unreachable in
budgeted slices; the trace line moved into wo_gc_slice (one format for pump
and in-program slices). rt_destroy frees traced remnants (trap paths, tests).
- WO_GC_GOAL joins WO_GC_BUDGET/WO_GC_TRACE as an rt-owned knob (default 256
KiB; a tiny goal forces mid-program cycles for testing).
- tests: test_cycle.c rewritten (abandoned cycle freed, rooted cycle survives,
slices bounded, cycle-through-multi, repeated-cycle leak-freedom, and the
spec's load-bearing DELETION-BARRIER test: an object hidden behind a black
object mid-mark must survive). test_rc.c re-pinned to owned drops + the
owned/traced boundary; test_obj.c asserts tracked-white-linked instead of
rc=1.
Verified: make test + test-iso (all suites, ASan/UBSan; test_cycle 42/0,
test_rc 14/0) + cli_smoke; oop-e2e 79/0 (gc corpus traces unchanged: the new
slice math reproduces steps=1/freed=2 and steps=2/freed=4); employee 8/0;
log-watcher 7/0. THE RING RUNS: docs/examples/gc-cycle prints
`ring a -> b -> c -> a`, is reclaimed post-exit (freed=3 remaining=0), is ASan
clean, and survives an in-program cycle while rooted (WO_GC_GOAL=64: mid-run
slice frees 0, post-exit frees 3).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- wo_row_update_field: encode new value, unique re-check against a
shadow BEFORE any mutation (violating update leaves the row
untouched, DB_ERR_UNIQUE), index entries moved old-hash -> new-hash,
old engine value freed; proven by test_table (unique refusal keeps
the row, released key becomes insertable)
- WAL UPDATE record: full-row re-log, replay = replace (remove +
re-create same id); prefix/suffix delta recorded as later
optimization; test_wal replays insert+update to the updated state
- builtins 62 DB_UPDATE_FIELD (cid,id,field,value) and 63 DB_DELETE
(cid,id), commit-before-ack like insert, WO_T_UNIQUE/WO_T_DB/WO_T_IO
mapping; dispatch range 61..63; loader arities; runner mirror
- plan Task 5 marked superseded-in-part with the recorded deviation:
the language surface (reads, queries, row views, delete statement)
is 9b's, where the comprehension design put it -- no interim brace-
select grammar to retire later
- gates: test_table 839/0, test_wal 102/0, 15 suites, oop-e2e 73/0
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- .wob v3: class records carry an index tail (flags bit0 = unique,
col_cnt, columns) -- @table(index:[a,b]) entries plus one unique
single-column entry per @unique field; loader validates columns in
range and scalar/Text-kinded; emitter validates the declarations
(unknown column, un-indexable kind => diagnostic)
- engine: db_index hash multimap per table, built from the class
table at first touch, maintained ONLY inside wo_row_insert/
wo_row_remove; unique checks re-compare actual column values (a
hash is a hint); replay re-indexes via wo_row_raw_commit AFTER
slots are filled, so recovered tables carry their indexes
- WO_T_UNIQUE = 10; a violating insert is un-applied whole (bitmap,
hash, count, and the never-observable id reclaimed) and traps
catchably -- the employee SEED-DUP pattern
- wo_row_insert gains err_kind so db.c maps UNIQUE/OOM/other to the
right trap; test images and the runner's loader mirror speak v3
- fixtures: trap/db-unique-violation (code 10 exact) and
run/db-unique-catch (catchable dup, composite index accepts
duplicates, next id dense after a refusal)
- gates: oop-e2e 73/0, all 15 runtime suites, woc-test green,
log-watcher 7/0
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- database/src/wal.{c,h}: framed records len|crc32|payload|mark
("WOL1" written last -- no mark, no record), typed-row payloads
walking the class-table kinds (nested records, containers, nil
encodings), little-endian like the loader
- commit order verbatim from the shipped phase-D pattern: RAM apply,
stage, ONE pwrite + ONE fdatasync for the batch, ack after -- group
commit is everything staged riding one sync
- replay decodes straight into engine-owned values (no VM at boot)
and re-enters rows through the choke-point row API, so Task 4's
indexes will rebuild for free; next_id advances past replayed ids
this shard owns (wo_row_create_raw)
- torn tail = short/CRC-fail/no-mark/zero-len: intact prefix applies,
tear dropped whole, wo_wal_open positions AT the tear so the next
commit overwrites it; CRC-valid-but-undecodable = corruption, loud
- wo_wal_check: offline oracle, no engine needed -- the crash
battery's verifier
- test_wal 90/0 ASan+UBSan incl. five crash-battery rounds (fork,
insert/commit/ack-over-pipe, SIGKILL mid-stream: zero acked-but-
missing, zero acked-but-wrong); all runtime suites green, oop-e2e
71/0; binding doc WAL section + CODE-LOGIC + plan Task 2 checked
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- database/src/table.{c,h}: per-shard per-class slabs (256 rows,
malloc'd, never moved -- row addresses stable for 9b's row views),
occupancy bitmap, LIFO slot reuse, open-addressing id hash with
tombstones (ids never 0, never reused)
- field encoding walks the same .wob class-table kinds the VM walks:
scalars raw (WO_NIL_SCALAR passes through), Texts copied to db_text,
owned objects flattened recursively to db_rec, containers
element-wise; GCREF refused at encode (the GC bulkhead, defensively)
- two one-way copy gates: insert copies VM values in, read allocates
fresh VM values out -- no VM pointer in a slab, no slab pointer in
the VM, proven by mutating originals after insert
- id discipline: per table per shard, S+1 step N; owner = (id-1) % N;
N-parametric, runs at N=1 until iteration 8, tested at N=3
- choke points: wo_row_insert/wo_row_remove carry the INDEX HOOK
sites Task 4 attaches to; nothing else mutates storage
- runtime/Makefile links database/src into every wovm + test binary
- test_table 827/0 ASan+UBSan; oop-e2e 71/0; log-watcher 7/0;
binding doc docs/plan/oop-vm/04-db-binding.md; CODE-LOGIC.md beside
the code; plan Task 1 checked off
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
docs/examples/log-watcher (1285 lines, 7 files) now compiles clean: 0
diagnostics, a 35KB .wob written. corpus 71/0, woc runtest 565/0, every wovm
unit gate green (both dispatch flavors).
- .wob v2: each class row gains three u32 per-field arrays — the field's NAME
constant, the CLASS it refers to (or the json-raw marker), and a container
field's ELEMENT kinds. json is then a runtime service driven by metadata
instead of per-type generated code. loader/emitter/disassembler/test
assembler all read and write v2; field-name constants are interned with the
rest of the pool (interning during serialization silently loses them)
- runtime/src/json.c (new): encode by static kind + object headers + class
table (nested records need no static knowledge); decode parses and BINDS
straight into the target class — keys matched to field names, nested objects
built as the field's class, arrays as a multi of the field's element kind,
unknown keys skipped, absent keys nil. Malformed input is nil, never a trap
- `as`: `json.decode(text) as T` is the one cast this language has (WO-E403
for any other `as`, and for a bare json.decode with no target type). Its
result is `?T`, which is why the decode and the target are one instruction
- json.Value: a reserved type name for a value the source does not inspect —
the raw JSON slice, kind TEXT, re-emitted verbatim by encode
- docs: 00-wob-format.md is now the v2 reference (class metadata, TRY/ENDTRY,
the whole builtin surface, WO_T_IO); 08-builtin-surface.md documents the
text/container builtins, the OS modules with their predeclared records, and
json's two documented limits (Bool encodes 0/1, floats truncate)
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- 16 tasks complete: arena, object model, borrow word, containers,
RC + budgeted cycle collector, wob_build, validating loader,
interpreter core (dual dispatch), object opcodes, drop-map unwinding,
builtins + DB_STUB + TRAP, ICALL, wovm CLI + just recipes
- 13 test suites × 2 dispatch flavors (ASan+UBSan) + CLI smoke, all green
- .wob v1 format pinned in src/wob.h + docs/plan/oop-vm/00-wob-format.md
- wo-rt.c reference event-loop preserved for sub-project 2
This is Iteration 2 of the OOP milestone; compiler front (Iteration 3)
is in progress on this branch. They meet at Iteration 4 (emitter+e2e).
