writeonce/docs/superpowers/specs/2026-08-31-schema-migrations-design.md

Schema migrations v1 — add + delete, declarative, auto on boot

Brainstormed and approved 2026-08-31. Story: databasev2 12. Scope settled during brainstorm: declarative model, add/delete only, zero-value fill, data/seed migrations deferred to v2.

The problem

A @table class is the schema and WO_DATA's log is the database, but the two are never compared. A record names its class by cid — the class's declaration index — and replay decodes every record against the compiled shape. So today, all verified:

• Adding a field makes every old record under-run its decode and replay refuses the whole log as "corruption".
• Deleting a field leaves trailing bytes — the same refusal.
• Reordering @table declarations renumbers cids and rows decode into the wrong class, silently when shapes happen to match.
• Nothing records which schema wrote the log. The durable/resident mode mismatch is checked at boot (databasev2 2); the shape is not.

Changing a @table class is a database migration. The language performs it.

The model, and why

Declarative, automatic, at boot. The binary already carries the schema — wo_classdesc has the class name and per-field names (v2 metadata), so no .wob format change is needed. At startup the runtime compares the shape recorded in the log against the compiled shape and either replays (match), migrates (add/delete), or refuses (everything else). No migration files, no offline step: writeonce's doctrine is one binary and no ops choreography, and a migration artifact would be a second source of truth for a schema the class declaration already states.

v1 verbs: add and delete only. They are the two changes whose data consequence is unambiguous: an added field has no stored values and takes the kind's zero value (Int 0, Float 0.0, Bool false, Text empty, optional nil); a deleted field's stored values are freed and physically dropped when the rewritten log is swapped in. Zero-fill is chosen over a default-value syntax because the grammar has no field defaults today and v1 refuses to grow compiler surface for a value the program can backfill itself.

Everything else refuses, precisely. Refusal messages follow the resident: keys house style: name the class, the field, the stored shape, the compiled shape, and what to do about it.

• A same-named field with a different kind: refused — no implicit conversions.
• A deleted field and an added field of the same kind in one step: refused — byte-for-byte indistinguishable from a rename, and one interpretation destroys data the other preserves. The message says to deploy the delete and the add as two separate steps when both are genuinely meant. (@renamed_from is the v2 answer.)
• A stored class absent from the compiled program: refused — its rows demand a verdict the runtime must not guess. (The existing durable-to-volatile refusal from databasev2 2 stays as-is.)
• Index or storage-flag changes: refused in v1.

Mechanics

The schema record. A new WAL record kind, WO_WAL_SCHEMA, CRC-framed like every other record, describing every durable class: class name, storage flags, and per field its name and kind plus the container/reference metadata, then the secondary-index layout. It is written as the first record of a fresh log and as the first record compaction writes, so the head of a log always states the shape of everything after it.

The boot diff. Before replay, the runtime peeks the log head.

• No schema record — a legacy log. Replay proceeds against the compiled shape exactly as today; the next compaction writes the record and the log is self-describing from then on. A legacy log whose shape already changed still fails as today: there is nothing recorded to diff against.
• Record present and equal to the compiled shape — normal replay. Equality is by class NAME and field NAME + kind, so pure declaration reordering is a match, which closes the silent cid-renumbering hole as a side effect.
• Record present and different — classify per class into migrate or refuse, as above.

The migration pass. Replay decodes each record with the OLD shape, reconstructed from the schema record — old kinds drive the value decoder, old names drive the mapping. Rows are built directly in the new shape through a per-class field map: surviving fields move to their new slot, deleted fields' values are freed, added fields take the zero value. Old cids map to new cids by class name. A delta record's field index is an old-shape index; the fold happens in old shape and maps afterward, so a delta on a deleted field folds to a no-op. The pass then writes a fresh log the way compaction does — schema record, one full-row record per live row, fsync, rename — and boot continues on the new log. Keys-resident tables get their offset maps rebuilt from the new log by the machinery compaction already carries. One stderr line per migrated class reports what changed, rows rewritten, and values dropped.

Crash safety is inherited, not added. The rename swap is atomic: a crash anywhere in the pass leaves the old log intact and the next boot re-migrates from it. This is databasev2 3's mutation-proven design, reused.

Acceptance criteria

• Given a log written with shape A and a binary compiled with shape A plus a new field, when the program boots, then it runs, the field reads as the kind's zero value, and the log head carries the new schema.
• Given shape A minus a field, when it boots, then the dropped values are freed (leak-clean under ASan) and no record of the field remains in the rewritten log.
• Given a simultaneous delete and add of the same kind, when it boots, then it refuses naming both fields and the two-step alternative.
• Given a retype or a vanished class with stored rows, when it boots, then it refuses naming the class, field and both shapes.
• Given pure declaration reordering, when it boots, then no migration runs and every row lands in its right class.
• Given a keys-resident table across a migration, when read after boot, then folds resolve through the new offsets and a delta on a deleted field is gone.
• Given a legacy log with an unchanged shape, when it boots, then replay is byte-for-byte today's behaviour and the next compaction writes the schema record.
• Given a kill between the new log's write and the rename, when the next boot runs, then it re-migrates from the intact old log.

Out of scope (v2 candidates, each with its reason)

• Rename — needs declared intent (@renamed_from); v1 refuses the ambiguous diff instead of guessing.
• Retype / conversions — even lossless ones invite silent surprises; refusal costs one explicit backfill program.
• Dropped-class data verdicts — refusing keeps the data; deleting it is a decision, not a migration.
• Data/seed migrations (@seed(n)) — deferred by decision; the site's trap 1 stays documented in the deploy guide.
• Field-default syntax — new grammar for what zero-fill plus app code already covers.
• Index migrations — indexes are rebuilt at boot anyway; the refusal is about flags, not data, and can be relaxed later with evidence.