shoney.arickathil
da30aa6527
- driving case: a 120 GB order table on a 32 GB host. Not a tuning problem; no eviction policy fixes it. Developer accepted reconsidering the principle - principle 7 rewritten: durability half UNCHANGED and unconditional (WAL-logged, fsync before ack, CRC-dropped torn tail); residency half demoted from law to per-table declaration. Old wording quoted in place so the amendment is legible, with the reason: a doctrine a real workload cannot satisfy gets ignored, and the failure it produced was an OOM kill - spec: docs/superpowers/specs/2026-08-26-table-residency-design.md One log-structured engine — the WAL already holds every row, so keep an in-RAM id->offset map and pread rows back. No second engine, no user-space row cache (the kernel page cache is the hot copy, which is already this repo's stated position and why it avoids O_DIRECT) - arithmetic that makes it work: 240M rows x 16 B of index = ~3.8 GB resident in 32 GB. Indexes stay resident, rows do not. Buys ~2 orders of magnitude, not infinity — stated plainly in the spec - grammar: two optional keys, `durable: true|false` and `resident: all|index`, both defaulting to today's behaviour, so all 28 existing @table declarations compile untouched and no golden is reblessed - rejected, with reasons recorded: mmap (rows are pointer-bearing — table.c returns (uintptr_t)t as the slot word), buffer pool (the Rust-era phase-12 design that died with that track), paged B-tree (stays rejected), a three-valued enum, automatic spill, disk-backed-by-default - self-review caught the budget defaulting to "none" while promising the ERP developer a diagnostic instead of the OOM killer — contradiction fixed: the budget defaults to a fraction of host memory, and its value comes from databasev2 1's swap-onset measurement - live docs that contradicted the amendment updated (subagent doctrine, its guide, discarded.md's two rows, iteration 04's read claim, 07, 38); dated specs/plans left as records. linkcheck 0 broken / 0 anchors Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
169 lines
9 KiB
Markdown
169 lines
9 KiB
Markdown
# The writeonce principles
|
||
|
||
The doctrine in one page. Every design argument in this repo eventually
|
||
lands on one of these thirteen; later documents link here instead of
|
||
re-arguing them. Each principle: what it is, why it holds, where it is
|
||
enforced.
|
||
|
||
## 1. One binary is the whole system
|
||
|
||
The application, the database, the API, and (later) the UI ship as a single
|
||
deployable — there is nothing else to install, operate, or version-skew.
|
||
*Why:* the assembled-stack tax (app + DB server + proxy + glue) is the
|
||
problem writeonce exists to delete.
|
||
*Enforced by:* [`01-problem.md`](01-problem.md), the single-binary story in
|
||
[the OOP spec](superpowers/specs/2026-08-01-oop-compiler-vm-design.md).
|
||
|
||
## 2. Zero dependencies — kernel primitives only
|
||
|
||
The runtime is C on libc; the compiler is OCaml on its stdlib; everything
|
||
else is epoll/io_uring, inotify, eventfd, signalfd, sendfile, mmap. The
|
||
kernel is the framework.
|
||
*Why:* every dependency is a supply chain, an upgrade treadmill, and a
|
||
black box in the one binary that must be understood end to end.
|
||
*Enforced by:* [the kernel-primitives catalogue](plan/exploration/linux/00-linux.md),
|
||
the dependency doctrine in [the OOP spec](superpowers/specs/2026-08-01-oop-compiler-vm-design.md).
|
||
|
||
## 3. Memory safety without a GC tax
|
||
|
||
Objects are owned values: one owner, moves on assignment, second-class
|
||
borrows checked mostly at compile time (mutable value semantics — the
|
||
Rust-borrow shape without lifetime inference). GC-ness is **inferred** by
|
||
the compiler (iteration 7b): a class in a reference cycle, or one whose
|
||
values must escape as long-lived aliases, is traced by an incremental
|
||
per-shard tri-color mark-sweep collector in budgeted slices — the developer
|
||
writes no memory annotation, and no global pause exists by construction.
|
||
*Why:* deterministic memory for the default case, aliasing freedom where
|
||
the design wants it, and never a stop-the-world in a runtime that is also
|
||
the database.
|
||
*Enforced by:* [the OOP spec §4](superpowers/specs/2026-08-01-oop-compiler-vm-design.md).
|
||
|
||
## 4. No inheritance, ever
|
||
|
||
No `extends`, no `override`, no virtual hierarchies. Is-a is a tagged
|
||
union; has-a is composition; polymorphism is structural interfaces.
|
||
*Why:* hierarchies fossilize early guesses and make dispatch, ownership,
|
||
and diagnostics all harder; composition keeps every unit flat and movable.
|
||
*Enforced by:* [the OOP spec](superpowers/specs/2026-08-01-oop-compiler-vm-design.md),
|
||
the reject rows of [the systems-track verdict table](superpowers/specs/2026-08-01-systems-track-design.md).
|
||
|
||
## 5. Thread-per-core shards; ownership moves, data never shares
|
||
|
||
One pinned worker per core, each owning its engine, heap, and event loop.
|
||
Cross-shard work is a message send that moves ownership. There is no
|
||
`Arc<Mutex<…>>` anywhere and never will be.
|
||
*Why:* sharing mutable state buys contention, locks, and heisenbugs;
|
||
moving ownership buys linear scaling and per-shard GC.
|
||
*Enforced by:* [the shard-fiber arc plan](superpowers/plans/2026-08-20-shard-fiber-arc.md)
|
||
(stages 1+2 landed; supersedes the discarded 2026-08-01 shard-actor plan
|
||
and the Rust-era plan 09, removed with that track 2026-08-18).
|
||
|
||
## 6. The runtime never stops
|
||
|
||
The executable is a systemd service that deploys without restarting: two
|
||
VM slots (Blue/Green), in-runtime compile of an approved proposal, atomic
|
||
dispatch switch, previous version resident for instant rollback — and the
|
||
binary embeds its own source, so prod is always self-describing.
|
||
*Why:* restarts drop connections, dump caches, and turn deploys into
|
||
events; a database that is also the app must not blink.
|
||
*Enforced by:* [the blue-green spec](superpowers/specs/2026-08-03-blue-green-vm-design.md).
|
||
|
||
## 7. The log is authoritative; residency is a declared per-table policy
|
||
|
||
**Amended 2026-08-26.** This principle read "RAM is authoritative; the WAL
|
||
makes it durable. All reads serve from memory." The durability half was never
|
||
under strain and is unchanged. The residency half was false for a real
|
||
workload, so it is now a declaration rather than a law.
|
||
|
||
**Durability, unconditional:** every mutation is WAL-logged and fsynced before
|
||
acknowledgment; boot replays the log; a torn tail is dropped whole by CRC; an
|
||
ack means the commit reached disk. Mirrors (Postgres) are reconstructible
|
||
backups that reads and acks never depend on. None of this is per-table and
|
||
none of it is negotiable.
|
||
|
||
**Residency, declared:** what a table keeps in memory is stated at the
|
||
declaration site. The default keeps every row resident and serves reads at
|
||
memory speed. A table that cannot fit says so, and then only its indexes are
|
||
resident while rows are read from the log by offset — the kernel page cache is
|
||
the hot copy, which is why the engine uses `pread` and deliberately not
|
||
`O_DIRECT`.
|
||
|
||
*Why the amendment:* the original wording is right for a knowledge-management
|
||
app and simply false for a 120 GB order table on a 32 GB host. A doctrine a
|
||
real workload cannot satisfy does not get followed, it gets ignored — and the
|
||
failure it produced was an OOM kill, which is the least debuggable outcome
|
||
available. The fix keeps one storage engine and one source of truth: the log
|
||
*is* the database, and RAM is how much of it you choose to serve fast. What was
|
||
rejected in 2026-08-18 and stays rejected is a *second* engine — a paged
|
||
B-tree with its own buffer pool ([`plan/discarded.md`](plan/discarded.md)).
|
||
Reading rows from the log we already write is not that.
|
||
|
||
*Enforced by:* [the db-engine binding plan](superpowers/plans/2026-08-01-db-engine-binding.md)
|
||
(typed WAL + boot replay, shipped); the residency declaration and its
|
||
enforcement are [databasev2 2](stories/databasev2/02-table-storage-modes.md);
|
||
the mirror-is-backup doctrine is recorded in
|
||
[`plan/discarded.md`](plan/discarded.md) (the Rust-era WAL and mirror plans
|
||
11/16 were removed with that track 2026-08-18).
|
||
|
||
## 8. Samples force the grammar
|
||
|
||
Language features exist when a sample program exercises them; the examples
|
||
directory is the de facto integration suite, and new surface is proven by
|
||
re-expressing real workloads (blog, ecommerce, pricing, log-watcher).
|
||
*Why:* grammars designed in the abstract grow features nobody needs and
|
||
miss the ones real programs demand.
|
||
*Enforced by:* [the web-app sample](examples/web-app/README.md)
|
||
(the blog sample left with the Rust track),
|
||
the sample-workload acceptance in [the systems-track spec](superpowers/specs/2026-08-01-systems-track-design.md).
|
||
|
||
## 9. Linux is the target
|
||
|
||
Not POSIX, not portable-someday: Linux syscalls, Linux fd semantics,
|
||
systemd as the process manager. Portability abstractions are refused.
|
||
*Why:* targeting one kernel lets the runtime use its sharpest primitives
|
||
directly instead of the lowest common denominator.
|
||
*Enforced by:* [the kernel-primitives catalogue](plan/exploration/linux/00-linux.md).
|
||
|
||
## 10. Capabilities are typed builtins — no FFI
|
||
|
||
Programs reach the system only through audited stdlib builtins — six
|
||
reserved namespaces (`fs`, `proc`, `net`, `time`, `json`, `env`): bounded
|
||
reads, args-array-only process runs, handles that close on drop. There is
|
||
no `extern`, no escape hatch.
|
||
*Why:* one FFI hole voids the entire memory-safety and security story;
|
||
typed capabilities make the safe path the only path.
|
||
*Enforced by:* [the systems-track spec Parts 2–3](superpowers/specs/2026-08-01-systems-track-design.md).
|
||
|
||
## 11. Plain diagnostics are the product
|
||
|
||
Stable `WO-E###` codes, `file:line:col`, source excerpts, ownership errors
|
||
naming both sites, many errors per run.
|
||
*Why:* mutable value semantics only beats Rust ergonomics if the errors
|
||
read like sentences; the compiler's error text is a first-class feature.
|
||
*Enforced by:* [the OOP spec §6](superpowers/specs/2026-08-01-oop-compiler-vm-design.md),
|
||
[the compiler architecture doctrine](plan/compiler/architecture.md).
|
||
|
||
## 12. The runtime is a recipe box
|
||
|
||
Transports, fibers, routing, subscriptions, the DB engine, deploy
|
||
machinery — each stays a separable capability. A web framework or a custom
|
||
database experience is a `.wo` library composing them; the runtime itself
|
||
stays framework-agnostic.
|
||
*Why:* the next stories (web framework, richer database surfaces) must be
|
||
buildable *on* the runtime without forking it.
|
||
*Enforced by:* [the blue-green vision §2](plan/exploration/blue-green-vm/00-vision.md).
|
||
|
||
## 13. Statically typed, all the way to the register
|
||
|
||
Every slot's type is known at compile time: no `Dynamic`, no `untyped`, no
|
||
`cast`, no runtime reflection. The VM runs untagged 64-bit registers
|
||
because the compiler already knows; JSON enters through checked decodes
|
||
(`as T` yielding `?T`), never through dynamic objects.
|
||
*Why:* the type system is the foundation the untagged VM, the borrow
|
||
checker, and the annotation ORM (`@table` classes, `ref`/`multi`
|
||
relations) all stand on — one dynamic hole collapses all three. The Haxe
|
||
reference workload shows the alternative: its transcompiled C++ pays a
|
||
hashed `__Field` lookup on every typedef access.
|
||
*Enforced by:* the `Dynamic`/`untyped`/`cast` reject rows of
|
||
[the systems-track verdict table](superpowers/specs/2026-08-01-systems-track-design.md),
|
||
untagged registers in [the OOP spec §5](superpowers/specs/2026-08-01-oop-compiler-vm-design.md).
|