shoney.arickathil
d7304f23ec
- Board renamed docs/plan/00-kanban.md -> docs/00-status.md and rebuilt: ▶ NEXT PLAN pointer (iteration 4 — emitter, corpus, `woc build`) then six buckets — stories, in progress, done, pending, discarded, learnings. It covered only the Rust runtime before, so the whole OOP track was invisible. All 16 inbound refs repointed; `Kanban:` banners renamed to `Status:`. - New discarded.md (settled rejections with reasons: inheritance, `abstract`, Money/SKU/Float, Dynamic/cast/macro/extern, AOT-to-C, Menhir, shared engine state) and learnings.md (plumbed≠enforced, vacuous goldens, exit-0-wrong- output, malloc-path ASan trick, deferred checks that never reach the VM). - RECOVERED docs/plan/exploration/blue-green-vm/00-vision.md — gone from disk, never committed (gitignored path), cited by five docs incl. principle 12. Root cause was broader: all seven forward-roadmap plans in docs/superpowers/plans/ were untracked and ignored, on one disk only. Dropped the docs ignore rules with a do-not-re-add note; added __pycache__/*.pyc. - Repaired broken links across docs/, 270 -> 36: fixes a regression from the earlier reference/ -> .dev/reference/ move (relative paths at ../../ and deeper were skipped), plus depth and reorg drift. The 36 residual point at content that does not exist and need decisions, not paths. - New spec docs/superpowers/specs/2026-08-10-logwatcher-gap-closure-design.md, applied: `and`/`or` verdict row; Part 3 gains `env` (six modules), swaps time.mono for iso/local, adds 22 bare core builtins; throw/time.mono/is cut (0 uses in the sample). Plan 8: Task 2 gains and/or, Task 5 drops throw, abstract+`is` task deleted, 8/9 renumber to 7/8. Plan 9 gains core builtins. Plan 10 gains the 307 -> 0 diagnostic gate. WO-E205 re-filed unreachable-by- design. types.ml header drops its false satisfaction-set claim. 00-code- review.md reduced to a stub — its rival Phase 1-4 roadmap retired.
92 lines
9.8 KiB
Markdown
92 lines
9.8 KiB
Markdown
# 16 — PostgreSQL mirror: RAM-authoritative database, Postgres as the backup
|
||
|
||
> **Status: 🔄 in progress (Track 3 — Storage & durability)** — 16a ✅, 16b ✅ shipped; 16c–16f ⬜. Board: [00-status.md](../00-status.md)
|
||
|
||
**Context sources:** [`README.md` § persistent database](../../README.md) (the product goal this implements: *"reads and writes database to RAM, persist data to postgres SQL"*), [`../runtime/database/03-inmemory-engine.md`](../runtime/database/03-inmemory-engine.md) (RAM-resident doctrine: disk sits behind the read path, never in front), [`./09-concurrency-scaleout.md`](./09-concurrency-scaleout.md) (per-shard WAL + ack-after-fsync this rides behind), [`./13-class-model-live-pricing.md`](./13-class-model-live-pricing.md) (the Product/Price worked example; `@table(name: "prices")` names the mirrored table), [`../runtime/database/07-wo-seg-migration.md`](../runtime/database/07-wo-seg-migration.md) (the dual-write precedent), `reference/postgresql/` (research symlink — `src/include/libpq/` for the wire protocol), PostgreSQL docs *Frontend/Backend Protocol*.
|
||
|
||
## Context
|
||
|
||
The engine is RAM-resident by design and durable through its own per-shard WAL (09c). What's missing is an **external, queryable, operator-friendly copy** of the data — something a DBA can point `psql`, Grafana, or a nightly `pg_dump` at. That's what Postgres is here: **a backup mechanism**, not a storage engine.
|
||
|
||
The contract, in one line each:
|
||
|
||
- **The entire database lives in RAM.** Reads never touch Postgres — ever.
|
||
- **Writes go to RAM (+ WAL) and to Postgres** — but the Postgres write is asynchronous: the client's ack gates on the WAL fsync exactly as before; the mirror follows behind.
|
||
- **Postgres is disposable.** RAM is authoritative, so backup repair is always "re-push RAM state" — never a merge.
|
||
|
||
Worked example throughout: `Product.set_price(amount)` from the [pricing demo](../examples/pricing/) → a `Price` row in RAM → the same row visible in `psql` as `SELECT * FROM prices`.
|
||
|
||
## Design decisions (locked)
|
||
|
||
1. **Async mirror, never a commit path.** The `wo-pg` thread is downstream of the commit: shard engines clone committed mutations onto a bounded channel (`try_send` — a full channel drops loudly, it never blocks a worker). Postgres being down costs clients nothing.
|
||
2. **Mirror what RAM holds.** The tap emits full post-merge rows (`MirrorRec::Upsert{ty, id, row}` — unlike `WalRec::Update`, which carries only the merge body), so a Postgres row is always byte-equivalent to its RAM row. Method transactions (13b) mirror as one `MirrorRec::Txn` → one `BEGIN…COMMIT` — an aborted method never reaches the channel at all.
|
||
3. **Hand-rolled wire client, zero new crates.** `crates/rt/src/pg.rs` speaks protocol v3 (startup, auth `trust`/`password`/`md5` with a hand-rolled MD5 — the CRC32 precedent; SCRAM is 16f) over a blocking `std::net::TcpStream`, **simple query protocol only**. Blocking is fine: the only caller is the dedicated mirror thread.
|
||
4. **One `wo-pg` thread per process.** N shard senders → one receiver; per-row ordering is preserved because a row's mutations always come from its owner shard (one FIFO sender). Batches drain opportunistically; statement errors are isolated per record (logged, skipped), socket errors reconnect with capped backoff.
|
||
5. **Boot = full resync.** Engines attach the mirror AFTER WAL replay and push their entire state as upserts (`mirror_sync_all`). Consequence: restarting `wo` against a fresh/empty/behind Postgres converges it — verified live (a record dropped during an outage reappeared after restart).
|
||
6. **Schema 16b: one JSONB table per type** — `"<storage_name>" (id BIGINT PRIMARY KEY, row JSONB NOT NULL)`, named by `@table(name: "prices")` (default: the type name). Upsert = `INSERT … ON CONFLICT (id) DO UPDATE`. Typed columns are 16c.
|
||
7. **Config: `WO_PG=postgres://user[:pass]@host[:port]/db`** env var (the `WO_DATA`/`WO_LISTEN` convention). Unset = mirror off, zero cost.
|
||
8. **The WAL stays the recovery source; Postgres is the backup of last resort.** Boot replay reads the WAL as today; restoring FROM Postgres (WAL lost) is 16e's explicit opt-in.
|
||
|
||
## Sub-phase sequence
|
||
|
||
### `16a` — hand-rolled wire client — ✅ shipped
|
||
|
||
`crates/rt/src/pg.rs` (~450 lines, stdlib only): `PgConfig::from_url`, `Conn::connect` (startup + auth trust/cleartext/md5, RFC-1321 MD5 hand-rolled with test vectors), `simple_query` (RowDescription/DataRow/CommandComplete/ErrorResponse/ReadyForQuery; a backend error drains to ready so the connection stays usable), `escape_literal`/`escape_ident`.
|
||
**Exit (met):** unit tests (MD5 vectors, URL forms, escaping) green; gated integration test (`WO_PG_TEST=…`) round-trips DDL/upsert/select/error-recovery/multi-statement against `postgres:16`; md5-auth container connects with the right password and fails cleanly with the wrong one.
|
||
|
||
### `16b` — the mirror pipeline — ✅ shipped
|
||
|
||
`crates/rt/src/mirror.rs`: `MirrorRec{Upsert, Delete, Txn}`, `spawn(cfg, rx, tables)` → the `wo-pg` thread (DDL bootstrap per (re)connect, batched apply, per-record error isolation, capped-backoff reconnect, loud drop accounting). Engine tap (`crates/rt/src/engine.rs`): `attach_mirror`, `mirror_send` (txn-buffered like the WAL buffer — dispatched as one `Txn` on commit, dropped on abort), `mirror_sync_all` boot push; taps sit AFTER `wal_log` accepts in `create`/`update`/`delete`/`commit_txn`. Wiring in `bin/wo.rs` behind `WO_PG`.
|
||
**Exit (met, verified live on 2 shards):** `set_price` rows appear in `psql` under the `@table` name `prices` with full JSONB; the abort case (`amount: 0` → 409) mirrors **nothing**; `docker stop` mid-writes → writes keep acking 200, reads unaffected; fresh empty container → reconnect + DDL bootstrap + new writes flow; `wo` restart → WAL replay + bulk sync **heals the gap** (the row written during the outage appeared). 69 unit tests green; blog/ecommerce/hello unchanged without `WO_PG`.
|
||
|
||
### `16c` — typed schema projection
|
||
|
||
Catalog scalar fields become real columns (`Id`/`Int`/`Money`/`ref` → `BIGINT`, `Text`/`Timestamp`/unions → `TEXT`, `Bool` → `BOOLEAN`; arrays/structs stay in a residual `row JSONB`); `@table(index: [product, at])` → `CREATE INDEX IF NOT EXISTS`; schema evolution via `ADD COLUMN IF NOT EXISTS`.
|
||
**Exit:** `SELECT avg((row->>'amount')::bigint)` becomes `SELECT avg(amount) FROM prices WHERE product = 1` in psql, using the mirrored index.
|
||
|
||
### `16d` — failure & lossless resync
|
||
|
||
Replace drop-and-log with dirty-flag repair: channel overflow or reconnect marks shards dirty; workers re-enqueue their tables at tick (the 09b mail-eventfd mechanism), so convergence no longer waits for a process restart. Mirror lag + queue depth + drop counters surface on `GET /`.
|
||
**Exit:** kill Postgres under sustained write load, restart it → row counts converge with zero client errors and no `wo` restart.
|
||
|
||
### `16e` — restore from Postgres
|
||
|
||
Boot source of last resort when the WAL is gone: `WO_PG_RESTORE=1` makes each shard `SELECT id, row FROM …` its own partition (`(id-1) % n = shard`) before arming accept, seeding RAM and re-logging a fresh WAL.
|
||
**Exit:** `rm -rf wo-data` → boot with restore → `current_price` answers from the restored state; id high-water marks keep the interleave.
|
||
|
||
### `16f` — SCRAM-SHA-256 auth
|
||
|
||
Hand-rolled SHA-256 + HMAC + PBKDF2 (RFC 7677 exchange) so stock `postgres:16` works without `pg_hba` changes.
|
||
**Exit:** connects to an out-of-the-box scram-auth server; wrong password fails with the server's error.
|
||
|
||
## Verification (16a/16b, reproducible)
|
||
|
||
```bash
|
||
docker run -d --rm --name wo-pg -e POSTGRES_HOST_AUTH_METHOD=trust -e POSTGRES_DB=wo -p 54329:5432 postgres:16
|
||
WO_PG_TEST=postgres://postgres@127.0.0.1:54329/wo cargo test --lib -- pg_ mirror_ # integration tests
|
||
just pricing-pg-demo # scripted end-to-end
|
||
```
|
||
|
||
| Check | Result |
|
||
| --- | --- |
|
||
| `set_price 4999/5999` → `psql: SELECT * FROM prices` | rows present, full JSONB, `@table` name honoured |
|
||
| Aborted method (`amount: 0` → 409) | nothing in Postgres — only committed txns mirror |
|
||
| Postgres stopped mid-writes | writes ack 200, reads unaffected, mirror retries with backoff |
|
||
| Fresh empty database on reconnect | DDL bootstrap recreates tables, stream resumes |
|
||
| `wo` restart against behind/empty Postgres | WAL replay + boot sync converges it (outage gap healed) |
|
||
| No `WO_PG` | zero behavioural change; 69 unit tests green |
|
||
|
||
## Non-scope
|
||
|
||
- **No reads from Postgres on any serving path** — doctrine; even 16e's restore happens before accept is armed.
|
||
- **No TLS** to Postgres (mirror a local/private endpoint; revisit with 16f).
|
||
- **No extended query protocol / prepared statements** — simple protocol is enough for a backup writer; revisit only if 16c profiling demands it.
|
||
- **No two-way sync / conflict resolution.** Postgres is write-only from writeonce's perspective (16e restore excepted); external writes to the mirrored tables are unsupported and will be overwritten.
|
||
- **No dependency creep.** `crates/rt` gains no crates for this plan — the wire client is part of the same hand-rolled surface as the HTTP layer.
|
||
|
||
## Cross-references
|
||
|
||
- [`./10-storage-foundations.md`](./10-storage-foundations.md) / [`11`](11-wal-and-recovery.md) / [`12`](12-engine-disk-cutover.md) — the native disk engine; the mirror is orthogonal (external queryable backup vs. native durability) and both sit behind the RAM read path.
|
||
- [`./13-class-model-live-pricing.md`](./13-class-model-live-pricing.md) — `@table(name:)` names the mirrored tables; 13b method txns map to Postgres txns; the pricing demo is the acceptance workload.
|
||
- [`../runtime/database/07-wo-seg-migration.md`](../runtime/database/07-wo-seg-migration.md) — the dual-write pattern precedent.
|
||
- [`./15-mcp-streamable-http.md`](./15-mcp-streamable-http.md) — the other "speak an established protocol, hand-rolled" track; 16a is to Postgres what 15a is to MCP.
|