shoney/writeonce
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

Update plan with kanban

Browse source
This commit is contained in:
shoney.arickathil 2026-07-12 04:25:49 +02:00
parent 619aa74933
commit 7a2e09cd19
12 changed files with 101 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

79
docs/plan/00-kanban.md Normal file
View file

@ -0,0 +1,79 @@
# Plan kanban — backend: runtime, database, API
Status board for every phase doc under `docs/plan/`. **Focus: the backend** — the runtime, the database engine, and the REST/`.wo`-language API. Frontend phases are parked, not deleted. Each phase file carries a matching status banner; this board is the index.
Statuses: ✅ **done** · 🔄 **in progress** · ⬜ **not started** · ⏸ **parked (out of backend focus)**
## Board
### Track 1 — Runtime foundations (dependency removal, kernel primitives)
| Status | Phase | Doc | Notes |
| --- | --- | --- | --- |
| ✅ | 01 crate scaffolding | [done/01](done/01-scafolding-crates.md) | 15 crates |
| ✅ | 02 epoll event loop | [done/02](done/02-event-loop-epoll.md) | `runtime/netpoll_epoll.rs` |
| ✅ | 03 hand-rolled HTTP | [done/03](done/03-hand-rolled-http.md) | + keep-alive & pipelining (follow-up under plan 09) |
| ✅ | 04 tokio/axum cutover | [done/04](done/04-cutover-remove-tokio-axum.md) | deps now: anyhow, serde, serde_json, libc |
| ⬜ | 05 hand-rolled JSON | [05](05-hand-rolled-json.md) | removes serde/serde_json — **next in this track** |
| ⬜ | 06 bespoke error type | [06](06-bespoke-error.md) | removes anyhow |
| ⬜ | 07 inotify content watcher | [07](07-inotify-content-watcher.md) | `wo dev` hot reload |
| ⬜ | 08 sendfile static assets | [08](08-sendfile-static-assets.md) | needed by parked UI track too |
### Track 2 — Concurrency & scale-out (plan 09) — 🔄 in progress
| Status | Sub-phase | Notes |
| --- | --- | --- |
| ✅ | 09a thread-per-core | `runtime/scheduler.rs`, `SO_REUSEPORT`, pinned `wo-shard-<t>` workers |
| ✅ | 09b sharded engine | `shard.rs` bus; `Arc<Mutex<Engine>>` deleted; interleaved ids; fan-out lists |
| ✅ | 09c per-shard WAL | `wal.rs`; ack-after-fsync; boot replay; `meta` shard guard |
| ✅ | — keep-alive (follow-up) | reads ×3.4 → 770k/s; C phase-C sequence |
| ✅ | — io_uring group commit (follow-up) | `netpoll_io_uring.rs` raw ring; 4.7× durable writes on real disk |
| ⬜ | 09d cross-shard subscriptions | LIVE fan-out, one message per shard — pairs with Stage 3 |
| ⬜ | 09e cross-shard transactions (2PC) | needed by `fn checkout` spanning shards |
| ⬜ | 09f observability & reshard | per-shard metrics, `WO_RESHARD` |
All numbers + find-and-fix stories: [09-concurrency-scaleout.md](09-concurrency-scaleout.md) shipped notes and the [benchmark table](../../prototypes/wo-rt-c/README.md).
### Track 3 — Storage & durability (plans 10–12)
| Status | Phase | Doc | Notes |
| --- | --- | --- | --- |
| ⬜ | 10 storage foundations | [10](10-storage-foundations.md) | scope reduced: WAL framing/fallocate landed via 09c |
| ⬜ | 11 WAL & recovery | [11](11-wal-and-recovery.md) | remaining: snapshots (`.data`), compaction, WAL rotation — replay core shipped in 09c |
| ⬜ | 12 engine disk cutover | [12](12-engine-disk-cutover.md) | mmap arena engine (C phase B is the proving ground) |
### Track 4 — Language & API (`.wo` on the wire) — 🔄 in progress
| Status | Phase | Doc | Notes |
| --- | --- | --- | --- |
| ✅ | 13a class surface | [13](13-class-model-live-pricing.md) | `class` parses, CRUD serves, spec amended |
| ⬜ | 13b method execution | [13](13-class-model-live-pricing.md) | `POST /api/<t>/:id/<method>`, row-scoped txn — **the next API milestone** |
| ⬜ | 13c LIVE pricing push | [13](13-class-model-live-pricing.md) | Stage 3 scoped: subscription registry, WS at `/api/<t>/live`, replaces the 501 stub |
| ⬜ | Stage 3 wire layer | [../runtime/database/04-client-api.md](../runtime/database/04-client-api.md) | full subscription engine + wire protocol; 13c is its beachhead |
| ⬜ | 13e pricing at scale | [13](13-class-model-live-pricing.md) | wires demo to 09; hot-row reads |
Ecommerce sample status (verified 2026-06-13, `api.rest` **17/17 expected statuses pass**): route wiring, empty lists, 405 for un-exposed ops, 404 for unregistered routes, 501 Stage-3 stubs — all exactly as documented. `fn checkout`, `on startup` seeding, and status-lifecycle triggers await 13b-style execution + 09e.
### Track 5 — C proving ground — ✅ done (A–F)
| Status | Phase | Doc |
| --- | --- | --- |
| ✅ | A threads · B arena · C io_uring · D WAL · E recovery · F bench+ACID | [exploration/c-runtime/00-plan.md](exploration/c-runtime/00-plan.md) |
859k reads/s, 618k durable commits/s; found the ack-ordering + fd-ABA bugs the Rust port then avoided. Optional phase G (splice `wo-db` C++ engine) remains an idea.
### Track 6 — Frontend — ⏸ parked (out of backend focus)
| Status | Phase | Doc | Notes |
| --- | --- | --- | --- |
| ⏸ | 13d pricing UI | [13](13-class-model-live-pricing.md) | MVC triplet exists as design artifact |
| ⏸ | 14 MVC UI implementation (14a–f) | [14](14-mvc-ui-implementation.md) | htmlx engine, SCSS subset, controllers, SSR, actions, live patching |
| ⏸ | UI exploration track | [exploration/ui/00-overview.md](exploration/ui/00-overview.md) | 00–08 design docs stay current |
## Suggested order of play (backend)
1. **13b — method execution** (unblocks `fn checkout` semantics, the ecommerce sample's core promise)
2. **13c / Stage 3 LIVE** with **09d** fan-out (turns every 501 stub real; the ecommerce order-ops board's backend)
3. **09e — 2PC** (cross-shard `fn checkout` — the canonical ACID demo end-to-end)
4. **05/06** dependency removal (mechanical, any time)
5. **10–12** storage completion (snapshots/compaction; arena engine)

2
docs/plan/05-hand-rolled-json.md
View file

@ -1,5 +1,7 @@
# 05 — Hand-Rolled JSON
> **Kanban: ⬜ not started** — Track 1 (runtime foundations), next in the dependency-removal sequence. Board: [00-kanban.md](00-kanban.md)
**Context sources:** [`./04-cutover-remove-tokio-axum.md`](./04-cutover-remove-tokio-axum.md), [`../../prototypes/wo-db/src/value.hpp`](../../prototypes/wo-db/src/value.hpp).
## Goal

2
docs/plan/06-bespoke-error.md
View file

@ -1,5 +1,7 @@
# 06 — Bespoke Error Type
> **Kanban: ⬜ not started** — Track 1 (runtime foundations). Board: [00-kanban.md](00-kanban.md)
**Context sources:** [`./05-hand-rolled-json.md`](./05-hand-rolled-json.md), [`../01-problem.md`](../01-problem.md).
## Goal

2
docs/plan/07-inotify-content-watcher.md
View file

@ -1,5 +1,7 @@
# 07 — `inotify` Content Watcher
> **Kanban: ⬜ not started** — Track 1 (runtime foundations). Board: [00-kanban.md](00-kanban.md)
**Context sources:** [`./02-event-loop-epoll.md`](./02-event-loop-epoll.md), [`./linux/00-linux.md`](./linux/00-linux.md) § File Watching, [`../02-recovery.md`](../02-recovery.md) § No AWS Infrastructure.
## Goal

2
docs/plan/08-sendfile-static-assets.md
View file

@ -1,5 +1,7 @@
# 08 — `sendfile` Zero-Copy Static Serving
> **Kanban: ⬜ not started** — Track 1 (runtime foundations); also a prerequisite of the parked UI track. Board: [00-kanban.md](00-kanban.md)
**Context sources:** [`./03-hand-rolled-http.md`](./03-hand-rolled-http.md), [`./linux/00-linux.md`](./linux/00-linux.md) § Efficient File Serving, [`../02-recovery.md`](../02-recovery.md).
## Goal

2
docs/plan/09-concurrency-scaleout.md
View file

@ -1,5 +1,7 @@
# 09 — Scale-out: thread-per-core for 10k concurrent users
> **Kanban: 🔄 in progress** — 09a/09b/09c ✅ shipped (+ keep-alive and io_uring group-commit follow-ups, measured in the shipped notes below); 09d/09e/09f ⬜ not started. Board: [00-kanban.md](00-kanban.md)
**Context sources:** [`./08-sendfile-static-assets.md`](./08-sendfile-static-assets.md) (last single-threaded phase), [`./assembly/02-writeonce-stance.md`](./assembly/02-writeonce-stance.md) (the "single-threaded" policy we're now refining), [`../runtime/database/02-wo-language.md#concurrency-model`](../runtime/database/02-wo-language.md#concurrency-model) (original concurrency stance), [`docs/examples/ecommerce/`](../examples/ecommerce/) (the target workload), [`./linux/`](./linux/) (kernel primitives), [`reference/go/src/runtime/`](../../reference/go/src/runtime/) (precedent for a runtime that scales across threads).
## Context

2
docs/plan/10-storage-foundations.md
View file

@ -1,5 +1,7 @@
# 10 — Storage Foundations: on-disk row codec + segment append path
> **Kanban: ⬜ not started (scope reduced)** — WAL framing/fallocate/CRC landed early via plan 09c. Board: [00-kanban.md](00-kanban.md)
**Context sources:** [`./done/04-cutover-remove-tokio-axum.md`](./done/04-cutover-remove-tokio-axum.md), [`../runtime/database/03-inmemory-engine.md`](../runtime/database/03-inmemory-engine.md), [`../runtime/database/07-wo-seg-migration.md`](../runtime/database/07-wo-seg-migration.md), [`./exploration/postgresql/smgr-and-md.md`](./exploration/postgresql/smgr-and-md.md), [`./exploration/postgresql/page-format.md`](./exploration/postgresql/page-format.md), [`./exploration/linux/12-pwrite-fsync.md`](./exploration/linux/12-pwrite-fsync.md), [`./exploration/linux/09-fallocate.md`](./exploration/linux/09-fallocate.md), [`reference/crates/wo-seg/src/`](../../reference/crates/wo-seg/src/).
## Goal

2
docs/plan/11-wal-and-recovery.md
View file

@ -1,5 +1,7 @@
# 11 — WAL + crash recovery
> **Kanban: ⬜ not started (scope reduced)** — replay + ack-after-fsync + group commit landed via 09c and its follow-ups; remaining here: snapshots (`.data`), compaction, WAL rotation. Board: [00-kanban.md](00-kanban.md)
**Context sources:** [`./10-storage-foundations.md`](./10-storage-foundations.md), [`../runtime/database/02-wo-language.md#concurrency-model`](../runtime/database/02-wo-language.md#concurrency-model), [`../runtime/database/03-inmemory-engine.md`](../runtime/database/03-inmemory-engine.md), [`./exploration/postgresql/wal.md`](./exploration/postgresql/wal.md), [`./exploration/postgresql/buffer-and-checkpoint.md`](./exploration/postgresql/buffer-and-checkpoint.md), [`./exploration/linux/12-pwrite-fsync.md`](./exploration/linux/12-pwrite-fsync.md), [`../02-recovery.md`](../02-recovery.md).
## Goal

2
docs/plan/12-engine-disk-cutover.md
View file

@ -1,5 +1,7 @@
# 12 — Engine cutover: rows live on disk
> **Kanban: ⬜ not started** — the C prototype's phase B (mmap arena) is the proving ground. Board: [00-kanban.md](00-kanban.md)
**Context sources:** [`./10-storage-foundations.md`](./10-storage-foundations.md), [`./11-wal-and-recovery.md`](./11-wal-and-recovery.md), [`../runtime/database/03-inmemory-engine.md`](../runtime/database/03-inmemory-engine.md), [`../runtime/database/07-wo-seg-migration.md`](../runtime/database/07-wo-seg-migration.md), [`./exploration/postgresql/buffer-and-checkpoint.md`](./exploration/postgresql/buffer-and-checkpoint.md), [`./exploration/postgresql/page-format.md`](./exploration/postgresql/page-format.md), [`./exploration/linux/12-pwrite-fsync.md`](./exploration/linux/12-pwrite-fsync.md).
## Goal

2
docs/plan/13-class-model-live-pricing.md
View file

@ -1,5 +1,7 @@
# 13 — Class model + live pricing: state and methods, no inheritance
> **Kanban: 🔄 in progress** — 13a ✅ shipped; 13b (method execution) is the next API milestone; 13c ⬜; 13d ⏸ parked (frontend); 13e ⬜. Board: [00-kanban.md](00-kanban.md)
**Context sources:** [`../runtime/database/02-wo-language.md`](../runtime/database/02-wo-language.md) (schema layer, § Schema-Layer DML brace disambiguation, § Cross-Paradigm Transaction Coordinator), [`../runtime/database/04-client-api.md`](../runtime/database/04-client-api.md) (subscription engine), [`./09-concurrency-scaleout.md`](./09-concurrency-scaleout.md) (thread-per-core scale-out), [`./exploration/ui/00-overview.md`](./exploration/ui/00-overview.md) + [`./exploration/ui/01-htmlx-format-spec.md`](./exploration/ui/01-htmlx-format-spec.md) (live UI), [`../examples/pricing/`](../examples/pricing/) (the demo this phase makes real), [`../examples/ecommerce/shared/logic/checkout.wo`](../examples/ecommerce/shared/logic/checkout.wo) (the existing `fn … in txn snapshot` signature style methods reuse).
## Context

2
docs/plan/14-mvc-ui-implementation.md
View file

@ -1,5 +1,7 @@
# 14 — MVC UI implementation: model = class, view = htmlx + scss, controller = .wo
> **Kanban: ⏸ parked (frontend)** — backend focus first; design stays current. Board: [00-kanban.md](00-kanban.md)
**Context sources:** [`./exploration/ui/08-mvc-structure.md`](./exploration/ui/08-mvc-structure.md) (the design this plan implements), [`./exploration/ui/01-htmlx-format-spec.md`](./exploration/ui/01-htmlx-format-spec.md) / [`02-ui-compiler.md`](./exploration/ui/02-ui-compiler.md) / [`03-client-runtime.md`](./exploration/ui/03-client-runtime.md) (the three UI-track pieces this plan sequences, each with port sources and LOC budgets), [`./13-class-model-live-pricing.md`](./13-class-model-live-pricing.md) (the class methods controllers call: 13a/13b; the LIVE deltas views consume: 13c), [`../examples/pricing/ui/pricing/`](../examples/pricing/ui/pricing/) (the reference MVC triplet), [`reference/crates/wo-htmlx/`](../../reference/crates/wo-htmlx/) (the v1 template engine, primary port source).
## Context

2
docs/plan/exploration/c-runtime/00-plan.md
View file

@ -1,5 +1,7 @@
# wo-rt-c roadmap — multi-threaded io_uring RAM database runtime, in C
> **Kanban: ✅ done** — phases A–F all shipped with measured exit evidence below. Board: [../../00-kanban.md](../../00-kanban.md)
**Context sources:** [`prototypes/wo-rt-c/wo-rt.c`](../../../../prototypes/wo-rt-c/wo-rt.c) (phase 0 — the single-threaded epoll baseline), [`../../09-concurrency-scaleout.md`](../../09-concurrency-scaleout.md) (the thread-per-core doctrine every phase here miniaturizes), [`../../10-storage-foundations.md`](../../10-storage-foundations.md) / [`11-wal-and-recovery.md`](../../11-wal-and-recovery.md) / [`12-engine-disk-cutover.md`](../../12-engine-disk-cutover.md) (the storage track), kernel reference cards [`../linux/07-io_uring.md`](../linux/07-io_uring.md), [`08-mmap.md`](../linux/08-mmap.md), [`09-fallocate.md`](../linux/09-fallocate.md), [`12-pwrite-fsync.md`](../linux/12-pwrite-fsync.md), [`02-eventfd.md`](../linux/02-eventfd.md).
## Goal