From 531b0283c6a9a9d57a0443ad83dce5802e30ca00 Mon Sep 17 00:00:00 2001 From: "shoney.arickathil" Date: Mon, 17 Aug 2026 20:06:36 +0200 Subject: [PATCH] docs: remove stale old-runtime docs; abandon the ##ui frontend track Analyzed the full 131-file docs tree (4 parallel classifiers) against the shipped woc/wovm toolchain. Removed 21 stale docs, kept all intentional history (Rust-track plans/done, the runtime/database design series cited by current specs, syscall/postgres/assembly/c-runtime studies, discarded/ learnings). Deleted: - old-runtime "front door": writeonce-pl.md, runtime/wo-language.md (pitched the Rust wo runtime -- REST/LiveView/SQL+Cypher -- as the current language; contradicted the new README) - v1 design set: 02-recovery, 03-data, 04-ui, 05-datalayer, 06-markdown-render, 07-ssl; runtime/database/05-go-sdk - future-scope/ai-agents-content-management (unfinished old-runtime CMS) - the ##ui/.htmlx LiveView frontend track (product decision to abandon): 9 plan/exploration/ui/*, plan/14-mvc-ui-implementation, superpowers/plans/2026-08-01-ui-htmlx-live; 13d pricing-UI board row Tree left link-clean: 46 dead links to the removed docs neutralized to plain text or deleted as pure see-also bullets across 20 kept docs; whole-tree link-resolving scan reports zero links to any deleted file. Removal recorded in discarded.md; board Frontend section + project-structure tree updated. Co-Authored-By: Claude Opus 5 (1M context) --- docs/00-status.md | 13 +- docs/01-problem.md | 2 +- docs/02-recovery.md | 172 -------- docs/03-data.md | 184 --------- docs/04-ui.md | 216 ---------- docs/05-datalayer.md | 139 ------- docs/06-markdown-render.md | 246 ------------ docs/07-ssl.md | 252 ------------ docs/08-project-structure.md | 11 +- .../ai-agents-content-management.md | 182 --------- docs/plan/07-inotify-content-watcher.md | 2 +- docs/plan/08-sendfile-static-assets.md | 2 +- docs/plan/11-wal-and-recovery.md | 2 +- docs/plan/13-class-model-live-pricing.md | 7 +- docs/plan/14-mvc-ui-implementation.md | 90 ----- docs/plan/discarded.md | 2 + docs/plan/done/02-event-loop-epoll.md | 2 +- docs/plan/done/03-hand-rolled-http.md | 2 +- .../exploration/c-runtime/01-architecture.md | 2 +- .../exploration/c-runtime/02-single-binary.md | 6 +- docs/plan/exploration/linux/00-linux.md | 2 +- docs/plan/exploration/ui/00-overview.md | 213 ---------- .../exploration/ui/01-htmlx-format-spec.md | 111 ------ docs/plan/exploration/ui/02-ui-compiler.md | 95 ----- docs/plan/exploration/ui/03-client-runtime.md | 109 ----- .../exploration/ui/04-workspace-layout.md | 139 ------- .../exploration/ui/05-per-app-binaries.md | 124 ------ .../exploration/ui/06-shared-db-daemon.md | 120 ------ .../exploration/ui/07-per-app-policies.md | 105 ----- docs/plan/exploration/ui/08-mvc-structure.md | 81 ---- docs/runtime/database.md | 9 +- docs/runtime/database/01-evaluation.md | 14 +- docs/runtime/database/02-wo-language.md | 2 +- docs/runtime/database/04-client-api.md | 6 +- docs/runtime/database/05-go-sdk.md | 371 ------------------ docs/runtime/database/06-lowcode-fullstack.md | 6 +- docs/runtime/database/07-wo-seg-migration.md | 4 +- docs/runtime/surreal-case-study.md | 2 +- docs/runtime/wo-language.md | 249 ------------ .../plans/2026-08-01-ui-htmlx-live.md | 94 ----- docs/writeonce-pl.md | 113 ------ 41 files changed, 45 insertions(+), 3458 deletions(-) delete mode 100644 docs/02-recovery.md delete mode 100644 docs/03-data.md delete mode 100644 docs/04-ui.md delete mode 100644 docs/05-datalayer.md delete mode 100644 docs/06-markdown-render.md delete mode 100644 docs/07-ssl.md delete mode 100644 docs/future-scope/ai-agents-content-management.md delete mode 100644 docs/plan/14-mvc-ui-implementation.md delete mode 100644 docs/plan/exploration/ui/00-overview.md delete mode 100644 docs/plan/exploration/ui/01-htmlx-format-spec.md delete mode 100644 docs/plan/exploration/ui/02-ui-compiler.md delete mode 100644 docs/plan/exploration/ui/03-client-runtime.md delete mode 100644 docs/plan/exploration/ui/04-workspace-layout.md delete mode 100644 docs/plan/exploration/ui/05-per-app-binaries.md delete mode 100644 docs/plan/exploration/ui/06-shared-db-daemon.md delete mode 100644 docs/plan/exploration/ui/07-per-app-policies.md delete mode 100644 docs/plan/exploration/ui/08-mvc-structure.md delete mode 100644 docs/runtime/database/05-go-sdk.md delete mode 100644 docs/runtime/wo-language.md delete mode 100644 docs/superpowers/plans/2026-08-01-ui-htmlx-live.md delete mode 100644 docs/writeonce-pl.md diff --git a/docs/00-status.md b/docs/00-status.md index 715f627..5e41ca6 100644 --- a/docs/00-status.md +++ b/docs/00-status.md @@ -374,13 +374,14 @@ log-watcher proof. | ⬜ | 15a–15e MCP over streamable HTTP | [15](plan/15-mcp-streamable-http.md) | 15e needs 13c + 09d | | ⬜ | 16c–16f typed columns, lossless resync, restore, SCRAM | [16](plan/16-postgres-mirror.md) | | -### Frontend — parked +### Frontend — removed as stale (2026-08-17) -| Status | Phase | Doc | -| ------ | -------------------------------- | ------------------------------------------------------------------- | -| ⏸ | 13d pricing UI | [13](plan/13-class-model-live-pricing.md) | -| ⏸ | 14 MVC UI implementation (14a–f) | [14](plan/14-mvc-ui-implementation.md) | -| ⏸ | UI exploration track | [exploration/ui/00-overview.md](plan/exploration/ui/00-overview.md) | +The `##ui` / `.htmlx` LiveView frontend track — 13d pricing UI, the 14-MVC-UI +implementation plan, the 7-of-7 `ui-htmlx-live` plan, and the 9-doc +`plan/exploration/ui/` design set — was **removed**. It was built entirely on +the non-advancing Rust runtime (`.dev/reference/crates/wo-htmlx`, `cargo run`, +WebSocket live-patches) and contradicts the current woc/wovm direction. Recorded +in [`discarded.md`](plan/discarded.md). --- diff --git a/docs/01-problem.md b/docs/01-problem.md index 584e2a8..afee068 100644 --- a/docs/01-problem.md +++ b/docs/01-problem.md @@ -1,6 +1,6 @@ # Problem Statement -The current writeonce architecture works, but it carries weight that the project doesn't need. This document identifies the structural problems that motivate the redesign described in [02-recovery.md](./02-recovery.md). +The current writeonce architecture works, but it carries weight that the project doesn't need. This document identifies the structural problems that motivate the redesign described in 02-recovery.md. ## Too Many Moving Parts diff --git a/docs/02-recovery.md b/docs/02-recovery.md deleted file mode 100644 index fd7b770..0000000 --- a/docs/02-recovery.md +++ /dev/null @@ -1,172 +0,0 @@ -# Recovery — The Target Architecture - -This document describes where writeonce is going: a single, self-contained binary that owns its own storage, serves its own content, and pushes updates to connected clients in real-time — with no external database, no cloud pipeline, and no separate API server. - -## Guiding Principle - -**Everything in one process.** The database, the server logic, and the client-facing interface all live in a single codebase and ship as a single executable. If you can run the binary, you have the full platform. - -## Own Database - -The current PostgreSQL instance is a derived cache — it stores JSONB copies of files that already exist as the source of truth. The recovery architecture eliminates this indirection entirely. - -### What Changes - -- **No external database.** No PostgreSQL, no Diesel ORM, no connection pooling, no migrations. -- **Local file storage.** Markdown files and JSON metadata files are stored in a local directory, just as they are today in `writeonce-articles-s3/`. The file system *is* the database. -- **Custom storage segments (.seg files).** Research area: segment files that provide efficient read access, indexing, and potentially append-only writes for content. Think of these as a lightweight, purpose-built storage layer — not a general-purpose database engine, but enough to support indexed lookups by `blog-title` and ordered listing by date. -- **Indexed by blog-title.** The `sys_title` / blog-title field remains the primary key for content retrieval. The embedded storage must support O(1) or O(log n) lookups by this field. - -### What Stays the Same - -- Articles are still structured as JSON metadata + Markdown content pairs. -- The `sys_title`, `published`, `tags`, `author`, and section structure remain the content model. -- Content is still the source of truth — but now it's read directly from local storage instead of being derived through a sync pipeline. - -## No AWS Infrastructure - -The current architecture uses S3 as a file host and Lambda as a sync trigger. In the target architecture, there is nothing to sync *to* — the files are already where they need to be. - -### What Gets Removed - -| Current Component | Why It Existed | Why It's No Longer Needed | -|---|---|---| -| S3 bucket | Remote file storage | Files live locally alongside the binary | -| Lambda function (Go) | Watch S3 for changes, call API | No remote store to watch — file changes are local | -| aws-infra service (Rust) | Bridge to AWS S3/EC2 APIs | No AWS dependency | -| Pulumi IaC | Manage Lambda + S3 resources | No cloud resources to manage | - -### What Replaces It - -The binary watches its own content directory. When a file changes (new article, updated metadata), the embedded database re-indexes and notifies subscribers. The deployment model becomes: - -``` -1. Place the binary on a server -2. Point it at a content directory -3. It serves -``` - -No credentials, no IAM roles, no SDK configuration. - -## No Separate API - -Today, `writeonce-api` is a standalone Actix-web server that mediates between the frontend and the database. In the target architecture, the server logic is embedded in the same process as the database and the content renderer. - -### What This Means - -- **No HTTP hop between database and server.** Queries go directly from the request handler to the storage engine in-process. No network serialization, no connection pool, no ORM layer. -- **Single codebase.** No multi-repo coordination. A new article field is added once — in the content model — and it flows through storage, indexing, and rendering in the same compilation unit. -- **Single deployment.** One binary, one container, one process. No docker-compose orchestrating API + database + infra services. - -The binary still exposes HTTP endpoints — it's still a web server. But it's a web server with an embedded database, not a web server that talks to an external one. - -## Real-Time Subscriptions Without WebSocket - -The current architecture has no mechanism for pushing content updates to connected clients. The target architecture adds real-time subscriptions, but explicitly without WebSocket. - -### Why Not WebSocket - -WebSocket adds connection state management, heartbeat logic, reconnection handling, and protocol upgrade complexity. For a content platform where updates are infrequent (articles are published, not streamed), the overhead isn't justified. - -### Subscription Model - -The target is a subscription mechanism where: - -- A client subscribes to a content query (e.g., "all published articles" or "article with sys_title X") -- When the underlying data changes, the server pushes the relevant diff to the subscriber -- No polling from the client side - -Candidate approaches to research: - -- **Server-Sent Events (SSE)** — unidirectional push over HTTP. Simple, well-supported, no protocol upgrade. Natural fit for infrequent content updates. -- **SpacetimeDB-style subscriptions** — clients register queries, the engine tracks which rows match, and only sends diffs when the result set changes. This is the aspirational model. -- **Long polling** — fallback option. Simple but less efficient than SSE for multiple subscribers. - -The key constraint: the subscription mechanism must work without requiring clients to maintain persistent bidirectional connections. - -## Target Architecture - -``` - content directory - (JSON + MD files, .seg index) - | - | file watch + re-index - v - +---------------------------+ - | writeonce binary | - | | - | +-------------------+ | - | | embedded storage | | .seg files, blog-title index - | | (read/write/index)| | - | +-------------------+ | - | | | - | +-------------------+ | - | | server logic | | route handlers, content queries - | | (HTTP endpoints) | | - | +-------------------+ | - | | | - | +-------------------+ | - | | subscription mgr | | SSE / query-based push - | | (real-time push) | | - | +-------------------+ | - | | - +---------------------------+ - | - HTTP / SSE - | - v - +-------------------+ - | frontend app | Angular or successor - | (browser client) | - +-------------------+ -``` - -## Single Repository - -The five current repos collapse into one: - -``` -writeonce/ - content/ # articles (JSON + MD), images, assets - storage/ # embedded database engine (.seg files, indexing) - server/ # HTTP handlers, subscription manager - frontend/ # client application - writeonce.toml # configuration (port, content dir, index settings) -``` - -One repo. One build. One deploy artifact. - -## What Needs Research - -| Area | Question | Notes | -|------|----------|-------| -| **.seg file format** | What storage format gives efficient indexed reads over JSON+MD content? | Look at LSM trees, append-only logs, SQLite's page format for inspiration | -| **File watching** | How to efficiently detect content changes on Linux/macOS? | `inotify` on Linux, `kqueue` on macOS, or cross-platform via `notify` crate | -| **SSE vs alternatives** | Is SSE sufficient for the subscription model, or is something custom needed? | SSE handles the "push diffs to subscribers" case well for low-frequency updates | -| **Index structure** | What index structure supports `blog-title` lookup + date-ordered listing? | B-tree or hash index for title, sorted set for date ordering | -| **Language choice** | Continue with Rust for the unified binary? | Rust fits: single binary output, no runtime, strong typing, existing team knowledge | -| **Frontend coupling** | Should the frontend be embedded in the binary (serve static assets) or remain separate? | Embedding simplifies deployment; separate allows independent frontend iteration | - -## Migration Path - -The transition from current to target doesn't have to be all-or-nothing: - -1. **Phase 1** — Build the embedded storage engine. Read JSON+MD files from a local directory, index by `blog-title`, serve via HTTP. No AWS, no PostgreSQL. This alone replaces `writeonce-api` + `aws-infra` + `lambda-function` + PostgreSQL. -2. **Phase 2** — Add real-time subscriptions (SSE). Clients subscribe to content queries and receive push updates when files change. -3. **Phase 3** — Collapse repositories. Move frontend into the unified codebase. Ship as a single binary that serves both API and static assets. - -Each phase produces a working system. The current architecture can run in parallel until the new one is ready. - -## Implementation phases - -The "embedded storage engine" of Phase 1 above lands in three numbered plan docs under [`docs/plan/`](./plan/): - -| Phase | Doc | What it ships | -| --- | --- | --- | -| 10 | [`plan/10-storage-foundations.md`](./plan/10-storage-foundations.md) | On-disk row codec (length-prefix + flags + LSN + CRC32C); per-type segment files (`data/.seg`); `posix_fallocate` preallocation; `pwrite`-only append path. Reads still in-memory. | -| 11 | [`plan/11-wal-and-recovery.md`](./plan/11-wal-and-recovery.md) | WAL log with `fdatasync` at commit; group commit per loop tick; control file with `last_durable_lsn` (rename-on-write); replay loop on startup. `kill -9` mid-write loses nothing acknowledged. | -| 12 | [`plan/12-engine-disk-cutover.md`](./plan/12-engine-disk-cutover.md) | `Engine`'s row payload moves to disk; in-memory map becomes `BTreeMap`. Periodic checkpoint flushes segments + advances the control file. RAM bounded by id-count, not row size. | - -Postgres' storage subsystem is the design reference — see [`docs/plan/exploration/postgresql/`](./plan/exploration/postgresql/) for which Postgres modules informed which decision and what writeonce skips (multi-process IPC, latches, separate writer processes). - -The durability syscalls themselves live in [`docs/plan/exploration/linux/12-pwrite-fsync.md`](./plan/exploration/linux/12-pwrite-fsync.md). diff --git a/docs/03-data.md b/docs/03-data.md deleted file mode 100644 index d444392..0000000 --- a/docs/03-data.md +++ /dev/null @@ -1,184 +0,0 @@ -# Data Layer — Local Storage with Subscriptions - -This document describes the embedded data layer that replaces PostgreSQL: local `.seg` files with indexing, and a subscription model where clients register queries and receive diffs on route visit — no polling required. - -## .seg File Storage - -The `.seg` (segment) format is the on-disk representation of article data. Each segment file holds serialized article content with positional indexing for fast lookups. - -### Design Goals - -- **No external database process.** The binary reads and writes `.seg` files directly. No socket connections, no protocol negotiation, no separate daemon. -- **Indexed by blog-title.** The primary access pattern is `GET /blog/:sys_title`. The storage layer must resolve a `sys_title` to its article content without scanning all files. -- **Append-friendly.** New articles and updates append to the segment. Deletes are tombstoned and compacted later. -- **Human-readable source.** The JSON + Markdown files remain the authoring format. `.seg` files are a derived index — if they're deleted, they can be rebuilt from the content directory. - -### Proposed Structure - -``` -content/ - linux-misc/ - linux-misc.json # authored metadata (source of truth) - linux-misc.md # authored content (source of truth) - aws-lambda-pulumi/ - aws-lambda-pulumi.json - aws-lambda-pulumi.md - -data/ - articles.seg # serialized article records - index/ - title.idx # blog-title -> offset mapping - date.idx # publish date -> offset (sorted) - tags.idx # tag -> [offsets] (inverted index) -``` - -The `content/` directory is what the author edits. The `data/` directory is what the engine builds and queries. Losing `data/` is a cold start, not data loss. - -### Segment File Internals - -``` -+------------------+ -| segment header | magic bytes, version, record count -+------------------+ -| record 0 | length-prefixed serialized article -+------------------+ -| record 1 | -+------------------+ -| ... | -+------------------+ -| record N | -+------------------+ -``` - -Each record is a length-prefixed byte sequence containing the full article (metadata + content merged). Records are addressed by byte offset from the start of the file. - -### Index Files - -**title.idx** — Hash map serialized to disk. Maps `sys_title` (string) to byte offset in `articles.seg`. Loaded into memory at startup for O(1) lookups. - -**date.idx** — Sorted array of `(timestamp, offset)` pairs. Supports range queries for "articles published between X and Y" and ordered listing for the homepage. - -**tags.idx** — Inverted index. Maps each tag string to a list of offsets. Supports "all articles tagged with X" queries. - -On startup, index files are memory-mapped or loaded into heap. On content change, affected indexes are rebuilt incrementally. - -## Subscription Model - -The subscription model is inspired by SpacetimeDB: clients register queries, and the engine tracks which results match. When underlying data changes, only the relevant diffs are pushed to subscribers. - -### How It Works - -``` - Client A Server Content Dir - | | | - |--- GET /blog/linux-misc -| | - | |-- read from .seg index ---->| - |<-- article + SSE stream -| | - | | | - | (subscribed to | | - | sys_title=linux-misc) | | - | | | - | |<-- file change detected ----| - | | | - | |-- re-index article -------->| - | |-- diff against last push -->| - | | | - |<-- SSE: updated content -| | - | | | -``` - -### Route-Based Subscription - -When a user visits a route, the response includes both the current content and an SSE stream. The client is automatically subscribed to changes for that query — no explicit subscription handshake needed. - -``` -GET /blog/linux-misc -``` - -Response: -``` -HTTP/1.1 200 OK -Content-Type: text/html - - - - - -``` - -The subscription lives as long as the browser tab is open. When the user navigates away, the EventSource closes and the server drops the subscription. No heartbeat management, no reconnection logic beyond what SSE provides natively (automatic reconnect is built into the EventSource API). - -### Query Registration - -Subscriptions are not limited to single-article lookups. The engine supports registering arbitrary content queries: - -| Query Type | Example | Subscription Behavior | -|---|---|---| -| Single article | `sys_title = "linux-misc"` | Push when this specific article changes | -| All published | `published = true` | Push when any article is published or unpublished | -| By tag | `tags contains "rust"` | Push when a rust-tagged article is added, removed, or updated | -| Homepage list | `published = true ORDER BY date DESC LIMIT 10` | Push when the top-10 list changes | - -The server maintains a registry of active subscriptions. On each content change, it evaluates which subscriptions are affected and pushes diffs only to those clients. - -### Diff Format - -When content changes, the server doesn't resend the full article. It sends a minimal diff: - -```json -{ - "type": "update", - "sys_title": "linux-misc", - "changes": { - "content.sections[2].paragraphs[0]": "Updated paragraph text...", - "content.tags": ["linux", "kernel", "new-tag"] - }, - "version": 42 -} -``` - -The `version` field enables clients to detect missed updates and request a full resync if needed. - -## Sample Dataset - -To validate the storage engine and subscription model, a sample dataset should exercise the core access patterns: - -### Articles - -| sys_title | tags | published | purpose | -|---|---|---|---| -| `sample-getting-started` | `[tutorial, beginner]` | true | Basic article, tests single-article subscription | -| `sample-rust-patterns` | `[rust, patterns]` | true | Tests tag-based queries | -| `sample-draft-wip` | `[draft]` | false | Tests published filter — should not appear in public queries | -| `sample-long-form` | `[deep-dive, rust]` | true | Multiple sections, images, code snippets — tests complex content rendering | -| `sample-frequently-updated` | `[changelog]` | true | Updated often — tests subscription diff delivery | - -### Test Scenarios - -1. **Cold start** — Delete `data/`, start the binary. It should rebuild `.seg` and index files from `content/` and serve all articles. -2. **Single article query** — `GET /blog/sample-getting-started` returns the article and opens an SSE subscription. -3. **Live update** — Edit `sample-frequently-updated.json` while a client is subscribed. The client should receive an SSE event with the diff. -4. **Tag query** — Subscribe to `tags contains "rust"`. Both `sample-rust-patterns` and `sample-long-form` should be in the result set. Adding a new article tagged `rust` should trigger a push. -5. **Publish toggle** — Change `sample-draft-wip` from `published: false` to `true`. Clients subscribed to the homepage list should receive a push with the new article added. - -## SpacetimeDB Reference - -SpacetimeDB is the primary architectural inspiration for the subscription model. Key concepts to study: - -- **Modules** — server logic that runs inside the database, not beside it -- **Subscription queries** — clients register SQL-like queries; the engine evaluates them incrementally on each transaction -- **Incremental view maintenance** — only recompute the parts of a query result that changed -- **Client SDK generation** — type-safe client code generated from the server schema - -Add SpacetimeDB as a reference submodule for quick access to their implementation patterns: - -```bash -git submodule add https://github.com/clockworklabs/SpacetimeDB.git references/spacetimedb -``` - -The goal is not to replicate SpacetimeDB — it's to take its subscription semantics and apply them to a much narrower domain (blog content), where the simplicity of the problem allows a simpler implementation. diff --git a/docs/04-ui.md b/docs/04-ui.md deleted file mode 100644 index 74dfd6f..0000000 --- a/docs/04-ui.md +++ /dev/null @@ -1,216 +0,0 @@ -# User Interface — Server-Rendered HTMLX - -No Angular. No React. No frontend framework. The UI is a set of `.htmlx` template files that the server parses, populates with content from the embedded database, and serves as plain HTML. Real-time updates arrive via SSE and are applied with minimal client-side scripting. - -## Why Not Angular - -The current `writeonce-app` is an Angular 18 SPA with Tailwind, PrismJS, ngx-markdown, and FontAwesome. It works, but it's a heavy delivery mechanism for what is fundamentally a read-heavy content site: - -- **~200MB of `node_modules`** for a site that renders markdown articles -- **Client-side routing** for content that doesn't need it — every article is a distinct URL, not an interactive application -- **JavaScript-dependent rendering** — content doesn't exist until Angular boots, hydrates, and fetches from the API -- **Separate build pipeline** — `npm run build` produces static assets that must be deployed to nginx independently of the API - -The content is static between updates. The interactivity is limited to navigation and code highlighting. A server-rendered approach matches the actual requirements. - -## HTMLX Templates - -The author defines the site layout using `.htmlx` files — HTML with embedded data bindings that the server resolves at render time. - -### Template Structure - -``` -templates/ - layout.htmlx # outer shell: , , - header.htmlx # site header, navigation - footer.htmlx # site footer - home.htmlx # homepage: article list - article.htmlx # single article view - about.htmlx # static page - contact.htmlx # static page - components/ - article-card.htmlx # summary card for article listings - code-snippet.htmlx # code block with language + title - img-caption.htmlx # image with caption - section.htmlx # article section (heading + paragraphs) -``` - -### Template Syntax - -Templates use a binding syntax that references content from the database. The server parses these bindings, resolves them against the current content, and outputs plain HTML. - -```html - -
- -
-``` - -```html - -
-

{{article.title}}

-

by {{article.author}} · {{article.tags}}

- - {{#each article.sections}} -
-

{{heading}}

- {{#each paragraphs}} -

{{this}}

- {{/each}} -
- {{/each}} - - {{#each article.codes}} - {{> code-snippet snippet=this}} - {{/each}} - - {{#each article.images}} - {{> img-caption image=this}} - {{/each}} -
-``` - -```html - -
-

articles

- {{#each articles}} - {{> article-card article=this}} - {{/each}} -
-``` - -The `{{> partial}}` syntax includes another `.htmlx` file as a component. The server resolves these at render time — no client-side component tree. - -### Content Subscription in Templates - -Templates declare what data they need. The server resolves these declarations against the embedded database and subscribes the client to changes: - -```html - - - -
-

{{article.title}}

- ... -
-``` - -```html - - - -
- {{#each articles}} - {{> article-card article=this}} - {{/each}} -
-``` - -The `` comment is a directive to the server. It declares the query that populates the template's data context. The same query is used to register an SSE subscription for live updates (as described in [03-data.md](./03-data.md)). - -## Rendering Pipeline - -``` - Browser request - | - v - Route match (/blog/linux-misc) - | - v - Load template (article.htmlx) - | - v - Parse subscribe directive - (article WHERE sys_title = "linux-misc") - | - v - Query embedded database (.seg index) - | - v - Resolve template bindings ({{article.title}}, etc.) - | - v - Compose with layout.htmlx + header.htmlx + footer.htmlx - | - v - Inject SSE subscription script - | - v - Send complete HTML response -``` - -The browser receives a fully rendered page on first load. No JavaScript framework boots. No API call fires. The content is already in the HTML. - -## Live Updates via SSE - -After the initial HTML is delivered, a small inline script opens an SSE connection for the page's subscription query: - -```html - -``` - -The `applyDiff` function is a lightweight client-side updater — it targets DOM elements by data attribute and patches their content. No virtual DOM, no reconciliation, no framework. For a content site where updates are infrequent and localized (a paragraph changed, a tag was added), direct DOM manipulation is sufficient. - -```html -

Linux Misc

-

First paragraph...

-``` - -When a diff arrives for `article.title`, the script finds the element with `data-bind="article.title"` and replaces its text content. This is the minimal client-side code the architecture requires. - -## Code Highlighting - -The current frontend uses PrismJS for syntax highlighting. In the server-rendered model, highlighting can happen at either layer: - -**Server-side (preferred):** The server parses code blocks during template rendering and emits pre-highlighted HTML with CSS classes. The browser only needs the PrismJS CSS theme, not the JavaScript library. This eliminates client-side parsing entirely. - -**Client-side (fallback):** Include PrismJS as a small script that runs on page load and on SSE update. Simpler to implement initially but adds a JavaScript dependency. - -## Markdown Rendering - -The current frontend uses `ngx-markdown` and `marked` to parse markdown in the browser. In the target architecture, markdown is rendered to HTML on the server during template composition. The browser never sees raw markdown. - -This aligns with the content model: the JSON metadata already defines the article structure (sections, paragraphs, code snippets, images). The markdown file provides prose content. The server combines both into final HTML — the template just places the pre-rendered blocks. - -## What Gets Removed - -| Current (Angular) | Target (HTMLX) | -|---|---| -| `writeonce-app/` (full Angular project) | `templates/` (handful of .htmlx files) | -| `node_modules/` (~200MB) | None | -| `angular.json`, `tsconfig.json`, `karma.conf.js` | None | -| npm build pipeline | Template parsed at request time | -| Nginx static file serving | Binary serves its own HTML | -| Client-side routing | Server-side route matching | -| Client-side markdown parsing | Server-side rendering | -| Client-side code highlighting | Server-side or minimal JS | - -## Styling - -Templates use plain CSS. Tailwind can optionally be used as a build-time utility (generating a static CSS file), but there is no runtime CSS framework. The author writes styles in a `styles.css` file that the server serves as a static asset. - -``` -templates/ - styles/ - main.css # site-wide styles - article.css # article-specific styles - code-theme.css # syntax highlighting theme (PrismJS compatible) -``` - -## Template Authoring Experience - -The `.htmlx` files are editable by the same author who writes articles. The template syntax is intentionally close to HTML — there's no JSX, no TypeScript, no build step. An author who knows HTML can modify the site layout. - -This closes the loop on the writeonce philosophy: the author writes content (markdown + JSON) and layout (`.htmlx` + CSS) as files, and the binary turns them into a live site. diff --git a/docs/05-datalayer.md b/docs/05-datalayer.md deleted file mode 100644 index 97150e3..0000000 --- a/docs/05-datalayer.md +++ /dev/null @@ -1,139 +0,0 @@ -# Data Layer — Implementation Status - -The embedded data layer described in [02-recovery.md](./02-recovery.md) and [03-data.md](./03-data.md) has been implemented as a Cargo workspace with 8 crates. All 44 tests pass. No external database, no AWS, no tokio — direct Linux syscalls on a custom event loop. - -## Workspace Structure - -``` -writeonce-all/ - Cargo.toml # workspace root - docs/ # architecture documentation - sample-content/ # 5 test articles for validation - crates/ - wo-model/ # content model - wo-seg/ # .seg file format - wo-index/ # index files - wo-store/ # unified storage engine - wo-watch/ # inotify file watcher - wo-event/ # epoll event loop - wo-sub/ # subscription system - wo-rt/ # custom runtime -``` - -## Crate Summary - -| Crate | Purpose | Tests | Key Types | -|-------|---------|-------|-----------| -| **wo-model** | Article structs matching existing JSON schema, `ContentLoader` for directory walking | 8 | `Article`, `ArticleContent`, `ArticleBody`, `Section`, `CodeSnippet`, `ContentLoader` | -| **wo-seg** | Binary `.seg` file format — length-prefixed records, tombstoning, positional I/O | 6 | `SegWriter`, `SegReader`, `SegHeader` | -| **wo-index** | Three index types for O(1) and O(log n) access patterns | 8 | `TitleIndex`, `DateIndex`, `TagIndex` | -| **wo-store** | Unified storage engine composing seg + indexes, cold-start rebuild | 3 | `Store` | -| **wo-watch** | Content directory watcher using inotify | 4 | `ContentWatcher`, `ContentChange` | -| **wo-event** | Custom event loop on epoll with eventfd, timerfd, signalfd | 5 | `EventLoop`, `EventFd`, `TimerFd`, `SignalFd` | -| **wo-sub** | Subscription manager with fd-based notifications, `register!` macro | 6 | `SubscriptionManager`, `Subscription`, `Notification` | -| **wo-rt** | Runtime tying all crates together — single process, single event loop | 4 | `Runtime`, `RuntimeHandle`, `Config` | - -## Linux Kernel Syscalls Used - -| Syscall | Crate | Purpose | -|---------|-------|---------| -| `pread` / `pwrite` | wo-seg | Positional read/write for .seg records without seeking | -| `fallocate` | wo-seg | Pre-allocate .seg file space to reduce fragmentation | -| `epoll_create1` / `epoll_ctl` / `epoll_wait` | wo-event | Event-driven I/O multiplexing for the main loop | -| `eventfd` | wo-event, wo-sub | Lightweight signaling between watcher and subscription manager | -| `timerfd_create` / `timerfd_settime` | wo-event | Periodic tasks (compaction, keepalive) as file descriptors | -| `signalfd` | wo-event | SIGINT/SIGTERM delivered as fd events for graceful shutdown | -| `inotify_init1` / `inotify_add_watch` | wo-watch | File system change detection on the content directory | -| `pipe2` | wo-sub (tests) | Mock subscriber fds for testing notification delivery | - -## .seg File Format - -``` -Offset Size Field -0 4 Magic: b"WOSF" -4 2 Version: u16 LE (1) -6 2 Flags: u16 LE (reserved) -8 8 Record count: u64 LE -16 8 Data start offset: u64 LE -24 8 Reserved -32+ variable Records: [u32 length][u8 flags][bincode payload]... -``` - -- Records are addressed by byte offset from file start -- Flags: `0x00` = active, `0x01` = tombstoned -- Payload: bincode-serialized `Article` struct - -## Index Files - -| File | Format | Access Pattern | -|------|--------|----------------| -| `title.idx` | On-disk hash table (Robin Hood, load factor 0.5), 138 bytes/slot | O(1) lookup by `sys_title` | -| `date.idx` | Sorted `(i64 timestamp, u64 offset)` array, 16 bytes/entry | Binary search for date ranges, latest N | -| `tags.idx` | Bincode-serialized `HashMap>` | Tag-to-offsets inverted index | - -All indexes are derived from `.seg` and rebuildable from `content/` on cold start. - -## Subscription Model - -No SSE. No WebSocket. Notifications are written directly to subscriber file descriptors. - -- **Subscribe**: `SubscriptionManager::subscribe(fd, Subscription::ByTitle("linux-misc"))` -- **Notify**: on content change, length-prefixed `Notification` written to matching fds -- **Cleanup**: `EPOLLHUP` on epoll triggers automatic `unsubscribe(fd)` -- **Dedup**: if a fd matches multiple patterns (title + tag), it receives only one notification - -Subscription patterns: -- `Subscription::ByTitle(sys_title)` — single article -- `Subscription::ByTag(tag)` — all articles with tag -- `Subscription::All` — all content changes - -## Store Query API - -```rust -store.get_by_title("linux-misc") -> Option
-store.list_published(skip, limit) -> Vec
-store.list_by_tag("rust") -> Vec
-store.list_by_date_range(start, end) -> Vec
-store.count_published() -> usize -store.article_version("linux-misc") -> Option -store.rebuild() // full rebuild from content/ -``` - -## Runtime Event Loop - -Single `epoll` instance multiplexing all file descriptors: - -| Token | Fd | Handler | -|-------|----|---------| -| `WATCHER` | inotify fd | Process file changes → update store → notify subscribers | -| `SIGNAL` | signalfd | SIGINT/SIGTERM → graceful shutdown | -| `TIMER` | timerfd | Periodic tasks (compaction, stats) | -| `NOTIFY` | eventfd | Subscription notification signal | -| `1000+` | subscriber fds | Hangup detection → unsubscribe + cleanup | - -## External Dependencies - -| Crate | Version | Purpose | -|-------|---------|---------| -| `serde` | 1.x | Serialization derives | -| `serde_json` | 1.x | JSON parsing for article files | -| `bincode` | 1.x | Compact binary serialization for .seg records and notifications | -| `libc` | 0.2.x | Raw Linux syscall bindings | - -No tokio. No async-std. No database driver. No HTTP framework (yet). - -## What Comes Next - -The data layer delivers everything the HTTP server and UI layers need: - -1. **`Store` with zero-copy query access** — all article queries resolve in-process -2. **Subscription system accepting raw fds** — HTTP layer hands socket fds to `subscribe()` -3. **Shared event loop** — HTTP listener socket registers on the same epoll -4. **Automatic cold-start** — if `data/` is missing, rebuilds from `content/` on startup -5. **Graceful shutdown** — SIGTERM triggers clean fd cleanup - -Next phases per [02-recovery.md](./02-recovery.md): -- **HTTP server** — route handlers using the `Store` query API, embedded in the same binary -- **HTMLX templates** — server-rendered HTML with `{{bindings}}` per [04-ui.md](./04-ui.md) -- **Frontend collapse** — serve static assets from the binary, eliminate the Angular app -3 \ No newline at end of file diff --git a/docs/06-markdown-render.md b/docs/06-markdown-render.md deleted file mode 100644 index 224b84f..0000000 --- a/docs/06-markdown-render.md +++ /dev/null @@ -1,246 +0,0 @@ -# Markdown File Rendering - -## Current State (writeonce-articles-s3) - -Each article is a directory containing a JSON metadata file and one or more `.md` files: - -``` -auto-scale-gitlab-runner-using-aws-spot-instance/ - docker-machine-test-with-t2.md - gitlab-runner-config.md - stop-test-gitlab-docker-machine.md - -gitlab-runner-with-kubernetes-executor/ - gitlab-runner-with-kubernetes-executor.json - deploy.md - permission.md - role-binding.md - role-defination.md - gitlab-runnergitlab-runner-deploy.md -``` - -The JSON metadata currently defines the full article structure — sections, headings, paragraphs, and code snippet references. Markdown files are limited to code blocks referenced via the `codes[].snippet` field. - -## Problem - -The JSON metadata carries too much content. Headings, paragraphs, prose — all of this is duplicated as JSON strings inside `content.content.sections`. The markdown files only hold code snippets, referenced by `sectionIndex` and `paragraphIndex`. - -This is backwards. The markdown file should be the content. The JSON should be minimal metadata. - -## Target: Markdown-First Content Model - -**The markdown file is the article.** All prose, headings, code blocks, and inline formatting live in the `.md` file. The JSON metadata file holds only what markdown cannot express: system fields, tags, publication state, and author. - -### Minimal JSON Metadata - -```json -{ - "sys_title": "gitlab-runner-with-kubernetes-executor", - "title": "Gitlab Runner with Kubernetes Executor", - "published": true, - "author": "Shoney Arickathil", - "tags": ["kubernetes", "gitlab", "ci-cd"], - "published_on": 1740950884 -} -``` - -No `content.content.sections`. No `content.content.codes`. No `paragraphs[]` arrays. No `sectionIndex`/`paragraphIndex` mapping. - -### Markdown File = Full Article Content - -````markdown -# Introduction - -Deploying a Gitlab runner using kubernetes is a great option to overcome -the limitations of other gitlab runner executor such as docker and docker machine. - -## Running Gitlab Runner in gitlab namespace - -Create the namespace and apply the deployment: - -```yaml -apiVersion: apps/v1 -kind: Deployment -metadata: - name: gitlab-runner - namespace: gitlab -``` -```` - -## Permissions - -The runner needs RBAC permissions to create pods: - -```yaml -apiVersion: rbac.authorization.k8s.io/v1 -kind: Role -metadata: - name: gitlab-runner -``` - -Everything is in the markdown — headings, paragraphs, code blocks with language hints, links, images. The rendering pipeline parses the markdown directly. - -### Directory Structure - -``` -content/ - gitlab-runner-with-kubernetes-executor/ - gitlab-runner-with-kubernetes-executor.json # minimal metadata - gitlab-runner-with-kubernetes-executor.md # full article content - linux-misc/ - linux-misc.json - linux-misc.md -``` - -One JSON for metadata. One markdown for content. No scattered `.md` files per code snippet. - -## What Changes - -| Before | After | -| -------------------------------------------------------------- | ----------------------------------------------------------------------- | -| JSON holds sections, headings, paragraphs as structured arrays | JSON holds only sys_title, title, published, author, tags, published_on | -| Markdown files hold only code snippets | Markdown file holds the entire article | -| `codes[].snippet` maps filename to sectionIndex/paragraphIndex | No mapping needed — headings and code blocks are inline in markdown | -| Renderer reads JSON structure, injects code from .md files | Renderer parses markdown directly into HTML | -| Multiple .md files per article (one per code snippet) | One .md file per article | - -## Impact on the Data Layer - -### wo-model - -The `Article` struct simplifies: - -```rust -pub struct Article { - pub sys_title: String, - pub title: String, - pub published: bool, - pub author: String, - pub tags: Vec, - pub published_on: Option, -} -``` - -The nested `ArticleContent` / `ArticleBody` / `Section` / `CodeSnippet` hierarchy is no longer needed. Article content comes from parsing the `.md` file at render time, not from the JSON. - -### wo-md - -Currently handles only inline markdown (`**bold**`, `` `code` ``, links). Needs to become a full markdown-to-HTML renderer: - -- Block elements: headings (`#`, `##`), paragraphs, code fences (` `lang ```), lists, blockquotes -- Inline elements: bold, italic, code, links, images -- Code fence language extraction for `wo-md::highlight()` -- The renderer reads `{sys_title}/{sys_title}.md`, parses it, and returns HTML - -### wo-htmlx - -The `article.htmlx` template simplifies. Instead of iterating `{{#each article.content.content.sections}}`, it renders the pre-parsed markdown HTML: - -```html -
-

{{article.title}}

-

by {{article.author}} · {{article.tags}}

- {{article.content_html}} -
-``` - -Where `content_html` is the full HTML output from the markdown renderer. - -### wo-store - -`ContentLoader` reads the `.json` for metadata and the `.md` for content. The `.seg` file stores both. At query time, the markdown is either: - -- Pre-rendered to HTML during ingestion (stored in .seg alongside metadata) -- Rendered on-demand at request time (read .md from disk) - -Pre-rendering is preferred — it avoids parsing markdown on every HTTP request. - -## Migration Path - -1. Update `wo-model` with the simplified `Article` struct -2. Extend `wo-md` to handle full markdown (block-level parsing, code fences) -3. Update `ContentLoader` to read `.json` + `.md` pairs -4. Update `wo-store` to store pre-rendered HTML in the .seg file -5. Simplify `article.htmlx` template -6. Migrate existing articles: extract prose from JSON into `.md` files - -Existing articles with the old JSON format can coexist during migration — `ContentLoader` checks for a `.md` file and falls back to the JSON structure if none exists. - -## Blog Subscription — Live Content Reload - -When a user visits `http://localhost:3000/blog/sample-rust-patterns`, the content should stay live. Any edit to `sample-content/sample-rust-patterns/sample-rust-patterns.md` must auto-reflect in the browser without a page refresh. - -### How It Works - -``` -Browser visits /blog/sample-rust-patterns - │ - ▼ -1. Server renders article HTML from .seg (pre-rendered from .md) -2. Server writes HTML response to socket fd -3. Server registers socket fd in subscription table: - register!(sub_manager, socket_fd, ByTitle("sample-rust-patterns")) -4. Connection transitions to Subscribed state (stays open) - │ - │ (user edits sample-rust-patterns.md) - │ - ▼ -5. inotify fires IN_MODIFY on sample-rust-patterns.md -6. ContentWatcher maps file → sys_title "sample-rust-patterns" -7. Store rebuilds: re-reads .json + .md, re-renders markdown to HTML, updates .seg + indexes -8. SubscriptionManager::notify("sample-rust-patterns", ...) fires -9. For each subscribed fd: write(fd, diff_payload) - │ - ▼ -10. Browser receives payload on the open connection -11. Client-side script applies the update to the DOM -``` - -### What Needs to Work - -| Component | Requirement | -|-----------|-------------| -| **inotify** (wo-watch) | Already watches `content/` directory. `.md` file changes must trigger `ContentChange::Modified(sys_title)` | -| **Store rebuild** (wo-store) | On `.md` change: re-read file, re-render markdown to HTML, update `.seg` and indexes | -| **Subscription table** (wo-sub) | Route handler registers the browser's socket fd via `register!` after sending initial HTML | -| **Notification** (wo-sub) | On content change, write updated `content_html` to all subscribed fds as JSON payload | -| **Event loop** (wo-rt) | After writing initial response, transition connection to `Subscribed` state. Keep fd on epoll for hangup detection. | -| **Client script** | Injected in the HTML. Reads payloads from the open connection. Replaces article content in the DOM. | - -### Client-Side Script - -Injected by the template renderer into every article page: - -```html - -``` - -### inotify and .md Files - -The current `ContentWatcher` watches for `.json` file changes. It must also trigger on `.md` file changes: - -- `IN_MODIFY` on `*.md` → `ContentChange::Modified(sys_title)` -- The sys_title is derived from the parent directory name (same as for JSON) -- Both `.json` and `.md` changes trigger a store rebuild and subscriber notification diff --git a/docs/07-ssl.md b/docs/07-ssl.md deleted file mode 100644 index 9c47f56..0000000 --- a/docs/07-ssl.md +++ /dev/null @@ -1,252 +0,0 @@ -# SSL and Deployment - -## Problem - -In [01-problem.md](./01-problem.md), the infrastructure overhead was identified — multiple repos, AWS dependencies, separate deployment pipelines. But one problem went unaddressed: the server-side infrastructure that sits in front of the application — nginx reverse proxy, SSL certificates, systemd service management, and deployment to the production host. - -Currently this requires manual SSH, manual nginx config, manual certbot runs. For a single-binary platform, the deployment should be as simple as the architecture. - -## Target - -Given: -- SSH access to `writeonce.de` is configured -- nginx exists at the default path `/etc/nginx/` -- The writeonce binary listens on a local port (e.g., `127.0.0.1:3000`) - -The deployment pipeline should: -1. Build the binary -2. Copy it to the server -3. Create/update the systemd service -4. Restart the service -5. Configure nginx as a reverse proxy -6. Obtain and auto-renew SSL certificates via Let's Encrypt - -## Systemd Service - -The writeonce binary runs as a systemd service for automatic restart, logging, and boot-start. - -### Service File - -```ini -# /etc/systemd/system/writeonce.service -[Unit] -Description=writeonce content platform -After=network.target - -[Service] -Type=simple -User=writeonce -Group=writeonce -WorkingDirectory=/opt/writeonce -ExecStart=/opt/writeonce/writeonce -Restart=on-failure -RestartSec=5 -StandardOutput=journal -StandardError=journal - -# Security hardening -NoNewPrivileges=true -ProtectSystem=strict -ProtectHome=true -ReadWritePaths=/opt/writeonce/data -PrivateTmp=true - -[Install] -WantedBy=multi-user.target -``` - -### Directory Layout on Server - -``` -/opt/writeonce/ - writeonce # the binary - content/ # article .json + .md files - data/ # derived .seg + .idx (rebuilt on start) - templates/ # .htmlx templates - static/ # CSS, images -``` - -### Service Management - -```bash -# Install / update -sudo systemctl daemon-reload -sudo systemctl enable writeonce -sudo systemctl restart writeonce - -# Check status -sudo systemctl status writeonce -journalctl -u writeonce -f -``` - -## Nginx Reverse Proxy - -Nginx sits in front of the writeonce binary, handling SSL termination and proxying requests to `127.0.0.1:3000`. - -### Nginx Config - -```nginx -# /etc/nginx/sites-available/writeonce.de -server { - listen 80; - server_name writeonce.de www.writeonce.de; - return 301 https://$server_name$request_uri; -} - -server { - listen 443 ssl http2; - server_name writeonce.de www.writeonce.de; - - ssl_certificate /etc/letsencrypt/live/writeonce.de/fullchain.pem; - ssl_certificate_key /etc/letsencrypt/live/writeonce.de/privkey.pem; - ssl_protocols TLSv1.2 TLSv1.3; - ssl_ciphers HIGH:!aNULL:!MD5; - ssl_prefer_server_ciphers on; - - # HSTS - add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always; - - location / { - proxy_pass http://127.0.0.1:3000; - proxy_set_header Host $host; - proxy_set_header X-Real-IP $remote_addr; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Forwarded-Proto $scheme; - - # Keep connections open for database subscriptions - proxy_http_version 1.1; - proxy_set_header Connection ""; - proxy_read_timeout 86400s; - proxy_send_timeout 86400s; - } - - # Static assets — let nginx serve directly for better caching - location /static/ { - alias /opt/writeonce/static/; - expires 1y; - add_header Cache-Control "public, immutable"; - } -} -``` - -### Enable Site - -```bash -sudo ln -sf /etc/nginx/sites-available/writeonce.de /etc/nginx/sites-enabled/ -sudo nginx -t -sudo systemctl reload nginx -``` - -## SSL with Let's Encrypt - -### Initial Certificate - -```bash -sudo apt install certbot python3-certbot-nginx -sudo certbot --nginx -d writeonce.de -d www.writeonce.de -``` - -Certbot modifies the nginx config to add SSL directives and obtains the certificate. - -### Auto-Renewal - -Certbot installs a systemd timer that runs twice daily: - -```bash -# Check timer -systemctl list-timers | grep certbot - -# Manual test -sudo certbot renew --dry-run -``` - -Certificates auto-renew before expiry. Nginx reloads automatically via certbot's deploy hook. - -### Deploy Hook for Nginx Reload - -```bash -# /etc/letsencrypt/renewal-hooks/deploy/reload-nginx.sh -#!/bin/bash -systemctl reload nginx -``` - -## Deployment Script - -A single script that builds, copies, and restarts: - -```bash -#!/bin/bash -# deploy.sh — run from the development machine -set -e - -SERVER="writeonce.de" -REMOTE_DIR="/opt/writeonce" - -echo "Building release binary..." -cargo build --release -p wo-rt --bin writeonce - -echo "Copying binary to server..." -scp target/release/writeonce $SERVER:$REMOTE_DIR/writeonce.new - -echo "Syncing content and templates..." -rsync -az --delete content/ $SERVER:$REMOTE_DIR/content/ -rsync -az --delete templates/ $SERVER:$REMOTE_DIR/templates/ -rsync -az --delete static/ $SERVER:$REMOTE_DIR/static/ - -echo "Swapping binary and restarting..." -ssh $SERVER " - sudo mv $REMOTE_DIR/writeonce.new $REMOTE_DIR/writeonce - sudo systemctl restart writeonce -" - -echo "Deployed. Checking status..." -ssh $SERVER "sudo systemctl status writeonce --no-pager" -``` - -### First-Time Setup - -Run once on the server to create the user, directory, and service: - -```bash -#!/bin/bash -# setup.sh — run on the server -set -e - -# Create user -sudo useradd -r -s /bin/false writeonce - -# Create directory -sudo mkdir -p /opt/writeonce/{content,data,templates,static} -sudo chown -R writeonce:writeonce /opt/writeonce - -# Install service -sudo cp writeonce.service /etc/systemd/system/ -sudo systemctl daemon-reload -sudo systemctl enable writeonce - -# Configure nginx -sudo cp writeonce.de.nginx /etc/nginx/sites-available/writeonce.de -sudo ln -sf /etc/nginx/sites-available/writeonce.de /etc/nginx/sites-enabled/ -sudo nginx -t -sudo systemctl reload nginx - -# SSL -sudo certbot --nginx -d writeonce.de -d www.writeonce.de -``` - -## What This Replaces - -| Before | After | -|--------|-------| -| Pulumi IaC managing Lambda + S3 | `deploy.sh` with scp + rsync | -| AWS Lambda deployment pipeline | `systemctl restart writeonce` | -| S3 bucket for content hosting | `rsync content/` to server | -| Docker Compose for API + DB | Single binary, one systemd service | -| Multiple nginx configs for API + frontend | One nginx config, one proxy_pass | -| Manual SSL setup | `certbot --nginx` with auto-renewal | - -## Connection Keepalive for Subscriptions - -The nginx config sets `proxy_read_timeout 86400s` (24 hours) to keep persistent connections open for the database subscription model. When a browser visits an article page and the connection transitions to `Subscribed` state, nginx must not timeout and close the upstream connection. - -If nginx is removed in the future (the binary handles TLS directly via `rustls`), this concern disappears — the binary owns the socket end-to-end. diff --git a/docs/08-project-structure.md b/docs/08-project-structure.md index f9e110d..777f958 100644 --- a/docs/08-project-structure.md +++ b/docs/08-project-structure.md @@ -90,19 +90,18 @@ Unchanged from CLAUDE.md's description: `rt` is the monolithic Stage-2 runtime p ``` docs/ -├── 01…08-*.md numbered design docs (this file is 08) -├── writeonce-pl.md language positioning -├── runtime/ user-facing language overview + the 7-phase database series -├── examples/ blog/, ecommerce/, pricing/ samples; ⏳ log-watcher/ (plan 10) +├── 00-*,01,08-*.md status / principles / problem / structure docs +├── runtime/ the 7-phase database design series + runtime concept refs +├── examples/ log-watcher/, employee/, employee-list/ samples ├── plan/ numbered engineering plans 00–16, linux/ cards, assembly/, -│ ├── exploration/ c-runtime/ (A–F, done), ui/ (htmlx track), colibri/ +│ ├── exploration/ c-runtime/ (A–F, done), linux/, postgresql/, assembly/ │ └── oop-vm/ ⏳ the OOP-track contracts: 00-wob-format, 01-error-catalog, │ 02-corpus, 03-shard-actor, 04-db-binding, 05-http-service, │ 06-ui-live, 07-systems-stdlib ├── superpowers/ │ ├── specs/ the two approved track specs (2026-08-01) │ └── plans/ implementation plans 1–10 (2026-08-01, prose-only) -└── future-scope/, cm.md legacy notes +└── cm.md legacy notes ``` Repo rule restated: documentation belongs here; code directories keep one orientation README each. diff --git a/docs/future-scope/ai-agents-content-management.md b/docs/future-scope/ai-agents-content-management.md deleted file mode 100644 index 59119ed..0000000 --- a/docs/future-scope/ai-agents-content-management.md +++ /dev/null @@ -1,182 +0,0 @@ -# AI Agents and Content Management - -## Context - -AI agents (Claude Code, Copilot, Cursor, custom agents) work within a specific project or working directory. Their sessions, context, and understanding are scoped to that directory. This works well when projects are completely different domains. - -But writeonce content is not isolated — articles reference each other, share tags, build on concepts from other articles. An agent editing `gitlab-runner-with-kubernetes-executor.md` would benefit from knowing that `auto-scale-gitlab-runner-using-aws-spot-instance.md` exists and covers related infrastructure. Without explicit mappings, the agent treats each article as an island. - -## Problem - -1. **Agents lack cross-article awareness.** When asked to write or update an article about Kubernetes, the agent doesn't know that related articles about Docker, CI/CD, or AWS already exist in the content directory — unless it manually searches. - -2. **No semantic grouping.** Tags provide flat categorization (`kubernetes`, `ci-cd`), but they don't express relationships: "this article is a prerequisite for that one", "these three articles form a series", "this article supersedes that one." - -3. **Context window waste.** Without mappings, the agent must scan all articles to find related content. With explicit mappings, it can load exactly the relevant files. - -## Solution: Metadata-Driven Content Mappings - -Users define relationships between articles in the JSON metadata. These mappings serve two purposes: - -1. **Human navigation** — rendered as "related articles" links on the site -2. **Agent context** — when an agent works on an article, it loads the mapped articles into its context for cross-referencing - -### Mapping Fields in JSON Metadata - -Per [06-markdown-render.md](../06-markdown-render.md), the JSON metadata is minimal. Add a `mappings` field: - -```json -{ - "sys_title": "gitlab-runner-with-kubernetes-executor", - "title": "Gitlab Runner with Kubernetes Executor", - "published": true, - "author": "Shoney Arickathil", - "tags": ["kubernetes", "gitlab", "ci-cd"], - "published_on": 1740950884, - "mappings": { - "related": ["auto-scale-gitlab-runner-using-aws-spot-instance"], - "prerequisite": ["linux-misc"], - "series": { - "name": "gitlab-runner", - "order": 2 - } - } -} -``` - -### Mapping Types - -| Type | Meaning | Agent Use | -| -------------- | ----------------------------------------------- | --------------------------------------------------------------------------------------- | -| `related` | Topically related articles | Agent loads these for cross-reference when editing | -| `prerequisite` | Articles the reader should read first | Agent ensures no concept duplication, references prerequisites instead of re-explaining | -| `series` | Articles that form an ordered sequence | Agent maintains narrative continuity across the series | -| `supersedes` | This article replaces an older one | Agent can mark the old article as outdated or unpublished | -| `references` | External articles or URLs the content builds on | Agent checks links are still valid, cites them properly | - -### Directory Structure with Mappings - -``` -content/ - gitlab-runner-with-kubernetes-executor/ - gitlab-runner-with-kubernetes-executor.json # metadata + mappings - gitlab-runner-with-kubernetes-executor.md # full article - auto-scale-gitlab-runner-using-aws-spot-instance/ - auto-scale-gitlab-runner-using-aws-spot-instance.json - auto-scale-gitlab-runner-using-aws-spot-instance.md - linux-misc/ - linux-misc.json - linux-misc.md -``` - -## Agent Workflows - -### 1. Writing a New Article - -The author asks an agent: "Write an article about deploying GitLab Runner on ECS." - -The agent: - -1. Scans the content directory for existing articles with tags `gitlab`, `ci-cd`, `aws` -2. Finds `gitlab-runner-with-kubernetes-executor` and `auto-scale-gitlab-runner-using-aws-spot-instance` -3. Reads their `.md` files to understand what's already covered -4. Writes the new article, referencing existing articles rather than re-explaining shared concepts -5. Suggests `mappings.related` entries for the new article's JSON - -### 2. Updating an Existing Article - -The author asks: "Update the Kubernetes executor article with the new runner token format." - -The agent: - -1. Reads the article's JSON metadata and `.md` content -2. Reads the `mappings.related` articles to check for consistency -3. Makes the update in the `.md` file -4. Checks if the change affects any prerequisite or series articles -5. inotify detects the `.md` change → store rebuilds → subscribers notified - -### 3. Content Audit - -The author asks: "Which articles reference outdated AWS configurations?" - -The agent: - -1. Loads all article metadata (the Store already indexes everything) -2. Follows `mappings` to build a dependency graph -3. Reads the `.md` files of articles tagged with `aws` -4. Identifies outdated patterns (old SDK versions, deprecated services) -5. Reports findings with links to specific articles and line numbers - -### 4. Series Management - -The author asks: "Add a new part to the gitlab-runner series." - -The agent: - -1. Finds all articles with `mappings.series.name == "gitlab-runner"` -2. Reads them in order to understand the narrative arc -3. Writes the new article continuing from where the series left off -4. Sets `mappings.series.order` to the next number -5. Updates the previous article's mappings to reference the new one - -## Integration with writeonce Architecture - -### Store Index - -Add a mappings index alongside the existing title, date, and tag indexes: - -``` -data/ - articles.seg - index/ - title.idx - date.idx - tags.idx - mappings.idx # sys_title → related sys_titles -``` - -The mappings index allows efficient traversal: "give me all articles related to X" without scanning every article's JSON. - -### Template Rendering - -The `article.htmlx` template can render related articles: - -```html -
-

{{article.title}}

- {{article.content_html}} {{#each article.related}} - - {{/each}} -
-``` - -### Subscription - -When a mapped article changes, subscribers to related articles can optionally be notified. If article A lists article B in `mappings.related`, and article B is updated, subscribers to article A can receive a notification that related content changed. - -## Agent Configuration - -For agents to use the mappings effectively, the project can include an agent instruction file (e.g., `CLAUDE.md` or `.agent/instructions.md`): - -```markdown -## Content Management - -- Articles are in `content/{sys_title}/{sys_title}.md` -- Metadata is in `content/{sys_title}/{sys_title}.json` -- Before writing or editing an article, read its `mappings` field and load related articles for context -- When creating a new article, suggest appropriate `mappings` based on tags and content overlap -- Maintain narrative continuity within `series` mappings -- Do not duplicate explanations that exist in `prerequisite` articles — reference them instead -``` - -This turns the content directory into an agent-navigable knowledge graph where the metadata provides the edges and the markdown files provide the nodes. - -## queryable graph database - -- traversable knowledge graphs available on RAM. -- which linux kernels, develop in C ++. User wants to learn it. diff --git a/docs/plan/07-inotify-content-watcher.md b/docs/plan/07-inotify-content-watcher.md index a2a2f26..0380339 100644 --- a/docs/plan/07-inotify-content-watcher.md +++ b/docs/plan/07-inotify-content-watcher.md @@ -2,7 +2,7 @@ > **Status: ⬜ not started** — Track 1 (runtime foundations). Board: [00-status.md](../00-status.md) -**Context sources:** [`./02-event-loop-epoll.md`](./done/02-event-loop-epoll.md), [`./linux/00-linux.md`](./exploration/linux/00-linux.md) § File Watching, [`../02-recovery.md`](../02-recovery.md) § No AWS Infrastructure. +**Context sources:** [`./02-event-loop-epoll.md`](./done/02-event-loop-epoll.md), [`./linux/00-linux.md`](./exploration/linux/00-linux.md) § File Watching, `../02-recovery.md` § No AWS Infrastructure. ## Goal diff --git a/docs/plan/08-sendfile-static-assets.md b/docs/plan/08-sendfile-static-assets.md index 92cf90f..9002f2f 100644 --- a/docs/plan/08-sendfile-static-assets.md +++ b/docs/plan/08-sendfile-static-assets.md @@ -2,7 +2,7 @@ > **Status: ⬜ not started** — Track 1 (runtime foundations); also a prerequisite of the parked UI track. Board: [00-status.md](../00-status.md) -**Context sources:** [`./03-hand-rolled-http.md`](./done/03-hand-rolled-http.md), [`./linux/00-linux.md`](./exploration/linux/00-linux.md) § Efficient File Serving, [`../02-recovery.md`](../02-recovery.md). +**Context sources:** [`./03-hand-rolled-http.md`](./done/03-hand-rolled-http.md), [`./linux/00-linux.md`](./exploration/linux/00-linux.md) § Efficient File Serving, `../02-recovery.md`. ## Goal diff --git a/docs/plan/11-wal-and-recovery.md b/docs/plan/11-wal-and-recovery.md index 3675e15..cb03269 100644 --- a/docs/plan/11-wal-and-recovery.md +++ b/docs/plan/11-wal-and-recovery.md @@ -2,7 +2,7 @@ > **Status: ⬜ not started (scope reduced)** — replay + ack-after-fsync + group commit landed via 09c and its follow-ups; remaining here: snapshots (`.data`), compaction, WAL rotation. Board: [00-status.md](../00-status.md) -**Context sources:** [`./10-storage-foundations.md`](./10-storage-foundations.md), [`../runtime/database/02-wo-language.md#concurrency-model`](../runtime/database/02-wo-language.md#concurrency-model), [`../runtime/database/03-inmemory-engine.md`](../runtime/database/03-inmemory-engine.md), [`./exploration/postgresql/wal.md`](./exploration/postgresql/wal.md), [`./exploration/postgresql/buffer-and-checkpoint.md`](./exploration/postgresql/buffer-and-checkpoint.md), [`./exploration/linux/12-pwrite-fsync.md`](./exploration/linux/12-pwrite-fsync.md), [`../02-recovery.md`](../02-recovery.md). +**Context sources:** [`./10-storage-foundations.md`](./10-storage-foundations.md), [`../runtime/database/02-wo-language.md#concurrency-model`](../runtime/database/02-wo-language.md#concurrency-model), [`../runtime/database/03-inmemory-engine.md`](../runtime/database/03-inmemory-engine.md), [`./exploration/postgresql/wal.md`](./exploration/postgresql/wal.md), [`./exploration/postgresql/buffer-and-checkpoint.md`](./exploration/postgresql/buffer-and-checkpoint.md), [`./exploration/linux/12-pwrite-fsync.md`](./exploration/linux/12-pwrite-fsync.md), `../02-recovery.md`. ## Goal diff --git a/docs/plan/13-class-model-live-pricing.md b/docs/plan/13-class-model-live-pricing.md index 43ef678..3bcb7dc 100644 --- a/docs/plan/13-class-model-live-pricing.md +++ b/docs/plan/13-class-model-live-pricing.md @@ -2,7 +2,7 @@ > **Status: 🔄 in progress** — 13a ✅ shipped; 13b ✅ shipped (methods execute over RPC); 13c (LIVE push) is next; 13d ⏸ parked (frontend); 13e ⬜. Board: [00-status.md](../00-status.md) -**Context sources:** [`../runtime/database/02-wo-language.md`](../runtime/database/02-wo-language.md) (schema layer, § Schema-Layer DML brace disambiguation, § Cross-Paradigm Transaction Coordinator), [`../runtime/database/04-client-api.md`](../runtime/database/04-client-api.md) (subscription engine), [`./09-concurrency-scaleout.md`](./09-concurrency-scaleout.md) (thread-per-core scale-out), [`./exploration/ui/00-overview.md`](./exploration/ui/00-overview.md) + [`./exploration/ui/01-htmlx-format-spec.md`](./exploration/ui/01-htmlx-format-spec.md) (live UI), [`../examples/pricing/`](../examples/pricing/) (the demo this phase makes real), [`../examples/ecommerce/shared/logic/checkout.wo`](../examples/ecommerce/shared/logic/checkout.wo) (the existing `fn … in txn snapshot` signature style methods reuse). +**Context sources:** [`../runtime/database/02-wo-language.md`](../runtime/database/02-wo-language.md) (schema layer, § Schema-Layer DML brace disambiguation, § Cross-Paradigm Transaction Coordinator), [`../runtime/database/04-client-api.md`](../runtime/database/04-client-api.md) (subscription engine), [`./09-concurrency-scaleout.md`](./09-concurrency-scaleout.md) (thread-per-core scale-out), `./exploration/ui/00-overview.md` + `./exploration/ui/01-htmlx-format-spec.md` (live UI), [`../examples/pricing/`](../examples/pricing/) (the demo this phase makes real), [`../examples/ecommerce/shared/logic/checkout.wo`](../examples/ecommerce/shared/logic/checkout.wo) (the existing `fn … in txn snapshot` signature style methods reuse). ## Context @@ -73,7 +73,7 @@ Each lands as its own numbered plan doc (`13a-…`, `13b-…`) when ready. The b ### `13a-class-surface.md` — lexer, parser, AST, spec amendments — ✅ shipped -`class` joins the keyword map (`crates/rt/src/lexer.rs` keyword match, ~line 202 — note `self` stays an ident per decision 4). `parse_type` (`crates/rt/src/parser.rs:120`) takes the leading keyword as a parameter and serves both constructs; `fn` members inside the body parse-and-discard through the existing brace-depth skip — the same mechanism that already swallows `on update … do { … }` triggers. `ast::TypeDecl` gains `is_class: bool`; `Catalog::from_schemas` ignores it (decision 5), so REST CRUD works the moment parsing does. Docs amended in the same change: a "Class Model" subsection in [`02-wo-language.md`](../runtime/database/02-wo-language.md) next to § Schema-Layer DML, the "Isn't OO" paragraph in [`wo-language.md`](../runtime/wo-language.md), the class line in [`writeonce-pl.md`](../writeonce-pl.md), and `just pricing` / `just pricing-demo` recipes. +`class` joins the keyword map (`crates/rt/src/lexer.rs` keyword match, ~line 202 — note `self` stays an ident per decision 4). `parse_type` (`crates/rt/src/parser.rs:120`) takes the leading keyword as a parameter and serves both constructs; `fn` members inside the body parse-and-discard through the existing brace-depth skip — the same mechanism that already swallows `on update … do { … }` triggers. `ast::TypeDecl` gains `is_class: bool`; `Catalog::from_schemas` ignores it (decision 5), so REST CRUD works the moment parsing does. Docs amended in the same change: a "Class Model" subsection in [`02-wo-language.md`](../runtime/database/02-wo-language.md) next to § Schema-Layer DML, the "Isn't OO" paragraph in `wo-language.md`, the class line in `writeonce-pl.md`, and `just pricing` / `just pricing-demo` recipes. **Exit (met):** `wo run docs/examples/pricing` parses 2 classes, serves `/api/products` CRUD; parser unit tests (`parses_class_with_methods`, `class_method_braces_do_not_truncate_body`) green; blog/ecommerce/hello unchanged. ### `13b-method-execution.md` — methods over RPC — ✅ shipped @@ -88,7 +88,7 @@ The subscription registry from [`04-client-api.md`](../runtime/database/04-clien ### `13d-pricing-ui.md` — the `/pricing` screen, MVC -The screen ships as an **MVC triplet** per [`exploration/ui/08-mvc-structure.md`](./exploration/ui/08-mvc-structure.md), built in the sub-phase sequence of [`14-mvc-ui-implementation.md`](./14-mvc-ui-implementation.md): model = the classes themselves, view = [`pricing.htmlx`](../examples/pricing/ui/pricing/pricing.htmlx) (plain htmlx, logic-free) + external [`pricing.scss`](../examples/pricing/ui/pricing/pricing.scss) (strict SCSS subset compiled at `wo build`, no external deps), controller = [`pricing.wo`](../examples/pricing/ui/pricing/pricing.wo) (`route:`/`view:`/`styles:`, `model:` bindings, `actions:` calling the 13b class methods). SSR per [`exploration/ui/01-htmlx-format-spec.md`](./exploration/ui/01-htmlx-format-spec.md), compiler glue per [`02-ui-compiler.md`](./exploration/ui/02-ui-compiler.md), and the vanilla-JS client runtime ([`03-client-runtime.md`](./exploration/ui/03-client-runtime.md)) patches the price cell when the 13c delta lands. The controller's `model:` block is the M→V binding; the watchlist narrows the subscription predicate server-side. +The screen ships as an **MVC triplet** per `exploration/ui/08-mvc-structure.md`, built in the sub-phase sequence of `14-mvc-ui-implementation.md`: model = the classes themselves, view = [`pricing.htmlx`](../examples/pricing/ui/pricing/pricing.htmlx) (plain htmlx, logic-free) + external [`pricing.scss`](../examples/pricing/ui/pricing/pricing.scss) (strict SCSS subset compiled at `wo build`, no external deps), controller = [`pricing.wo`](../examples/pricing/ui/pricing/pricing.wo) (`route:`/`view:`/`styles:`, `model:` bindings, `actions:` calling the 13b class methods). SSR per `exploration/ui/01-htmlx-format-spec.md`, compiler glue per `02-ui-compiler.md`, and the vanilla-JS client runtime (`03-client-runtime.md`) patches the price cell when the 13c delta lands. The controller's `model:` block is the M→V binding; the watchlist narrows the subscription predicate server-side. **Exit:** browser at `/pricing` shows selected products; a `set_price` commit from curl changes the price cell in every open browser without reload. ### `13e-pricing-at-scale.md` — millions of readers, millions of live updates @@ -122,5 +122,4 @@ No new architecture — this sub-phase wires the demo to [`09-concurrency-scaleo - [`../runtime/database/02-wo-language.md`](../runtime/database/02-wo-language.md) — schema layer the class grammar extends; transaction coordinator methods reuse. - [`../runtime/database/04-client-api.md`](../runtime/database/04-client-api.md) — subscription engine 13c scopes down. - [`./09-concurrency-scaleout.md`](./09-concurrency-scaleout.md) — the scale architecture 13e instantiates. -- [`./exploration/ui/00-overview.md`](./exploration/ui/00-overview.md) — UI track 13d draws on. - [`../examples/hello/main.wo`](../examples/hello/main.wo) — the minimal example whose `Revision`-trigger pattern is the declarative ancestor of methods. diff --git a/docs/plan/14-mvc-ui-implementation.md b/docs/plan/14-mvc-ui-implementation.md deleted file mode 100644 index ecef360..0000000 --- a/docs/plan/14-mvc-ui-implementation.md +++ /dev/null @@ -1,90 +0,0 @@ -# 14 — MVC UI implementation: model = class, view = htmlx + scss, controller = .wo - -> **Status: ⏸ parked (frontend)** — backend focus first; design stays current. Board: [00-status.md](../00-status.md) - -**Context sources:** [`./exploration/ui/08-mvc-structure.md`](./exploration/ui/08-mvc-structure.md) (the design this plan implements), [`./exploration/ui/01-htmlx-format-spec.md`](./exploration/ui/01-htmlx-format-spec.md) / [`02-ui-compiler.md`](./exploration/ui/02-ui-compiler.md) / [`03-client-runtime.md`](./exploration/ui/03-client-runtime.md) (the three UI-track pieces this plan sequences, each with port sources and LOC budgets), [`./13-class-model-live-pricing.md`](./13-class-model-live-pricing.md) (the class methods controllers call: 13a/13b; the LIVE deltas views consume: 13c), [`../examples/pricing/ui/pricing/`](../examples/pricing/ui/pricing/) (the reference MVC triplet), [`reference/crates/wo-htmlx/`](../../.dev/reference/crates/wo-htmlx/) (the v1 template engine, primary port source). - -## Context - -[`exploration/ui/08-mvc-structure.md`](./exploration/ui/08-mvc-structure.md) locks the screen anatomy: **model** = the `class`/`type` itself, **view** = plain `.htmlx` + external `.scss`, **controller** = a `.wo` file (`route:`/`view:`/`styles:`/`model:`/`actions:`) that binds the model into the view and is the only place UI may call class methods. The UI exploration docs 01–03 already specify the htmlx engine, the `##ui` compiler, and the client runtime in implementable detail. What's missing is the build order, the two genuinely new pieces (the controller format and the SCSS subset compiler), and the wiring into the `13` class-model track. This doc is that sequence. - -Everything lands in **`crates/ui`** (currently a placeholder) and small deltas to `crates/rt` — consistent with the crate inventory in [`crates/README.md`](../../crates/README.md). The deployment shape never changes: one binary serving SSR + database + API on the kernel-primitive runtime. - -## Goal - -`cargo run --bin wo -- run docs/examples/pricing` (after plan 13a–13c land) serves `GET /pricing` as styled SSR HTML; clicking ☆ dispatches a controller action; an Ops `set-price` action calls `Product.set_price`, the commit pushes a delta, and the price cell patches in every open browser without reload — the [plan 13d exit criterion](./13-class-model-live-pricing.md), implemented MVC-shaped. - -## Dependency graph - -``` -14a htmlx engine ──────┬─→ 14c controller format ─→ 14d SSR routes ─→ 14e actions ─→ 14f live patch -14b scss compiler ─────┘ │ │ │ - (14a ∥ 14b — no shared code) needs engine needs 13a+13b needs 13c - (exists today) -``` - -Phases 05/06 (hand-rolled JSON / bespoke error) are orthogonal: `crates/ui` adopts `serde`/`serde_json` per the ui/01 decision and migrates when 05 lands. Phase 08 (`sendfile`) upgrades static-asset serving in 14d when it arrives; 14d ships with plain buffered writes first. - -## Sub-phase sequence - -### `14a-htmlx-engine.md` — port the view engine into `crates/ui` - -Execute [`exploration/ui/01-htmlx-format-spec.md`](./exploration/ui/01-htmlx-format-spec.md) as written: port `reference/crates/wo-htmlx` (585 LOC — `parser.rs`, `ast.rs`, `value.rs`, `registry.rs`, `render.rs` carried over per its table) into `crates/ui/src/htmlx/`, extend with `` structured nodes, `wo:bind` capture, and the `data-wo-manifest` JSON emitter (~250 LOC new). One addition beyond the 01 spec, from the MVC design: `` records whether `source` is a bare name (controller model binding, resolved in 14d) or an inline query — a one-field change to `LiveSubscription`. -**Exit:** the 01 spec's criteria — `cargo build -p ui` green, golden parse+render for every `.htmlx` under `docs/examples/{blog,ecommerce}` **plus** [`pricing/ui/pricing/pricing.htmlx`](../examples/pricing/ui/pricing/pricing.htmlx), manifest matches the 01 schema. - -### `14b-scss-subset.md` — the stylesheet compiler - -New, no port source (~400 LOC at `crates/ui/src/scss/`): scanner → rule tree → flattener. Exactly the subset locked in [08-mvc decision 3](./exploration/ui/08-mvc-structure.md): `$variables`, nesting (including `&`-less descendant flattening), `@use "partials"` (`ui/styles/_*.scss`), comments. **No mixins, functions, `@extend`, or color math** — `rgba($accent, 0.06)` in the reference file compiles by literal substitution of `$accent` and is the only function-form supported. Output is one flat `.css` per screen, written to `target/wo//static/`. -**Exit:** [`pricing.scss`](../examples/pricing/ui/pricing/pricing.scss) → golden-file CSS; unknown construct = compile error naming file:line (never silent passthrough); runs standalone (`wo build` integration is 14d). - -### `14c-controller-format.md` — parse the controller, keep the shorthand - -The `Kind::HashHash("ui")` skip arm in `crates/rt/src/parser.rs` (L80–88) parses for real, into one of two IRs by key-shape dispatch: - -- **Controller form** (`route:`/`view:`/`styles:`/`model:`/`actions:` — the MVC triplet's `pricing.wo`) → new `Controller` IR: route pattern, view/styles paths resolved relative to the screen directory, `model:` entries as named query strings (`LIVE` flag captured, execution deferred), `actions:` entries as `(name, params, target-method-or-fn, role-set)`. -- **Shorthand form** (`source:`/`columns:`/… — the existing ecommerce/blog screens) → the `Screen` IR of [`exploration/ui/02-ui-compiler.md`](./exploration/ui/02-ui-compiler.md), whose codegen emits a generated view + a synthesized `Controller` — [08-mvc decision 5](./exploration/ui/08-mvc-structure.md): the shorthand is sugar over the triplet, one downstream path. - -`##app` and `##component` keep their current skip behaviour (owned by ui/05 and ui/02 respectively). -**Exit:** `pricing.wo` parses to a `Controller` with 2 model bindings + 3 actions; every existing `##ui` screen in blog/ecommerce parses to `Screen` and compiles to an `.htmlx` that 14a round-trips; a controller naming a missing view file is a compile error. - -### `14d-ssr-routes.md` — the single binary serves the screen - -Wire controllers into `crates/rt`'s router (`crates/rt/src/server.rs`): each `Controller.route` becomes a GET route; the handler resolves `model:` bindings against the in-process engine (snapshot `select` now — the `LIVE` flag additionally registers a 13c subscription when that phase is live), renders the view via 14a with the model names as root scope, and emits HTML + manifest + ``. `wo build`/`wo run` gain the asset step: compile SCSS (14b), bake `/_wo/runtime.js` (`include_bytes!`, per ui/03 decision 6), serve `target/wo/.../static/` with buffered writes (upgraded to `sendfile` when [phase 08](./08-sendfile-static-assets.md) lands). -**Exit:** `GET /pricing` returns styled SSR HTML with a valid manifest and resolvable CSS/JS links; `GET /` on the blog sample is unaffected; route table printed at boot includes UI routes alongside REST. - -### `14e-action-dispatch.md` — controller actions call class methods - -`wo:action` buttons POST to `/_wo/action//` with `wo:args` + form payload. The dispatcher looks up the controller's action table, enforces the `role:` set server-side (per-app policy model of [`exploration/ui/07-per-app-policies.md`](./exploration/ui/07-per-app-policies.md)), and invokes the target: a class method via the 13b row-scoped RPC path (`Product{ id == id }.set_price(amount)`) or a free `fn`. Response is 204 — the UI never re-renders from the action response; the visible change arrives as a 13c delta, keeping one update path. -**Requires:** 13a + 13b. **Exit:** the ☆/★ watch toggle round-trips; `set-price` with an Ops session commits a Price; without the role it's 403 and no transaction starts. - -### `14f-live-patching.md` — the browser follows commits - -Execute [`exploration/ui/03-client-runtime.md`](./exploration/ui/03-client-runtime.md) as written (~500 LOC vanilla JS at `crates/ui/assets/wo-runtime.js`, JSON frames over one WebSocket, targeted DOM patching by `data-key` + `wo:bind`, coalescing backpressure, snapshot resync on reconnect), pointed at the 13c subscription endpoint. -**Requires:** 13c. **Exit:** the plan-13d criterion — `set_price` via curl in one terminal, the price cell changes in every open `/pricing` browser without reload; kill the server, restart, the page resyncs on reconnect. - -## Verification targets (after 14f) - -| Check | Target | How | -| --- | --- | --- | -| Golden corpus | every `.htmlx` in blog/ecommerce/pricing parses + renders byte-stable | `cargo test -p ui` golden files | -| SCSS | `pricing.scss` → golden CSS; errors carry file:line | `cargo test -p ui scss` | -| SSR | `GET /pricing` < 5 ms p99 on the dev box (RAM engine, no I/O on read path) | scripted curl loop | -| End-to-end | 13d criterion green | two-terminal demo, scripted in `just pricing-demo` | -| Single binary | UI + DB + API + WS in one `wo build` output, no Node anywhere | `ldd` shows libc only; no build-step JS | -| Dep budget | `crates/ui`: `serde`/`serde_json` only (dropped when phase 05 lands) | `Cargo.toml` review | - -## Non-scope - -- **No SPA router, no client-side templates.** Navigation is full-page SSR; only `wo:bind` cells and `` subtrees mutate in place. (ui/00 decision; unchanged.) -- **No SCSS mixins/functions/`@extend`/color math** beyond literal variable substitution — the subset is a floor, widened only by demonstrated need in the sample corpus. -- **No theme system / design tokens.** Shared partials under `ui/styles/_*.scss` are the only sharing mechanism for now. -- **No component framework.** `##component` partials render server-side via the 14a engine; they have no client behaviour beyond inherited `wo:bind` sites. -- **No changes to the REST API surface.** UI routes live beside `/api/*`; nothing under `/api` changes shape in this plan. - -## Cross-references - -- [`./exploration/ui/08-mvc-structure.md`](./exploration/ui/08-mvc-structure.md) — the design; its exit criteria are satisfied by 14c/14b/14f respectively. -- [`./13-class-model-live-pricing.md`](./13-class-model-live-pricing.md) — 13a/13b gate 14e; 13c gates 14f; 13d's exit criterion is this plan's end-to-end target. -- [`./exploration/ui/00-overview.md`](./exploration/ui/00-overview.md) — the UI track's master frame (per-app binaries, shared DB daemon) that 14d's asset/serving choices stay compatible with. -- [`reference/crates/wo-htmlx/`](../../.dev/reference/crates/wo-htmlx/) — primary port source (585 LOC), per ui/01. -- [`../examples/pricing/ui/pricing/`](../examples/pricing/ui/pricing/) — the reference triplet every sub-phase tests against. diff --git a/docs/plan/discarded.md b/docs/plan/discarded.md index dc7ef25..a0257c2 100644 --- a/docs/plan/discarded.md +++ b/docs/plan/discarded.md @@ -52,3 +52,5 @@ Status board: [`00-status.md`](../00-status.md) · Doctrine: [`../00-principles. | **Per-example `principle.md` files** | 2026-08-08: one canonical repo-level [`docs/00-principles.md`](../00-principles.md) instead; examples link to it. | | **Minimal 3-file log-watcher sample** | Breaks the file-for-file `.hx` → `.wo` mapping and leaves the "could not express" column unproven — which is the sample's entire acceptance criterion. | | **Raw code in plan documents** | Plans carry concept, reason, and required behavior in words; the executor writes the code. | +| **`##ui` / `.htmlx` LiveView frontend track** | 2026-08-17: removed the 9-doc `exploration/ui/` design set, the `14-mvc-ui-implementation` plan, and the `ui-htmlx-live` plan. All were built on the non-advancing Rust runtime (`.dev/reference/crates/wo-htmlx`, `cargo run`, WebSocket live-patches) and contradict the current woc/wovm direction. The 13d pricing-UI row went with them. Revisit only if a UI story is re-opened on the woc/wovm stack. | +| **Old-runtime "front door" + v1 design docs** | 2026-08-17: removed `writeonce-pl.md`, `runtime/wo-language.md`, `future-scope/ai-agents-content-management.md`, the numbered v1 set `02-recovery`/`03-data`/`04-ui`/`05-datalayer`/`06-markdown-render`/`07-ssl`, and `runtime/database/05-go-sdk.md`. They pitched the old Rust `wo` runtime (REST + LiveView + SQL/Cypher) as the current language and contradicted the shipped woc/wovm toolchain. The `runtime/database/` design series is kept as cited design history; the Rust-track plans/`done` are kept per the status board. | diff --git a/docs/plan/done/02-event-loop-epoll.md b/docs/plan/done/02-event-loop-epoll.md index 9ca2a92..f0106f4 100644 --- a/docs/plan/done/02-event-loop-epoll.md +++ b/docs/plan/done/02-event-loop-epoll.md @@ -2,7 +2,7 @@ > **Status: ✅ done** (Rust Stage 2 — shipped, maintained, not advancing) — `runtime/netpoll_epoll.rs`: the hand-rolled `epoll` loop that replaced the async runtime. Board: [00-status.md](../../00-status.md) -**Context sources:** [`../01-problem.md`](../../01-problem.md), [`../02-recovery.md`](../../02-recovery.md), [`./linux/00-linux.md`](../exploration/linux/00-linux.md), [`./done/01-scafolding-crates.md`](01-scafolding-crates.md). +**Context sources:** [`../01-problem.md`](../../01-problem.md), `../02-recovery.md`, [`./linux/00-linux.md`](../exploration/linux/00-linux.md), [`./done/01-scafolding-crates.md`](01-scafolding-crates.md). ## Goal diff --git a/docs/plan/done/03-hand-rolled-http.md b/docs/plan/done/03-hand-rolled-http.md index c66f6b1..0265dc2 100644 --- a/docs/plan/done/03-hand-rolled-http.md +++ b/docs/plan/done/03-hand-rolled-http.md @@ -2,7 +2,7 @@ > **Status: ✅ done** (Rust Stage 2 — shipped, maintained, not advancing) — hand-rolled HTTP/1.1, plus keep-alive and pipelining. Board: [00-status.md](../../00-status.md) -**Context sources:** [`./02-event-loop-epoll.md`](./02-event-loop-epoll.md), [`./linux/00-linux.md`](../exploration/linux/00-linux.md), [`../02-recovery.md`](../../02-recovery.md). +**Context sources:** [`./02-event-loop-epoll.md`](./02-event-loop-epoll.md), [`./linux/00-linux.md`](../exploration/linux/00-linux.md), `../02-recovery.md`. ## Goal diff --git a/docs/plan/exploration/c-runtime/01-architecture.md b/docs/plan/exploration/c-runtime/01-architecture.md index b5206e3..ae7d820 100644 --- a/docs/plan/exploration/c-runtime/01-architecture.md +++ b/docs/plan/exploration/c-runtime/01-architecture.md @@ -146,4 +146,4 @@ Work stealing (breaks single-writer ACID), shared-heap locking (the doctrine exi - [`../../docs/plan/09-concurrency-scaleout.md`](../../09-concurrency-scaleout.md) — the thread-per-core doctrine. - [`../../docs/plan/exploration/linux/07-io_uring.md`](../linux/07-io_uring.md), [`08-mmap.md`](../linux/08-mmap.md) — the two shared-page mechanisms. - [`../../docs/plan/13-class-model-live-pricing.md`](../../13-class-model-live-pricing.md) — 13e's read-replica alternative, contrasted in improvement 1. -- [`../../docs/writeonce-pl.md`](../../../writeonce-pl.md) — the C/assembly "one address" pedagogy this doc extends to a full runtime. +- [`README.md`](../../../../README.md) — the C/assembly "one address" pedagogy the single-binary story extends to a full runtime. diff --git a/docs/plan/exploration/c-runtime/02-single-binary.md b/docs/plan/exploration/c-runtime/02-single-binary.md index a6a2be7..65e3bb2 100644 --- a/docs/plan/exploration/c-runtime/02-single-binary.md +++ b/docs/plan/exploration/c-runtime/02-single-binary.md @@ -1,6 +1,6 @@ # 02 — The end goal: the writeonce single binary on this runtime environment -**Context sources:** [`00-plan.md`](./00-plan.md) (the runtime-environment phases, A–B ✅), [`01-architecture.md`](./01-architecture.md) (the one-address trace), [`../../../runtime/wo-language.md`](../../../runtime/wo-language.md) ("one binary per project; no runtime to install on the target host"; `.wo` has "its own lexer, parser, analyzer, and bytecode"), [`../../09-concurrency-scaleout.md`](../../09-concurrency-scaleout.md)–[`12`](../../12-engine-disk-cutover.md) (the Rust product track this proves out), [`../../../runtime/database/02-wo-language.md`](../../../runtime/database/02-wo-language.md) (catalog + transaction semantics the payload carries). +**Context sources:** [`00-plan.md`](./00-plan.md) (the runtime-environment phases, A–B ✅), [`01-architecture.md`](./01-architecture.md) (the one-address trace), `../../../runtime/wo-language.md` ("one binary per project; no runtime to install on the target host"; `.wo` has "its own lexer, parser, analyzer, and bytecode"), [`../../09-concurrency-scaleout.md`](../../09-concurrency-scaleout.md)–[`12`](../../12-engine-disk-cutover.md) (the Rust product track this proves out), [`../../../runtime/database/02-wo-language.md`](../../../runtime/database/02-wo-language.md) (catalog + transaction semantics the payload carries). ## The end goal, stated once @@ -85,6 +85,6 @@ Steps 1–2 and 4–6 exist in `wo-rt-c` today with the notes store standing in ## Cross-references - [`00-plan.md`](./00-plan.md) — the kernel phases; [`01-architecture.md`](./01-architecture.md) — the one-address trace through the same stack. -- [`../../../runtime/wo-language.md`](../../../runtime/wo-language.md) — the user-facing single-binary promise this document implements. -- [`../../13-class-model-live-pricing.md`](../../13-class-model-live-pricing.md) (13b methods), [`../../14-mvc-ui-implementation.md`](../../14-mvc-ui-implementation.md) (UI assets) — the payload-side tracks. +- [`README.md`](../../../../README.md) — the user-facing single-binary promise this document implements. +- [`../../13-class-model-live-pricing.md`](../../13-class-model-live-pricing.md) (13b methods) — the payload-side track. - [`../../09-concurrency-scaleout.md`](../../09-concurrency-scaleout.md) — shard-key routing and 2PC the contract defers to. diff --git a/docs/plan/exploration/linux/00-linux.md b/docs/plan/exploration/linux/00-linux.md index d9a8a4d..2a00420 100644 --- a/docs/plan/exploration/linux/00-linux.md +++ b/docs/plan/exploration/linux/00-linux.md @@ -1,6 +1,6 @@ ## Linux Kernel Features -Kernel primitives that the writeonce binary can leverage, mapped to the architectural needs identified in [docs/01-problem.md](../../../01-problem.md) and [docs/02-recovery.md](../../../02-recovery.md). +Kernel primitives that the writeonce binary can leverage, mapped to the architectural needs identified in [docs/01-problem.md](../../../01-problem.md) and docs/02-recovery.md. ### Per-primitive reference cards diff --git a/docs/plan/exploration/ui/00-overview.md b/docs/plan/exploration/ui/00-overview.md deleted file mode 100644 index 45d5d29..0000000 --- a/docs/plan/exploration/ui/00-overview.md +++ /dev/null @@ -1,213 +0,0 @@ -# UI track — `.htmlx` live templates + Angular-style monorepo - -**Context sources:** [`docs/examples/ecommerce/ui/`](../../examples/ecommerce/ui/) (current `##ui` screens — storefront, order_tracker, admin_orders), [`docs/examples/ecommerce/types/`](../../examples/ecommerce/types/) + [`docs/examples/ecommerce/logic/`](../../examples/ecommerce/logic/) (the shared-schema + shared-fn anchor), [`reference/crates/wo-htmlx/`](../../../../.dev/reference/crates/wo-htmlx/) (v1 template engine — `{{path}}`, `{{#each}}`, `{{> partial}}`, `data-bind` attributes), [`templates/`](../../../../templates/) (v1 blog's concrete `.htmlx` usage), [`docs/runtime/database/06-lowcode-fullstack.md`](../../../runtime/database/06-lowcode-fullstack.md) (Phase 6's `##ui` + `##app` block spec). - -## Context - -Three threads converge into one plan: - -1. **`##ui` needs a concrete output format.** Phase 6's spec says screens "compile to a render tree" served as SSR HTML with a thin client runtime, but the actual template format isn't named. The v1 `.htmlx` engine at [`reference/crates/wo-htmlx/`](../../../../.dev/reference/crates/wo-htmlx/) already speaks `{{bindings}}`, `{{#each}}`, `{{> partials}}`, and `data-bind` attributes — it's 90% of what the new runtime needs and already has a working parser + renderer. Adopting it (and extending it with live-subscription semantics) is cheaper than inventing a new format. - -2. **The samples want a home that matches how real frontends are organised.** The ecommerce sample today is one flat directory with `types/`, `logic/`, and `ui/` beside each other. A real deployment has *multiple apps* against the same data: a customer storefront, an admin dashboard, a fulfillment console, maybe a read-only analytics viewer. Each has its own routes, its own policies, its own ideal binary shape. Angular (via Nx / Angular CLI workspaces) solved this with `apps/*` + `libs/*` on top of a shared root config — writeonce adopts the same shape. - -3. **Each app wants to be its own binary but share a database.** Running storefront and admin as one monolith conflates concerns: a CPU spike in admin stalls customer checkout; an admin auth bug opens customer data paths. Splitting into per-app binaries that share a single database backend (via the Phase-4 native wire protocol) gives blast-radius isolation without duplicating data. - -Intended outcome: `docs/examples/ecommerce/` refactors into a workspace with `shared/` + `apps/storefront/` + `apps/admin/`. `wo build apps/storefront` produces a `storefront` binary. `wo db serve` runs the shared database. The apps connect over `wo://…` and serve `.htmlx` SSR pages that subscribe to LIVE queries without a page reload. - -## Goal - -After this track's sub-phases land: - -- A **monorepo workspace** at `docs/examples/ecommerce/` with `shared/{types,logic,components}` + `apps/{storefront,admin}` structure. -- A **per-app binary** for each app under `apps/`: `wo build apps/storefront` → `./target/wo/storefront`, `wo build apps/admin` → `./target/wo/admin`. Each binary includes only its own `##ui` / `##app` / app-local types and logic; shared code compiles in by path reference. -- A **shared DB daemon** (`wo db serve`) — one process, no UI, just the engine and wire-protocol server. Each app binary connects as a client via the Phase-4 native protocol. -- **`.htmlx` as the compiled UI output.** Every `##ui` screen compiles into an `.htmlx` template file that the app binary serves; hand-written `.htmlx` files under `apps/X/ui/*.htmlx` are accepted as a first-class authoring alternative. -- **`.htmlx` subscribes.** A `` subtree in the template registers a LIVE query at page load; a ~20 KB client JS runtime patches DOM nodes on delta frames without reloading the page. -- **Per-app users + policies.** Each app declares its role set in `apps/X/app.wo`; row-level policies on shared types stay global, app-scoped policies layer on top per route. - -## Design decisions (locked) - -1. **`.htmlx` is the template format; `##ui` is the DSL that emits it.** Authors choose per-screen: declare `##ui #home { source: Product, columns: [...] }` in `.wo` and let the compiler produce `home.htmlx`; OR hand-write `home.htmlx` for a custom page. Both flow through the same `wo-htmlx` renderer. -2. **Extend v1 `.htmlx` with two new constructs** — `...` (subscription subtree) and `wo:bind="field"` (field-level live binding). The rest of the v1 syntax (`{{path}}`, `{{#each}}`, `{{> partial}}`) carries through unchanged. -3. **One binary per app, shared database process.** Not a shared library, not a monolith. Apps connect via the Phase-4 wire protocol (`wo://host:port`) — the same connection a Go/TS client would use. No in-process shared state between apps; their isolation is enforced by the OS process boundary. -4. **Angular-parallel workspace layout.** `apps/*` for deployable binaries, `shared/*` for libs shared across apps (types, logic, UI components), `wo.toml` at the workspace root. Each app also has its own `wo.toml` that names which `shared/` directories it depends on. -5. **File structure mirrors Angular component organisation.** Each UI screen lives in its own directory: `apps/storefront/ui/home/{home.wo, home.htmlx, home.css}`. Tests go in `home.test.wo`. Matches the Angular component pattern (`home.component.ts`, `home.component.html`, `home.component.scss`). -6. **Per-app routes, not per-screen routes.** `apps/storefront/app.wo` declares route table; each route maps to a `ui.` declared under `apps/storefront/ui/*/`. Cross-app navigation is an external redirect, not an internal route. -7. **Policy composition.** Global policies live in `shared/types/.wo` next to the `type` declaration (today). App-scoped policies live in `apps/X/app.wo` and AND with the global set — an admin app might relax a storefront policy for ops roles but can never relax beyond what the type's own policy permits. - -## Angular parallels — what writeonce copies, what it doesn't - -| Angular feature | Writeonce translation | Notes | -| --- | --- | --- | -| `nx workspace` / `angular.json` | Root `wo.toml` with `[workspace] apps = [...], shared = [...]` | Path references, not package registry | -| `apps//` | `apps//` with `app.wo` + `ui/` + local `types/` + local `logic/` | 1:1 naming | -| `libs//` | `shared//` | Used `shared/` instead of `libs/` — matches the more common monorepo idiom (Nx defaults to `libs`, but `shared` is clearer for this audience) | -| `.ts` + `.html` + `.scss` | `.wo` + `.htmlx` + `.css` under `ui//` | One-directory-per-screen | -| `ng build ` | `wo build apps/` | Per-app binary output | -| Dependency injection | Service-block resolution — `service rest` blocks in shared types are callable from any app by import | No runtime DI container; bindings are resolved at compile time | -| RxJS observables | LIVE subscription frames on a WebSocket | Declarative `live` attribute instead of imperative `.subscribe(...)` | -| Zone.js change detection | Per-row delta dispatch + field-level `wo:bind` | No full-tree change detection — only the rows/fields the delta names get repainted | -| `HttpClient` | Built-in wire-protocol client inside the app binary | No separate library to import; always present | - -### What we don't copy - -- **No TypeScript.** Authoring is `.wo` (for logic) + `.htmlx` (for templates) + `.css`. If a page needs bespoke JS interactivity beyond what `wo:bind` covers, a `` and consumed only by phase 03's client runtime. Schema below; `version: 1` is a constant for this phase. - -## Scope - -### New files inside `crates/ui/src/htmlx/` - -| File | Responsibility | Port source | -| --- | --- | --- | -| `mod.rs` | Re-exports `Template`, `Manifest`, `LiveSubscription`, `BindSite`, `ParseError`, `RenderError` | [`reference/crates/wo-htmlx/src/lib.rs`](../../../../.dev/reference/crates/wo-htmlx/src/lib.rs) (11 LOC) | -| `ast.rs` | Adds `Node::Live { attrs, body }` and `wo_bind: Option` on element nodes | [`reference/crates/wo-htmlx/src/ast.rs`](../../../../.dev/reference/crates/wo-htmlx/src/ast.rs) (18 LOC) — extend by ~50 LOC | -| `parser.rs` | Adds `` body in `
` for the runtime | [`reference/crates/wo-htmlx/src/render.rs`](../../../../.dev/reference/crates/wo-htmlx/src/render.rs) (140 LOC) — extend by ~70 LOC | -| `manifest.rs` | Walks the AST, collects subscriptions + bind sites, serialises JSON | new (~150 LOC) | - -Total: ~835 LOC (585 ported + ~250 new). - -### `Cargo.toml` change - -```toml -[dependencies] -serde = { version = "1", features = ["derive"] } -serde_json = "1" -``` - -(Both already in `crates/rt`; `crates/ui` adopts them rather than hand-rolling JSON for the manifest at this phase.) - -### Manifest schema - -```json -{ - "version": 1, - "subscriptions": [ - { - "id": "orders-live-0", - "source": "Order{ status != Cancelled }", - "key": "id", - "sort": "placed_at desc", - "filter": null, - "root_selector": "[data-wo-subscription=\"orders-live-0\"]" - } - ], - "bind_sites": [ - { "subscription_id": "orders-live-0", "key": "id", "field": "status" }, - { "subscription_id": "orders-live-0", "key": "id", "field": "total" } - ] -} -``` - -## API shape (target) - -```rust -use ui::htmlx::{Template, Manifest, HelperRegistry}; - -let tmpl = Template::parse(src)?; // Result -let html = tmpl.render(&ctx, ®istry)?; // Result -let mani = tmpl.manifest(); // Manifest - -// SSR pattern: page = head + html + "" -``` - -## Exit criteria - -1. `cargo build -p ui` green; no new top-level dependencies beyond `serde`/`serde_json`. -2. **Golden parse + render** for every `.htmlx` file under [`docs/examples/blog/ui/components/`](../../examples/blog/ui/components/) and [`docs/examples/ecommerce/shared/components/`](../../examples/ecommerce/shared/components/) — output is HTML and parses back into an isomorphic AST. -3. **Manifest emission** for `…` produces a `LiveSubscription` with the source string preserved verbatim and the body wrapped under `data-wo-subscription="orders-live-0"`. -4. **Bind-site collection** for `{{status}}` inside `` records `(subscription_id, key="id", field="status")` once and only once. -5. **v1 regression**: `cargo run --bin wo -- run docs/examples/blog` continues to start without parser errors. The blog sample has no `` or `wo:bind` today; nothing should regress. -6. All 14 existing `crates/rt` tests pass. - -## Non-scope - -- **No SSR.** Phase 02 emits templates; phase 03 ships the runtime; serving them is a downstream concern that the per-app binary in phase 05 wires together. -- **No nested ``.** Parse error in this phase. Author can compose live subtrees by partial inclusion (`{{> child}}`). -- **No author-extensible helpers.** The closed enum is the contract for this phase. -- **No streaming render.** Templates are rendered to a single `String`. -- **No manifest version negotiation.** Wire format is `version: 1` always. - -## Verification - -```bash -cargo build -p ui -cargo test -p ui --test golden_v1 # blog/ecommerce templates byte-identical -cargo test -p ui --test golden_extensions # + wo:bind cases -cargo test -p ui --test manifest # manifest emission - -# v1 regression -cargo run --bin wo -- run docs/examples/blog & -PID=$!; sleep 1; curl -fsS http://127.0.0.1:8080/ >/dev/null; kill $PID - -cd reference/crates && cargo build && cargo test -``` - -## After this phase - -Phase 02 (`02-ui-compiler.md`) consumes the `Template` + `Manifest` types defined here as its emission target — every `##ui` block compiles down to an `.htmlx` file that parses cleanly under this phase's parser. Phase 03 (`03-client-runtime.md`) consumes the manifest JSON schema as its wire input. diff --git a/docs/plan/exploration/ui/02-ui-compiler.md b/docs/plan/exploration/ui/02-ui-compiler.md deleted file mode 100644 index 292cdd4..0000000 --- a/docs/plan/exploration/ui/02-ui-compiler.md +++ /dev/null @@ -1,95 +0,0 @@ -# 02 — `##ui` → `.htmlx` compiler - -**Context sources:** [`./00-overview.md`](./00-overview.md) §§ "Sub-phase sequence" (L172–180), "Design decisions" 1–6, [`./01-htmlx-format-spec.md`](./01-htmlx-format-spec.md) (the emission target), [`../../runtime/database/06-lowcode-fullstack.md`](../../../runtime/database/06-lowcode-fullstack.md) (the `##ui` block spec), [`docs/examples/blog/ui/article_list.wo`](../../../examples/blog/ui/article_list.wo), [`docs/examples/blog/ui/article_detail.wo`](../../../examples/blog/ui/article_detail.wo), [`docs/examples/ecommerce/apps/admin/ui/orders/orders.wo`](../../examples/ecommerce/apps/admin/ui/orders/orders.wo) (the test corpus), [`crates/rt/src/parser.rs:80–116`](../../../../crates/rt/src/parser.rs) (the parse-and-discard call site to replace). - -## Goal - -Walk a parsed `##ui` block — every key listed in 00-overview's locked grammar (`title`, `source`, `live`, `role`, `filter`, `quick-filters`, `columns`, `sort`, `actions`, `pagination`, `refresh`, `highlight-new`, `key`, `sections`, `inputs`, `use`, `with`, `template`, `styles`) — and emit a complete `.htmlx` template that parses under phase 01's grammar. When a hand-written `.htmlx` sits beside the `.wo`, the compiler honours it and only validates that the manifest still aligns. - -## Design decisions (locked) - -1. **File-presence dispatch.** If `apps//ui//.htmlx` exists alongside `.wo`, the hand-written file wins. The compiler still emits a manifest; it errors if the manifest's `bind_sites` reference fields the hand-written template doesn't expose. Anchored in [`./00-overview.md`](./00-overview.md) L24, L30, L46. -2. **`live: true` ⇒ `` wrapper.** The compiler emits `` around the auto-generated ``. `key` defaults to `id` when not declared. -3. **`renderer: ` ⇒ helper invocation by rule table.** A static rule table maps each renderer name to its emission form: `markdown` → `{{markdown }}`, `money` → `{{> money amount=}}`, `relative-date` → `{{relative }}`, `code` / `tag-chips` / `pill` / `image` / `stock-badge` / `list` similarly. The set is closed and matches phase 01's helper registry. -4. **`actions: row-* / bulk-*` ⇒ `data-action` + `data-role` attributes.** Click → POST `/api/fn/`. Role gating is a `data-role=""` attribute the runtime hides on; phase 07 wires the server-side check. -5. **Generated templates land in `target/wo//ui/.htmlx`.** Same path the per-app binary in phase 05 reads from at startup. Build artefact, not committed. -6. **`crates/rt/src/parser.rs:80–88` no longer skips `##ui`.** The `Kind::HashHash` arm parses into a `Screen` IR (this phase's new type). All other `##` blocks (`##app`, `##component`) keep their current `skip_top_level_chunk` behaviour for now — `##app` is owned by phase 05 and `##component` parses inline as a sibling of `##ui` but emits no template (it's a partial that other screens reference). - -## Scope - -### New files inside `crates/ui/src/compiler/` - -| File | Responsibility | Port source | -| --- | --- | --- | -| `mod.rs` | Re-exports `compile_screen`, `Screen`, `Column`, `Action`, `CompileError` | new (~30 LOC) | -| `screen.rs` | `Screen` IR — every key listed in the goal section above | new (~150 LOC) | -| `codegen.rs` | Walk `Screen` → emit `.htmlx` source string | new (~280 LOC) | -| `renderers.rs` | Closed `renderer:` → helper-emission rule table | new (~120 LOC) | -| `fallback.rs` | File-presence dispatch + manifest cross-check | new (~80 LOC) | - -### Modified file - -| File | Change | Notes | -| --- | --- | --- | -| `crates/rt/src/parser.rs` | Replace the `Kind::HashHash(_)` skip arm at L80–88 with a real parse into `Screen` when the tag is `ui` | +60 LOC delta | - -Total: ~720 LOC (all new — the v1 codebase has no `##ui` precedent to port). - -### `Cargo.toml` change - -`crates/ui` already depends on `serde`/`serde_json` from phase 01. No new deps. - -## API shape (target) - -```rust -use ui::compiler::{compile_screen, compile_app, Screen}; - -let screen: Screen = ql::parse_ui_block(src)?; -let template: String = compile_screen(&screen, &ctx)?; // an .htmlx string -let mani = ui::htmlx::Template::parse(&template)?.manifest(); - -// Whole-app pipeline used by phase 05's `wo build`: -let outputs: Vec<(PathBuf, String)> = compile_app(&app_dir)?; -for (path, src) in outputs { fs::write(path, src)?; } -``` - -## Exit criteria - -1. `cargo build -p ui` and `cargo build -p rt` green. -2. **Compile every sample `##ui`** in the test corpus: blog `article_list.wo`, blog `article_detail.wo`, ecommerce `apps/storefront/ui/home/home.wo`, `apps/storefront/ui/orders/orders.wo`, `apps/admin/ui/orders/orders.wo`. Output template parses cleanly under phase 01. -3. **Hand-written fallback honoured.** With a hand-written `apps/admin/ui/orders/orders.htmlx` present, the compiler returns its source unchanged but still emits the manifest. -4. **Manifest cross-check fires.** Renaming `body` to `text` in a hand-written template that the `##ui` block expects under `wo:bind="body"` produces a `CompileError::HandWrittenMissingField` diagnostic. -5. **Parser change is non-breaking.** `crates/rt`'s 14 unit tests still pass; `cargo run --bin wo -- run docs/examples/blog` boots and serves REST as before. -6. `cd reference/crates && cargo build && cargo test`. - -## Non-scope - -- **No SSR.** This phase emits files only — the runtime in phase 05 reads them at startup. -- **No author-defined renderers.** The closed table from phase 01's helper registry is the contract. -- **No build-output caching.** The compiler runs on every `wo build`. Caching is a future concern. -- **No partial recovery.** First parse or codegen error aborts compilation for the screen; whole-app compilation reports per-screen status. -- **No `##component` codegen in this phase.** Components remain their existing partial-include shape (`{{> name args}}`) — only `##ui` screens drive new emission. - -## Verification - -```bash -cargo build -p ui -p rt -cargo test -p ui --test compile_blog -cargo test -p ui --test compile_ecommerce -cargo test -p ui --test fallback_handwritten - -# end-to-end: emit templates for the storefront and inspect them -cargo run --bin wo -- build docs/examples/ecommerce/apps/storefront --emit-templates-only -ls target/wo/storefront/ui/ # home.htmlx, orders.htmlx -head -1 target/wo/storefront/ui/orders.htmlx # starts with - -# v1 regression -cargo run --bin wo -- run docs/examples/blog & -PID=$!; sleep 1; curl -fsS http://127.0.0.1:8080/ >/dev/null; kill $PID - -cd reference/crates && cargo build && cargo test -``` - -## After this phase - -Phase 03 (`03-client-runtime.md`) is now unblocked: every screen has a manifest the client runtime can read. Phase 04 (`04-workspace-layout.md`) places the compiled outputs under `target/wo//ui/`, which phase 05 then bakes into the per-app binary. diff --git a/docs/plan/exploration/ui/03-client-runtime.md b/docs/plan/exploration/ui/03-client-runtime.md deleted file mode 100644 index fe90e3d..0000000 --- a/docs/plan/exploration/ui/03-client-runtime.md +++ /dev/null @@ -1,109 +0,0 @@ -# 03 — Client runtime - -**Context sources:** [`./00-overview.md`](./00-overview.md) §§ "`.htmlx` with live subscriptions — target format" (L127–166) and decisions 1–2, [`./01-htmlx-format-spec.md`](./01-htmlx-format-spec.md) (the manifest schema this runtime consumes), [`reference/crates/wo-sub/src/lib.rs`](../../../../.dev/reference/crates/wo-sub/src/lib.rs) (the v1 frame model the wire format mirrors), [`docs/examples/ecommerce/shared/components/order-row.htmlx`](../../../examples/ecommerce/shared/components/order-row.htmlx) (the live workload the runtime must update without reload). - -## Goal - -Ship a ~500-line vanilla-JS client at `crates/ui/assets/wo-runtime.js` that, on page load, reads the `