writeonce/docs/runtime/database/01-evaluation.md

Phase 1 — Database Evaluation

Why build a custom database instead of adopting CouchDB, Postgres, Neo4j, or an in-memory graph library?

Next: Phase 2 — The .wo Language & ACID Engine | Index: database.md

---

Reference repositories:

• github.com/apache/couchdb — document DB with Mango query language
• github.com/postgres/postgres — relational DB with JSONB
• github.com/neo4j/neo4j — native graph DB, Cypher query language
• github.com/petgraph/petgraph — Rust in-memory graph library (NetworkX analogue)

The Question

writeonce articles have two shapes at once: they are documents (per-article JSON metadata + markdown body, per 06-markdown-render.md) and they form a graph (the mappings field — related, prerequisite, series, supersedes, references — per ai-agents-content-management.md).

Should writeonce adopt an off-the-shelf document or graph database to back these two shapes, or keep the flat-file .seg + .idx storage already implemented in 05-datalayer.md?

Short answer: no external DB. The dataset is small (hundreds of articles, not millions of rows), single-writer (author commits), and read-heavy. A full rebuild on change is cheap. The mappings graph fits entirely in RAM. External databases would add a process, a protocol, a driver, and a failure mode — none of which writeonce needs.

This doc walks through each option, what it buys, and why writeonce does not adopt it.

The Data Shape

Every article is a pair of files in content/{sys_title}/:

{
  "sys_title": "linux-misc",
  "title": "Linux Miscellaneous",
  "published": true,
  "author": "Shoney Arickathil",
  "tags": ["Linux"],
  "published_on": 1740950884,
  "mappings": {
    "related": ["auto-scale-gitlab-runner-using-aws-spot-instance"],
    "prerequisite": ["linux-misc"],
    "series": { "name": "gitlab-runner", "order": 2 }
  }
}

Plus {sys_title}.md with the full article body.

The access patterns, per 05-datalayer.md:

Pattern	Frequency	Current Implementation
get_by_title(sys_title)	Every /blog/:sys_title hit	title.idx hash — O(1)
list_published(skip, limit)	Homepage, pagination	date.idx sorted array — O(log n)
list_by_tag(tag)	Tag pages	tags.idx inverted index — O(1) + scan
list_by_date_range	Archive views	date.idx binary search
Graph traversal (mappings)	Agent workflows, "related" widget	Not yet implemented

The last row is the gap. Everything else already works on .seg + .idx.

Option 1: CouchDB + Mango

CouchDB is an HTTP-native document store. Each article JSON would be a document; Mango queries are JSON selectors expressive enough for most writeonce reads:

{ "selector": { "published": true, "tags": { "$in": ["rust"] } }, "sort": [{ "published_on": "desc" }] }

What it buys:

• Built-in revisions (MVCC) — each edit gets a _rev
• Multi-master replication, which matters if content is authored from multiple machines
• A _changes feed, conceptually similar to writeonce's subscription model

What it costs:

• Separate Erlang process, HTTP driver, JSON over the wire on every read
• No native graph traversal — mappings would require client-side joins or view functions
• Duplicates what inotify + .seg already do (change feed, indexing)

Comparison

Aspect	CouchDB	writeonce
Transport	HTTP per query	In-process function call
Storage	B-tree per database, append-only	.seg append-only, tombstoned records
Change feed	_changes HTTP long-poll	inotify → epoll → fd write
Index	View functions (JavaScript map/reduce), Mango indexes	title.idx, date.idx, tags.idx rebuilt on change
Revisions	Every write creates _rev	Git already does this for content/

CouchDB's replication and revision model are attractive, but git already covers revision history for content/, and the dataset is too small to justify a separate daemon.

Option 2: PostgreSQL + JSONB

Postgres with a jsonb column gives you SQL over the article metadata, GIN indexes for tag queries, and a real query planner. Schema would be roughly:

CREATE TABLE articles (
  sys_title TEXT PRIMARY KEY,
  metadata  JSONB NOT NULL,
  body_md   TEXT NOT NULL,
  updated   TIMESTAMPTZ DEFAULT now()
);
CREATE INDEX ON articles USING GIN ((metadata->'tags'));
CREATE INDEX ON articles (((metadata->>'published_on')::bigint));

What it buys:

• Mature: WAL, point-in-time recovery, replication, tooling
• jsonb_path_query and @> containment make most writeonce queries one-liners
• Recursive CTEs (WITH RECURSIVE) can traverse mappings — adequate for shallow graphs

What it costs:

• A Postgres process, a driver (tokio-postgres or raw libpq), connection pooling — writeonce's 05-datalayer.md explicitly removed all of this
• JSONB query planning is excellent but still pays per-query cost that an in-process hash does not
• Recursive CTEs on deep mapping chains are slower than a RAM graph walk

Comparison

Aspect	Postgres JSONB	writeonce
Process model	External daemon, TCP or unix socket	Single binary, single process
Query language	SQL + JSONB operators	Rust method calls on Store
Graph traversal	WITH RECURSIVE CTE	(Future) in-memory adjacency list
Backup	pg_dump, WAL archive	content/ directory + git
Failure modes	Connection loss, pool exhaustion, vacuum stalls	File not found

Postgres is the default reflex for "I have structured data." But writeonce's structured data is ~500 rows that change when the author saves a file. The mismatch is two orders of magnitude on the dataset size and one process on the deployment surface.

Option 3: Neo4j — Native Graph DB

Neo4j models articles as nodes and mappings as typed edges. Cypher makes the traversals natural:

MATCH (a:Article {sys_title: 'linux-misc'})-[:PREREQUISITE*1..3]->(p:Article)
RETURN p.sys_title

What it buys:

• Native graph storage — constant-time edge traversal regardless of dataset size
• Cypher is the right query language for the mappings problem
• Useful when the graph is the primary shape

What it costs:

• JVM process, Bolt protocol, driver — heaviest option on this list
• Document storage is secondary (properties on nodes), so article body and metadata are awkwardly split
• Dataset is tiny — the graph fits in a few KB of RAM; Neo4j's disk-backed adjacency is overkill

Comparison

Aspect	Neo4j	writeonce
Storage	Native adjacency on disk	(Future) HashMap<sys_title, Vec<Edge>> in memory
Traversal	Cypher over Bolt	Rust iteration over adjacency
Transactions	ACID with MVCC	Full rebuild on change
Deployment	JVM daemon + driver	Statically linked binary
Fit for writeonce dataset	Overprovisioned by 3+ orders of magnitude	Right-sized

Option 4: In-Memory Graph — NetworkX / petgraph

NetworkX (Python) and petgraph (Rust) are libraries, not databases. You load the graph into process memory and traverse it directly. This is the model gestured at in ai-agents-content-management.md line 181 — "traversable knowledge graphs available on RAM."

For writeonce, petgraph is the right shape:

use petgraph::graph::DiGraph;

let mut g: DiGraph<String, MappingKind> = DiGraph::new();
// nodes: one per sys_title
// edges: one per mapping (related, prerequisite, series, ...)

What it buys:

• Zero extra processes — compiles into the wo-store crate
• Constant-time neighbor lookup, standard BFS / Dijkstra / SCC algorithms included
• Rebuilt cheaply on any content/ change by walking the .seg and following mappings

What it costs:

• No persistence layer — but the graph is derived from JSON, same as title.idx, so it rebuilds on cold start for free
• No query language — but the traversals agents need (all prerequisites of X, next article in series Y) are short Rust functions

Comparison

Aspect	petgraph (in-memory)	Neo4j
Location	Same process as wo-store	External JVM
Build time	Single pass over .seg	Bulk import via CSV / Cypher
Cost per traversal	Pointer chase in RAM	Network round trip + disk I/O
Query language	Rust	Cypher
Scales to	Millions of nodes in RAM (plenty of headroom)	Billions on disk

For writeonce's hundreds of articles, petgraph is the correct answer. It fits the existing architecture — derived from content/, rebuildable, no external process — the same shape as title.idx already has.

Decision Matrix

Option	Dataset Fit	Process Count	Graph Support	Matches writeonce Philosophy
CouchDB Mango	Overprovisioned	+1 (Erlang)	Manual joins	No — external daemon
Postgres JSONB	Overprovisioned	+1 (Postgres)	Recursive CTE	No — external daemon
Neo4j	Overprovisioned	+1 (JVM)	Native, excellent	No — external daemon
petgraph in-memory	Right-sized	0	Native, in-process	Yes
Current .seg + .idx	Right-sized	0	None yet	Already here

Proposed Addition: mappings.idx Backed by petgraph

Per 05-datalayer.md, indexes live alongside .seg. Add a fourth index:

data/
  articles.seg
  index/
    title.idx        # existing — O(1) sys_title lookup
    date.idx         # existing — sorted by published_on
    tags.idx         # existing — inverted index by tag
    mappings.idx     # NEW — serialized petgraph adjacency

New wo-graph crate (or extension to wo-index):

pub struct MappingGraph {
    graph: DiGraph<SysTitle, MappingKind>,
    by_title: HashMap<String, NodeIndex>,
}

impl MappingGraph {
    pub fn neighbors(&self, sys_title: &str, kind: MappingKind) -> Vec<&str> { ... }
    pub fn prerequisites_transitive(&self, sys_title: &str) -> Vec<&str> { ... }
    pub fn series(&self, name: &str) -> Vec<&str> { ... }  // ordered by .order
}

Rebuild on every Store::rebuild(). Drop and recompute on any ContentChange — the graph is small enough that incremental updates are not worth the bug surface.

Key Takeaways

1. A document+graph database is the right model — but not the right dependency. writeonce's articles really are documents with graph edges. That does not mean importing CouchDB, Postgres, or Neo4j. It means writing ~200 lines that give you the document and graph operations the product actually uses.

2. Dataset size dictates architecture. SurrealDB, Postgres, and Neo4j all assume millions of rows and concurrent writers. writeonce has hundreds of articles and one author. The gap is where external databases become overhead, not infrastructure.

3. Git already solves the problems CouchDB's revisions solve. Content lives in content/ under version control. _rev, replication, and change history are the author's git history.

4. Query languages are a cost, not a benefit, at this scale. Mango selectors, JSONB operators, and Cypher exist because production queries are written by humans against large evolving datasets. writeonce's queries are fixed (get_by_title, list_by_tag, list_published) and written once in Rust.

5. The graph belongs in RAM. Article mappings form a small, mostly static DAG. petgraph holds it with zero protocol overhead and rebuilds from content/ on cold start — same lifecycle as title.idx.

Reference

If the graph side of this ever grows beyond what petgraph comfortably handles, the reference points are:

git submodule add https://github.com/neo4j/neo4j.git references/neo4j
git submodule add https://github.com/apache/couchdb.git references/couchdb
git submodule add https://github.com/petgraph/petgraph.git references/petgraph

Key files to study:

• petgraph/src/graph_impl/ — adjacency list implementation, the smallest viable graph backend
• couchdb/src/mango/ — Mango query compilation, if a selector-style query API ever becomes useful
• neo4j/community/cypher/ — Cypher planner, for how a real graph query language is structured

See also:

• surreal-case-study.md — why writeonce does not use a multi-model DB for live queries