writeonce/docs/runtime/surreal-case-study.md

SurrealDB — Runtime Case Study

Reference repository: github.com/surrealdb/surrealdb

git submodule add https://github.com/surrealdb/surrealdb.git references/surrealdb

The Question

Does SurrealDB only rely on async/await for concurrency?

No. SurrealDB uses a layered concurrency model — async/await is one layer, but it also uses OS thread pools, CPU-affinity-pinned workers, lock-free data structures, and parallel computation frameworks. Each layer serves a different purpose.

Architecture Overview

SurrealDB is a single Rust binary that ships a multi-model database (documents, graphs, key-value) with real-time live queries. It supports multiple deployment modes:

• Server: surreal start runs HTTP/WebSocket API via Axum + storage engine
• Embedded: the library crate embeds directly in Rust applications
• WASM: runs in the browser with IndexedDB backend

Concurrency Layers

Layer 1: Tokio — Async I/O and Request Handling

The primary runtime. Handles:

• HTTP/WebSocket connections (via Axum)
• Network I/O (accept, read, write)
• Timer-based operations
• Task scheduling (M:N scheduling of futures onto OS threads)

Client request → Axum handler (async) → parse query → execute → respond

Every request handler is an async function. Tokio's multi-threaded executor distributes tasks across OS threads using work-stealing.

Layer 2: Rayon — Parallel CPU-Bound Computation

For operations that are compute-heavy, not I/O-bound:

• Query plan execution across partitions
• Data processing and transformation
• Parallel iteration over result sets

Rayon provides par_iter() — automatic parallelism across CPU cores. It has its own thread pool, separate from tokio's.

Layer 3: affinitypool — CPU-Pinned Storage I/O

SurrealDB's custom crate. Runs blocking storage operations on a dedicated thread pool where each thread is pinned to a specific CPU core via libc CPU affinity syscalls.

Used by:

• RocksDB backend (blocking disk I/O)
• SurrealKV embedded storage
• In-memory engine for heavy operations

This bridges the async world (tokio) and the blocking world (disk I/O) without polluting the tokio thread pool with blocking calls.

Layer 4: Lock-Free Data Structures

The hot path in storage engines uses concurrent data structures that avoid locks entirely:

Crate	Data Structure	Used For
crossbeam-skiplist	Concurrent skip list	Index structures in SurrealKV and surrealmx
crossbeam-deque	Work-stealing deque	Task distribution
crossbeam-queue	Lock-free queue	Message passing
papaya	Concurrent HashMap	In-memory engine (surrealmx)
dashmap	Sharded concurrent map	Pub/sub routing for live queries
arc-swap	Atomic pointer swap	Hot-swapping data structures without locks
parking_lot	Fast mutex/rwlock	Where locking is needed (faster than std)

No Fibers

SurrealDB does not use fibers, green threads, or any custom scheduling mechanism. The concurrency model is:

Tokio tasks (async/await)      — for I/O-bound work
Rayon threads (par_iter)       — for CPU-bound work
affinitypool threads (pinned)  — for blocking storage I/O
Lock-free structures           — for concurrent data access

This is pragmatic — each concurrency mechanism is used where it fits, rather than forcing everything through one model.

Live Queries / Real-Time Subscriptions

SurrealDB's live query system pushes changes to connected clients in real-time:

1. Client registers a live query via WebSocket: LIVE SELECT * FROM person WHERE age > 21
2. Server tracks the query in a dashmap (concurrent map)
3. When a transaction commits changes to person, the engine evaluates which live queries are affected
4. Matching subscribers receive the diff via their WebSocket connection
5. Transport: tokio-tungstenite for WebSocket, async-channel for internal pub/sub routing

Comparison with writeonce Subscriptions

Aspect	SurrealDB	writeonce
Transport	WebSocket (tokio-tungstenite)	Raw socket fd (kernel-level write)
Query registration	SQL-like live query over WebSocket	register! macro binding fd to content pattern
Change detection	Transaction commit triggers evaluation	inotify detects file change
Notification routing	dashmap + async-channel	SubscriptionManager HashMap + direct write(fd)
Runtime	Tokio multi-threaded executor	Single-threaded epoll event loop
Protocol framing	WebSocket frames	Length-prefixed payloads (no protocol)

SurrealDB's live queries are the architectural inspiration for writeonce's subscription model (as noted in 03-data.md), but the implementation is fundamentally different — SurrealDB uses a full async runtime with WebSocket transport, while writeonce uses kernel fd notifications with no protocol layer.

Storage Engine Architecture

SurrealDB supports 5 backends:

Backend	Type	Concurrency
surrealmx	In-memory	Lock-free (papaya, crossbeam-skiplist, arc-swap)
surrealkv	Embedded persistent	Tokio async + crossbeam + parking_lot
RocksDB	Embedded persistent	affinitypool (CPU-pinned blocking threads)
TiKV	Distributed	Async TiKV client over gRPC
IndxDB	Browser/WASM	IndexedDB via wasm-bindgen-futures

Comparison with writeonce Storage

Aspect	SurrealDB	writeonce
Storage format	Key-value entries in LSM trees (RocksDB) or custom B-trees (SurrealKV)	.seg files with length-prefixed bincode records
Index	Built into storage engine	Separate .idx files (title hash, date sorted, tags inverted)
Concurrency	Multi-threaded with locks/lock-free structures	Single-threaded, positional I/O (pread/pwrite)
Transaction	ACID with MVCC	Full rebuild on change (article count is small)
Complexity	~100K+ lines across storage crates	~300 lines (wo-seg + wo-index)

writeonce's storage is intentionally simple — the dataset is small (hundreds of articles, not millions of rows), so a full rebuild on change is fast enough and avoids the complexity of concurrent transactions.

Key Takeaways

1. Async/await alone is not enough for a database. SurrealDB uses four concurrency mechanisms, each for a different workload profile.

2. Blocking I/O needs its own thread pool. The affinitypool pattern — CPU-pinned threads for storage operations — keeps blocking work off the async executor. writeonce avoids this entirely by using pread (non-blocking positional reads) in a single-threaded loop.

3. Lock-free data structures matter at scale. SurrealDB's hot path avoids mutexes. writeonce doesn't need this — single-threaded access means no contention.

4. Live queries are the hard problem. Both SurrealDB and writeonce solve "push changes to subscribers," but at vastly different scales. SurrealDB handles arbitrary SQL predicates over millions of rows. writeonce handles content queries over hundreds of articles.

5. The right amount of complexity depends on the problem. SurrealDB is a general-purpose database — it needs the complexity. writeonce is a content platform — the event loop model is sufficient and far simpler.

Reference

Add SurrealDB as a submodule for code reference:

git submodule add https://github.com/surrealdb/surrealdb.git references/surrealdb

Key files to study:

• crates/core/src/kvs/ — storage engine abstraction and transaction handling
• crates/sdk/src/api/engine/ — live query subscription routing
• lib/affinitypool/ — CPU-pinned thread pool for blocking I/O
• crates/core/src/sql/ — query parser and execution engine