every class with @table anotation is queriable table

This commit is contained in:
shoney.arickathil 2026-07-15 00:44:09 +02:00
parent 1aa0defbd3
commit 7ae3a20af1
16 changed files with 832 additions and 46 deletions

View file

@ -127,7 +127,8 @@ Stage-3 stubs (501) and policy-shaped 405/404 responses are **intentional and do
- **`rt` is monolithic on purpose.** Splitting it into the 14 sibling crates is Phase-by-Phase work, not a Stage-2 refactor.
- **`reference/crates/` is its own workspace.** Running `cargo build` at the root does not build v1. Running it in `reference/crates/` does.
- **Parser identifiers vs. keywords.** `subscribe`, `receive`, `expect_abort`, `me`, `self`, and lowercase `insert` are NOT keywords in the lexer — they stay as plain idents (only SQL-layer `INSERT` is a keyword). The 13b statement parser matches `insert` as an ident; adding these to the keyword map breaks `service rest "..." expose subscribe` and method bodies.
- **Parser identifiers vs. keywords.** `subscribe`, `receive`, `expect_abort`, `me`, `self`, and lowercase `insert`/`select` are NOT keywords in the lexer — they stay as plain idents (only SQL-layer `INSERT`/`SELECT` are keywords). The 13b statement parser matches `insert` as an ident; the select expression is recognised by the two-token shape `select <Ident> {`. Adding these to the keyword map breaks `service rest "..." expose subscribe` and method bodies.
- **Type-level annotations.** `@table(name: "...", index: [a, b])` before a `type`/`class` configures storage (it never toggles table-ness — every type IS a table). Unknown keys inside `@table(...)` are parse errors; unknown annotation *names* (`@foo`) skip silently. Engine secondary indexes are maintained ONLY via `Engine::row_insert`/`row_remove` — never touch `tables` directly or indexes drift.
- **Parser skip-on-block.** Unknown triggers (`on update do ...`) are parsed-and-discarded by brace-depth-aware skipping. Object literals like `{ article_id: self.id }` inside trigger actions contain `}` that must not be mistaken for the type's outer close brace — the depth counter exists specifically because of this. Exception since 13b: `fn` inside a `class` parses into a real `MethodDecl` (body statements + expressions, executed by `method.rs`); `fn` inside a plain `type` still skips.
- **Newline significance.** The lexer emits `Kind::Newline` tokens and the parser uses them to end policy/trigger lines. Do not filter newlines globally.
- **Default-value parsing.** `= now()` is recognised explicitly as `DefaultExpr::Now`; anything else falls into an opaque-expression path that `engine::eval_default` then **omits from created rows** (computed fields display as empty, not as debug-printed tokens).

View file

@ -22,6 +22,23 @@ pub struct TypeDecl {
pub is_class: bool,
/// Row-scoped methods (plan 13b). Only populated for classes.
pub methods: Vec<MethodDecl>,
/// Storage configuration from a type-level `@table(...)` annotation.
/// Every type/class IS a table regardless (plan 13 decisions 3/5) —
/// `@table` configures storage, it never toggles it.
pub table: TableCfg,
}
/// `@table(name: "prices", index: [product, at], index: [sku])` — optional
/// storage configuration. `shard_key:`/`retention:` are reserved for later
/// phases and rejected by the parser until they land.
#[derive(Debug, Clone, Default)]
pub struct TableCfg {
/// Storage/table name override. Defaults to the type name. Consumed by
/// the SQL layer and plans 10–12; WAL records keep the type name as the
/// stable identifier.
pub name: Option<String>,
/// Composite secondary indexes — one `index: [a, b]` entry each.
pub indexes: Vec<Vec<String>>,
}
/// `fn name(args) -> Ret [in txn [snapshot]] { body }` — a row-scoped
@ -73,6 +90,17 @@ pub enum Expr {
Field(Box<Expr>, String),
/// `name(args)` — builtins: `latest`, `count`, `now`.
Call(String, Vec<Expr>),
/// `select Type{ field == expr, other_field }` — schema-layer select per
/// § Brace Disambiguation: operator entries are predicates, bare idents
/// are projections. Evaluates to a set (array) of row objects, shaped by
/// the projection when one is given. Equality predicates route through
/// the engine's secondary indexes when the type declares a matching
/// `@table(index: ...)`.
Select {
ty: String,
predicates: Vec<(String, BinOp, Box<Expr>)>,
projection: Vec<String>,
},
Unary(UnOp, Box<Expr>),
Binary(BinOp, Box<Expr>, Box<Expr>),
}

View file

@ -27,11 +27,19 @@ pub struct CompiledType {
pub methods: Vec<MethodDecl>,
/// True iff the type declared an `id: Id` column. Auto-populated on insert.
pub has_id: bool,
/// Storage/table name — `@table(name: "...")` override or the type name.
/// Metadata for the SQL layer and plans 10–12; the engine and WAL key
/// everything by type name (the stable identifier).
pub storage_name: String,
/// Composite secondary indexes from `@table(index: [...])`, validated
/// against the fields. The engine maintains these on every mutation.
pub indexes: Vec<Vec<String>>,
}
impl Catalog {
pub fn from_schemas(schemas: Vec<Schema>) -> Result<Self> {
let mut cat = Catalog::default();
let mut storage_names = std::collections::HashSet::new();
for s in schemas {
for t in s.types {
if cat.types.contains_key(&t.name) {
@ -41,6 +49,33 @@ impl Catalog {
f.name == "id" &&
matches!(f.ty, FieldTy::Scalar(ref n) if n == "Id")
);
// @table validation (plan 13 follow-up): the name must be
// catalog-unique; index columns must be stored scalar
// columns (scalars, unions, and `ref` FKs — not relations
// without a column, not arrays/structs).
let storage_name = t.table.name.clone().unwrap_or_else(|| t.name.clone());
if !storage_names.insert(storage_name.clone()) {
bail!("{}: @table name \"{storage_name}\" is already used by another type",
t.name);
}
for cols in &t.table.indexes {
for col in cols {
let Some(f) = t.fields.iter().find(|f| &f.name == col) else {
bail!("{}: @table index names unknown field `{col}`", t.name);
};
match &f.ty {
FieldTy::Scalar(_) | FieldTy::Union(_) | FieldTy::Ref(_) => {}
FieldTy::MultiEdge { .. } | FieldTy::MultiVia { .. }
| FieldTy::Backlink { .. } => bail!(
"{}: @table index field `{col}` is a relation without a \
stored column — index the `ref` side instead", t.name),
FieldTy::Array(_) | FieldTy::Struct(_) => bail!(
"{}: @table index field `{col}` is not a scalar column", t.name),
}
}
}
cat.order.push(t.name.clone());
cat.types.insert(t.name.clone(), CompiledType {
name: t.name.clone(),
@ -48,6 +83,8 @@ impl Catalog {
services: t.services,
methods: t.methods,
has_id,
storage_name,
indexes: t.table.indexes,
});
}
}
@ -64,6 +101,37 @@ mod tests {
use super::*;
use crate::parser::parse;
#[test]
fn table_annotation_validation() {
// Duplicate storage names collide across types.
let sch = parse("@table(name: \"t\")\ntype A { id: Id }\n@table(name: \"t\")\ntype B { id: Id }").unwrap();
let err = Catalog::from_schemas(vec![sch]).unwrap_err().to_string();
assert!(err.contains("already used"), "{err}");
// A name override colliding with another type's default name.
let sch = parse("@table(name: \"B\")\ntype A { id: Id }\ntype B { id: Id }").unwrap();
assert!(Catalog::from_schemas(vec![sch]).is_err());
// Index on a missing field.
let sch = parse("@table(index: [nope])\ntype C { id: Id }").unwrap();
let err = Catalog::from_schemas(vec![sch]).unwrap_err().to_string();
assert!(err.contains("unknown field `nope`"), "{err}");
// Index on a relation without a stored column.
let sch = parse("@table(index: [prices])\nclass P { id: Id\n prices: multi Price }").unwrap();
let err = Catalog::from_schemas(vec![sch]).unwrap_err().to_string();
assert!(err.contains("relation without a stored column"), "{err}");
// Valid: ref FK + scalar composite; storage_name defaults to type name.
let sch = parse(
"@table(name: \"prices\", index: [product, at])\nclass Price { id: Id\n product: ref Product\n at: Text }"
).unwrap();
let cat = Catalog::from_schemas(vec![sch]).unwrap();
let t = cat.get("Price").unwrap();
assert_eq!(t.storage_name, "prices");
assert_eq!(t.indexes, vec![vec!["product".to_string(), "at".to_string()]]);
}
#[test]
fn catalog_collects_types_and_detects_id() {
let sch = parse(r#"

View file

@ -9,16 +9,63 @@ use crate::compile::{Catalog, CompiledType};
use anyhow::Result;
use serde_json::{json, Map, Value};
use std::collections::BTreeMap;
use std::collections::{BTreeMap, BTreeSet};
use std::time::{SystemTime, UNIX_EPOCH};
pub type Row = Map<String, Value>;
/// Comparable encoding of an indexed column value — the key space of the
/// secondary indexes (`@table(index: [...])`). Ordering: Null < Bool < Int
/// < Str, then natural order within each. Residual filters always re-check
/// with real JSON equality, so encoding collisions cannot produce wrong
/// results — only wasted candidates.
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord)]
enum IndexKey {
Null,
Bool(bool),
Int(i64),
Str(String),
}
impl IndexKey {
fn from_value(v: Option<&Value>) -> IndexKey {
match v {
None | Some(Value::Null) => IndexKey::Null,
Some(Value::Bool(b)) => IndexKey::Bool(*b),
Some(other) => match other.as_i64() {
Some(n) => IndexKey::Int(n),
None => match other {
Value::String(s) => IndexKey::Str(s.clone()),
v => IndexKey::Str(v.to_string()),
},
},
}
}
}
/// One composite secondary index: ordered key tuples → row ids. Per-shard,
/// in RAM, maintained incrementally by [`Engine::row_insert`]/[`row_remove`].
#[derive(Debug)]
struct Index {
cols: Vec<String>,
map: BTreeMap<Vec<IndexKey>, BTreeSet<i64>>,
}
impl Index {
fn key_for(&self, row: &Row) -> Vec<IndexKey> {
self.cols.iter().map(|c| IndexKey::from_value(row.get(c))).collect()
}
}
#[derive(Debug, Default)]
pub struct Engine {
catalog: Catalog,
/// type_name → { id → row }
tables: std::collections::HashMap<String, BTreeMap<i64, Row>>,
/// type_name → its secondary indexes (`@table(index: [...])`). Only
/// mutated by `row_insert`/`row_remove` — every table mutation path
/// (CRUD, replay, txn undo) goes through those two helpers.
indexes: std::collections::HashMap<String, Vec<Index>>,
/// per-type id allocator
next_id: std::collections::HashMap<String, i64>,
/// id stride — 1 for a standalone engine, `n_shards` for a 09b shard so
@ -67,14 +114,22 @@ impl Engine {
/// One shard of a thread-per-core deployment (plan 09b): same engine,
/// interleaved id minting.
pub fn for_shard(catalog: Catalog, shard: usize, n_shards: usize) -> Self {
let mut tables = std::collections::HashMap::new();
let mut tables = std::collections::HashMap::new();
let mut next_id = std::collections::HashMap::new();
let mut indexes = std::collections::HashMap::new();
for name in catalog.order.iter() {
tables.insert(name.clone(), BTreeMap::new());
next_id.insert(name.clone(), shard as i64 + 1);
let t = catalog.get(name).expect("type present");
if !t.indexes.is_empty() {
indexes.insert(name.clone(), t.indexes.iter().map(|cols| Index {
cols: cols.clone(),
map: BTreeMap::new(),
}).collect());
}
}
Self { catalog, tables, next_id, id_step: n_shards.max(1) as i64, wal: None, staged: false,
txn: None }
Self { catalog, tables, indexes, next_id, id_step: n_shards.max(1) as i64,
wal: None, staged: false, txn: None }
}
/// Attach a per-commit WAL (fsync inside each mutation). Must happen
@ -139,24 +194,25 @@ impl Engine {
match rec {
WalRec::Create { ty, row } => {
let Some(id) = row.get("id").and_then(|v| v.as_i64()) else { return };
if let Some(table) = self.tables.get_mut(ty) {
table.insert(id, row.clone());
if self.tables.contains_key(ty) {
self.row_insert(ty, id, row.clone());
let step = self.id_step;
let counter = self.next_id.entry(ty.clone()).or_insert(1);
while *counter <= id { *counter += step; }
}
}
WalRec::Update { ty, id, body } => {
if let Some(row) = self.tables.get_mut(ty).and_then(|t| t.get_mut(id)) {
if let Value::Object(input) = body {
for (k, v) in input {
if k != "id" { row.insert(k.clone(), v.clone()); }
}
// Remove-then-insert keeps the secondary indexes in step.
let Some(mut row) = self.row_remove(ty, *id) else { return };
if let Value::Object(input) = body {
for (k, v) in input {
if k != "id" { row.insert(k.clone(), v.clone()); }
}
}
self.row_insert(ty, *id, row);
}
WalRec::Delete { ty, id } => {
if let Some(table) = self.tables.get_mut(ty) { table.remove(id); }
self.row_remove(ty, *id);
}
// A method's mutations — the frame validated whole, apply all.
WalRec::Txn { recs } => {
@ -203,10 +259,13 @@ impl Engine {
for u in undo.into_iter().rev() {
match u {
Undo::Created { ty, id } => {
if let Some(t) = self.tables.get_mut(&ty) { t.remove(&id); }
self.row_remove(&ty, id);
}
Undo::Updated { ty, id, prev } | Undo::Deleted { ty, id, row: prev } => {
if let Some(t) = self.tables.get_mut(&ty) { t.insert(id, prev); }
// Clear the current version's index keys (if any row is
// present) before restoring the previous one.
self.row_remove(&ty, id);
self.row_insert(&ty, id, prev);
}
}
}
@ -268,24 +327,23 @@ impl Engine {
.unwrap_or_else(|| self.mint_id(ty));
row.insert("id".into(), json!(id));
self.tables.get_mut(ty).unwrap().insert(id, row.clone());
self.row_insert(ty, id, row.clone());
if let Some(t) = self.txn.as_mut() {
t.undo.push(Undo::Created { ty: ty.into(), id });
}
// Dual-write order: RAM applied above, durable now, ack after return.
if let Err(e) = self.wal_log(crate::wal::WalRec::Create { ty: ty.into(), row: row.clone() }) {
self.tables.get_mut(ty).unwrap().remove(&id); // never ack non-durable
self.row_remove(ty, id); // never ack non-durable
return Err(e);
}
Ok(row)
}
/// Merge-update a row.
/// Merge-update a row. Remove-then-insert so the secondary indexes see
/// both the old and the new key tuples.
pub fn update(&mut self, ty: &str, id: i64, body: Value) -> Result<Option<Row>> {
let table = self.tables.get_mut(ty)
.ok_or_else(|| anyhow::anyhow!("no such type: {ty}"))?;
let Some(row) = table.get_mut(&id) else { return Ok(None); };
let prev = row.clone();
let Some(prev) = self.table(ty)?.get(&id).cloned() else { return Ok(None); };
let mut row = prev.clone();
if let Value::Object(input) = &body {
for (k, v) in input {
if k == "id" { continue; } // don't let the client mutate the primary key
@ -293,36 +351,120 @@ impl Engine {
}
}
let updated = row.clone();
self.row_remove(ty, id);
self.row_insert(ty, id, row);
if let Some(t) = self.txn.as_mut() {
t.undo.push(Undo::Updated { ty: ty.into(), id, prev: prev.clone() });
}
if let Err(e) = self.wal_log(crate::wal::WalRec::Update { ty: ty.into(), id, body }) {
self.tables.get_mut(ty).unwrap().insert(id, prev); // undo: never ack non-durable
self.row_remove(ty, id); // undo: never ack non-durable
self.row_insert(ty, id, prev);
return Err(e);
}
Ok(Some(updated))
}
pub fn delete(&mut self, ty: &str, id: i64) -> Result<bool> {
let table = self.tables.get_mut(ty)
.ok_or_else(|| anyhow::anyhow!("no such type: {ty}"))?;
let Some(removed) = table.remove(&id) else { return Ok(false) };
self.table(ty)?; // surface unknown-type as an error, not a silent false
let Some(removed) = self.row_remove(ty, id) else { return Ok(false) };
if let Some(t) = self.txn.as_mut() {
t.undo.push(Undo::Deleted { ty: ty.into(), id, row: removed.clone() });
}
if let Err(e) = self.wal_log(crate::wal::WalRec::Delete { ty: ty.into(), id }) {
self.tables.get_mut(ty).unwrap().insert(id, removed); // undo
self.row_insert(ty, id, removed); // undo
return Err(e);
}
Ok(true)
}
/// Equality lookup, index-accelerated. Picks the index whose leading
/// columns form the longest prefix of the queried fields (prefix range
/// scan on its BTreeMap); remaining predicates filter the candidates;
/// no matching index → full scan. Results in id order. `eq` empty =
/// plain `list`.
pub fn find_by(&self, ty: &str, eq: &[(String, Value)]) -> Result<Vec<Row>> {
let table = self.table(ty)?;
if eq.is_empty() {
return Ok(table.values().cloned().collect());
}
// Real-equality re-check over ALL queried fields — the index only
// narrows candidates, it never decides membership.
let matches = |row: &Row| eq.iter().all(|(f, v)| {
match row.get(f) {
Some(rv) => rv == v,
None => v.is_null(),
}
});
let mut best: Option<(&Index, usize)> = None;
if let Some(idxs) = self.indexes.get(ty) {
for idx in idxs {
let mut k = 0;
for col in &idx.cols {
if eq.iter().any(|(f, _)| f == col) { k += 1; } else { break; }
}
if k > 0 && best.map_or(true, |(_, bk)| k > bk) {
best = Some((idx, k));
}
}
}
let Some((idx, k)) = best else {
return Ok(table.values().filter(|r| matches(r)).cloned().collect());
};
let prefix: Vec<IndexKey> = idx.cols[..k].iter()
.map(|c| IndexKey::from_value(eq.iter().find(|(f, _)| f == c).map(|(_, v)| v)))
.collect();
let mut ids: Vec<i64> = Vec::new();
// A shorter Vec sorts before any longer Vec sharing its prefix, so
// range(prefix..) starts exactly at the first candidate key.
for (key, set) in idx.map.range(prefix.clone()..) {
if key.len() < k || key[..k] != prefix[..] { break; }
ids.extend(set.iter().copied());
}
ids.sort_unstable();
Ok(ids.into_iter()
.filter_map(|id| table.get(&id))
.filter(|r| matches(r))
.cloned()
.collect())
}
// --- helpers ---
fn table(&self, ty: &str) -> Result<&BTreeMap<i64, Row>> {
self.tables.get(ty).ok_or_else(|| anyhow::anyhow!("no such type: {ty}"))
}
/// THE two table-mutation primitives — every path that changes a row
/// (CRUD, WAL replay, txn undo) goes through these so the secondary
/// indexes can never drift from the tables.
fn row_insert(&mut self, ty: &str, id: i64, row: Row) {
if let Some(idxs) = self.indexes.get_mut(ty) {
for idx in idxs {
let key = idx.key_for(&row);
idx.map.entry(key).or_default().insert(id);
}
}
if let Some(t) = self.tables.get_mut(ty) {
t.insert(id, row);
}
}
fn row_remove(&mut self, ty: &str, id: i64) -> Option<Row> {
let row = self.tables.get_mut(ty)?.remove(&id)?;
if let Some(idxs) = self.indexes.get_mut(ty) {
for idx in idxs {
let key = idx.key_for(&row);
if let Some(set) = idx.map.get_mut(&key) {
set.remove(&id);
if set.is_empty() { idx.map.remove(&key); }
}
}
}
Some(row)
}
fn compiled(&self, ty: &str) -> Result<&CompiledType> {
self.catalog.get(ty).ok_or_else(|| anyhow::anyhow!("no such type: {ty}"))
}
@ -444,6 +586,63 @@ mod tests {
Engine::new(cat)
}
const INDEXED: &str = r#"
@table(index: [owner, at])
type Item { id: Id
owner: Int
at: Text
service rest "/api/items" expose list }
"#;
#[test]
fn secondary_index_tracks_create_update_delete() {
let mut eng = engine_from(INDEXED);
for i in 0..3 {
eng.create("Item", json!({"owner": 1, "at": format!("t{i}")})).unwrap();
}
eng.create("Item", json!({"owner": 2, "at": "t9"})).unwrap();
// prefix match (owner) and full composite (owner, at)
let one = eng.find_by("Item", &[("owner".into(), json!(1))]).unwrap();
assert_eq!(one.len(), 3);
let exact = eng.find_by("Item",
&[("owner".into(), json!(1)), ("at".into(), json!("t1"))]).unwrap();
assert_eq!(exact.len(), 1);
// update moves the row between index keys
let id = exact[0]["id"].as_i64().unwrap();
eng.update("Item", id, json!({"owner": 2})).unwrap();
assert_eq!(eng.find_by("Item", &[("owner".into(), json!(1))]).unwrap().len(), 2);
assert_eq!(eng.find_by("Item", &[("owner".into(), json!(2))]).unwrap().len(), 2);
// delete clears its entries
eng.delete("Item", id).unwrap();
assert_eq!(eng.find_by("Item", &[("owner".into(), json!(2))]).unwrap().len(), 1);
// non-indexed field → scan fallback, same semantics
assert_eq!(eng.find_by("Item", &[("at".into(), json!("t0"))]).unwrap().len(), 1);
// index answers equal scan answers (ground truth)
let scan: Vec<_> = eng.list("Item").unwrap().into_iter()
.filter(|r| r["owner"] == json!(1)).collect();
assert_eq!(eng.find_by("Item", &[("owner".into(), json!(1))]).unwrap(), scan);
}
#[test]
fn txn_abort_restores_index_state() {
let mut eng = engine_from(INDEXED);
eng.create("Item", json!({"owner": 1, "at": "a"})).unwrap(); // id 1
eng.begin_txn().unwrap();
eng.create("Item", json!({"owner": 1, "at": "b"})).unwrap();
eng.update("Item", 1, json!({"owner": 5})).unwrap();
eng.abort_txn();
assert_eq!(eng.find_by("Item", &[("owner".into(), json!(1))]).unwrap().len(), 1);
assert!(eng.find_by("Item", &[("owner".into(), json!(5))]).unwrap().is_empty());
assert_eq!(eng.list("Item").unwrap().len(), 1);
}
#[test]
fn crud_roundtrip_auto_id() {
let mut eng = engine_from(r#"

View file

@ -178,10 +178,58 @@ fn eval(cx: &mut Cx, e: &Expr) -> Result<Value, MethodError> {
match b {
Value::Object(m) => m.get(field).cloned().ok_or_else(||
MethodError::Exec(format!("no field `{field}`"))),
// Dotted access distributes over a set — the spec's
// cardinality rule: `select Price{...}.amount` is the set
// of amounts.
Value::Array(items) => {
let mut out = Vec::with_capacity(items.len());
for it in items {
match it {
Value::Object(m) => out.push(m.get(field).cloned()
.ok_or_else(|| MethodError::Exec(
format!("no field `{field}` in set element")))?),
other => return Err(MethodError::Exec(format!(
"`.{field}` on a non-object set element ({other})"))),
}
}
Ok(Value::Array(out))
}
other => Err(MethodError::Exec(format!(
"`.{field}` on a non-object value ({other})"))),
}
}
Expr::Select { ty, predicates, projection } => {
// Equality predicates route through the engine's secondary
// indexes (`@table(index: ...)`) via find_by; other comparison
// operators filter the candidates.
let mut eq = Vec::new();
let mut rest = Vec::new();
for (field, op, rhs) in predicates {
let v = eval(cx, rhs)?;
if *op == BinOp::Eq { eq.push((field.clone(), v)); }
else { rest.push((field.as_str(), *op, v)); }
}
let rows = cx.e.find_by(ty, &eq)
.map_err(|e| MethodError::Exec(format!("select {ty}: {e}")))?;
let mut out = Vec::new();
for row in rows {
if !rest.iter().all(|(f, op, v)| pred_holds(row.get(*f), *op, v)) {
continue;
}
out.push(if projection.is_empty() {
Value::Object(row)
} else {
let mut shaped = Map::new();
for p in projection {
if let Some(v) = row.get(p) {
shaped.insert(p.clone(), v.clone());
}
}
Value::Object(shaped)
});
}
Ok(Value::Array(out))
}
Expr::Call(name, args) => {
let mut vals = Vec::with_capacity(args.len());
for a in args { vals.push(eval(cx, a)?); }
@ -249,6 +297,33 @@ fn as_i64(v: &Value) -> Result<i64, MethodError> {
v.as_i64().ok_or_else(|| MethodError::Exec(format!("expected a number, got {v}")))
}
/// Non-equality select predicate over a row column. Numbers compare
/// numerically, strings lexicographically (covers `at > "2026-…"`);
/// mismatched or missing values fail the predicate rather than erroring —
/// a filter, not an expression.
fn pred_holds(actual: Option<&Value>, op: BinOp, wanted: &Value) -> bool {
let Some(a) = actual else { return op == BinOp::Ne && !wanted.is_null() };
match op {
BinOp::Ne => a != wanted,
BinOp::Lt | BinOp::Le | BinOp::Gt | BinOp::Ge => {
let ord = match (a.as_i64(), wanted.as_i64()) {
(Some(x), Some(y)) => x.cmp(&y),
_ => match (a.as_str(), wanted.as_str()) {
(Some(x), Some(y)) => x.cmp(y),
_ => return false,
},
};
match op {
BinOp::Lt => ord.is_lt(),
BinOp::Le => ord.is_le(),
BinOp::Gt => ord.is_gt(),
_ => ord.is_ge(),
}
}
_ => unreachable!("equality predicates go through find_by"),
}
}
/// If `field` names a relation on the receiving type, materialize it:
/// `multi T` / `backlink T.f` → array of the related rows, in id order.
/// Returns `Ok(None)` when `field` is not a relation (plain column access).
@ -285,10 +360,11 @@ fn relation_rows(cx: &mut Cx, field: &str) -> Result<Option<Value>, MethodError>
};
let self_id = cx.self_row.get("id").cloned().unwrap_or(Value::Null);
let rows = cx.e.list(&target)
// Index-accelerated when the target declares @table(index: [<link>, …]);
// find_by falls back to a scan otherwise.
let rows = cx.e.find_by(&target, &[(link_field, self_id)])
.map_err(|e| MethodError::Exec(e.to_string()))?
.into_iter()
.filter(|r| r.get(&link_field) == Some(&self_id))
.map(Value::Object)
.collect::<Vec<_>>();
Ok(Some(Value::Array(rows)))
@ -326,11 +402,13 @@ mod tests {
use crate::parser::parse;
const PRICING: &str = r#"
@table(name: "prices", index: [product, at])
class Price {
id: Id
product: ref Product
amount: Money
currency: Text = "EUR"
at: Text = "t0"
fn discounted(pct: Int) -> Money {
return self.amount * (100 - pct) / 100;
@ -352,6 +430,10 @@ class Product {
insert Price { product: self.id, amount: amount };
}
fn history() -> [Money] in txn {
return select Price{ product == self.id, amount };
}
service rest "/api/products" expose list, get, create
}
"#;
@ -390,6 +472,31 @@ class Product {
assert_eq!(v, json!(5999));
}
#[test]
fn select_expression_filters_and_projects() {
let mut e = engine();
let p1 = e.create("Product", json!({"sku": "A", "name": "A"})).unwrap();
let p2 = e.create("Product", json!({"sku": "B", "name": "B"})).unwrap();
let (id1, id2) = (p1["id"].as_i64().unwrap(), p2["id"].as_i64().unwrap());
let set = method(&e, "Product", "set_price");
call(&mut e, "Product", id1, &set, &args(json!({"amount": 100}))).unwrap();
call(&mut e, "Product", id1, &set, &args(json!({"amount": 200}))).unwrap();
call(&mut e, "Product", id2, &set, &args(json!({"amount": 999}))).unwrap();
// `history` = select Price{ product == self.id, amount } — the
// equality predicate rides the (product, at) index; the projection
// shapes each row down to { amount }.
let hist = method(&e, "Product", "history");
let v = call(&mut e, "Product", id1, &hist, &Map::new()).unwrap();
assert_eq!(v, json!([{"amount": 100}, {"amount": 200}]));
// Indexed relation read (self.prices) equals the select's row set.
let cur = method(&e, "Product", "current_price");
assert_eq!(call(&mut e, "Product", id1, &cur, &Map::new()).unwrap(), json!(200));
assert_eq!(call(&mut e, "Product", id2, &cur, &Map::new()).unwrap(), json!(999));
}
#[test]
fn pure_method_computes_from_self() {
let mut e = engine();

View file

@ -75,7 +75,21 @@ impl Parser {
self.skip_newlines();
if self.at_end() { break; }
match self.peek() {
Kind::KwType | Kind::KwClass => sch.types.push(self.parse_type()?),
Kind::KwType | Kind::KwClass =>
sch.types.push(self.parse_type(TableCfg::default())?),
// Type-level annotation: `@table(...)` configures the
// declaration that follows; unknown names skip silently
// (the field-annotation precedent).
Kind::At => {
if let Some(table) = self.parse_type_annotations()? {
self.skip_newlines();
if !matches!(self.peek(), Kind::KwType | Kind::KwClass) {
bail!("line {}: expected `type` or `class` after @table, got {}",
self.peek_line(), self.peek());
}
sch.types.push(self.parse_type(table)?);
}
}
// Skip constructs we don't execute yet.
Kind::HashHash(_)
| Kind::KwFn
@ -92,6 +106,79 @@ impl Parser {
Ok(sch)
}
/// Parse the type-level annotation list ahead of a `type`/`class`.
/// Returns `Some(cfg)` when a recognised `@table` was consumed, `None`
/// when the annotation was unknown and skipped (caller resumes the loop).
fn parse_type_annotations(&mut self) -> Result<Option<TableCfg>> {
self.expect(&Kind::At, "'@'")?;
let name = self.expect_ident("annotation name")?;
if name != "table" {
// Unknown type-level annotation: consume an optional (...) block
// and let the schema loop decide what the next token means.
if matches!(self.peek(), Kind::LParen) {
let mut depth = 1i32;
self.advance();
while depth > 0 && !self.at_end() {
match self.peek() {
Kind::LParen => { depth += 1; self.advance(); }
Kind::RParen => { depth -= 1; self.advance(); }
_ => { self.advance(); }
}
}
}
return Ok(None);
}
let mut cfg = TableCfg::default();
if !self.accept(&Kind::LParen) {
return Ok(Some(cfg)); // bare `@table` — legal no-op
}
loop {
self.skip_newlines();
if self.accept(&Kind::RParen) { break; }
let key = self.expect_ident("@table argument")?;
self.expect(&Kind::Colon, "':'")?;
match key.as_str() {
"name" => {
if cfg.name.is_some() {
bail!("line {}: @table(name: ...) given twice", self.peek_line());
}
match self.peek().clone() {
Kind::Str(s) => { self.advance(); cfg.name = Some(s); }
other => bail!("line {}: @table name must be a string, got {other}",
self.peek_line()),
}
}
"index" => {
self.expect(&Kind::LBracket, "'['")?;
let mut cols = Vec::new();
loop {
cols.push(self.expect_ident("index column")?);
if !self.accept(&Kind::Comma) { break; }
}
self.expect(&Kind::RBracket, "']'")?;
if cols.is_empty() {
bail!("line {}: @table index needs at least one column", self.peek_line());
}
cfg.indexes.push(cols);
}
// `shard_key`/`retention` are reserved for later phases —
// reject loudly rather than silently ignoring (no silent
// passthrough on surface we own).
other => bail!(
"line {}: unknown @table argument `{other}` \
(supported: name, index)", self.peek_line()),
}
self.skip_newlines();
if !self.accept(&Kind::Comma) {
self.skip_newlines();
self.expect(&Kind::RParen, "')' or ','")?;
break;
}
}
Ok(Some(cfg))
}
/// Walk forward until we reach the start of the next top-level construct
/// (another `type`, `##…`, or EOF), balancing braces in between.
fn skip_top_level_chunk(&mut self) -> Result<()> {
@ -118,7 +205,7 @@ impl Parser {
// --- type declaration ---
fn parse_type(&mut self) -> Result<TypeDecl> {
fn parse_type(&mut self, table: TableCfg) -> Result<TypeDecl> {
// `class` is the behavior-bearing sibling of `type` — identical field
// grammar plus `fn` methods (plan 13a). Storage/REST are class-blind.
let is_class = matches!(self.peek(), Kind::KwClass);
@ -143,7 +230,7 @@ impl Parser {
let mut decl = TypeDecl {
name, fields: Vec::new(), services: Vec::new(), is_class,
methods: Vec::new(),
methods: Vec::new(), table,
};
loop {
self.skip_newlines();
@ -507,7 +594,14 @@ impl Parser {
let mut ret = None;
if self.accept(&Kind::Arrow) {
ret = Some(self.expect_ident("return type")?);
// `-> Money` or `-> [Price]` — diagnostic-only in Stage 2.
if self.accept(&Kind::LBracket) {
let inner = self.expect_ident("return type")?;
self.expect(&Kind::RBracket, "']'")?;
ret = Some(format!("[{inner}]"));
} else {
ret = Some(self.expect_ident("return type")?);
}
}
let mut txn = TxnMode::None;
@ -729,6 +823,16 @@ impl Parser {
}
Kind::Ident(name) => {
self.advance();
// `select Type{ ... }` — schema-layer select expression.
// Lowercase `select` is an ident (only SQL `SELECT` is a
// keyword); the two-token shape Ident + LBrace disambiguates
// it from a variable named `select`.
if name == "select"
&& matches!(self.peek(), Kind::Ident(_))
&& matches!(self.toks.get(self.pos + 1).map(|t| &t.kind), Some(Kind::LBrace))
{
return self.parse_select_expr();
}
// `name(args)` — call form.
if self.accept(&Kind::LParen) {
let mut args = Vec::new();
@ -747,6 +851,42 @@ impl Parser {
other => bail!("line {}: expected expression, got {other}", self.peek_line()),
}
}
/// `select Type{ entries }` — per § Brace Disambiguation: an entry with a
/// comparison operator is a predicate; a bare identifier is a projection.
/// (`select` and the type name are already consumed up to the ident.)
fn parse_select_expr(&mut self) -> Result<Expr> {
let ty = self.expect_ident("type name after `select`")?;
self.expect(&Kind::LBrace, "'{'")?;
let mut predicates = Vec::new();
let mut projection = Vec::new();
loop {
self.skip_newlines();
if self.accept(&Kind::RBrace) { break; }
let field = self.expect_ident("field name")?;
let op = match self.peek() {
Kind::EqEq => Some(BinOp::Eq),
Kind::NotEq => Some(BinOp::Ne),
Kind::Lt => Some(BinOp::Lt),
Kind::LtEq => Some(BinOp::Le),
Kind::Gt => Some(BinOp::Gt),
Kind::GtEq => Some(BinOp::Ge),
_ => None,
};
match op {
Some(op) => {
self.advance();
// RHS is an additive expression — comparisons don't chain.
let rhs = self.parse_add()?;
predicates.push((field, op, Box::new(rhs)));
}
None => projection.push(field),
}
self.skip_newlines();
self.accept(&Kind::Comma);
}
Ok(Expr::Select { ty, predicates, projection })
}
}
#[cfg(test)]
@ -928,6 +1068,74 @@ class Product {
Stmt::Insert { ty, fields } if ty == "Price" && fields.len() == 2));
}
#[test]
fn parses_table_annotation() {
let src = r#"
@table(name: "prices", index: [product, at], index: [sku])
class Price {
id: Id
product: ref Product
amount: Money
at: Timestamp = now()
sku: SKU
}
@table
type Note { id: Id }
type Plain { id: Id }
"#;
let sch = parse(src).unwrap();
assert_eq!(sch.types.len(), 3);
let p = &sch.types[0];
assert_eq!(p.table.name.as_deref(), Some("prices"));
assert_eq!(p.table.indexes, vec![
vec!["product".to_string(), "at".to_string()],
vec!["sku".to_string()],
]);
// bare @table = legal no-op config
let n = &sch.types[1];
assert!(n.table.name.is_none() && n.table.indexes.is_empty());
assert!(sch.types[2].table.indexes.is_empty());
}
#[test]
fn table_annotation_error_cases() {
// Reserved-for-later keys error loudly — no silent passthrough.
let err = parse("@table(shard_key: sku)\ntype T { id: Id }").unwrap_err().to_string();
assert!(err.contains("unknown @table argument `shard_key`"), "{err}");
// @table must be followed by a type/class.
assert!(parse("@table(name: \"x\")\nfn stray() {}").is_err());
// Unknown annotation NAMES skip silently (field-annotation precedent).
let sch = parse("@experimental(anything, at: all)\ntype T { id: Id }").unwrap();
assert_eq!(sch.types.len(), 1);
assert_eq!(sch.types[0].name, "T");
}
#[test]
fn parses_select_expression() {
let src = r#"
class Product {
id: Id
fn history() -> [Price] in txn {
return select Price{ product == self.id, amount, at };
}
}
"#;
let m = &parse(src).unwrap().types[0].methods[0];
let Stmt::Return { expr: Some(Expr::Select { ty, predicates, projection }) } = &m.body[0]
else { panic!("expected return select, got {:?}", m.body[0]) };
assert_eq!(ty, "Price");
assert_eq!(predicates.len(), 1);
assert_eq!(predicates[0].0, "product");
assert!(matches!(predicates[0].1, BinOp::Eq));
assert!(matches!(&*predicates[0].2, Expr::Field(b, f) if f == "id"
&& matches!(&**b, Expr::Ident(s) if s == "self")));
assert_eq!(projection, &vec!["amount".to_string(), "at".to_string()]);
}
#[test]
fn method_body_expression_ast() {
let src = r#"

View file

@ -134,11 +134,17 @@ fn shard_gone() -> Response {
Response::status(Status::INTERNAL_SERVER_ERROR).text("owning shard unavailable")
}
fn list_h(ctx: &Rc<ShardCtx>, ty: &str, _req: &Request, _params: &RouteParams) -> Response {
fn list_h(ctx: &Rc<ShardCtx>, ty: &str, req: &Request, _params: &RouteParams) -> Response {
// `?field=value` filters run through the engine's find_by — secondary
// indexes (`@table(index: ...)`) accelerate, scan is the fallback.
let filters = match query_filters(ctx, ty, req.query.as_deref()) {
Ok(f) => f,
Err(r) => return r,
};
let ty_owned = ty.to_string();
// Fan out to every shard, merge by id — the cross-shard read per 09b.
let per_shard: Vec<Result<Vec<Row>, String>> =
ctx.fanout(move |e| e.list(&ty_owned).map_err(|e| e.to_string()));
ctx.fanout(move |e| e.find_by(&ty_owned, &filters).map_err(|e| e.to_string()));
let mut rows = Vec::new();
for r in per_shard {
match r {
@ -150,6 +156,60 @@ fn list_h(ctx: &Rc<ShardCtx>, ty: &str, _req: &Request, _params: &RouteParams) -
Response::ok().json(&json!(rows))
}
/// Parse `k=v&k2=v2` into equality filters. Fields must be stored columns
/// of the type (unknown → 400); values coerce int → bool → string.
fn query_filters(
ctx: &Rc<ShardCtx>,
ty: &str,
query: Option<&str>,
) -> Result<Vec<(String, Value)>, Response> {
let Some(q) = query.filter(|q| !q.is_empty()) else { return Ok(Vec::new()) };
let mut out = Vec::new();
let engine = ctx.engine.borrow();
let t = engine.catalog().get(ty)
.ok_or_else(|| Response::status(Status::INTERNAL_SERVER_ERROR).text("type missing"))?;
for pair in q.split('&').filter(|s| !s.is_empty()) {
let (k, v) = pair.split_once('=').unwrap_or((pair, ""));
let (k, v) = (url_decode(k), url_decode(v));
let stored = k == "id" || t.fields.iter().any(|f| f.name == k && matches!(
f.ty,
crate::ast::FieldTy::Scalar(_) | crate::ast::FieldTy::Union(_)
| crate::ast::FieldTy::Ref(_)
));
if !stored {
return Err(Response::status(Status::BAD_REQUEST)
.text(format!("unknown filter field `{k}` for {ty}")));
}
let val = if let Ok(n) = v.parse::<i64>() { json!(n) }
else if v == "true" { json!(true) }
else if v == "false" { json!(false) }
else { Value::String(v) };
out.push((k, val));
}
Ok(out)
}
/// Minimal percent-decoding for query values (`%XX` and `+` → space).
fn url_decode(s: &str) -> String {
let b = s.as_bytes();
let mut out = Vec::with_capacity(b.len());
let mut i = 0;
while i < b.len() {
match b[i] {
b'%' if i + 2 < b.len() => {
let hex = |c: u8| (c as char).to_digit(16);
match (hex(b[i + 1]), hex(b[i + 2])) {
(Some(h), Some(l)) => { out.push((h * 16 + l) as u8); i += 3; }
_ => { out.push(b[i]); i += 1; }
}
}
b'+' => { out.push(b' '); i += 1; }
c => { out.push(c); i += 1; }
}
}
String::from_utf8_lossy(&out).into_owned()
}
fn get_h(ctx: &Rc<ShardCtx>, ty: &str, _req: &Request, params: &RouteParams) -> Response {
let id = match parse_id(params) {
Ok(id) => id,
@ -317,6 +377,17 @@ pub fn describe_routes(catalog: &Catalog) -> String {
};
out.push_str(&format!(" POST {p:<30} method {}.{}{mode}\n", name, m.name));
}
// @table storage configuration, when it says anything non-default.
if t.storage_name != *name || !t.indexes.is_empty() {
let mut cfg = format!("table \"{}\"", t.storage_name);
if !t.indexes.is_empty() {
let idx = t.indexes.iter()
.map(|c| c.join("+"))
.collect::<Vec<_>>().join(", ");
cfg.push_str(&format!(", index [{idx}]"));
}
out.push_str(&format!(" {:<30} @table {cfg}\n", name));
}
}
}
out
@ -404,6 +475,7 @@ type Article { id: Id
}
const PRICING: &str = r#"
@table(name: "prices", index: [product])
class Price {
id: Id
product: ref Product
@ -477,4 +549,53 @@ class Product {
let resp = r.dispatch(&req(Method::Get, "/api/products/1/set_price", b""));
assert_eq!(resp.status.0, 405);
}
fn req_q(path: &str, query: &str) -> Request {
Request {
method: Method::Get,
path: path.into(),
query: Some(query.to_string()),
headers: Default::default(),
body: vec![],
keep_alive: true,
}
}
#[test]
fn list_query_filter_is_index_backed() {
let (_ctx, r) = build(PRICING);
r.dispatch(&req(Method::Post, "/api/products", br#"{"sku":"A","name":"A"}"#));
r.dispatch(&req(Method::Post, "/api/products", br#"{"sku":"B","name":"B"}"#));
r.dispatch(&req(Method::Post, "/api/products/1/set_price", br#"{"amount": 100}"#));
r.dispatch(&req(Method::Post, "/api/products/1/set_price", br#"{"amount": 200}"#));
r.dispatch(&req(Method::Post, "/api/products/2/set_price", br#"{"amount": 999}"#));
// Unfiltered list unchanged.
let resp = r.dispatch(&req(Method::Get, "/api/prices", b""));
let all: Vec<serde_json::Value> = serde_json::from_slice(&resp.body).unwrap();
assert_eq!(all.len(), 3);
// ?product=1 → only that product's prices, via the (product) index.
let resp = r.dispatch(&req_q("/api/prices", "product=1"));
assert_eq!(resp.status.0, 200);
let rows: Vec<serde_json::Value> = serde_json::from_slice(&resp.body).unwrap();
assert_eq!(rows.len(), 2);
assert!(rows.iter().all(|r| r["product"] == 1));
// Multiple filters combine (equality AND).
let resp = r.dispatch(&req_q("/api/prices", "product=1&amount=200"));
let rows: Vec<serde_json::Value> = serde_json::from_slice(&resp.body).unwrap();
assert_eq!(rows.len(), 1);
assert_eq!(rows[0]["amount"], 200);
// Unknown field → 400.
let resp = r.dispatch(&req_q("/api/prices", "nope=1"));
assert_eq!(resp.status.0, 400);
// Filtering on a non-indexed stored column falls back to scan.
let resp = r.dispatch(&req_q("/api/prices", "amount=999"));
let rows: Vec<serde_json::Value> = serde_json::from_slice(&resp.body).unwrap();
assert_eq!(rows.len(), 1);
assert_eq!(rows[0]["product"], 2);
}
}

View file

@ -260,7 +260,8 @@ mod tests {
fn catalog() -> Catalog {
Catalog::from_schemas(vec![parse(
r#"type Note { id: Id
r#"@table(index: [title])
type Note { id: Id
title: Text
service rest "/api/notes" expose list, get, create, update, delete }"#,
).unwrap()]).unwrap()
@ -294,6 +295,12 @@ mod tests {
let ids: Vec<i64> = rows.iter().map(|r| r["id"].as_i64().unwrap()).collect();
assert_eq!(ids, vec![1, 5]);
assert_eq!(rows[0]["title"], "a2");
// Replay went through row_insert/row_remove — the secondary index is
// rebuilt: the update moved id 1 from "a" to "a2", the delete cleared
// id 3's entry.
assert_eq!(e.find_by("Note", &[("title".into(), json!("a2"))]).unwrap().len(), 1);
assert!(e.find_by("Note", &[("title".into(), json!("a"))]).unwrap().is_empty());
assert!(e.find_by("Note", &[("title".into(), json!("b"))]).unwrap().is_empty());
// id high-water restored: the next mint must not collide (and must
// keep the shard-0-of-2 stride: odd ids).
e.attach_wal(wal);

View file

@ -33,6 +33,7 @@ pricing/
| --- | --- | --- |
| `class` parses; `/api/products` CRUD serves | **13a ✅ shipped** | lexer/parser/AST + spec amendments |
| `set_price` / `current_price` over RPC (`POST /api/products/:id/set_price`) | **13b ✅ shipped** | method execution, row-scoped txn, one WAL frame per call |
| `@table(name: "prices", index: [product, at])` + indexed DML — `history()` via `select Price{ product == self.id }`, `GET /api/prices?product=1` | **✅ shipped (13 follow-up)** | engine secondary indexes, `find_by`, REST filters |
| `subscribe` / `LIVE select` — delta on every commit, WebSocket at `/api/products/live` | **13c** | subscription registry (scoped Stage 3) |
| `/pricing` screen patches price cells in open browsers | **13d** | SSR + `wo:live`/`wo:bind` client runtime |
| Millions of readers + millions of live recipients | **13e** | scale targets + load harness |

View file

@ -4,7 +4,13 @@
--
-- Prices are append-only history rows: setting a new price inserts a Price,
-- it never mutates an old one. A product's "current price" is the latest row.
--
-- @table configures storage — it never toggles it (every class IS a table):
-- name: the storage/table name for the SQL layer and plans 10–12
-- index: composite secondary index — accelerates `self.prices`,
-- `select Price{ product == ... }`, and GET /api/prices?product=N
@table(name: "prices", index: [product, at])
class Price {
id: Id
product: ref Product -- owning product (foreign key)
@ -17,4 +23,9 @@ class Price {
fn discounted(pct: Int) -> Money {
return self.amount * (100 - pct) / 100;
}
-- Read-only history endpoint; `GET /api/prices?product=1` filters via
-- the (product, at) index above.
service rest "/api/prices"
expose list
}

View file

@ -26,6 +26,12 @@ class Product {
insert Price { product: self.id, amount: amount };
}
-- Schema-layer select (§ Brace Disambiguation): `product == self.id` is a
-- predicate (rides Price's (product, at) index), `amount`/`at` project.
fn history() -> [Price] in txn {
return select Price{ product == self.id, amount, at };
}
-- CRUD works from 13a (classes are storage-identical to types);
-- subscribe goes live in 13c.
service rest "/api/products"

View file

@ -38,7 +38,7 @@ All numbers + find-and-fix stories: [09-concurrency-scaleout.md](09-concurrency-
| Status | Phase | Doc | Notes |
| --- | --- | --- | --- |
| ⬜ | 10 storage foundations | [10](10-storage-foundations.md) | scope reduced: WAL framing/fallocate landed via 09c |
| ⬜ | 10 storage foundations | [10](10-storage-foundations.md) | scope reduced: WAL framing/fallocate landed via 09c; `@table(name:, index:)` surface + RAM secondary indexes landed via 13 follow-up |
| ⬜ | 11 WAL & recovery | [11](11-wal-and-recovery.md) | remaining: snapshots (`.data`), compaction, WAL rotation — replay core shipped in 09c |
| ⬜ | 12 engine disk cutover | [12](12-engine-disk-cutover.md) | mmap arena engine (C phase B is the proving ground) |
@ -48,6 +48,7 @@ All numbers + find-and-fix stories: [09-concurrency-scaleout.md](09-concurrency-
| --- | --- | --- | --- |
| ✅ | 13a class surface | [13](13-class-model-live-pricing.md) | `class` parses, CRUD serves, spec amended |
| ✅ | 13b method execution | [13](13-class-model-live-pricing.md) | `POST /api/<t>/:id/<method>`; row-scoped txn; one `WalRec::Txn` frame per call; abort → 409 rollback |
| ✅ | — `@table` + indexed DML (follow-up) | [13](13-class-model-live-pricing.md) | `@table(name:, index:)`; engine secondary indexes + `find_by`; `select Type{…}` in methods; `?field=value` REST filters |
| ⬜ | 13c LIVE pricing push | [13](13-class-model-live-pricing.md) | Stage 3 scoped: subscription registry, WS at `/api/<t>/live`, replaces the 501 stub |
| ⬜ | Stage 3 wire layer | [../runtime/database/04-client-api.md](../runtime/database/04-client-api.md) | full subscription engine + wire protocol; 13c is its beachhead |
| ⬜ | 13e pricing at scale | [13](13-class-model-live-pricing.md) | wires demo to 09; hot-row reads |

View file

@ -1,6 +1,6 @@
# 10 — Storage Foundations: on-disk row codec + segment append path
> **Kanban: ⬜ not started (scope reduced)** — WAL framing/fallocate/CRC landed early via plan 09c. Board: [00-kanban.md](00-kanban.md)
> **Kanban: ⬜ not started (scope reduced)** — WAL framing/fallocate/CRC landed early via plan 09c; the `@table(name:, index:)` storage-config surface and in-RAM secondary indexes (`Engine::find_by`) landed via the plan-13 follow-up (spec: [`02-wo-language.md § Type-Level Annotations`](../runtime/database/02-wo-language.md)) — this plan inherits the surface and gives indexes their on-disk form. Board: [00-kanban.md](00-kanban.md)
**Context sources:** [`./done/04-cutover-remove-tokio-axum.md`](./done/04-cutover-remove-tokio-axum.md), [`../runtime/database/03-inmemory-engine.md`](../runtime/database/03-inmemory-engine.md), [`../runtime/database/07-wo-seg-migration.md`](../runtime/database/07-wo-seg-migration.md), [`./exploration/postgresql/smgr-and-md.md`](./exploration/postgresql/smgr-and-md.md), [`./exploration/postgresql/page-format.md`](./exploration/postgresql/page-format.md), [`./exploration/linux/12-pwrite-fsync.md`](./exploration/linux/12-pwrite-fsync.md), [`./exploration/linux/09-fallocate.md`](./exploration/linux/09-fallocate.md), [`reference/crates/wo-seg/src/`](../../reference/crates/wo-seg/src/).

View file

@ -30,6 +30,7 @@ This doc is a **master plan** in the style of [`09-concurrency-scaleout.md`](./0
## The class surface (normative example)
```wo
@table(name: "prices", index: [product, at])
class Price {
id: Id
product: ref Product
@ -43,6 +44,8 @@ class Price {
}
}
@table
class Product {
id: Id
sku: SKU @unique
@ -62,6 +65,8 @@ class Product {
}
```
**`@table` — storage configuration, never storage declaration** (✅ shipped, with DML support). Every `type`/`class` IS a table (decisions 3/5 stand); the optional type-level `@table(...)` annotation *configures* it: `name: "prices"` sets the storage/table name (catalog-unique; the surface plans 10–12 and the SQL layer consume — WAL records keep the type name as the stable identifier), and each `index: [product, at]` declares a composite secondary index that the engine maintains on every mutation path (CRUD, method-txn undo, WAL replay) and that accelerates `self.prices` relation reads, the schema-layer `select Price{ product == self.id }` expression in method bodies, and `GET /api/prices?product=1` REST filters — all through one `Engine::find_by` path with scan fallback. Bare `@table` is a legal no-op. Unknown keys (`shard_key`, `retention`) are parse errors until their phases land; unknown annotation *names* skip silently. Spec: [`02-wo-language.md § Type-Level Annotations`](../runtime/database/02-wo-language.md).
## Sub-phase sequence
Each lands as its own numbered plan doc (`13a-…`, `13b-…`) when ready. The blog + ecommerce smoke stays green after every sub-phase.
@ -96,13 +101,13 @@ No new architecture — this sub-phase wires the demo to [`09-concurrency-scaleo
**Exit: verification targets defined and a load harness scripted** (not necessarily met on dev hardware):
| Metric | Target | How measured |
| --- | --- | --- |
| Concurrent readers of one product | 1 M req/s aggregate on 16 cores | `wrk -c 10000` against `GET /api/products/1`, hot-row replicas on |
| Live subscribers receiving one price update | 1 M open sockets, delta delivered p99 < 250 ms | `websocat` fan-out harness, timestamped frames |
| Commit→first-delta latency | p99 < 10 ms | in-process timestamp at commit vs first socket write |
| Memory | ~2 KB/connection + engine working set | RSS under subscriber load |
| Dep count | 1 (`libc`) | `crates/rt/Cargo.toml` unchanged by this phase |
| Metric | Target | How measured |
| ------------------------------------------- | ---------------------------------------------- | ----------------------------------------------------------------- |
| Concurrent readers of one product | 1 M req/s aggregate on 16 cores | `wrk -c 10000` against `GET /api/products/1`, hot-row replicas on |
| Live subscribers receiving one price update | 1 M open sockets, delta delivered p99 < 250 ms | `websocat` fan-out harness, timestamped frames |
| Commit→first-delta latency | p99 < 10 ms | in-process timestamp at commit vs first socket write |
| Memory | ~2 KB/connection + engine working set | RSS under subscriber load |
| Dep count | 1 (`libc`) | `crates/rt/Cargo.toml` unchanged by this phase |
## Non-scope

View file

@ -201,7 +201,28 @@ Rules:
- **No inheritance.** No `extends`, no override, no virtual dispatch. "Is-a" is a tagged union; "has-a" is `ref`/`multi`. This also kills the table-per-class storage-mapping problem: a class IS one table (+ doc/graph parts), exactly like a type.
- **Methods are row-scoped transactional functions.** `fn name(args) -> Ret [in txn [snapshot]]` — the same signature grammar and coordinator as a free-standing `fn` (see `fn checkout` in the ecommerce sample); `self` binds to the receiving row. Bodies are the schema-layer DML of the section above. Exposed as RPC: `POST /api/products/:id/current_price` (plan 13b).
- **Storage and REST are class-blind.** The catalog treats `class` exactly like `type`; converting between them is a no-op for stored data. `self` stays a plain identifier in the lexer (same rule as `subscribe`/`me`).
- **Status:** parsing + CRUD shipped (13a, `ast::TypeDecl::is_class`); method execution shipped (13b, `crates/rt/src/method.rs`) — bodies compile to `ast::{Stmt, Expr}` (`let`, `insert`, `return`, `assert … otherwise abort`, `if/else`) and run on the row's owning shard, committing as one atomic `WalRec::Txn` frame; an abort rolls back completely (HTTP 409). `fn` inside a plain `type` is still parsed-and-discarded.
- **Status:** parsing + CRUD shipped (13a, `ast::TypeDecl::is_class`); method execution shipped (13b, `crates/rt/src/method.rs`) — bodies compile to `ast::{Stmt, Expr}` (`let`, `insert`, `select` expressions, `return`, `assert … otherwise abort`, `if/else`) and run on the row's owning shard, committing as one atomic `WalRec::Txn` frame; an abort rolls back completely (HTTP 409). `fn` inside a plain `type` is still parsed-and-discarded.
### Type-Level Annotations — `@table`
An optional annotation list may precede a `type`/`class` declaration. The first one is `@table` — **storage configuration, never storage declaration**: every `type`/`class` IS a table regardless (see Class Model rule 3 above and plan 13 decisions 3/5); `@table` only configures how.
```wo
@table(name: "prices", index: [product, at], index: [sku])
class Price { ... }
```
| Argument | Meaning |
| --- | --- |
| `name: "…"` | Storage/table name (default: the type name). Catalog-unique, enforced at compile. Consumed by the SQL layer and the storage phases (plans 10–12); the WAL keeps the *type name* as the stable identifier, so renaming a table is replay-safe. |
| `index: [a, b]` | One composite secondary index per entry (repeatable). Columns must be stored scalar columns — scalars, unions, and `ref` FKs; `multi`/`backlink` have no column and are compile errors. |
Rules:
- **Bare `@table` is a legal no-op.** Annotating changes nothing by itself.
- **Indexes are engine-maintained on every mutation path** — CRUD, method-transaction undo, and WAL replay — and serve three DML surfaces through one lookup (`Engine::find_by`, longest-prefix index selection, scan fallback): relation reads (`self.prices`), schema-layer `select Type{ field == expr }` expressions in method bodies, and REST list filters (`GET /api/prices?product=1`; unknown field → 400).
- **Unknown `@table` keys are parse errors** (`shard_key:` and `retention:` are reserved for later phases — no silent passthrough on owned surface). Unknown annotation *names* (`@foo`) skip silently, like unknown field annotations.
- Indexes are per-shard structures over that shard's rows (shared-nothing, plan 09); cross-shard list filters fan out and merge exactly like unfiltered lists.
## Query Layer — Hybrid SQL + Cypher, Fixed Glue

View file

@ -68,6 +68,8 @@ pricing-demo port="8092":
echo "--- current_price (expect 5999):"; curl -s -X POST "$base/api/products/1/current_price"; echo
echo "--- set_price 0 (assert aborts, expect 409):"; curl -s -X POST "$base/api/products/1/set_price" -d '{"amount":0}'; echo
echo "--- current_price unchanged (expect 5999):"; curl -s -X POST "$base/api/products/1/current_price"; echo
echo "--- price history via select (projected amount+at):"; curl -s -X POST "$base/api/products/1/history"; echo
echo "--- indexed REST filter ?product=1:"; curl -s "$base/api/prices?product=1"; echo
echo "--- live (13c pending, expect 501):"; curl -s -o /dev/null -w '%{http_code}\n' "$base/api/products/live"
echo "--- delete 1 (expect 204):"; curl -s -X DELETE "$base/api/products/1" -o /dev/null -w '%{http_code}\n'