test(db2-chains): the oracle closes databasev2 11's last criterion — keys vs all across 3×K flattenings and a replay
- test_oracle_all_vs_keys_same_update_sequence continues the shared
sequence 3×WO_DELTA_MAX_HOPS steps, alternating scalar and Text, and
asserts the `resident: all` and `resident: keys` rows equal after EVERY
step; the fold's hop count proves the chain terminated at least twice and
never exceeded K; then the keys log replays into a fresh store and is
compared against the oracle once more — the criterion as written, which
the story carried as ⚠ "an expected value, not an oracle table"
- wal.h: wo_wal_append_row_image's comment claimed the flattened image is
written as WO_WAL_INSERT; it is WO_WAL_UPDATE — an INSERT would replay as
a duplicate id; compaction alone writes INSERT, into a FRESH log — as 11
landed it and its story recorded
- story 11: the criterion flips to ✅ naming the test; the sequencing note
and out-of-scope bullet record task 7's 2026-08-30 measurement (16× vs
105× collapse under a cap, 1.53× faster than swapping) — the work stands
- test_wal 6295 → 6880 pass, 0 fail; 21 runtime suites 0 fail
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
(cherry picked from commit d841390f3087a0c2542ddf23f25f15037a7d4d71)
This commit is contained in:
parent
296efb52ed
commit
7acd085f46
3 changed files with 75 additions and 15 deletions
|
|
@ -513,9 +513,11 @@ int wo_wal_fold_row_at(wo_wal *w, wo_db *db, uint64_t off, uint32_t *class_out,
|
||||||
* rather than one looked up by id. wo_wal_append_insert sources its values via
|
* rather than one looked up by id. wo_wal_append_insert sources its values via
|
||||||
* wo_row_ptr, which is NULL for a keys-resident row whose payload has been
|
* wo_row_ptr, which is NULL for a keys-resident row whose payload has been
|
||||||
* dropped; the update path holds a materialised row and needs to log it as a
|
* dropped; the update path holds a materialised row and needs to log it as a
|
||||||
* chain-terminating record. Written as WO_WAL_INSERT because that is what a
|
* chain-terminating record. Written as WO_WAL_UPDATE, not WO_WAL_INSERT: the
|
||||||
* chain's base must be: it has to replay into a database where nothing
|
* live log already carries the row's insert, so an INSERT here would replay as
|
||||||
* precedes it. */
|
* a duplicate id (corruption). UPDATE replays as remove-then-recreate and the
|
||||||
|
* fold terminates on either full-row kind. (Compaction's own flattening writes
|
||||||
|
* INSERT because it builds a FRESH log — see wal.c.) */
|
||||||
int wo_wal_append_row_image(wo_wal *w, wo_db *db, uint32_t class_id, uint64_t id,
|
int wo_wal_append_row_image(wo_wal *w, wo_db *db, uint32_t class_id, uint64_t id,
|
||||||
const db_row *r);
|
const db_row *r);
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -111,11 +111,14 @@ All verified but one, which is narrowed rather than dropped. Tests live in
|
||||||
- ✅ **Given** the same row, **when** the process restarts, **then** replay is
|
- ✅ **Given** the same row, **when** the process restarts, **then** replay is
|
||||||
correct and its cost does not grow with the total updates ever applied.
|
correct and its cost does not grow with the total updates ever applied.
|
||||||
*`test_delta_chain_flatten_replays`.*
|
*`test_delta_chain_flatten_replays`.*
|
||||||
- ⚠️ **Given** a flattening update, **when** replayed, **then** the row matches
|
- ✅ **Given** a flattening update, **when** replayed, **then** the row matches
|
||||||
the same row in a `resident: all` table under the same update sequence.
|
the same row in a `resident: all` table under the same update sequence.
|
||||||
*Asserted against an expected value, not against a `resident: all` oracle
|
*`test_oracle_all_vs_keys_same_update_sequence` (closed 2026-09-09): the
|
||||||
table. Weaker than written: it catches a wrong value, but it would not catch
|
same sequence runs 3×K further updates, alternating scalar and Text, against
|
||||||
the two modes disagreeing in a way that also fooled the expectation.*
|
a `resident: all` and a `resident: keys` table, asserting equal rows after
|
||||||
|
every step; the fold's hop count proves the chain was terminated at least
|
||||||
|
twice and never exceeded K; then the keys log is replayed into a fresh store
|
||||||
|
and compared against the oracle once more.*
|
||||||
- ✅ **Given** a flattening update to an indexed column, **when** queried through
|
- ✅ **Given** a flattening update to an indexed column, **when** queried through
|
||||||
that index, **then** the row is found by its new value and not its old, before
|
that index, **then** the row is found by its new value and not its old, before
|
||||||
and after a restart. *`test_keys_resident_indexed_across_flatten`, checked at
|
and after a restart. *`test_keys_resident_indexed_across_flatten`, checked at
|
||||||
|
|
@ -143,8 +146,8 @@ compose, and it now pins that.
|
||||||
before.
|
before.
|
||||||
- **A time-based compaction trigger.** Records are durable at commit, so an idle
|
- **A time-based compaction trigger.** Records are durable at commit, so an idle
|
||||||
log does not grow.
|
log does not grow.
|
||||||
- **Whether `resident: keys` earns its place at all.** That is iteration 2's
|
- **Whether `resident: keys` earns its place at all.** That was iteration 2's
|
||||||
task 7, and it should arguably run *before* this work — see below.
|
task 7, measured 2026-08-30 after this landed — see below.
|
||||||
|
|
||||||
## Info — the forks, settled
|
## Info — the forks, settled
|
||||||
|
|
||||||
|
|
@ -165,8 +168,10 @@ compose, and it now pins that.
|
||||||
|
|
||||||
## Sequencing note
|
## Sequencing note
|
||||||
|
|
||||||
This iteration is **ready but arguably should not be next**. Iteration 2's
|
This iteration was written as **ready but arguably not next**: iteration 2's
|
||||||
task 7 has still never measured whether `resident: keys` beats the kernel's own
|
task 7 had not yet measured whether `resident: keys` beats the kernel's own
|
||||||
paging, and everything built on it — including this — assumes it does. If that
|
paging, and everything built on it — including this — assumes it does. The
|
||||||
measurement comes back poorly, this work is optimising something that should be
|
recommended order (measure first) was not followed; task 7 measured on
|
||||||
deleted. Recommended order: measure first, then this.
|
2026-08-30 and the answer is qualified but positive — under a memory cap
|
||||||
|
`resident: keys` collapses 16× where `resident: all` collapses 105×, 1.53×
|
||||||
|
faster than swapping, with a 2.55× smaller resident set — so this work stands.
|
||||||
|
|
|
||||||
|
|
@ -3678,9 +3678,62 @@ static void test_oracle_all_vs_keys_same_update_sequence(void) {
|
||||||
assert_rows_equal(&db_all, id_all, &db_keys, id_keys, &rt, label);
|
assert_rows_equal(&db_all, id_all, &db_keys, id_keys, &rt, label);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/* databasev2 11 closure: the same sequence continued 3×K steps past the
|
||||||
|
six above, alternating scalar and Text, so the keys chain is TERMINATED
|
||||||
|
by a full-row image more than once while the oracle keeps mutating its
|
||||||
|
slab. Equality is re-asserted after every step; the fold's hop count
|
||||||
|
proves the boundary was crossed, not merely approached. */
|
||||||
|
uint32_t peak = 0, resets = 0;
|
||||||
|
for (uint32_t n = 1; n <= WO_DELTA_MAX_HOPS * 3u; n++) {
|
||||||
|
uint32_t field = n & 1u;
|
||||||
|
uint64_t val_all, val_keys;
|
||||||
|
if (field == 0) {
|
||||||
|
val_all = val_keys = 100u + n;
|
||||||
|
} else {
|
||||||
|
char text[16];
|
||||||
|
snprintf(text, sizeof text, "t%u", n);
|
||||||
|
uint32_t tl = (uint32_t)strlen(text);
|
||||||
|
val_all = (uint64_t)(uintptr_t)wo_str_new(&rt, text, tl);
|
||||||
|
val_keys = (uint64_t)(uintptr_t)wo_str_new(&rt, text, tl);
|
||||||
|
}
|
||||||
|
int ek = 0;
|
||||||
|
T_EQ(wo_row_update_field(&db_all, 0, id_all, field, val_all, &msg, &ek), 0);
|
||||||
|
T_EQ(ek, DB_ERR_NONE);
|
||||||
|
chain_update(&db_keys, &w, 0, id_keys, field, val_keys);
|
||||||
|
|
||||||
|
uint32_t hops = 0;
|
||||||
|
uint64_t out[2] = {0, 0};
|
||||||
|
const char *fm = "";
|
||||||
|
uint64_t o1 = wo_row_offset1(&db_keys, 0, id_keys);
|
||||||
|
T_CHECK(o1 != 0);
|
||||||
|
T_EQ(wo_wal_fold_row_at(&w, &db_keys, o1 - 1, NULL, NULL, out, &hops, &fm), 0);
|
||||||
|
for (uint32_t i = 0; i < 2; i++) wo_db_val_free(&db_keys, KEYS_CLASSES[0].kinds[i], out[i]);
|
||||||
|
if (hops > peak) peak = hops;
|
||||||
|
if (hops == 0) resets++;
|
||||||
|
|
||||||
|
char label[32];
|
||||||
|
snprintf(label, sizeof label, "flatten step %u", n);
|
||||||
|
assert_rows_equal(&db_all, id_all, &db_keys, id_keys, &rt, label);
|
||||||
|
}
|
||||||
|
T_CHECK(peak <= WO_DELTA_MAX_HOPS); /* the bound held throughout */
|
||||||
|
T_CHECK(resets >= 2); /* and was actually crossed, twice */
|
||||||
|
|
||||||
|
/* and across a restart: replay the keys log into a fresh store and compare
|
||||||
|
it against the oracle, which never left RAM — the criterion as written */
|
||||||
wo_wal_close(&w);
|
wo_wal_close(&w);
|
||||||
wo_db_destroy(&db_all);
|
|
||||||
wo_db_destroy(&db_keys);
|
wo_db_destroy(&db_keys);
|
||||||
|
wo_db db_keys2;
|
||||||
|
T_EQ(wo_db_init(&db_keys2, KEYS_CLASSES, 1, 0, 1), 0);
|
||||||
|
db_keys2.rt = &rt; rt.wal = NULL; rt.db = &db_keys2;
|
||||||
|
T_CHECK(wo_wal_replay(path, &db_keys2) >= 0);
|
||||||
|
wo_wal w2;
|
||||||
|
T_EQ(wo_wal_open(&w2, path, 1 << 16), 0);
|
||||||
|
rt.wal = &w2;
|
||||||
|
assert_rows_equal(&db_all, id_all, &db_keys2, id_keys, &rt, "after replay");
|
||||||
|
|
||||||
|
wo_wal_close(&w2);
|
||||||
|
wo_db_destroy(&db_all);
|
||||||
|
wo_db_destroy(&db_keys2);
|
||||||
wo_rt_destroy(&rt);
|
wo_rt_destroy(&rt);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue