feat(json): Bool encodes true/false; fraction/exponent decode fails honestly
Closes json's two documented fidelity limits (iteration 5 strictness): - field_class gains WOB_FIELD_BOOL (a plain `Bool` field) and WOB_FIELD_NIL_BOOL (a `?Bool`: WO_NIL_SCALAR nil + bool encoding) — the kind byte alone cannot tell a Bool slot from an Int slot, so the metadata carries it. Emitter writes them (field_class_meta); loader whitelists them; json.c encodes `true`/`false` (and `null` for a ?Bool nil), decode's null/omitted-key pre-write covers NIL_BOOL. - A JSON number with a fraction or exponent is MALFORMED for an Int field: the checked decode (`json.decode(t) as T`) yields nil for the whole document instead of silently truncating 3.7 to 3 — the language has no float, and corrupting data quietly was the one thing a "checked decode" must never do. Floats stay representable through a raw `json.Value` field. - corpus: run/json-bool-fidelity pins the round-trip (true/false both ways, ?Bool null both ways, fraction AND exponent rejected). - Board's two known-gap entries struck; format doc's field_class marker list extended. Verified: oop-e2e 87/0; runtime test + test-iso OK; woc-test 540/0; log-watcher 7/0; employee 8/0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
a43232dc72
commit
f282451867
8 changed files with 77 additions and 29 deletions
|
|
@ -1358,6 +1358,8 @@ let check_field_idx (p : pctx) (f : fstate) (pos : Ast.pos) (v : int) : int =
|
||||||
per-type generated code. *)
|
per-type generated code. *)
|
||||||
let wob_field_json_raw = 0xFFFFFFFE
|
let wob_field_json_raw = 0xFFFFFFFE
|
||||||
let wob_field_nil_scalar = 0xFFFFFFFD
|
let wob_field_nil_scalar = 0xFFFFFFFD
|
||||||
|
let wob_field_bool = 0xFFFFFFFC (* a plain `Bool` field: encode true/false *)
|
||||||
|
let wob_field_nil_bool = 0xFFFFFFFB (* a `?Bool` field: NIL_SCALAR nil + bool encoding *)
|
||||||
|
|
||||||
(* nil for a nullable SCALAR is not the zero word: `0` is a real Int, and the
|
(* nil for a nullable SCALAR is not the zero word: `0` is a real Int, and the
|
||||||
driving workload stores it in a `?Int` (a cron `*` field expands to `0`), so
|
driving workload stores it in a `?Int` (a cron `*` field expands to `0`), so
|
||||||
|
|
@ -1379,10 +1381,14 @@ let is_nullable_scalar (p : pctx) (ty : Ast.field_ty) : bool =
|
||||||
|
|
||||||
let field_class_meta (p : pctx) (ty : Ast.field_ty) : int =
|
let field_class_meta (p : pctx) (ty : Ast.field_ty) : int =
|
||||||
let name_of t = match t with Ast.Scalar n -> Some n | _ -> None in
|
let name_of t = match t with Ast.Scalar n -> Some n | _ -> None in
|
||||||
if is_nullable_scalar p ty then wob_field_nil_scalar
|
if is_nullable_scalar p ty then
|
||||||
|
(match ty with
|
||||||
|
| Ast.Nullable (Ast.Scalar "Bool") -> wob_field_nil_bool
|
||||||
|
| _ -> wob_field_nil_scalar)
|
||||||
else
|
else
|
||||||
match unwrap ty with
|
match unwrap ty with
|
||||||
| Ast.Scalar n when n = Types.json_value_type -> wob_field_json_raw
|
| Ast.Scalar n when n = Types.json_value_type -> wob_field_json_raw
|
||||||
|
| Ast.Scalar "Bool" -> wob_field_bool
|
||||||
| Ast.Scalar n -> ( match class_of_name p n with Some cid -> cid | None -> wob_none)
|
| Ast.Scalar n -> ( match class_of_name p n with Some cid -> cid | None -> wob_none)
|
||||||
| Ast.Multi e | Ast.Map (_, e) -> (
|
| Ast.Multi e | Ast.Map (_, e) -> (
|
||||||
match name_of (Ast.Scalar e) with
|
match name_of (Ast.Scalar e) with
|
||||||
|
|
|
||||||
|
|
@ -254,17 +254,18 @@ recorded, not silently owed:
|
||||||
`for e in parse_dir(dir).entries` keeps the entries alive (good) but leaks
|
`for e in parse_dir(dir).entries` keeps the entries alive (good) but leaks
|
||||||
the `ParseResult` shell (its drop is recorded for no register). Found in the
|
the `ParseResult` shell (its drop is recorded for no register). Found in the
|
||||||
same disassembly; a leak, not a corruption.
|
same disassembly; a leak, not a corruption.
|
||||||
- **json's two documented limits**: a `Bool` field encodes as `0`/`1` (the
|
- ~~json's two documented limits~~ — **closed 2026-08-18** (branch
|
||||||
class-table kind byte does not distinguish it from an integer), and a JSON
|
`json-fidelity`): a `Bool` field encodes `true`/`false` (WOB_FIELD_BOOL /
|
||||||
number with a fraction or exponent decodes by truncation.
|
WOB_FIELD_NIL_BOOL in the field metadata), and a fraction/exponent is
|
||||||
|
malformed for an Int field — the checked decode yields nil instead of
|
||||||
|
truncating (floats stay representable via a raw `json.Value` field).
|
||||||
- **`net` fd lifetime is the program's problem.** `net.close` exists; the
|
- **`net` fd lifetime is the program's problem.** `net.close` exists; the
|
||||||
sample's MCP server never calls it, so a long-running `mcp` session leaks
|
sample's MCP server never calls it, so a long-running `mcp` session leaks
|
||||||
descriptors. That is the sample's bug to fix, not the runtime's.
|
descriptors. That is the sample's bug to fix, not the runtime's.
|
||||||
- **The workload has never run under ASan**, and iteration 4's `gc/held-cycle`
|
- **The workload has never run under ASan**, and iteration 4's `gc/held-cycle`
|
||||||
leak (above) is still open. The corpus itself stays ASan-clean.
|
leak (above) is still open. The corpus itself stays ASan-clean.
|
||||||
- **`json.encode` of a `Bool` and of a nil scalar are asymmetric**: a nullable
|
- ~~`json.encode` Bool/nil-scalar asymmetry~~ — **closed 2026-08-18** with the
|
||||||
scalar encodes as `null` (the field metadata says so), a plain `Bool` still
|
same change: `Bool` encodes `true`/`false`, `?Bool` nil encodes `null`.
|
||||||
encodes as `0`/`1`.
|
|
||||||
- **No corpus fixtures cover the new surface.** By explicit direction
|
- **No corpus fixtures cover the new surface.** By explicit direction
|
||||||
(2026-08-14) the acceptance for this work is the log-watcher program itself,
|
(2026-08-14) the acceptance for this work is the log-watcher program itself,
|
||||||
not fixture pairs; `tests/corpus/` still gates every pre-existing behavior
|
not fixture pairs; `tests/corpus/` still gates every pre-existing behavior
|
||||||
|
|
|
||||||
|
|
@ -18,7 +18,7 @@ All integers little-endian; offsets are absolute file offsets.
|
||||||
**Class table** — per class: name constant index, flags u32 (bit0 = instances are `@gc`), field count, then one kind byte per field padded to a 4-byte boundary, then **three u32 arrays of per-field metadata** (v2), one entry per field each, in declaration order:
|
**Class table** — per class: name constant index, flags u32 (bit0 = instances are `@gc`), field count, then one kind byte per field padded to a 4-byte boundary, then **three u32 arrays of per-field metadata** (v2), one entry per field each, in declaration order:
|
||||||
|
|
||||||
1. `field_names[i]` — constant index of the field's name, or all-ones for "not recorded" (what a hand-built test image writes).
|
1. `field_names[i]` — constant index of the field's name, or all-ones for "not recorded" (what a hand-built test image writes).
|
||||||
2. `field_class[i]` — the class id the field refers to: its own class for an OWNED/GCREF field, its *element's* class for a container of records; `0xFFFFFFFE` marks a `json.Value` field, whose Text holds a raw JSON slice; all-ones for none.
|
2. `field_class[i]` — the class id the field refers to: its own class for an OWNED/GCREF field, its *element's* class for a container of records; `0xFFFFFFFE` marks a `json.Value` field, whose Text holds a raw JSON slice; `0xFFFFFFFD` a nullable scalar (`WO_NIL_SCALAR` nil); `0xFFFFFFFC` a plain `Bool` (json encodes `true`/`false`); `0xFFFFFFFB` a `?Bool` (both); all-ones for none.
|
||||||
3. `field_elem[i]` — a container field's element kinds: a MULTI's element kind, or a MAP's key kind in the low nibble and value kind in the next; 0 otherwise.
|
3. `field_elem[i]` — a container field's element kinds: a MULTI's element kind, or a MAP's key kind in the low nibble and value kind in the next; 0 otherwise.
|
||||||
|
|
||||||
Field kinds: 0 SCALAR, 1 OWNED, 2 GCREF, 3 TEXT, 4 MULTI, 5 MAP. Runtime object layout: 16-byte header then one 8-byte slot per field, in declaration order.
|
Field kinds: 0 SCALAR, 1 OWNED, 2 GCREF, 3 TEXT, 4 MULTI, 5 MAP. Runtime object layout: 16-byte header then one 8-byte slot per field, in declaration order.
|
||||||
|
|
|
||||||
|
|
@ -17,10 +17,12 @@
|
||||||
* `?T` spells nil. Malformed input yields nil, never a trap —
|
* `?T` spells nil. Malformed input yields nil, never a trap —
|
||||||
* that is what makes `json.decode(t) as T` a *checked* decode.
|
* that is what makes `json.decode(t) as T` a *checked* decode.
|
||||||
*
|
*
|
||||||
* Two deliberate, documented limits: a `Bool` field is a WO_K_SCALAR slot
|
* Fidelity (iteration 5 strictness closed both old documented limits): a
|
||||||
* like every other integer, so it encodes as 0/1 rather than false/true (the
|
* `Bool` field carries WOB_FIELD_BOOL / WOB_FIELD_NIL_BOOL in field_class,
|
||||||
* kind byte does not distinguish them); and a JSON number with a fraction or
|
* so it encodes true/false and its `?Bool` nil is null; a JSON number with a
|
||||||
* an exponent decodes by truncation to i64, since the language has no float.
|
* fraction or an exponent is MALFORMED for an Int field (the language has no
|
||||||
|
* float) — the whole decode yields nil instead of silently truncating.
|
||||||
|
* Floats stay representable through a raw `json.Value` field.
|
||||||
* A `json.Value` field (field_class == WOB_FIELD_JSON_RAW) holds the raw JSON
|
* A `json.Value` field (field_class == WOB_FIELD_JSON_RAW) holds the raw JSON
|
||||||
* slice it was decoded from, and encodes back verbatim.
|
* slice it was decoded from, and encodes back verbatim.
|
||||||
*/
|
*/
|
||||||
|
|
@ -126,8 +128,13 @@ static void enc_value(jbuf *b, const wo_module *mod, uint64_t v, uint8_t kind, u
|
||||||
switch (kind) {
|
switch (kind) {
|
||||||
case WO_K_SCALAR:
|
case WO_K_SCALAR:
|
||||||
/* a nullable scalar holding its nil word is JSON null, not a number */
|
/* a nullable scalar holding its nil word is JSON null, not a number */
|
||||||
if (fclass == WOB_FIELD_NIL_SCALAR && v == WO_NIL_SCALAR) jb_put(b, "null", 4);
|
if ((fclass == WOB_FIELD_NIL_SCALAR || fclass == WOB_FIELD_NIL_BOOL) &&
|
||||||
else jb_int(b, (int64_t)v);
|
v == WO_NIL_SCALAR)
|
||||||
|
jb_put(b, "null", 4);
|
||||||
|
else if (fclass == WOB_FIELD_BOOL || fclass == WOB_FIELD_NIL_BOOL)
|
||||||
|
jb_put(b, v ? "true" : "false", v ? 4 : 5);
|
||||||
|
else
|
||||||
|
jb_int(b, (int64_t)v);
|
||||||
return;
|
return;
|
||||||
case WO_K_TEXT: {
|
case WO_K_TEXT: {
|
||||||
const wo_str *s = (const wo_str *)(uintptr_t)v;
|
const wo_str *s = (const wo_str *)(uintptr_t)v;
|
||||||
|
|
@ -323,7 +330,8 @@ static int jparse_object(jp *j, uint32_t class_id, uint64_t *out) {
|
||||||
`0` for a scalar one — so a nullable scalar starts at its own nil word
|
`0` for a scalar one — so a nullable scalar starts at its own nil word
|
||||||
(wob.h's WO_NIL_SCALAR) and stays there if the object omits the key. */
|
(wob.h's WO_NIL_SCALAR) and stays there if the object omits the key. */
|
||||||
for (uint32_t i = 0; i < c->field_cnt; i++)
|
for (uint32_t i = 0; i < c->field_cnt; i++)
|
||||||
if (c->field_class && c->field_class[i] == WOB_FIELD_NIL_SCALAR)
|
if (c->field_class && (c->field_class[i] == WOB_FIELD_NIL_SCALAR ||
|
||||||
|
c->field_class[i] == WOB_FIELD_NIL_BOOL))
|
||||||
fs[i] = WO_NIL_SCALAR;
|
fs[i] = WO_NIL_SCALAR;
|
||||||
jskip_ws(j);
|
jskip_ws(j);
|
||||||
if (j->p >= j->end || *j->p != '{') {
|
if (j->p >= j->end || *j->p != '{') {
|
||||||
|
|
@ -406,7 +414,10 @@ static int jparse_value(jp *j, uint8_t kind, uint32_t fclass, uint32_t felem, ui
|
||||||
}
|
}
|
||||||
char c = *j->p;
|
char c = *j->p;
|
||||||
if (c == 'n') { /* null: this field's own nil word */
|
if (c == 'n') { /* null: this field's own nil word */
|
||||||
uint64_t nilw = fclass == WOB_FIELD_NIL_SCALAR ? WO_NIL_SCALAR : 0;
|
uint64_t nilw =
|
||||||
|
(fclass == WOB_FIELD_NIL_SCALAR || fclass == WOB_FIELD_NIL_BOOL)
|
||||||
|
? WO_NIL_SCALAR
|
||||||
|
: 0;
|
||||||
return jskip_value(j) == 0 ? (*out = nilw, 0) : -1;
|
return jskip_value(j) == 0 ? (*out = nilw, 0) : -1;
|
||||||
}
|
}
|
||||||
if (c == '{') {
|
if (c == '{') {
|
||||||
|
|
@ -534,18 +545,12 @@ static int jparse_value(jp *j, uint8_t kind, uint32_t fclass, uint32_t felem, ui
|
||||||
digits++;
|
digits++;
|
||||||
}
|
}
|
||||||
if (!digits) return -1;
|
if (!digits) return -1;
|
||||||
if (j->p < j->end && (*j->p == '.' || *j->p == 'e' || *j->p == 'E')) {
|
if (j->p < j->end && (*j->p == '.' || *j->p == 'e' || *j->p == 'E'))
|
||||||
/* consume the fraction/exponent; the integer part is the value */
|
/* a fraction or exponent cannot round-trip an i64 slot: MALFORMED
|
||||||
if (*j->p == '.') {
|
* for this language (no float), so the checked decode fails whole
|
||||||
j->p++;
|
* instead of silently truncating. Floats belong in a raw
|
||||||
while (j->p < j->end && *j->p >= '0' && *j->p <= '9') j->p++;
|
* `json.Value` field. */
|
||||||
}
|
return -1;
|
||||||
if (j->p < j->end && (*j->p == 'e' || *j->p == 'E')) {
|
|
||||||
j->p++;
|
|
||||||
if (j->p < j->end && (*j->p == '-' || *j->p == '+')) j->p++;
|
|
||||||
while (j->p < j->end && *j->p >= '0' && *j->p <= '9') j->p++;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
*out = kind == WO_K_SCALAR ? (uint64_t)(neg ? -acc : acc) : 0;
|
*out = kind == WO_K_SCALAR ? (uint64_t)(neg ? -acc : acc) : 0;
|
||||||
return 0;
|
return 0;
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -196,7 +196,7 @@ int wo_load_buf(wo_module *m, const uint8_t *buf, size_t len, char *err,
|
||||||
BAIL("class %u field %u: bad name constant", (unsigned)i, (unsigned)j);
|
BAIL("class %u field %u: bad name constant", (unsigned)i, (unsigned)j);
|
||||||
uint32_t fc = m->metapool[meta_pool + fcnt + j];
|
uint32_t fc = m->metapool[meta_pool + fcnt + j];
|
||||||
if (fc != WOB_NONE && fc != WOB_FIELD_JSON_RAW && fc != WOB_FIELD_NIL_SCALAR &&
|
if (fc != WOB_NONE && fc != WOB_FIELD_JSON_RAW && fc != WOB_FIELD_NIL_SCALAR &&
|
||||||
fc >= kcnt)
|
fc != WOB_FIELD_BOOL && fc != WOB_FIELD_NIL_BOOL && fc >= kcnt)
|
||||||
BAIL("class %u field %u: field class out of range", (unsigned)i, (unsigned)j);
|
BAIL("class %u field %u: field class out of range", (unsigned)i, (unsigned)j);
|
||||||
}
|
}
|
||||||
m->classes[i].name = name;
|
m->classes[i].name = name;
|
||||||
|
|
|
||||||
|
|
@ -49,6 +49,13 @@
|
||||||
* exists so the runtime can tell the two apart where it must write absence
|
* exists so the runtime can tell the two apart where it must write absence
|
||||||
* itself, which today is json.decode leaving an absent key nil. */
|
* itself, which today is json.decode leaving an absent key nil. */
|
||||||
#define WOB_FIELD_NIL_SCALAR 0xFFFFFFFDu
|
#define WOB_FIELD_NIL_SCALAR 0xFFFFFFFDu
|
||||||
|
/* json fidelity (iteration 5 strictness): the kind byte cannot distinguish a
|
||||||
|
* Bool slot from an Int slot, so json.encode used to emit 0/1 for a `Bool` —
|
||||||
|
* invalid for any JSON consumer expecting a boolean. field_class carries the
|
||||||
|
* distinction instead: BOOL marks a plain `Bool` field, NIL_BOOL a `?Bool`
|
||||||
|
* (nil spelled WO_NIL_SCALAR, exactly like NIL_SCALAR, plus bool encoding). */
|
||||||
|
#define WOB_FIELD_BOOL 0xFFFFFFFCu
|
||||||
|
#define WOB_FIELD_NIL_BOOL 0xFFFFFFFBu
|
||||||
|
|
||||||
/* nil for a nullable scalar: -(2^62). Not INT64_MIN, deliberately — the
|
/* nil for a nullable scalar: -(2^62). Not INT64_MIN, deliberately — the
|
||||||
* compiler's own integers are OCaml's 63-bit native ints, so INT64_MIN is not
|
* compiler's own integers are OCaml's 63-bit native ints, so INT64_MIN is not
|
||||||
|
|
|
||||||
4
tests/corpus/run/json-bool-fidelity/fixture.out
Normal file
4
tests/corpus/run/json-bool-fidelity/fixture.out
Normal file
|
|
@ -0,0 +1,4 @@
|
||||||
|
{"on":true,"maybe":null,"n":7}
|
||||||
|
{"on":false,"maybe":true,"n":9}
|
||||||
|
fraction rejected
|
||||||
|
exponent rejected
|
||||||
25
tests/corpus/run/json-bool-fidelity/fixture.wo
Normal file
25
tests/corpus/run/json-bool-fidelity/fixture.wo
Normal file
|
|
@ -0,0 +1,25 @@
|
||||||
|
-- json fidelity (iteration 5 strictness): a Bool field encodes true/false
|
||||||
|
-- (WOB_FIELD_BOOL / WOB_FIELD_NIL_BOOL in field_class — the kind byte alone
|
||||||
|
-- cannot tell Bool from Int), a ?Bool nil is null both ways, and a JSON
|
||||||
|
-- number with a fraction or exponent is MALFORMED for an Int field: the
|
||||||
|
-- checked decode yields nil instead of silently truncating.
|
||||||
|
use json
|
||||||
|
|
||||||
|
class Flags {
|
||||||
|
on: Bool
|
||||||
|
maybe: ?Bool
|
||||||
|
n: Int
|
||||||
|
}
|
||||||
|
|
||||||
|
fn main() -> Int {
|
||||||
|
print(json.encode(Flags { on: true, maybe: nil, n: 7 }));
|
||||||
|
let back = json.decode("{\"on\":false,\"maybe\":true,\"n\":9}") as Flags;
|
||||||
|
if back != nil {
|
||||||
|
print(json.encode(Flags { on: back.on, maybe: back.maybe, n: back.n }));
|
||||||
|
}
|
||||||
|
let frac = json.decode("{\"on\":true,\"maybe\":null,\"n\":3.7}") as Flags;
|
||||||
|
if frac == nil { print("fraction rejected"); }
|
||||||
|
let expo = json.decode("{\"on\":true,\"maybe\":null,\"n\":2e3}") as Flags;
|
||||||
|
if expo == nil { print("exponent rejected"); }
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
Loading…
Reference in a new issue