Compared §F3c-net's forks against gofiber v3's client (fasthttp + Go crypto/tls/x509, .dev/reference/fiber). Two gaps my defaults had vs Go, now locked as decisions 5 and 6: - (5) bounded handshake deadline: the blocking model would let a stalled server hang the shard's one thread indefinitely (the DoS DoTimeout closes). connect_tls now bounds connect+handshake via non-blocking connect+poll + SO_RCVTIMEO/SNDTIMEO, default WO_TLS_HANDSHAKE_MS (10s); expiry traps WO_T_IO. _dl variant + park handshake stay follow-ups - (6) chain hardening: signatures+validity+SAN alone let a leaf act as a CA. Now every non-leaf must assert basicConstraints CA:TRUE (+pathLen) and the leaf must carry EKU serverAuth — what Go's crypto/x509 enforces - acceptance criteria added (stalled-server timeout; leaf-as-CA + no-EKU rejected); connect_tls bullet, frontmatter review_pending, status NEXT PLAN updated to six locked forks Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> (cherry picked from commit 9662cd8b040f417b4886dae9ce97b70083e24e40)
278 lines
20 KiB
Markdown
278 lines
20 KiB
Markdown
---
|
||
track: runtime-v2
|
||
iteration: "9"
|
||
status: in-progress
|
||
readiness: ready
|
||
review_pending: "forks auto-approved 2026-09-08/09 for autonomous execution — developer second review before this ships. Landed + KAT'd (RFC 8448 / real certs): A–E crypto, F1 record, F2 key schedule, F3a messages, F3b offline verify, F3c-core sans-io driver, SAN/hostname, F3c-net chain validation. §F3c-net (socket/VM slice) brainstormed to READY 2026-09-09 with six integration forks locked (blocking connect+handshake then park data I/O; per-shard fd-keyed slot table no-locks; failures trap WO_T_IO; per-shard lazy read-only CA bundle; a bounded handshake deadline WO_TLS_HANDSHAKE_MS; chain hardening = basicConstraints CA:TRUE + EKU serverAuth — the last two added from the gofiber/Go crypto/x509 comparison). Remaining to BUILD: F3c-net (net.connect_tls/read_tls/write_tls, ids 115-117, live-gated), then G inbound server"
|
||
---
|
||
|
||
# runtime-v2 9 — in-process TLS: retiring the proxy-termination doctrine
|
||
|
||
> Created 2026-09-07 from the gap [`jarvis`](../jarvis/00-story.md) surfaces — an
|
||
> assistant must dial an LLM over HTTPS, and the runtime has no outbound TLS. The
|
||
> developer chose the **full overturn**: the runtime gains TLS **both
|
||
> directions**, and the standing "TLS is the proxy's job" doctrine is retired.
|
||
> **Brainstormed to `ready` 2026-09-07**: hand-rolled TLS 1.3, RSA+ECDSA+X.509
|
||
> cert verification, decomposed into the bottom-up phase ladder below. The load-
|
||
> bearing implementation fork is settled — hand-roll, not vendor — with eyes open
|
||
> to the risk (Info).
|
||
|
||
## Why this exists — and what it overturns
|
||
|
||
Three documents record the same standing decision, and this story reverses it:
|
||
|
||
- *"TLS — permanently the proxy's job (framework doctrine)"* —
|
||
[language 34](../language-runtime-database/34-crypto-builtins.md) (crypto
|
||
builtins, line ~83).
|
||
- *"TLS — proxy-terminated, by doctrine, unchanged… the story says so out loud
|
||
rather than implying HTTPS clients"* —
|
||
[language 38](../language-runtime-database/38-content-platform-capabilities.md)
|
||
(which adds `net.connect` as **plaintext** outbound TCP and explicitly refuses
|
||
HTTPS).
|
||
- *"TLS, HTTP/2 | nobody — proxy-terminated by doctrine"* —
|
||
[porch](../porch/00-story.md)'s "what this track does NOT own".
|
||
|
||
The doctrine was reasonable while nothing in-tree needed to *dial* anything: a
|
||
front proxy terminates inbound TLS, and there were no outbound callers. jarvis
|
||
breaks that — its whole job is to reach a remote API — and the developer's
|
||
direct-HTTPS choice for it means the runtime, not a companion, owns the
|
||
connection. Rather than carve out a one-directional exception, the decision is to
|
||
give the runtime TLS in **both** directions: outbound so a `.wo` program can dial
|
||
HTTPS, and inbound so porch can terminate TLS itself instead of mandating a
|
||
proxy in front of every deployment.
|
||
|
||
This is **not a builtin-sized seam** like the rest of this track. TLS 1.3 plus
|
||
X.509 certificate validation is a large, security-critical subsystem — the one
|
||
place the runtime's hand-roll-everything habit (the sha256 precedent) should not
|
||
be assumed to extend. That tension is the load-bearing fork below.
|
||
|
||
## Decisions locked (brainstorm 2026-09-07)
|
||
|
||
1. **Hand-roll TLS 1.3 in C — no vendored library.** The developer chose the
|
||
hand-roll over vendoring mbedTLS/BearSSL, extending the runtime's
|
||
hand-roll-everything habit (the sha256 precedent) to the hardest place it has
|
||
reached. This keeps the pure single-static-binary, zero-external-dependency
|
||
story intact — and it is, stated plainly, the largest and highest-risk
|
||
undertaking in the project. See the risk note in Info; it is not a caveat to
|
||
bury.
|
||
2. **TLS 1.3 only.** No 1.2 legacy — smallest attack surface, one handshake to
|
||
get right.
|
||
3. **Cert verification is full: RSA + ECDSA + X.509.** To reach real endpoints
|
||
(Anthropic, OpenAI and most HTTPS servers present RSA-signed chains), the
|
||
verifier does RSA-PSS and RSA-PKCS#1v1.5 plus ECDSA-P256, over a real
|
||
ASN.1/DER + X.509 chain validator with a system trust store, validity-date and
|
||
hostname (SAN) checks. This is the biggest, most CVE-prone slice, and it is in
|
||
scope because EC-only cannot talk to the APIs jarvis needs.
|
||
4. **Bottom-up, outbound-first.** Build the primitives before the protocol, and
|
||
the client (jarvis's need) before the server (porch's), because the primitives
|
||
are shared and only the role differs.
|
||
|
||
## The phase ladder
|
||
|
||
Each rung is a security-critical slice; C, D and E are each large enough that
|
||
they may split into their own runtime-v2 iterations as they are picked up.
|
||
|
||
| Phase | Delivers | Notes |
|
||
| --- | --- | --- |
|
||
| A — AEAD | AES-128/256-GCM (TLS 1.3 mandates AES-128-GCM) and ChaCha20-Poly1305 | **is runtime-v2 [8](08-symmetric-cipher.md)** — so 8 must include AES-GCM, not only ChaCha; this rung consumes it |
|
||
| B — key schedule | ✅ **LANDED 2026-09-08** — `wo_hkdf_sha256_extract`/`expand` (RFC 5869) + `expand_label` (RFC 8446 §7.1), internal C over `hmac_sha256`; SHA-256 (the mandatory suites' hash; SHA-384 a later add). KAT-gated in `test_crypto.c` (RFC 5869 case 1 + Expand-Label vectors), ASan/UBSan clean. No builtin, no compiler change |
|
||
| C — key exchange | ✅ **LANDED 2026-09-08** — `wo_x25519` (RFC 7748), constant-time Montgomery ladder + mask-based cswap, radix-2⁵¹ field arithmetic (curve25519-donna-c64, `__int128`). Internal C. KAT-gated in `test_crypto.c`: RFC 7748 §5.2 both direct vectors **and the 1000-iteration test**, ASan/UBSan clean |
|
||
| D — signatures | ✅ **LANDED 2026-09-08** — **RSA** `wo_rsa_pkcs1_sha256_verify` + `wo_rsa_pss_sha256_verify` (bignum Montgomery modexp) and **ECDSA-P256** `wo_ecdsa_p256_sha256_verify` (Jacobian point arithmetic, a=-3, on-curve check, Fermat inverses reusing the bignum). Verification is public data so **not** constant-time by design. Both match python vectors (RSA-2048 PKCS1+PSS; P-256), tamper/wrong-hash rejected, KAT-gated, ASan/UBSan clean |
|
||
| E — X.509 | 🔄 **CORE LANDED 2026-09-08** — a defensive ASN.1/DER reader (every length/bound checked, malformation is rejection not over-read) + certificate parse (tbsCertificate span, sig-alg OID, signature, SubjectPublicKeyInfo→RSA n/e or EC P-256 x/y, validity) + `wo_x509_verify_one` (one chain link's signature, dispatching to D's RSA-PKCS1/PSS + ECDSA-P256) + `wo_x509_parse_spki` + `wo_x509_check_validity` (caller supplies the time). KAT-gated in `test_crypto.c` against **real python-generated chains** — RSA CA+leaf (SHA256withRSA) and EC P-256 CA+leaf (ecdsa-with-SHA256): leaf-vs-CA, self-signed CA, wrong-issuer/tampered/truncated rejected, validity window, SPKI extraction — ASan/UBSan clean. **Deferred to F**: SAN/hostname match (needs the target host) and the multi-cert chain walk to a system CA bundle | notoriously bug-prone; consumes D |
|
||
| F — record + handshake (client) | 🔄 **F1–F3b LANDED 2026-09-08** — new `tls.c`/`tls.h`. **F1 record layer** (`wo_tls_record_seal`/`open`, RFC 8446 §5.2, per-record nonce = iv XOR seq, both suites) KAT'd byte-for-byte vs python. **F2 key schedule** (`wo_tls_derive_handshake`/`_application`/`_traffic_keys`/`_finished_verify`, §7.1) KAT'd byte-for-byte vs **RFC 8448 §3**. **F3a message layer** (`wo_tls_parse_server_hello` — attacker input, bounded, rejects HRR/bad suite/truncation; `wo_tls_build_client_hello` — SNI, x25519, sig-algs) KAT'd vs RFC 8448 SH + validated by an independent parser. **F3b offline handshake verification** (`wo_tls_verify_cert_verify` over phase E+D; server + client Finished) — the whole handshake **crypto** proven end-to-end offline vs RFC 8448. **F3c-core sans-io driver** (`wo_tls_client` — pure FSM, caller frames records: CH→SH→flight→Finished, message reassembly, per-message transcript timing, constant-time Finished, application encrypt/decrypt) KAT'd against the **full RFC 8448 record trace** — client Finished + first app record byte-for-byte, NewSessionTicket + server app data decrypt, tampered flight refused. **SAN/hostname** (`wo_x509_check_host`, RFC 6125) + driver enforcement landed. **F3c-net chain validation** (`wo_tls_verify_chain` — chain-link + trust anchor + host + validity, no partial trust) KAT'd offline vs the phase-E RSA + EC chains. **Remaining to build (F3c-net, spec now `ready` — see §F3c-net below)**: `getrandom` ephemeral, per-shard lazy CA-bundle loader, and the `net.connect_tls` / `net.read_tls` / `net.write_tls` builtins (ids 115–117, blocking connect+handshake then park the data plane, per-shard fd-keyed slot table), gated live against `openssl s_server` | jarvis's path; the reason the story exists |
|
||
| G — server (inbound) | the server handshake half, cert+key loading, signing CertificateVerify; porch terminates TLS | retires the inbound proxy requirement, and the doctrine docs |
|
||
|
||
## F3c-net — the socket/VM slice (READY — decisions locked 2026-09-09; forks auto-approved, `review_pending`)
|
||
|
||
Everything security-critical is landed and offline-KAT'd. What is left is I/O
|
||
integration that can only be gated **live** (a local `openssl s_server` / python
|
||
TLS server), so it is one cohesive slice, not further split. The integration
|
||
forks are settled below — the first four grounded in the existing runtime, the
|
||
last two added 2026-09-09 from a comparison against **gofiber v3's client**
|
||
(fasthttp + Go `crypto/tls`/`crypto/x509`, in `.dev/reference/fiber`), which
|
||
bounds every request with a timeout and delegates full chain checks to
|
||
`crypto/x509`. This section is `ready`: the decisions are locked, the acceptance
|
||
criteria are stated, and code may start once a developer signs off the
|
||
`review_pending` marker.
|
||
|
||
### The locked decisions
|
||
|
||
1. **Blocking connect + blocking handshake, then park the data plane.** This
|
||
mirrors `net.connect` exactly (`sysio.c` `WO_B_NET_CONNECT`): the socket is
|
||
**blocking** through TCP connect and the whole TLS handshake, then switched to
|
||
`O_NONBLOCK` once ESTABLISHED. `net.connect`'s own comment already accepts a
|
||
blocking connect ("can stall the shard during the handshake, tolerable while
|
||
connect is rare"); a TLS connection is likewise rare and long-lived (jarvis
|
||
streams a whole conversation over one), so the extra few handshake round-trips
|
||
are the same tolerable stall. Application I/O then **parks the fiber** exactly
|
||
like `net.read`/`net.write` (`O_NONBLOCK` + `park_fd` on POLLIN/POLLOUT +
|
||
retry). A **park-based handshake** is a named follow-up — the same deferral
|
||
`net.connect` made for its `_dl`/park variant, not a v1 requirement.
|
||
2. **Per-shard fd-keyed slot table, no locks.** TLS connection state lives in a
|
||
`wo_tls_conn` slot array **in the shard's own vm**, keyed by fd — the exact
|
||
pattern of `wo_child children[WO_PROC_MAX]` (`vm.h`: "live in the owning
|
||
shard's vm — no locks, one thread"). One pinned OS thread per shard and fds
|
||
that never migrate cross-shard make this thread-safe by construction, with no
|
||
new locking. Each slot holds the `wo_tls_client` (keys, seqs, driver state), a
|
||
**partial-record read buffer** (a record may arrive in fragments over a
|
||
non-blocking socket), and a **leftover-plaintext buffer** (a decrypted record
|
||
larger than the caller's `max`). Capped like `WO_PROC_MAX`.
|
||
3. **Failures trap `WO_T_IO`, loudly.** Every failure of a secure connect —
|
||
DNS, TCP connect, the handshake, and critically the **certificate chain and
|
||
hostname** checks (and any later record auth failure) — returns a `WO_T_IO`
|
||
trap with a descriptive message, mirroring `net.connect`. A secure-connection
|
||
failure is never a silent `nil`; this is the "refuse loudly / no partial
|
||
trust" rule made concrete. `net.read_tls`/`net.write_tls` otherwise mirror
|
||
`net.read`/`net.write` (EOF is the empty Bytes; a partial write resumes via a
|
||
`park_wr_at`-style cursor; a decrypt/auth failure traps).
|
||
4. **Per-shard, lazy, read-only CA bundle.** On the first `net.connect_tls` a
|
||
shard loads the system PEM bundle into its own vm (read-only thereafter) and
|
||
reuses it for every later dial — no cross-shard sharing, no locks, consistent
|
||
with (2). Path: `/etc/ssl/certs/ca-certificates.crt` (confirmed present on the
|
||
dev box), overridable by the `WO_CA_BUNDLE` environment variable — which is
|
||
also how the live gate points the client at its self-signed test CA.
|
||
5. **A bounded handshake deadline (no unbounded shard stall).** The blocking
|
||
model of decision (1) would otherwise let a slow or hostile server stall the
|
||
shard's one thread indefinitely during connect + handshake — the DoS that
|
||
gofiber closes with `DoTimeout`. So `net.connect_tls` bounds the whole
|
||
connect+handshake by a deadline: **non-blocking `connect()` + `poll` for the
|
||
TCP step, and `SO_RCVTIMEO`/`SO_SNDTIMEO` on the blocking socket across the
|
||
handshake**, capping the stall without needing the full park refit. Default
|
||
from `WO_TLS_HANDSHAKE_MS` (10 000 ms if unset); expiry aborts and traps
|
||
`WO_T_IO` ("tls: handshake timeout"). A per-call `_dl` variant and the
|
||
park-based handshake remain the named follow-ups.
|
||
6. **Chain hardening: basicConstraints + EKU (not just signatures).** Signature
|
||
+ validity + SAN is not enough — Go's `crypto/x509` also enforces the
|
||
constraints that stop a leaf from masquerading as a CA. So the phase-E
|
||
extension walk and `wo_tls_verify_chain` gain: every **non-leaf** cert must
|
||
assert `basicConstraints` CA:TRUE and satisfy `pathLenConstraint`, and the
|
||
**leaf** must carry Extended Key Usage `id-kp-serverAuth` (or omit EKU
|
||
entirely). A `keyUsage` `keyCertSign` check on issuers is included where
|
||
present. Failure is a rejection like any other chain fault (no partial trust).
|
||
|
||
### The builtin surface
|
||
|
||
Three new builtins on the `net` module (one numeric id space; `WO_B_MAX` moves
|
||
114 → 117):
|
||
|
||
- `net.connect_tls(host, port) -> Int` — id **115**, arity 2. Blocking TCP
|
||
connect (reusing the `net.connect` DNS/connect path), `getrandom(2)` ephemeral
|
||
X25519 key + ClientHello random/session-id, run the sans-io driver over the
|
||
blocking socket (frame each record: read the 5-byte header, then the body;
|
||
flush `take_output`) to ESTABLISHED, set the host on the driver so the leaf
|
||
SAN is enforced, then `wo_tls_verify_chain` against the lazily-loaded anchors
|
||
(with the decision-6 basicConstraints/EKU checks). The whole connect+handshake
|
||
is bounded by the decision-5 deadline. Returns the fd (a slot is claimed for
|
||
it); traps on any failure.
|
||
- `net.read_tls(fd, max) -> Bytes` — id **116**, arity 2. Reads/decrypts one
|
||
application record via the slot, returning up to `max` plaintext bytes (EOF is
|
||
the empty Bytes), buffering a partial record and parking on POLLIN, and
|
||
draining any leftover plaintext first.
|
||
- `net.write_tls(fd, bytes) -> Int` — id **117**, arity 2. Seals `bytes` into an
|
||
application record and writes it, parking on POLLOUT for a partial write.
|
||
|
||
`net.close` (existing) additionally frees any `wo_tls_conn` slot for the fd.
|
||
VM wiring touches `wob.h` (ids + `WO_B_MAX`), `emit.ml`/`types.ml`
|
||
(registration + return types), `loader.c` (arities), `builtin.c` (sysio dispatch
|
||
range), and `sysio.c` (the implementations + the slot/bundle helpers). No `.wob`
|
||
consumer change beyond the id additions.
|
||
|
||
### Acceptance criteria
|
||
|
||
- **Given** a reachable TLS 1.3 server with a chain to a trusted anchor, **when**
|
||
a `.wo` program calls `net.connect_tls` for its hostname, **then** the
|
||
handshake completes, the chain + hostname validate, and an fd is returned.
|
||
- **Given** that fd, **when** the program `net.write_tls`es a request and
|
||
`net.read_tls`es, **then** it exchanges application data, and `net.close`
|
||
frees the socket and the slot.
|
||
- **Given** a server whose certificate does not chain to a trusted anchor, whose
|
||
SAN does not match the host, or is expired, **when** `net.connect_tls` runs,
|
||
**then** it traps `WO_T_IO` — no connection is returned.
|
||
- **Given** a server that accepts the TCP connection but then stalls (never
|
||
finishing the handshake), **when** the decision-5 deadline elapses,
|
||
**then** `net.connect_tls` aborts and traps `WO_T_IO` rather than stalling the
|
||
shard indefinitely — proven with a stub that connects then sleeps.
|
||
- **Given** a chain whose issuer lacks `basicConstraints` CA:TRUE (a leaf used
|
||
to sign another cert), or a leaf lacking EKU `serverAuth`, **when**
|
||
`net.connect_tls` validates it, **then** it is rejected — with negative KATs
|
||
in `test_tls` alongside the existing chain cases.
|
||
- **Given** two shards each dialing TLS, **when** they run concurrently, **then**
|
||
neither reads the other's slot or bundle (per-shard, no locks), proven under
|
||
ASan/TSan.
|
||
- **Given** the live gate, **when** it runs, **then** it dials a local TLS
|
||
server (trusting a test CA via `WO_CA_BUNDLE`), does a request/response
|
||
round-trip, and refuses each negative (wrong host, untrusted chain, expired).
|
||
|
||
### Out of scope (named, deferred)
|
||
|
||
- **A park-based handshake** — the async refit of decision (1); a first-class
|
||
`TlsConn` language object over the fd — both later, only if measured need or
|
||
the developer prefers them.
|
||
- **The HTTP layer.** `net.connect_tls` is a TLS byte pipe; HTTP/1.1 framing
|
||
over it is the caller's (jarvis 1's `.wo`), not this slice's.
|
||
- **Inbound TLS (server).** Phase **G**, a separate slice for porch.
|
||
|
||
## Consumers
|
||
|
||
Named, so this is not a capability shipped as decoration:
|
||
|
||
- **[jarvis 1](../jarvis/00-story.md)** — outbound HTTPS to the LLM API (the
|
||
reason this story exists).
|
||
- **porch** — inbound TLS termination, retiring the mandatory front proxy for a
|
||
single-binary deployment.
|
||
- **language 38** — the outbound HTTPS half it excluded by doctrine; this story
|
||
is where that exclusion is lifted.
|
||
|
||
## Dependencies
|
||
|
||
- **runtime-v2 [8](08-symmetric-cipher.md)** — the AEAD (phase A). This story
|
||
forces 8 to include **AES-GCM** (TLS 1.3 mandates AES-128-GCM), not ChaCha
|
||
alone — a consequence to record in 8's own fork.
|
||
- **language [34](../language-runtime-database/34-crypto-builtins.md)** —
|
||
SHA-256/HMAC for the key schedule (phase B) and the transcript hash.
|
||
- **`net.connect`** (id 110, **landed 2026-09-07**) — the outbound TCP socket the
|
||
client handshake runs over; the client half sits directly on it.
|
||
|
||
## Out of scope
|
||
|
||
- **HTTP/2.** A separate protocol concern, parked behind language iteration 23
|
||
regardless; TLS is its prerequisite, not its owner.
|
||
- **Mutual TLS / client certificates.** A later slice if a consumer asks; the
|
||
first cut authenticates the server, not the client.
|
||
- **Updating the doctrine documents.** Retiring "TLS is the proxy's job" means
|
||
correcting [language 34](../language-runtime-database/34-crypto-builtins.md),
|
||
[language 38](../language-runtime-database/38-content-platform-capabilities.md)
|
||
and [porch](../porch/00-story.md) when this lands — a follow-up bookkeeping
|
||
pass, named here so it is not forgotten, not part of the runtime work.
|
||
|
||
## Risk and test strategy
|
||
|
||
**This is the highest-risk work in the project, and hand-rolling it raises that
|
||
risk, not lowers it.** Hand-rolled RSA, ECDSA, X25519 and ASN.1/X.509 are the
|
||
classic sources of real-world CVEs (timing side-channels, padding oracles, chain-
|
||
validation bypasses, parser memory bugs). The decision to hand-roll is recorded
|
||
and owned; the mitigations are non-negotiable:
|
||
|
||
- **Constant-time** for every secret-dependent operation (X25519, RSA/ECDSA,
|
||
AEAD) — verified, not assumed.
|
||
- **Reference-tested**: every phase gated against a reference implementation —
|
||
`openssl s_client`/`s_server`, real published cert chains, and the RFC 8448
|
||
TLS 1.3 test vectors — plus an ASan/UBSan leg on the parser and bignum code.
|
||
- **Negative tests as first-class**: an expired cert, a wrong hostname, a broken
|
||
chain, a tampered CertificateVerify and a downgrade attempt must each be
|
||
refused, with a test that fails if they are accepted.
|
||
- **No partial-trust states**: a validation that cannot complete refuses the
|
||
connection; there is no "warn and continue".
|
||
|
||
## Info
|
||
|
||
This is the heaviest iteration in the runtime-v2 track by a wide margin — a
|
||
subsystem, not a builtin-sized seam — and the only one that reverses a project
|
||
doctrine. It is pure I/O-plane and compute work (a handshake layer over the
|
||
existing socket verbs plus the crypto ladder); no actors, so it is not exposed to
|
||
the lang-41 hang. It gates jarvis entirely: until at least phases A–F land,
|
||
jarvis cannot reach a model at all. Realistically it is a multi-phase effort
|
||
measured in weeks, and phases C (X25519), D (signatures/RSA) and E (X.509) may
|
||
each become their own iteration when picked up. Implementation order is the
|
||
ladder, bottom-up: A (via rv2 8) → B → C → D → E → F, with G (inbound server)
|
||
last.
|