writeonce/docs/stories/language-runtime-database/28-skillhost-host-workload.md
shoney.arickathil 1fe808b7a4 docs(stories): add readiness, retire status: refine, sweep all 47 iterations
- `readiness: ready | refine` is a SECOND axis, orthogonal to status.
  `ready` = the brainstorm is complete and the decisions are LOCKED (a spec
  approved, or the forks explicitly confirmed). `refine` = open forks remain
  and it cannot be planned yet
- `status: refine` RETIRED because it carried both meanings at once, so a held
  iteration with an approved spec (language 18, 26) was indistinguishable from
  one nobody had thought about. status is now purely where the WORK is:
  done | in-progress | pending | hold — `pending` was already the board's own
  rendering word, so nothing new was invented
- all 47 iterations classified from EVIDENCE in their own text, not by guess:
  "the four forks are SETTLED" / "spec + plan approved" / "Approved spec:" for
  ready; "Forks the spec must settle" / "no spec exists yet" for refine. Every
  shipped iteration is ready by definition. 19 done, 5 in-progress, 15
  pending, 8 hold; 27 ready, 20 refine
- two iterations moved refine -> in-progress rather than -> pending: language
  31 and 34 are absorbed into 24 and work on them is literally happening, which
  the board already showed as 🔄 while their frontmatter said otherwise. That
  disagreement is now gone
- board legend, board-views' frontmatter contract, and two new Dataview
  queries updated — the useful one being `readiness: ready AND status:
  pending`, the startable set

WHAT THE NEW AXIS IMMEDIATELY SURFACED: of 15 pending iterations, exactly ONE
is startable — databasev2 4, io_uring group-commit, whose forks were confirmed
settled 2026-08-20. Everything else pending needs a brainstorm first. That was
invisible while one key carried both meanings, and it is now on the board.

Also caught by the sweep, unrelated to readiness but found by cross-checking
frontmatter against the board: SIX duplicate rows. Every iteration moved into
databasev2 was still listed in the LANGUAGE pending table under its retired id
(23, 32, 33, 20, 21, 27) as well as its new one. Stale copies removed. And two
databasev2 rows made claims the sweep contradicts — iteration 1 was billed
"startable today" while its forks are open, and 6 still called itself the
ceiling-raiser after 2 took that role.

Docs only. linkcheck 0 broken / 0 anchors.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 16:54:45 +02:00

205 lines
10 KiB
Markdown

---
iteration: "28"
status: hold
readiness: refine
---
# Iteration 28 — skillhost: a host-shaped workload, and the capability gaps it exposes
> Format: `product/story-iteration-template`. Part of
> [Story — one language, one runtime, one database, one binary](00-story.md).
>
> **Inserted 2026-08-16.** A driving-workload iteration, the way iteration 7's
> log-watcher drove the systems stdlib. The workload is a writeonce port of
> **`~/projects/skillhost`** (a C++ MCP host that links `libllama` in-process,
> discovers filesystem skills, and runs their scripts under confinement). The
> point is not the port for its own sake — it is that a *host-shaped* program
> (an MCP tool server that orchestrates subprocesses) exercises runtime
> capabilities no prior sample needed, and this iteration names each gap so it
> becomes schedulable.
>
> **No spec exists yet.** This iteration frames the workload and records the
> gaps (surveyed 2026-08-16 against the running compiler + the skillhost
> source); each gap below is a candidate iteration of its own.
## Goals
- **A writeonce program that is skillhost's shape**: one MCP tool
(`delegate_task`) exposed to a hosted caller, an on-disk skill catalog
discovered at startup, and a confined subprocess runner that executes a
skill's scripts and returns exit code + captured output. Target sample:
`docs/examples/skillhost`.
- **Drive the runtime's missing host capabilities into the open.** Each thing
the port cannot express today is a named gap with a decision attached — the
same discipline log-watcher used to grow `fs`/`proc`/`net`/`time`.
- **Ship the expressible part now, block honestly on the rest.** Much of
skillhost is already writable (below); the port lands in the pragmatic
variant that avoids the hard blockers, and the blockers become their own
iterations.
## What is already expressible (verified 2026-08-16)
- **The skill catalog** — `@table` with the query surface already exceeds
skillhost's in-memory SQLite `skills` table; iteration 27's
`docs/examples/skill-catalog` is literally this table, running. (Or a plain
`map`/`multi` would do — the catalog is a lookup cache, not persistence.)
- **Discovery** — `fs.exists`/`fs.list` (one level) + `fs.read_all` walk
`.agents/*/SKILL.md` exactly as skillhost does one level deep.
- **Frontmatter** — skillhost's own parser is a hand-rolled YAML subset
(`--- … key: value …`); that is plain Text parsing in writeonce, no YAML
library needed.
- **Config** — `env.get` (the `SKILLHOST_*` overrides), `fs.read_all` +
`json.decode` (the config file), `fn main(args)` (flags). The "context
gate" is pure Int arithmetic (`tokens + max_tokens >= n_ctx`); skillhost
does **no** runtime VRAM query, so none is needed.
- **Single-threaded serving** — skillhost handles one request at a time (its
mutexes are defensive only); writeonce's blocking single-thread model maps
cleanly, as log-watcher's MCP mode already proved.
## The gaps (each a candidate iteration)
### Blocker A — in-process native library binding (FFI)
skillhost links `libllama`'s C API in-process (`llama_model_load_from_file`,
`llama_decode`, the sampler chain, GBNF grammar, tokenizer, chat template).
writeonce has **no FFI**, so none of it can be linked or called. The
architecture doc is explicit that in-process linkage is *what forces C++* —
so this gap is the whole reason skillhost is not already portable.
Two directions, and they are a real fork, not a detail:
- **FFI as a language capability** — a way to declare and call C functions
from writeonce. This is a large, doctrine-level addition (the runtime is
libc-only by principle; the one sanctioned exception so far is the vendored
Ed25519, 21). FFI would reopen the dependency-sprawl question the whole
project is built to avoid. Likely its own spec, likely contested.
- **Out-of-process model, no FFI** — drive a llama.cpp binary via `proc`
(`llama-cli`) or `llama-server` over `net` + `json` (it accepts a GBNF
`grammar` param, so grammar-constrained tool calls survive). This needs
**nothing new** and is how a host in any language would do it — the doc
notes `llama-server` "a host in any language could drive." It loses
skillhost's per-turn `llama_memory_clear` + custom sampler control, which
become server request options. **This is the variant the port should use.**
Leaning: build the port on the out-of-process model; record FFI as a separate
iteration to be opened only if a workload genuinely needs in-process linkage,
and expect it to be weighed hard against the libc-only doctrine.
### Blocker B — stdio transport (process stdin/stdout)
skillhost speaks MCP as **newline-delimited JSON-RPC over stdin/stdout**
(`server.serve(std::cin, std::cout)`), which is what a stdio-launching MCP
client expects. writeonce has **no stdin builtin and no stdout-write
builtin** — only `net` sockets and `print`. So a faithful stdio transport is
impossible today.
- The pragmatic port re-hosts MCP on a **TCP socket** (`net.listen/accept/
read/write` + newline framing + `json`), exactly as log-watcher's MCP mode
does — a legitimate but *different* transport than skillhost's stdio.
- The real capability gap: **raw stdin/stdout byte I/O** (read fd 0, write fd
1, flush), so a writeonce program can be a stdio-transport MCP server — the
default an MCP client launches. Small stdlib addition (`io.stdin_read` /
`io.stdout_write`, or an `fd` surface). A candidate iteration.
### Blocker C — bounded, killable subprocesses
skillhost's `run_script` spawns a script, drains stdout+stderr, and — on a
120s deadline — `SIGKILL`s the whole **process group** so backgrounded
grandchildren die too (`poll`-with-deadline + `kill(-pid, SIGKILL)`).
writeonce's `proc.run` captures stdout/stderr/exit but has **no timeout, no
signal control, no process-group kill**, and blocking-only I/O with no
threads means it cannot wait on a child with a wall-clock deadline. A runaway
skill script cannot be bounded — unacceptable for a host that runs untrusted
scripts.
The capability gap: **`proc.run` with a timeout that kills the child (group)
on expiry** — a deadline-bounded subprocess. This composes with the
log-watcher Task-4 stop-flag work (interruptible blocking calls) and is a
clear candidate iteration.
### Partial cluster — filesystem metadata
Smaller gaps, all in `fs`:
- **Recursive directory walk** — skillhost lists a skill's executables/
references recursively; `fs.list` is one level, so this is built by hand
(fine) or `fs` grows a recursive walk.
- **Executable-bit detection** (`access(X_OK)`) — skillhost only offers
runnable scripts to the model; unclear whether `fs.stat` exposes mode bits.
Candidate `fs.stat` extension.
- **Symlink-resolving path confinement** (`weakly_canonical`/`realpath`) —
skillhost resolves symlinks before its prefix check so a link pointing out
of the skill dir is rejected. writeonce has no `realpath`; a string-prefix
check alone is **weaker** (a symlink escape). Candidate `fs` addition —
and a security-relevant one, since confinement is the host's trust boundary.
## Acceptance Criteria
- What to achieve?
- **Given** the expressible subset (catalog + discovery + frontmatter +
config + a TCP-hosted `delegate_task` tool + a subprocess runner),
- **when** `docs/examples/skillhost` is written and compiled,
- **then** it runs as an MCP server over a socket, discovers `.agents/*`
skills, answers `initialize`/`tools/list`/`tools/call`, and runs a
skill's script returning its exit code and captured output — the
skillhost shape, minus the named blockers.
- What to achieve?
- **Given** a skill script that runs forever,
- **when** the runner has no bounded-subprocess capability (Blocker C),
- **then** the port documents that it cannot yet bound it — the gap is
demonstrated, not hidden, and the acceptance records it as the reason
Blocker C's iteration exists.
- What to achieve?
- **Given** each gap iteration lands (bounded subprocess, stdio
transport, fs metadata, and — if ever — FFI),
- **when** the port adopts it,
- **then** the port moves one step closer to skillhost's exact behavior,
and the divergence list in its README shrinks by exactly that gap.
## Out Of Scope
- **In-process `libllama` / CUDA offload** — requires Blocker A (FFI); the
port uses an out-of-process model instead and says so.
- **Runtime VRAM/NVML introspection** — skillhost does none (the context
gate is arithmetic); nothing to build.
- **A resident MCP-over-stdio server** until Blocker B lands — the port uses
a socket transport meanwhile.
- **Reproducing skillhost's exact sampler chain / per-turn memory clear** —
those are in-process libllama specifics; the out-of-process variant
approximates them with server request options.
## Info
The three blockers are independent and each merits its own iteration; the
filesystem partials could ride together as one small `fs`-metadata iteration.
Priority order by leverage:
1. **Bounded subprocess (Blocker C)** — the smallest, most broadly useful, and
a hard requirement for any host that runs untrusted scripts. Do first.
2. **stdio transport (Blocker B)** — small, unblocks writeonce being a
*standard* MCP server (stdio is the default an MCP client launches), useful
far beyond skillhost.
3. **fs metadata (the partials)** — small, security-relevant (symlink
confinement).
4. **FFI (Blocker A)** — largest and most contentious; open only on genuine
demand, and expect the out-of-process variant to make it unnecessary for
this workload.
The port itself is buildable today in the out-of-process / TCP-transport
variant *once Blocker C exists* (a host that cannot bound a runaway script is
not honest to ship) — so the natural first step is Blocker C, then the port,
then B and the partials narrow the gap to skillhost's real behavior.
## Proposed Solution
- **Brainstorm each gap iteration on demand**, starting with the bounded
subprocess (Blocker C) since the port cannot responsibly run scripts
without it.
- **Write `docs/examples/skillhost`** in the out-of-process / socket-transport
variant, with a README whose "divergence from the C++ skillhost" list is
exactly the open gaps (A: out-of-process model, B: socket not stdio, C:
bounded once its iteration lands, plus the fs partials) — the list is the
iteration's own scoreboard.
- Reuse iteration 27's `skill-catalog` as the catalog layer, log-watcher's
MCP mode as the transport skeleton, and the systems stdlib for discovery
and execution.