- README: shipped concurrency/HTTP/WebSockets sat in the roadmap as "not yet available"; "no package manager" contradicted [deps]; the deps example would not have compiled (the key IS the module name) - runtime/README: leads with wovm, wo-rt.c demoted to a historical section; dropped 2 nonexistent recipes, crates/rt, @gc refcounting, 13 suites -> 18 - employee + log-watcher READMEs claimed "does not compile"; both are gates - error catalog: +10 emitted codes incl WO-E250, the only diagnostic the shipped query surface raises; recorded why the sweep rotted - language-surface: group-by parses, then the typechecker refuses it - 00-code-review + 00-link-audit re-run; history kept, not rewritten - 48 dead Rust-era exploration links de-linked rather than re-pointed (their prose names the retired plan by number); successor map -> discarded.md - 08-project-structure: compiler/plan/ never existed; corpus has 9 dirs, 5 empty - releasing.md: dropped a --draft step the workflow never had - new docs/00-doc-audit.md: findings + disposition, incl one row where the audit was wrong and the doc it accused was right - status folders removed: 34 stories flat, status only in frontmatter; 252 links recomputed from resolved paths; board/board-views/structure retaught - story 24 -> in-progress, since frontmatter is now the only truth - new iteration 38: fs mutation verbs + net.connect, the two capability families no iteration owned - new iteration 39: gofiber/fiber v3.5.0 parity study. The ledger called CSRF/sessions unblocked by iteration 34's HMAC, but the runtime has no source of randomness at all - linkcheck skips .dev/.superpowers: 0 broken paths, 0 bad anchors Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
78 lines
3.9 KiB
Markdown
78 lines
3.9 KiB
Markdown
# 06 — `sendfile`
|
|
|
|
Zero-copy transfer from a file fd to a socket fd. The kernel splices pages directly from the page cache into the socket's send buffer — userspace never touches the bytes. One syscall, one copy (DMA → NIC), no userspace buffer.
|
|
|
|
## Kernel source
|
|
|
|
| Path | What |
|
|
| --- | --- |
|
|
| [`reference/linux/fs/read_write.c`](../../../../.dev/reference/linux/fs/read_write.c) | `SYSCALL_DEFINE4(sendfile, ...)` and `SYSCALL_DEFINE4(sendfile64, ...)`. Modern glibc aliases the first to the second; the syscalls are distinguished by the offset type. |
|
|
| [`reference/linux/fs/splice.c`](../../../../.dev/reference/linux/fs/splice.c) | Internally `sendfile` delegates to `splice_direct_to_actor`. Related — see `07-splice.md` if you ever need the more general fd-to-fd pipe path. |
|
|
|
|
## Man pages
|
|
|
|
`man 2 sendfile`.
|
|
|
|
## Rust FFI via `libc`
|
|
|
|
```rust
|
|
use libc::{sendfile, off_t};
|
|
// sendfile64 is the same syscall on 64-bit Linux; libc::sendfile already uses it
|
|
```
|
|
|
|
## Direct-syscall example
|
|
|
|
```rust
|
|
unsafe {
|
|
// out_fd must be a socket; in_fd must be a regular file opened O_RDONLY.
|
|
// Pass NULL for the offset pointer to advance the file's own file offset;
|
|
// pass a &mut offset to keep the file position untouched and step through explicitly.
|
|
let mut sent: isize = 0;
|
|
let mut remaining = file_len;
|
|
let mut offset: off_t = 0;
|
|
|
|
while remaining > 0 {
|
|
let n = libc::sendfile(sock_fd, file_fd, &mut offset as *mut _, remaining as usize);
|
|
if n < 0 {
|
|
let e = io::Error::last_os_error();
|
|
match e.raw_os_error() {
|
|
Some(libc::EAGAIN) | Some(libc::EWOULDBLOCK) => {
|
|
// re-arm EPOLLOUT on sock_fd and yield back to the loop;
|
|
// resume this call when the loop wakes us
|
|
return Err(e);
|
|
}
|
|
_ => return Err(e),
|
|
}
|
|
}
|
|
sent += n;
|
|
remaining -= n as u64;
|
|
if n == 0 { break; } // peer closed
|
|
}
|
|
}
|
|
```
|
|
|
|
## Key behaviour
|
|
|
|
| Detail | Notes |
|
|
| --- | --- |
|
|
| Input fd | Must support `mmap`-like access — regular files, shared memory, some block devices. **Not** sockets, pipes, or character devices. |
|
|
| Output fd | Must be a socket (kernel 2.6.33+ lifted the restriction to anything, but practically: sockets). |
|
|
| Max per-call | Kernel caps at ~2 GB per syscall regardless of what you request. Loop for bigger files. |
|
|
| `offset` pointer | If NULL, updates the input fd's internal offset (like `read` does). If non-NULL, updates only the pointed-to variable. **Always use non-NULL** when sharing the fd across concurrent readers. |
|
|
| Return | Bytes sent (possibly less than requested — partial send; re-arm `EPOLLOUT` and resume). |
|
|
|
|
## Gotchas
|
|
|
|
- **`EAGAIN` is the common partial-write case.** The socket's send buffer filled; register `EPOLLOUT`, drain the CQ when the loop fires, keep calling `sendfile` with the updated `offset`.
|
|
- **TLS and `sendfile` don't mix** (without kTLS). Encryption requires a userspace copy. If / when TLS lands, use kTLS (`setsockopt(TCP_ULP, "tls")`, kernel 4.13+, AES-GCM only in practice).
|
|
- **Chunked transfer encoding and `sendfile` also don't mix** — the chunk framing has to go around the payload. Send headers with `write`, then `sendfile` the body, then write the trailing zero-length chunk.
|
|
- **Compression can't happen on the wire** — the kernel doesn't gzip. Pre-compressed sibling files (`.gz`, `.br`) + `Content-Encoding` header is the idiom.
|
|
- **`fstat` before `sendfile`** to get the file length for `Content-Length` — spares the client from guessing when the body ends.
|
|
|
|
## Used by
|
|
|
|
`08-sendfile-static-assets.md` — the `GET /static/...` handler. Future `##ui` SSR output bundles go through the same path.
|
|
|
|
## v1 port source
|
|
|
|
[`reference/crates/wo-serve/src/sendfile.rs`](../../../../.dev/reference/crates/wo-serve/src/sendfile.rs) (109 LOC) — `send_file(sock, path) -> Result` wrapping the loop + `EAGAIN` handling.
|