writeonce/docs/runtime/async.md

Creating an Async Runtime Environment

Goal

Build a minimal async runtime from scratch using Linux kernel primitives. The runtime is not specific to writeonce — it is a general-purpose event loop that any project can use to multiplex I/O without threads, without tokio, and without any external async framework.

The writeonce project uses this runtime (the wo-event and wo-rt crates), but the concepts apply to any server, daemon, or event-driven application on Linux.

What Is a Runtime?

A runtime is the loop that decides what code runs next. In a synchronous program, the OS scheduler picks the next thread. In an async runtime, a single thread asks the kernel: "which of my file descriptors are ready?" — and runs the corresponding handler.

loop {
    ready_fds = ask_kernel_which_fds_are_ready()
    for fd in ready_fds {
        run_handler(fd)
    }
}

That's the entire concept. Everything else — epoll, tokens, interest flags — is implementation detail around this loop.

Explaining It with C

Before Rust, before abstractions, here is a minimal async runtime in C that watches two file descriptors on one thread.

Step 1: Create an epoll instance

#include <sys/epoll.h>
#include <unistd.h>
#include <stdio.h>

int main() {
    // Create the event loop
    int epoll_fd = epoll_create1(0);

    // This single fd is the "runtime" — all other fds register on it
    printf("epoll fd: %d\n", epoll_fd);
}

epoll_create1 returns a file descriptor. This fd is the runtime. Every other fd in the system registers itself on this one fd, and the kernel tracks readiness for all of them.

Step 2: Register file descriptors

#include <sys/epoll.h>
#include <sys/eventfd.h>
#include <sys/timerfd.h>
#include <time.h>

int main() {
    int epoll_fd = epoll_create1(0);

    // Create an eventfd (like a semaphore as a file descriptor)
    int event_fd = eventfd(0, EFD_NONBLOCK);

    // Create a timerfd (fires every 2 seconds)
    int timer_fd = timerfd_create(CLOCK_MONOTONIC, TFD_NONBLOCK);
    struct itimerspec spec = {
        .it_interval = { .tv_sec = 2, .tv_nsec = 0 },
        .it_value    = { .tv_sec = 2, .tv_nsec = 0 }
    };
    timerfd_settime(timer_fd, 0, &spec, NULL);

    // Register both on epoll
    struct epoll_event ev1 = { .events = EPOLLIN, .data.fd = event_fd };
    epoll_ctl(epoll_fd, EPOLL_CTL_ADD, event_fd, &ev1);

    struct epoll_event ev2 = { .events = EPOLLIN, .data.fd = timer_fd };
    epoll_ctl(epoll_fd, EPOLL_CTL_ADD, timer_fd, &ev2);
}

Two different kinds of fd — an event signal and a timer — both registered on the same epoll instance. The kernel will wake us when either is ready.

Step 3: The event loop

#include <sys/epoll.h>
#include <sys/eventfd.h>
#include <sys/timerfd.h>
#include <unistd.h>
#include <stdio.h>
#include <stdint.h>
#include <time.h>

int main() {
    int epoll_fd = epoll_create1(0);

    int event_fd = eventfd(0, EFD_NONBLOCK);
    int timer_fd = timerfd_create(CLOCK_MONOTONIC, TFD_NONBLOCK);

    struct itimerspec spec = {
        .it_interval = { .tv_sec = 2, .tv_nsec = 0 },
        .it_value    = { .tv_sec = 2, .tv_nsec = 0 }
    };
    timerfd_settime(timer_fd, 0, &spec, NULL);

    struct epoll_event ev1 = { .events = EPOLLIN, .data.fd = event_fd };
    epoll_ctl(epoll_fd, EPOLL_CTL_ADD, event_fd, &ev1);

    struct epoll_event ev2 = { .events = EPOLLIN, .data.fd = timer_fd };
    epoll_ctl(epoll_fd, EPOLL_CTL_ADD, timer_fd, &ev2);

    printf("Runtime started. Timer fires every 2s.\n");
    printf("Write to eventfd to trigger it: echo 1 > /proc/%d/fd/%d\n",
           getpid(), event_fd);

    // The event loop — this IS the runtime
    struct epoll_event events[10];
    while (1) {
        int n = epoll_wait(epoll_fd, events, 10, -1);  // block until ready

        for (int i = 0; i < n; i++) {
            int fd = events[i].data.fd;

            if (fd == timer_fd) {
                uint64_t expirations;
                read(timer_fd, &expirations, sizeof(expirations));
                printf("[timer] fired (%lu expirations)\n", expirations);
            }
            else if (fd == event_fd) {
                uint64_t val;
                read(event_fd, &val, sizeof(val));
                printf("[event] signaled (value: %lu)\n", val);
            }
        }
    }

    close(epoll_fd);
    close(event_fd);
    close(timer_fd);
    return 0;
}

Compile and run:

gcc -o runtime runtime.c
./runtime

Output:

Runtime started. Timer fires every 2s.
[timer] fired (1 expirations)
[timer] fired (1 expirations)
[timer] fired (1 expirations)
...

One thread. Two fd types. One loop. The kernel does the scheduling.

Step 4: Add a TCP server to the same loop

#include <sys/epoll.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <unistd.h>
#include <stdio.h>
#include <fcntl.h>
#include <string.h>

int main() {
    int epoll_fd = epoll_create1(0);

    // Create a non-blocking TCP listener
    int listen_fd = socket(AF_INET, SOCK_STREAM | SOCK_NONBLOCK, 0);
    int opt = 1;
    setsockopt(listen_fd, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(opt));

    struct sockaddr_in addr = {
        .sin_family = AF_INET,
        .sin_port = htons(8080),
        .sin_addr.s_addr = INADDR_ANY
    };
    bind(listen_fd, (struct sockaddr*)&addr, sizeof(addr));
    listen(listen_fd, 128);

    // Register listener on epoll
    struct epoll_event ev = { .events = EPOLLIN, .data.fd = listen_fd };
    epoll_ctl(epoll_fd, EPOLL_CTL_ADD, listen_fd, &ev);

    printf("Listening on port 8080\n");

    struct epoll_event events[64];
    while (1) {
        int n = epoll_wait(epoll_fd, events, 64, -1);

        for (int i = 0; i < n; i++) {
            int fd = events[i].data.fd;

            if (fd == listen_fd) {
                // Accept new connection
                int client_fd = accept4(listen_fd, NULL, NULL, SOCK_NONBLOCK);
                if (client_fd >= 0) {
                    struct epoll_event cev = { .events = EPOLLIN, .data.fd = client_fd };
                    epoll_ctl(epoll_fd, EPOLL_CTL_ADD, client_fd, &cev);
                    printf("[accept] client fd=%d\n", client_fd);
                }
            } else {
                // Read from client
                char buf[4096];
                int nbytes = read(fd, buf, sizeof(buf));
                if (nbytes <= 0) {
                    // Client disconnected
                    epoll_ctl(epoll_fd, EPOLL_CTL_DEL, fd, NULL);
                    close(fd);
                    printf("[close] fd=%d\n", fd);
                } else {
                    // Echo response
                    const char *response =
                        "HTTP/1.1 200 OK\r\n"
                        "Content-Length: 13\r\n"
                        "\r\n"
                        "Hello, world!";
                    write(fd, response, strlen(response));
                    epoll_ctl(epoll_fd, EPOLL_CTL_DEL, fd, NULL);
                    close(fd);
                }
            }
        }
    }
}

This is a complete HTTP server — no threads, no framework, no library. One epoll_wait drives accept, read, write, and close for every connection.

From C to Rust: The writeonce Runtime

The writeonce wo-event crate wraps these same syscalls in safe Rust:

C syscall	Rust wrapper	Crate
epoll_create1	EventLoop::new()	wo-event
epoll_ctl(ADD)	EventLoop::register(fd, interest, token)	wo-event
epoll_ctl(MOD)	EventLoop::modify(fd, interest, token)	wo-event
epoll_ctl(DEL)	EventLoop::deregister(fd)	wo-event
epoll_wait	EventLoop::poll(timeout)	wo-event
eventfd	EventFd::new()	wo-event
timerfd_create	TimerFd::new()	wo-event
signalfd	SignalFd::new()	wo-event

The key difference from the C examples: instead of matching on raw fd numbers, the Rust runtime assigns a token (u64) to each fd. The event loop returns tokens, and the runtime dispatches on them:

let events = event_loop.poll(Some(Duration::from_millis(500)))?;

for event in events {
    match event.token {
        TOKEN_WATCHER  => handle_file_change(),
        TOKEN_SIGNAL   => handle_shutdown(),
        TOKEN_TIMER    => handle_periodic_task(),
        TOKEN_LISTENER => handle_new_connection(),
        token if token >= 10000 => handle_http(token),
        _ => {}
    }
}

Why Not tokio?

tokio is a production-grade async runtime. It handles everything — epoll, thread pools, work stealing, timers, I/O drivers. So why build a custom one?

Concern	tokio	Custom runtime
Binary size	Adds ~2-3 MB	Zero — just libc syscalls
Dependencies	50+ transitive crates	1 crate (libc)
Complexity	Work-stealing scheduler, multi-threaded executor	Single-threaded loop, ~200 lines
Control	Opaque — runtime internals hidden behind .await	Every fd, every syscall, every state transition is explicit
Learning	Abstracts the kernel away	Forces understanding of what the kernel actually does

For a content platform serving markdown files, the workload is: accept connection, read request, query in-memory index, render template, write response. This is microseconds of work per request. A single-threaded event loop handles thousands of concurrent connections without the complexity of a multi-threaded executor.

Reusing the Runtime in Other Projects

The wo-event crate has no dependency on writeonce. It provides:

• EventLoop — epoll wrapper with register/deregister/poll
• EventFd — lightweight signaling
• TimerFd — periodic timers as fds
• SignalFd — SIGINT/SIGTERM as fd events
• Event — readable/writable/hangup status
• Token — u64 identifier for dispatch

Any project that needs non-blocking I/O on Linux can use it:

use wo_event::{EventLoop, EventFd, Interest};
use std::time::Duration;

fn main() {
    let eloop = EventLoop::new().unwrap();
    let efd = EventFd::new().unwrap();

    eloop.register(efd.fd(), Interest::Readable, 1).unwrap();

    // Signal from another thread
    std::thread::spawn(move || {
        std::thread::sleep(Duration::from_secs(1));
        efd.write(42).unwrap();
    });

    // Wait for the signal
    let events = eloop.poll(Some(Duration::from_secs(5))).unwrap();
    assert_eq!(events[0].token, 1);
    println!("Event received!");
}

The Mental Model

    ┌─────────────────────────────────┐
    │           your code             │
    │  (handlers, business logic)     │
    └──────────────┬──────────────────┘
                   │ dispatches on token
    ┌──────────────┴──────────────────┐
    │         event loop              │
    │    epoll_wait → Vec<Event>      │
    └──────────────┬──────────────────┘
                   │ registered fds
    ┌──────────────┴──────────────────┐
    │        Linux kernel             │
    │  tracks readiness for all fds   │
    │  inotify, sockets, timers,      │
    │  signals, eventfds — all fds    │
    └─────────────────────────────────┘

The kernel is the scheduler. The event loop is the dispatcher. Your code is the handler. Everything in the system — files, sockets, timers, signals — is a file descriptor. One loop to rule them all.