Inside the Flow-Wing event loop

Flow-Wing's scheduler is cooperative. It can suspend a task and run another one. On its own, though, it knows exactly one reason to wake a task up again: a deadline has passed. That is enough for sys::sleep and nothing else.

Now ask it to wait for a socket. There is no deadline. The only honest answer the bare scheduler can give is "call read() and block" — and a blocked thread is a stopped program, because every task shares that one thread.

task A: read(socket)   ← thread blocked here
task B: cannot run
task C: cannot run
timer:  cannot fire

task A: parked, waiting for bytes
task B: running
task C: running
timer:  fires on time

libuv is what turns "wait for a socket" into something the scheduler can idle on, alongside its timers, in a single kernel call.

The scheduler does not know libuv exists. It holds one function pointer, and libuv fills it in:

/* fw_sched.h */
typedef void (*FWWaitFn)(long long max_wait_ns);
void fw_sched_set_waiter(FWWaitFn fn);   /* NULL = fall back to nanosleep */

That single line is the whole dependency. Link a program that never opens a socket or a file, and no loop is ever created: the scheduler idles in nanosleep and libuv's code is never reached.

static uv_loop_t  g_loop;      /* the loop itself                       */
static uv_async_t g_wake;      /* the one handle another THREAD may touch */
static uv_timer_t g_deadline;  /* caps how long uv_run is allowed to wait */
static int        g_ready = 0; /* has the loop been created yet?        */

There is exactly one loop per process, created the first time anything asks for it. A server, a client and a file read all share it — so a program doing all three still has one thread and one loop.

uv_loop_t *fw_uv_loop(void) {
  if (g_ready) return &g_loop;                /* idempotent */

  if (uv_loop_init(&g_loop) != 0) return NULL;
  uv_async_init(&g_loop, &g_wake, on_wake);
  uv_timer_init(&g_loop, &g_deadline);

  uv_unref((uv_handle_t *)&g_deadline);       /* see below */

  g_ready = 1;
  fw_sched_set_waiter(fw_uv_wait);            /* ← the handshake */
  return &g_loop;
}

static void fw_uv_wait(long long max_wait_ns) {
  if (!g_ready) return;

  if (max_wait_ns >= 0) {
    uint64_t ms = (uint64_t)(max_wait_ns / 1000000LL);
    if (max_wait_ns > 0 && ms == 0) ms = 1;   /* never round a real wait to zero */
    uv_timer_start(&g_deadline, on_deadline, ms, 0);
  }

  uv_run(&g_loop, UV_RUN_ONCE);               /* ← sleeps in the kernel */

  if (max_wait_ns >= 0) uv_timer_stop(&g_deadline);
}

Three different requests share one function:

uv_loop_t  *fw_uv_loop(void);     /* get (creating on first call) the shared loop */
int         fw_uv_ready(void);    /* does the loop exist yet?                    */
void        fw_uv_wake(void);     /* THE cross-thread call — see section 7       */
const char *fw_uv_version(void);  /* uv_version_string()                         */

void fw_sched_park_io(void) {
  FWTask *t = g_current;
  if (t == NULL) return;      /* not in a task: no-op */

  t->wake_at_ns = FW_WAKE_NEVER;   /* = -1 */
  t->shadow_top = fw_gc_shadow_top;
  t->state      = FW_TASK_SUSPENDED;
  queue_push(t);

  swapcontext(&t->ctx, &g_sched_ctx);
}

void fw_sched_wake_io(void) {
  for (size_t i = g_head; i < g_tail; i++) {
    FWTask *t = g_ready[i];
    if (t != NULL && t->wake_at_ns == FW_WAKE_NEVER)
      t->wake_at_ns = 0;      /* runnable now */
  }
}

A parked task stays in the ready queue. It is not moved to a side list. The single field wake_at_ns encodes all three states:

while (g_head < g_tail) {

  /* (A) THROTTLED POLL — tasks ARE runnable, but check for I/O anyway */
  if (g_waiter != NULL && fw_sched_io_waiting() > 0) {
    long long now = now_ns();
    if (now - last_poll_ns >= 1000000LL) {      /* at most once per ms */
      last_poll_ns = now;
      g_waiter(0);                              /* poll, never block */
    }
  }

  FWTask *t = pop_ready();

  if (t == NULL) {
    /* (B) NOTHING CAN RUN — work out what could change that */
    long long     due = earliest_deadline();
    unsigned long io  = fw_sched_io_waiting();

    if (due == 0 && io == 0) break;             /* no timers, no events: done */

    if (g_waiter != NULL) {
      g_waiter(due != 0 ? (due - now_ns()) : FW_WAKE_NEVER);
    } else if (due != 0) {
      block_ns(due - now_ns());                 /* no event layer: nanosleep */
    } else {
      break;                                    /* (C) see below */
    }
    continue;
  }

  /* ... swap to the task's stack ... */
}

task A sleeps 200 ms   →  wake_at_ns = now + 200ms
task B parks on socket →  wake_at_ns = -1
nothing ready

drain:  due = (A's deadline)          io = 1
        g_waiter(due - now)  ==  fw_uv_wait(200_000_000)
              ↓
        uv_timer_start(&g_deadline, on_deadline, 200, 0)
        uv_run(UV_RUN_ONCE)   ←  ONE kernel sleep watching
                                 the socket AND the 200 ms cap
              ↓
   whichever happens first wins:
     socket speaks  → on_read → ... → fw_sched_wake_io() → B runnable
     200 ms elapse  → on_deadline (does nothing) → uv_run returns
                      → pop_ready() finds A due → A runs

There is no polling and no spare thread. One epoll_wait / kqueue / IOCP call covers every timer and every socket in the program at once.

int fw_http_server_listen(int64_t handle, int port) {
  uv_tcp_init(fw_uv_loop(), &s->listener);    /* shared loop */
  s->listener.data = s;

  uv_ip4_addr("0.0.0.0", port, &addr);
  uv_tcp_bind(&s->listener, ..., 0);
  uv_listen((uv_stream_t *)&s->listener, 128, on_connection);
  ...
}

128 is the kernel's backlog: connections the OS accepts and holds for you while your code is busy. A brief burst does not get refused — it queues.

void on_connection(uv_stream_t *server_stream, int status) {
  FwConn *c = new FwConn();
  uv_tcp_init(fw_uv_loop(), &c->handle);
  uv_accept(server_stream, (uv_stream_t *)&c->handle);

  llhttp_init(&c->parser, HTTP_REQUEST, &c->settings);   /* per-connection parser */

  uv_timer_init(fw_uv_loop(), &c->idle);
  uv_unref((uv_handle_t *)&c->idle);   /* must not keep the loop alive alone */
  arm_idle(c);

  uv_read_start((uv_stream_t *)&c->handle, alloc_buffer, on_read);
}

Every connection gets its own llhttp parser and its own idle timer. The parser is a state machine, so a request split across five TCP packets is fine — each on_read feeds bytes in and the parser remembers where it got to.

int64_t fw_http_accept(int64_t handle) {
  FwServer *s = ...;

  for (;;) {
    if (!s->ready.empty()) {            /* a request is waiting: take it */
      FwRequest *r = s->ready.front();
      s->ready.pop_front();
      return (int64_t)r;
    }
    if (!fw_sched_in_task()) {          /* no scheduler to yield to    */
      uv_run(fw_uv_loop(), UV_RUN_ONCE);/* drive the loop directly     */
      continue;
    }
    fw_sched_park_io();                 /* suspend just this task      */
  }
}

All four are enforced before your Flow-Wing code ever sees the request. A client that opens a socket, sends one byte and goes quiet is dropped by its own idle timer without any task waking up for it.

There is no portable way to wait on a regular file with epoll. libuv's answer is a small worker pool: hand it a blocking job, it runs it on a spare thread and calls you back on the loop thread when it is finished.

const char *file_read_all(const char *path) {
  if (fw_sched_in_task() && fw_uv_loop() != nullptr) {
    FwReadJob job;                         /* lives on the TASK's stack */
    job.path     = path;
    job.req.data = &job;

    if (uv_queue_work(fw_uv_loop(), &job.req, read_work, read_done) == 0) {
      while (!job.done) {
        fw_sched_park_io();                /* other tasks run here */
      }
      return alloc_gc_string(job.data);    /* back on the FW thread */
    }
  }

  /* Outside a task there is nothing to switch to: read inline. */
  ...
}

void fw_uv_wake(void) {
  if (g_ready) uv_async_send(&g_wake);
}

static void on_wake(uv_async_t *handle) {   /* runs on the FW thread, in uv_run */
  (void)handle;
  fw_sched_wake_io();
}

uv_async_send is the only function libuv documents as safe to call from a thread other than the loop's own. Everything else in libuv — every uv_read_start, every uv_timer_start — must happen on the loop thread.

The HTTP client in libflowwing_vortex.cpp runs its request on a std::thread (it uses cpp-httplib, which is blocking). Meanwhile the Flow-Wing thread may be fast asleep inside uv_run. When a chunk arrives on the worker thread, something has to reach across and wake the sleeper:

[chunk arrives on worker thread]
  std::lock_guard<std::mutex> lock(ctx->mtx);
  ctx->chunks.push(std::string(data, len));
  ctx->cv.notify_all();
  fw_uv_wake();          /* ← nudge the loop thread */

[Flow-Wing thread, parked in uv_run]
  on_wake  →  fw_sched_wake_io()  →  the reading task becomes runnable
  it re-takes the mutex, sees a chunk, and returns it

Without the wake, the chunk would sit in the queue and the loop would keep sleeping — there is no socket it is watching, because the socket belongs to the other thread.