Skip to content

perf(server): stop idle-client CPU spin (arm EPOLLOUT on demand)#3

Merged
LESdylan merged 1 commit into
Univers42:mainfrom
vjan-nie:testing/audit
Jun 15, 2026
Merged

perf(server): stop idle-client CPU spin (arm EPOLLOUT on demand)#3
LESdylan merged 1 commit into
Univers42:mainfrom
vjan-nie:testing/audit

Conversation

@vjan-nie

Copy link
Copy Markdown
Collaborator

Closes #2


Problem

Each accepted client was registered with EPOLLIN | EPOLLOUT permanently
(Server::acceptClient). Because epoll is level-triggered and a TCP socket is
almost always writable, epoll_wait() returned on every loop iteration even
with no traffic, so Server::run() spun at ~100% CPU for each connected idle
client. The existing modifyEpoll() helper was defined but never called.

Approach

Reconcile write-interest once per loop iteration (chosen over instrumenting
every enqueue site):

  • Register clients with EPOLLIN only.
  • After all handlers, timeouts and extension ticks have run, sweep the clients
    and set each one's interest to EPOLLIN | (hasPendingData() ? EPOLLOUT : 0),
    issuing epoll_ctl only when the interest actually changed (_epollMask).

Why this and not "arm EPOLLOUT at each queueMessage call site": output is
queued from many places (command handlers, channel broadcasts into other
clients' buffers, PING in checkTimeouts, extensions). The sweep covers all of
them without touching any call site, so there is no path that can be forgotten
and leave a client's output stuck. Cost is an O(n) walk over a bounded client
set per iteration, syscall-only-on-change. Delivery latency is unchanged (both
designs flush on the next epoll_wait).

Changes

  • src/Server.cpp, include/Server.hpp: the fix (register EPOLLIN only;
    _epollMask; updateEpollInterest(); the reconcile sweep in run(); mask
    cleanup in disconnectClient).
  • tests/test_eventloop.cpp (+ tests/Makefile): regression test — counts
    event-loop iterations via the IServerExtension::onTick seam (zero
    production code added).
  • CLAUDE.md: corrected a stale note; added the workflow-artifacts policy.
  • .gitignore: trailing-newline fix (separate housekeeping commit).

Verification

Check Before fix After fix
Event-loop iterations in ~1s, 1 idle client 1,810,757 1
EventLoopTest.IdleClientDoesNotSpinEventLoop FAIL PASS
CPU, 1 idle client ~100% ~0%
  • Builds clean: make mandatory and make (full), -Wall -Wextra -Werror -std=c++98.
  • Full suite: 139/139 passed (138 prior + 1 new).
  • Registration replies (001) and channel broadcasts still flush correctly.

The regression test discriminates: it fails on the unfixed tree and passes on
the fixed tree (numbers above).

Process

Followed the audit → plan → adversarial-review workflow. The reviewer returned
APPROVE WITH CHANGES; all points addressed (regression test added, the
.gitignore change split into its own commit, raw verification captured). The
workflow documents (01-audit.md04-review.md) are kept out of git per the
artifacts policy; their findings are distilled above. I can paste the review
verdict as a comment if useful.

What I'd like you to validate

  • Builds and the full suite passes in your environment.
  • You agree with the reconcile-sweep approach over per-call-site arming.
  • Scope is clean: the fix touches only src/Server.cpp and
    include/Server.hpp; everything else is the test, docs, and the
    housekeeping commit.
  • The regression test's threshold/margin looks sound to you.

Client fds were registered with a permanent EPOLLOUT. Since a TCP
socket is almost always writable and epoll is level-triggered,
epoll_wait() returned on every tick even with no traffic, spinning
the event loop at ~100% CPU per connected idle client.

Register clients with EPOLLIN only and reconcile each client's
write-interest once per loop iteration: EPOLLOUT is armed only while
output is queued and dropped once the send buffer drains. epoll_ctl
is issued only when the interest actually changes.

Measured (1 idle client, no traffic): 99.3% -> 0.0% CPU.
Verified registration replies and channel broadcasts still flush.
@LESdylan
LESdylan merged commit 4e78d69 into Univers42:main Jun 15, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Event loop spins at ~100% CPU with a single idle client

2 participants