Serv

Partial implementation of HTTP/1.1 in C++98 for linux or macOS. Minimal config DSL.

Problem

Serve static and dynamic data (via CGI, see Common Gateway Interface RFC 3875). Multiple concurrent chunked -- and multipart, but its not the focus -- transfers, rife with slow and/or impatient clients, partial messages, fd exhaustion, and fairness across requests.

Architecture

Streaming everything with explicit FSMs. The dependencies between CgiSessionHandler and ClientHandler made simple event-based functions brittle. By switching to an Unreal style tick approach (poll rounds instead of frames?) -- where poll event registration/de-registrations are consolidated around a 2d-array transition table as opposed to driven by event callbacks -- errors were GREATLY reduced during development.

Polling is done with kqueue() or epoll(). A socket read EOF is interpreted as a peer close; no half-close support.

Note

Epoll spins on HUPs! The docs don't tell you this!!

States

╔════════════════════════════════════════════════════════════════════════╗
║                          ClientHandler FSM                             ║
╚════════════════════════════════════════════════════════════════════════╝

    IDLE ──────> PROCESSING
     |               |
     |         (READING_HEADERS) ─────> (READING_BODY) ───> (RESPONSE QUEUED)
     │               │                         │                    │
     │               └────> ERROR  <───────────┘                    │
     │                        │                                     │
     |                    RESETTING ◄───────────────────────────────┘
     |                        │
     |                        ▼
     └──────────────────> CLOSED / IDLE

  IDLE               : Socket accepted, reads registered, awaiting first byte
  PROCESSING         : Read request, dispatch to resource handler, queue response
  (READING_HEADERS)  : Accumulating request line + headers
  (READING_BODY)     : POST body ingestion (streaming)
  RESETTING          : Sending response, async session cleanup, reset state
  CLOSE              : Request connection closure


╔════════════════════════════════════════════════════════════════════════╗
║                       AsyncHandler (CGI) FSM                           ║
╚════════════════════════════════════════════════════════════════════════╝

    INIT ──> WRITING_BODY <──> THROTTLED <───> READING_OUTPUT ──> DONE
                │                 │                 │
                |                 ▼                 |
                └─────────────> ERROR <─────────────┘
                                               

  INIT            : Fork + pipe setup, child exec'd
  WRITING_BODY    : Feeding request body to CGI process
  READING_OUTPUT  : Buffering script output (CGI headers + body)
  DONE            : Script exited 0, response ready for ClientHandler
  ERROR           : Script crashed / pipe HUP, CGI parse error


╔════════════════════════════════════════════════════════════════════════╗
║                        Ownership & Interaction                         ║
╚════════════════════════════════════════════════════════════════════════╝

  ClientHandler                         AsyncHandler
       │                                       │
       │  creates & holds ptr / onClientData() │
       ├──────────────────────────────────────>│
       │                                       │
       │  onTimeout() / cancel()               │
       ├──────────────────────────────────────>│
       │                                       │
       │    produceOutput(): signals _haveresp │
       │◄──────────────────────────────────────│
       │  ClientHandler -> RESETTING           │
       │                                       │
       └─> destroys AsyncHandler               │
           ClientHandler -> IDLE/CLOSED        X

Performance

Survives seige benchmark mode with GET/POST at maxxed concurrency without dropping a single request, unless the somaxxconn file descriptor backlog limit is exceeded. The OG fork()'d CGI keeps things quite slow (hey, the goal was correctness). This should look like ~20K RPS using trivial message bodies, <1024 connections, on a basic linux box. Config file samples are provided.

Building it

Requires: g++ or clang++ with C++98 support, make, linux or macOS.

git clone <repo>
cd serv
make
./serv conf/siegeconf

Running it

Config vars: config.hpp has some policy macros, TCP_NODELAY for more large xfer speed, TMPFILE cleanup on start, besides fairness caps and buffer sizes.

Makefile flags: PROXY_MODE drains message bodies if an error arises (so far only that the request is too long) to preserve the TCP connection, useful were this ever reverse proxied. LOGLVL with 7 settings. EDGET for edge-triggered epoll() in linux only, for theoretical performance gain at high concurrency and slow clients.

CLI args: config file is necessary. -d for daemon mode.

Signals: Ctrl+c SIGINT kills it.

Other Makefile targets

• You can make bench for siege: ./serv conf/siegeconf
• You can make test for ~50 pytests that are dependent on config values: ./serv conf/pyconf
• You can make -j if you're impatient, but not advised for a tiny VM

Config DSL

See samples in the conf/ directory.

listen : the IP/port to bind for a server

root : filesys directory for serving files

methods : list the http methods allowed in a location block

location : defines a URI location block between {...}

client_max_body_size : max allowed request body size

server : defines a virtual server block between {...}

server_name : hostname for virtual hosting

error_page : custom error pages (e.g. '404 /404.html')

autoindex : "on" for generic directory listing

index : specifies default index file

upload_store : dir where uploads are saved

cgi_extension : file extensions for scripts

redirect : return a redirect

cgi : defines a global cgi override block between {...}

cgi_root : base dir for CGI scripts

alias : maps URL path to a different fsys path (ignores URI)

Contributors

• Equinox201: initial CGI module commits.

Known Limitations

• Thread pool (FastCGI; avoid fork() latency)
• O(n) timeout loop vs O(1) timing wheel
• Zero-copy handoffs (CGIParser::produceOutput() serializes pipe reader queue into std::string)
• io_uring for threaded polling
• UTF-8