feat: add --standalone mode (single node, no Raft)#2
Open
alnr wants to merge 3 commits into
Open
Conversation
Run min(thread-pool-size, cores) h2o event loops instead of one. Each loop has its own SO_REUSEPORT listener (the kernel load-balances accepts), h2o context, message dispatcher and SSL refresh; loop 0 runs on the run() thread. A connection is owned by exactly one loop: it is accepted, read and written only there. Every request is tagged with its owning loop's dispatcher (http_req::res_dispatcher), and all response/request-proceed/deferral messages are delivered through it via the new helpers HttpServer::deliver_stream_response / deliver_request_proceed / deliver_defer_processing. For requests with no owning loop (writes replayed from the log, tests) the helpers mirror the dead-request paths of the corresponding message handlers. The global dispatcher access paths are deleted -- HttpServer:: get_message_dispatcher(), the unused HttpServer::send_message() wrapper and ReplicationState's stashed message_dispatcher -- so no delivery site can bypass the owning loop. That makes multiple loops safe in Raft mode too: the BatchedIndexer apply path, follower->leader forwarding (HttpClient), the SSE proxy and all write-rejection paths now deliver on the request's own loop. Deferred-processing timers are linked on the owning loop (the dispatcher carries its h2o loop), so streamed responses (e.g. /documents/export) no longer stall under backpressure when the request belongs to a loop other than loop 0. Removes the single-event-loop ceiling on HTTP intake and response I/O. Three adjustments follow from requests now being minted on N threads: http_req::start_ts, which doubles as the unique write-request ID in the BatchedIndexer (request map and write-log chunk keys), comes from a global atomic monotonic counter since wall-clock microseconds are no longer unique across accept threads; deliver_defer_processing drives dispatcher-less (replayed) write continuations directly on the worker pool, so chunked handlers such as batched deletes still apply fully on followers and during log replay; and the access log's ofstream is guarded by a mutex now that requests are logged from multiple loops.
Replace the per-loop SO_REUSEPORT listeners with h2o's default model (dup_listener in h2o's src/main.c): bind a single socket and give every event loop its own dup() of it. All loops poll the shared accept queue (level-triggered evloop) and race non-blocking accepts; losing the race yields EAGAIN and is a no-op. Why: - A second process binding the same port fails with EADDRINUSE again. SO_REUSEPORT let a stray second server bind successfully and silently steal a share of the connections. - The shared accept queue is pull-based: idle loops win the race, so load follows capacity. SO_REUSEPORT hash-pins each connection to one loop's private queue at handshake time regardless of that loop's backlog. - Portability: plain SO_REUSEPORT does not load-balance TCP on macOS/BSDs; h2o itself only enables reuseport on Linux and FreeBSD. A loop accepts one connection per wakeup -- deliberately not adopting h2o's 10-accept batch -- so accepts interleave across loops as finely as possible, favoring load spreading over wakeup overhead. Also capture errno before LOG() in the bind and dup failure paths: the LogMessage constructor runs first and can clobber errno, which made bind failures report e.g. ENOENT instead of EADDRINUSE. Review cleanups: internal linkage for the file-private tls_res_dispatcher, and <atomic>/<algorithm> includes in http_data.h for http_req::next_start_ts instead of relying on transitive includes.
In --standalone, writes bypass Raft entirely: ReplicationState::write applies the request directly on a thread pool, serialized per collection by a stripe mutex (different collections run in parallel), instead of node->apply -> single on_apply thread -> BatchedIndexer. This removes the single-threaded Raft commit/apply pipeline that caps write throughput, and also sidesteps the BatchedIndexer RocksDB request round-trip. - config: --standalone / TYPESENSE_STANDALONE / ini server.standalone - durability: with no Raft log to act as the write-ahead log, the document store opens with the RocksDB WAL enabled. Writes are recovered on the next DB open and replayed into the in-memory index by the existing, Raft-independent CollectionManager::load() path, so a process restart rebuilds the index from disk. Note: durability only covers persisted fields; a store:false field is not written to disk and will not survive a restart. - start_standalone(): load collections from the durable store and mark the node ready/caught-up/leader; no brpc peering server is started - is_leader()/node_state() report a healthy leader so reads, /health and the leader-only paths (analytics, conversations) work unchanged; on-demand snapshots/vote/peer operations are rejected (node is null) - chunked (async_req) write bodies aggregate in request->body and the handler runs once with the complete body; non-final chunks send a request-proceed message to the owning event loop so streamed uploads keep flowing, and gzip bodies are inflated in one call once complete - writes are striped by BatchedIndexer::get_collection_name, so a collection's creation and subsequent writes to it serialize in order while writes to different collections run in parallel; unlike the Raft path there is no cross-request ordering for referenced collections - harden the boot path: _populate_referenced_ins parsed each collection-meta row with the throwing nlohmann::json::parse, so a single unparseable row terminated the server on boot (the is_discarded() guard below it was dead code). Use the non-throwing overload so a bad row is logged and skipped, matching the main load loop's fail-closed behavior. - single node only: no replication or consensus in this mode
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
In --standalone, writes bypass Raft entirely: ReplicationState::write
applies the request directly on a thread pool, serialized per collection
by a stripe mutex (different collections run in parallel), instead of
node->apply -> single on_apply thread -> BatchedIndexer. This removes the
single-threaded Raft commit/apply pipeline that caps write throughput,
and also sidesteps the BatchedIndexer RocksDB request round-trip.
document store opens with the RocksDB WAL enabled. Writes are recovered
on the next DB open and replayed into the in-memory index by the
existing, Raft-independent CollectionManager::load() path, so a process
restart rebuilds the index from disk. Note: durability only covers
persisted fields; a store:false field is not written to disk and will
not survive a restart.
the node ready/caught-up/leader; no brpc peering server is started
the leader-only paths (analytics, conversations) work unchanged;
on-demand snapshots/vote/peer operations are rejected (node is null)
handler runs once with the complete body; non-final chunks send a
request-proceed message to the owning event loop so streamed uploads
keep flowing, and gzip bodies are inflated in one call once complete
collection's creation and subsequent writes to it serialize in order
while writes to different collections run in parallel; unlike the Raft
path there is no cross-request ordering for referenced collections
collection-meta row with the throwing nlohmann::json::parse, so a
single unparseable row terminated the server on boot (the
is_discarded() guard below it was dead code). Use the non-throwing
overload so a bad row is logged and skipped, matching the main load
loop's fail-closed behavior.