nghttp3: a streamed response at the send-retention high-water waits for acks instead of spinning - #277
Merged
Conversation
…or acks instead of spinning At the high-water PumpEgress stops pulling, so a writer's chunk is not taken until acks drain retention - and a writer flushing from the dispatch pass looped on PumpAsync, which outside a drain pumped once and returned at once. The reactor never got back to reading the acks: 0 of 512 one-MiB responses completed under h2load, both reactors at 100% CPU for good. PumpAsync now parks it there, and the capacity signal runs the full drain, which releases parked writers where PumpEgress alone left them waiting.
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A streamed nghttp3 response whose handler flushes from the dispatch pass (outside the streamed drain) spun once the connection reached the 16 MiB send-retention high-water. There
PumpEgressstops taking chunks until acks drain retention, butPumpAsyncoutside the drain pumped once and returned at once, soFlushAsyncasked again - forever, on the reactor thread, which therefore never read the acks that would have ended it. Under h2load (16 conns × 32 streams of 16 × 64 KiB) none of 512 responses completed, and both reactors stayed at 100% CPU after the client had gone.PumpAsyncnow parks the writer when the connection cannot queue more, and the capacity signal runs the full drain - which releases parked writers, wherePumpEgressalone left them waiting.Tests
h3: a streamed response past the send-retention high-water completes(new) - 20 MiB in 64 KiB flushes from the dispatch pass:All suites pass: E2E 232, Unit 60, Chaos 47, Http 44, Tls 151 (the 7 kTLS tests skip without sudo), File 4.
Bench
16 × 64 KiB streamed (Nghttp3Response, 16 conns × 32 streams): main 0 req/s with the reactors spinning; this PR 2,119 req/s.
h2load, 1 KiB, 16 conns × 32 streams, 2 reactors, 4 interleaved rounds with a second build of main as control; within-round median vs main: