Follow-ups from #807: shutdown-drain bytes missing from BytesWritten, no test of the cancel path's budget race

[FumingPower3925]

Follow-ups from the round-1 review of #807 (celeris#760), none blocking; the blocking one (a Shutdown ctx without a deadline got the 250 ms floor) is fixed in #807 itself.

1. Bytes the shutdown drain sends are not counted. drainSends flushes with onLoopThread=true (engine/epoll/loop.go, l.flushWrites(cs, true)), which adds to l.bytesWrittenBatch; that batch is moved into bytesWritten only in run(), which has returned by the time shutdown() runs. So the bytes the drain sends never reach Metrics().BytesWritten. Passing false (the atomic path) would count them.
2. No test covers the cancel path's race. On a cancel of StartWithContext's context the loops start draining before the watcher's Engine.Shutdown stores the budget; the drain re-reads it each 20 ms round, within the 250 ms floor. In TestShutdownSendsTheWholeResponse the handler holds the loop (sync) or asyncWG (async) for 200 ms after the cancel, so the budget is always stored before the drain starts. A drain that read the budget once at its start would pass. A test would release the handler before the watcher can run (or delay the watcher with a hook) and check the drain still extends to the budget.

(The review's third point, TestShutdownSendDrainIsBounded allowing budget + 5 s, is fixed in #807: the limit is budget + 1 s.)

[FumingPower3925]

More follow-ups, from CodeRabbit's review of #807 at 0cefbb0 (both rated Minor) and from merging #807 with main in merge train 9. None blocks #807.

3. An adaptive switch that lands during Shutdown can publish an epoll slot the budget hand-over missed (CodeRabbit, adaptive/engine.go around 965-978). Shutdown snapshots primary/secondary under e.mu and calls each one's Shutdown(ctx) before it cancels Listen. If runEvalLoop has already released switchMu and calls performSwitch after that snapshot, the epoll sub-engine it builds never gets the budget before the cancel. Its drain then gets only the 250 ms floor. It is still bounded, just not extended. CodeRabbit suggests a shuttingDown flag set at the top of Shutdown and checked under e.mu in performSwitch. A test would force a switch into that window.
4. TestShutdownSendsTheWholeResponse orders the client after the drain with fixed sleeps (CodeRabbit, shutdown_send_drain_linux_test.go:79-82). The client starts reading 500 ms after the handler returns, which is 200 ms into the shutdown, and nothing proves the loop has reached drainSends by then. On a slow runner the unfixed code could still be sending when the client starts reading, and the case would pass without exercising the drain. The negative control does fail today (main a73afb6's loop.go/engine.go: 12/18 cases, see fix(epoll): shutdown sends what the sockets have not taken yet before it closes the conns (celeris#760) #807's controls table), so the test is not vacuous on the laptop. A test-only hook in Loop.shutdown (drain entered) would make that ordering deterministic.
5. engine.Engine.Shutdown's doc is stale for epoll (found in the merge with std: a cancel of StartWithContext's context ignores Config.ShutdownTimeout; the drain, the OnShutdown hooks and Start wait for every handler #753's text on main). engine/engine.go says "On epoll and io_uring the drain runs in Listen once Listen's ctx is cancelled, and Shutdown itself does nothing". Since fix(epoll): shutdown sends what the sockets have not taken yet before it closes the conns (celeris#760) #807, epoll's Shutdown hands its ctx to the loops as the send drain's budget. The sentence should say so, as engine/epoll/engine.go's own doc does.

[FumingPower3925]

One more, from CodeRabbit's review of the companion docs PR goceleris/docs#81 (rated Minor). The docs repo has no milestones, so it is tracked here.

6. graceful-shutdown.md: two sentences do not repeat the async HTTP/2 exception (docs#81, src/content/docs/graceful-shutdown.md around 221-223 and 672-675). Step 3 of the shutdown sequence says the workers "wait for their async handlers". The FAQ says a handler that is still running "keeps its connection, and its response still reaches the client". Neither sentence mentions that an HTTP/2 stream on an async route is not waited for, and that on the native engines its response is lost (celeris#759). The same page's "What the drain waits for" section does state the exception, and both sentences predate docs#81. The fix is a qualifier in each sentence that links that section.