Server.Shutdown runs OnShutdown hooks and returns before epoll/io_uring drain in-flight requests, breaking its documented order

[FumingPower3925]

Summary

Server.Shutdown's godoc promises that the OnShutdown hooks fire after the engine "stops accepting new connections and drains in-flight requests" (server.go:413-415 at main 9f4d89b). On epoll and io_uring, the two default Linux engines, that is not what happens. Server.Shutdown runs every hook immediately and returns immediately, while a request is still being handled. std and adaptive keep the promise.

Why (main 9f4d89b)

Server.Shutdown (server.go:427-447):

err := eng.Shutdown(ctx)   // epoll / io_uring: a documented no-op
s.cancelListen()           // only SIGNALS Listen's goroutine to drain
s.closeCPUMonitor()
for _, fn := range s.shutdownHooks { fn(ctx) }   // runs now
return err                                       // returns now

• epoll.Engine.Shutdown (engine/epoll/engine.go:209) is return nil. Its comment says graceful shutdown is "driven by context cancellation on Listen's parent context", and that Loop.shutdown closes connections and joins async dispatch.
• iouring.Engine.Shutdown (engine/iouring/engine.go:453) is the same: a no-op behind e.mu, with the drain in the workers on ctx.Done.
• So the drain happens in the Listen goroutine after cancelListen(), and Server.Shutdown never waits for it. On std, Engine.Shutdown is the drain (http.Server.Shutdown), so the order holds there. Adaptive's Shutdown also waits (measured below).

Measurement

A GET /slow handler sleeps 500 ms. The shutdown starts at t=0 while that request is in flight (a handler-entry signal gates it). The container is Linux 7.0.12-linuxkit with --security-opt seccomp=unconfined. Measured at #692's head 5a05c2e; #692 does not change Server.Shutdown, and main's body is identical. One run per arm:

engine	mode	handler finished	hook started	call returned
std	direct Shutdown(5s ctx)	502 ms	540 ms	540 ms
epoll	direct Shutdown(5s ctx)	500 ms	0 s	0 s
io_uring	direct Shutdown(5s ctx)	501 ms	0 s	0 s
adaptive	direct Shutdown(5s ctx)	501 ms	501 ms	501 ms
std	cancel StartWithContext's ctx	503 ms	540 ms	540 ms
epoll	cancel	500 ms	0 s	500 ms
io_uring	cancel	501 ms	0 s	501 ms
adaptive	cancel	500 ms	500 ms	500 ms

On the cancel path #692 makes StartWithContext return only after the drain, but the hook has already run at 0 s. On the direct path, Shutdown itself returns at 0 s.

Impact

• A program that does srv.Shutdown(ctx); os.Exit(0), the usual pattern and the one net/http's semantics teach, can exit while requests are still being served on epoll and io_uring. It was not measured whether the in-flight response still reaches the client in that case.
• A hook that releases a resource (closes a DB pool, flushes and closes a store, flips readiness) runs while handlers still use it.
• The behaviour differs by engine, so the same program is graceful on std and adaptive but not on the default Linux engines.

Fix direction

Server.Shutdown should, after cancelListen(), wait (bounded by ctx) for the engine's Listen to return, which is when epoll's Loop.shutdown and io_uring's Worker.shutdown have drained and joined async dispatch. Only then should it run the hooks. #692 already introduces a listen-done / watcher-done join on the Start*Context path, and the fix should reuse that signal rather than add a second one. Deadlock check (the cancel path runs Server.Shutdown from the watcher): the wait must be on Listen's return, never on Start*Context's return.

Tests: the experiment above as a real test that asserts hook start >= handler done, and call return >= both, on all four engines and both modes. It must fail on this main for epoll and io_uring. As a second control, a mutant that removes the new wait must fail it.

Related

• fix(server): shut down on every context cancel, and return only after it (celeris#673) #692 (adaptive: TestRouteAdaptive_NoReopenerWhenEngineCreationFails flakes on a shared runner (2 ms polls plus a runtime.Stack goroutine census) #673): joins the watcher on the cancel path; the fix builds on it.
• docs: EngineMetrics.Throughput always reads 0; the parser is not SIMD (celeris#653, celeris#424) docs#73 currently documents this per engine (hooks run at once on epoll/io_uring). When this is fixed, those statements go back to one engine-independent rule.
• Found while verifying the docs' graceful-shutdown page for adaptive: TestRouteAdaptive_NoReopenerWhenEngineCreationFails flakes on a shared runner (2 ms polls plus a runtime.Stack goroutine census) #673.

Test used (not committed; run in a copy of the tree)

The test drives each engine with StartWithContext, sends GET /slow over raw TCP, waits for the handler to start, then either cancels the ctx or calls Server.Shutdown(5s). It records when the handler finished, when the OnShutdown hook started, and when the call returned, all relative to t=0. It asserts nothing and only logs the RESULT lines tabulated above.