You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Follow-ups from #773: an AsyncHandlers arm after #774, io_uring multishot strings read before Hijack, the copy on std, the engine-side cost, doc and witness nits #785
The review of #773 (celeris#733) left these minor findings and nits. None of them blocks #773, whose failing-first result, mutants and suites the review reproduced. Each item says what to do.
1. #773 and #774 depend on each other, and no committed test covers the path they share (minor)
#773's epoll fix covers hijackConn's inline branch, which is sync mode and an async-mode conn not yet promoted. With Config{AsyncHandlers: true} and no Async route, the handler runs inline on an async loop: on #773 alone that path still crashes (#769, fixed by #774), which is why #773's test marks its async arm's route Async instead. Once both merge, nothing committed pins #733 on the AsyncHandlers-inline path.
The review's probe (TestHijackKeepsRequestViews' rig with Config{Engine: Epoll, AsyncHandlers: true} and no Async route; kept in the maintainer's evidence tree as evidence/lanes-20260927/EPOLL-HIJACK/followups/review-probes/zz_review733_asynchandlers_test.go) gave, per the review: main + #773 + #774 + #776rounds=20 wrong=0 PASS 3/3; #774's head alone wrong 15 and 16 of 20 (B's Authorization bytes), FAIL 2/2; #773's head alone a nil-pointer panic at engine/epoll/loop.go:1398 (#769).
To do: merge #774 before #773 (or together), then add a Config{AsyncHandlers: true} arm, with no Async route, to TestHijackKeepsRequestViews, and to CI's exact tally step (arm PASS count).
2. io_uring's opt-in multishot mode still leaks strings read before Hijack (minor)
In multishot receive mode (CELERIS_IOURING_MULTISHOT_RECV=1) the request lives in a provided-ring buffer that goes back to the kernel when the handler returns (engine/iouring/worker.go: PushBuffer before the ErrHijacked return after ProcessH1). #773 makes strings read afterHijack safe (the Context copies them) and documents that strings read before it must be cloned (context_response.go:1266-1269 at 6842891). The most natural usage, read the path and headers, then hijack and hand them to the serving goroutine, still reads other connections' bytes there: #773's own multishot test shows it as its witness (before-Hijack "TTP/1.1\r|ETSECRETSECRET|/w HTTP/1.1\r"). It is a documented limitation, not a fix.
To do: either keep the buffer for a hijacked conn (give the ring a fresh replacement entry instead of pushing the request's buffer back), or make Hijack's doc lead with the read-after-Hijack pattern. Multishot is opt-in, so this is a limitation to close or accept explicitly, not a regression.
3. Context.Hijack copies on std too, where the values are already copies (minor)
context_response.go:1280 calls c.cloneRequestValues() before h.Hijack(c.stream) with no engine check. middleware/websocket upgrades through c.Hijack() on std (websocket.go:291, hijackUpgrade), so every std WebSocket upgrade now pays the copy for nothing: the mock-engine benchmark measured 257.6 ns → 1810 ns, 0 → 994 B, 0 → 37 allocs per Hijack. It was never measured in the std upgrade shape. (#773's body said "Only Hijack pays, never an ordinary request: the request path is unchanged"; an upgrade is a per-connection request path, and the body now says so.)
To do: measure a std WebSocket upgrade before and after; if it matters, skip the copy when the engine serves copies already (std), e.g. a capability the std ResponseWriter reports.
4. The engine-side cost is stated, not measured (minor)
After a hijack, epoll gives up one receive buffer (cs.buf = nil before releaseConnState(cs); acquireConnState allocates a new one when cap(cs.buf) < bufSize, conn.go:280-284) and io_uring one connState (queuePendingReleaseDetached(cs)). The benchmark uses a mock engine, so no number covers this; the body says "one allocation" without a figure.
To do: a hijack-then-accept benchmark on each native engine (hijack, close, accept the next conn), main vs #773, or an argument that bounds it by the accept path's existing allocations, with a number.
5. queuePendingReleaseDetached's doc does not name hijack (nit)
Its comment (engine/iouring/worker.go:3893-3907 at 6842891) lists only the detached close paths (async dispatch, WebSocket, SSE) and goroutine closures as its reason. hijackConn now uses it for another reason: the request's views must outlive the handler (:2295).
To do: add hijack and its reason to the comment.
6. TestHijackKeepsRequestViews has no reuse witness (nit)
Within a round nothing shows that B actually received into A's recycled buffer (hijack_keeps_request_views_linux_test.go:170 counts only h.k != want). On the fixed code, 0 wrong rounds cannot be told from "no reuse happened", and sync.Pool drops Puts at random under -race. Today the failing-first runs and the mutants show the test has power (epoll 11-18 and io_uring 10-16 wrong of 20 on main), but a later pool change could make it vacuous silently.
To do: a witness per arm that B was served in the conditions the defect needs, counted per round, with a minimum number of such rounds required: on epoll, that B's connState is A's recycled one (the fix still recycles the connState, only not its buffer); on io_uring, that B's connState came from the pool. A pool change that stops the reuse then fails the test instead of passing it.
7. From CodeRabbit's review of #773 (2026-09-28, head 6842891)
(b) Negative controls for both tests (CodeRabbit: Minor) (thread). Already in fix: keep a hijacked request's receive buffer out of the pool on epoll and io_uring, and copy the request values at Hijack (celeris#733) #773's body. On test-only commit 8e03dd3, TestHijackKeepsRequestViews fails at hijack_keeps_request_views_linux_test.go:184 on its epoll and io_uring arms and TestHijackCopiesRequestValuesUnderMultishotRecv at :285, 5 of 5 runs each. On the head, reverting each fix alone is killed 3 of 3 by the arm or test that covers it. Needs no change. Keeping that power visible after a later pool change is item 6.
Evidence for item 7: evidence/lanes-20260927/EPOLL-HIJACK/733/logs/ff-8e03dd3-m8-arm64.log and mutants-6842891-m8-arm64.log.
The review of #773 (celeris#733) left these minor findings and nits. None of them blocks #773, whose failing-first result, mutants and suites the review reproduced. Each item says what to do.
1. #773 and #774 depend on each other, and no committed test covers the path they share (minor)
#773's epoll fix covers
hijackConn's inline branch, which is sync mode and an async-mode conn not yet promoted. WithConfig{AsyncHandlers: true}and noAsyncroute, the handler runs inline on an async loop: on #773 alone that path still crashes (#769, fixed by #774), which is why #773's test marks its async arm's routeAsyncinstead. Once both merge, nothing committed pins #733 on the AsyncHandlers-inline path.The review's probe (
TestHijackKeepsRequestViews' rig withConfig{Engine: Epoll, AsyncHandlers: true}and noAsyncroute; kept in the maintainer's evidence tree asevidence/lanes-20260927/EPOLL-HIJACK/followups/review-probes/zz_review733_asynchandlers_test.go) gave, per the review: main + #773 + #774 + #776rounds=20 wrong=0PASS 3/3; #774's head alone wrong 15 and 16 of 20 (B'sAuthorizationbytes), FAIL 2/2; #773's head alone a nil-pointer panic atengine/epoll/loop.go:1398(#769).To do: merge #774 before #773 (or together), then add a
Config{AsyncHandlers: true}arm, with noAsyncroute, toTestHijackKeepsRequestViews, and to CI's exact tally step (arm PASScount).2. io_uring's opt-in multishot mode still leaks strings read before
Hijack(minor)In multishot receive mode (
CELERIS_IOURING_MULTISHOT_RECV=1) the request lives in a provided-ring buffer that goes back to the kernel when the handler returns (engine/iouring/worker.go:PushBufferbefore theErrHijackedreturn afterProcessH1). #773 makes strings read afterHijacksafe (the Context copies them) and documents that strings read before it must be cloned (context_response.go:1266-1269at 6842891). The most natural usage, read the path and headers, then hijack and hand them to the serving goroutine, still reads other connections' bytes there: #773's own multishot test shows it as its witness (before-Hijack "TTP/1.1\r|ETSECRETSECRET|/w HTTP/1.1\r"). It is a documented limitation, not a fix.To do: either keep the buffer for a hijacked conn (give the ring a fresh replacement entry instead of pushing the request's buffer back), or make
Hijack's doc lead with the read-after-Hijackpattern. Multishot is opt-in, so this is a limitation to close or accept explicitly, not a regression.3.
Context.Hijackcopies on std too, where the values are already copies (minor)context_response.go:1280callsc.cloneRequestValues()beforeh.Hijack(c.stream)with no engine check.middleware/websocketupgrades throughc.Hijack()on std (websocket.go:291,hijackUpgrade), so every std WebSocket upgrade now pays the copy for nothing: the mock-engine benchmark measured 257.6 ns → 1810 ns, 0 → 994 B, 0 → 37 allocs perHijack. It was never measured in the std upgrade shape. (#773's body said "Only Hijack pays, never an ordinary request: the request path is unchanged"; an upgrade is a per-connection request path, and the body now says so.)To do: measure a std WebSocket upgrade before and after; if it matters, skip the copy when the engine serves copies already (std), e.g. a capability the std ResponseWriter reports.
4. The engine-side cost is stated, not measured (minor)
After a hijack, epoll gives up one receive buffer (
cs.buf = nilbeforereleaseConnState(cs);acquireConnStateallocates a new one whencap(cs.buf) < bufSize,conn.go:280-284) and io_uring one connState (queuePendingReleaseDetached(cs)). The benchmark uses a mock engine, so no number covers this; the body says "one allocation" without a figure.To do: a hijack-then-accept benchmark on each native engine (hijack, close, accept the next conn), main vs #773, or an argument that bounds it by the accept path's existing allocations, with a number.
5.
queuePendingReleaseDetached's doc does not name hijack (nit)Its comment (
engine/iouring/worker.go:3893-3907at 6842891) lists only the detached close paths (async dispatch, WebSocket, SSE) and goroutine closures as its reason.hijackConnnow uses it for another reason: the request's views must outlive the handler (:2295).To do: add hijack and its reason to the comment.
6.
TestHijackKeepsRequestViewshas no reuse witness (nit)Within a round nothing shows that B actually received into A's recycled buffer (
hijack_keeps_request_views_linux_test.go:170counts onlyh.k != want). On the fixed code, 0 wrong rounds cannot be told from "no reuse happened", andsync.Pooldrops Puts at random under-race. Today the failing-first runs and the mutants show the test has power (epoll 11-18 and io_uring 10-16 wrong of 20 on main), but a later pool change could make it vacuous silently.To do: a witness per arm that B was served in the conditions the defect needs, counted per round, with a minimum number of such rounds required: on epoll, that B's connState is A's recycled one (the fix still recycles the connState, only not its buffer); on io_uring, that B's connState came from the pool. A pool change that stops the reuse then fails the test instead of passing it.
7. From CodeRabbit's review of #773 (2026-09-28, head
6842891)pendingReleasebackstop can let go of a hijacked conn's receive buffer while a receive it could not cancel still holds it (CodeRabbit: Major, disputed as a finding against fix: keep a hijacked request's receive buffer out of the pool on epoll and io_uring, and copy the request values at Hijack (celeris#733) #773, recorded here) (thread). IfcancelConnOpsgets no SQE for the receive's cancel (getCancelSQEreturns nil only when the SQ is still full after aSubmit,engine/iouring/worker.go:3834-3844at6842891), the receive stays armed on the socket the hijacker now owns. After the 5 s backstop,drainPendingRelease(:3936-3958) drops the entry and itsclosedOpsidentity, and the kernel can still complete that receive intocs.buf. fix: keep a hijacked request's receive buffer out of the pool on epoll and io_uring, and copy the request values at Hijack (celeris#733) #773 does not introduce this. Main'shijackConnqueued the samecson the same backstop (queuePendingRelease,worker.go:2248-2250at00d985c), whose release putcsintoconnStatePoolwithcs.buf. Every close path shares the backstop (finishClose:4034,finishCloseDetached:4150at00d985c). fix: keep a hijacked request's receive buffer out of the pool on epoll and io_uring, and copy the request values at Hijack (celeris#733) #773 changes only which release the hijack path gets: the GC drop thatfinishCloseDetachedalready uses, instead of the pool. To do: decide it with io_uring: the pendingRelease backstop returns a connection's send buffer to the pool while a SEND_ZC notification is still owed, so a peer that stalls >10 s then reads receives another connection's bytes #812. io_uring: the pendingRelease backstop returns a connection's send buffer to the pool while a SEND_ZC notification is still owed, so a peer that stalls >10 s then reads receives another connection's bytes #812's fix direction holds SEND_ZC entries past the backstop and keeps the backstop's release of anything else as the anomaly. Either extend that hold to a receive whose cancel was never submitted (keep the buffer, and itsclosedOpsidentity, until the terminal CQE), or record why the anomaly release stays.8e03dd3,TestHijackKeepsRequestViewsfails athijack_keeps_request_views_linux_test.go:184on its epoll and io_uring arms andTestHijackCopiesRequestValuesUnderMultishotRecvat:285, 5 of 5 runs each. On the head, reverting each fix alone is killed 3 of 3 by the arm or test that covers it. Needs no change. Keeping that power visible after a later pool change is item 6.Evidence for item 7:
evidence/lanes-20260927/EPOLL-HIJACK/733/logs/ff-8e03dd3-m8-arm64.logandmutants-6842891-m8-arm64.log.