fix(proxy): close the activity entry a request opened, exactly once - #181
Open
ak2k wants to merge 3 commits into
Open
fix(proxy): close the activity entry a request opened, exactly once#181ak2k wants to merge 3 commits into
ak2k wants to merge 3 commits into
Conversation
…lient The pin segment is percent-encoded by the client, and decodeURIComponent throws URIError on a malformed escape. "/tc-acct/%/v1/messages", "/tc-acct/%zz/v1/messages" and a truncated "/tc-acct/%E0%A4/v1/messages" are all ordinary request lines, and all three throw out of the pin parsing into a catch that logs and returns. The client never gets a response and waits until its own timeout expires. An undecodable pin is unusable for the same reason an unknown one is, so the decode is guarded and falls through to the existing unknown-pin 404. The reply quotes the token as it arrived, since there is no decoded form to show. The new test races the request against a timer, so a hang shows up as a failed assertion instead of a stuck run.
The 502 in forwardRequest covers the inner try only. The code above it (the egress hold, pin parsing, body buffering, the activity hooks) runs under a catch that logs and returns, and createProxyServer's request handler has the same catch around the auth gate, the CSRF gate, the forward-proxy relay and the status/reload/switch endpoints. A throw in either window leaves the socket open and silent, and the client waits until its own timeout expires. getStatusExtra is a hook the application installs, so the second window is reachable from a plain GET. Answering only before headersSent still leaves a hang: the status endpoint serializes the hook's value after writeHead, so a hook returning something JSON.stringify rejects (a cycle, a BigInt) throws with the 200 already sent, and nothing ends the response. Both catches now go through one answerUnhandled helper carrying the same pair of arms forwardRequest already uses: a 502 while nothing has been written, and destroy once something has. destroy rather than end() on the second arm, because end() delivers truncated bytes as an apparently complete reply and the client has no reason to retry. The headersSent guard matters in the other direction too: the inner finally calls onRequestEnd after the response has streamed, so a throw from that hook reaches the catch with the headers long sent, and an unguarded writeHead would raise ERR_HTTP_HEADERS_SENT from inside the catch. Four new tests, each racing its request against a timer so a hang is a failed assertion instead of a stuck run.
Every consumer of the activity hooks holds a request's row until it is told the request ended. The TUI keeps it in `active` and keeps its animation running while one is open; a headless consumer counts it as in flight. Nothing reclaims a row that is never closed, so on a daemon that runs for weeks each leak is permanent. Only the inner path had a `finally`, so a throw above it opened a row that nothing would ever close. The ordinary trigger is a client cancelling mid-upload, which makes `for await (const chunk of req)` reject: Ctrl+C in Claude Code does that routinely. A start hook that throws leaks the same way, and that is the shipped hook's shape, since the TUI registers the row and then renders, and the render can rethrow. The listener now tracks the open entry in one marker. It is set before the start hook, so a hook that registers its row and then throws is still accounted for. Every closing site clears the marker before calling the end hook, because a hook that throws would otherwise still look open to the outer catch, which would then call that same hook a second time for one request. The outer catch closes whatever is left, as 499 when the client is gone or the response is past saying anything, and 502 when that is what it is about to write. Its own call to the hook is guarded, because the throw that sent it there may be that hook. Unguarded, the second throw escapes an async request listener with nothing above it, which is an unhandled rejection, and crash-log.js turns that into exit(1). A broken activity hook could take the daemon down. The recovery also has to survive its own logging, because the console it logs through is the same component whose failure it is recovering from. Under the TUI the console is the TUI: `console.error` appends to the activity log and repaints, so a render that throws makes the console throw. The report is the first statement of each of these paths, so an unguarded one skips the whole recovery: the row still leaks, the socket is never answered, and the throw escapes as an unhandled rejection after all. Every report on a recovery path now goes through one helper that falls back to stderr, the way the TUI already does when its own activity stream fails. The fallback writes with `writeSync` rather than `process.stderr.write`, because it has to fail the way the helper promises. A stderr whose reader is gone makes the stream surface EPIPE asynchronously, as an error event no `try` around the call can see, and an uncaught EPIPE is fatal here. Written that way, a helper meant to keep a broken render from killing the daemon would kill it on a closed pipe instead. `writeSync` throws where it is called, so the catch is real. Eight tests: an abort mid-body, a start hook that registers and then throws, an end hook that throws, a hook that throws on every call, the blocklist's early return, a console that throws on each of the two recovery paths, and a report written to a stderr nobody is reading.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Builds on #180, which added the outer catch this uses. Review that one first; the diff here is the last commit only.
Cancel a request in Claude Code with Ctrl+C and the proxy never learns it ended: the TUI's activity row spins forever, and a headless consumer's in-flight count grows by one, permanently, per cancelled upload.
repro-leak.mjsWhy
The entry is opened by
onRequestStartbut only closed by thefinallyaroundforwardRequest, so anything that throws above the inner try leaves it open forever. Two ordinary things do: a client cancelling mid-upload (for await (const chunk of req)rejects above the inner try), and a start hook that registers its row and then throws, which is the shipped TUI hook's shape (active.set(id, ...), then a render that can rethrow).The fix
The listener tracks the open entry in one marker: set before the start hook, cleared before the end hook at every closing site, and the outer catch closes whatever is left (499 if the client is gone or the response is past answering, 502 when that is what it is about to write). Each ordering trades a spurious close, which every shipped consumer tolerates, against a permanent leak. The outer catch's own hook call is guarded, because the throw that sent it there may be that hook; unguarded, it escapes as an unhandled rejection, which
crash-log.jsturns intoexit(1).The recovery has to survive its own logging
Under the TUI the console is the TUI:
console.errorrenders, so a broken render makes the first line of every recovery throw. That half exists on plainmastertoday.onRequestRoutedis called insideforwardRequest, whose catch answers 502; run the same request twice and vary only the console:probe-master-logging.mjsInside this change, one broken render causes both throws at once. Measured before this part was added:
{ rowsStillOpen: 1, clientGot: "HUNG", escaped: ["TUI render failed"] }. After:{ rowsStillOpen: 0, clientGot: "HTTP 502", escaped: [] }.So every recovery-path report goes through one
reportFailurehelper that falls back to stderr with the full stack, astui.jsalready does when its own activity stream fails. The fallback writes withwriteSync, notprocess.stderr.write: a closed stderr surfaces EPIPE asynchronously where notrycan catch it, and this daemon treats an uncaught EPIPE as fatal.Three of the four converted sites are each held by a test; the fourth, the inner catch around
forwardRequest, is converted for consistency and carries no claim.Tests
Eight in
test/activity-entries.test.js. Six fail on this PR's base:The three console tests run the proxy in a child with a throwing (not muted)
console.error, since a raw fd-2 write bypasses any in-process stub. The other two pass on the base and pin the clear-before-call orderings; moving either clear turns its test red.538/538 (530 on the base), #180's five tests unmodified,
npx eslint .clean.Compatibility
Accounting and hooks only; no routing, selection, retry or upstream changes. Clients see a difference in two places, neither a loss: an abandoned request's entry now closes as 499 where the client previously got nothing, and under a throwing TUI render, requests that used to hang now get their 502.
onRequestEndconsumers can now seeaccount: null, model: nullwith a 499 or 502 for a request that never reached selection, the same shape an early failure already produced.