fix(stack): log target failures and handle disconnections quietly - #6691
Conversation
A wake failure reached clients only as a dead connection: the request path answered 502 and the upgrade path destroyed the socket, both discarding the cause, while preparation failures never reached the service observation. Any failed wake was therefore indistinguishable from a flake. Both proxy arms now log the route and the underlying cause, with a client that disconnects first carrying its own tagged error so ordinary aborts stay quiet, and a failed wake preparation is recorded on the observation so status names it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LcUaThuQfDKPwi9mCE9XeD
… them The auto scan walked contiguously from a random start and gave up after 64 consecutive bind failures, so a start inside a reserved block wider than 64 ports failed with "No public port is available" while tens of thousands of ports were free. Windows publishes such excluded ranges, which is where this surfaced. Each probe is now redrawn, and the exhausted error carries the last bind failure so a reserved range reads differently from an occupied port. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LcUaThuQfDKPwi9mCE9XeD
There was a problem hiding this comment.
🤖 AI Review
Three substantive findings are confirmed after merging the overlapping reviews: randomized port probing can falsely exhaust candidates, stale preparation failures can overwrite current observations, and post-acquisition client disconnects are logged as proxy errors. Two Claude-only consistency/documentation findings are refuted with concrete counter-evidence. No uncertain findings remain.
Findings
| Severity | Location | Category | Sources | Claim |
|---|---|---|---|---|
| 🟡 MINOR | packages/stack/src/HttpProxy.ts:292 |
observability |
claude+codex | Client disconnects after target acquisition are classified as proxy failures and logged at error level. |
| 🟡 MINOR | packages/stack/src/Ports.ts:66 |
correctness |
claude+codex | Automatic port probing samples with replacement and omits port 49999, allowing false exhaustion while usable candidates remain. |
| 🟡 MINOR | packages/stack/src/Service.ts:433 |
concurrency |
claude+codex | A stale preparation failure can overwrite the error field of a newer running service observation. |
Refuted findings (kept for transparency, not posted as review comments)
packages/stack/src/Service.ts:431(consistency): Recording preparation failures for every startAt call but not restart is an inconsistent loss of restart failure observability.
Refuted: A restart preparation failure is not dropped: it is returned to the restart caller, and the currently running service remains valid. The comment identifies the distinct wake case, which lacks such a caller; it does not require failed restart attempts to mark a healthy running observation as errored.packages/stack/src/Ports.ts:66(documentation): This PR newly makes automatic port probing inconsistent with ADR 0017.
Refuted: The ADR/code divergence predates this PR, so the change did not introduce the claimed documentation regression. The newly introduced exclusive-upper-bound problem is preserved in the merged port-correctness finding.
Stats
Claude findings: 5 · Codex findings: 3 · Confirmed: 3 · Refuted: 2 · Uncertain: 0
Models: claude-opus-5 + gpt-5.6-sol · Trigger: auto · Workflow run
This review runs once per PR. A maintainer can request another with a /ai-review comment.
Drawing a random port per probe cleared reserved ranges only by chance. A stride co-prime with the span visits every port once and keeps consecutive probes far apart instead, so a reserved range narrower than the stride costs at most one probe and can never exhaust the failure budget. The scan starts from an offset derived from the stack's checkout, id, and key, so separate checkouts and keys stay apart while one stack reassigns the same port across runs. Ports no longer needs the platform crypto service, and its callers stop carrying that requirement. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LcUaThuQfDKPwi9mCE9XeD
The auto scan now follows the architecture ADR again: it walks `20000..32767` with stride 257, staying below the Linux ephemeral range and visiting every candidate once, so repeated probes can neither skip a port nor spend the bind allowance twice. Only the start differs from the ADR, deriving from the stack instead of a random draw; the ADR records that. A client that goes away mid-response or resets an established upgrade now fails with the disconnect error rather than a proxy error, so ordinary client exits stay out of the log, and a preparation failure reaches the observation only while the service is still the stopped, registered attempt that prepared it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LcUaThuQfDKPwi9mCE9XeD
jgoux
left a comment
There was a problem hiding this comment.
Requesting changes for the reproduced mid-response disconnect regression below. The existing 42 targeted integration tests pass, but a client that closes after receiving the first response chunk produces a spurious error log; the same reproduction passes against the base.
The port scan is a separate, reasonable fix; its description overstates the guarantee, as noted inline. A separate pre-existing limitation is that detached hosts discard stdout/stderr, so the new proxy logs are not retained in that topology.
Destroying a partially received upstream response emits `aborted` synchronously, and the abort listener re-entered the abandon path while the operation was still unsettled. A client that closed after receiving a body chunk therefore resettled as `HttpProxyError: undefined` and logged, defeating the quiet classification. Both proxy paths now settle and drop their listeners before destroying, and the mid-response disconnect is covered: the previous coverage closed the client before any upstream response arrived. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LcUaThuQfDKPwi9mCE9XeD
TL;DR
502or a dead socket, with the cause discarded anderror=undefinedinstack status. The proxy now logs the route and cause, and the failure reaches the service observation.stack port tests (windows-latest)could not allocate a port. The auto scan walked contiguously and died inside a Windows reserved block. It now walks20000..32767with stride257per ADR 0017, so a reserved block up to 12511 of the 12768 ports is escaped within the bounded attempts.Details
Two independent stack failures, both surfaced while investigating the 2026-09-21
Testbreakage.A failed lazy wake was unobservable
A wake that failed reached clients only as a dead connection.
HttpProxydiscarded the cause in both arms — the request path answered502and the upgrade path destroyed the socket — andService.startAtrunsdefinition.prepare(where native artifact preparation happens) before it touches the observation, so a preparation failure lefterrorundefined. Every wake failure was therefore indistinguishable from a flake.Both proxy arms now log the route id and the underlying cause before responding, and a failed wake preparation reaches the service observation, so
stack statusand the reopened-stack diagnostics name the cause instead of printingerror=undefined.The two halves cover different topologies. A detached owner is spawned with
stdout/stderrignored, so the proxy log lines land only where the runtime shares the caller's process; the recorded observation is what names the cause throughstack statusand the e2e diagnostics in the detached topology. Retaining a detached owner's output is a separate pre-existing gap.Client exits stay quiet, and are told apart from upstream faults by type rather than by message: a client that disconnects while the target is waking, goes away mid-response, or resets an established upgrade fails with
HttpProxyDisconnected, while upstream faults keep failing withHttpProxyErrorand log. Both paths settle before destroying their sockets, because destroying a partially received upstream response emitsabortedsynchronously and would otherwise resettle a disconnect as a proxy failure. A preparation failure is recorded only while the service is still the stopped, registered attempt that prepared it, so a concurrent start that already relaunched keeps its own state.stack port tests (windows-latest)could not allocate an auto portThe auto scan walked contiguously from a random start over
20000..49999and gave up after 64 consecutive bind failures. Windows publishes reserved TCP ranges as contiguous blocks commonly wider than 64 ports, so a start landing inside one failed withNo public port is availablewhile tens of thousands of ports were free. That range also overlapped the Linux default ephemeral range.The scan follows ADR 0017 again:
20000..32767with stride257, which is co-prime with the span and so traverses every candidate once across a full scan. Within the ADR's 64-attempt bound, a contiguous reserved block up to 12511 of the 12768 ports is always escaped, worst case 49 failures, verified over every start and block offset. Wider blocks can still exhaust the bound — that limit is the ADR's, and the stride does not remove it — but they are far beyond the width of a published excluded range.The one deliberate divergence from the ADR is the scan start, which derives from the stack's project root, id, and listener key rather than a random draw. Allocation is reproducible: a stack reassigns the same port across runs, separate checkouts stay apart, and a future failure repeats instead of depending on the draw. The ADR records this.
The exhausted error also carries the last bind failure, so a reserved range (
EACCES) reads differently from an occupied port (EADDRINUSE) from the log alone.Changes
HttpProxy.ts: log route request and upgrade failures with the route id and cause; give client-originated disconnects a distinct tagged error; settle before destroying socketsPorts.ts: walk the documented range with a co-prime stride from a derived start; attach the last bind failure to the exhausted error. No longer needs the platform crypto service, soHostProcessstops carrying that requirementService.ts: record a failed wake preparation on the observation, guarded against a concurrent relaunchdocs/adr/0017: record the derived scan startService.tshere is the wake path only;restartis unchanged because its caller already receives the error.🤖 Generated with Claude Code
https://claude.ai/code/session_01LcUaThuQfDKPwi9mCE9XeD