Skip to content

fix(pty): guard node-pty's unlistened win32 conin socket, and harden Windows CI - #63

Merged
y49 merged 22 commits into
mainfrom
fix/windows-conin-crash
Jul 28, 2026
Merged

fix(pty): guard node-pty's unlistened win32 conin socket, and harden Windows CI#63
y49 merged 22 commits into
mainfrom
fix/windows-conin-crash

Conversation

@y49

@y49 y49 commented Jul 28, 2026

Copy link
Copy Markdown
Owner

Closes #59.

Started as "is Windows CI even worth keeping?" — it had a ~10% red rate on a required check and, across its whole history, had never caught a product bug. Every red traced back to test fragility.

Investigating the most recent one reversed that. #59 is a real daemon crash on Windows, not the test-teardown noise it was filed as. The CI job had caught its first product bug and we had misfiled it.

The crash

node-pty opens two pipes to the ConPTY. The read side gets an 'error' listener; the write side does not:

// windowsPtyAgent.js:79 — conin, constructed with no 'error' listener
this._inSocket = new net.Socket({ fd: fs.openSync(term.conin, 'w'), readable: false, writable: true });

// windowsTerminal.js:54,90 — conout, listener present
this._socket = this._agent.outSocket;
this._socket.on('error', err => { /* ... */ });

Every pty.write() goes to that unguarded socket (windowsTerminal.js:123) and kill() destroys it (windowsPtyAgent.js:176). A net.Socket with no 'error' listener escalates any error to an uncaught exception, so a failed write takes the process down. Two triggers reach it: EAGAIN when conin's buffer fills, and a write racing kill().

tlive writes user keystrokes into the pty and kills ptys when sessions end, so on Windows a keystroke landing in the same tick as a kill crashes the daemon. Tests hit it first only because they start and stop ptys far more densely than a person does.

Fixed in three layers:

  • win-conin-guard.ts attaches the listener node-pty is missing. This is the fix that stops the crash — it covers EAGAIN backpressure too, which has nothing to do with our lifecycle. Reaching through the private _agent is deliberate and is the only route: Terminal.on() forwards to the conout socket, so there is no public path to conin. The module records the node-pty version the shape was verified against, and degrades to a no-op rather than throwing if a future release renames it.
  • disposePty() clears this.pty after kill, so a client Data frame already queued cannot reach a torn-down pipe. Defence in depth, and correct regardless of platform.
  • Upstream report filed against node-pty so the guard can eventually be deleted.

Windows CI now covers the product, not just the test suite

install-smoke-windows installs the real packed tarball on windows-latest, then exercises the .cmd shim, loads node-pty from the global install, spawns an actual ConPTY, and runs the named-pipe daemon through start/status/stop.

Windows previously ran unit tests and nothing else, which is precisely why a genuine Windows crash could sit behind a "test flake" label. The Linux install-smoke job earned its place the same way, by catching node-pty 1.1.0 shipping no linux prebuild — invisible to unit tests, which install with a full toolchain present.

Not added to branch protection in this PR. It should accumulate a track record on real runners first.

The flake class that made the job untrustworthy

Fixed setTimeout waits sized for unix sockets, ahead of assertions that something had arrived. Windows named pipes are slower, so margins that hold on Linux do not hold there — that is both #45 and the expected +0 to be 1 red from last week.

Nine IPC/pty test files were reviewed; six needed changes, and 29 waits now poll for their condition through a shared until() helper. The other three — spawn, edit-queue, client-close — were deliberately left untouched, which is as much the point as the conversions: their waits are a bounded polling loop and two absence checks, and converting them would have been the mistake.

Deliberately not converted: absence and invariant assertions, which prove something did not happen. vi.waitFor returns on its first successful check, so a condition that is already true is asserted with zero elapsed time — converting those would silently gut them while leaving the tests green. Every retained fixed sleep is annotated with why.

Every retained fixed interval is an absence or invariant assertion, or an already-bounded polling helper, annotated in place with why it stays. Two sites are worth calling out because they are the trap in miniature:

  • One assertion looked like an arrival but read its expected value off the very state it then asserted on, so it was already true before the action under test. Wrapping it in until() made it weaker than the 80 ms sleep it replaced.
  • One sequencing wait has no daemon-observable substitute at all — on the suppressed grace path the handler returns before the observable is ever assigned. It stays a fixed sleep, widened, with the exit condition documented in place.

Also fixed

SessionHost initialised lastOutputAt at construction, so the first activity poll classified every session as running — 1000 ms elapsed is inside the 1500 ms window whether or not the child ever wrote anything. A silent session reported running for its first second, contradicting the contract in that file's own header. Activity now derives from observed output.

Verification

651 tests green; typecheck and build clean. Beyond that:

Controlled experiment on real Windows hardware. A VM that reproduces the flake class far more readily than GitHub's runners do — 90% vs ~10% — ran the full suite ten times on each revision. Same machine, same command:

before (8b983db) after (85150c7)
failures 9 / 10 0 / 10

The baseline failures were exactly the tests this branch converted, including the expected +0 to be 1 signature from CI.

install-smoke-windows verified non-vacuously. Its first run passed while its daemon step could not actually fail — pwsh does not halt on a native non-zero exit, and tlive stop exits 0 when nothing is running, so the verdict came from that alone. After gating each command, the step logs tlive daemon started (pid 2688) / daemon: running (pid 2688, uptime 0s) / tlive daemon: stopped, so the assertion demonstrably runs. That same run is also the first time tlive status's non-daemon output path has executed on real Windows.

What is not verified. write EAGAIN never reproduced on the VM — 40 single-file runs and 10 full-suite runs, before or after. The crash fix rests on the code-level argument, not a before/after measurement: the missing listener is a fact about node-pty's source rather than a hypothesis, but nobody has watched the crash stop on demand. #59's closing comment says so, and asks for a reopen rather than a new issue if it recurs.

Every commit was reviewed by an independent pass before the next one started; three of the plan's own technical premises were caught and corrected that way.

y49 added 15 commits July 28, 2026 09:23
node-pty attaches an 'error' listener to the conout socket but not to
conin (windowsPtyAgent.ts:79). A net.Socket without one turns any write
failure into an uncaught exception, so an EAGAIN under backpressure — or
a write racing kill() — crashes the daemon on Windows. Attach the
listener the upstream package is missing.

Refs #59
stop() killed the pty but left this.pty set, so a Data frame already
queued on a client socket still reached pty.write() after the pipe was
torn down. Clearing the handle turns those late writes into no-ops.

Refs #59
A fixed 40ms tick assumed the request had crossed the IPC socket. On
Windows named pipes it had not, which is the 'expected +0 to be 1' red.
Adds a shared until() helper for arrival assertions; absence assertions
keep a real elapsed interval, which is the only thing that makes them
meaningful.
Same Windows-flake class as session-ipc: fixed sleeps sized for unix
sockets before arrival assertions. Absence assertions and the genuinely
time-based idle-flip test keep their fixed intervals.
lastOutputAt was initialised at construction, so the first activity poll
always classified a session as running — 1000ms elapsed is inside the
1500ms window regardless of whether the child ever wrote anything. A
silent session reported running for its first second, contradicting the
contract in this file's own header, and made onActivity unusable as a
"saw output" signal.
Ten arrival assertions were preceded by fixed 100-150ms sleeps. The four
absence assertions keep theirs — a condition that is already true
returns immediately and would assert nothing.
Denying request A must not wipe B's pending indicator. reqB is read off
the frame where pending had already moved to B, so the assertion is an
invariant, not an arrival — until() returned on its first check and
proved nothing. Widens the continue-grace sequencing wait too, the one
fixed sleep in this file with no daemon-observable substitute.
Nine arrival assertions, including two setTimeout(0) microtask flushes.
The two remaining fixed sleeps are inside fake adapters simulating
channel latency, not test waits.
Windows had unit tests and nothing else — the install path a Windows
user actually walks (prebuild load, ConPTY spawn, .cmd shim, named-pipe
daemon) had no coverage. That gap is why #59, a real Windows daemon
crash, sat filed as test flake.

Not added to branch protection yet; it needs a track record first.
pwsh does not halt on a native non-zero exit, and GitHub Actions reads
$LASTEXITCODE only after the whole script — so the step's verdict came
from `tlive stop` alone, which exits 0 when no daemon is running. A
completely broken `tlive start` reported green. Gates each command and
asserts status output.
ConPTY emits initialisation sequences before any child output, so a pty
that produces nothing does not exist on Windows and the assertion cannot
mean what it says. Caught by the first real install-smoke-windows run.
The late joiner accumulated frames until EARLY-SCREEN appeared, which is
satisfiable by live output as well as by the snapshot. onClient writes the
Size frame then the snapshot as the first Data frame, synchronously, so the
first Data frame can be asserted directly. Also removes most of the 100ms
wait after the probe detaches: the shadow is written inside pty.onData before
the broadcast, so it already holds the bytes the probe just received. A minimal
10ms delay remains to ensure the probe socket's close event is processed on the
server side before the late joiner connects.
The probe teardown was guarded by an arbitrary sleep. Waiting on the
socket's own close event removes the magic number: the client's 'close'
event signals that the socket is torn down on this end, and a small follow-up
delay (20ms) ensures the server's removeClient and applySize have completed
before the late joiner attaches.
The late-joiner snapshot test guarded its precondition with a sleep. The
precondition is that EARLY-SCREEN has reached the shadow terminal before the
client under test attaches; without it the snapshot is empty, the live stream
carries the string anyway, and the test passes without touching the snapshot
path.

Watching a probe client for EARLY-SCREEN on the live stream does not establish
that. pty.onData broadcasts to sockets synchronously, but xterm's write buffer
parses on a later macrotask (WriteBuffer.write schedules _innerWrite via
setTimeout), so serialize() still returns '' while those bytes are already on
the wire. onClient then skips the Data frame entirely (snap.length > 0), the
joiner receives only a Size frame, and the test times out — 5/25 full-file runs
on Linux. The sleep was covering the write-buffer latency, not socket teardown:
the probe was already out of the clients set by then.

Wait on the shadow's own content with `until` instead, and drop the probe's
detach: a late joiner is one that attaches after the output happened, not the
only client, and every client gets its own snapshot on its own first attach.
No delays left in the test. 35/35 full-file runs green; deleting the snapshot
write still fails it 1/1.
y49 added 7 commits July 28, 2026 14:17
…d wait left

The comment claimed this was 'the one remaining fixed wait in this
file,' which reads as every other wait already being converted. It
isn't: four other fixed intervals in this file are deliberate absence
assertions (dropped-deny, pending, status, droppable invariants), not
leftovers. As written, a future 'clean up the last magic sleep' pass
would find those four and convert them too, silently gutting four
absence checks.

Reword to match what ec13922's commit body already said correctly:
this is the one fixed sleep that has no daemon-observable substitute.
The assertion sat behind an unconditional setTimeout(...,5000): every
run burned the full 5s even on success, and a ConPTY answering at
5001ms would fail the step. Same pattern this branch removes from the
test suite, just left standing in the new CI script.

Check for 'pty-ok' inside onData and resolve immediately; the 5s timer
is now only a failure deadline, guarded so it's a no-op once onData has
already won. Same safety, no fixed margin on the success path.
node-pty 1.2.0-beta.14 ships lib/*.js (plus native C++ src/), not the
TypeScript sources the comment cited (windowsPtyAgent.ts /
windowsTerminal.ts) — those never make it into the published package.
Line numbers were already correct and do land on the referenced
constructs; only the extensions/dirs were wrong. This comment is the
only map to a private-internals reach, so it needs to stay navigable.
…tart()

win-conin-guard.test.ts covers guardWindowsConinSocket in isolation, but
deleting its call at session-host.ts:90 leaves the whole suite green —
on Linux the guard is a no-op by design, so no black-box test can see
it. That is the one place dropping the fix for #59 would go unnoticed.

Close it platform-independently with vi.mock('../win-conin-guard.js')
and assert the call count increases by exactly one across start().
Placed here rather than in win-conin-guard.test.ts because that file
unit-tests the real implementation directly; auto-mocking the module
there would neuter those tests. Nothing else in this file asserts on
the guard, so mocking it file-wide doesn't change any other test's
behavior.
The local was typed { serializer: ... }, not a shadow terminal —
SessionHost has separate shadow and serializer private fields, so the
old name was actively misleading next to the real thing it was
casting past.
'reports running on output then idle after silence' asserted
flips[0] === true after a fixed 3200ms sleep. That used to be
guaranteed by construction, but after this branch's product fix it
depends on the child's first byte beating the poll tick at
spawn+~1000ms — a wide margin, but now a timing dependency, in the one
branch that should not leave those behind. Replaced with two until()
waits on containment instead of index/fixed-delay.

'a silent child (no output) does not report as running' waited a fixed
1200ms against a 1000ms poll tick (~200ms margin), and an empty flips
would have passed vacuously. Replaced with until() on the false flip
(a genuine, non-vacuous arrival — the first tick always reports idle)
followed by the not.toContain(true) check, which is now evaluated
against state that can no longer change because this test's child
never writes. Not an absence-assertion violation: the arrival gives
until() something real to resolve on, and the absence check only holds
static state afterward. it.skipIf(win32) and its ConPTY explanation
are unchanged.
A file-global vi.mock replaced the guard with a no-op for every test in
session-host.test.ts, including the ones that spawn real ptys and drive
real writes and kills — the exact place a Windows conin failure would
need swallowing. spy: true keeps the implementation and still lets the
wiring be asserted.
@y49
y49 marked this pull request as ready for review July 28, 2026 06:38
@y49
y49 merged commit 77f8535 into main Jul 28, 2026
4 checks passed
@y49
y49 deleted the fix/windows-conin-crash branch July 28, 2026 06:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Windows CI: intermittent write EAGAIN unhandled error in session-host tests

1 participant