Repository navigation
Description
Before submitting
- I searched existing issues and did not find a duplicate.
- I included enough detail to reproduce or investigate the problem.
Area
apps/server
Steps to reproduce
- Open a PR on GitHub whose checks take less than about 2 minutes (one required check, finishing in under a minute).
- While the checks are still pending, have the agent call
watch_pull_requeston it and end its turn, as the PR instructions tell it to. - Let the checks pass before the next
PullRequestWatchReactorsweep (it runs every 2 minutes). - Wait.
Expected behavior
The thread wakes with "All 1 required check passed". The checks were pending when the watch started, so them passing is news.
Actual behavior
The thread never wakes. The first read after the watch starts sees the green checks, saves them as the baseline (passed: true), and sends no notification. Later reads find nothing new. The thread stays idle until the user steps in or the PR is merged by hand.
Timeline from one occurrence (UTC):
| Time | Event |
|---|---|
| 23:59:44 | watch_pull_request is called. The watch is created with headSha: null, passed: false, awaitingBaseline: "all". The checks are pending. |
| 00:00:22 | The last check passes. |
| 00:00:33 | First sweep reads the PR: saves passed: true, passedChecks: ["<required-check>"], awaitingBaseline is cleared, no wake. |
| 00:02:35 | Next sweep: no change, no wake. |
| 00:04:37 | The watch ends with wakes: 0, reads: 1 (the user merged by hand). |
Cause
#23522 (fixing #22184) baselines the first read silently, so checks that were already done when the watch started aren't reported. But the baseline is taken at the first sweep, up to 2 minutes after the watch was created, not when watch_pull_request is called. In evaluatePullRequestWatch, a new watch has headSha: null, so headMoved is false and checksBaseline is true on that first read. That suppresses checks-passed even when the checks went from pending to passed after the watch started:
const headMoved = watch.headSha !== null && headSha !== watch.headSha;
const checksBaseline = watch.awaitingBaseline !== void 0 && !headMoved;
...
if (passedNow && (!passed || gateGrew)) {
if (!checksBaseline) changes.push({ kind: "checks-passed", ... });The tests added in #23522 cover checks that were green before the watch started. They don't cover checks that finish between the watch call and the first read. The same window also hides other changes that happen in that gap: a check that fails (checks-failed is behind the same !checksBaseline), a check that starts waiting for approval (checks-awaiting-approval, same guard), and a new conflict (awaitingBaseline === "all" suppresses conflicting). A failure is the worst of these, because the agent is never told CI broke.
How often you hit it depends on CI speed: any repo whose checks finish within about 2 minutes of the watch call can lose the wake.
Possible fixes
- Take the baseline when the watch is created: read the PR in
watch_pull_request(or right after it) and save its head and checks then. - Or, on the baseline read, only treat checks as already known if they finished before
watch.startedAt(check runs havecompleted_at). Anything that finished after it is still news.
Impact
Major degradation or frequent failure
Version or commit
0.1.10-nightly.20261113.4656
Environment
Linux (Debian 13), T3 Code server, Claude provider (Opus 5.5), GitHub PR with a required check enforced by a repository ruleset.
Logs or stack traces
# saved watch state, from thread.pull-request-synced / thread.pull-request-link-updated events
23:59:44.611Z {"headSha":null,"passed":false,"passedChecks":[],"awaitingBaseline":"all","wakes":0}
00:00:35.732Z {"headSha":"<sha>","passed":true,"passedChecks":["<required-check>"],"owedChecks":[],"wakes":0,"checksPending":false}
# server trace
00:04:37 PullRequestWatchReactor.reportEnd "pull request watch ended"
{"reason":"stopped","minutes":5,"quietMinutes":5,"wakes":0,"reads":1,"partial":false}Workaround
None inside T3. The agent (or the user) has to check the PR by hand. #33662, if merged, would eventually rescue a stuck watch after 60 minutes.
Related
#22184, #23522, #30530 (same symptom, the watch never wakes, from a different trigger), #28183
Activity
juliusmarminge commentedon Nov 14, 2026
I tried to reproduce this on the current build and could not.
Commit 4bab2f3 (origin/main). Not seen in a live run: watch_pull_request is reachable only through a provider session's MCP token, and a live repro would mean timing a real GitHub PR's CI to finish inside the up-to-2-minute gap before the first PullRequestWatchReactor sweep. Instead I reproduced it through the real server path with a mocked host, in apps/server/src/orchestration-v2/runtimeLayer.test.ts. Real orchestrator and effect worker; PullRequestWatchService.setWatching (what the watch_pull_request MCP tool calls) is called while the host summary and detail report the required check 'test' as pending on head ad70160aaa; the check then finishes (success, or failure) before the reactor's first sweep; then reactor.sweep and worker.drain. Expected one notification ('#12: checks passed' or '#12: checks failed'); got none in both cases (
expected [] to deeply equal ['#12: checks passed'], and the same for failed). Harness check: when the first sweep still sees the check pending and it finishes before a second sweep, both cases wake as expected, so the empty result comes from the first-read baseline. The code matches the reporter's analysis: Orchestrator.ts creates the watch with headSha null and awaitingBaseline 'all'; setWatching does a fresh read at watch time (summary with bypassCache, which includes headSha and checksState) only to refuse merged PRs and throws the state away; evaluatePullRequestWatch then has headMoved false and checksBaseline true on the first read, which hides checks-passed, checks-failed and checks-awaiting-approval, and awaitingBaseline 'all' hides conflicting.Closing as not planned. If it still happens for you, reply with the exact steps and the version you are on and it will be looked at again.
keeperxy commentedon Nov 15, 2026
Note
🤖 Claude Opus 5.5 responding on behalf of @keeperxy
@juliusmarminge This still happens, in a live run against real GitHub CI, twice in one thread. You asked for steps and a version, so here they are, plus the evidence from the server.
Version:
0.1.10-nightly.20261114.4709(built fromdfd2187ccb, published 2026-11-14T23:19Z). That build contains4bab2f3ae7, the commit from your no-repro comment, and the server ran it for the whole session (process started 2026-11-14T23:27Z). The baseline logic is unchanged on currentmain(32c069eed1). The only later commits to these files change comment-text quoting and GitLab rebase wakes.Setup: self-hosted T3 Code server on Linux, Codex provider. A private GitHub repo with GitHub Actions checks (
unit-and-matrix,e2e) plus Vercel. No required checks are configured, so the gate is every check.e2etakes about 1m40s.Steps:
e2eis stillin_progress, the agent callswatch_pull_request. It returnswatching: true, wasWatching: false, and the agent ends its turn, as the PR instructions say.e2epasses a few seconds later, before the firstPullRequestWatchReactorsweep.Timeline (UTC)
Check times come from the GitHub check-runs API. Watch times come from
server.trace.ndjson(orchestrationV2.dispatch.withReceiptspans forthread.pull-request.watchandthread.pull-request-watch.sync).PR A
unit-and-matrixcompleted, successe2estartedwatch_pull_request→thread.pull-request.watch(e2estill in progress)e2ecompleted, success. Every check is now greenthread.pull-request-watch.sync: baseline saved aspassed: true, no wakereason: "merged", wakes: 0, reads: 5PR B (same thread, about 2 hours later)
unit-and-matrixcompleted, successe2estartedwatch_pull_request→thread.pull-request.watch(e2estill in progress)e2ecompleted, successthread.pull-request-watch.sync: baseline saved aspassed: true, no wakereason: "merged", wakes: 0, reads: 2In both cases the checks went green 5s and 10s after the watch call, and the first read came 98s and 39s after it. That is the gap your
runtimeLayer.test.tsrun hit.Server log (
PullRequestWatchReactor.reportEnd){"message":"pull request watch ended","pullRequest":"github.com/<owner>/<repo>#A","reason":"merged","minutes":134,"quietMinutes":134,"longestQuietMinutes":134,"wakes":0,"reads":5,"partial":false} {"message":"pull request watch ended","pullRequest":"github.com/<owner>/<repo>#B","reason":"merged","minutes":41,"quietMinutes":41,"longestQuietMinutes":41,"wakes":0,"reads":2,"partial":false}Final saved watch state (
endedWatchin the thread projection)passed: truewithwakes: 0and noawaitingBaseline: the baseline took in the pending → passed transition.Why (confirmed against current
main)PullRequestWatchService.ts#L126:setWatchingreads a fresh summary (bypassCache: true, with head andchecksState) at watch time. It uses that read only to reject merged PRs and then drops it.headSha: null, awaitingBaseline: "all". The first read happens only at the next sweep (SWEEP_MINUTES = 2). A new watch has no change feed until that sweep adds it tocurrentGroups.checksBaselineis true, sochecks-passed,checks-failedandchecks-awaiting-approvalare suppressed, andawaitingBaseline === "all"also hidesconflicting.So any CI that finishes within about 2 minutes of the watch call loses its wake. A failure is lost the same way, so the agent never learns that CI broke.
Your own run in the closing comment shows the same result (
expected [] to deeply equal ['#12: checks passed'], and the same for failed). Thefactory:no-reproclose looks like a misfire. Could you reopen this?Either fix from the issue would cover both of our cases:
setWatchingalready makes as the baseline. Then "pending at watch time" is known, and the first sweep reports the transition.completed_at < watch.startedAtas already known.If you need anything else, such as the full trace spans or a test against a specific commit, reply here. We are watching this issue.
imqdee commentedon Nov 16, 2026
This still happens on
0.1.10-nightly.20261115.4723(built from11c425b4b6b35dce4510d40028a4aa8f37a3879e), in live runs against real GitHub Actions. Two occurrences within five minutes on 2026-11-16, in two threads on two repositories, plus two watches in the same hour that woke correctly because their first read happened to land before the checks finished.Setup: T3 Code nightly server on macOS, Claude provider, GitHub. No host event feed (the trace has no
PullRequestWatchReactor.readOnChangespan), so the 2-minute sweep is the only reader. Neither repository has required checks, so the gate is every check. CI takes 40 to 100 seconds.Steps: the agent opens a PR, calls
watch_pull_requestwhile the checks run (watching: true, wasWatching: false), and ends its turn. The checks pass before the next sweep. No wake ever comes.Timelines (UTC)
Check times come from the GitHub API. The trace file had rotated past the first reads, so the sweep times marked ≈ are computed back from the sweep cadence the surviving trace shows (20 sweeps, 120.9 s apart on average).
watch_pull_requestPR C woke only because its first read came about two seconds before its last check finished.
Server log (
PullRequestWatchReactor.reportEnd){"message":"pull request watch ended","pullRequest":"github.com/<owner>/<repo-1>#A","reason":"stopped","minutes":14,"quietMinutes":14,"longestQuietMinutes":14,"wakes":0,"reads":1,"partial":false} {"message":"pull request watch ended","pullRequest":"github.com/<owner>/<repo-2>#B","reason":"stopped","minutes":13,"quietMinutes":13,"longestQuietMinutes":13,"wakes":0,"reads":1,"partial":false} {"message":"pull request watch ended","pullRequest":"github.com/<owner>/<repo-1>#D","reason":"stopped","minutes":5,"quietMinutes":5,"longestQuietMinutes":5,"wakes":1,"reads":2,"partial":false}wakes: 0, reads: 1over 13 and 14 minutes: the single read is the baseline, taken after the checks went green, and every later sweep finds the fingerprint unchanged.How often
The first read lands uniformly within one sweep interval after the watch call. With
rseconds of CI left when the agent calls the watch, the chance of a lost wake is about1 - r/121forr < 121, and zero above. The two repositories here had 30 to 86 seconds left, so between one watch in three and three in four. Any repository whose CI ends within two minutes of the watch call is exposed on every PR.Open pull requests
#37494 and #37492 shorten the delay for watches whose last read saw pending checks. A watch that has just been created has no last read and is not in the last sweep's groups, so its first read still waits for the sweep and still baselines whatever it finds. The cause described in this issue is unchanged on
mainas of12b9845d1c5aec2bfede36b313adfc5098a2fb93: none of the commits since the build above touchpullRequestWatch.ts,PullRequestWatchReactor.tsorPullRequestWatchService.ts.Taking the baseline at registration, from the summary
setWatchingalready reads and discards, would close the window.SwiftlyDaniel commentedon Nov 17, 2026
This comment was prepared by GPT-6.1 Sol (Codex), reporting an occurrence observed in Dani’s T3 Code session.
We observed a missed wakeup on self-hosted GitLab that appears consistent with this issue.
Environment: T3 Code Nightly
0.1.11-nightly.20261117.4825, macOS 27.2.1 arm64, Codex provider (gpt-6.1-sol), glab 1.130.0.Timeline on 17 November 2026, UTC:
watch_pull_requestregistered successfully, returningwatching: true, wasWatching: false.thread.pull-request-watch.sync.reason: "stopped", wakes: 0, reads: 2, partial: false.The agent ended its preceding turn at 15:56:51.001. The retained run history shows no watcher-triggered continuation before the user’s next message.
This timing fits the registration-to-first-read window described here. However, we have not captured the saved evaluator baseline or the exact checks returned by its first read, so the internal cause remains unconfirmed. A GitLab-specific read or refresh failure has not been ruled out.
A preceding MR also received no wake, but its checks had already completed before registration. We are keeping that separate because suppressing already-completed checks may be intentional.
Could this issue be reopened to investigate this additional GitLab occurrence?
Private repository URLs and thread identifiers have been omitted.
fix(server): baseline PR watches at registration so fast check comple…
fix(server): baseline PR watches at registration so fast check comple…
mertcanekiz commentedon Nov 18, 2026
Note
🤖 Claude Opus 5.5 responding on behalf of @mertcanekiz
Another live occurrence, in a variant the open fixes don't cover: watching a PR again on a head the last watch already reported passing. A new watch inherits that
passed: true. CI then re-runs on the same head (here because the PR was reopened) and passes before the first sweep, so nothing ever changes from the watch's point of view.Version:
0.1.8-nightly.20261109.4433, macOS, Claude provider, GitHub, no host event feed. The inheritance is unchanged onmain(8cf97b5f06).Steps:
unwatch_pull_request.dff7660. Here the base branch was deleted, GitHub closed the PR, and reopening it started a new run. A re-run, or a retarget that triggers CI, would do the same.watch_pull_request(watching: true, wasWatching: false) and ends its turn.Timeline (UTC, 2026-11-18)
dff7660starts (PR reopened)watch_pull_request, whilelintandtestare in progresslint,testpasspassed→passed, no change{"message":"pull request watch ended","pullRequest":"github.com/<owner>/<repo>#X","reason":"stopped","minutes":56,"quietMinutes":56,"longestQuietMinutes":56,"wakes":0,"reads":2,"partial":false}Final
endedWatch:{"startedAt":"2026-11-18T14:57:02.983Z","headSha":"dff7660…","failedChecks":[],"passed":true,"passedChecks":["lint","test","bundle","production","isler"],"owedChecks":[],"remarksThrough":"2026-11-18T14:57:02.983Z","remarkIds":[],"conflicting":false,"wakes":0,"checksAwaitedSince":null}Why
Orchestrator.ts#L5079-L5087seeds a new watch fromendedWatch: sameheadSha,passed,passedChecks("Watching again must not repeat check or conflict news the last watch told"). The head didn't move, so the first read findspassedalready true andpassedNow && !passednever fires. A first read that lands while the re-run is pending would clearpassed, and the pass would wake. So this is the same up-to-2-minute gap as this issue, plus one more way in.This matters for the fix proposed above. A baseline taken from the read
setWatchingalready makes would cover this case only if a pending read there replaces the inheritedpassed. And #32276 proposes reporting a pass once per head, which would make this case silent by design. The agent re-watched specifically to wait for the new run, so to it the pass is news. Which behavior do you want for a same-head re-run after a re-watch?fix(server): baseline PR watches at registration so fast check comple…
fix(server): baseline PR watches at registration so fast check comple…
fix(server): baseline PR watches at registration so fast check comple…
fix(server): baseline PR watches at registration so fast check comple…
fix(server): baseline PR watches at registration so fast check comple…
fix(server): baseline PR watches at registration so fast check comple…
fix(server): baseline PR watches at registration so fast check comple…
Bishi commentedon Nov 22, 2026
This still happens on 0.1.11 (server 0.1.11, Windows 10, Codex provider, GitHub, web client). It also shows that #39116 fixed the failure side of this window but not the pass side.
Timeline (UTC, one PR, no required checks, one check "Doc size"):
pull_request_readreturns Doc sizependingcompleted_at, conclusion successwatch_pull_request(until: "merged")and ends its turnFinal saved watch:
{"passed":true,"passedChecks":["Doc size"],"wakes":0,"checksPending":false,"until":"merged"}.Earlier in the same thread, a second PR's watch woke correctly ("checks passed") because its check finished about 25 minutes after the watch started. So the miss happens only when checks finish before the first read.
Cause (v0.1.11,
pullRequestWatch.ts): #39116 seedschecksPending: truefrom the agent's last read and exempts failures from the baseline (newlyFailed.length > 0 && (!checksBaseline || watch.checksPending === true)). The passed wake is still behind a bareif (!checksBaseline). A watch seeded withchecksPending: truewhose first read finds the checks green is a pending→passed change, and it should wake just as a pending→failed one does.mainhas the same guard today.Suggested fix: mirror the failure exemption, so the
checks-passedchange is reported when!checksBaseline || watch.checksPending === true.