Repository navigation
Delegated results: remind for linked children a stop lost, and tighten the reminder - #641
Merged
Merged
Conversation
…n the reminder - At start, read each linked environment's threads made before the start; a parent here that one names as delegatedFrom but no task follows gets one queued message (ids from the orphans, so a rerun or the next restart never repeats it). Retries while an environment does not answer or the parent is busy. - The startup check also finds parents with unread results but no reserved wake. - Results with no delivery state count as unread for a day; an open question holds the reminder, background work does not. - task_list: a linked child still running there is read there; one timeout marks the rest of that environment unknown; the orphan search says when it read only the newest threads. Fixes #625 Part of #386 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ults-followups # Conflicts: # FORK_CHANGES.md # apps/server/src/fork/forkLayers.ts
Owner
Author
Review of #641 (correctness and state lens): SAFEHead BlockingNone. Major
Minor
Checked and fine
|
… what is left - Each orphan gets its own key in the message and command ids; before a send the scan reads the parent's sent message ids and accepted receipts, so a grown set (late machine, moved one-day window after a restart) sends only the new ones. At most 10 orphans per message; the first scan's per-set id still counts. - A busy parent (also one waiting on a question) is no longer held: the queued message is read once its run ends. Orphans are worked out first, so a busy parent with none forces no rescan. - A later page that does not answer is its own state (cut-short): scanned again, and named in task_list's next. After the last scan a warning names the environments and parents still unchecked. The page-limit line says so. - Neutral wording (a failed record also leaves an orphan), a minute's clock tolerance between machines, a test for the held-start skip, and the upstream gap in the docs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Owner
Author
Fix for the correctness and state reviewHead Major 1: an orphan is never named twice
Major 2: no silent give-up
Minors
Tests fail on the old code
|
scuttletoby
added a commit
that referenced
this pull request
Nov 21, 2026
…n the reminder (#641) * Delegated results: remind for linked children a stop lost, and tighten the reminder - At start, read each linked environment's threads made before the start; a parent here that one names as delegatedFrom but no task follows gets one queued message (ids from the orphans, so a rerun or the next restart never repeats it). Retries while an environment does not answer or the parent is busy. - The startup check also finds parents with unread results but no reserved wake. - Results with no delivery state count as unread for a day; an open question holds the reminder, background work does not. - task_list: a linked child still running there is read there; one timeout marks the rest of that environment unknown; the orphan search says when it read only the newest threads. Fixes #625 Part of #386 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * FORK_CHANGES: link the delegated results follow-ups on their row Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Delegated orphans: name each orphan once, queue for busy parents, log what is left - Each orphan gets its own key in the message and command ids; before a send the scan reads the parent's sent message ids and accepted receipts, so a grown set (late machine, moved one-day window after a restart) sends only the new ones. At most 10 orphans per message; the first scan's per-set id still counts. - A busy parent (also one waiting on a question) is no longer held: the queued message is read once its run ends. Orphans are worked out first, so a busy parent with none forces no rescan. - A later page that does not answer is its own state (cut-short): scanned again, and named in task_list's next. After the last scan a warning names the environments and parents still unchecked. The page-limit line says so. - Neutral wording (a failed record also leaves an orphan), a minute's clock tolerance between machines, a test for the held-start skip, and the upstream gap in the docs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
scuttletoby
added a commit
that referenced
this pull request
Nov 21, 2026
…n the reminder (#641) * Delegated results: remind for linked children a stop lost, and tighten the reminder - At start, read each linked environment's threads made before the start; a parent here that one names as delegatedFrom but no task follows gets one queued message (ids from the orphans, so a rerun or the next restart never repeats it). Retries while an environment does not answer or the parent is busy. - The startup check also finds parents with unread results but no reserved wake. - Results with no delivery state count as unread for a day; an open question holds the reminder, background work does not. - task_list: a linked child still running there is read there; one timeout marks the rest of that environment unknown; the orphan search says when it read only the newest threads. Fixes #625 Part of #386 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * FORK_CHANGES: link the delegated results follow-ups on their row Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Delegated orphans: name each orphan once, queue for busy parents, log what is left - Each orphan gets its own key in the message and command ids; before a send the scan reads the parent's sent message ids and accepted receipts, so a grown set (late machine, moved one-day window after a restart) sends only the new ones. At most 10 orphans per message; the first scan's per-set id still counts. - A busy parent (also one waiting on a question) is no longer held: the queued message is read once its run ends. Orphans are worked out first, so a busy parent with none forces no rescan. - A later page that does not answer is its own state (cut-short): scanned again, and named in task_list's next. After the last scan a warning names the environments and parents still unchecked. The page-limit line says so. - Neutral wording (a failed record also leaves an orphan), a minute's clock tolerance between machines, a test for the held-start skip, and the upstream gap in the docs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
scuttletoby
added a commit
that referenced
this pull request
Nov 21, 2026
…n the reminder (#641) * Delegated results: remind for linked children a stop lost, and tighten the reminder - At start, read each linked environment's threads made before the start; a parent here that one names as delegatedFrom but no task follows gets one queued message (ids from the orphans, so a rerun or the next restart never repeats it). Retries while an environment does not answer or the parent is busy. - The startup check also finds parents with unread results but no reserved wake. - Results with no delivery state count as unread for a day; an open question holds the reminder, background work does not. - task_list: a linked child still running there is read there; one timeout marks the rest of that environment unknown; the orphan search says when it read only the newest threads. Fixes #625 Part of #386 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * FORK_CHANGES: link the delegated results follow-ups on their row Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Delegated orphans: name each orphan once, queue for busy parents, log what is left - Each orphan gets its own key in the message and command ids; before a send the scan reads the parent's sent message ids and accepted receipts, so a grown set (late machine, moved one-day window after a restart) sends only the new ones. At most 10 orphans per message; the first scan's per-set id still counts. - A busy parent (also one waiting on a question) is no longer held: the queued message is read once its run ends. Orphans are worked out first, so a busy parent with none forces no rescan. - A later page that does not answer is its own state (cut-short): scanned again, and named in task_list's next. After the last scan a warning names the environments and parents still unchecked. The page-limit line says so. - Neutral wording (a failed record also leaves an orphan), a minute's clock tolerance between machines, a test for the held-start skip, and the upstream gap in the docs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #625
Part of #386
Follow-ups from the review of #622. Every point of #625 is fixed below, except item 8 of the review (no change needed) and the table padding (reason given).
Major: crash gap for linked children
delegate_taskto a linked environment makes the thread there before it records the task here. A server that stopped in between left a thread there that no task here follows, and nothing told the parent.New
DelegatedOrphanScan, started a few minutes after server start:t3_thread_listpage by page (allProjects, 100 a page, at most 20 pages, 10 s a page). It stops at threads last updated over a day before the start.delegatedFromnames a thread here and that were made before this server started, with one minute of tolerance for clocks that differ between machines (CLOCK_SKEW_MS). A launch after the start is recorded by this server long before the first scan (3 min after start), so it is never raced.DelegatedResultsStore.remoteChildrenOf, a narrow SQL read, settled tasks included. A parent with orphans gets a queued message throughWaitDelivery(at most 10 orphans each). It names each thread and the tools that read and stop it.delegated-orphan:<parent>:<key>.<key>…. Before a send,DelegatedResultsStore.sentIdsreads the parent's message ids and accepted receipt ids with that prefix. Only orphans not named yet are sent, so a grown set (a machine that answers late, a one-day window that moved after a restart) sends only the new ones.WaitDeliverysends withmode: "queue"(queue_after_active), so the message waits behind the run and is read once it ends. Orphans are worked out before the parent's shell is read, so a busy parent with none costs nothing.cut-short), or a parent could not be read. After the last scan, a warning names the environments and parents still unchecked. The next start scans them again.server.waitForis off, while Linked environments is off, or on a held start (--hold-work). It never writes to a parent that is stopped, archived or deleted.Upstream's "record before launch" change ("recover remote delegated launches after source restart") is not ported. Upstream has not merged it yet and it is large. The scan closes the gap without touching upstream's launch path, and it also covers threads lost before this PR. When a sync brings it, slim the scan or drop it (noted in the scan's header and
docs/fork/README.md).The orphan message's wording is neutral ("delegate_task did not record it here"): a failed record leaves such a thread too, not only a stopped server.
Minors
task_list'sunrecordedtext says "an old row, or a wait that never acknowledged it".parentsWithUnreadResults), not only those with a reserved wake. Parents that already have a timer are skipped.stoppedThere: false) is read there. After one read on an environment times out, that environment's other tasks areunknownwithout waiting again.task_listnow reads up to 5 pages (500 threads) per environment, andnextnames any environment where it read only the newest 500, and separately any environment whose later page did not answer. The startup scan reads up to 2000 and logs "the page limit was reached" only for that limit; a page that did not answer is scanned again instead.watchedand re-triggered by its next run end;run.updatedis no trigger;includeSettled(exact key set);FiberMap.hasskip (checked by mutation: removing it fails "the startup scan leaves a parent to the check an event already moved later").waitForturned on later: no change, as the review concluded.task_listfor children per place andwait_listfor this agent's own slot use.FORK_CHANGES.mdpadding: not reverted. Re-padding the table back would change all rows again and conflict with every open PR that edits it. This PR appends its link to row 26 and keeps the existing column widths.General vs project
Everything here is general fork behaviour for any T3 install with linked environments. There are no project names, hosts or paths, and no settings beyond upstream's
server.waitFor. New code lives in fork-owned paths (apps/server/src/fork/delegatedResults/, the fork'stask_listtoolkit). It builds on upstream'scompletionDelivery,isLiveSubagentShell,getRecoveryThreadIds,t3_thread_listandPeerForwarding.State question (in tests)
WaitDelivery's message and receipt lookups dedupe them.retry, an environment that does not answer (fully), or an unreadable parent leads to another scan.delegated-orphans:<parent>:<set digest>) counts as sent. That format was never onmain.unreadable, retried, nothing sent);gone);held, never restarted); a busy one (queued);--hold-work: nothing read, nothing sent);waitForoff (off);DelegatedResultsStoreruns both SQL reads against the real schema in memory.Tests fail on the old code
Review fix round: this PR's previous sources with the new tests: 20 of 75 failed. One mutant of the new code per fix (no per-orphan record, no earlier-format check, busy held again, cut-short not rescanned, no final warning, busy checked before orphans, old wording, no clock tolerance, no held-start skip, cut-short as limit, receipts ignored, rejected receipts counted): 12 of 12 killed. This head: 75/75 pass; typecheck, lint, knip and format are clean.
First round:
Main's versions of
delegatedResults.ts,DelegatedResultReminder.ts,taskList.tsandtools.tswere put back and the suites were run against them: 22 of 62 tests failed. They span all five suites:delegatedResults: none remindable, background work not busy, "may" wording, orphan ids,orphansOfenvironment checks;DelegatedResultReminder: a wait task with no state, background work, a startup parent with no wake;DelegatedOrphanScan: 7 tests;taskList:stoppedThere, one timeout per environment, the partial note;With this branch: 62/62 pass, and typecheck, lint, knip and format are clean.
🤖 Generated with Claude Code