Skip to content

Run managed issue work through upstream delegation primitives #418

Closed

Description

@tannerpolley

Plan step 2 of #416. Owner decision: keep the managed policy, run on upstream execution.

Keep issue eligibility, root ownership, role selection, handoff and attempt policy in services and command modules that the issue code owns. Translate those decisions into upstream's existing launch, cancel and complete primitives.

Reduce fork edits in:

  • Orchestrator;
  • EffectWorker;
  • ProjectionStore;
  • the Codex and Claude adapters;
  • OrchestratorMcpService.

Keep the atomic receipt/event/outbox guarantees. Don't move work into an after-the-fact observer that can lose claims.

Success: fewer changed lines in those upstream files and fewer repeat conflict paths at the next sync. Do this after the sync sub-issue.

Activity

  1. tannerpolley commented on Nov 17, 2026

    @tannerpolley
    OwnerAuthor

    Design: managed policy over upstream delegation

    Pinned inspection: fork main 1e072987b65fc34e7099dc459c33d57c8309252f; upstream nightly v0.1.10-nightly.20261116.4800, commit 7d32075e3757ded707914e2450c3e392a9665765. Main was clean. Read-only source/history/issue inspection; no runtime tests or live userdata reads.

    Authority: #416 findings, owner decisions, #417 replacement ledger, and #429 triage with its correction.

    Outcome and boundaries

    Retain managed queue, issue eligibility, repository ownership, attempt reuse, role selection, separate reader/worker pools, Writer owner routing, suspension and handoff. Use upstream delegation for execution and delivery. No native-provider-subagent substitution for managed attempts. No new workflow, plugin framework, status vocabulary, issue-host work, reset-credit redesign or sidebar redesign. Codex/Claude and web/desktop are the fork's target surfaces; preserve common execution behavior for other adapters and existing remote delegation.

    This is consolidation, not a new execution engine. Already today, IssueWorkService.startWorker dispatches through ThreadManagement and calls ThreadLaunchService.prepareRun; makeIssueWorkCommands receives dispatchDelegatedTaskRequest. Preserve these paths and remove remaining parallel policy/result decisions. Do not route internal commands through MCP or split an atomic command into successive service calls.

    Current shared-file footprint

    Measured with git diff --numstat 7d32075e3757ded707914e2450c3e392a9665765 1e072987b65fc34e7099dc459c33d57c8309252f -- <six paths>. These are whole-file additions/deletions relative to the synced nightly, not issue-only line counts or promised deletions. Tests are excluded.

    File under apps/server/src Added Deleted Current fork responsibilities and disposition
    orchestration-v2/Orchestrator.ts 1,548 685 Issue/repository command wiring, root-scoped admission/locking, queue continuation; thread-owned completion cohort support; recovery/result selection and request routing, plus unrelated fork features. Move policy into existing IssueWorkCommands/RepositoryWorkCommands/IssueQueueCommands and issue-owned completion policy. Keep upstream dispatch, event collection and commit ownership.
    orchestration-v2/EffectWorker.ts 359 16 issue.change.execute, issue.suspension.interrupt/notify, issue.status.refresh, issue.github.comment, usage-sprint notification and keyed issue-write locks, alongside unrelated runtime additions. Move issue handlers/locks into an issue-owned executor; retain upstream claiming, retry, cancellation, lane/drain and settlement mechanics.
    orchestration-v2/ProjectionStore.ts 450 43 Issue reads/revision, routed-request reads, thread-owned completion recovery discovery, usage recovery, native-child shell metadata and title refresh. Extend existing issue-owned read/projection modules; keep shared indexed selectors and generic delivery recovery. Do not sweep title/native-shell/usage changes into this issue.
    orchestration-v2/Adapters/CodexAdapterV2.ts 477 124 managedIssue/modelRoles instruction inputs and managed-worker tool policy; also approval prompts, plan artifacts, native-child metadata, command output and stop/release handling. Extract only managed policy assembly; retain protocol translation and unrelated fixes.
    orchestration-v2/Adapters/ClaudeAdapterV2.ts 365 44 managedIssue/modelRoles query overrides, policy-key inputs and tool restrictions; also native/background ownership, output paths, question/cancellation/wake attachment behavior. Extract managed policy while preserving SDK lifecycle behavior.
    mcp/OrchestratorMcpService.ts 352 202 Managed issue validation/start/link, repository methods, model/usage policy, ancestor-send guard, task result/branch/worktree/request reporting. Extend ManagedIssueDelegation, RepositoryOrchestrationMcp and delegateTaskTarget; upstream remains the task/wait/read/cancel facade.

    Total: +3,551 / −1,114, 4,665 changed lines. At each step remeasure against this same upstream commit and classify remaining hunks by owner. Acceptance requires reduced managed-policy edits in these files, not arbitrary percentage targets or moving the whole engine to a fork file. Repeat-conflict reduction can only be assessed at a subsequent sync.

    Decisions mapped to existing primitives

    All upstream references below are pinned to the nightly:

    • Start/admit: Orchestrator.dispatchDelegatedTaskRequest, command delegated_task.request, and ThreadLaunchService.prepareRun. Issue-owned planners supply validated issue/attempt/role/workspace decisions into the same event/effect collectors. Runless managed admission remains a small extension; do not invent a parent run.
    • Stop/suspend: upstream delegated-tasks.stop in EffectWorker.layerExecutor and provider-turn interruption. Issue modules decide eligibility/suspension and emit the existing stop intent; shared execution owns cancellation. Preserve immediate-parent authority, pending request disposition and worktree retention.
    • Completion: upstream readDelegatedCompletionPlanRecords, planDelegatedCompletionDelivery, dispatchDelegatedTaskCompletionDeliveryResolution and dispatchDelegatedTaskWakePolicy in Orchestrator. One delivery planner for ordinary and managed tasks. Keep a narrow cohort-owner accessor for run-owned versus thread-owned state, currently completionCohort/completionCohortEvent. Extract issue closeout effects from ForkDelegatedCompletion; keep necessary provider-recovery mapping as a policy input, not a second delivery algorithm.
    • What counts as finished: RestartBackgroundNote.delegatedTaskRuns / delegatedTaskAwaitsWork / delegatedTaskResultRun and subagentProjection.delegatedTaskProgress. Preserve upstream's untagged descendant-completion wake hold before applying fork promotion/usage-recovery mapping. MCP status, wait termination and completion must consume the same result/progress decision.
    • Reads and persistence: upstream ProjectionStore.getThreadRecords, EventSink.commitCommand, and EffectOutbox. IssueWorkProjection/IssueWorkStore and issue-owned SQL helpers own issue state, within the existing projection transaction.
    • Provider boundaries: retain makeClaudeQueryOptions, claudeMcpQueryOverrides, claudeEffectiveQueryPolicyKey and Codex session configuration as protocol consumers. Compose managed instructions/tool restrictions outside adapters using existing McpProviderSession policy inputs. Role or policy changes must still invalidate Claude's effective query policy key. Upstream ProviderSessionManager's writeReleasedRosterEvents/writeReleasedToolCallEvents/writeReleasedSubagentEvents remain release owners.
    • MCP: preserve upstream McpToolAccess.routed, McpSchema.EnabledWhen and McpInvocationContext.hiddenFromOutsideClients; extraction must retain registration metadata and handler authority checks. Preserve pageIncludesTerminalTaskResult acknowledgement semantics: prompt-only/truncated reads are not result delivery.

    Use existing owned modules first: IssueWorkService, IssueEligibility, IssueWorkDispatch, IssueWorkCommands, RepositoryWorkCommands, IssueQueueCommands, IssueWorkRecovery, IssueWorkProjection, IssueCommentEffects, ManagedIssueDelegation, RepositoryOrchestrationMcp, delegateTaskTarget and DelegatedRequestRouting. A small new issue-effects executor is justified; a general extension registry is not.

    Atomicity and concurrency invariants

    1. Keep external issue reads before planning locks, as IssueWorkDispatch does. Missing/failed relationship reads cannot admit work. Recheck local root ownership/revision, lineage, suspension, attempt identity and pool capacity under the existing root-scoped lock; retain repository handoff lock ordering.
    2. A managed admission's claim/attempt metadata, child/task/run events, projections, command receipt and required outbox intents commit together through EventSink. EventSink reserves the receipt, appends/applies events, enqueues effects and finalizes the receipt in one SQL transaction. Duplicate command IDs return the committed result and cannot create a second worker or consume a second slot.
    3. No claim writer, queue subscriber or completion observer after commit may become the authority for admission. Existing reactors may request idempotent commands; they cannot repair a deliberately non-atomic claim.
    4. Completion result, cohort generation/reservation, queued wake message and task membership must agree in committed state. Preserve both cohort owners. Selective reads must retain the spawning run AND queued delivery run/message/reserved siblings. A second finisher joins the queued message without replacing its ID/generation/reservation.
    5. “Claimed” is not “delivered.” Mark delivery/acknowledgement only from the existing accepted-delivery/read protocol. Failed/held/cancelled wakes must remain recoverable or explicitly disposed with a reason; no silent success. Late owner follow-ups need delivery identity per result, preserving An issue owner's result from a wake-up run is dropped when its task already published a result #396.
    6. Outbox execution is retryable, not an exactly-once external side effect. Keep deterministic IDs, issue comment markers/receipts and per-issue write serialization. GitHub writes cannot be atomic with SQLite. Preserve stale-attempt suppression and restart reconciliation. Publish events/notify workers after commit; rollback leaves no partial claims or wake membership.

    Upgrade and recovery

    Recommendation: retain persisted contracts for this refactor, including nullable parentRunId, both cohort locations, command/task/run/attempt IDs, message IDs, delivery generations and existing outbox request types. Old accepted command receipts must replay even if GitHub eligibility changes. Do not rebuild claims from current live runs or synthesize fresh launches from pending effects.

    Test upgrade/restart using disposable fixtures produced in the baseline format: queued/unadmitted readers and workers; preparing workspace; running worker interrupted by restart; pending owner question; finished-undelivered child; claimed queued wake with two siblings; delivery interrupted after commit; already acknowledged/disposed result; late owner follow-up; handoff/replaced attempt; held queue and usage recovery continuation. Recovery must neither free a live slot nor resurrect a retired worker, lose the original brief, duplicate a wake, or acknowledge an unseen result.

    Preserve existing ForkMigrations identities and historical event decoding. No schema migration is expected. If implementation discovers a required format change, stop for a revised design: additive fork migration and old-format reader first, cutover second, retirement only after compatibility proof. Do not delete old receipts/events/outbox rows. Downgrade is not assumed safe after any new persisted format.

    Deployment pause/drain is an owner choice, not a substitute for restart correctness. No live-data experimentation; fixture-based recovery proof first. “Live upgrade” cannot promise preservation of provider subprocesses across server restart.

    Small separately mergeable sequence

    Each numbered step is one bounded sub-issue, with small commits/PRs inside it if needed. Dependencies are sequential; no simultaneous editing of shared files.

    1. Admission and policy seam. Preserve current behavior; isolate remaining eligibility/root/attempt/handoff/role decisions in owned planners, passing their decisions through existing dispatch collectors. Reuse launch/stop primitives; no schema changes. Extend existing IssueWorkStore/IssueEligibility, IssueReviewDelegation and managed delegation tests for duplicate requests, pool contention, ownership changes, Writer exemption and request reuse. Add transaction failure injection at EventSink only where existing tests do not prove rollback of the managed claim plus child/outbox. Done: one committed claim owner, same results/authority, narrower Orchestrator/MCP policy hunks, all focused tests pass.
    2. Issue effect and projection boundary. Extract issue effect handlers and locks; consolidate issue-only reads/projection updates through existing owned modules without moving commit ownership. Preserve lane concurrency, retry classification, drain, comment markers and snapshot freshness. Extend IssueCommentEffects, IssueWorkProjection/Store and relevant EffectWorker/EventSink tests: retry after remote write, stale attempt, rollback and startup replay. Done: shared worker retains execution mechanics with small typed hooks, no after-commit claim writes, fewer EffectWorker/ProjectionStore issue hunks.
    3. Single completion/recovery path; Make child and worker state visible and completion wakes reliable #429 backend wake work. Consolidate progress/result selection and keep one upstream planner for both cohort owners. Move issue closeout into issue policy; preserve Sync with upstream nightly v0.1.10-nightly.20261116.4800, with a behavior replacement ledger #417 selective-reader fixes, follow-up hold, One definition of "done" for a delegated task: task_status, waits and the parent wake agree #350 background-work hold and An issue owner's result from a wake-up run is dropped when its task already published a result #396 late result behavior. Extend existing DelegatedCompletionDelivery(.fork), ManagedFollowupRecovery, ProjectionRecovery and ProviderRuntimeRecoveryService tests only where gaps remain. Cases: concurrent siblings for both owners, restart at pending/claimed/delivered boundaries, untagged descendant wake, background task, failed workspace preparation, stopped/archived/deleted parent and explicit result read. Use persisted events/Deferred/outbox drain, no sleeps. Done: task_status, wait and parent wake agree; all eligible results are delivered or have an explicit recoverable/disposed outcome; no duplicate logical delivery; old-format fixtures recover. If a new Make child and worker state visible and completion wakes reliable #429 defect is found, demonstrate it failing before its fix.
    4. Provider/MCP policy boundary and integrated proof. Move managed instruction/tool policy assembly out of Codex/Claude adapters; finish thin MCP composition. Preserve native execution fixes and Writer routing. Extend ManagedIssueDelegation, relevant adapter fork suites, MCP toolkit registration/outside-client tests and result-read acknowledgement cases. If model-facing text changes, follow evals/README.md and add a failing behavior case first. Done: managed policy is outside adapters, registration succeeds with EnabledWhen intact, upstream ordinary delegation/remote behavior remains covered, focused suites and required check:changed pass. One Replay-backed web/desktop pass demonstrates start/queued work, question, cancel, sibling completion and restart result recovery using isolated state; primary integrator owns this pass. Capture any user-reproducible new bug with scenarios:prove as required.

    Run checks through agent-heavy; no repo-wide suite. Preserve existing regression cases rather than copying them across layers. Publish before/after numstat and a residual-hunk ledger on this issue at closeout. Historical sync test counts are evidence from #417, not tests rerun by this design.

    #429 division

    Inside #418: shared backend definition of done; final-result parent wake; cohort correctness; restart discovery/retry of undelivered results; durable delivery-failure facts using existing state. Preserve previous fixes, prove gaps before adding new behavior. #417's two cohort failures and follow-up hold are already repaired and are acceptance regressions here, not new feature work.

    Stays in #429: visible per-child/issue queued/running/question/approval/done state; direct owner response controls; thread/branch links; “In progress” derived from actual worker activity; visible presentation of missed/failed wakes; background-task/sidebar visibility. Those UI consumers must use #418's persisted facts rather than a second completion state machine. Also retain the queued-reader brief-loss investigation as its own bug: inspect queued initial-message/context-transfer/start recovery, reproduce first, and fix at that owner. No evidence establishes when it began; do not attribute it to #426/#427. If its reproduction reaches code #418 changes, land the focused regression/fix before step 3 and share it.

    #429 should audit #43/#84, #36/#252, #363 and #344/#345 against current UI behavior; this design does not assert those historical fixes still work. #350/#391/#396 are closed, with relevant behavior retained as backend acceptance cases. No UI redesign or speculative watchdog is required by #418.

    Owner decisions (options and recommendation)

    1. Upgrade compatibility: retain both persisted cohort owners and IDs during extraction (recommended) / convert runless work to synthetic parent runs with a separate migration. The latter materially enlarges risk and is not needed for the desired reduction.
    2. Make child and worker state visible and completion wakes reliable #429 scope: put completion correctness/recovery in Run managed issue work through upstream delegation primitives #418 and visibility/brief-loss work in Make child and worker state visible and completion wakes reliable #429 (recommended) / keep all wake work in Make child and worker state visible and completion wakes reliable #429 and block step 3 on it. Do not implement competing wake fixes.
    3. Deployment: pause new admissions and let active workers drain before the first rollout (recommended, still test in-flight restart recovery) / allow normal restart with active workers after recovery fixtures pass. Restarting the owner's server requires separate operational approval.

    Existing decisions to keep managed policy, roles and Writer are settled; do not ask them again. These recommendations are proposed design choices, not recorded owner approval. No implementation was performed.

  2. tannerpolley commented on Nov 17, 2026

    @tannerpolley
    OwnerAuthor

    Split handoff

    The design is large enough for four implementors in sequence. propose_issue_change(type: split) was attempted with request ID design-418-split-20261117-v1, but returned invalid_request: “Only a managed thread in this repository can propose issue changes.” No sub-issues were created. The registered repository orchestrator should submit the split using the exact step scopes/tests/done criteria in the canonical plan:

    1. Isolate managed admission policy behind upstream delegation.
    2. Move issue effects and reads out of shared execution files (depends on 1).
    3. Unify managed completion and restart recovery with upstream delegation (depends on 2).
    4. Isolate provider and MCP managed policy and verify delegation integration (depends on 3).

    Use no milestone unless the owner selects one. Owner choices are listed in the plan; they remain proposed, not approved. No implementation has started.

  3. tannerpolley commented on Nov 17, 2026

    @tannerpolley
    OwnerAuthor

    Owner decisions (2026-11-17)

    The owner took all three recommendations:

    1. Upgrade compatibility: retain both persisted cohort owners and their IDs during extraction. No synthetic-run migration.
    2. Make child and worker state visible and completion wakes reliable #429 scope: completion correctness and restart recovery live in Run managed issue work through upstream delegation primitives #418. Visibility, direct answer controls and the queued-brief-loss bug stay in Make child and worker state visible and completion wakes reliable #429, built on Run managed issue work through upstream delegation primitives #418's persisted facts.
    3. Deployment: before the first rollout, pause new admissions and let active workers drain, while still proving in-flight restart recovery with fixtures.

    The four steps become top-level issues, each blocked by the one before it. The root can only start workers on top-level issues. Each body says it is part of #418.

  4. tannerpolley commented on Nov 18, 2026

    @tannerpolley
    OwnerAuthor

    Closeout (2026-11-18)

    All four steps are merged:

    Step Issue PR Review
    1. Admission policy #433 #443 Checker: merge
    2. Issue effects and reads #434 #459 Checker: merge
    3. Completion and restart recovery #435 #471 Checker: merge. Fixed a late unseen read acknowledging run-less results (the upstream pingdotgg#32573 rule)
    4. Managed policy outside the adapters #436 #475 Checker: merge. Composed instructions are identical for every role, apart from two blank-line fixes

    The numstat against 7d32075e37 and the residual-hunk ledger are in the PR #475 body. Orchestrator.ts, EffectWorker.ts and ProjectionStore.ts are smaller in each step's measured numstat.

    Rollout gate, from the owner decision. Builds 2.12.152 (#471) and later change how results are handed back. Before installing them: pause repository admissions, let running workers finish, install, then resume.

    Not done, all optional review notes, tracked here rather than as work:

    #440 and #441 were blocked on this issue. #440 is still blocked on #455 (done) and is ready to schedule.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions