Skip to content

Two delegated follow-ups that finish in the same wake run settle only one; the other stays waiting #39204

Open

Description

@Dropje97

What happened

Two follow-ups sent to the same delegated child both finished, but only one settled. The other stayed waiting with delivery pending and completedAt: null. The parent kept listing it as running background work, and the Lineage panel showed the child as working after it had finished and been archived.

Diagnosis

Follow-ups A and B each start a nested delegate. B's nested completion opens a wake run, and a later nested completion for A is steered into that run automatically. delegatedTaskRuns (RestartBackgroundNote.ts) follows delegatedCompletion.parentRunId on every user message in a run, including one that joined as a steer, so A and B both select that wake as their result run. A publishes a subagent_result transfer from it. finalizeAppOwnedSubagentTask (Orchestrator.ts) looks for an existing transfer by source run and thread pair, not by task, finds A's transfer, and skips B. Later runs do not change B's selected result run, so B never settles.

Steps to reproduce

  1. Start an async delegated child and wait for its first result.
  2. Send follow-up A. The child starts a nested delegate and ends its turn.
  3. Send follow-up B before A settles. The child starts another nested delegate and ends its turn.
  4. Let A's nested delegate wake the child. During that wake the child starts one more nested delegate for A.
  5. Let B's nested delegate complete and open a new wake. While it runs, let A's remaining nested delegate complete, so its notice steers into that wake. This needs a provider that steers delegated completions automatically; Codex does.
  6. Let the wake finish and read both tasks.

We have a deterministic regression test for these steps (scripted Codex adapter, real orchestration services, projections and workers) that fails on main, and can share it.

Version

Observed on 0.1.11-nightly.20261118.4890 with a local change that lets Claude steer delegated completions into a running turn (stock Claude queues them). Reproduced on unmodified main at 2526691ea5 with the test above; main at 8d5e0b241d has the same code.

Environment

Linux x86-64, self-hosted t3 serve. Observed with a Claude parent and child; reproduced with a scripted Codex adapter.

Evidence

| | A | B |
|---|---|---|
| Selected result run | the wake | the wake |
| Transfer from the wake | yes | none |
| Final task row | `completed`, delivered | `waiting`, delivery `pending` |

Related issues

#23300 handles a related collision: a queued follow-up promoted to Steer shares the joined run's transfer. Merged locally with main, it does not change this case; its moved-message lookup applies only to a cancelled queued run. #17512 / #22709 and #28448 / #37901 fixed other ways a delegated task stays waiting.

Fix applied or workaround

No workaround found; from the source, task_cancel and task_status leave the row waiting. A local change fixes the reproduction: record the destination task on each result transfer (targetTaskId) and deduplicate on it in the finalizer and in the MCP result lookup, while explicitly promoted follow-ups keep sharing the joined run's transfer. Tested on main with #23300 merged: the regression test and the steering, delegated-completion and Orchestrator* suites pass (287 tests).

Filed by

Measured and written by Claude (Opus 5.5) in T3 Code; posted with the owner's approval.

Activity

  1. Dropje97 commented on Nov 19, 2026

    @Dropje97
    Author

    The regression test for the steps above, as a diff against main (8d5e0b241d). It adds one case to SteeringCompletion.integration.test.ts, delivers both follow-up results when a nested completion steers into another wake, and fails on main with B left waiting. Run it from apps/server with pnpm exec vp test run src/orchestration-v2/SteeringCompletion.integration.test.ts -t "delivers both follow-up results".

    Regression test (330-line diff)
    diff --git a/apps/server/src/orchestration-v2/SteeringCompletion.integration.test.ts b/apps/server/src/orchestration-v2/SteeringCompletion.integration.test.ts
    index ca79456323..073b4033b6 100644
    --- a/apps/server/src/orchestration-v2/SteeringCompletion.integration.test.ts
    +++ b/apps/server/src/orchestration-v2/SteeringCompletion.integration.test.ts
    @@ -15,10 +15,12 @@ import {
       ProviderThreadId,
       ProviderTurnId,
       PROVIDER_SEND_TURN_MAX_INPUT_CHARS,
    +  type RunId,
       ScheduledTaskId,
       ThreadId,
       TurnItemId,
       type ModelSelection,
    +  type OrchestrationV2ConversationMessage,
       type OrchestrationV2DomainEvent,
     } from "@t3tools/contracts";
     import {
    @@ -602,7 +604,7 @@ const nextTurnSelectionHarness = Effect.fn("nextTurnSelectionHarness")(function*
                   providerSessionId: input.providerSessionId,
                   appThreadId: threadId,
                   ownerNodeId: null,
    -              nativeThreadRef: { driver, nativeId: "native-thread", strength: "strong" },
    +              nativeThreadRef: { driver, nativeId: `native:${threadId}`, strength: "strong" },
                   nativeConversationHeadRef: null,
                   status: "idle",
                   firstRunOrdinal: null,
    @@ -706,9 +708,302 @@ const nextTurnSelectionHarness = Effect.fn("nextTurnSelectionHarness")(function*
         yield* (yield* ThreadCommandExecutor.ThreadCommandExecutor).withLock(threadId, Effect.void);
         return threadId;
       });
    -  return { cwd, events, started, steered, steeredSelections, interrupted, layer, startFirstTurn };
    +  return {
    +    adapter,
    +    cwd,
    +    events,
    +    started,
    +    steered,
    +    steeredSelections,
    +    interrupted,
    +    layer,
    +    startFirstTurn,
    +  };
     });
     
    +it.effect("delivers both follow-up results when a nested completion steers into another wake", () =>
    +  Effect.gen(function* () {
    +    const name = "overlapping-nested-completions";
    +    const harness = yield* nextTurnSelectionHarness(name);
    +    const layer = ProviderReplayHarness.layerWithRegistry(
    +      { name },
    +      ProviderAdapterRegistry.layerSingle(harness.adapter),
    +      { runEffectWorker: false, runContinuationWorker: true },
    +    );
    +    yield* Effect.gen(function* () {
    +      const orchestrator = yield* Orchestrator.OrchestratorV2;
    +      const worker = yield* EffectWorker.OrchestrationEffectWorkerV2;
    +      const executor = yield* ThreadCommandExecutor.ThreadCommandExecutor;
    +      const parentThread = yield* harness.startFirstTurn;
    +      const read = orchestrator.getThreadProjection;
    +      const sequence = orchestrator.getThreadEventSequence;
    +      const waitForEvent = Effect.fnUntraced(function* (
    +        threadId: ThreadId,
    +        afterSequence: number,
    +        predicate: (event: OrchestrationV2DomainEvent) => boolean,
    +      ) {
    +        yield* orchestrator.streamStoredEventsFrom({ threadId, afterSequence }).pipe(
    +          Stream.filter(({ event }) => predicate(event)),
    +          Stream.take(1),
    +          Stream.runDrain,
    +        );
    +        yield* executor.withLock(threadId, Effect.void);
    +      });
    +      const start = Effect.fnUntraced(function* (threadId: ThreadId, runId: RunId) {
    +        yield* orchestrator.resumeQueuedRuns;
    +        yield* worker.drain();
    +        yield* waitForEvent(
    +          threadId,
    +          0,
    +          (event) =>
    +            event.type === "provider-turn.updated" &&
    +            event.runId === runId &&
    +            event.payload.status === "running",
    +        );
    +        return harness.started.find((turn) => turn.runId === runId)!;
    +      });
    +      const finish = Effect.fnUntraced(function* (
    +        turn: ProviderAdapter.ProviderAdapterV2TurnInput,
    +        text: string,
    +      ) {
    +        const now = yield* DateTime.now;
    +        const providerTurn = (yield* read(turn.threadId)).providerTurns.find(
    +          (candidate) => candidate.runAttemptId === turn.attemptId,
    +        )!;
    +        const after = yield* sequence(turn.threadId);
    +        yield* Queue.offerAll(harness.events, [
    +          {
    +            type: "turn_item.updated",
    +            driver,
    +            turnItem: {
    +              id: TurnItemId.make(`reply:${turn.runId}`),
    +              threadId: turn.threadId,
    +              runId: turn.runId,
    +              nodeId: turn.rootNodeId,
    +              providerThreadId: providerTurn.providerThreadId,
    +              providerTurnId: providerTurn.id,
    +              nativeItemRef: null,
    +              parentItemId: null,
    +              ordinal: 1,
    +              status: "completed",
    +              title: null,
    +              startedAt: now,
    +              completedAt: now,
    +              updatedAt: now,
    +              type: "assistant_message",
    +              messageId: MessageId.make(`reply:${turn.runId}`),
    +              text,
    +              streaming: false,
    +            },
    +          },
    +          {
    +            type: "provider_turn.updated",
    +            driver,
    +            providerTurn: { ...providerTurn, status: "completed", completedAt: now },
    +          },
    +          {
    +            type: "turn.terminal",
    +            driver,
    +            providerThreadId: providerTurn.providerThreadId,
    +            providerTurnId: providerTurn.id,
    +            runOrdinal: turn.runOrdinal,
    +            status: "completed",
    +            failure: null,
    +            threadDisposition: "reusable",
    +          },
    +        ]);
    +        yield* waitForEvent(
    +          turn.threadId,
    +          after,
    +          (event) =>
    +            event.type === "run.updated" &&
    +            event.payload.id === turn.runId &&
    +            event.payload.status === "waiting",
    +        );
    +        yield* worker.drain();
    +        yield* waitForEvent(
    +          turn.threadId,
    +          after,
    +          (event) =>
    +            event.type === "run.updated" &&
    +            event.payload.id === turn.runId &&
    +            event.payload.status === "completed",
    +        );
    +        yield* orchestrator.recoverDelegatedTasks;
    +      });
    +      const delegate = Effect.fnUntraced(function* (
    +        from: ProviderAdapter.ProviderAdapterV2TurnInput,
    +        label: string,
    +      ) {
    +        yield* orchestrator.dispatch({
    +          type: "delegated_task.request",
    +          createdBy: "agent",
    +          creationSource: "mcp",
    +          commandId: CommandId.make(`delegate:${label}`),
    +          parentThreadId: from.threadId,
    +          parentRunId: from.runId,
    +          parentNodeId: from.rootNodeId,
    +          task: label,
    +          modelSelection: runSelection,
    +          runtimeMode: "full-access",
    +          interactionMode: "default",
    +          completionWake: "always",
    +        });
    +        const task = (yield* read(from.threadId)).subagents.find((row) => row.prompt === label)!;
    +        const threadId = task.childThreadId!;
    +        const runId = (yield* read(threadId)).runs[0]!.id;
    +        return { task, turn: yield* start(threadId, runId) };
    +      });
    +      const send = Effect.fnUntraced(function* (
    +        threadId: ThreadId,
    +        label: string,
    +        followup: boolean,
    +      ) {
    +        const messageId = MessageId.make(`send:${label}`);
    +        yield* orchestrator.dispatch({
    +          type: "message.dispatch",
    +          createdBy: followup ? "agent" : "user",
    +          creationSource: followup ? "mcp" : "web",
    +          ...(followup ? { senderThreadId: parentThread } : {}),
    +          commandId: CommandId.make(`send:${label}`),
    +          threadId,
    +          messageId,
    +          text: label,
    +          attachments: [],
    +          dispatchMode: { type: "start_immediately" },
    +        });
    +        const run = (yield* read(threadId)).runs.find(
    +          (candidate) => candidate.userMessageId === messageId,
    +        )!;
    +        return { run, turn: yield* start(threadId, run.id) };
    +      });
    +      const noticeFrom = Effect.fnUntraced(function* (
    +        threadId: ThreadId,
    +        from: RunId,
    +        taskId: NodeId,
    +        after: number,
    +      ) {
    +        const isNotice = (
    +          message: Pick<OrchestrationV2ConversationMessage, "delegatedCompletion">,
    +        ) =>
    +          message.delegatedCompletion?.parentRunId === from &&
    +          message.delegatedCompletion.taskIds.includes(taskId);
    +        yield* waitForEvent(
    +          threadId,
    +          after,
    +          (event) => event.type === "message.updated" && isNotice(event.payload),
    +        );
    +        return (yield* read(threadId)).messages.find(isNotice)!;
    +      });
    +
    +      const initialParent = harness.started[0]!;
    +      const original = yield* delegate(initialParent, "original child");
    +      const childThread = original.turn.threadId;
    +      const beforeOriginal = yield* sequence(parentThread);
    +      yield* finish(original.turn, "initial result");
    +      const originalNotice = yield* noticeFrom(
    +        parentThread,
    +        initialParent.runId,
    +        original.task.id,
    +        beforeOriginal,
    +      );
    +      assert.equal(originalNotice.runId, initialParent.runId);
    +      yield* worker.drain();
    +      yield* finish(initialParent, "original delegation done");
    +
    +      const parentA = yield* send(parentThread, "send A", false);
    +      const a = yield* send(childThread, "follow-up A", true);
    +      const nestedA = yield* delegate(a.turn, "A first delegate");
    +      yield* finish(a.turn, "A is waiting");
    +      yield* finish(parentA.turn, "A sent");
    +      const parentB = yield* send(parentThread, "send B", false);
    +      const b = yield* send(childThread, "follow-up B", true);
    +      const nestedB = yield* delegate(b.turn, "B delegate");
    +      yield* finish(b.turn, "B is waiting");
    +      yield* finish(parentB.turn, "B sent");
    +
    +      const beforeWakeA = yield* sequence(childThread);
    +      yield* finish(nestedA.turn, "A first delegate done");
    +      const aOpening = yield* noticeFrom(childThread, a.run.id, nestedA.task.id, beforeWakeA);
    +      const wakeA = yield* start(childThread, aOpening.runId!);
    +      const nestedA2 = yield* delegate(wakeA, "A second delegate");
    +      yield* finish(wakeA, "A is still waiting");
    +
    +      const beforeWakeB = yield* sequence(childThread);
    +      yield* finish(nestedB.turn, "B delegate done");
    +      const bOpening = yield* noticeFrom(childThread, b.run.id, nestedB.task.id, beforeWakeB);
    +      const wakeB = yield* start(childThread, bOpening.runId!);
    +      const beforeSteer = yield* sequence(childThread);
    +      yield* finish(nestedA2.turn, "A second delegate done");
    +      const steered = yield* noticeFrom(childThread, wakeA.runId, nestedA2.task.id, beforeSteer);
    +      yield* worker.drain();
    +      const child = yield* read(childThread);
    +      const joined = child.runs.find((run) => run.id === wakeB.runId)!;
    +      assert.equal(joined.userMessageId, bOpening.id);
    +      assert.isUndefined(joined.delegatedTaskId);
    +      assert.equal(steered.runId, joined.id);
    +      assert.include(harness.steered.at(-1), nestedA2.task.id);
    +      assert.equal(
    +        child.subagents.find((task) => task.id === nestedA2.task.id)?.completionDelivery?.state,
    +        "delivered",
    +      );
    +
    +      const followups = [a, b].map(({ run }) => run.delegatedTaskId!);
    +      assert.equal(new Set(followups).size, 2);
    +      const held = (yield* read(parentThread)).subagents.filter((task) =>
    +        followups.includes(task.id),
    +      );
    +      assert.deepEqual(
    +        held.map((task) => task.result),
    +        [null, null],
    +      );
    +      const beforeResults = yield* sequence(parentThread);
    +      const result = "A and B finished in the joined turn";
    +      yield* finish(wakeB, result);
    +      const after = yield* read(parentThread);
    +      const transfers = after.contextTransfers.filter(
    +        (transfer) =>
    +          transfer.type === "subagent_result" &&
    +          transfer.sourceThreadId === childThread &&
    +          transfer.sourcePoint.runId === wakeB.runId,
    +      );
    +      assert.deepEqual(
    +        {
    +          tasks: followups.map((id) => {
    +            const task = after.subagents.find((row) => row.id === id)!;
    +            return [task.status, task.result, task.completionDelivery?.state];
    +          }),
    +          targets: transfers.map((transfer) => transfer.targetRunId).sort(),
    +        },
    +        {
    +          tasks: [
    +            ["completed", result, "claimed"],
    +            ["completed", result, "claimed"],
    +          ],
    +          targets: [parentA.run.id, parentB.run.id].sort(),
    +        },
    +      );
    +
    +      const parentNotices = yield* Effect.all([
    +        noticeFrom(parentThread, parentA.run.id, followups[0]!, beforeResults),
    +        noticeFrom(parentThread, parentB.run.id, followups[1]!, beforeResults),
    +      ]);
    +      const parentRunIds = new Set(parentNotices.map((message) => message.runId));
    +      const parentRuns = (yield* read(parentThread)).runs.filter((run) => parentRunIds.has(run.id));
    +      for (const run of parentRuns.toSorted((a, b) => a.ordinal - b.ordinal)) {
    +        yield* finish(yield* start(parentThread, run.id), "received result");
    +      }
    +      const delivered = (yield* read(parentThread)).subagents.filter((task) =>
    +        followups.includes(task.id),
    +      );
    +      assert.deepEqual(
    +        delivered.map((task) => task.completionDelivery?.state),
    +        ["delivered", "delivered"],
    +      );
    +    }).pipe(Effect.provide(layer));
    +  }),
    +);
    +
     it.effect.each([
       { notice: "delegated task question", goal: null, steers: true },
       { notice: "background command", goal: "active", steers: true },

    Measured and written by Claude (Opus 5.5) in T3 Code; posted with the owner's approval.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    via-triageFiled through npx t3 triage

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions