Skip to content

[Async stack 3/6] Dispatch cleanup for coroutine orchestrators - #360

Draft
andystaples wants to merge 7 commits into
andystaples-async-stack-2-asyncio-compositionfrom
andystaples-async-stack-3-dispatch-cleanup
Draft

andystaples wants to merge 7 commits into
andystaples-async-stack-2-asyncio-compositionfrom
andystaples-async-stack-3-dispatch-cleanup

Conversation

@andystaples

@andystaples andystaples commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Do not merge this PR, or any PR in this stack, individually. The whole stack will be squash-merged only after every stack PR is approved.

Layer 3/6 of the review stack that splits the closed #293. The base is layer 2/6, #358.

Scope

Dispatch teardown for the per-dispatch durable event loop:

  • DurableEventLoop installs the async-generator hooks for each dispatch and restores the host's previous hooks afterwards. close() drains, then cancels, pending tasks and closes tracked async generators. close() is idempotent.
  • Cooperative cleanup is capped at about 10,000 loop steps per cleanup phase (more for dispatches with many tasks or generators). Each async-generator sweep spends from a budget fixed at sweep entry, so tasks or generators created during the sweep cannot extend it. Truncated cleanup is reported once as a warning without changing the snapshotted response.
  • Cleanup errors are collected per handle. The first unexpected failure is kept; a process-control exception such as KeyboardInterrupt or SystemExit takes precedence, even when grouped or raised after other errors. The existing dispatch response is preserved, and background and cleanup errors are logged.
  • wait_durable cannot schedule durable work once the loop is closing. The orchestration context also rejects sequence-number allocation during async cleanup, and exiting an entity critical section during teardown does not schedule extra release messages.
  • Control-flow exception classification moves from worker into the shared durabletask.internal.exceptions.is_control_flow_exception. Its BaseExceptionGroup handling is unchanged.

Docs

docs/async-orchestrators.md gains a warning callout describing the cleanup cap and truncation, and documents async-generator ownership during dispatch, explicit aclose() / contextlib.aclosing(...), and teardown handling of entity critical sections and process-control exits.

CI

Adds an async-runtime-windows job to .github/workflows/durabletask.yml that runs the asyncio, replay, and dispatch cleanup tests on Windows across Python 3.11-3.15. Layer 4/6 (#361) extends it with test_async_context.py.

Intermediate adaptations reversed in layer 4/6

Async orchestrators still receive the eager task.OrchestrationContext at this layer, so tests annotate contexts as task.OrchestrationContext instead of task.AsyncOrchestrationContext, and operations that will later go through ctx.as_task_context() (for example call_activity(...) and create_timer(...) that need eager handles) are called directly on ctx.

No changelog changes; release notes are added in layer 6/6 (#364).

Comment thread durabletask/internal/asyncio_loop.py Resolved
Comment thread durabletask/internal/asyncio_loop.py Resolved
@andystaples
andystaples force-pushed the andystaples-async-stack-3-dispatch-cleanup branch from fe8e8bf to 466196e Compare November 12, 2026 18:18
andystaples and others added 3 commits November 12, 2026 13:35
Close async generators and drain or cancel pending tasks at dispatch teardown, share control-flow exception classification, and add a Windows async-runtime CI job.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@andystaples
andystaples force-pushed the andystaples-async-stack-3-dispatch-cleanup branch from ed84f44 to c164cb6 Compare November 12, 2026 20:41
andystaples and others added 4 commits November 12, 2026 15:33
…async-stack-3-dispatch-cleanup

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…async-stack-3-dispatch-cleanup

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…async-stack-3-dispatch-cleanup

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Spend each async-generator sweep's drains from the budget fixed at sweep
entry, report truncation once, keep the first custom cleanup failure unless a
process-control exit supersedes it, and restore the closed-loop guard. Add
regression tests for these and for restoring host async-generator hooks, and
document the cleanup cap prominently. Remove changelog entries for unreleased
behavior.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant