Skip to content

Commit 5a77c20

Browse files
andystaplesCopilot
andcommitted
Merge andystaples-async-stack-3-dispatch-cleanup into andystaples-async-stack-4-async-context
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2 parents f8347b8 + f6a45e8 commit 5a77c20

1 file changed

Lines changed: 87 additions & 23 deletions

File tree

‎docs/async-orchestrators.md‎

Lines changed: 87 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,45 @@ batch before scheduling its next operation.
3535
> when the workflow finishes. Keep durable compensation in normal orchestration
3636
> control flow; see [dispatch teardown](#dispatch-teardown-is-not-workflow-cancellation).
3737
38+
## Motivation and scope
39+
40+
Generator orchestrators cannot call `async def` helpers or use ordinary
41+
asyncio composition, so async-only libraries need adapters or rewrites before
42+
they can participate in a durable workflow
43+
([#294](https://github.com/microsoft/durabletask-python/issues/294)).
44+
45+
Goals:
46+
47+
- Let orchestrators be written with `async def` and `await`, sharing the
48+
existing replay engine, history format, and provider integrations.
49+
- Support ordinary deterministic asyncio composition, such as
50+
`asyncio.gather`, `asyncio.TaskGroup`, `asyncio.timeout`, and
51+
`asyncio.wait_for`, with durable timers as the clock.
52+
- Allow async helpers and libraries to compose durable operations, as long as
53+
they use only supported asyncio APIs.
54+
- Keep generator orchestrators and their recorded histories unchanged.
55+
56+
Non-goals:
57+
58+
- A general-purpose asyncio event loop. Real network, storage, or subprocess
59+
I/O, threads, and executors remain activity work.
60+
- Set-based composition such as `asyncio.wait` and `asyncio.as_completed`,
61+
whose ordering is not replay-stable.
62+
- Automatic recording of arbitrary `await` results or local activities.
63+
64+
Alternatives considered:
65+
66+
- **A minimal await-to-generator driver without an event loop.** Driving the
67+
coroutine directly would support sequential awaits, but not asyncio tasks,
68+
task groups, timeouts, or libraries that call `asyncio.get_running_loop()`.
69+
- **Combinator-only APIs.** Extending `task.when_all` / `task.when_any` keeps the
70+
generator model but still excludes async-only code and standard asyncio
71+
composition.
72+
73+
A deterministic, dispatch-local event loop supports the standard composition
74+
primitives while deriving its clock and task ordering from durable history,
75+
so replays reconstruct the same local scheduling decisions.
76+
3877
## Cold operations and reusable handles
3978

4079
An `async def` orchestrator automatically receives an
@@ -58,9 +97,9 @@ One-way commands `ctx.send_event(...)`, `ctx.signal_entity(...)`, and
5897
Use these cold coroutines with `asyncio.create_task`, `TaskGroup.create_task`,
5998
and ordered `gather`. Constructing a coroutine does not validate the operation,
6099
allocate action IDs, serialize input, or consume buffered events. Cancelling its local task before
61-
its first step starts no durable operation. Mutable inputs are read when the
62-
coroutine executes, and relative timer delays start from that execution point.
63-
Keep or copy inputs accordingly.
100+
its first step starts no durable operation. Operation inputs are not copied:
101+
mutations made before the coroutine's first step are observed, and relative
102+
timer delays start from that execution point. Keep or copy inputs accordingly.
64103

65104
Cold coroutines are single-use, like ordinary Python coroutines. For an
66105
explicitly started operation with multiple independent waiters, retain its
@@ -79,11 +118,20 @@ Cancelling either borrower does not cancel the underlying operation. Direct
79118
an awaitable durable handle is not a coroutine. Do not retry that expression
80119
assuming the eager operation was rolled back.
81120

82-
Async function detection also recognizes `functools.wraps`, partials, and
83-
callable instances; existing registration naming requirements still apply.
84-
Use `functools.wraps` when a synchronous decorator wraps an async orchestrator;
85-
an unmarked synchronous wrapper returning a coroutine is rejected because it
86-
would have received the generator context.
121+
The worker classifies each orchestrator once, when it is registered:
122+
123+
1. A callable that is itself a coroutine function (including partials, bound
124+
methods, and callable instances whose `__call__` is `async def`) is async.
125+
2. A callable that is itself a generator function is a generator orchestrator.
126+
3. Otherwise, a `functools.wraps` wrapper is classified by the function it
127+
wraps, so a synchronous decorator around an async orchestrator remains async.
128+
4. Any other callable is synchronous.
129+
130+
Async orchestrators receive an `AsyncOrchestrationContext` and must return a
131+
coroutine; synchronous orchestrators receive `OrchestrationContext`. An
132+
unmarked synchronous wrapper returning a coroutine is rejected because it would
133+
have received the generator context. Cyclic `__wrapped__` chains are rejected at
134+
registration. Existing registration naming requirements still apply.
87135

88136
## Azure Functions compatibility
89137

@@ -164,14 +212,18 @@ require history events, not elapsed time in the worker.
164212
> raise errors, but the SDK cannot detect every unsafe operation, especially
165213
> synchronous side effects or already-completed foreign awaitables.
166214
167-
`asyncio.wait` and `as_completed` are **not supported deterministic composition
168-
APIs**. `wait` returns unordered sets; `as_completed` converts inputs to a set
169-
and can create tasks in hash-dependent order. Pre-completed/tied tasks and
170-
differences between Python implementations can change ordering even when a
171-
particular example executes successfully. Do not iterate sets to schedule
172-
durable actions. Use ordered `gather`, task groups, or durable `when_any` and
173-
explicitly handle the selected durable task. Other asyncio APIs are not
174-
implicitly supported because they happen to execute.
215+
Other asyncio APIs are not implicitly supported because they happen to execute.
216+
Do not iterate sets to schedule durable actions.
217+
218+
> [!WARNING]
219+
> `asyncio.wait` and `asyncio.as_completed` are **not supported** in async
220+
> orchestrators. They are not rejected at run time, but `wait` returns
221+
> unordered sets and `as_completed` converts its inputs to a set and can create
222+
> tasks in hash-dependent order. Pre-completed or tied tasks and differences
223+
> between Python implementations can change durable action order on replay,
224+
> even when a particular example executes successfully. Use ordered `gather`,
225+
> task groups, or durable `when_any` and explicitly handle the selected durable
226+
> task.
175227
176228
## Failures, timeouts, and cancellation
177229

@@ -237,15 +289,20 @@ to follow the existing worker recovery policy.
237289

238290
## Dispatch teardown is not workflow cancellation
239291

292+
> [!WARNING]
293+
> Dispatch teardown is not workflow cancellation. Nested `finally` blocks and
294+
> async context-manager exits may run at **every dispatch boundary**, including
295+
> dispatches that simply suspend while waiting for durable work, long before
296+
> the workflow completes.
297+
240298
Every dispatch reconstructs the coroutine and its local tasks from history.
241299
After the response and entity-release obligations are snapshotted, teardown
242300
cancels remaining local tasks, drains cooperative cleanup, closes tracked async
243301
generators, and disposes the loop. This includes dispatches that suspend,
244302
continue as new, encounter a history failure, or reject a version.
245303

246-
Nested `finally` blocks and async context-manager exits may run at **every
247-
dispatch boundary**, before workflow completion. Local cleanup can await ready
248-
work such as `sleep(0)`; it must not wait for new durable or external work.
304+
Local cleanup can await ready work such as `sleep(0)`; it must not wait for new
305+
durable or external work.
249306
Scheduling/awaiting durable work during async teardown raises an actionable
250307
error. Teardown must cooperate with cancellation; a non-quiescent cleanup
251308
queue is bounded and reported, not allowed to run indefinitely. Each
@@ -276,6 +333,15 @@ explicitly with `await gen.aclose()` or iterate it inside
276333

277334
## Replay and framework adapters
278335

336+
> [!WARNING]
337+
> Recorded histories are coupled to the Python minor version. Python 3.11 and
338+
> 3.12+ schedule coroutine operands of `asyncio.wait_for()` differently, which
339+
> can change durable action order inside concurrent branches, and the async
340+
> context does not normalize arbitrary standard-library scheduling. Keep the
341+
> Python minor version (and dependencies) fixed for in-flight instances; drain
342+
> or version affected instances before upgrading. See
343+
> [#337](https://github.com/microsoft/durabletask-python/issues/337).
344+
279345
Ordinary async helpers rerun during replay. Their return values are not
280346
automatically checkpointed. There are no local/virtual activities and no
281347
automatic recording or interception of arbitrary await expressions.
@@ -307,10 +373,8 @@ construction order is not execution order.
307373
Supporting several Python versions does not guarantee recorded-history
308374
portability between them. Pin the runtime and dependencies for active
309375
instances and validate histories before upgrades. All workers processing an
310-
instance must use that pinned environment. In particular, Python 3.11
311-
and 3.12+ schedule coroutine operands of `asyncio.wait_for()` differently, which
312-
can change durable action order inside concurrent branches. The async context
313-
does not normalize arbitrary standard-library scheduling.
376+
instance must use that pinned environment; see the Python version warning
377+
above.
314378

315379
The Functions `orchestrator_generator_wrapper` helper supports direct durable
316380
awaits in synchronous and async mock-based unit tests; it does not simulate the

0 commit comments

Comments
 (0)