Skip to content

Commit 4d82397

Browse files
hardikkauraniandystaplesCopilot
authored
fix(serialization): support dict, Mapping, and tuple container traversal in can_reconstruct (#346)
* fix(serialization): add dict and Mapping container support to can_reconstruct Signed-off-by: Hardik Kaurani <hardikkaurani1@gmail.com> * fix(serialization): add homogeneous and heterogeneous tuple container support to can_reconstruct Signed-off-by: Hardik Kaurani <hardikkaurani1@gmail.com> * test(serialization): verify can_reconstruct with dict, Mapping, and tuple types Signed-off-by: Hardik Kaurani <hardikkaurani1@gmail.com> * test(worker): verify type discovery and activity input reconstruction for dict and tuple containers Signed-off-by: Hardik Kaurani <hardikkaurani1@gmail.com> * docs(serialization): document dict, Mapping, and tuple traversal in can_reconstruct Signed-off-by: Hardik Kaurani <hardikkaurani1@gmail.com> * test(executor): verify old-history replay behavior and nondeterminism for dict and tuple containers Signed-off-by: Hardik Kaurani <hardikkaurani1@gmail.com> * test(serialization): verify custom converter policies and azure functions unaffected coverage Signed-off-by: Hardik Kaurani <hardikkaurani1@gmail.com> * docs(changelog): document replay-breaking container reconstruction in core and azuremanaged changelogs Signed-off-by: Hardik Kaurani <hardikkaurani1@gmail.com> * docs(changelog): narrow and accurately scope replay-breaking container reconstruction warning Signed-off-by: Hardik Kaurani <hardikkaurani1@gmail.com> * test: remove trailing blank line to fix lint Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --------- Signed-off-by: Hardik Kaurani <hardikkaurani1@gmail.com> Co-authored-by: Andy Staples <andystaples@microsoft.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
1 parent 17ec055 commit 4d82397

6 files changed

Lines changed: 468 additions & 9 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,26 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
77

88
## Unreleased
99

10+
FIXED
11+
12+
- Type discovery now recognizes `dict` / `Mapping` and `tuple` containers wrapping
13+
reconstructable types (such as dataclasses and `from_json()`-capable types) via
14+
`JsonDataConverter.can_reconstruct()`. Inbound activity and orchestrator inputs as
15+
well as inferred activity return types annotated with these containers are now
16+
automatically reconstructed into typed model instances without requiring an explicit
17+
`return_type` override.
18+
19+
> [!WARNING]
20+
> **Replay-breaking bug fix:** Container annotations involving `dict`, `Mapping`,
21+
> or `tuple` wrapping reconstructable types, including nested annotations, now
22+
> reconstruct orchestration inputs and inferred activity results by default.
23+
> Existing instances that relied on raw dictionaries or lists can change behavior
24+
> or fail replay with `NonDeterminismError`.
25+
>
26+
> Drain affected instances on the previous SDK, or keep their task hub on the
27+
> previous SDK and use a separate task hub for new instances. Pin or lock the core
28+
> `durabletask` dependency, including when using the Azure Managed provider.
29+
1030
## v1.13.1
1131

1232
ADDED

‎durabletask-azuremanaged/CHANGELOG.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,14 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
77

88
## Unreleased
99

10+
FIXED
11+
12+
- With the corresponding core `durabletask` SDK update, type discovery now recognizes
13+
`dict`, `Mapping`, and `tuple` container annotations wrapping reconstructable types,
14+
automatically reconstructing recorded inputs and inferred activity results into typed model
15+
instances. See the [core upgrade warning](../CHANGELOG.md#unreleased) for replay-breaking
16+
guidance and dependency pinning recommendations.
17+
1018
## v1.13.1
1119

1220
ADDED

‎durabletask/serialization.py‎

Lines changed: 15 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -267,15 +267,14 @@ def _is_supported_set_element(converter: DataConverter, elem_type: Any) -> bool:
267267
def _can_reconstruct(converter: DataConverter, target_type: Any) -> bool:
268268
""":class:`JsonDataConverter`'s reconstruction policy.
269269
270-
Recognizes dataclasses and ``from_json()``-capable types, plus ``Optional``
271-
/ ``list`` / ``Sequence`` hints wrapping them; builtins and unknown
272-
annotations are excluded. When set support is enabled on the converter
273-
(via ``enable_generic_sets=True``), ``set`` / ``frozenset`` / ``Set``
274-
containers with supported element types are also recognized. Recurses
275-
through ``converter.can_reconstruct`` (not itself) so a
270+
Recognizes dataclasses and ``from_json()``-capable types, plus ``Optional``,
271+
``list`` / ``Sequence``, ``dict`` / ``Mapping``, and ``tuple`` hints wrapping
272+
them; builtins and unknown annotations are excluded. When set support is
273+
enabled on the converter (via ``enable_generic_sets=True``), ``set`` /
274+
``frozenset`` / ``Set`` containers with supported element types are also
275+
recognized. Recurses through ``converter.can_reconstruct`` (not itself) so a
276276
:class:`JsonDataConverter` subclass that overrides ``can_reconstruct``
277-
still participates in the element-type checks of ``Optional`` / ``list``
278-
hints.
277+
still participates in the element-type checks of wrapped hints.
279278
"""
280279
origin = typing.get_origin(target_type)
281280
if origin is not None:
@@ -292,6 +291,14 @@ def _can_reconstruct(converter: DataConverter, target_type: Any) -> bool:
292291
if not args:
293292
return True
294293
return _is_supported_set_element(converter, args[0])
294+
if origin in (dict, Mapping):
295+
return len(args) == 2 and converter.can_reconstruct(args[1])
296+
if origin is tuple:
297+
if not args:
298+
return False
299+
if len(args) == 2 and args[1] is Ellipsis:
300+
return converter.can_reconstruct(args[0])
301+
return any(converter.can_reconstruct(a) for a in args)
295302
return False
296303
if not isinstance(target_type, type):
297304
return False

‎tests/azure-functions-durable/test_converters.py‎

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -361,3 +361,54 @@ def test_durable_client_decode_preserves_host_configuration():
361361
value = DurableClientConverter.decode(
362362
meta.Datum(type="string", value=config), trigger_metadata=None)
363363
assert value == config
364+
365+
366+
# ---------------------------------------------------------------------------
367+
# FunctionsDataConverter reconstruction and discovery
368+
# ---------------------------------------------------------------------------
369+
370+
def test_functions_data_converter_can_reconstruct_and_discovery_unaffected():
371+
"""Azure Functions overrides can_reconstruct() to return True uniformly.
372+
373+
The core SDK's expanded dict/tuple traversal in JsonDataConverter does not
374+
alter Azure Functions discovery behavior, because FunctionsDataConverter
375+
already returns True for all types (delegating decoding to df_loads).
376+
"""
377+
from collections.abc import Mapping
378+
from dataclasses import dataclass
379+
from durabletask.internal import type_discovery
380+
381+
@dataclass
382+
class Order:
383+
item: str
384+
quantity: int
385+
386+
conv = FunctionsDataConverter()
387+
388+
# Always returns True for affected container types
389+
assert conv.can_reconstruct(dict[str, Order]) is True
390+
assert conv.can_reconstruct(Mapping[str, Order]) is True
391+
assert conv.can_reconstruct(tuple[Order, ...]) is True
392+
assert conv.can_reconstruct(tuple[Order, int]) is True
393+
394+
# Always returns True even for builtins and unknown types (distinct from core)
395+
assert conv.can_reconstruct(dict[str, int]) is True
396+
assert conv.can_reconstruct(tuple[int, str]) is True
397+
assert conv.can_reconstruct(int) is True
398+
assert conv.can_reconstruct(dict) is True
399+
assert conv.can_reconstruct(tuple) is True
400+
401+
# Inbound and outbound type discovery with FunctionsDataConverter hands declared types through
402+
def sample_func(ctx, orders: dict[str, Order]) -> tuple[Order, ...]:
403+
pass
404+
405+
assert type_discovery.orchestrator_input_type(sample_func, conv) == dict[str, Order]
406+
assert type_discovery.activity_input_type(sample_func, conv) == dict[str, Order]
407+
assert type_discovery.activity_output_type(sample_func, conv) == tuple[Order, ...]
408+
409+
def builtin_func(ctx, data: dict[str, int]) -> tuple[int, ...]:
410+
pass
411+
412+
# Builtins are also passed through to df_loads under FunctionsDataConverter
413+
assert type_discovery.orchestrator_input_type(builtin_func, conv) == dict[str, int]
414+
assert type_discovery.activity_output_type(builtin_func, conv) == tuple[int, ...]

0 commit comments

Comments
 (0)