docs: propose serialization compatibility policy

.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,7 @@
+# Summary
+
+## Serialization compatibility
+
+- [ ] If this changes default serialization, I reviewed the serialization
+  compatibility policy in `CONTRIBUTING.md` and described the replay impact
+  and any applicable exception. Otherwise, this item does not apply.

.github/copilot-instructions.md

@@ -80,6 +80,22 @@ Examples:
 - Add a regression test that confirms a new concrete API is absent from the
   public ABC's `__abstractmethods__`.
 
+## Serialization Compatibility
+
+- Follow the [serialization policy](../docs/serialization.md) for changes
+  to default serialization, including provider adapters. Use the affected
+  provider's documented contract to establish existing support.
+- Preserve encoded payloads, reconstructed values, raw fallback, and failure
+  behavior between major releases, subject to the exceptions below. Defer
+  consideration of new types and unsupported edge cases until a breaking major
+  release rather than adding feature-specific opt-in flags.
+- Non-major replay-breaking changes are limited to security fixes and repairs
+  to previously supported behavior. An unsupported annotation is not supported
+  merely because a related type is supported or its raw value is returned.
+- For an allowed replay-impacting change, add focused regression coverage,
+  including replay where appropriate, and document the risk and upgrade
+  mitigation. A previous failure can itself affect replay if it is caught.
+
 ## Copyright Headers
 
 Every new Python (`.py`) source file MUST begin with the following copyright

CHANGELOG.md

@@ -31,6 +31,8 @@ automatically reconstructed into typed model instances without requiring an expl
 > Drain affected instances on the previous SDK, or keep their task hub on the
 > previous SDK and use a separate task hub for new instances. Pin or lock the core
 > `durabletask` dependency, including when using the Azure Managed provider.
+> See [serialization and replay](docs/serialization.md) for supported types and
+> compatibility guidance.
 
 ## v1.13.1
 

CONTRIBUTING.md

@@ -13,3 +13,31 @@ This project has adopted the [Microsoft Open Source Code of
 Conduct](https://opensource.microsoft.com/codeofconduct/). For more information see the [Code of
 Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact
 [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.
+
+## Serialization compatibility
+
+Changes to default payload serialization can alter the replay of existing
+orchestrations. The [serialization guide](docs/serialization.md) describes the
+policy and the current `JsonDataConverter` support matrix. Apply the same replay
+assessment to provider adapters, while using each provider's own documented
+serialization contract.
+
+For non-major releases:
+
+- Preserve existing encoded payloads, reconstruction, raw fallback, and failure
+  behavior except as described below. Defer consideration of new types and
+  previously unsupported cases until a breaking major release instead of adding
+  feature-specific opt-in flags.
+- Replay-breaking changes are limited to security fixes and repairs to
+  previously documented supported behavior. A related type or a raw fallback
+  alone does not establish support. A change from failure to success can also
+  affect replay.
+- For an allowed change that can affect replay, explain the exception and impact
+  in the pull request, add focused regression coverage including replay where
+  appropriate, and provide a changelog warning with actionable upgrade
+  guidance. Prefer a compatibility mitigation when feasible and safe.
+
+Use the pull request checklist to identify changes to payloads, reconstructed
+values, or failures and describe any replay impact and applicable exception.
+Applications can use their own `to_json()` / `from_json()` hooks or a custom
+converter rather than expanding the default converter.

README.md

@@ -25,6 +25,7 @@ The Azure Functions Durable provider requires Python 3.13 or later.
 
 - [Supported Patterns](./docs/supported-patterns.md)
 - [Available Features](./docs/features.md)
+- [Serialization and Replay](./docs/serialization.md)
 - [Getting Started](./docs/getting-started.md)
 - [Development Guide](./docs/development.md)
 - [Azure Functions Durable 2.x](./azure-functions-durable/README.md)
@@ -67,6 +68,9 @@ Azure Functions uses its own shared codec rather than this converter.
 > replay behavior. Iteration over reconstructed sets is not replay-safe; sort
 > values explicitly before using their order to schedule durable work.
 
+See [serialization and replay](./docs/serialization.md) for supported types,
+custom converters, and compatibility guidance.
+
 ### Entity IDs
 
 `EntityInstanceId` values are immutable, hashable, and usable as dictionary keys

docs/serialization.md

@@ -0,0 +1,158 @@
+# Serialization and replay
+
+The core SDK and Azure Managed provider use `JsonDataConverter` by default for
+durable payloads. This guide describes that converter's current behavior and the
+compatibility policy for changes to default serialization. The support matrix
+reflects the current repository code; check the [changelog](../CHANGELOG.md)
+for changes not yet present in an installed release.
+
+Azure Functions Durable uses the shared Azure Functions codec instead. Its
+serialization behavior is not described by the type matrix below; see the
+[provider documentation](../azure-functions-durable/README.md#unicode-json-serialization).
+The compatibility policy also guides changes in this repository to that
+provider's default serialization.
+
+## Compatibility policy
+
+Orchestrations can replay payloads written by an earlier SDK version. Changing
+the serialized representation, reconstructed value or type, raw fallback, or
+failure behavior can change orchestration decisions during replay.
+
+The project follows these rules for changes to default serialization:
+
+- New types and previously unsupported serialization or reconstruction cases
+  are considered for the next breaking major version. They are not added in a
+  non-major release, including behind a new feature-specific opt-in flag.
+- Between major releases, existing payload representations, reconstructed
+  values and types, fallbacks, and failures are preserved except for security
+  fixes and repairs to previously documented supported behavior. Changes that
+  can affect replay include a release warning and upgrade guidance.
+- Existing options, including the default-off generic set reconstruction
+  option, retain their current defaults between major versions, subject to the
+  same exceptions.
+
+A type is not supported merely because it is related to a supported type or
+because the converter returns a raw value for it. That raw fallback can still
+affect replay. A previous failure can also affect replay if application code
+catches it; a change from failure to success is not automatically replay-safe.
+
+> [!WARNING]
+> Changing annotations, hooks, converters, or their configuration can also
+> change replay behavior. Test representative histories before deployment. If
+> old behavior cannot be preserved, drain affected instances on the old
+> configuration or isolate new instances in a separate task hub.
+
+## JSON representation
+
+The default converter serializes JSON-compatible values, dataclasses,
+`SimpleNamespace` objects, and enums with JSON-compatible values. Tuples become
+JSON arrays. Sets and frozensets also become arrays, ordered by canonical JSON
+representation when their elements can be deterministically encoded. Custom
+objects can provide a `to_json()` hook, as described below. A top-level `None`
+represents no payload.
+
+The converter does not add or infer a Python destination type from the JSON
+payload. Without a target type, decoding normally returns JSON values such as
+dictionaries and lists. For compatibility, the decoder still recognizes legacy
+object markers.
+
+## Reconstruction support
+
+`deserialize()` and `coerce()` accept an explicit target type, and dataclass
+fields are reconstructed using their annotations. For orchestrator, activity,
+and entity inputs, and for inferred activity results when a callable is
+provided, the SDK first asks `can_reconstruct()` whether to use the annotation.
+Explicit target types do not depend on that discovery check.
+
+In the table, `Sequence`, `Mapping`, and `Set` refer to `collections.abc`
+types; equivalent `typing` aliases with the same origin are also recognized.
+
+| Target annotation | Reconstruction behavior |
+| --- | --- |
+| Dataclass or `from_json()` class | Rebuilt when supplied; automatically discoverable. |
+| `list[T]`, `Sequence[T]` | Elements are coerced; discovery requires reconstructable `T`. |
+| `dict[str, T]`, `Mapping[str, T]` | Values are coerced; discovery requires reconstructable `T`. |
+| `tuple[T, ...]`, `tuple[T1, T2]` | Coerces elements; discovery needs a reconstructable element. |
+| `Optional[T]` | Non-`None` values use `T`; discovery follows `T`. |
+| `set[T]`, `frozenset[T]`, `Set[T]` | Generic reconstruction is opt-in; see below. |
+
+JSON object keys remain strings; mapping value annotations do not coerce keys.
+The converter recognizes the listed container origins, not arbitrary related
+ABCs or subclasses. Other generic annotations are generally passed through as
+parsed JSON rather than recursively reconstructed. `T | None` is equivalent to
+`Optional[T]`.
+
+Primitive-only annotations do not normally trigger automatic discovery. For
+example, `tuple[int, ...]` is not inferred as an activity input type, so the
+input remains a list; explicitly requesting that target type reconstructs a
+tuple. Enums can similarly be reconstructed when explicitly requested or used
+in a typed dataclass field, but an enum annotation alone is not automatically
+discovered. For unions with multiple non-`None` members, the converter does not
+guess which member should reconstruct an unmatched value.
+
+Reconstruction is best-effort: if coercion fails, `JsonDataConverter` returns
+the raw value and logs the failure at debug level. A custom converter can choose
+stricter behavior.
+
+### Set reconstruction
+
+Generic `set[T]`, `frozenset[T]`, and `Set[T]` annotations retain JSON arrays as
+lists by default, preserving their recorded order and duplicates. Set
+reconstruction for supported, hashable elements is enabled with
+`JsonDataConverter(enable_generic_sets=True)`; `enable_set_coercion=True` is an
+alias. Existing bare `set` and `frozenset` target types, including dataclass
+fields, can still be coerced through their constructors without this option.
+
+> [!WARNING]
+> Enabling generic set reconstruction for existing instances can change replay
+> behavior. Set iteration order is not replay-safe; order values by a
+> deterministic key before using their order to schedule durable work.
+
+## Custom types and converters
+
+For an application-owned type, define `to_json()` and `from_json()` hooks. The
+encoder calls `to_json()` for custom values not already handled by Python's
+JSON encoder; the hook returns a JSON-compatible value, not a JSON document.
+Define `from_json()` as a class method or static method. The decoder calls it
+with the parsed value when the type is supplied or discovered:
+
+```python
+from durabletask.serialization import JsonDataConverter
+
+
+class Money:
+    def __init__(self, cents: int) -> None:
+        self.cents = cents
+
+    def to_json(self) -> dict[str, int]:
+        return {"cents": self.cents}
+
+    @classmethod
+    def from_json(cls, value: dict[str, int]) -> "Money":
+        return cls(value["cents"])
+
+
+converter = JsonDataConverter()
+restored = converter.deserialize(converter.serialize(Money(125)), Money)
+assert restored.cents == 125
+```
+
+The `from_json()` hook is responsible for its own nested values. It can accept
+the active converter as a second parameter named `converter` and use its public
+`coerce()` method to reconstruct them.
+
+For other cases, implement a `DataConverter` or subclass `JsonDataConverter` and
+delegate unhandled values to its public methods. Implement matching
+`deserialize()` and `coerce()` behavior, and override `can_reconstruct()` when
+the SDK should discover your additional annotated types. A custom converter
+must traverse any additional container types itself: changing discovery alone
+does not add reconstruction. The
+[custom converter example](../examples/custom_data_converter/README.md) shows
+delegation and explains its top-level generic limitation.
+
+Where appropriate, an annotation listed in the support table can be used
+instead. Changing existing annotations can affect replay, so apply the upgrade
+precautions above.
+
+Keep custom serialization deterministic and configure compatible converters on
+both clients and workers that exchange durable payloads.

docs/supported-patterns.md

@@ -174,12 +174,13 @@ def purchase_order_workflow(ctx: task.OrchestrationContext, order: Order):
     return f"Approved by '{approval_details.approver}'"
 ```
 
-As an aside, you'll also notice that the example orchestration above works with custom business
-objects. Custom classes, data classes, and named tuples are serialized to plain JSON automatically.
-To reconstruct the original type on the receiving side, supply the type — for example via the
-`data_type` argument to `wait_for_external_event` (shown above), the `return_type` argument to
-`call_activity` / `call_sub_orchestrator` / `call_entity`, or by annotating the consuming function's
-input parameter. Without a type, the payload is returned as plain JSON (a `dict` or `list`).
+The example also works with custom business objects. Dataclasses, named tuples, and objects with
+`to_json()` hooks can be serialized as plain JSON. To reconstruct a supported type on the receiving
+side, supply the type — for example via the `data_type` argument to `wait_for_external_event`
+(shown above), the `return_type` argument to `call_activity` / `call_sub_orchestrator` /
+`call_entity`, or a supported annotation on the consuming function's input parameter. Without a
+type, the payload is normally returned as plain JSON (a `dict` or `list`). See
+[serialization and replay](serialization.md) for supported types and compatibility guidance.
 
 See the full [human interaction sample](../examples/human_interaction.py).