Add 0050 example coverage: degrade mode and call-level retry#152
Merged
Conversation
Demonstrate FailureIsolationMiddleware as a third per-instance failure posture beside fail_fast and collect: wrapped outermost in the instance middleware (retry stays inner), an instance whose retries exhaust is caught and degraded to a placeholder so the batch finishes intact. Rename the demo env knob COLLECT_MODE to MODE (fail_fast/collect/degrade) and update the example and middleware concept docs.
Pass complete(retry=RetryConfig(...)) on the respond node to retry the LLM wire call on transient categories, and grow the example's failure-handling rundown from three placements to four. Update the example and LLMs concept docs.
There was a problem hiding this comment.
Pull request overview
Updates existing worked examples and documentation to demonstrate proposal 0050’s two primitives (failure isolation “degrade” posture and call-level retry) without adding new standalone demos.
Changes:
- Extend
fan-out-with-retrywith a newMODE=degradeposture usingFailureIsolationMiddlewareas outermost instance middleware, and rename the env knob fromCOLLECT_MODEtoMODE. - Extend
chat-with-multimodalto use call-level retry viacomplete(retry=RetryConfig(...))and update the failure-handling discussion accordingly. - Add docs backlinks from middleware/LLM concept pages to the updated demonstrations.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| examples/fan-out-with-retry/main.py | Adds MODE handling and degrade posture via FailureIsolationMiddleware in instance middleware. |
| examples/chat-with-multimodal/main.py | Demonstrates call-level retry on complete(retry=...) and updates explanatory text/output. |
| docs/examples/fan-out-with-retry.md | Updates example docs for MODE (fail_fast / collect / degrade) and degrade behavior. |
| docs/examples/chat-with-multimodal.md | Documents call-level retry usage and expands failure-handling discussion. |
| docs/concepts/middleware.md | Adds backlink to the fan-out degrade composition demonstration. |
| docs/concepts/llms.md | Adds backlink to the chat example demonstrating call-level retry. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Raise ValueError on an unknown mode instead of silently falling back to fail_fast, so a mistyped MODE fails loudly. The docs assert the three valid values.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fold proposal 0050's two new primitives into existing worked examples and their docs, rather than adding standalone demos.
fan-out-with-retrygains adegradefailure posture alongsidefail_fastandcollect.FailureIsolationMiddlewarewraps each fan-out instance as the outermost layer (retry stays inner), so an instance whose retries exhaust is caught and degraded to a placeholder result and the batch finishes intact. The demo env knob is renamedCOLLECT_MODEtoMODE(fail_fast/collect/degrade).chat-with-multimodalgains call-levelcomplete(retry=RetryConfig(...))on the respond node, retrying just the LLM wire call on transient categories. Its failure-handling rundown grows from three placements to four.Each example's docs page is updated, and the middleware and LLMs concept pages get backlinks to the new demonstrations.
Notes
FailureIsolatedEventcurrently reports the engine's wrapper category rather than the originating cause. That fidelity question is tracked separately and does not affect the degrade behavior shown here.src/changes. Both 0050 primitives shipped already (in Add FailureIsolationMiddleware (proposal 0050) #149 / Add RetryConfig record for RetryMiddleware #150 / Add call-level retry to Provider.complete() #151); this is example and docs coverage only.Validation
ruff format/check,pyright, andpytest tests/test_examples_smoke.py(13/13) pass;mkdocs buildis clean.fail_fastandMODE=degrade(the sentinel degrades to a placeholder, the rest summarize, no error block), and the chat demo across its four turns.