fix(execution): commit context manager resources before yielding to downstream consumers (#126) - #127
Merged
Conversation
added 3 commits
July 28, 2026 07:58
…ownstream consumers (#126) ### Summary Fixes #126 where context-manager resources in EACH-mode steps remained uncommitted while yielded items were being processed by downstream consumers, causing stale/uncommitted reads in multi-step outbox pipelines. ### Changes - Sync & Async step runners: execute `self.fn(**item_args)` inside the context manager block and save the result, allowing the context manager block (`with ExitStack()`) to exit and commit DB transactions *before* `yield result` passes the item downstream. - Corrected exception handling flow when `PipelineStopException` or step execution exceptions occur. ### Tests - Added sync & async unit tests verifying that resource context managers commit prior to downstream step execution (`test_cm_resource_exited_before_downstream_consumer_receives_item`). - All 798 tests passing in full test suite.
…/exc_to_raise flags
- Restructure try/except to wrap the with ExitStack block, keeping yield inside try but outside with (commit before yield, errors still handled) - Add tests for CM factory errors in EACH mode with OnError.CONTINUE (skips failed item) and OnError.STOP (raises PipelineStopException)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #126 where context-manager resources in EACH-mode steps remained uncommitted while yielded items were being processed by downstream consumers, causing stale/uncommitted reads in multi-step outbox pipelines.
Root Cause
In
StepRunner(both sync & async),yield self.fn(**item_args)was located inside thewith ExitStack()block. As a result, when a step yielded a processed item to a downstream consumer, the context manager block was suspended mid-execution before entering__exit__(where DB commits occur). Downstream steps reading from the database saw uncommitted state.Solution
Execute
self.fn(**item_args)inside theExitStack/AsyncExitStackblock to store the result, allowing the context manager block to close (and commit) beforeyield resultemits the item to downstream steps.Tests
test_cm_resource_exited_before_downstream_consumer_receives_item)