fix(retain): fail when oversized chunk cannot split - #3655
Closed
kamilfurtak wants to merge 3 commits into
Closed
Conversation
kamilfurtak
force-pushed
the
agent/retain-fail-on-unsplittable
branch
from
August 19, 2026 20:50
1b310d6 to
483f57f
Compare
The recursive split gathered sub-chunk results without return_exceptions, so a single unsplittable half discarded the facts the other half had already extracted, and the failure propagated to the whole document. Losing good data is exactly the failure mode this path exists to prevent. Collect sub-chunk results tolerantly: salvage what succeeded, log the partial failure at error level with counts, and only re-raise when the split produced nothing at all.
kamilfurtak
force-pushed
the
agent/retain-fail-on-unsplittable
branch
from
August 19, 2026 20:54
483f57f to
c5a27cd
Compare
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
In Hindsight 0.9.1, the retain output-retry splitter logs a warning and returns an empty result when an overlong sub-chunk cannot be split any further. The caller then treats that result as successful extraction, which can allow an operation to report
completedwhile the sub-chunk's data was silently lost.This draft PR changes only that terminal branch:
_split_chunk_for_output_retry;RuntimeErrorwhen anOutputTooLongErrorcannot be followed by a valid split;extract_facts_from_textfailure propagation mark the retain as failed/retryable instead of committing a partial result.Reproduction / impact
OutputTooLongError._split_chunk_for_output_retryreturnsNonebecause the chunk is too small or cannot be reduced._extract_facts_with_auto_splitreturned([], TokenUsage())and logged that it was dropping the sub-chunk.The regression test uses a one-character chunk and a mocked
OutputTooLongError; it now verifies that the helper raises and makes exactly one extraction attempt. Splittable chunks and the standard successful extraction path are unchanged.Scope
[[{turn}, ...]]splitter fix in fix(retain): preserve nested conversation chunks during output retry #3652.Validation
tests/test_fact_extraction_retry.py: 25 passedThe repository hook was not used because this checkout does not have
uvinstalled; the equivalent targeted Ruff checks were run directly.Depends on #3652
Without #3652, conversation payloads wrapped in an extra array cannot be split at
all, so this change would turn a recoverable case into a hard failure. #3652 makes
those splits succeed; this PR then only fails when a chunk genuinely cannot be
reduced further.
Partial results are preserved
The recursive split previously gathered sub-chunk results without
return_exceptions, so one unsplittable half discarded facts the other half hadalready extracted. That is the same data-loss shape this PR is meant to remove, so
sub-chunk results are now collected tolerantly:
failure at error level with counts
The remaining trade-off is deliberate: a partially-failed chunk still completes,
with the loss visible in logs rather than silent at the operation level.