fix(langchain): make RunnableRails.transform/atransform proper (async) iterators#2090
Open
nac7 wants to merge 1 commit into
Open
fix(langchain): make RunnableRails.transform/atransform proper (async) iterators#2090nac7 wants to merge 1 commit into
nac7 wants to merge 1 commit into
Conversation
…) iterators
RunnableRails.transform and atransform were aliased to invoke/ainvoke, which
violates the LangChain Runnable streaming protocol: transform must consume an
iterator of inputs and return an iterator, and atransform must be an async
iterator. Because atransform returned a coroutine, nesting RunnableRails in a
RunnableSequence and streaming it raised:
TypeError: 'async for' requires an object with __aiter__ method, got coroutine
Defer to the base Runnable implementations, which buffer the input stream and
delegate to the already-correct stream/astream methods, so guardrails run once
over the fully assembled input (no per-item re-invocation).
Fixes NVIDIA-NeMo#1692
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #1692.
RunnableRails.transformandRunnableRails.atransformwere aliased toinvoke/ainvoke, which violates the LangChainRunnablestreaming protocol:transformmust consume an iterator of inputs and return an iterator of outputs.atransformmust consume an async iterator of inputs and must itself be an async iterator.Because
atransformwas anasync defthatreturned a value, calling it produced a coroutine rather than an async iterator. As a result, placingRunnableRailsinside aRunnableSequenceand streaming it raised:Reproduction (from the issue)
Fix
Defer to the base
Runnable.transform/atransformimplementations, which buffer the input stream and delegate to the already-correctstream/astreammethods. Guardrails therefore run once over the fully assembled input, with no per-item re-invocation.Tests
Added regression tests in
tests/integrations/langchain/runnable_rails/test_streaming.py:RunnableRailsnested in aRunnableSequenceis streamable viaastream(the exact issue scenario) andstream.atransformreturns an async iterator (not a coroutine).transformconsumes an iterator and yields outputs.All four fail on the previous code and pass with this fix. Full
runnable_railssuite: 165 passed, 2 skipped.