fix(langgraph): offload sync remote tool coroutines - #229
Conversation
Signed-off-by: Mohamed Ali <mohamed.ali@openinnovation.ai>
|
Thank you for your pull request and welcome to our community! To contribute, please sign the Oracle Contributor Agreement (OCA).
To sign the OCA, please create an Oracle account and sign the OCA in Oracle's Contributor Agreement Application. When signing the OCA, please provide your GitHub username. After signing the OCA and getting an OCA approval from Oracle, this PR will be automatically updated. If you are an Oracle employee, please make sure that you are a member of the main Oracle GitHub organization, and your membership in this organization is public. |
|
@fashkl Thank you for the contribution! To start, could you please sign the Oracle Contributor Agreement (OCA)? You can find more info in the comment at #229 (comment). Please make sure to use the same email associated to your Github account, that should be the one reported in the comment. |
Hello @cesarebernardis |
Problem
Fixes #228.
LangGraph
RemoteToolconversion exposes a coroutine for sync remote-tool callables, but the coroutine currently calls the synchronous function directly. ForRemoteTool, that function executeshttpx.request(...), so async LangGraph execution can block the event loop while waiting on network I/O.This is worse than leaving
StructuredTool.coroutineunset. LangChain's ownStructuredTool._arunchecksself.coroutinefirst; when it isNone, it falls back to the base async implementation, whose source comment says it is expected to delegate_runto a separate thread. By supplying a fake async wrapper, pyagentspec bypasses that safe default and runs blocking sync I/O on the event-loop thread.This also makes
RemoteToolinconsistent with the existingApiNodeexecutor split in this codebase:ApiNodeExecutor._execute()uses synchttpx.request(...), whileApiNodeExecutor._aexecute()useshttpx.AsyncClient. RemoteTool is the outlier because its async entrypoint is only async-shaped, not async-safe.In services with event-loop based liveness/readiness handling, a slow or stuck remote tool request can therefore freeze unrelated async work long enough to trip health checks and restart the service.
Fix
Offload synchronous structured-tool coroutine execution with
asyncio.to_thread(...)while preserving native async callables unchanged.This is a small general safety net for sync callables that pyagentspec intentionally exposes through a coroutine, including the current
RemoteToolandClientToolconversion paths. A future native asyncRemoteToolimplementation usinghttpx.AsyncClientwould still be compatible with this change and could further reduce thread usage for RemoteTool specifically.Testing
test_remote_tool_coroutine_does_not_block_event_loopSKIP_LLM_TESTS=1 uv run python -m pytest -q tests/adapters/langgraph/test_tools.pyCommit is signed off per the contribution guide.