@langchain/typesafe (TypeSafeClassifier, a LangChain Runnable) calls Jev with fetch directly. It does not use @typesafe-ai/sdk, so our TypeSafe integration does not see these calls.
Our LangChain callback handler sees the run through handleChainStart. It records the run as an invoke_agent TypeSafeClassifier span (gen_ai.invoke_agent). This span has no request model, response model, token usage, questions or answers.
Expected
An evaluate <model> span (gen_ai.evaluate) with the same attributes as the TypeSafe integration (packages/server-utils/src/ai/typesafe/): gen_ai.input.messages (state + questions), gen_ai.output.messages (answers), request/response model and token usage.
Notes
@langchain/typesafe(TypeSafeClassifier, a LangChainRunnable) calls Jev withfetchdirectly. It does not use@typesafe-ai/sdk, so our TypeSafe integration does not see these calls.Our LangChain callback handler sees the run through
handleChainStart. It records the run as aninvoke_agent TypeSafeClassifierspan (gen_ai.invoke_agent). This span has no request model, response model, token usage, questions or answers.Expected
An
evaluate <model>span (gen_ai.evaluate) with the same attributes as the TypeSafe integration (packages/server-utils/src/ai/typesafe/):gen_ai.input.messages(state+questions),gen_ai.output.messages(answers), request/response model and token usage.Notes
@langchain/typesafe0.0.1, peer@langchain/core^1.0.0. Default modeljev-latest, endpointPOST {baseUrl}/v1/systemone.TypeSafeClassifierinhandleChainStart(lc_namespaceislangchain/classifiers/typesafe) and reuse the TypeSafe attribute helpers (getEvaluationInputMessages,addResponseAttributes). The chain outputs are the parsed response.experimental_evaluate(feat(server-utils): Instrument Vercel AI experimental_evaluate #24694), TypeSafe integration (feat(server-utils): Add TypeSafe integration #24703), Workers AI Jev (feat(cloudflare): Trace TypeSafe Jev calls in Workers AI as evaluate spans #24833).