Repository navigation
feat: TanStack AI POC - #26039
Closed
JPeer264 wants to merge 6 commits into
Closed
feat: TanStack AI POC#26039JPeer264 wants to merge 6 commits into
JPeer264 wants to merge 6 commits into
Conversation
Record invoke_agent, chat and execute_tool spans for @tanstack/ai chat() calls through its chat middleware API, and inject that middleware automatically via an orchestrion channel on chat(). Refs #18690 Co-Authored-By: Jan VamPeer Stöcklmair <jan.peer@sentry.io>
Record a gen_ai.embeddings span for each @tanstack/ai embed() call, with the input, the response model and the usage. The span is active while the call runs, so the provider request nests under it. embed() takes observe-only middleware that gets neither the input nor the result and cannot make a span active, so the call itself is traced through an orchestrion channel, as for LangChain embeddings. As for chat(), the integrations of the vendor SDKs are skipped only during the calls of the adapter, and a call that the app aborts gets the cancelled status. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Record a gen_ai.rerank span for each @tanstack/ai rerank() call, with the total tokens and the provider cost. Like embed(), the call is traced through an orchestrion channel, so the provider request nests under the span. Rerank models bill search units. The OpenRouter adapter reports 0 input and 0 output tokens and makes the response id itself, so the span leaves these out. When rerank() throws because the provider returned an index out of range, the span still records the usage, because the provider bills the call. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Record a gen_ai.decide span for each @tanstack/ai decide() call, with the state and questions as input messages, the answers as output messages, the response model and the usage, in the same form as for Vercel AI's experimental_decide(). Like embed(), the call is traced through an orchestrion channel, and the integrations of the vendor SDKs are skipped during the calls of the decider. A refused decide() call is still billed. TanStack AI puts the usage and the model on the EvaluateError that it throws, so the span records them with the error status. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The gen-ai-libraries app now calls embed(), rerank() and decide(), so CI checks these activities on Node (runtime injection) and on Cloudflare (build-time injection). embed() goes through the OpenAI adapter, and the openai tests that run after it check that the OpenAI integration is skipped only during the TanStack AI call. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Member
Author
|
Closing for now. Can be reopened when we get further into it. cc @andreiborza |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
POC