fix(server-utils): Derive the Vercel AI conversation id from the OpenAI conversation option - #24979
RulaKhaled wants to merge 1 commit into
Conversation
…AI conversation option `gen_ai.conversation.id` was filled with `providerMetadata.openai.responseId`, the id of the response that just came back, so every call got its own value under a grouping attribute. The id now comes from `providerOptions.openai.conversation` (or `azure`), the Conversations API id that is the same on every turn. Child spans inherit it from the active operation span, and `Sentry.setConversationId()` still wins. Part of #24832 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
size-limit report 📦
|
isaacs
left a comment
isaacs
left a comment
There was a problem hiding this comment.
There are some choices here that we should probably call out and I'd maybe add a follow-up for the openai inconsistency, but this is good overall.
Deferring reading the id from runtimeContext / experimental_telemetry.metadata until #24721 lands is sensible. That should be ready to go once the convention is approved.
| @@ -417,11 +437,16 @@ export function createSpanFromMessage( | |||
| recordToolDescriptions(callId, event.tools); | |||
| } | |||
|
|
|||
| // Only an operation's start event carries `providerOptions`; its model-call and tool events start | |||
| // while the operation span is active, so they inherit the id from it. | |||
| const conversationId = getOpenAiConversationId(event.providerOptions) ?? getActiveSpanConversationId(); | |||
There was a problem hiding this comment.
getActiveSpanConversationId() runs for every event type, root operations included. The PR description and the code comment describe it as "child model-call and tool spans inherit it from the active operation span". But for a root generateText/streamText/embed/rerank with no conversation option, the active span is whatever the user's code has active. If that span has gen_ai.conversation.id, the new operation takes it.
Two cases I found where this happens:
- A
generateTextcalled from inside a tool'sexecute(a sub-agent) takes the outer conversation id from theexecute_toolspan. This holds even when the inner call usespreviousResponseIdonly, which the scenario says must produce no id. - A Vercel AI call made inside a LangGraph node takes the LangGraph
thread_id(packages/server-utils/src/ai/langgraph/index.tsline 127). A call inside a Mastra or Flue span takes their conversation id in the same way.
This could be what we want! A sub-agent turn is arguably part of the same conversation. But the behavior is not stated, and it reaches past the "operation => child" scope that the description claims. It also means the result depends on which span is active at the call site, not on the call's own arguments.
Suggestion: Pick one approach and make it explicit. To match the description, I'd copy the id only for child events:
| const conversationId = getOpenAiConversationId(event.providerOptions) ?? getActiveSpanConversationId(); | |
| const conversationId = | |
| getOpenAiConversationId(event.providerOptions) ?? | |
| (ROOT_OPERATION_TYPES.has(type) ? undefined : getActiveSpanConversationId()); |
If cross-operation copying is wanted, we should say so in the comment and the PR description, and add a test scenario turn that runs generateText inside a tool execute and asserts the result, so that it's clearly intended.
| }, | ||
| }); | ||
|
|
||
| // Chaining on the previous response names a response, not a thread, so no conversation id. |
There was a problem hiding this comment.
Definitely out of scope for this PR, but I noticed that the direct OpenAI integration does the reverse of this comment: extractConversationId in packages/server-utils/src/ai/openai/utils.ts line 161 maps previous_response_id to gen_ai.conversation.id. That has the same problem you're fixing in this PR, because the value differs on each turn of a chain. So the same OpenAI conversation now gets different gen_ai.conversation.id values depending on whether the user calls OpenAI directly or through the AI SDK.
I'd suggest naming it in the description here as a todo, and adding a follow-up task to #24832 so we can converge on a single rule.
| const openaiOptions = providerOptions.openai ?? providerOptions.azure; | ||
| return isObjectLike(openaiOptions) ? asString(openaiOptions.conversation) : undefined; |
There was a problem hiding this comment.
Hm, this will stop at the first object-like openai option, not the first that has a conversation ID. Maybe unlikely, but someone might have multiple options for different providers or something, like { openai: { reasoningEffort: 'low' }, azure: { conversation: 'conv_xyz' } }
| const openaiOptions = providerOptions.openai ?? providerOptions.azure; | |
| return isObjectLike(openaiOptions) ? asString(openaiOptions.conversation) : undefined; | |
| for (const key of ['openai', 'azure']) { | |
| const options = providerOptions[key]; | |
| const conversation = isObjectLike(options) ? asString(options.conversation) : undefined; | |
| if (conversation) { | |
| return conversation; | |
| } | |
| } | |
| return undefined; |
| // Cache/reasoning token breakdown is derived from the model's `providerMetadata` — by the | ||
| // OTel processor on v6 and by the channel subscriber on v7, both via the shared | ||
| // `getProviderMetadataAttributes` helper, so the shape is identical. |
There was a problem hiding this comment.
OTel processor no longer exists, right?
| // Cache/reasoning token breakdown is derived from the model's `providerMetadata` — by the | |
| // OTel processor on v6 and by the channel subscriber on v7, both via the shared | |
| // `getProviderMetadataAttributes` helper, so the shape is identical. | |
| // Cache/reasoning token breakdown is derived from the model's `providerMetadata` by the | |
| // channel subscriber, which v6 reaches through the orchestrion adapter, so the shape is the | |
| // same on both versions. |
This branch has not been deployed
2 participants
The Vercel AI integration filled
gen_ai.conversation.idwithproviderMetadata.openai.responseId. That is the id of the response that just came back, so every call got a different value under a grouping attribute and showed up as its own one-span conversation. The same value is already ongen_ai.response.id. The mapping came in with #16992 and #19903 later had to guard it from overwriting user-set ids.The id now comes from
providerOptions.openai.conversation(orazure), the Conversations APIconv_id, which is the same on every turn. Child model-call and tool spans inherit it from the active operation span.Sentry.setConversationId()still wins, sinceconversationIdIntegrationwrites the scope value onspanStartafter these start attributes. The v6 adapter forwardsproviderOptionssoai4 to 6 behave the same.Not in this PR: reading the id from
runtimeContext/experimental_telemetry.metadatagoes with #24706 once #24721 lands.Part of #24832
🤖 Generated with Claude Code