fix(ai): only capture terminal Responses statuses as stop reasons - #4736
Open
bernatixer wants to merge 3 commits into
Open
fix(ai): only capture terminal Responses statuses as stop reasons#4736bernatixer wants to merge 3 commits into
bernatixer wants to merge 3 commits into
Conversation
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
marandaneto
approved these changes
Sep 2, 2026
Member
|
@bernatixer make sure to tag @PostHog/team-ai-observability for reviewer as well |
Radu-Raicea
reviewed
Sep 2, 2026
| // by what cut it short, and only terminal statuses count as stop reasons. | ||
| // Shared with the native OpenAI Responses wrapper. | ||
| responsesStopReason(messageResponseMetadata) || | ||
| responsesStopReason(generationResponseMetadata) |
Member
There was a problem hiding this comment.
This can return incomplete before checking the reason in generationResponseMetadata. Is that fine? Could the reason be max_output_tokens?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
background: truereturns while still queued, and the callback records$ai_stop_reason: 'queued'- a lifecycle state, not a stop reason.max_output_tokens, the native OpenAI wrapper reportsincomplete. The PostHog trace view only recognizes the first, so native-wrapper truncations render as normal endings.Follow-up to #4700, from its post-merge review.
Changes
$ai_stop_reasonfrom both the LangChain callback and the native wrapper, and queued or in-progress background runs carry none.responsesStopReasonhelper does the mapping: non-terminal statuses yield nothing, an incomplete run is named byincomplete_details.reason(e.g.max_output_tokens), the other terminal statuses stand for themselves.Note
Behavior change: the native wrapper reports a truncated run as
max_output_tokensinstead ofincomplete. Dashboards grouping on$ai_stop_reasonsee the new value after this ships. Cancelled runs staycancelled.Release info Sub-libraries affected
Libraries affected
Checklist
If releasing new changes
pnpm changesetto generate a changeset fileTesting
status: 'queued'produces no stop reason - fails without the terminal gate.incomplete_details: null, the shape every real completed run has.openai,azure-openai,openai-stream-accumulators, andopenai-background-responsessuites expectmax_output_tokensfor the incomplete status.statusfield:stopReasonis set only alongsideterminalResponsenow, so the old cast could never hold a real status.🤖 Agent context
Autonomy: Human-driven (agent-assisted)
Written by Claude Code from the confirmed findings of a max-level review of #4700: the missing terminal-status gate, the stop-reason vocabulary split between the two wrappers, and the stale comment. The remaining review findings (streamed truncations on @langchain/openai <=1.4.6, untested generation-level metadata legs) were left out as lower priority. Skills invoked: code-review, writing-pr-descriptions.