Skip to content

fix(models): map Anthropic model_context_window_exceeded to MAX_TOKENS - #7584

Open
pcbeingused333 wants to merge 1 commit into
google:mainfrom
pcbeingused333:fix/anthropic-context-window-finish-reason
Open

pcbeingused333 wants to merge 1 commit into
google:mainfrom
pcbeingused333:fix/anthropic-context-window-finish-reason

Conversation

@pcbeingused333

Copy link
Copy Markdown

Please ensure you have read the contribution guide before creating a pull request.

Link to Issue or Description of Change

2. Or, if no issue exists, describe the change:

Problem:

Claude Sonnet 4.5 and newer models end a turn with stop_reason: "model_context_window_exceeded" when generation stops because the context window is full. AnthropicLlm's _STOP_REASON_MAPPING doesn't list it, so to_google_genai_finish_reason maps it to FINISH_REASON_UNSPECIFIED, in both the streaming and non-streaming paths.

The output is truncated just as with max_tokens, but nothing downstream sees that:

  • telemetry's is_reported_finish_reason treats UNSPECIFIED as "no finish reason reported";
  • callers that check for MAX_TOKENS, such as the LLM classifier's "ran out of output tokens" guard, accept the truncated text as a complete answer.

Solution:

Map model_context_window_exceeded to FinishReason.MAX_TOKENS, the same as max_tokens. The mapping is now keyed by str, because the stop reason is newer than the StopReason literal in the minimum supported anthropic (0.78). Unknown values still map to FINISH_REASON_UNSPECIFIED.

Testing Plan

Unit Tests:

  • I have added or updated unit tests for my change.
  • All unit tests pass locally.

Added a model_context_window_exceeded case to test_to_google_genai_finish_reason. pytest tests/unittests/models/test_anthropic_llm.py: 184 passed (anthropic 1.15.0). On main only the new case fails. pyink --check and isort --check-only are clean on the touched files, and mypy reports the same errors on anthropic_llm.py as on main.

Manual End-to-End (E2E) Tests:

Not run: triggering this stop reason needs a Claude request that fills the context window. The change only affects the stop reason → finish reason mapping, which the unit test exercises directly.

Checklist

  • I have read the CONTRIBUTING.md document.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass locally with my changes.
  • I have manually tested my changes end-to-end.
  • Any dependent changes have been merged and published in downstream modules.

Additional context

This change was drafted with Claude Code; the tests above were run before opening the PR.

🤖 Generated with Claude Code

Claude Sonnet 4.5 and newer end a turn with stop_reason
"model_context_window_exceeded" when generation fills the context
window. It was missing from the stop reason mapping and fell through to
FINISH_REASON_UNSPECIFIED, so a truncated response looked like one
without a reported finish reason, and callers that check for MAX_TOKENS
(e.g. the LLM classifier) accepted the truncated output.
@google-cla

google-cla Bot commented Oct 17, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant