[api][plan][python] Use request-scoped metric groups for chat token metrics#861
[api][plan][python] Use request-scoped metric groups for chat token metrics#861jlon wants to merge 1 commit into
Conversation
| this resource's currently bound metric group is used. | ||
| """ | ||
| metric_group = self.metric_group | ||
| if metric_group is None: |
There was a problem hiding this comment.
The two languages now document slightly different contracts for the absent-group case: Java's 4-arg recordTokenMetrics treats a null group as "skip recording" (BaseChatModelSetup.java:135), while here metric_group is None falls back to self.metric_group — the live, possibly-rebound group. Every real call site passes a captured group (chat_model_action.py:334), so there's no impact on today's flow. The part I'm less sure about: a future caller passing None expecting Java parity would record under whatever action is currently bound — the #859 scenario again on the Python side, where Java would no-op instead. Is the asymmetry intentional? If the = None default is only there to keep the param optional for the test helper, dropping it so callers must pass the captured group would match Java's no-fallback contract; alternatively a one-line javadoc note that Java's null means skip would at least make each side self-documenting.
What
Why
Cached resources can be rebound when another action retrieves the same resource before the original request records token usage. The delayed token metrics should stay under the action that initiated the request.
Closes #859.
Tests