Skip to content

Minor cleanup around tool invocation tool call args / results #292

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
DylanRussell wants to merge 5 commits into
base: main
Choose a base branch
from
Draft

Minor cleanup around tool invocation tool call args / results #292

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
fe0712b
Minor changes to tools
DylanRussell Jul 20, 2026
b3a2682
Cleanup
DylanRussell Jul 20, 2026
8f3b189
Revert small thing
DylanRussell Jul 20, 2026
b6c4533
Changelog and precommit
DylanRussell Jul 20, 2026
dabd597
Remove empty file
DylanRussell Jul 20, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions instrumentation/opentelemetry-instrumentation-genai-langchain/.changelog/292.changed
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Avoid redundant JSON encoding of tool arguments.
33 changes: 21 additions & 12 deletions ...ion-genai-langchain/src/opentelemetry/instrumentation/genai/langchain/callback_handler.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@

from __future__ import annotations

import json
from typing import Any, Optional, cast
from uuid import UUID

Expand Down Expand Up @@ -40,6 +39,7 @@
Text,
ToolCallRequest,
)
from opentelemetry.util.genai.utils import gen_ai_json_dumps


class OpenTelemetryLangChainCallbackHandler(BaseCallbackHandler):
Expand Down Expand Up @@ -431,19 +431,22 @@ def on_tool_start(
if serialized is not None:
name = serialized.get("name") or "unknown"
description = serialized.get("description")

arguments: Any
if inputs is not None:
arguments = inputs
else:
try:
arguments = json.loads(input_str)
except (json.JSONDecodeError, ValueError):
arguments = input_str
tool_invocation = self._telemetry_handler.tool(
name=name, tool_description=description, tool_type="function"
)
tool_invocation.arguments = arguments
if self._telemetry_handler.should_capture_content():
arguments: Any
if inputs is not None:
arguments = inputs
else:
arguments = input_str

tool_invocation.arguments = (

@lmolkova lmolkova Jul 20, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

they are json dumped only for the time being until we can populate complex attributes on spans. I'd prefer to either keep deserialization in instrumentations or make utils deserialize them when they come as string (since it's common part for all instrumentations). If we do the latter, we should also do this in other instrumentation libs and eventually util-genai will switch to complex attrs AND will start deserailizing these at the same time.

@DylanRussell DylanRussell Jul 20, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does complex attributes help here ? The hard problem here is converting arguments (which can be essentially anything) into the narrow otel attributes type, complex attributes just expands the type a tiny bit

At the moment I see only 3 instrumentations with tool invocations:

opentelemetry-python-genai/instrumentation/opentelemetry-instrumentation-genai-langchain/src/opentelemetry/instrumentation/genai/langchain/callback_handler.py

Lines 436 to 442 in 2d0d28f

if inputs is not None:
arguments = inputs
else:
try:
arguments = json.loads(input_str)
except (json.JSONDecodeError, ValueError):
arguments = input_str

instrumentation/opentelemetry-instrumentation-google-genai/src/opentelemetry/instrumentation/google_genai/tool_call_wrapper.py

opentelemetry-python-genai/instrumentation/opentelemetry-instrumentation-genai-openai-agents/src/opentelemetry/instrumentation/genai/openai_agents/processor.py

Line 106 in 2d0d28f

invocation.arguments = span_data.input

^^ I don't really see what common code to pull out of this into the utils library.. I also still prefer pulling that serialization out of the utils library where it's kind of hidden.

So in general you prefer the tool arguments / results are not serialized and somehow better represented ?

@lmolkova lmolkova Jul 20, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

semconv tell to have them not serialized

https://github.com/open-telemetry/semantic-conventions-genai/blob/c26a2c21d1ee70d5231bd440c7b48d3c94ee506a/model/gen-ai/registry.yaml#L345

  It's expected to be an object - in case a serialized string is available
 to the instrumentation, the instrumentation SHOULD do the best effort to
 deserialize it to an object.

how I see it can work:

  • instrumentations either
    • deserialize them into ToolCallArguments - we'll add a type even though it's just an alias to AnyValue
    • keep them as string since they've got string from model
  • utils in the future once complex attributes on spans are supported:
    • if it's string, try deserializing it into json object (AnyValue)
    • if it's a ToolCallArguments already, just set it
    • ignore otherwise
  • utils now:
    • if it's a string, stamp as is
    • serialize into string otherwise

@DylanRussell DylanRussell Jul 20, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay i think we are mostly aligned then, just leave it up to instrumentations to translate arguments into AnyValue and accept that as the param ?

Maybe lets just revisit once complex attributes is available as that should be pretty soon..

arguments
if isinstance(arguments, str)
else gen_ai_json_dumps(arguments)
)

self._invocation_manager.add_invocation_state(
run_id, parent_run_id, tool_invocation
)
Expand All @@ -460,7 +463,13 @@ def on_tool_end(
if not isinstance(tool_invocation, ToolInvocation):
return
tool_invocation.tool_call_id = getattr(output, "tool_call_id", None)
tool_invocation.tool_result = getattr(output, "content", None)
raw_result = getattr(output, "content", output)
if self._telemetry_handler.should_capture_content():
tool_invocation.tool_result = (
raw_result
if isinstance(raw_result, str)
else gen_ai_json_dumps(raw_result)
Comment on lines 466 to +472
)
tool_invocation.stop()
if not tool_invocation.span.is_recording():
self._invocation_manager.delete_invocation_state(run_id=run_id)
Expand Down
7 changes: 5 additions & 2 deletions ...on-genai-openai-agents/src/opentelemetry/instrumentation/genai/openai_agents/processor.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@
GenAIInvocation,
ToolInvocation,
)
from opentelemetry.util.genai.utils import gen_ai_json_dumps

# Non-semconv attribute: surfaces the workflow name on the workflow span
# so callers can query/filter by it. util-genai's WorkflowInvocation
Expand Down Expand Up @@ -126,9 +127,11 @@ def on_span_end(self, span: Span[Any]) -> None:
span.span_data, FunctionSpanData
):
output = span.span_data.output
if output is not None:
if self._handler.should_capture_content() and output is not None:
invocation.tool_result = (
output if isinstance(output, str) else str(output)
output
if isinstance(output, str)
else gen_ai_json_dumps(output)
)
Comment on lines +130 to 135
invocation.stop()

Expand Down
10 changes: 5 additions & 5 deletions ...entation-google-genai/src/opentelemetry/instrumentation/google_genai/tool_call_wrapper.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@

import functools
import inspect
import json
from typing import Any, Callable, Optional, Union

from google.genai.types import (
Expand All @@ -13,6 +12,7 @@
)

from opentelemetry.util.genai.handler import TelemetryHandler
from opentelemetry.util.genai.utils import gen_ai_json_dumps

ToolFunction = Callable[..., Any]

Expand Down Expand Up @@ -84,12 +84,12 @@ async def wrapped_function(*args, **kwargs):
) as tool_invocation:
# Do this before calling the tool in case that crashes.
if tool_invocation.should_capture_content_on_span:
tool_invocation.arguments = json.dumps(
tool_invocation.arguments = gen_ai_json_dumps(
_get_function_args(tool_function, args, kwargs)
)
result = await tool_function(*args, **kwargs)
if tool_invocation.should_capture_content_on_span:
tool_invocation.tool_result = json.dumps(
tool_invocation.tool_result = gen_ai_json_dumps(
_to_otel_value(result)
)
return result
Expand All @@ -103,12 +103,12 @@ def wrapped_function(*args, **kwargs):
) as tool_invocation:
# Do this before calling the tool in case that crashes.
if tool_invocation.should_capture_content_on_span:
tool_invocation.arguments = json.dumps(
tool_invocation.arguments = gen_ai_json_dumps(
_get_function_args(tool_function, args, kwargs)
)
result = tool_function(*args, **kwargs)
if tool_invocation.should_capture_content_on_span:
tool_invocation.tool_result = json.dumps(
tool_invocation.tool_result = gen_ai_json_dumps(
_to_otel_value(result)
)
return result
Expand Down
1 change: 1 addition & 0 deletions util/opentelemetry-util-genai/.changelog/292.changed
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Update ``ToolInvocation`` arguments and results to accept ``AnyValue`` attribute types directly.
37 changes: 10 additions & 27 deletions util/opentelemetry-util-genai/src/opentelemetry/util/genai/_tool_invocation.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,23 +11,8 @@
from opentelemetry.util.genai._invocation import Error, GenAIInvocation
from opentelemetry.util.genai.completion_hook import CompletionHook
from opentelemetry.util.genai.metrics import InvocationMetricsRecorder
from opentelemetry.util.genai.utils import (
gen_ai_json_dumps,
should_capture_content_on_spans,
)
from opentelemetry.util.types import AnyValue, AttributeValue


def _any_value_to_attribute_value(value: AnyValue) -> AttributeValue | None:
"""Serialize an AnyValue to an AttributeValue for OTel span attributes."""
if value is None:
return None
if isinstance(value, (bool, str, bytes, int, float)):
return value
try:
return gen_ai_json_dumps(value)
except (TypeError, ValueError):
return str(value)
from opentelemetry.util.genai.utils import should_capture_content_on_spans
from opentelemetry.util.types import AnyValue


class ToolInvocation(GenAIInvocation):
Expand Down Expand Up @@ -75,18 +60,18 @@ def __init__(
)
self.should_capture_content_on_span = should_capture_content_on_spans()
self.name = name
self.tool_result: AnyValue | None = None
self.tool_result: AnyValue = None
# Since arguments and tool_result can be expensive to serialize,
# it's recommended to check the content capture flag in the
# instrumentation library before assigning these attributes
# to the invocation.
self.arguments: AnyValue | None = None
self.arguments: AnyValue = None
self.tool_call_id = tool_call_id
self.tool_type = tool_type
self.tool_description = tool_description
self._start(self._get_base_attributes())

def _get_base_attributes(self) -> dict[str, AttributeValue]:
def _get_base_attributes(self) -> dict[str, AnyValue]:
"""Return sampling-relevant attributes available at span creation time."""
optional_attrs = (
(GenAI.GEN_AI_TOOL_NAME, self.name),
Expand All @@ -99,8 +84,8 @@ def _get_base_attributes(self) -> dict[str, AttributeValue]:
**{k: v for k, v in optional_attrs if v is not None},
}

def _get_metric_attributes(self) -> dict[str, AttributeValue]:
attrs: dict[str, AttributeValue] = {
def _get_metric_attributes(self) -> dict[str, AnyValue]:
attrs: dict[str, AnyValue] = {
Comment on lines +87 to +88
GenAI.GEN_AI_OPERATION_NAME: self._operation_name,
}
attrs.update(self.metric_attributes)
Expand All @@ -116,20 +101,18 @@ def _apply_finish(self, error: Error | None = None) -> None:
(GenAI.GEN_AI_TOOL_DESCRIPTION, self.tool_description),
(
GenAI.GEN_AI_TOOL_CALL_ARGUMENTS,
_any_value_to_attribute_value(self.arguments)
self.arguments
if self.should_capture_content_on_span
and self.arguments is not None
else None,
Comment on lines +104 to 106
),
(
GenAI.GEN_AI_TOOL_CALL_RESULT,
_any_value_to_attribute_value(self.tool_result)
self.tool_result
if self.should_capture_content_on_span
and self.tool_result is not None
else None,
),
)
attributes: dict[str, AttributeValue] = {
attributes: dict[str, AnyValue] = {
Comment on lines +110 to +115
GenAI.GEN_AI_OPERATION_NAME: self._operation_name,
**{k: v for k, v in optional_attrs if v is not None},
}
Expand Down
21 changes: 1 addition & 20 deletions util/opentelemetry-util-genai/tests/test_toolcall.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,6 @@
ServerToolCallResponse,
ToolCallRequest,
)
from opentelemetry.util.genai.utils import gen_ai_json_dumps


def _make_handler() -> TelemetryHandler:
Expand Down Expand Up @@ -208,24 +207,6 @@ def _make_span_exporter_and_handler():
return span_exporter, handler


@patch.dict(
os.environ,
{OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT: "SPAN_ONLY"},
)
def test_arguments_dict_serialized_to_json():
"""dict arguments are JSON-serialized onto the span attribute."""
span_exporter, handler = _make_span_exporter_and_handler()
invocation = handler.tool("get_weather")
invocation.arguments = {"location": "Paris", "unit": "celsius"}
invocation.stop()

attrs = span_exporter.get_finished_spans()[0].attributes
assert GenAI.GEN_AI_TOOL_CALL_ARGUMENTS in attrs
assert attrs[GenAI.GEN_AI_TOOL_CALL_ARGUMENTS] == gen_ai_json_dumps(
{"location": "Paris", "unit": "celsius"}
)


@patch.dict(
os.environ,
{OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT: "SPAN_ONLY"},
Expand Down Expand Up @@ -275,7 +256,7 @@ def test_arguments_omitted_when_content_capture_disabled():
"""arguments must not appear on the span when content capture is off."""
span_exporter, handler = _make_span_exporter_and_handler()
invocation = handler.tool("get_weather")
invocation.arguments = {"location": "Paris"}
invocation.arguments = "abasfasf"
invocation.stop()

attrs = span_exporter.get_finished_spans()[0].attributes
Expand Down
Loading