Add opentelemetry-instrumentation-genai-smolagents instrumentation package

[alexander-akhmetov]

Description

Ports openinference-instrumentation-smolagents into a new opentelemetry-instrumentation-genai-smolagents package. All telemetry is emitted through the public opentelemetry-util-genai API.

What is instrumented:

• MultiStepAgent.run, sync and streaming, as invoke_agent
• the model classes that define generate, as chat
• Tool.__call__, as execute_tool

It also wraps smolagents' local_python_executor.timeout, so tool spans started from agent-generated code keep the agent span as their parent instead of becoming root spans.

Known gaps, also listed in the package README.rst:

• No span for an agent step. Neither the semantic conventions nor opentelemetry-util-genai have a step or chain concept, so the OpenInference "Step N" CHAIN spans are dropped and chat and execute_tool spans nest directly under invoke_agent. See Adding ReAct Iterations Spans in Reasoning-Acting Agents semantic-conventions-genai#81.
• Model.generate_stream is not instrumented, so an agent created with stream_outputs=True reports no chat span and no token usage.
• A custom Model subclass that overrides generate shadows the patched base method and reports no chat span. A subclass that inherits generate is instrumented.
• A tool subclassing PipelineTool, including the shipped SpeechToTextTool, overrides Tool.__call__ and reports no execute_tool span.
• gen_ai.tool.call.id is not recorded. Tool.__call__ receives only the argument values, so the provider's tool call id never reaches the instrumentation.

Part of #141

Type of change

• New feature (non-breaking change which adds functionality)

How has this been tested?

Unit tests for the agent, model and tool patches, VCR-backed, and Weaver conformance scenarios for invoke_agent, chat and multimodal input. Run against smolagents 1.24.0:

uv run tox -e py310-test-instrumentation-genai-smolagents-oldest
uv run tox -e py314-test-instrumentation-genai-smolagents-latest
uv run tox -e py314-test-instrumentation-genai-smolagents-conformance

Checklist

See CONTRIBUTING.md
for the style guide, changelog guidance, and more.

• Followed the style guidelines of this project
• Changelog updated if the change requires an entry
• Unit tests added
• Documentation updated

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: alexander-akhmetov / name: Alexander Akhmetov (e734f71)

[opentelemetry-pr-dashboard]

Pull request dashboard status

Waiting on the author · refreshed 2026-08-04 00:09 UTC

Move out of draft to request review.

Status above doesn't look right?

• Just replied or pushed? Anything around or after the refresh time above may not be picked up yet — give it a few minutes.
• Anything look wrong? Report it with what you expected; it helps us improve the dashboard.

[alexander-akhmetov]

Split the PR, first part is the skeleton: #349

[alexander-akhmetov]

Migration review: opentelemetry-instrumentation-genai-smolagents

Mode: greenfield migration (the predecessor commit 2cabe19 added only a
skeleton: a no-op instrumentor, lifecycle tests, and the tox/workspace wiring.
Commit 67581e2 under review adds all instrumentation, all telemetry tests, and
the conformance scenarios.)

Compared against:

• OpenInference: openinference-instrumentation-smolagents
  (~/projects/donation-openinference/python/instrumentation/openinference-instrumentation-smolagents,
  https://github.com/open-telemetry/donation-openinference)

1. Instrumented API surface

API method	OpenInference	This package	Notes
smolagents.CodeAgent._step_stream	✅ (CHAIN span Step N)	❌	no semconv operation defined yet — semantic-conventions-genai#81
smolagents.ToolCallingAgent._step_stream	✅ (CHAIN span Step N)	❌	no semconv operation defined yet — semantic-conventions-genai#81
smolagents.MultiStepAgent.run	✅	✅	invoke_agent; covers managed agents, which are invoked through __call__ -> run
smolagents.models.Model.generate	✅	✅	chat
smolagents.models.MLXModel.generate	✅	✅	chat
smolagents.models.TransformersModel.generate	✅	✅	chat
smolagents.models.InferenceClientModel.generate	✅	✅	chat
smolagents.models.LiteLLMModel.generate	✅	✅	chat
smolagents.models.OpenAIModel.generate	✅	✅	chat; also exported as OpenAIServerModel. Upstream wraps both names and so wraps the class twice
smolagents.models.VLLMModel.generate	✅	✅	chat
smolagents.models.AmazonBedrockModel.generate	✅	✅	chat; also exported as AmazonBedrockServerModel
smolagents.models.ApiModel.generate	✅ (wraps the inherited method on the subclass)	✅	chat; here the base Model.generate patch covers it
smolagents.models.AzureOpenAIModel.generate	✅ (wraps the inherited method on the subclass)	✅	chat; here the OpenAIModel patch covers it. Upstream needs _has_active_llm_parent_span() to drop the duplicate span this produces
smolagents.models.LiteLLMRouterModel.generate	✅ (wraps the inherited method on the subclass)	✅	chat; here the LiteLLMModel patch covers it
smolagents.models.TransformersModel.generate_stream	—	✅	chat, streamed
smolagents.models.InferenceClientModel.generate_stream	—	✅	chat, streamed
smolagents.models.LiteLLMModel.generate_stream	—	✅	chat, streamed
smolagents.models.OpenAIModel.generate_stream	—	✅	chat, streamed
smolagents.tools.Tool.__call__	✅	✅	execute_tool
smolagents.tools.PipelineTool.__call__	—	✅	execute_tool; PipelineTool overrides Tool.__call__ without delegating, so upstream emits no span for SpeechToTextTool
smolagents.ToolCallingAgent.process_tool_calls	—	✅	no span; publishes the step's tool call ids for the execute_tool spans
smolagents.local_python_executor context shim	✅ (ThreadPoolExecutor)	✅ (timeout)	no span either way. timeout() holds the module's only ThreadPoolExecutor use, so coverage is the same

2. Gaps and open issues

Gap	File / test	Upstream issue	Notes
No span per agent step. The CHAIN span upstream emits per ReAct iteration has no semconv counterpart, so chat and execute_tool nest directly under invoke_agent	src/.../smolagents/__init__.py (TODO before the timeout patch); no test	semantic-conventions-genai#81	Loses per-step grouping and the step-level input/output values
TelemetryHandler.invoke_local_agent() takes no provider argument, although AgentInvocation.__init__ accepts one and gen_ai.provider.name is conditionally required on gen_ai.client.operation.duration. The package writes the metric attribute by hand	src/.../smolagents/patch.py:797; tests/test_agents.py::test_agent_run_metrics	none filed	Same workaround as opentelemetry-instrumentation-genai-openai-agents/.../processor.py:108 for tool spans, so the gap is repo-wide
SyncStreamWrapper ties one invocation= argument to two behaviors: the gen_ai.request.stream marker and the per-chunk timing metrics. An agent run wants the marker but not the metrics (its chunks are step objects, not response chunks), so it passes nothing and a streamed agent.run(stream=True) is indistinguishable from a non-streamed one	src/.../smolagents/patch.py _AgentRunStreamWrapper.__init__; tests/test_agents.py::test_streaming_run_records_no_chunk_metrics	none filed	Needs the wrapper to separate the two, or an invocation-level opt-out for chunk metrics

3. Significant behavioral changes

Aspect	Upstream	This package	Notes
Token counting	Sums agent.monitor totals onto the run span as well as the model spans	Reports tokens only on the chat span of the call that produced them	Documented in README.rst. An agent whose model class is not one of the patched ones (a user subclass overriding generate) now reports no tokens anywhere
Recoverable tool errors	_record_step_error turns AgentToolCallError / AgentToolExecutionError into an agent.step_recovery span event and keeps the step span OK	No step span exists, so the execute_tool span records error.type and ERROR status, and the invoke_agent span still ends UNSET	tests/test_agents.py::test_expected_tool_error_is_recorded_and_agent_continues. A consumer counting error spans now sees the tool failure the agent recovered from
Error recording	span.record_exception(e) plus ERROR status, so the span carries an exception event with the stack trace	invocation.fail(error), so the span carries error.type and ERROR status and no stack trace	util-genai owns this; no instrumentation in this repo records exception events
Request parameters	Dumps the merged model and call kwargs into llm.invocation_parameters as JSON	Maps only the parameters semconv names (temperature, top_p, max_tokens, seed, stop_sequences, output_type, ...)	Provider-specific parameters are dropped, for example the thinking={"type": "enabled", ...} used by the LiteLLM reasoning test
Agent-level custom attributes	smolagents.task, smolagents.additional_args, smolagents.max_steps, smolagents.tools_names, smolagents.managed_agents.*	Task and additional args go into gen_ai.input.messages; tools and managed agents into gen_ai.tool.definitions	smolagents.max_steps has no semconv counterpart and is not recorded, which README.rst states. A run that gives up is still visible through gen_ai.response.finish_reasons=length
Instrumentation suppression	Every wrapper returns early when context_api._SUPPRESS_INSTRUMENTATION_KEY is set	Not checked	suppress_instrumentation no longer silences smolagents spans. util-genai does not check the key either, and no package in this repo does, so this is a repo-wide behavior

4. Test coverage

py310-…-oldest (smolagents 1.24.0) and py312-…-latest (1.26.0): 154 passed.
py314-…-conformance: 5 passed. lint-instrumentation-genai-smolagents,
ruff format --check, readme, and typecheck are clean. tox -e precommit
could not run here: the nono sandbox denies the POSIX semaphore that
pre-commit needs (PermissionError: [Errno 1] Operation not permitted), so
the ruff, ruff-format, uv-lock, and rstcheck hooks were run individually
instead, minus rstcheck.

4a. Unit-test matrix per wrapped method

none

smolagents exposes no async API in 1.24.0 or 1.26.0 (no async def anywhere in
the package), so the async rows do not apply.

Cassettes: all 5 under tests/cassettes/ are referenced, and all 5 come from the
upstream cassette directory (same request bodies and response ids). No missing
cassette, no unreferenced cassette.

4b. Conformance scenarios

Operation	Scenario file	Status
chat	tests/conformance/inference.py, tests/conformance/multimodal.py	pass
invoke_agent	tests/conformance/agent.py	pass
execute_tool	tests/conformance/agent.py (nested in the agent run)	pass

ReasoningScenario declares one ExpectedViolation:
genai_expected_attribute_missing for server.address, because a
LiteLLMModel built without an explicit api_base exposes no endpoint URL.
That is accurate, not a workaround. No scenario is skipped.

4c. Docstring / README coverage

none

5. Follow-up work

Nothing blocks this PR.

Follow-ups, each its own PR:

API surface

• "smolagents: emit a span per agent step once semconv defines one" — closes the
  only ❌ rows in §1 (CodeAgent._step_stream, ToolCallingAgent._step_stream,
  covered by OpenInference as CHAIN spans). Blocked on
  semantic-conventions-genai#81.

util-genai gaps

• "util-genai: let invoke_local_agent() take a provider" — removes the manual
  metric_attributes[gen_ai.provider.name] write in smolagents and the same
  workaround in openai-agents.
• "util-genai: separate the stream marker from the per-chunk metrics in
  SyncStreamWrapper" — would let an agent-run wrapper record
  gen_ai.request.stream without claiming time_to_first_chunk /
  time_per_output_chunk.

Behavioral parity

• "smolagents: add the package to [tool.pyright] include" — the package is
  fully annotated but nothing checks it; pyright in strict mode currently reports
  54 errors for it, almost all from the untyped smolagents surfaces it calls.