feat(util-genai): add per-signal context-scoped attributes

[RKest]

Description

Adds set_context_scoped_attributes to opentelemetry-util-genai: a caller
attaches attributes to an OpenTelemetry context, and GenAI telemetry emitted
within that context carries them. Each attribute declares which signal it
applies to.

token = context.attach(
    set_context_scoped_attributes(
        span_attributes={"gen_ai.agent.name": "trip-planner"},
        log_attributes={"user.id": user_id},
    )
)
try:
    client.chat.completions.create(...)  # instrumented elsewhere
finally:
    context.detach(token)

Motivation

GenAI instrumentation is layered across packages that share no call path. An
agentic framework knows which agent, workflow, or conversation is running. The
model-client instrumentation that emits the inference telemetry
(opentelemetry-instrumentation-genai-openai and friends) sits a layer below
it and has no way to learn any of it:

• Span parentage is traces-only, the event carries no parent attributes, and
  OTel spans are write-only — an inference instrumentation cannot read
  gen_ai.agent.name off its parent span even when one exists.
• Threading it through the call is impossible; you cannot pass an agent name
  through openai.chat.completions.create().
• Instrumentation config is static per-process; agent identity is
  per-invocation.
• Baggage propagates cross-service (wrong scope) and is string-only.

The context is the only in-process channel between a component that knows the
fact and a component that emits the telemetry needing it. Concretely,
gen_ai.agent.name, gen_ai.agent.id, and gen_ai.conversation.id are
semconv-defined attributes that plainly belong on inference telemetry and that
no code path can currently populate — a spec-defined population gap rather than
a vendor extension.

Per-signal targeting is the second half of the requirement: content that is
unsafe on a sampled, widely-read span (user identifiers, tenant data) can still
be recorded on the event.

Semantics

• Attributes apply to GenAI telemetry emitted by this package only. Other
  instrumentation, and telemetry the application emits directly, are unaffected.
• Attributes are never propagated out of the process — they live on the context
  under a private key, not in Baggage or any wire format.
• span_attributes are merged into the start_span call, so they are visible
  to samplers.
• log_attributes apply to gen_ai.client.inference.operation.details, the only
  event this package emits — so they reach inference invocations only.
• Attributes an invocation sets itself take precedence over context-scoped ones.
• Nested calls merge, with the inner call winning for keys it sets.
• An invocation reads the context once, when it starts.
• Metrics are not supported (see below).

Why metrics are excluded

Spans and logs absorb an extra attribute harmlessly. Metrics do not: every
distinct value forks a new time series, so one context-scoped user.id or
gen_ai.conversation.id on the inference histograms is an unbounded cardinality
explosion — expensive, and often noticed only once a backend starts dropping
series. The failure is easy to cause, hard to detect, and hard to undo.

Rather than ship a foot-gun with a warning, there is no metrics target member at
all. If a pressing need appears, the safe form is an explicit allow-list of
attributes permitted on metrics, not a free-form bag. Adding that later is
straightforward; taking a free-form bag away after users depend on it is not.

Alternatives considered

The requirement — attributes on the inference event but not the span, set by a
layer above the instrumentation — was first attacked outside the
instrumentation, via the log pipeline. None of those hold up:

• A custom LogRecordProcessor wrapping the exporting processor
  (add_log_record_processor(BatchLogRecordProcessor(StampingProcessor(exporter)))).
  Requires knowing and wrapping every processor the application installs, and
  re-wrapping whenever that set changes. Fragile by construction.
• A stamping processor installed first, relying on processor ordering.
  Processor order is not a stability guarantee to build on.
• A custom LoggerProvider overriding emit. The provider is global and set
  by the application; a library cannot force its own implementation on users.
• Installing a custom provider only on one deployment target. Telemetry would
  then differ by where the agent runs, and deployment-specific code leaks into
  framework core.

A SpanProcessor counterpart to any of these is possible for the span half,
with the caveat the OTEP itself notes: OnStart runs after the sampler, so
those attributes would not reach sampling decisions.

• Implementing OTEP 4931 in the Python SDK. The right long-term home, but the
  OTEP requires the feature be off unless the user opts in per-signal on the
  provider, which is exactly the provider control a library does not have. It
  also cannot express per-attribute signal targeting (below).
• Generalising Prototype showing gen_ai.conversation_root span attribute to mark the root GenAI span of a conversation #187 in place. Prototype showing gen_ai.conversation_root span attribute to mark the root GenAI span of a conversation #187 stamps one hardcoded attribute at span
  finish — after sampling — and merges outermost-wins. Both are wrong for this
  use case; this module is the generalisation it should rebase onto.

Relation to OTEP 4931

This borrows the vocabulary of
OTEP 4931
but is not an implementation of it, and does not claim conformance.

	OTEP 4931	This PR
Who stamps telemetry	the SDK	opentelemetry-util-genai
Scope	any telemetry from any component	GenAI telemetry from this package
Gating	per-signal, on the provider	per-attribute, at the write site
Metrics	MUST, for sync instruments	not supported

The OTEP assigns stamping exclusively to the SDK, and only when the user has
opted in per-signal; it never contemplates an instrumentation doing the
stamping, so its SDK requirements do not bind here. What this package does is
read a context variable to decide which attributes to put on its own telemetry —
ordinary instrumentation behaviour, and what
GENERATE_CONTENT_EXTRA_ATTRIBUTES_CONTEXT_KEY in
opentelemetry-instrumentation-google-genai already does today in a hardcoded,
single-package form.

The OTEP does consider a non-SDK-core stamper, in the rejected "built-in
Processor" alternative, and rejects it for two mechanical reasons: a processor's
OnStart runs after the sampler, and it would need the SDK's internal context
key. Neither applies here — attributes are merged into the start_span call, so
samplers see them, and this module owns its own key.

Two deliberate consequences worth flagging:

• Not a stepping stone. When the Python SDK implements 4931, this does not
  become it. 4931 cannot express "on the event, not on the span":
  AddContextScopedAttributes(Context, Attributes) has no signal parameter, and
  its gating is on the provider, so disabling traces would drop every
  context-scoped attribute from spans rather than the one that is sensitive.
• The setter opts in, not the provider. Nothing is stamped unless a caller
  explicitly calls set_context_scoped_attributes, so the feature costs nothing
  when unused and needs no provider-level configuration — which is what makes it
  usable by libraries that do not control SDK setup.

Scope

Deliberately limited to the public API plus the two stamping sites. Follow-ups:

• Deprecating GENERATE_CONTENT_EXTRA_ATTRIBUTES_CONTEXT_KEY in favour of this.
• Rebasing Prototype showing gen_ai.conversation_root span attribute to mark the root GenAI span of a conversation #187 (gen_ai.conversation_root) onto this module.

Open question

Should there be an env kill switch (OTEL_INSTRUMENTATION_GENAI_CONTEXT_SCOPED_ATTRIBUTES=false)
so an application owner can disable stamping when a dependency writes attributes
they do not want? Not included; happy to add if maintainers prefer it. This is
adjacent to the OTEP's own open question about whether instrumentation relying
on this feature must always expose it as opt-in.

How has this been tested?

New util/opentelemetry-util-genai/tests/test_context_attributes.py (5 tests)
covering: per-signal targeting, invocation attributes winning over
context-scoped ones, nested inner-wins merge, visibility to a recording sampler,
and no leakage outside the attached context.

Also verified end-to-end against a live provider: opentelemetry-instrumentation-google-genai
calling Gemini on Vertex AI, with attributes set as in the example above and
both signals exported over OTLP to a real backend (Google Cloud). Reading the
telemetry back from the backend:

• the span carries gen_ai.agent.name=trip-planner and no user.id;

• the gen_ai.client.inference.operation.details log record carries
  user.id=<generated> and no gen_ai.agent.name;

• both correlate to the same trace/span IDs.

So the per-signal split survives serialisation, export, and ingestion, not just
the in-process exporter assertions.

uv run tox -e typecheck passes. uv run tox -e precommit passes except the
uv-lock hook, which rewrites indexes in my environment for unrelated reasons.

Checklist

• Followed the style guidelines of this project
• Changelog updated if the change requires an entry
• Unit tests added
• Documentation updated

[opentelemetry-pr-dashboard]

Pull request dashboard status

Closed · refreshed 2026-08-04 15:04 UTC

Status above doesn't look right?

• Anything look wrong? Report it with what you expected; it helps us improve the dashboard.

[Copilot]

Pull request overview

Adds a new opentelemetry-util-genai utility for attaching per-signal attribute bags (span vs log/event) to an OpenTelemetry Context, and wires the util’s span/event emission points to merge those context-scoped attributes at invocation start/finish.

Changes:

• Introduce set_context_scoped_attributes(...) to build a derived Context carrying GenAI-only, non-propagating attribute bags for spans and inference events.
• Apply context-scoped span attributes at invocation start (so samplers can see them) and context-scoped log attributes when emitting the inference details event.
• Add unit tests, README documentation, and a towncrier changelog fragment for the new API.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
util/opentelemetry-util-genai/src/opentelemetry/util/genai/context_attributes.py	New context helper storing per-signal attribute bags under a private context key.
util/opentelemetry-util-genai/src/opentelemetry/util/genai/_invocation.py	Reads context-scoped attributes once at invocation start and merges span-targeted attributes into start_span(...).
util/opentelemetry-util-genai/src/opentelemetry/util/genai/_inference_invocation.py	Merges log-targeted context attributes into the inference event attribute set.
util/opentelemetry-util-genai/tests/test_context_attributes.py	New tests validating per-signal targeting, precedence, nesting merge behavior, sampler visibility, and scoping.
util/opentelemetry-util-genai/README.rst	Documents the new context-scoped attributes feature and its intended semantics/limitations.
util/opentelemetry-util-genai/.changelog/337.added	Changelog fragment announcing the new API.

Suppressed comments (2)

util/opentelemetry-util-genai/tests/test_context_attributes.py:163

• Use GenAI.GEN_AI_AGENT_NAME instead of hardcoding "gen_ai.agent.name" so the test stays aligned with the semconv constants used elsewhere in this repo’s tests.

        with self._inference(
            set_context_scoped_attributes(
                span_attributes={"gen_ai.agent.name": "trip-planner"}
            )
        ):
            pass

        (seen,) = self.sampler.seen
        self.assertEqual(seen["gen_ai.agent.name"], "trip-planner")

util/opentelemetry-util-genai/tests/test_context_attributes.py:177

• Use GenAI.GEN_AI_AGENT_NAME instead of hardcoding "gen_ai.agent.name" for the semconv attribute key, consistent with other tests in this package.

        with self._inference(
            set_context_scoped_attributes(
                span_attributes={"gen_ai.agent.name": "trip-planner"}
            )
        ):
            pass
        self.span_exporter.clear()

        with self.handler.inference("test-provider"):
            pass

        self.assertNotIn("gen_ai.agent.name", self.span_attributes)

[lmolkova]

This should come from otel-python following https://github.com/open-telemetry/opentelemetry-specification/blob/main/oteps/4931-context-scoped-attributes.md and the spec changes (not spec-ed out yet).

We should not polyfill missing SDK features here

[RKest]

This should come from otel-python following https://github.com/open-telemetry/opentelemetry-specification/blob/main/oteps/4931-context-scoped-attributes.md and the spec changes (not spec-ed out yet).

We should not polyfill missing SDK features here

Are you proposing then making our own OTEP to extend https://github.com/open-telemetry/opentelemetry-specification/blob/main/oteps/4931-context-scoped-attributes.md?

What's currently missing for our use-case is:

1. The ability to set attributes on a per-signal basis. OTEP only specifies per-signal opt-in, but AFAIU the bag of attributes is shared across all signals. We have a privacy requirement in ADK, to store user.id attribute only in logs, and a narrower set of context-scoped span attributes.

2. Ideally the attributes would be default-on. Then we could maintain the canonical set of telemetry emitted for the default provider setup, and we could confidently assure telemetry consumers which attributes will be present, and are safe to be relied upon.

   Also the quote "Instrumentation libraries desiring to set Context-scoped attributes SHOULD offer this as an opt-in behavior" means users would have to go through two opt-ins, to get the canonical set of telemetry in ADK.

   I can see how default-on context scoped attributes across all of OTel SDK, could be problematic, because it would allow for large blast radius changes. Default-on limited to GenAI instrumentation, I hope should be more defendable, especially given how quickly the GenAI space evolves, and how fragmented across different technologies, standards and ultimately -- instrumentors it is. In other words, I hope the pro of flexibility outweighs the con of inconsistency, in order to make this carve out acceptable.

3. I imagine at some point, there will likely be a need to set context-scoped attributes on a per-metric/span/log basis. I can imagine a metric in the inference-specific (i.e. outside of any agentic framework) instrumentor, which has a gen_ai.agent.name attribute, which shouldn't be shared with other metrics.

   I didn't implement this here, but a more experimental/malleable API, seems like a good opportunity to see how truly necessary this is, before fully committing to OTEP.

In any case, I'd love to hear your thoughts on the general direction we should take

[lmolkova]

@RKest I'm saying context-scoped attributes should be implemented in https://github.com/open-telemetry/opentelemetry-python, not in this repo, but before that the API and implementation details should be covered in the specification. For now, user apps can implement context-based processors

[RKest]

@RKest I'm saying context-scoped attributes should be implemented in https://github.com/open-telemetry/opentelemetry-python, not in this repo, but before that the API and implementation details should be covered in the specification. For now, user apps can implement context-based processors

SG, thank you.

I'll get the ball rolling on the OTEP proposal, and in the meantime I'll try to have an interim quick fix in ADK to get around the current limitations.

[RKest]

Opened open-telemetry/opentelemetry-specification#5247