feat: add Rhesis integration for OpenTelemetry tracing - #3669
Open
Arman-Beykmohammadi wants to merge 16 commits into
Open
feat: add Rhesis integration for OpenTelemetry tracing#3669Arman-Beykmohammadi wants to merge 16 commits into
Arman-Beykmohammadi wants to merge 16 commits into
Conversation
Add the rhesis-haystack integration, providing a RhesisConnector component and an OpenTelemetry-based tracer that exports Haystack pipeline, component, and agent spans to Rhesis. Includes the Haystack-to-Rhesis semantic mapping layer, packaging (pyproject, pydoc config, py.typed), examples, and tests. Co-authored-by: Cursor <cursoragent@cursor.com>
…ntry Wire the new Rhesis integration into repo automation: add the test workflow, labeler rule, coverage-comment trigger, and the README integrations table row. Co-authored-by: Cursor <cursoragent@cursor.com>
Arman-Beykmohammadi
requested review from
davidsbatista
and removed request for
a team
July 27, 2026 05:47
|
|
github-actions
Bot
added
topic:CI
type:documentation
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
Contributor
|
@Arman-Beykmohammadi do you want to keep this PR open or move into your own repo? |
Author
|
@davidsbatista |
Haystack 3.0 removed `serialize_class_instance` and
`deserialize_class_instance` from `haystack.utils.base_serialization`, so
importing them made `RhesisConnector` fail to import on 3.0.
The span handler is an arbitrary user class rather than a component, so it
still needs the type-tagged `{"type": ..., "data": ...}` envelope those
helpers produced. Rebuild it locally on top of
`generate_qualified_class_name` and `import_class_by_name`, keeping the
serialized format byte-compatible so pipelines serialized before this
change keep deserializing.
Deserialization now also rejects a type that is not a `SpanHandler`
subclass, instead of failing later on a missing `from_dict`.
Haystack 3.0 moved the Agent loop off `Pipeline._run_component`: every iteration now opens its own `haystack.agent.step` span, and the LLM call and each tool call are traced directly as `haystack.agent.step.llm` and `haystack.agent.step.tool` instead of going through a `ToolInvoker` component span. Those spans carry none of the `haystack.component.*` tags, so the existing span-kind rules — which all key off component type — left them unresolved. An agent's LLM and tool work was exported as untyped `function.*` spans with no model, token, or tool attributes attached. Add an `_OPERATION_ONLY` rule form that matches on operation name alone, and map the new operations to `ai.llm.invoke`, `ai.tool.invoke`, and a `function.haystack.agent.step` grouping span. `RhesisSpan` now retains its operation name, which is the only discriminator available at enrichment time and is what lets reply metadata be promoted from the in-agent LLM span; that promotion moves into `_apply_chat_reply_metadata`, shared with the existing `ChatGenerator` component path. Tool arguments and results are stamped as `ai.tool.input`/`ai.tool.output` content rather than `ai.prompt`/`ai.completion` events, since they are not a prompt. The haystack 2.x `ToolInvoker` rules are kept, so 2.x pipelines trace exactly as before.
Haystack models delegation to a specialist agent as an ordinary tool call: the tool's function runs `Agent.run`, so the nested agent shows up as an `ai.agent.invoke` span under a plain `ai.tool.invoke` parent. Nothing in the exported trace said the call was a handoff, or which agent handed off to which. When an agent span opens directly inside a tool span, re-label that still-open tool span as `ai.agent.handoff` and stamp `ai.agent.handoff.to` with the tool name and `ai.agent.handoff.from` with the enclosing agent — the nearest ancestor tool span for a specialist, or the nearest component name for a top-level agent. The nested agent span itself picks up `ai.agent.name`. Add an end-to-end test that runs a coordinator agent delegating to a specialist through a tool, driven by a scripted chat generator, and asserts the shape of the exported span tree: a single pipeline root, one `agent.invoke` per agent, an `llm.invoke` per step, and the nested agent parented by the handoff span.
A pipeline root span stamped `conversation.input`/`conversation.output` by
running the entire pipeline input and output mappings through
`_stringify_content`. A turn was therefore displayed as a serialized dict —
`{"chat": {"messages": [...]}}` — rather than what the user asked and what
the pipeline replied.
Walk the per-component payloads instead and take the last user message and
the last assistant message from whichever component carries a chat history,
preferring an Agent's `last_message` when it reports one. Assistant turns
that only request tool calls carry no text, so they are skipped rather than
ending the search.
When no chat messages can be found the span now stamps no conversation text
at all. A serialized payload is never a valid rendering of a turn, so showing
nothing is the better failure mode.
Note that this is a fallback for pipelines traced with no Rhesis SDK endpoint
above them, not an authoritative record: only the application knows how it
derives its reply.
The role-matching logic is factored out and reused by the agent-span
extraction, which as a result also skips text-less messages instead of
stopping at the first one.
… owns it When a Haystack pipeline runs inside a Rhesis SDK `@endpoint` / `@observe` call, both the SDK root span and the Haystack root span stamped `is_turn_root`. Only one span per turn may carry that flag, so the exporter stripped the Haystack root's real parent and the whole Haystack subtree detached into a second conversation turn — with the pipeline payload restated as that turn's input and output. Decide turn ownership once, at span creation, from the SDK's `get_root_trace_id` context var: when the SDK has already opened the turn root, the Haystack root span is a child of that turn rather than a turn of its own. Such a span drops the `is_turn_root` flag and skips conversation input/output stamping altogether, since the SDK span already carries the mapped user message and reply. Session and conversation ids are kept — they are useful on a nested span and the exporter propagates them anyway. The check is deliberately keyed on the SDK context rather than on `trace.get_current_span()`, so that unrelated ambient instrumentation (an HTTP server span, say) cannot leave a turn with no root at all. Standalone Haystack, with nothing above it, still owns the turn.
Every published `rhesis-sdk` version from 0.9.1 on declares `requires-python >= 3.12`, so the Python 3.10 CI jobs could not even resolve the environment. Raise `requires-python` to match the SDK and move the test matrix (plus the lint and coverage jobs pinned to the lowest supported version) to 3.12. Raising the floor also raises ruff's inferred target version, which enables UP042 on `MappingPromotion`; `StrEnum` is available from 3.11. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The tree this test asserts — a span per agent step, a span per tool call, and the handoff promoted from the tool span that runs a nested Agent — only exists from Haystack 3.0 on. On 2.x an agent routes all of a step's tool calls through a single batched ToolInvoker component span, so the test failed in the "lowest direct dependencies" job, which resolves haystack-ai to the declared floor of 2.22.0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The lint job installs the latest ruff into a fresh environment, so it picked up two ruff 0.16 changes the moment the job started running: - `PLR0917` (too many positional arguments) left preview and now flags `RhesisConnector.__init__`. Ignore it, as every other integration in the repo that selects `PLR` already does, next to the `PLR0913` argument-count rule it complements. - `ruff format` now formats Python code blocks inside Markdown, which wants blank lines around the README's top-level definitions. Neither surfaced before: on Python 3.10 the lint job failed while resolving the environment, before ruff ever ran. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`RhesisConnector` covers the common case: add it to a pipeline and every run is traced. An application that owns its own loop — a chat REPL, a batch script, a server handling one turn per request — needs two things a component sitting inside the pipeline cannot provide: tracing switched on without a pipeline to attach to, and a span wrapping a whole pipeline run so that a conversation turn has a root of its own. Without that root span the Haystack pipeline span claims the turn and reports the serialized pipeline input and output as the conversation text, so the Rhesis conversation view shows a dict dump instead of what the user asked and what the application replied. Add `RhesisTracing`, which constructs the connector for its side effects and never adds it to a pipeline. `turn()` opens the turn root, stamps the conversation input, and sets the SDK's `root_trace_id` context var so the Haystack tracer defers instead of claiming the turn. Turns after the first attach to a synthetic non-recording parent carrying the first turn's trace id — the approach the Rhesis SDK uses for turns it serves itself — so a conversation reads as one trace rather than one per exchange. The exporter strips that placeholder parent, so every turn is still stored as a root span. The reply is assigned by the caller through `ConversationTurn.output`: only the application knows which part of a pipeline result is the user-facing answer, which may be a tool result or a value held in agent state rather than the last assistant message. Construction never raises. A missing `RHESIS_API_KEY`, a rejected configuration, or an explicit `enabled=False` all yield an inert instance whose `turn()` still works, so an application runs untraced rather than failing to start and callers need no branching.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a new
rhesis-haystackintegration that connects Haystack pipelines to Rhesis for OpenTelemetry-based tracing and observability.RhesisConnectorcomponent: enable tracing by adding it to a pipeline (no wiring to other components required); returnsname,trace_url, andtrace_id.RhesisTracer/DefaultSpanHandler: bridge Haystack spans to OpenTelemetry, exporting pipeline, component, and agent spans to Rhesis.ai.operation.type, content/token attributes, invocation-context propagation).pyproject.toml,pydocconfig,py.typed, examples, README, and unit tests..github/workflows/rhesis.yml), labeler rule, coverage-comment trigger, and README integrations table entry.Test plan
hatch run fmt-checkpasseshatch run test:typespasses (mypy, 5 source files)hatch run test:unitpasses (31 passed)RHESIS_API_KEY; run in CI)Made with Cursor