Repository navigation
feat(observability): export Langfuse v4 traces over OTLP - #467
Conversation
Add an OpenTelemetry Protocol exporter for trace spans so traces can be shipped to an OTLP collector. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
Adds an OTLP ingestion path for trace export so spans can be received over the OpenTelemetry protocol instead of only being written out. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
Add an OTLP ingestion path for trace export alongside the existing Langfuse integration, with tests covering the new ingestion behaviour. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
Add a langfuse observability module to the harness so traces can be exported to a langfuse endpoint. The module includes tests covering the new tracing behaviour. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
Adds a langfuse exporter so graph runs can report traces to a langfuse endpoint, with tests covering the new module. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
Adds a test asserting that the OTLP export produces a v4 trace with the expected trace-level attributes on every span, a single root span, and correct parent-child nesting for steps and nodes. Also verifies error level and node metadata on a failing node and the event type on the run.failed span. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
Langfuse v4 ingests traces only over OTLP, and Langfuse Cloud removes the legacy ingestion trace path on 2026-11-16, so the harness now converts a locally built ingestion batch into v4 OTLP spans and sends them with the v4 ingestion version header. Trace attributes ride on every span, trace input/output move to the root observation, and a 2xx answer with rejected spans is reported as an error. The batch-path methods stay for legacy deployments. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Note Currently processing new changes in this PR. This may take a few minutes, please wait... ⚙️ Run configuration
📒 Files selected for processing: 12
Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 71496c9394
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| .get("id") | ||
| .and_then(Value::as_str) | ||
| .ok_or_else(|| TinyAgentsError::Validation("trace-create has no id".into()))?; | ||
| let trace_hex = hex_id(trace_id, 16); |
There was a problem hiding this comment.
When a trace exported through the new OTLP path is later graded through create_score, this unconditionally changes the configured trace ID into the SHA-derived OTLP traceId (and observation_span does the same for observation IDs), while build_score_batch still writes score.trace_id and score.observation_id unchanged in langfuse/mod.rs:283-285. Thus the documented LangfuseScore::numeric("trace-1", ...) flow points at a nonexistent trace after an OTLP export, and observation-scoped scores are orphaned as well; expose/return this ID transformation and apply it to score targets, or otherwise preserve the same externally visible target IDs.
Useful? React with 👍 / 👎.
| let payload = self.build_ingestion_batch(trace, observations)?; | ||
| self.send_batch_otlp(&payload, service).await |
There was a problem hiding this comment.
When callers follow langfuse/README.md:9-13 and export an accumulated journal on a timer (or retry an export), this method rebuilds and sends every previously seen span again. Langfuse v4 duplicates an already-received span ID instead of merging it (trace_export/README.md:61-62), so repeated calls silently duplicate generations and their token/cost totals; even passing only newly read events re-synthesizes the deterministic run span on every call. Track which finalized spans were sent, or restrict this API to a documented one-shot terminal batch rather than recommending it for periodic export.
AGENTS.md reference: AGENTS.md:L86-L90
Useful? React with 👍 / 👎.
| //! [`ExportBrand`] so product identity strings stay out of this crate. | ||
|
|
||
| pub mod ingestion_batch; | ||
| pub mod ingestion_otlp; |
There was a problem hiding this comment.
For downstream users, this new public module exposes ingestion_batch_to_otlp_spans and ingestion_batch_otlp_requests only through tinyagents_harness::observability::trace_export, and the new chunk_otlp_spans helper is likewise absent from src/lib.rs. Re-export the intended public helpers from the package root (or keep them private) so this addition follows the repository's centralized public-surface convention.
AGENTS.md reference: AGENTS.md:L68-L70
Useful? React with 👍 / 👎.
Tiny Sweeper reviewTiny Sweeper reviewed this change across 5 lane(s) and found 0 active actionable finding(s). Detailed lane evidence and any incomplete work are listed below. State: Incomplete Review snapshot
Completeness: Incomplete FeaturesNone identified with supported citations. TestsNo supported feature-to-test mapping was produced. Test execution is not inferred.
FindingsNo active actionable findings. Could not review: crates/tinyagents-graph/src/observability/README.md, crates/tinyagents-graph/src/observability/langfuse.rs, crates/tinyagents-graph/src/observability/langfuse/langfuse_tests.rs, crates/tinyagents-harness/src/observability/langfuse/mod.rs, crates/tinyagents-harness/src/observability/trace_export/README.md, crates/tinyagents-harness/src/observability/trace_export/ingestion_otlp.rs, crates/tinyagents-harness/src/observability/trace_export/ingestion_otlp_tests.rs, crates/tinyagents-harness/src/observability/trace_export/mod.rs, crates/tinyagents-harness/src/observability/trace_export/otlp.rs, tinysweeper/tests Before merge
How this fits togetherflowchart LR
n0["...s_root_identity_without_an_explicit_trace<br/>changed"]:::changed
n1["...ce_endpoint_beside_the_ingestion_endpoint<br/>changed"]:::changed
n2["reports_a_refused_otlp_request<br/>changed"]:::changed
n3["...son_to_the_trace_endpoint_with_basic_auth<br/>changed"]:::changed
n4["unwrap"]:::impacted
n5["Value"]:::impacted
n6["build_graph_langfuse_batch"]:::impacted
n7["...ata_injector_merges_and_wins_on_collision"]:::impacted
n0 -->|calls| n4
n0 -->|tests| n4
n0 -->|calls| n6
n0 -->|tests| n6
n1 -->|calls| n4
n1 -->|tests| n4
n2 -->|calls| n4
n2 -->|tests| n4
n3 -->|calls| n4
n3 -->|tests| n4
n4 -->|uses| n5
n6 -->|uses| n5
n7 -->|calls| n4
n7 -->|tests| n4
classDef changed fill:#0d4429,stroke:#238636,color:#e6edf3
classDef impacted fill:#161b22,stroke:#6e7681,color:#c9d1d9
classDef flagged fill:#5a1e02,stroke:#d93f0b,color:#ffffff
classDef blocking fill:#67060c,stroke:#f85149,color:#ffffff
Agent review detailscritique
security
tests
commits
description
Evidence and run details
|
![tinysweeper[bot]](https://avatars.githubusercontent.com/in/4526585?s=60&v=4){kind=small}
There was a problem hiding this comment.
tinysweeper found nothing blocking, but could not review everything, so this is not an approval: crates/tinyagents-graph/src/observability/README.md, crates/tinyagents-graph/src/observability/langfuse.rs, crates/tinyagents-graph/src/observability/langfuse/langfuse_tests.rs, crates/tinyagents-harness/src/observability/langfuse/mod.rs, crates/tinyagents-harness/src/observability/trace_export/README.md, crates/tinyagents-harness/src/observability/trace_export/ingestion_otlp.rs, crates/tinyagents-harness/src/observability/trace_export/ingestion_otlp_tests.rs, crates/tinyagents-harness/src/observability/trace_export/mod.rs and 2 more.
$0.0042 · 290,311 in / 15,922 out · 0 cached (0%) · ladder/vectors-oai3, glm-5.3-flash, · 793 embedded
critique: $0.0034 · 213,914 in / 14,900 out · 0 cached (0%) · glm-5.3-flash,
security: $0.0005 · 55,251 in / 781 out · 0 cached (0%) · glm-5.3-flash,
description: $0.0002 · 21,146 in / 241 out · 0 cached (0%) · glm-5.3-flash
tinysweeper
Bot
added
the
priority: p3
1 participant
Summary
Langfuse v4 (changelog, ingestion migration) ingests traces only over OTLP, and Langfuse Cloud removes the legacy
/api/public/ingestiontrace path on 2026-11-16. Agent turns already export over OTLP. The graph exporter, and the harnesssend_observations, still sent traces only as legacy batches. This PR adds a v4 OTLP path for both and hardenssend_otlp.LangfuseClient::send_otlpnow sendsx-langfuse-ingestion-version: 4(LANGFUSE_INGESTION_VERSION_HEADER), which selects v4's observations-first write path. A 2xx answer whose OTLPpartialSuccess.rejectedSpansis non-zero is now an error; before, it counted as a success.trace_export::ingestion_otlp(pure):ingestion_batch_to_otlp_spans/ingestion_batch_otlp_requestsconvert a locally built ingestion batch into v4 OTLP spans, following the migration guide:trace.metadata.*, environment, release and version go on every span. v4 filters observations directly, so a value set only on the root is invisible on a child.langfuse.observation.input/output). Nothing writeslangfuse.trace.input.unfinished. Scores and other non-trace events are skipped and keep using the scores API.LangfuseClient::send_observations_otlp/send_batch_otlpandGraphLangfuseExporter::send_observations_otlp/build_otlp_requestsare the v4 export entry points. The batch methods stay, with docs pointing to the OTLP ones.trace_export::otlp::chunk_otlp_spansexposes the existing 20-span / ~8 MB request chunking so both converters share it.Docs: the READMEs for
trace_export,langfuseand graphobservability, plus a CHANGELOG entry.Tests
trace_export::ingestion_otlptests: trace attributes on every span, root-only trace IO, parentage/level/generation mapping, deterministic ids, open-span handling, request wrapping, rejection of a batch with no trace.langfusetests: v4 header on direct and proxy clients (failed before the fix), partial rejection is an error (failed before the fix), emptypartialSuccessis a success,send_observations_otlpend to end against a loopback server.otlp_export_is_a_v4_trace_with_trace_attributes_on_every_span.cargo test -p tinyagents-graph: 508 passed.cargo test -p tinyagents-harness: 2784 passed. The 3providers::claude_code::driverretry tests fail locally withNotFoundon a fixture binary; that is environmental and unrelated to this change. Clippy is clean.The OpenHuman host change (flow-run export moves to OTLP) follows in a separate PR that bumps the gitlink.
Summary by CodeRabbit