Skip to content

Export every trace payload in a tail batch - #227

Open
ayaangazali wants to merge 1 commit into
pydantic:mainfrom
ayaangazali:tail-export-all-trace-entries
Open

Export every trace payload in a tail batch#227
ayaangazali wants to merge 1 commit into
pydantic:mainfrom
ayaangazali:tail-export-all-trace-entries

Conversation

@ayaangazali

Copy link
Copy Markdown
Contributor

What is wrong

exportTailEventsToLogfire stops at the first payload it finds:

function findTraceEntry(events: TraceItem[]): Record<string, unknown> | null {
  for (const event of events) {
    for (const log of event.logs) {
      if (Array.isArray(log.message)) {
        for (const entry of log.message) {
          if (isTraceEntry(entry)) {
            return entry
          }
        }
      }
    }
  }
  return null
}

Its input is a batch. A tail worker is handed one TraceItem per producing invocation, and TailWorkerExporter emits a payload per flush:

const exportMessage = this.cleanNullValues(response)
console.log(exportMessage)

So a batch normally carries several payloads, and every one after the first is discarded. Nothing warns and nothing fails: the export returns a 200 for the payload it did send, so the loss is invisible from both ends.

Evidence

Three payloads across two batched invocations, one of which flushed twice, on 79bcac0:

const events = [
  { logs: [{ message: [{ resourceSpans: ['first'] }] }, { message: [{ resourceSpans: ['second'] }] }] },
  { logs: [{ message: [{ resourceSpans: ['third'] }] }] },
]

posts {"resourceSpans":["first"]}. Two of the three traces never leave the worker. The proportion lost scales with how busy the producing worker is, so it is worst exactly when the traces matter most.

What this does

Collects every entry and merges their resourceSpans into one request. OTLP already models resourceSpans as a repeated field, so this stays a single POST and the Promise<Response | undefined> return is unchanged, rather than becoming N requests with N results to reconcile.

Behaviour for a batch with one payload is byte-identical, which the existing User-Agent test pins. Verified by restoring the early return, which fails the new test and no others.


Built this with Claude Code's help and reviewed the diff myself.

Copilot AI lite review requested due to automatic review settings August 12, 2026 19:05

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@coderabbitai

coderabbitai Bot commented Aug 12, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: c83757d9-e569-49ba-a67c-925a5b6af613

📥 Commits

Reviewing files that changed from the base of the PR and between 79bcac0 and 29636cd.

📒 Files selected for processing (3)
  • .changeset/tail-export-all-trace-entries.md
  • packages/logfire-cf-workers/src/exportTailEventsToLogfire.ts
  • packages/logfire-cf-workers/src/index.test.ts
🔗 Linked repositories identified

CodeRabbit considers these linked repositories for cross-repo context during reviews:

  • pydantic/logfire (manual)
  • pydantic/pydantic-ai (manual)

📝 Walkthrough

Walkthrough

The Cloudflare Workers exporter now collects all matching trace entries from a tail-event batch. It merges their resourceSpans arrays into one OTLP request. It skips transmission when no trace entries exist. A test verifies aggregation across multiple events and one fetch call. A patch changeset documents the release.

Possibly related PRs

  • pydantic/logfire-js#224: Updates the @pydantic/logfire-cf-workers release and changelog for the same tail-event export fix.
🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Description check ✅ Passed The description clearly explains the lost trace payloads and the aggregation change.
Title check ✅ Passed The title clearly and concisely states that every trace payload in a tail batch will be exported.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

findTraceEntry returned on the first entry carrying resourceSpans, so
exportTailEventsToLogfire forwarded one payload per tail invocation. A
tail worker is handed a batch with one TraceItem per producing
invocation, and TailWorkerExporter console.logs a payload per flush, so
everything after the first was dropped silently.

Collect every entry and merge their resourceSpans into one request,
which OTLP already models as a repeated field, so the single Response
return stays as it was.
@ayaangazali

ayaangazali commented Aug 19, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto 8effa79 and green there.

Worth separating this from the tail-sampling work in #242, since the names collide: this one is Cloudflare tail-worker event export in logfire-cf-workers, and it does not touch TailSamplingProcessor at all. Nothing in #242 changes what this PR does.

Still reproducing on current main. Happy to close it if the CF tail-worker path is not somewhere you want changes right now.

@ayaangazali
ayaangazali force-pushed the tail-export-all-trace-entries branch from 29636cd to a00a518 Compare August 19, 2026 20:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants