feat(llm-gateway): accept caller trace id, protect gateway-owned event properties

[brandonleung]

Problem

Callers of the Python LLM gateway have no supported way to correlate their own records with the captured $ai_generation: the OpenAI route derives $ai_trace_id internally. And the property merge runs caller-last with no protection, so any x-posthog-property-$ai_* header silently overwrites gateway-derived fields ($ai_input, $ai_model, token counts, $ai_trace_id).

Changes

• Accept an x-posthog-trace-id request header (stripped, capped at 256 chars) and use it as $ai_trace_id on both success and failure captures. Stored verbatim so a caller's saved id matches the join; the metadata.user_id derivation still applies when the header is absent.
• Make the caller-property merge use setdefault, so a caller can add its own properties (e.g. $ai_span_name, which the gateway never derives) but cannot overwrite a value the gateway already set.
• Carry the trace id through the handle_llm_request context rebuild.
• Log when an over-256 trace-id header is dropped, so a caller whose join comes back empty has a signal. Delete the unused set_posthog_context helper.

First consumer: the surveys gateway helper (#72670), so survey summary ratings can join the generation they rate.

Verification

llm-gateway suite (1342 passed), ruff, format. Route-level coverage on both /v1/chat/completions and /{product}/v1/chat/completions; callback tests pin header-wins, verbatim storage, the unchanged metadata.user_id fallback, and that a caller $ai_trace_id/$ai_model property loses to the derived value while $ai_span_name passes through.

Known gaps

Low: a caller can still set $set/$set_once and, when the gateway computes no cost, $ai_total_cost_usd, since the gateway does not set those before the merge. Both are pre-existing (unchanged from master); tracked separately.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Claude (Opus 4.8) wrote this under my direction. Requires human review.

[trunk-io]

Merging to master in this repository is managed by Trunk.

• To merge this pull request, check the box to the left or comment /trunk merge below.

After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here

[greptile-apps]

_{Reviews (1): Last reviewed commit: "feat(llm-gateway): accept caller trace i..." | Re-trigger Greptile}

[veria-ai]

Medium: Billing bypass via caller-controlled trace ID

A caller can assign a billed generation the ID of a known non-billable trace, such as a docs-search-only trace. The usage report joins generations to billing exemptions using only $ai_trace_id, so the generation is then excluded from credit accounting; either restrict caller-selected IDs for billable products or scope the billing join to trusted product and customer attribution.

[veria-ai]

PR overview

This pull request updates the LLM gateway to accept a caller-provided trace ID while protecting gateway-owned event properties in PostHog callback handling. The changes center on tracing and event attribution behavior for gateway-generated LLM telemetry.

There is one open security issue affecting billing integrity: caller-controlled trace IDs can be reused to associate billable generations with non-billable traces, causing usage to be excluded from credit accounting. No issues have been fixed yet, so the PR still needs a guardrail that prevents untrusted trace IDs from influencing billing exemptions or scopes those exemptions to trusted product and customer attribution.

Open issues (1)

• Medium: Billing bypass via caller-controlled trace ID — services/llm-gateway/src/llm_gateway/callbacks/posthog.py:198

Fixed/addressed: 0 · PR risk: 6/10