Skip to content

Support DD_APM_TRACING_ENABLED=false for AI Guard traces#11885

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 12 commits into
masterfrom
clara.poncet/ai-guard-standalone
Jul 22, 2026
Merged

Support DD_APM_TRACING_ENABLED=false for AI Guard traces#11885
gh-worker-dd-mergequeue-cf854d[bot] merged 12 commits into
masterfrom
clara.poncet/ai-guard-standalone

Conversation

@claponcet

@claponcet claponcet commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

What Does This Do

When DD_APM_TRACING_ENABLED=false, AI Guard traces are now preserved and sent
to the backend. The tracer no longer lets the sampler override the USER_KEEP
priority that AI Guard sets via ai_guard.keep.

Concretely, TraceCollector.setSamplingPriorityIfNecessary gains a guard: if
the root span's propagated trace source (_dd.p.ts) is marked for AI_GUARD
(new ProductTraceSource.AI_GUARD bit), the sampler is skipped entirely
(whether that sampler is the regular priority sampler or the
AsmStandaloneSampler rate-limiter used in ASM standalone mode) — mirroring
the existing bypass already in place for ASM.

AIGuardInternal now sets Tags.PROPAGATED_TRACE_SOURCE (_dd.p.ts) to
ProductTraceSource.AI_GUARD on the local root span alongside ai_guard.keep,
so the trace source itself signals "do not let the sampler touch this."

The ai_guard.event tag is also promoted from a local string constant in
AIGuardInternal to a canonical constant in Tags (Tags.AI_GUARD_EVENT),
alongside ai_guard.keep. This tag is purely descriptive (marks a span as
originating from an AI Guard evaluation) and plays no role in the
sampler-bypass logic.

Root spans for AI Guard evaluations carry:

  • _sampling_priority_v1:2 (USER_KEEP) — set by forceKeep(SamplingMechanism.AI_GUARD)
  • _dd.p.dm:-13 (decision maker: AI Guard) — set by the same mechanism
  • _dd.p.ts marked with the AI_GUARD bit — set via Tags.PROPAGATED_TRACE_SOURCE, causes the sampler to be skipped
  • ai_guard.event:true

Motivation

Part of APPSEC-61460 (AI Guard without APM billing). AI Guard customers should
not require APM host billing. Unlike ASM standalone mode, AI Guard does not need
the "1 trace per minute" allowance — every AI Guard trace must go through.

Additional Notes

The AsmStandaloneSampler (used when APM is disabled and AppSec is enabled)
has a rate limiter that allows only 1 trace/minute for regular APM traces. The
new bypass ensures AI Guard traces are never dropped by this limiter, while
regular traces remain subject to it. The bypass is keyed off the propagated
trace source (_dd.p.ts) rather than the ai_guard.event tag, so it composes
with the existing ASM bypass via ProductTraceSource.isProductMarked(..., ASM, AI_GUARD).

Contributor Checklist

  • Format the title according to the contribution guidelines
  • Assign the type: and (comp: or inst:) labels in addition to any other useful labels
  • Avoid using close, fix, or any linking keywords when referencing an issue
    Use solves instead, and assign the PR milestone to the issue
  • Update the CODEOWNERS file on source file addition, migration, or deletion
  • Update public documentation with any new configuration flags or behaviors
  • Add your completed PR to the merge queue by commenting /merge. You can also:
    • Customize the commit message associated with the merge with /merge --commit-message "..."
    • Remove your PR from the merge queue with /merge -c
    • Skip all merge queue checks with /merge -f --reason "reason"; please use this judiciously, as some checks do not run at the PR-level (note: the PR still needs to be mergeable, this will only skip the pre-merge build)
    • Get more information in this doc

Jira ticket: APPSEC-68487

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@claponcet claponcet added type: feature Enhancements and improvements comp: ai-guard AI Guard labels Jul 8, 2026
@claponcet

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 42d77deb22

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@claponcet
claponcet marked this pull request as ready for review July 8, 2026 12:41
@claponcet
claponcet requested review from a team as code owners July 8, 2026 12:41
@claponcet
claponcet requested review from dromanol, manuel-alvarez-alvarez and mcculls and removed request for a team July 8, 2026 12:41
@dd-octo-sts dd-octo-sts Bot added the tag: ai generated Largely based on code generated by an AI or LLM label Jul 8, 2026
@datadog-prod-us1-5

This comment has been minimized.

Comment thread dd-trace-core/src/main/java/datadog/trace/core/TraceCollector.java Outdated
@dd-octo-sts

dd-octo-sts Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 13.92 s 14.01 s [-1.4%; +0.1%] (no difference)
startup:insecure-bank:tracing:Agent 12.88 s 12.97 s [-1.4%; +0.0%] (no difference)
startup:petclinic:appsec:Agent 16.94 s 16.61 s [+1.0%; +2.9%] (significantly worse)
startup:petclinic:iast:Agent 16.94 s 16.98 s [-1.1%; +0.6%] (no difference)
startup:petclinic:profiling:Agent 16.81 s 16.88 s [-1.6%; +0.8%] (no difference)
startup:petclinic:sca:Agent 16.95 s 16.83 s [-0.2%; +1.6%] (no difference)
startup:petclinic:tracing:Agent 16.11 s 16.19 s [-1.5%; +0.4%] (no difference)

Commit: 5c40eea3 · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

@claponcet
claponcet requested a review from smola July 10, 2026 14:44
Comment thread internal-api/src/main/java/datadog/trace/api/ProductTraceSource.java Outdated

@jandro996 jandro996 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One minor non-blocking nit inline about varargs allocation in isProductMarked.
No other issues found

@claponcet
claponcet enabled auto-merge July 22, 2026 09:39
@claponcet
claponcet added this pull request to the merge queue Jul 22, 2026
@dd-octo-sts

dd-octo-sts Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

/merge

@gh-worker-devflow-routing-ef8351

gh-worker-devflow-routing-ef8351 Bot commented Jul 22, 2026

Copy link
Copy Markdown

View all feedbacks in Devflow UI.

2026-07-22 09:56:32 UTC ℹ️ Start processing command /merge


2026-07-22 09:56:37 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 1h (p90).


2026-07-22 10:54:57 UTC ℹ️ MergeQueue: This merge request was merged

@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jul 22, 2026
@gh-worker-dd-mergequeue-cf854d
gh-worker-dd-mergequeue-cf854d Bot merged commit 1358e8e into master Jul 22, 2026
587 checks passed
@gh-worker-dd-mergequeue-cf854d
gh-worker-dd-mergequeue-cf854d Bot deleted the clara.poncet/ai-guard-standalone branch July 22, 2026 10:54
@github-actions github-actions Bot added this to the 1.65.0 milestone Jul 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: ai-guard AI Guard tag: ai generated Largely based on code generated by an AI or LLM type: feature Enhancements and improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants