Add code coverage report flags by juan-fernandez · Pull Request #11981 · DataDog/dd-trace-java

juan-fernandez · 2026-07-17T09:24:47Z

What Does This Do

Adds support for DD_CODE_COVERAGE_FLAGS to Test Optimization coverage-report uploads. Values are parsed as comma-separated flags, trimmed, and included in the multipart event as the JSON string array report.flags.

The configuration preserves flag order and duplicates, removes empty entries, and applies the same immutable snapshot to every report uploaded by the process. More than 32 flags logs a warning, omits report.flags, and continues the upload.

Registers the configuration metadata and migrates CoverageReportUploaderTest from Spock/Groovy to JUnit 5 Java.

Motivation

Coverage reports uploaded by the Java tracer could not use the same grouping flags supported by datadog-ci coverage upload --flags. This makes the Java uploader compatible with that payload shape while preserving fail-open tracer behavior.

Additional Notes

Validation:

./gradlew :internal-api:test --tests datadog.trace.api.ConfigCodeCoverageFlagsTest
./gradlew :dd-java-agent:agent-ci-visibility:test --tests datadog.trace.civisibility.coverage.report.CoverageReportUploaderTest
./gradlew :internal-api:test --tests datadog.trace.api.telemetry.ConfigInversionMetricCollectorImplTest
./gradlew :dd-trace-api:spotlessCheck :internal-api:spotlessCheck :dd-java-agent:agent-ci-visibility:spotlessCheck
./gradlew :internal-api:compileJava

Contributor Checklist

Format the title according to the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any other useful labels
Avoid using close, fix, or any linking keywords when referencing an issue
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, migration, or deletion
Update public documentation with any new configuration flags or behaviors
Once approved, use merge queue to merge the PR

Jira ticket: [PROJ-IDENT]

cit-pr-commenter-54b7da · 2026-07-17T09:37:54Z

Test Environment - sbt-scalatest

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	54.84	55.43	$\color{green}{\blacktriangledown}$ -0.59	55.43	$\color{green}{\blacktriangledown}$ -0.59	58/246
agentEvpProxy	54.94	n/a	n/a	n/a	n/a	-

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

cit-pr-commenter-54b7da · 2026-07-17T09:40:45Z

Test Environment - nebula-release-plugin

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	35.45	36.42	$\color{green}{\blacktriangledown}$ -0.97	36.42	$\color{green}{\blacktriangledown}$ -0.97	31/124
agentless	35.19	36.42	$\color{green}{\blacktriangledown}$ -1.23	36.42	$\color{green}{\blacktriangledown}$ -1.23	31/124
agentlessCodeCoverage	45.04	45.38	$\color{green}{\blacktriangledown}$ -0.34	44.48	$\color{red}{\blacktriangle}$ +0.56	31/124
agentlessLineCoverage	72.15	74.82	$\color{green}{\blacktriangledown}$ -2.67	74.82	$\color{green}{\blacktriangledown}$ -2.67	31/124

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

cit-pr-commenter-54b7da · 2026-07-17T09:42:17Z

Test Environment - netflix-zuul

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	88.41	87.80	$\color{red}{\blacktriangle}$ +0.61	87.80	$\color{red}{\blacktriangle}$ +0.61	34/129
agentless	81.33	81.05	$\color{red}{\blacktriangle}$ +0.28	81.05	$\color{red}{\blacktriangle}$ +0.28	31/127
agentlessCodeCoverage	97.05	97.04	$\color{red}{\blacktriangle}$ +0.01	95.12	$\color{red}{\blacktriangle}$ +1.93	31/125
agentlessLineCoverage	112.22	111.62	$\color{red}{\blacktriangle}$ +0.60	111.62	$\color{red}{\blacktriangle}$ +0.60	30/124

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

cit-pr-commenter-54b7da · 2026-07-17T09:42:58Z

Test Environment - pass4s

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	13.34	10.54	$\color{red}{\blacktriangle}$ +2.80	10.33	$\color{red}{\blacktriangle}$ +3.01	25/116
agentless	8.49	9.73	$\color{green}{\blacktriangledown}$ -1.24	10.13	$\color{green}{\blacktriangledown}$ -1.64	25/116
agentlessCodeCoverage	16.39	16.69	$\color{green}{\blacktriangledown}$ -0.30	17.03	$\color{green}{\blacktriangledown}$ -0.64	24/114

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

cit-pr-commenter-54b7da · 2026-07-17T09:43:59Z

Test Environment - reactive-streams-jvm

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	22.50	21.65	$\color{red}{\blacktriangle}$ +0.85	21.65	$\color{red}{\blacktriangle}$ +0.85	39/133
agentless	19.39	18.82	$\color{red}{\blacktriangle}$ +0.57	18.82	$\color{red}{\blacktriangle}$ +0.57	35/128
agentlessCodeCoverage	20.69	19.99	$\color{red}{\blacktriangle}$ +0.70	19.99	$\color{red}{\blacktriangle}$ +0.70	33/125
agentlessLineCoverage	31.98	30.42	$\color{red}{\blacktriangle}$ +1.56	29.82	$\color{red}{\blacktriangle}$ +2.16	33/125

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

cit-pr-commenter-54b7da · 2026-07-17T09:44:28Z

Test Environment - heliboard

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	9.78	9.73	$\color{red}{\blacktriangle}$ +0.05	9.54	$\color{red}{\blacktriangle}$ +0.24	28/59

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

dd-octo-sts · 2026-07-17T09:49:21Z

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite	Status
Startup	🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Scenario	Candidate	master	Δ (95% CI of mean)
startup:insecure-bank:iast:Agent	14.73 s	14.66 s	[-0.3%; +1.3%] (no difference)
startup:insecure-bank:tracing:Agent	13.71 s	13.79 s	[-1.4%; +0.3%] (no difference)
startup:petclinic:appsec:Agent	16.88 s	16.75 s	[-0.5%; +2.0%] (no difference)
startup:petclinic:iast:Agent	16.41 s	16.94 s	[-7.6%; +1.3%] (no difference)
startup:petclinic:profiling:Agent	16.66 s	16.50 s	[-0.0%; +2.0%] (no difference)
startup:petclinic:sca:Agent	16.89 s	16.81 s	[-0.3%; +1.2%] (no difference)
startup:petclinic:tracing:Agent	16.08 s	16.18 s	[-1.5%; +0.3%] (no difference)

Commit: 29fadab7 · CI Pipeline · Benchmarking Platform UI

Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

cit-pr-commenter-54b7da · 2026-07-17T09:49:33Z

Test Environment - sonar-kotlin

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	12.00	12.87	$\color{green}{\blacktriangledown}$ -0.87	12.87	$\color{green}{\blacktriangledown}$ -0.87	31/121
agentless	10.39	11.88	$\color{green}{\blacktriangledown}$ -1.49	12.12	$\color{green}{\blacktriangledown}$ -1.73	28/117
agentlessCodeCoverage	14.31	14.23	$\color{red}{\blacktriangle}$ +0.08	15.11	$\color{green}{\blacktriangledown}$ -0.80	28/117
agentlessLineCoverage	19.59	19.20	$\color{red}{\blacktriangle}$ +0.39	19.20	$\color{red}{\blacktriangle}$ +0.39	29/120

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

datadog-prod-us1-3 · 2026-07-17T09:50:28Z

View session in Datadog

Bits Code status: ✅ Done

Comment @DataDog to request changes

cit-pr-commenter-54b7da · 2026-07-17T09:52:01Z

Test Environment - jolokia

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	94.06	93.23	$\color{red}{\blacktriangle}$ +0.83	93.23	$\color{red}{\blacktriangle}$ +0.83	42/133
agentless	88.82	89.58	$\color{green}{\blacktriangledown}$ -0.76	89.58	$\color{green}{\blacktriangledown}$ -0.76	43/131
agentlessCodeCoverage	98.22	99.00	$\color{green}{\blacktriangledown}$ -0.78	99.00	$\color{green}{\blacktriangledown}$ -0.78	42/128
agentlessLineCoverage	99.64	99.00	$\color{red}{\blacktriangle}$ +0.64	99.00	$\color{red}{\blacktriangle}$ +0.64	40/126

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

cit-pr-commenter-54b7da · 2026-07-17T09:57:58Z

Test Environment - okhttp

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	18.00	19.20	$\color{green}{\blacktriangledown}$ -1.20	19.20	$\color{green}{\blacktriangledown}$ -1.20	28/128
agentless	18.15	18.82	$\color{green}{\blacktriangledown}$ -0.67	18.82	$\color{green}{\blacktriangledown}$ -0.67	28/125
agentlessCodeCoverage	21.69	22.09	$\color{green}{\blacktriangledown}$ -0.40	22.09	$\color{green}{\blacktriangledown}$ -0.40	28/124
agentlessLineCoverage	44.51	44.48	$\color{red}{\blacktriangle}$ +0.03	44.48	$\color{red}{\blacktriangle}$ +0.03	28/130

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

daniel-mohedano

LGTM, left a small nit 🚀

cit-pr-commenter-54b7da · 2026-07-17T10:05:25Z

Test Environment - spring_boot

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	16.12	16.36	$\color{green}{\blacktriangledown}$ -0.24	16.36	$\color{green}{\blacktriangledown}$ -0.24	31/118
agentless	9.73	9.73	$\color{gray}{\bullet}$ 0.00	9.73	$\color{gray}{\bullet}$ 0.00	31/119
agentlessCodeCoverage	13.73	13.40	$\color{red}{\blacktriangle}$ +0.33	13.40	$\color{red}{\blacktriangle}$ +0.33	32/119
agentlessLineCoverage	33.18	32.95	$\color{red}{\blacktriangle}$ +0.23	32.95	$\color{red}{\blacktriangle}$ +0.23	31/119

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

datadog-prod-us1-3

More details

Configuration parsing correctly handles all edge cases: empty entries are filtered by ConfigProvider.parseList before reaching the validation code, max-32-flags limit is enforced with fail-open warning, flags are immutably copied and properly serialized as JSON array in the upload event. Event map type changed from Map<String, String> to Map<String, Object> with Moshi adapter type updated accordingly. Test migration from Groovy to Java is complete with comprehensive edge-case coverage. No behavioral regressions detected.

Was this helpful? React 👍 or 👎

_{📊 Validated against 8 scenarios · Open Bits AI session}

_{🤖 Datadog Autotest · Commit 34c0678 · What is Autotest? · Any feedback? Reach out in #autotest}

cit-pr-commenter-54b7da · 2026-07-17T10:22:44Z

Test Environment - sonar-java

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	14.30	8.13	$\color{red}{\blacktriangle}$ +6.17	13.40	$\color{red}{\blacktriangle}$ +0.90	34/128
agentless	49.61	8.46	$\color{red}{\blacktriangle}$ +41.15	16.04	$\color{red}{\blacktriangle}$ +33.57	34/127
agentlessCodeCoverage	95.46	84.36	$\color{red}{\blacktriangle}$ +11.10	86.07	$\color{red}{\blacktriangle}$ +9.39	34/127
agentlessLineCoverage	183.71	130.99	$\color{red}{\blacktriangle}$ +52.72	136.34	$\color{red}{\blacktriangle}$ +47.37	33/126

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

datadog-prod-us1-3 · 2026-07-17T10:33:57Z

Bits has a CI fix ready

🟢 Investigated · 🟢 Fix prepared · ⚪ Validation skipped · 🟠 Ready

ConfigCodeCoverageFlagsTest imported WithConfigExtension from the old package path. Updated it to use datadog.trace.test.junit.utils.config.WithConfigExtension, matching the relocated test utility.

Commit fix to this PR

View in Datadog | Reviewed commit 508270c

dd-octo-sts · 2026-07-20T10:12:47Z

/merge

gh-worker-devflow-routing-ef8351 · 2026-07-20T10:12:51Z

View all feedbacks in Devflow UI.

2026-07-20 10:12:50 UTC ℹ️ Start processing command /merge

2026-07-20 10:12:55 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 2h (p90).

2026-07-20 10:21:19 UTC ❌ MergeQueue: The build pipeline contains failing jobs for this merge request

Build pipeline has failing jobs for dce6116:

build_tests: [:baseTest, base]

⚠️ Do NOT retry failed jobs directly (why?).

What to do next?

Investigate the failures and when ready, re-add your pull request to the queue!
If your PR checks are green, try to rebase/merge. It might be because the CI run is a bit old.
Any question, go check the FAQ.

Details

Since those jobs are not marked as being allowed to fail, the pipeline will most likely fail.
Therefore, and to allow other builds to be processed, this merge request has been rejected and the pipeline got canceled.

dd-octo-sts · 2026-07-20T13:17:40Z

/merge

gh-worker-devflow-routing-ef8351 · 2026-07-20T13:17:45Z

View all feedbacks in Devflow UI.

2026-07-20 13:17:44 UTC ℹ️ Start processing command /merge

2026-07-20 13:17:50 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 2h (p90).

2026-07-20 14:26:20 UTC ℹ️ MergeQueue: This merge request was merged

Add code coverage report flags

f6fd6c1

juan-fernandez added comp: ci visibility Continuous Integration Visibility type: feature Enhancements and improvements tag: ai generated Largely based on code generated by an AI or LLM labels Jul 17, 2026 — with ChatGPT Codex Connector

This comment has been minimized.

Sign in to view

daniel-mohedano approved these changes Jul 17, 2026

View reviewed changes

Comment thread internal-api/src/main/java/datadog/trace/api/Config.java Outdated

Remove redundant coverage flag trimming

34c0678

juan-fernandez marked this pull request as ready for review July 17, 2026 10:15

juan-fernandez requested review from a team as code owners July 17, 2026 10:15

juan-fernandez requested review from AlexeyKuznetsov-DD and amarziali and removed request for a team July 17, 2026 10:15

datadog-prod-us1-3 Bot reviewed Jul 17, 2026

View reviewed changes

Correct code coverage flags configuration type

2e68457

juan-fernandez added this pull request to the merge queue Jul 20, 2026

github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jul 20, 2026

juan-fernandez and others added 2 commits July 20, 2026 12:21

Merge branch 'master' into juan-fernandez/coverage-report-flags

508270c

Fix coverage flags test after JUnit utilities move

29fadab

juan-fernandez added this pull request to the merge queue Jul 20, 2026

github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jul 20, 2026

gh-worker-dd-mergequeue-cf854d Bot merged commit 554134a into master Jul 20, 2026
586 checks passed

gh-worker-dd-mergequeue-cf854d Bot deleted the juan-fernandez/coverage-report-flags branch July 20, 2026 14:26

github-actions Bot added this to the 1.65.0 milestone Jul 20, 2026

Uh oh!

Conversation

juan-fernandez commented Jul 17, 2026

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

cit-pr-commenter-54b7da Bot commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Environment - sbt-scalatest

Uh oh!

cit-pr-commenter-54b7da Bot commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Environment - nebula-release-plugin

Uh oh!

This comment has been minimized.

cit-pr-commenter-54b7da Bot commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Environment - netflix-zuul

Uh oh!

cit-pr-commenter-54b7da Bot commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Environment - pass4s

Uh oh!

cit-pr-commenter-54b7da Bot commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Environment - reactive-streams-jvm

Uh oh!

cit-pr-commenter-54b7da Bot commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Environment - heliboard

Uh oh!

dd-octo-sts Bot commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🟢 Java Benchmark SLOs — All performance SLOs passed

Uh oh!

cit-pr-commenter-54b7da Bot commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Environment - sonar-kotlin

Uh oh!

datadog-prod-us1-3 Bot commented Jul 17, 2026 • edited by datadog-datadog-prod-us1-2 Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cit-pr-commenter-54b7da Bot commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Environment - jolokia

Uh oh!

cit-pr-commenter-54b7da Bot commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Environment - okhttp

Uh oh!

daniel-mohedano left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cit-pr-commenter-54b7da Bot commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Environment - spring_boot

Uh oh!

datadog-prod-us1-3 Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cit-pr-commenter-54b7da Bot commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Environment - sonar-java

Uh oh!

datadog-prod-us1-3 Bot commented Jul 17, 2026 • edited by datadog-prod-us1-6 Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bits has a CI fix ready

Uh oh!

dd-octo-sts Bot commented Jul 20, 2026

Uh oh!

gh-worker-devflow-routing-ef8351 Bot commented Jul 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What to do next?

Uh oh!

Uh oh!

dd-octo-sts Bot commented Jul 20, 2026

Uh oh!

cit-pr-commenter-54b7da Bot commented Jul 17, 2026 •

edited

Loading

cit-pr-commenter-54b7da Bot commented Jul 17, 2026 •

edited

Loading

cit-pr-commenter-54b7da Bot commented Jul 17, 2026 •

edited

Loading

cit-pr-commenter-54b7da Bot commented Jul 17, 2026 •

edited

Loading

cit-pr-commenter-54b7da Bot commented Jul 17, 2026 •

edited

Loading

cit-pr-commenter-54b7da Bot commented Jul 17, 2026 •

edited

Loading

dd-octo-sts Bot commented Jul 17, 2026 •

edited

Loading

cit-pr-commenter-54b7da Bot commented Jul 17, 2026 •

edited

Loading

datadog-prod-us1-3 Bot commented Jul 17, 2026 •

edited by datadog-datadog-prod-us1-2 Bot

Loading

cit-pr-commenter-54b7da Bot commented Jul 17, 2026 •

edited

Loading

cit-pr-commenter-54b7da Bot commented Jul 17, 2026 •

edited

Loading

cit-pr-commenter-54b7da Bot commented Jul 17, 2026 •

edited

Loading

cit-pr-commenter-54b7da Bot commented Jul 17, 2026 •

edited

Loading

datadog-prod-us1-3 Bot commented Jul 17, 2026 •

edited by datadog-prod-us1-6 Bot

Loading

gh-worker-devflow-routing-ef8351 Bot commented Jul 20, 2026 •

edited

Loading

gh-worker-devflow-routing-ef8351 Bot commented Jul 20, 2026 •

edited

Loading