feat(rust): axum weblog by MilanGarnier · Pull Request #7288 · DataDog/system-tests

MilanGarnier · 2026-07-10T09:33:18Z

Motivation

Rust is lacking a weblog to run end to end tests. The goal is to provide a canonical weblog based on axum and community otel integrations.

Changes

Workflow

⚠️ Create your PR as draft ⚠️
Work on you PR until the CI passes
Mark it as ready for review
- Test logic is modified? -> Get a review from RFC owner.
- Framework is modified, or non obvious usage of it -> get a review from R&P team

🚀 Once your PR is reviewed and the CI green, you can merge it!

🛟 #apm-shared-testing 🛟

Reviewer checklist

Anything but tests/ or manifests/ is modified ? I have the approval from R&P team
A docker base image is modified?
- the relevant build-XXX-image label is present
A scenario is added, removed or renamed?
- Get a review from R&P team

fix: error at building weblog)

datadog-official · 2026-07-10T09:33:37Z

Tests

✨ Fix all issues with BitsAI

⚠️ Warnings

🚦 3 Pipeline jobs failed

Testing the test | Test the test

DataDog/system-tests | java7-app.arm64.DOA9: [public.ecr.aws/lts/ubuntu:22.04, linux/arm64, ]

Testing the test | all-jobs-are-green

ℹ️ Info

No other issues found (see more)

🧪 All tests passed
❄️ No new flaky tests detected

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 69344c6 | Docs | Datadog PR Page | Give us feedback!}

github-actions · 2026-07-10T09:33:54Z

CODEOWNERS have been resolved as:

utils/build/docker/rust/axum.Dockerfile                                 @DataDog/apm-rust @DataDog/system-tests-core
utils/build/docker/rust/axum/.gitignore                                 @DataDog/apm-rust @DataDog/system-tests-core
utils/build/docker/rust/axum/Cargo.lock                                 @DataDog/apm-rust @DataDog/system-tests-core
utils/build/docker/rust/axum/Cargo.toml                                 @DataDog/apm-rust @DataDog/system-tests-core
utils/build/docker/rust/axum/app.sh                                     @DataDog/apm-rust @DataDog/system-tests-core
utils/build/docker/rust/axum/src/integration.rs                         @DataDog/apm-rust @DataDog/system-tests-core
utils/build/docker/rust/axum/src/main.rs                                @DataDog/apm-rust @DataDog/system-tests-core
utils/build/docker/rust/axum/system_tests_library_version.sh            @DataDog/apm-rust @DataDog/system-tests-core
manifests/rust.yml                                                      @DataDog/apm-rust
mirror_images.lock.yaml                                                 @DataDog/system-tests-core
mirror_images.yaml                                                      @DataDog/system-tests-core
tests/schemas/utils/library/v0.7/config-request.json                    @DataDog/system-tests-core
tests/test_telemetry.py                                                 @DataDog/libdatadog-telemetry @DataDog/apm-sdk-capabilities @DataDog/system-tests-core
utils/build/docker/rust/install_ddtrace.sh                              @DataDog/apm-rust @DataDog/system-tests-core
utils/build/docker/rust/parametric/Cargo.lock                           @DataDog/apm-rust @DataDog/system-tests-core

MilanGarnier · 2026-07-15T12:22:38Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dc1864abb4

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector · 2026-07-15T12:36:39Z

+# docs/understand/weblogs/weblog-metadata.md
+
+axum:
+  categories: [dd_trace]


Remove the unsupported weblog metadata key

When the rust end-to-end workflow loads weblogs, WeblogMetaData._load_explicit_metadata() passes each YAML value directly into WeblogMetaData(name=..., library=..., **kwargs), but that dataclass only accepts build_mode, framework_versions, and artifact_name. This new categories key therefore raises TypeError: unexpected keyword argument 'categories' before the axum weblog can be returned, blocking rust CI/workflow parameter computation; omit this file or use only supported metadata fields.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-07-15T12:36:39Z

+        .route("/sample_rate_route/{i}", get(sample_rate_route))
+        .route("/flush", get(flush))
+        .route("/createextraservice", get(create_extra_service))
+        .route("/e2e_otel_span", get(e2e_otel_span))


Skip or implement the mixed-contrib OTel endpoint

In APM_TRACING_E2E_OTEL, Test_Otel_Span::setup_distributed_otel_trace requests /e2e_otel_span/mixed_contrib; other non-Go manifests mark that test irrelevant because it is Go-contrib-specific. The Rust manifest does not skip it, and this router only registers the exact /e2e_otel_span path, so the new Axum weblog will return 404 and the scenario will fail as soon as it runs that test. Add the route or mark test_distributed_otel_trace unsupported for Rust.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-07-15T12:36:39Z

+    cargo remove datadog-opentelemetry >/dev/null 2>&1 || true
+    if ! cargo add datadog-opentelemetry --features metrics-http,metrics-grpc,logs-http,logs-grpc >/dev/null 2>&1; then


Preserve Axum's pinned tracer dependency

For the normal Axum build with no /binaries/dd-trace-rs, this branch removes the datadog-opentelemetry git revision that utils/build/docker/rust/axum/Cargo.toml says is required to keep all OTel crates on 0.32, then resolves crates.io instead. With the published 0.4.0 still on OTel 0.31, align_opentelemetry pulls direct deps back to 0.31 while opentelemetry-instrumentation-tower/tracing-opentelemetry remain 0.32, so the new check_single_opentelemetry_version rejects the graph or server spans become no-ops. Keep the existing git dependency for Axum unless a local checkout is supplied.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-07-15T12:36:39Z

+    let mut parent = tracer
+        .span_builder(parent_name.to_owned())
+        .with_kind(SpanKind::Internal)


Set the expected OTel parent span ID

When APM_TRACING_E2E_OTEL runs Test_Otel_Span::test_datadog_otel_span for Rust, it asserts the root span named by parentName has span ID 10000 for every language except JVM/PHP; the Go weblog explicitly starts that span with WithSpanID(10000). This implementation starts a normal OTel span here, so Rust will emit a random span ID and fail the enabled test unless the ID is forced or the test is marked unsupported for Rust.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-07-15T12:36:39Z

+            if is_sensitive_key(key) {
+                // Replace the entire key=value pair with <redacted>
+                parts.push("<redacted>".to_owned());
+            } else {
+                parts.push(pair.to_owned());


Redact sensitive values inside JSON query strings

In the default scenario, tests/test_scrubbing.py::Test_UrlQuery is not disabled for Rust and sends ?json={"sign":"leak-url-main-v4"} before asserting no leak-url-main appears in tracer payloads. This scrubber only checks the query parameter name (json), so that value is copied into the http.url attribute unchanged; either implement the same JSON/sign obfuscation the test expects or mark the scrubbing tests missing for Rust.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-07-15T12:36:39Z

+COPY --from=builder /usr/app/weblog /app/weblog
+COPY --from=builder /usr/app/SYSTEM_TESTS_LIBRARY_VERSION /app/SYSTEM_TESTS_LIBRARY_VERSION
+COPY utils/build/docker/rust/axum/app.sh /app/app.sh
+RUN chmod +x /app/app.sh


Keep tracer log level out of the Docker image

Setting DD_LOG_LEVEL=debug in the Axum Dockerfile enables debug tracer logging for every Rust end-to-end scenario, not just a targeted debugging run, which can inflate CI logs and change telemetry/config assertions. The repo's PR review guidance says tracer/agent behavior env vars belong in scenario weblog_env/agent_env, so remove this or set it only on scenarios that need debug logging.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-07-15T12:36:39Z

+        .route("/flush", get(flush))
+        .route("/createextraservice", get(create_extra_service))
+        .route("/e2e_otel_span", get(e2e_otel_span))
+        .route("/e2e_single_span", get(e2e_single_span))


Disable single-span E2E until Rust supports SSS

Registering /e2e_single_span makes the new Axum weblog participate in APM_TRACING_E2E_SINGLE_SPAN, but the Rust manifest still marks the equivalent root/child single-span sampling parametric cases as missing_feature. The end-to-end test asserts those spans carry single-span sampling metrics and that only the selected span reaches the backend, so this scenario will fail for Rust unless SSS support is actually added or tests/apm_tracing_e2e/test_single_span.py::Test_SingleSpan is marked unsupported.

Useful? React with 👍 / 👎.

Client-IP resolution (forwarded-header priority + public-IP filtering) and referer-hostname extraction are tracer/appsec responsibilities. The weblog hand-rolled them, so Test_StandardTagsClientIp and Test_StandardTagsReferrerHostname were validating the fixture rather than dd-trace-rs. Remove the logic and mark those tests missing_feature.

Query-string obfuscation of http.url is a tracer responsibility. The weblog's hand-rolled scrubber only served fixture-tested scrubbing cases (all already missing_feature for Rust), so drop it entirely and report the URL verbatim — the weblog now does near-zero tracer-side work. The outgoing-call span still rebuilds its URL from host_str(), so userinfo is dropped and Test_UrlField is unaffected.

MilanGarnier added 9 commits July 7, 2026 11:16

poc axum-weblog (index, healthcheck & make_distant_call)

4df74d8

update mirror images

6900093

use existing middleware for reqwest and axum

3fbe4aa

fix opentelemetry having simultaneaous different versions. (temporary

e08425d

fix: error at building weblog)

rust manifest edits

51dae86

make_extract_call matches spec (passing on test extract behavior)

e98854b

Restructure code

5e064a2

skip some missing endpoints

d4a926b

enrich span fix

c6b1fef

MilanGarnier changed the title ~~Milan.garnier/axum weblog~~ feat(rust): axum weblog Jul 10, 2026

MilanGarnier added 7 commits July 10, 2026 13:50

implement /status endpoint

cc693f4

/status endpoint

aad1265

more simple endpoints

fbcb8d5

add endpoints and enable telemetry tests

e439cc2

disable missing tests for current weblog state

84f8454

test: declare Rust Axum weblog metadata

5a29dea

fix: clarify Rust dependency resolution failures

dc1864a

chatgpt-codex-connector Bot reviewed Jul 15, 2026

View reviewed changes

MilanGarnier added 3 commits July 15, 2026 15:18

address codex review comments

fa2382a

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(rust): axum weblog#7288

feat(rust): axum weblog#7288
MilanGarnier wants to merge 19 commits into
mainfrom
milan.garnier/axum-weblog

MilanGarnier commented Jul 10, 2026

Uh oh!

datadog-official Bot commented Jul 10, 2026 •

edited by datadog-datadog-prod-us1-2 Bot

Loading

Uh oh!

github-actions Bot commented Jul 10, 2026 •

edited

Loading

Uh oh!

MilanGarnier commented Jul 15, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

chatgpt-codex-connector Bot Jul 15, 2026

Uh oh!

chatgpt-codex-connector Bot Jul 15, 2026

Uh oh!

chatgpt-codex-connector Bot Jul 15, 2026

Uh oh!

chatgpt-codex-connector Bot Jul 15, 2026

Uh oh!

chatgpt-codex-connector Bot Jul 15, 2026

Uh oh!

chatgpt-codex-connector Bot Jul 15, 2026

Uh oh!

chatgpt-codex-connector Bot Jul 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

		cargo remove datadog-opentelemetry >/dev/null 2>&1 \|\| true
		if ! cargo add datadog-opentelemetry --features metrics-http,metrics-grpc,logs-http,logs-grpc >/dev/null 2>&1; then

Uh oh!

Conversation

MilanGarnier commented Jul 10, 2026

Motivation

Changes

Workflow

Reviewer checklist

Uh oh!

datadog-official Bot commented Jul 10, 2026 • edited by datadog-datadog-prod-us1-2 Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ Warnings

ℹ️ Info

Uh oh!

github-actions Bot commented Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MilanGarnier commented Jul 15, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

datadog-official Bot commented Jul 10, 2026 •

edited by datadog-datadog-prod-us1-2 Bot

Loading

github-actions Bot commented Jul 10, 2026 •

edited

Loading