chore(config): preserve provenance in typed configuration

[webern]

Human Summary

WIP: not ready for review

Reviewer's Guide

• lib/saluki-config/src/dynamic/event.rs introduces types to carry provenance in the Agent updates.
• lib/agent-data-plane-config/src/provenance.rs introduces types to carry provenance in typed config.
• bin/agent-data-plane/src/internal/remote_agent.rs is where we preserve the information in the config stream.
• lib/agent-data-plane-config-system/src/translators/datadog_translator.rs is where we preserve the information for those fields where we need it in typed config.

Most of the code churn is tests creating config stream events for themselves which now have to add the provenance metadata.

AI Summary

The Core Agent configuration stream publishes a complete reified configuration, including values materialized from schema defaults, and source metadata for each key. Once that configuration is deserialized into a value-only model, an explicitly supplied value equal to the schema default is indistinguishable from a defaulted value. That distinction is required for source-aware translation and is the key to resolving #1965.

This PR carries the two-state provenance (Default/Explicit) from the stream through the typed configuration system and exposes it to SalukiConfiguration consumers with ConfigValue<T>. Translators and consumers can now determine whether a setting was supplied without hardcoding or comparing schema-default values. Provenance is preserved across local bootstrap values, snapshots, partial updates, resets, and live changes while retaining the existing serialized configuration shape.

As a side effect, default-sourced Agent values no longer shadow explicit local file or environment values in the typed path; explicit Agent values still override them. This is useful for local and test configuration, but is secondary to making source-aware configuration decisions possible. The legacy GenericConfiguration compatibility view remains value-only, and affected components still need to migrate before #1965 is fully resolved.

Change Type

• Non-functional (chore, refactoring, docs)

How did you test this PR?

• cargo check --workspace --tests
• cargo test -p agent-data-plane-config -p agent-data-plane-config-system -p saluki-config -p agent-data-plane
• git diff --check

All tests passed.

References

• Progresses Investigate whether dd_url's schema default makes site unreachable in ADP. #1965

[pr-commenter]

Binary Size Analysis (Agent Data Plane)

Baseline: be50909 · Comparison: de7bb0f · diff
Analysis Configuration: stripped binaries · Pass/Fail Threshold: +5%
Sizes: 41.69 MiB (baseline) vs 41.63 MiB (comparison)
Size Change: -66.18 KiB (-0.16%)

✅ Binary size difference within threshold

Changes by Module

Module	File Size	Symbols
agent_data_plane_config_system::saluki_env_overlay::PathRecorder	-55.46 KiB	19
agent_data_plane_config_system::saluki_only::_	+43.63 KiB	8
core	-26.40 KiB	2503
agent_data_plane::dogstatsd_contexts::artifact	+25.36 KiB	7
alloc	+23.65 KiB	571
serde_json	-20.86 KiB	91
figment	-19.23 KiB	141
anon.4d7b2df5eb1b3946a70196cadfabc2c4.518.llvm.1716889922181735544	-17.71 KiB	1
anon.90eaefd0caafd678f7882ded20c08702.603.llvm.7541402005584103405	+17.62 KiB	1
serde_core	+14.64 KiB	77
anon.e11d83d0b5088fc60b7097f54cc60ee9.929.llvm.11962382711851653337	+12.46 KiB	1
anon.4d7b2df5eb1b3946a70196cadfabc2c4.590.llvm.1716889922181735544	-12.37 KiB	1
anyhow	-10.36 KiB	263
agent_data_plane_config::_::_<impl serde_core	-10.12 KiB	1
saluki_core::topology::blueprint	+9.68 KiB	36
anon.90eaefd0caafd678f7882ded20c08702.682.llvm.7541402005584103405	+9.62 KiB	1
anon.4d7b2df5eb1b3946a70196cadfabc2c4.264.llvm.1716889922181735544	-9.53 KiB	1
anon.90eaefd0caafd678f7882ded20c08702.584.llvm.7541402005584103405	+9.06 KiB	1
anon.4d7b2df5eb1b3946a70196cadfabc2c4.515.llvm.1716889922181735544	-8.97 KiB	1
anon.90eaefd0caafd678f7882ded20c08702.10.llvm.7541402005584103405	+8.74 KiB	1

Detailed Symbol Changes

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  [NEW] +35.2Ki  [NEW] +34.9Ki    _<agent_data_plane_config_system::saluki_only::_::<impl serde_core::de::Deserialize for agent_data_plane_config_system::saluki_only::SalukiOnly>::deserialize::__Visitor as serde_core::de::Visitor>::visit_map::h9cec17db56ba97fb
  +669% +24.8Ki  +689% +24.8Ki    agent_data_plane::dogstatsd_contexts::artifact::for_each_record::ha4e06ef9d824f235
  [NEW] +17.6Ki  [NEW]     +76    anon.90eaefd0caafd678f7882ded20c08702.603.llvm.7541402005584103405
  [NEW] +14.4Ki  [NEW] +14.4Ki    datadog_agent_config::generated::witness::drive::h16e0048f54c273e5
  [NEW] +13.2Ki  [NEW] +13.1Ki    _<tokio::runtime::task::trace::Root<T> as core::future::future::Future>::poll::hbdb002c3fb5ddb9e
  [NEW] +12.5Ki  [NEW]     +81    anon.e11d83d0b5088fc60b7097f54cc60ee9.929.llvm.11962382711851653337
  [NEW] +12.3Ki  [NEW] +12.0Ki    _<agent_data_plane_config_system::saluki_only::_::<impl serde_core::de::Deserialize for agent_data_plane_config_system::saluki_only::OtlpConfig>::deserialize::__Visitor as serde_core::de::Visitor>::visit_map::h936abc0aa4583620
  [NEW] +9.62Ki  [NEW]     +79    anon.90eaefd0caafd678f7882ded20c08702.682.llvm.7541402005584103405
  [NEW] +9.06Ki  [NEW]     +91    anon.90eaefd0caafd678f7882ded20c08702.584.llvm.7541402005584103405
  [DEL] -8.97Ki  [DEL]     -91    anon.4d7b2df5eb1b3946a70196cadfabc2c4.515.llvm.1716889922181735544
  [DEL] -9.53Ki  [DEL]     -79    anon.4d7b2df5eb1b3946a70196cadfabc2c4.264.llvm.1716889922181735544
 -93.9% -10.1Ki -95.5% -10.1Ki    agent_data_plane_config::_::_<impl serde_core::ser::Serialize for agent_data_plane_config::SalukiConfiguration>::serialize::ha4890af58c866150
  [DEL] -12.4Ki  [DEL]     -81    anon.4d7b2df5eb1b3946a70196cadfabc2c4.590.llvm.1716889922181735544
  [DEL] -12.4Ki  [DEL] -12.3Ki    agent_data_plane::state::metrics::rules::dogstatsd::get_dogstatsd_remappings::ha71516d8223bc1b6
  [DEL] -12.5Ki  [DEL] -12.3Ki    _<serde_json::de::StreamDeserializer<R,T> as core::iter::traits::iterator::Iterator>::next::h10bc41daec578a9b
  [DEL] -12.6Ki  [DEL] -12.4Ki    _<serde_json::de::StreamDeserializer<R,T> as core::iter::traits::iterator::Iterator>::next::h6054a357390c9fb4
  [DEL] -13.0Ki  [DEL] -12.8Ki    _<tokio::runtime::task::trace::Root<T> as core::future::future::Future>::poll::h79d3bd14fcf0c785
  [DEL] -14.5Ki  [DEL] -14.4Ki    datadog_agent_config::generated::witness::drive::h1f4ddadd82aeab06
  [DEL] -15.6Ki  [DEL] -15.5Ki    _<agent_data_plane_config_system::saluki_env_overlay::PathRecorder as serde_core::de::Deserializer>::deserialize_struct::hb4aae3736c21710c
  [DEL] -17.7Ki  [DEL]     -76    anon.4d7b2df5eb1b3946a70196cadfabc2c4.518.llvm.1716889922181735544
  -0.6% -75.5Ki  -0.7% -77.3Ki    [14924 Others]
  -0.2% -66.2Ki  -0.2% -68.1Ki    TOTAL

[pr-commenter]

Regression Detector (Agent Data Plane)

Run ID: db9b3135-be50-4d8d-b533-d9124092a32e
Baseline: be50909a · Comparison: de7bb0f9 · diff

Optimization Goals: ✅ No significant changes detected

Fine details of change detection per experiment (5)

Experiments configured erratic: true are tagged (ignored) and skipped when determining which experiments regressed or improved. Experiments which are detected as erratic at runtime are tagged (erratic) to flag that the run's sample dispersion was high, but their regression / improvement signal still counts.

experiment	goal	Δ mean %	links
quality_gates_rss_dsd_low	memory	⚪ -0.10	metrics profiles logs
quality_gates_rss_idle	memory	⚪ -0.11	metrics profiles logs
quality_gates_rss_dsd_ultraheavy	memory	⚪ -0.71	metrics profiles logs
quality_gates_rss_dsd_medium	memory	⚪ -0.83	metrics profiles logs
quality_gates_rss_dsd_heavy	memory	⚪ -1.39	metrics profiles logs

Bounds Checks: ✅ Passed (5)

experiment	check	replicates	observed	links
quality_gates_rss_dsd_heavy	memory_usage	10/10	✅ 226 MiB ≤ 250 MiB	metrics profiles logs
quality_gates_rss_dsd_low	memory_usage	10/10	✅ 51.5 MiB ≤ 60 MiB	metrics profiles logs
quality_gates_rss_dsd_medium	memory_usage	10/10	✅ 90.1 MiB ≤ 100 MiB	metrics profiles logs
quality_gates_rss_dsd_ultraheavy	memory_usage	10/10	✅ 384 MiB ≤ 420 MiB	metrics profiles logs
quality_gates_rss_idle	memory_usage	10/10	✅ 31.8 MiB ≤ 40 MiB	metrics profiles logs

Explanation

A change is flagged as a regression when |Δ mean %| > 5.00% in the regressing direction for its optimization goal AND SMP marks the experiment as a regression (is_regression: true). Improvements use the matching criteria for the improving direction. Experiments configured erratic: true (tagged (ignored)) are skipped outright; experiments detected as erratic at runtime (tagged (erratic)) still count, since that flag describes sample dispersion rather than directional certainty. The Δ mean % cell is colored accordingly: 🟢 = improvement, 🔴 = regression, ⚪ = neutral. Reduction in CPU or memory is an improvement; reduction in ingress throughput is a regression.