chore(antithesis): Alter dogstatsd generation to is_malformed regime - #2215
chore(antithesis): Alter dogstatsd generation to is_malformed regime#2215blt wants to merge 1 commit into
Conversation
dd-octo-sts
Bot
added
the
area/test
This comment has been minimized.
This comment has been minimized.
Binary Size Analysis (Agent Data Plane)Baseline: 7f0ba13 · Comparison: eaf9e7e · diff ✅ Binary size difference within thresholdChanges by Module
Detailed Symbol Changes |
![datadog-datadog-prod-us1[bot]](https://avatars.githubusercontent.com/u/365230?s=60&v=4){kind=small}
There was a problem hiding this comment.
More details
The generator redesign did not reveal a clear diff-only behavioral regression. Adversarial cases covered delimiter-bearing fields, packed values, optional fields, framing, fallback sizing, and malformed-input repair; harness execution was blocked by the sandbox's unavailable hyper-http-proxy Git dependency.
🤖 Datadog Autotest · Commit e4ff7fa · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest
Regression Detector (Agent Data Plane)Run ID: Optimization Goals: ✅ No significant changes detectedFine details of change detection per experiment (5)Experiments configured
Bounds Checks: ✅ Passed (5)
ExplanationA change is flagged as a regression when |Δ mean %| > 5.00% in the regressing direction for its optimization goal AND SMP marks the experiment as a regression ( |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e4ff7fa8ae
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
e4ff7fa to
f89d2be
Compare
aad38fb to
e6183a6
Compare
f89d2be to
07e1995
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 07e1995aed
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
e6183a6 to
ae4f370
Compare
07e1995 to
b943695
Compare
ae4f370 to
dbfb781
Compare
b943695 to
a6dd29d
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a6dd29db83
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
a6dd29d to
212be47
Compare
c3941f6 to
cd8f1f4
Compare
212be47 to
57f3d76
Compare
cd8f1f4 to
c1edde3
Compare
170d7ed to
56c2a83
Compare
c1edde3 to
513666b
Compare
e6fcbf2 to
8f3b592
Compare
513666b to
da79efc
Compare
0e9a907 to
4bcdae0
Compare
da79efc to
5974b21
Compare
| /// index but are never malformed. | ||
| /// | ||
| /// One variant per Agent drop rule, across all three message types. The metric `T` timestamp rule is | ||
| /// intentionally absent. See the note in [`classify_metric`]. |
There was a problem hiding this comment.
We should to fix this, because it's easy to fix... but I'll look into having the API docs job skip trying to generate docs for anything under test/antithesis since none of this code needs polished docs that way that we want Saluki to have.
4bcdae0 to
c33ffb4
Compare
Our previous approach to generating dogstatsd load for use in the Antithesis rig was very Datadog/lading inspired: generate something that is correct-by-construction and then bias the result to be 'interesting'. This works well but the biasing systematically avoids conditions that are not-malformed -- that is, accepted by the SUT -- but are demonstrative of goofy behavior. This commit changes the generator to be a hybrid of that approach with a state search approach. We define a predicate `is_malformed` that confirms whether or not a payload will be accepted by the SUT and then payloads which are not malformed, however strange, are transmitted.
c33ffb4 to
eaf9e7e
Compare
There was a problem hiding this comment.
More details
The generator now constructs and validates serialized DogStatsD lines before sending, including delimiter, malformed-input, and boundary-budget paths. No clear diff-only behavioral regression was identified; focused execution was blocked by the sandbox being unable to fetch the pinned Rust git dependency.
🤖 Datadog Autotest · Commit c33ffb4 · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest
3 participants
Summary
Our previous approach to generating dogstatsd load for use in the
Antithesis rig was very Datadog/lading inspired: generate something
that is correct-by-construction and then bias the result to be
'interesting'. This works well but the biasing systematically avoids
conditions that are not-malformed -- that is, accepted by the SUT --
but are demonstrative of goofy behavior. This commit changes the
generator to be a hybrid of that approach with a state search approach.
We define a predicate
is_malformedthat confirms whether or nota payload will be accepted by the SUT and then payloads which are not
malformed, however strange, are transmitted.
Change Type
How did you test this PR?
References