Skip to content

fix(audit): stop under-reporting suppressed_count as a maximum - #86

Merged
plusky merged 1 commit into
mainfrom
fix/suppressed-count-total
Aug 11, 2026
Merged

fix(audit): stop under-reporting suppressed_count as a maximum#86
plusky merged 1 commit into
mainfrom
fix/suppressed-count-total

Conversation

@plusky

@plusky plusky commented Aug 11, 2026

Copy link
Copy Markdown
Owner

Closes #68.

AuditCell::into_guard_info computed suppressed_count as max(suppressed_ids.len(), suppressed_extra) — the larger of two tallies, not a total. A call that dropped 3 private comments while hiding 2 duplicate-marker bug ids recorded suppressed_count: 3 beside a two-element id list, where the truth was 5. It under-reported silently: the record validated and looked complete.

I3 forbids the client learning anything was withheld, so the audit stream is the only place this surfaces, and suppressed_count is authoritative by design — suppressed_ids sits behind a config switch, so with ids elided the count is all an operator has.

The decision, and a conscious departure from AC 2

Sum the two tallies. Stay on SCHEMA_VERSION = 1. No field added, removed, renamed or retyped.

#68's second acceptance criterion asked that SCHEMA_VERSION move with any semantics change, coordinated with #34. It deliberately does not. A maximum of two disjoint populations is not a count of anything, so this restores the contract the field already documented ("authoritative … never infer the count from the id list") rather than replacing a working one; and a bump would fork every reader for a record whose fields, names and types are identical. The project's own precedent points the same way — scan was added to the record and stayed v1.

The cost that buys is recorded in DESIGN.md, not just here. There is no discriminator in the record: AuditEvent carries v/ts/seq/session only, initialize records the client's version and never bugwarden's build, policy_hash is unchanged absent a policy edit, and suppressed_count >= len(suppressed_ids) holds under both readings, so there is no structural tell either. A v: 1 corpus spanning this change carries both readings, with the deployed build as the boundary. The safe reading of a pre-upgrade record is "at least max(suppressed_count, len(suppressed_ids)), possibly more". SCHEMA_VERSION's rustdoc now states the structure-not-meaning rule this relies on, which it previously left unstated.

Why the sum is safe

The populations are disjoint by construction, not merely by category. Three sites feed the id-less tally: bug_comments (server.rs:1889), list_attachments (:2723), summarize_bug (:3117). The two that also name ids harvest Guard::duplicate_marker_ids from the post-filter comment list (server.rs:1881, :3111), so a dropped private comment contributes its id to nothing even when its text carries a marker. list_attachments names no ids. bugs_quicksearch feeds only ids, into a BTreeSet that dedupes; note_scan never touches the id-less tally.

Both harvest sites now carry a LOAD-BEARING: comment — that is where the mistake would be made, and the I14 rationale sitting beside them is an active invitation to move the harvest pre-filter.

Rollout impact

Operators running suppressed_ids = true with a consumer that alerts on suppressed_count != len(suppressed_ids) will see new firings on mixed bug_comments/summarize_bug calls after deploy. The record is now correct; the alert's premise was only ever true under the old maximum.

Adversarial review before opening

Three reviewers attacked the uncommitted diff — disjointness, invariants, and convention lenses. Findings and resolutions:

Finding Resolution
The load-bearing premise was completely untested. Mutating either harvest site to read the pre-filter list — the one edit that destroys disjointness — left all 420 tests green, because the fixture's private comments carried no duplicate markers. That mutant records count=6, ids=[666,667,668] where the truth is 5: one withheld comment counted twice, an over-report strictly worse than the bug being fixed. The fixture's third private comment is now itself a duplicate marker naming a hidden bug 668, with the id=666,667,668 classify variant mounted so the mutant dies on the number rather than a 404. Both pre-filter mutants now fail.
summarize_bug's recorded number changes with this diff but had zero integration coverage — deleting its counter call left the suite green. New summarize_bug_records_the_same_total_as_bug_comments; the deletion mutant now fails 2-vs-5.
The disjointness contract lived in audit.rs, but the mistake is made in server.rs. LOAD-BEARING: comments at both harvest sites.
Staying on v1 was justified purely on wire shape; the accepted cost and the compatibility rule it leans on were recorded nowhere. Both now in DESIGN.md and SCHEMA_VERSION's rustdoc.
The new rustdoc contradicted itself — "the sites that count id-less content name no ids at all" is false for the two the argument depends on. Corrected.
Both end-to-end tests passed with the classify mock broken (Guard::disclosable fails closed and withholds anyway), making the policy fixture decorative. .expect(1) on the mock; all three tests now fail if it goes unmatched.

Verified under refutation and left alone: ScanInfo::dropped's analogy is about elidability, not max-vs-sum; the serialization golden builds a hand-written GuardInfo literal and never drains a cell; the README worked example is a bugs_quicksearch record where no id-less path exists. A repo-wide sweep found no survivor of the old max() wording. Disjointness also holds under include_private = true, total - filtered.len() cannot underflow (filter_private is a pure .filter()), and no as truncation shadows the saturation.

Verification

cargo fmt --check, both clippy invocations, cargo test --workspace --all-targets --locked (421 passed, 0 failed) and cargo deny check all pass — re-run independently, not taken on report. cargo doc --no-deps --workspace is byte-identical to the main baseline.

Reverting saturating_add to .max( fails five tests. Every test added or rewritten here was mutation-proven to fail against the change it claims to pin.

Follow-ups filed

#85 (AGENTS.md still calls the invariant set I1–I13 while DESIGN.md defines through I16). Also noted, not yet filed: removing note_suppressed_count's if n == 0 { return; } guard leaves the suite green, so nothing asserts a clean bug_comments call records served rather than served_filtered — pre-existing at HEAD.

The field took the larger of two tallies — the withheld bug ids and the
id-less suppressed content — rather than their total. A summarize_bug
call that dropped three private comments while hiding two duplicate
marker ids recorded 3 beside a two-element id list, where the truth was
5. It under-reported silently: the record validated and looked complete.

That matters because I3 forbids the client learning anything was
withheld, so the audit stream is the only place the fact surfaces, and
suppressed_count is authoritative by design — suppressed_ids sits behind
a config switch, so with ids off the count is all an operator has. A
maximum of two disjoint populations is not a count of anything, and
cannot satisfy the "authoritative, never infer it from the id list"
contract the field already documented.

Sum them. The populations are disjoint by construction, not merely by
category: the two tools that feed both tallies, bug_comments and
summarize_bug, harvest their duplicate-marker ids from the comments that
survived the private filter, so a dropped comment contributes its id to
nothing even when it carries one. list_attachments, the third id-less
site, names no ids at all. Comment at both harvest sites, since that is
where the mistake would be made — the I14 rationale beside them is an
active invitation to move the harvest pre-filter, which would silently
turn this under-report into an over-report.

Keep SCHEMA_VERSION at 1. No field is added, removed, renamed or
retyped, and a bump would fork the reader for a structurally identical
record. #68 asked for the version to move with any semantics change; it
deliberately does not, and DESIGN.md records the cost that buys — a v: 1
corpus spanning this change carries both readings, with the deployed
build and not the schema version as the boundary.

Closes #68
@plusky
plusky merged commit 898c80d into main Aug 11, 2026
11 checks passed
@plusky
plusky deleted the fix/suppressed-count-total branch August 11, 2026 21:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

suppressed_count is a max() of two unrelated tallies, not a count

1 participant