Skip to content

Make SizeLimit tests deterministic by deduplicating generator output#1098

Draft
dwcullop wants to merge 1 commit into
reactivemarbles:mainfrom
dwcullop:fix/sizelimit-test-flake
Draft

Make SizeLimit tests deterministic by deduplicating generator output#1098
dwcullop wants to merge 1 commit into
reactivemarbles:mainfrom
dwcullop:fix/sizelimit-test-flake

Conversation

@dwcullop
Copy link
Copy Markdown
Member

Problem

SizeLimitFixture.InvokeLimitSizeToWhenOverLimit and SizeLimitFixture.AddMoreThanLimitInBatched fail intermittently in CI with errors of the form:

Expected _results.Messages[1].Adds to be 10 because Should be 10 adds in the second update, but found 9 (difference of -1).

Both tests build a 10-item batch, then build a second 10-item batch, and assert that the two AddOrUpdate calls produced exactly 10 Adds each.

RandomPersonGenerator.Take(N) draws random Person rows from a finite name pool (~21 first names for girls + ~30 first names for boys, each cross-joined with 24 lastnames squared). Person.Key is Person.Name. Two independent .Take(10) calls can return rows that share the same generated Name, so the second batch's AddOrUpdate sees one of its rows as an existing key and emits 9 Adds + 1 Update instead of 10 Adds + 0 Updates.

The probability is small per row, but two Take(10) calls draw 20 rows into the same key space; birthday-paradox math puts the collision rate at roughly 0.3%. That matches the observed CI failure frequency for these two tests.

Fix

Draw a larger candidate pool (60 rows), dedupe by Key, take the first 20 distinct, and split into two non-overlapping batches of 10. No production code changes.

Verification

50 consecutive runs of SizeLimitFixture on the branch, zero failures. Targeted command:

for ($i=1; $i -le 50; $i++) {
  dotnet test ... --filter "FullyQualifiedName~SizeLimitFixture"
}

RandomPersonGenerator emits Person rows drawn from a finite name pool
(~21 girls + ~30 boys cross-joined with 24 lastnames squared). Person.Key
is Person.Name, so two independent .Take(10) calls can produce overlapping
keys with non-trivial probability. When they collide, the second batch's
AddOrUpdate produces 9 Adds + 1 Update instead of 10 Adds, breaking the
per-message assertions in:

- InvokeLimitSizeToWhenOverLimit
- AddMoreThanLimitInBatched

Both tests now draw 60 candidates up front, dedupe by Key, take the first
20, and split into two non-overlapping batches of 10.

Verified: 50/50 consecutive runs of SizeLimitFixture pass with no failures.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant