Question
Five faults surfaced in the fingerprint feature within hours of it reaching test, and 210 unit tests plus the integration and e2e suites passed through every one of them:
| Fault |
Fixed in |
Why tests missed it |
| A candidate 404 abandoned the whole sweep |
#32 |
No fixture returned 404 mid-roster; the live measurement saw 23 of 393 |
| Ownership gate excluded never-claimed characters |
#34 |
Fixtures only covered "withheld", never "no account" |
discord_profile: "" raised schema drift |
#35 |
Every fixture used a non-empty string or null |
403 profile_is_private retried as an outage |
#36 |
No fixture returned 403 |
| Roster class name required but never sent |
#38 |
The fixture invented playable_class: { name: "Mage" } |
The pattern is not "insufficient coverage" — coverage was good. Every one of these fixtures encoded a guessed upstream shape, each guess was wrong, and a passing test then certified the guess. The roster case is the clearest: the fixture asserted a field that does not exist in Blizzard's response, so the test could only ever have passed while production could only ever have failed.
To decide:
- Whether the Blizzard and Raider.IO fixtures get rebuilt from recorded live payloads (redacted) rather than hand-written objects, and where those recordings live.
- Whether a periodic check re-fetches a known character and roster and diffs the shape against the fixtures, so upstream drift fails CI instead of production.
- Whether the negative cases each upstream really returns — 403, 404 mid-roster, empty strings, absent optional objects — become required fixtures per gateway rather than ad hoc additions.
A recording harness already exists in spirit: scripts/prototypes/fingerprint-sweep-measurement.ts parsed the live roster correctly precisely because it read only what was there and skipped the rest. It was deleted from the plan's scope as disposable; its leniency was the thing worth keeping.
Related: #30 — nothing here would have been caught without a live run, which is the argument for that smoke test gating promotion.
Question
Five faults surfaced in the fingerprint feature within hours of it reaching
test, and 210 unit tests plus the integration and e2e suites passed through every one of them:discord_profile: ""raised schema driftprofile_is_privateretried as an outageplayable_class: { name: "Mage" }The pattern is not "insufficient coverage" — coverage was good. Every one of these fixtures encoded a guessed upstream shape, each guess was wrong, and a passing test then certified the guess. The roster case is the clearest: the fixture asserted a field that does not exist in Blizzard's response, so the test could only ever have passed while production could only ever have failed.
To decide:
A recording harness already exists in spirit:
scripts/prototypes/fingerprint-sweep-measurement.tsparsed the live roster correctly precisely because it read only what was there and skipped the rest. It was deleted from the plan's scope as disposable; its leniency was the thing worth keeping.Related: #30 — nothing here would have been caught without a live run, which is the argument for that smoke test gating promotion.