Skip to content

Blizzard and Raider.IO fixtures encode guessed upstream shapes #39

Description

@Erilla

Question

Five faults surfaced in the fingerprint feature within hours of it reaching test, and 210 unit tests plus the integration and e2e suites passed through every one of them:

Fault Fixed in Why tests missed it
A candidate 404 abandoned the whole sweep #32 No fixture returned 404 mid-roster; the live measurement saw 23 of 393
Ownership gate excluded never-claimed characters #34 Fixtures only covered "withheld", never "no account"
discord_profile: "" raised schema drift #35 Every fixture used a non-empty string or null
403 profile_is_private retried as an outage #36 No fixture returned 403
Roster class name required but never sent #38 The fixture invented playable_class: { name: "Mage" }

The pattern is not "insufficient coverage" — coverage was good. Every one of these fixtures encoded a guessed upstream shape, each guess was wrong, and a passing test then certified the guess. The roster case is the clearest: the fixture asserted a field that does not exist in Blizzard's response, so the test could only ever have passed while production could only ever have failed.

To decide:

  • Whether the Blizzard and Raider.IO fixtures get rebuilt from recorded live payloads (redacted) rather than hand-written objects, and where those recordings live.
  • Whether a periodic check re-fetches a known character and roster and diffs the shape against the fixtures, so upstream drift fails CI instead of production.
  • Whether the negative cases each upstream really returns — 403, 404 mid-roster, empty strings, absent optional objects — become required fixtures per gateway rather than ad hoc additions.

A recording harness already exists in spirit: scripts/prototypes/fingerprint-sweep-measurement.ts parsed the live roster correctly precisely because it read only what was there and skipped the rest. It was deleted from the plan's scope as disposable; its leniency was the thing worth keeping.

Related: #30 — nothing here would have been caught without a live run, which is the argument for that smoke test gating promotion.

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs-triageMaintainer needs to evaluate this issue

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions