Skip to content

Data quality: capture numbers that don't match the tags (#168) - #169

Merged
Luen merged 1 commit into
mainfrom
feat/recapture-flags
Nov 9, 2026
Merged

Luen merged 1 commit into
mainfrom
feat/recapture-flags

Conversation

@Luen

@Luen Luen commented Nov 9, 2026

Copy link
Copy Markdown
Collaborator

Closes #168.

The site counts a shark once per tag (plus the 2013-14 and 2014-15 Recaptures-sheet groups) and never reads the sheets' capture numbers. This adds import/recaptures.ts, which compares the two for each shark and logs disagreements under a new recaptures category on the Data quality page:

Check Severity
Marked capture 2 or more, but no earlier capture of that shark is in the sheets. The message notes when the tag is in the lab records. warning
Marked capture 1, but its tag was caught earlier the same season info
One tag on two captures on the same day, neither marked as a recapture (a mistyped tag, or a row entered twice?) warning

Two things are not flagged:

  • Repeats across seasons, because capture numbers restart each season.
  • The 2013-14 Catch data, where the capture-number column holds how many times the shark was caught (both rows of a shark caught twice say 2), not the order. A shark with at least N rows all marked N passes.

Real data: 25 warnings and 7 notes, with no change to any count (blacktip individuals stay at 1,843; lemons at 657):

Blacktip Sicklefin lemon
Recapture with no earlier capture (tagged) 7 (2 of them in the lab records) 7
Recapture with no earlier capture (untagged) 4 3
Capture 1, caught earlier that season 4 3
Tag on two captures in one day 1 3

Changes

  • checkRecaptures runs once per individual in the importer.
  • IMPORTER_VERSION is bumped to 11.
  • Unit tests use synthetic captures.
  • The Data quality lede mentions the new check.
  • AGENTS.md importer notes now say capture numbers never decide identity and describe the 2013-14 convention.

Checks: pnpm lint, pnpm check, pnpm test (118 passed), the real-data test, pnpm build, and a look at /data-quality filtered to the new category on a local server with the real data.

🤖 Generated with Claude Code

The importer counts a shark once per tag and never reads the sheets'
capture numbers. It now logs where the two disagree, under a
"recaptures" category on the Data quality page:

- capture 2 or more with no earlier capture of that shark (noting when
  the tag is in the lab records);
- capture 1 whose tag was caught earlier the same season;
- one tag on two unnumbered or first captures on the same day.

Repeats across seasons are expected (numbers restart each season) and
2013-14's habit of writing the number of catches on every row (2, 2)
is accepted. How individuals are counted doesn't change; real data:
25 warnings and 7 notes, blacktip individuals still 1,843.

Closes #168

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Luen Luen added the data-quality Issues in the source spreadsheets for the lab to review label Nov 9, 2026
@Luen
Luen merged commit afd3696 into main Nov 9, 2026
6 checks passed
@Luen
Luen deleted the feat/recapture-flags branch November 9, 2026 01:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

data-quality Issues in the source spreadsheets for the lab to review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Data quality: capture numbers that don't match the tags

1 participant