Skip to content

fix(cli): enrich-agencies reports distinct rows, not upsert counts (#142) - #202

Merged
alexwolson merged 1 commit into
mainfrom
fix-142-agency-row-counts
Jul 28, 2026
Merged

fix(cli): enrich-agencies reports distinct rows, not upsert counts (#142)#202
alexwolson merged 1 commit into
mainfrom
fix-142-agency-row-counts

Conversation

@alexwolson

Copy link
Copy Markdown
Collaborator

Closes #142.

The bug

tb enrich-agencies printed the number of upserts issued as though it were the table's
contents. The tables dedup on (buyer_id, native_ref) and the award line key — a report and its
amendment, or two items sharing a fallback ref, collapse into one row — so a reader who saw
107 solicitations went looking in the DB and found 74.

The dedup is correct and intended. Only the messaging was wrong.

The fix

Took the issue's first option: report distinct rows. The leading figures now come from two
real row-count queries. #177 already made the delta honest this way; this is the same
correction applied to the totals sitting beside it.

The upsert counts are kept, in a trailing bracket that names them for what they are —
0 upserts and 107 upserts that all deduped are very different runs, and a flat (+0) alone
cannot tell them apart. That diagnostic would have been lost by simply swapping the numbers out.

_source_row_counts gains solicitations (it counted awards/bids only), so all three figures come
from the same source-scoped query rather than mixing sources.

Verification

Run against a copy of the production store, with every figure checked independently against the
DB:

trca stored : 225 solicitations (+0), 439 awards (+0), 408 bids (+0)  [upserts 446/693/527]
zoo stored  : 46 solicitations (+0), 53 awards (+0)                   [upserts 58/58]
ep stored   : 74 solicitations (+0), 88 awards (+0), 115 bids (+0)    [upserts 107/107/115]
trca_board   DB rows: 225 solicitations, 439 awards, 408 bids
zoo_board    DB rows: 46 solicitations, 53 awards, 0 bids
ep_board     DB rows: 74 solicitations, 88 awards, 115 bids

Exact match on all nine figures. EP is the 107-vs-74 case from the issue — and the gap turned
out wider than reported: TRCA was claiming 446 solicitations for a table holding 225.

778 tests passing — 2 new (distinct-rows-not-upserts; the dedup that causes the gap, pinned
directly by upserting two rows under one native_ref and asserting one row results), and 4
updated to the widened 3-tuple _source_row_counts contract.

🤖 Generated with Claude Code

https://claude.ai/code/session_01W53WHx8mm2UHuLFAQWeF62

)

`tb enrich-agencies` printed the number of upserts ISSUED as though it
were the table's contents. The tables dedup on `(buyer_id, native_ref)`
and the award line key -- a report and its amendment collapse into one row
-- so a reader who saw "107 solicitations" went looking and found 74.

The leading figures are now distinct ROWS, from two real row-count
queries. #177 already made the DELTA honest this way; #142 is the same
correction applied to the totals beside it.

The upsert counts are kept, in a trailing bracket that names them: "0
upserts" and "107 upserts that all deduped" are very different runs and
otherwise look identical from a flat (+0).

`_source_row_counts` gains solicitations (it counted awards/bids only),
so all three figures come from the same source-scoped query.

Verified against a copy of the production store -- every printed figure
matches the DB, checked independently:

    trca stored : 225 solicitations (+0), 439 awards (+0), 408 bids (+0)  [upserts 446/693/527]
    zoo stored  : 46 solicitations (+0), 53 awards (+0)                   [upserts 58/58]
    ep stored   : 74 solicitations (+0), 88 awards (+0), 115 bids (+0)    [upserts 107/107/115]

EP is the 107-vs-74 case from the issue. The gap turned out wider than
reported: TRCA was claiming 446 solicitations for a table holding 225.

778 tests passing (2 new, 4 updated to the widened 3-tuple contract).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W53WHx8mm2UHuLFAQWeF62
@alexwolson
alexwolson merged commit c9dfe10 into main Jul 28, 2026
1 check passed
@alexwolson
alexwolson deleted the fix-142-agency-row-counts branch July 28, 2026 17:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Low priority: enrich-agencies prints store-call counts, not distinct rows

1 participant