geo data refresh + curated alias fixes (0.1.10) - #16
Merged
Conversation
Bundled-data rebuild fixing geo misresolutions: World Bank short names
("Congo, Rep." etc.), "Taiwan Province of China" -> country/TWN, and
"EU Institutions" -> DAC/EUInstitutions. Adds DAC/Non-DAC aggregates,
the Euro Area, and a World entity.
- GraphContribution.names_to_delete: enrichers can delete an individual
alias row, not just suppress re-insertion (suppression leaks on
cache-reuse builds where the staging store accumulates prior output).
- Build resilience: Data Commons discovery cache + transient-5xx retry;
Wikidata per-batch cache + QID validation; calibrate_geo re-stamps the
pack metadata calibrator checksum on install.
- BYOD: overlay composition, bulk on_missing for resolved-but-uncoded
entities, intake availability union.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rebuilds the bundled geo packs (in-place data refresh) and fixes a set of geo resolution bugs:
country/TWN(the competing UN region aggregate is removed).DAC/EUInstitutionsinstead of the European Union.Code:
GraphContribution.names_to_delete(delete an alias row, not just suppress re-insertion — suppression leaks on cache-reuse builds); Data Commons discovery cache + transient-5xx retry; Wikidata per-batch cache + QID validation; BYOD overlay composition,bulk(on_missing=...), and intake availability union.Note: the remote deep tiers (admin2–5, cities) need a separate
release_datarun to publish refreshed assets. Theeval_geogate is informational in CI (no remote tiers); no regression observed (country-tier accuracy 0.827, recalibration ≥ prior on the same pack).