Add oecd:dotstat code system (OECD Data Explorer codes) - #19
Merged
Conversation
Preserve the OECD Data Explorer code (the post-2024 identifier scheme) that the OECD codelists feed already delivers but the ingest projector was discarding, and expose it as a new oecd:dotstat code system on DAC provider and recipient entities, alongside the existing legacy numeric oecd:provider/oecd:recipient codes. This makes resolvekit an offline crosswalk between the old OECD.Stat codes and the new Data Explorer codes. - refresh_oecd_dac.py: projector keeps the dotstatcode field (providers, recipients; channels and agencies carry none) - oecd_dac.py: guarded _attach_dotstat_code helper at the five provider/recipient emission sites - oecd_dac.yaml: regenerated from the live OECD feed (additive dotstatcode only) - kept distinct from iso3 (Kosovo XKX vs XKV; region/multilateral aggregates have no iso3) and excluded from input auto-detection - docs + changelog (0.1.12) Bundled data packs are rebuilt in a follow-up commit.
Rebuild of the four bundled packs that carry DAC provider/recipient entities (geo.countries, geo.regions, org.providers, geo.continental_unions) so the wheel ships the new oecd:dotstat code system. Purely additive: 349 oecd:dotstat codes added across the packs; entities, names, relations, and all other code systems are unchanged. Manifest checksums re-stamped via sync_manifest.
The OECD live codelist publishes only currently-active codes, but old ODA data references graduated territories (Anguilla, Aruba, Hong Kong, Macao, the Gulf states, ...) that left the recipient list. Add those 28 historical recipient codes (collision-checked against active codes; entities already exist) so the old<->new crosswalk can translate older data. Rewrite the refresh script's regenerate step to merge ADDITIVELY for recipients and providers: preserve committed codes the live feed omits (historical), add new active codes, adopt live values for shared codes but report changes for review. A wholesale regenerate would have proposed deleting every historical code. Channels and agencies (duplicate rows, no unique key, no historical need) pass through from the live fetch. Dissolved states whose numeric codes the OECD reassigned (Netherlands Antilles 236->Benin, Serbia and Montenegro 382->Saint Kitts) are deferred — they need date-scoped codes to resolve unambiguously.
Rebuild of geo.countries carrying the 28 graduated-territory recipient codes: +28 oecd:recipient, +17 oecd:dotstat (11 territories already had dotstat via the provider path), +6 OECD-name aliases. Entities, relations, and all non-OECD codes unchanged. Manifest re-stamped.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a new
oecd:dotstatcode system carrying the OECD Data Explorer codes (the post-2024 identifier scheme) on DAC provider and recipient entities, alongside the existing legacy numericoecd:provider/oecd:recipientcodes. This makes resolvekit an offline crosswalk between the old OECD.Stat codes and the new Data Explorer codes: a value in either system resolves to the same entity and converts to the other.The OECD feed already delivered the
dotstatcodefield; the ingest projector was discarding it. It is preserved for providers and recipients (channels and agencies carry none), kept distinct fromiso3(they diverge for Kosovo and for region/multilateral aggregates with no ISO3), and excluded from input auto-detection.Bundled data packs (geo.countries, geo.regions, org.providers, geo.continental_unions) are rebuilt to ship the codes — purely additive, 349 codes added, nothing else changed. Targets 0.1.12.