Skip to content

Declare each calibration hierarchy label once: diagnostics schema 9 catalogs #1341

Open

Description

@anth-volk

Problem

calibration_diagnostics.json schema 8 copies every provider, category, geography and dimension label onto every target row. Labels come from each source fact (Chronicle packages from different publishers) or from Microcosm builders. Microcosm checks that the facts inside one target agree, and since #1335 that a US registry gives each geography ID one label, but nothing checks dimension values across targets, and other countries only warn. So a published file can label one identifier two ways:

  • US national release populace-us-2024-0fb05b6-4b57d15a287c-20260930T150401Z: AGI band under_1 is "No adjusted gross income" on 106 targets and "No adjusted gross income and deficit" on 2; all is "All returns" (47) and "All returns, total" (53); age 85_plus is "Age 85 plus" (51) and "Age 85 and over" (1). The current US feed pin (Chronicle 51aa40f) contains these facts unchanged, so the next national build would repeat them.
  • US ACS local release populace-us-2024-buildo-acs-local-766b7c7-20261017T030317Z: 51 states named both ways ("Maine" / "ME"), fixed in the producer by Give each US geography one label across target sources #1336.
  • UK: Chronicle's local facts carry the area as a dimension named geography whose value is the lower-case GSS code, labelled with each publisher's name. The October 2026 dense release gives 88 areas two names ("Hartlepool UA" / "Hartlepool", "Ynys Mon" / "Ynys Môn"); the October national releases give "Yorkshire And The Humber" / "Yorkshire and The Humber".

The calibration dashboard has to choose one spelling per identifier; a disagreement it did not expect stopped every US release tree publication from 2026-10-02 (calibration-diagnostics#246, #249).

Wanted

A diagnostics schema 9 in which one geography ID or one dimension value ID cannot carry two labels: provider, category, geography and dimension labels declared once in catalogs and referred to by ID from the rows, as the country contracts already declare providers and categories. The writer builds the catalogs from the targets and refuses a document whose rows give one identifier two labels, except where a reviewed label choice resolves it. The JSON Schema from #1298 needs a schema 9 counterpart. Releases already published keep schema 8 and stay readable.

This must merge only after the calibration dashboard reads schema 9 in production.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions