Skip to content

Manifest-driven validate(): read the sidecar schema block as the spec, shared by builders and a PR-validation workflow #119

Closed

Description

@mmcky

The builder template (builders/_template.py) and builders/business_cycle.py both carry a hand-written validate(): business_cycle asserts its own column grid, economy set, value band and null placement, and none of it is read from lectures/business_cycle_data.csv.yml. The manifest's schema block is meant to be the single source of truth (#14, point 3 of the architecture), and today nothing reads it.

Deliverable. A validate(frame, spec, previous) that takes the sidecar's schema block and enforces columns (exact name entries and pattern runs), row_count_floor, known_nulls and date_range, plus the bounded overlap window against the previous vintage. It lives in shared machinery (builders/_validate.py or similar, the extraction #14 deferred until the second dynamic builder landed; the FRED composite is that second builder), and two callers use it unchanged:

  • every dynamic-snapshot builder's validate() stage, with builder-specific checks (business_cycle's value band, for instance) layered on top rather than replacing it;
  • a PR-validation workflow that runs the same checks over every dataset a PR touches, alongside the manifest schema check (manifest-schema.yml conformance) and the CATALOG.md freshness gate. PLAN Phase 5 lists this as "PR validation"; consumed-file-check.yml is the narrow forerunner it subsumes.

Depends on the three schema decisions filed beside this one: the column pattern semantics, whether known_nulls is exact or a ceiling, and the dtype vocabulary. The code can start before they close, but the manifest-conformance half of the PR check cannot be made a required status check until they do.

Out of scope. Consumer fan-out and packaging the refresh job as a reusable workflow stay on PLAN Phase 5 as separate unchecked items.

Sub-issue of #14.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions