Phase 38: Matrix generated-model evidence aggregation

[gparadis]

Phase

Phase 38: Matrix generated-model evidence aggregation.

Status

Active on branch feature/p38-matrix-generated-model-evidence.

Goal

Add generic Modelwright evidence aggregation for FreshForge matrix runs of generated-model workflows so downstream packages can summarize repeated generated-model validation cases without parsing raw artifacts or copying bulky/private data.

Scope

• Add compact matrix evidence path, case, and summary records.
• Aggregate FreshForge matrix run or matrix summary JSON.
• Optionally inspect per-case Modelwright validation-evidence directories or artifact roots.
• Write compact sanitized summary.json and summary.md.
• Add modelwright validation matrix-evidence CLI support.
• Keep FABLE output-ref strategy and scenario semantics out of Modelwright.

Out of scope

• Running FreshForge matrices.
• Rerunning Modelwright generation, execution, or validation.
• FABLE-specific path conventions or workbook-version labels.
• Raw generated source, raw generated values, source workbooks, or full validation reports in compact summaries.
• Stable public API claims.

Child tasks

• P38.1 Define matrix evidence aggregation records #244 P38.1 Define matrix evidence aggregation records.
• P38.2 Add aggregation API and compact writers #245 P38.2 Add aggregation API and compact writers.
• P38.3 Add CLI command for matrix evidence packaging #246 P38.3 Add CLI command for matrix evidence packaging.
• P38.4 Update docs, examples, and tests #247 P38.4 Update docs, examples, and tests.
• P38.5 Verify, PR, deploy docs, and close phase #248 P38.5 Verify, PR, deploy docs, and close phase.

Verification

• .venv/bin/python -m ruff check .
• .venv/bin/python -m pytest
• .venv/bin/sphinx-build -b html docs _build/html -W
• .venv/bin/python scripts/verify_docs_theme.py _build/html
• scripts/check_release_artifacts.sh
• git diff --check

[gparadis]

Phase 38 completed by PR #249. Implemented generic matrix generated-model evidence aggregation, the modelwright validation matrix-evidence CLI, docs, public exports, and tests. Local verification passed Ruff, focused tests, full pytest, Sphinx docs, docs theme verification, release artifact checks, and git diff check; PR checks passed before merge.