Skip to content

Native evidence reader rejects real run manifests by item count #1187

Closed

Description

@juaristi22

The native-evidence reader introduced in #1144 applies the compiler-schema complexity budget (16 bytes charged per JSON item, 32 MiB total) to every evidence file it opens. Real run manifests fail that budget by item count while being well under the byte limit, so a licensed dense build refuses before its first phase and a spine's own evidence bundle cannot be exported.

Observed (f10 rung on the merged drivers, 2026-10-10, machine-local run; runs/uk-f10-1144-0bee8bfb6/failure.json):

  • The licensed spine built normally (6 min; evidence capture 6.9 s, peak RSS unchanged at 11.41 GiB; two gate-report summaries, none failed).
  • The dense role then stopped at _save_graph_presentation with Graph schema exceeds the metadata size limit. The checkpoint boundary hashes the four files the spine sidecar links, and it read them through microcosm.graph.evidence.read_json.
  • The spine run manifest is 12.5 MB on disk but holds 1,422,795 JSON items, which the reader charged as 48.9 MiB. The cgt_incidence_clone receipt alone is 14.6 MiB of that charge; spi_support_channel is 4.0 MiB.
  • load_run_evidence uses the same reader, so collect_execution_evidence on the spine bundle fails the same way: the drivers can save evidence that the exporter cannot load.
  • The graph itself is small (29 operations, 0.3 MB, schema builds fine), and the sidecar is 0.7 MB. Other ordinary UK artifacts carry hundreds of thousands of items in under 15 MB (uk.full.national_targets.registry.json: 437k items, 13.4 MB).

Why it is a design issue, not a tuning issue. The per-item charge exists to keep compiler schemas and Orrery documents bounded, since those are projected into the viewer. Native evidence files are exact copies of manifests and graphs a build already holds in memory; they need a byte and nesting bound, not an item budget. Hashing a file for a checkpoint link or a published-evidence check should not parse it at all.

Fix (PR to follow): bound native evidence inputs by bytes (1 GiB) and nesting depth, keep refusing duplicate keys and non-finite numbers, keep the per-item budget for schemas and Orrery documents only, and hash checkpoint links, copied upstream bytes and published evidence by bytes. On the real spine bundle the export path then loads in 11.2 s, collects in 9.5 s and projects in 1.5 s (4,030 nodes, 17.9 MB).

Related: #1079, #1080, #1144.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions