test(conformance): self-contained, offline-verifiable ACTION fixture bundles

[miyannishar]

In the #36 thread, @Ahmedibrahim222 proposed self-contained per-case ACTION bundles (parent/child TRACE records, delegation credential, key material) to make the offline-verification requirement concrete, and @imran-siddique suggested it become its own issue while preferring the originator file it. @Ahmedibrahim222 — do you still want to take this? I'm happy to pick it up if not.

Problem

ACTION-001–016 evidence is currently assembled in memory inside tests/conformance/test_profile_conformance.py (the _action_chain / _records / _action_evidence / _verify_action_evidence helpers). Nothing serializes a case to disk and re-verifies it with the shipped offline verifier, so cA2A's "offline-verifiable provenance" claim isn't exercised end-to-end from files — only through live Python objects.

Proposal

Add a small set of self-contained bundle directories under tests/fixtures/action/<case>/, each containing:

• chain.json — the serialized signed delegation chain (the shape ca2a verify-chain already consumes)
• records.json — the serialized provenance records (the shape ca2a verify-dag already consumes)
• expected.json — the documented offline verdict plus trusted_root_issuer and optional at_time
• a one-line README naming which behavior the bundle embodies

…plus a generator script that derives these from the existing chain/record builders, and a loader test that runs the real offline path (ca2a verify-dag --dag records.json --chain chain.json --trusted-root-issuer <iss>, i.e. verify_dag / verify_chain / cross_check_chain) against each bundle and asserts the verdict matches expected.json.

Honest boundary

The offline path verifies provenance (chain signatures / attenuation / validity windows), the DAG, the chain↔record cross-check, and any recorded denial outcome. It does not exercise the holder-proof authorization-replay axis, which needs live audience / secret / challenge material and isn't offline-replayable evidence — consistent with the README note that the ACTION helper "does not check holder-proof binding." I'd keep that axis in-code and say so in the fixtures' README.

Scope

Additive only: new fixtures + generator + loader test. Does not touch ACTION-014/015 (#36) or the reporting design (#144).

Could a maintainer confirm the preferred fixture location/shape and /vouch me so I can open the PR?

[imran-siddique]

Take it. @Ahmedibrahim222 proposed the shape on 25 August and has not come back to it in 18 days, so the originator preference has had its run. The idea stays credited to them in this issue and in whatever the fixtures' README says; the work is yours.

Vouching you below so the gate does not close your PR.

Confirming your reading first. The four helpers are where you say (_records at line 120, _action_chain 129, _action_evidence 142, _verify_action_evidence 159), nothing in that file serializes anything, and the holder-proof boundary you want to document is already written at tests/conformance/README.md:127. Point the fixtures' README at it rather than restating it. Your count is right to exclude ACTION-014 and 015: grep -c '^def test_action_' gives 14, covering 001 to 013 and 016.

One thing that changes the design, and it is the whole reason to reuse rather than invent. This pattern already exists. tests/unit/test_committed_examples_verify.py runs the real offline CLI path against examples/cross-operator-delegation and examples/rejection-with-proof, and it reads each blob out of git with a _committed() helper rather than from the working tree. Its docstring says why, and it is not hypothetical:

changing the hashed record body left examples/cross-operator-delegation/dag.json broken on disk, with every test still green and the README still quoting ca2a verify-dag as working.

A generator that derives fixtures from the in-memory builders, plus a loader that reads those fixtures off disk, is self-fulfilling if anything in CI can run the generator first. The loader has to read the committed blob. Reuse _committed() from that file, or lift it somewhere both tests can import, and then a bundle nobody regenerated fails instead of passing.

Three smaller calls so you are not guessing.

tests/fixtures/action/<case>/ is the right location. The examples/ bundles are reader-facing artifacts and these are not.

Name the record file dag.json, not records.json. It is the same shape, and the existing committed examples and ca2a verify-dag --dag already use that name, so one loader and one CLI invocation cover both sets instead of two of each.

Keep expected.json carrying the verdict triple plus trusted_root_issuer and optional at_time, as you propose. Add the denial reason code where the case has one, since the conformance README already specifies those per case and a bundle that agrees on the verdict while disagreeing on the reason is the failure worth catching.

Additive, and it does not touch #36 or #144. Send it when ready.

[imran-siddique]

/vouch @miyannishar

[github-actions]

@miyannishar is vouched by @imran-siddique. Pull requests from that account will not be closed by the vouch check.