Skip to content

test(conformance): separate ACTION provenance, authorization, and controller reporting #162

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
imran-siddique merged 4 commits into from
Sep 11, 2026
Merged

test(conformance): separate ACTION provenance, authorization, and controller reporting #162

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
0e2240e
Add files via upload
Bashirloyan Sep 8, 2026
60dccf3
Update READMe
Bashirloyan Sep 8, 2026
813002a
update test_profile_conformance.py
Bashirloyan Sep 8, 2026
2393080
update readme
Bashirloyan Sep 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 68 additions & 13 deletions tests/conformance/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,21 +95,76 @@ Spec: [call-graph.md](../../docs/spec/call-graph.md)

Spec: [trace-a2a-profile.md](../../docs/spec/trace-a2a-profile.md), [provenance-dag.md](../../docs/spec/provenance-dag.md), [call-graph.md](../../docs/spec/call-graph.md)

The ACTION helper checks delegation-linked action evidence and reports its
verdict on three separate axes, so a correct denial or a correct rejection
is never confused with a broken or absent provenance chain:

| Field | Question it answers | Values |
|---|---|---|
| `provenance_status` | Did the delegation chain and its linked TRACE/provenance records check out? | `verified`, `invalid`, `missing`, `not_evaluated` |
| `authorization_decision` | Was the requested capability actually allowed by the delegated scope and local policy? | `allowed`, `denied`, `not_evaluated` |
| `controller_outcome` | What did the controller report for the action? | `accepted`, `rejected`, `not_evaluated` |

**Security order.** `provenance_status` is checked first. `authorization_decision`
is only reported once provenance is verified; a provenance failure reports
authorization and controller as `not_evaluated` rather than guessing at
either. `controller_outcome` is reported only once authorization allows the
request, for the same reason. A local-policy or scope denial (ACTION-005,
ACTION-006) is `verified` / `denied` / `not_evaluated`: the denial shows the
control worked correctly, and is not evidence of bad provenance.

**`provenance_status` boundary.** `invalid` means the chain or its records
were present and a check found a defect in them (a tampered signature, a
broken or mismatched link, scope escalation, or a credential outside its
validity window). `missing` means a chain is claimed but the record set
needed to complete the check was never fully supplied — for example a
non-root record without its parent — which is a different fact from a
check that ran and failed. `not_evaluated` means no delegation chain was
claimed at all (ACTION-016): there is nothing here for a delegation-
provenance check to evaluate, which is a different fact again from evidence
that is missing or invalid. `verified` means every check this helper
performs on the chain and its records passed. `provenance_status` does not
cover holder-proof binding: this helper replays recorded evidence offline,
so it does not check whether the original caller proved it held the leaf
key, and it never claims that absence as either `verified` or `invalid`.

**`controller_outcome` boundary.** `controller_outcome` is currently a
claimed test-helper input (`_ActionEvidence.controller_decision`), not
evidence cryptographically proven by a signed ACTION or TRACE record.

`classification` and `code` are the pre-existing broad category and exact
reason, kept on the result for compatibility with callers that only look at
those two fields.

The five minimum reporting cases agreed for this issue:

1. Policy denial: `verified` / `denied` / `not_evaluated` (ACTION-005, ACTION-006).
2. Controller rejection: `verified` / `allowed` / `rejected` (ACTION-007).
3. Invalid signature: `invalid` / `not_evaluated` / `not_evaluated` (ACTION-008).
4. Missing evidence: `missing` / `not_evaluated` / `not_evaluated` (ACTION-003).
5. Outside helper scope: `not_evaluated` / `not_evaluated` / `not_evaluated`
(ACTION-016) — a check this helper is deliberately not evaluating, never
reported as `verified` or `invalid`.

| ID | Level | Requirement | Expected outcome |
|---|---|---|---|
| ACTION-001 | MUST | A delegated action with a parent-linked TRACE/provenance record, matching credential id, and permitted capability verifies. | `verified`. |
| ACTION-002 | MUST | A child action record whose parent record hash does not match the canonical parent record hash is rejected as provenance-invalid. | `PROVENANCE_LINK_BROKEN`. |
| ACTION-003 | MUST | A non-root delegated action record without its parent record is rejected as provenance-invalid. | `PROVENANCE_LINK_BROKEN`. |
| ACTION-004 | MUST | Action evidence naming a delegation credential id that is not the verified leaf credential is rejected as provenance-invalid. | `PROVENANCE_LINK_BROKEN`. |
| ACTION-005 | MUST | A requested action outside the effective delegated scope is classified as authorization-invalid, not malformed provenance. | `SCOPE_NOT_PERMITTED`. |
| ACTION-006 | MUST | A valid delegated action denied by local policy is classified as authorization-invalid, not malformed provenance. | `SCOPE_NOT_PERMITTED`. |
| ACTION-007 | MUST | A valid delegated action whose controller outcome is negative remains valid evidence of a negative outcome. | `valid_negative_outcome`. |
| ACTION-008 | MUST | An action whose delegation chain contains a credential with an invalid signature is rejected as provenance-invalid before authorization or outcome handling. | `INVALID_CREDENTIAL`. |
| ACTION-009 | MUST | A delegated action with a strictly attenuating multi-hop credential chain verifies. | `verified`. |
| ACTION-010 | MUST | An action evidence chain with scope widening at an intermediate hop is rejected as provenance-invalid. | `SCOPE_ESCALATION`. |
| ACTION-011 | MUST | Action evidence whose delegatee differs from the subject of the referenced credential is rejected as provenance-invalid. | `PROVENANCE_LINK_BROKEN`. |
| ACTION-012 | MUST | Action evidence whose delegation chain contains an expired credential is rejected as provenance-invalid. | `CREDENTIAL_EXPIRED`. |
| ACTION-013 | MUST | Action evidence whose delegation chain contains a not-yet-valid credential is rejected as provenance-invalid. | `CREDENTIAL_NOT_YET_VALID`. |
| ACTION-001 | MUST | A delegated action with a parent-linked TRACE/provenance record, matching credential id, and permitted capability verifies. | `verified` / `allowed` / `accepted` (`verified`, `ACCEPTED`). |
| ACTION-002 | MUST | A child action record whose parent record hash does not match the canonical parent record hash is rejected as provenance-invalid. | `invalid` / `not_evaluated` / `not_evaluated` (`provenance_invalid`, `PROVENANCE_LINK_BROKEN`). |
| ACTION-003 | MUST | A non-root delegated action record without its parent record is rejected as provenance-missing. | `missing` / `not_evaluated` / `not_evaluated` (`provenance_invalid`, `PROVENANCE_LINK_BROKEN`). |
| ACTION-004 | MUST | Action evidence naming a delegation credential id that is not the verified leaf credential is rejected as provenance-invalid. | `invalid` / `not_evaluated` / `not_evaluated` (`provenance_invalid`, `PROVENANCE_LINK_BROKEN`). |
| ACTION-005 | MUST | A requested action outside the effective delegated scope is classified as authorization-invalid, not malformed provenance. | `verified` / `denied` / `not_evaluated` (`authorization_invalid`, `SCOPE_NOT_PERMITTED`). |
| ACTION-006 | MUST | A valid delegated action denied by local policy is classified as authorization-invalid, not malformed provenance. | `verified` / `denied` / `not_evaluated` (`authorization_invalid`, `SCOPE_NOT_PERMITTED`). |
| ACTION-007 | MUST | A valid delegated action whose controller outcome is negative remains valid evidence of a negative outcome. | `verified` / `allowed` / `rejected` (`valid_negative_outcome`, `CONTROLLER_REJECTED`). |
| ACTION-008 | MUST | An action whose delegation chain contains a credential with an invalid signature is rejected as provenance-invalid before authorization or outcome handling. | `invalid` / `not_evaluated` / `not_evaluated` (`provenance_invalid`, `INVALID_CREDENTIAL`). |
| ACTION-009 | MUST | A delegated action with a strictly attenuating multi-hop credential chain verifies. | `verified` / `allowed` / `accepted` (`verified`, `ACCEPTED`). |
| ACTION-010 | MUST | An action evidence chain with scope widening at an intermediate hop is rejected as provenance-invalid. | `invalid` / `not_evaluated` / `not_evaluated` (`provenance_invalid`, `SCOPE_ESCALATION`). |
| ACTION-011 | MUST | Action evidence whose delegatee differs from the subject of the referenced credential is rejected as provenance-invalid. | `invalid` / `not_evaluated` / `not_evaluated` (`provenance_invalid`, `PROVENANCE_LINK_BROKEN`). |
| ACTION-012 | MUST | Action evidence whose delegation chain contains an expired credential is rejected as provenance-invalid. | `invalid` / `not_evaluated` / `not_evaluated` (`provenance_invalid`, `CREDENTIAL_EXPIRED`). |
| ACTION-013 | MUST | Action evidence whose delegation chain contains a not-yet-valid credential is rejected as provenance-invalid. | `invalid` / `not_evaluated` / `not_evaluated` (`provenance_invalid`, `CREDENTIAL_NOT_YET_VALID`). |
| ACTION-016 | MUST | Action evidence naming no delegation chain at all is outside this helper's scope, not a failed or incomplete check. | `not_evaluated` / `not_evaluated` / `not_evaluated` (`outside_helper_scope`, `NO_DELEGATION_CHAIN`). |

ACTION-014 and ACTION-015 are reserved for the historical-replay work
tracked separately; this issue does not use them.

## Group 8: Holder binding

Expand Down
Loading
Loading