diff --git a/index.md b/index.md index 61bf00a..5fd04b7 100644 --- a/index.md +++ b/index.md @@ -1,44 +1,19 @@ --- -title: Verify your TRACE implementation +title: TRACE conformance suite description: Run this suite against a TRACE record to see which conformance level it reaches, and produce a report anyone can reproduce from the record and the suite version. --- -# TRACE Test Suite +[04 · Evidence: can a third party verify all of it offline, years later?](https://agentrust-io.com/#chain) -Check a TRACE record, inspect the findings, and produce a reproducible conformance report. The suite checks the record and supplied evidence; a passing report does not establish that an entire implementation meets every requirement of the [TRACE specification](https://trace.agentrust-io.com). +# Score a TRACE record against the specification -The [eight modules](docs/modules.md) cover envelope, signature, runtime, policy, appraisal, transcript, transparency, and provenance checks. Read the [limitations](LIMITATIONS.md) to interpret what each result establishes. - -!!! tip "TL;DR" - - `verify` answers a question for the person running it. `report` produces an artifact for somebody who was not there. - - `report` runs every level up to `--max-level`, because the useful answer is the highest level a record reaches, not whether it cleared the level someone happened to pick. - - The HTML report is self-contained: no scripts, no fonts, no external CSS, no badge service, nothing fetched when it is opened. - - The report is not evidence, and it says so on its face. It carries the record digest, the suite and library versions, and the command to reproduce the result. - -```bash -pip install agentrust-trace-tests -trace-tests verify --record path/to/trust-record.jwt --level 1 -``` - -## A report you can hand to someone else - -```bash -trace-tests report --record trust-record.json --html report.html --json report.json --badge trace.svg -``` +The suite checks one record and the evidence supplied with it, reports the highest conformance level reached, and writes a report anyone can reproduce from the record digest and suite version. A passing report does not establish that an entire implementation meets every requirement of the [TRACE specification](https://trace.agentrust-io.com). -Use `--fail-under 1` to gate CI on a level. Without it the command always exits `0`, which is what you want when you are producing an artifact rather than enforcing a threshold. `report.json` is stable under `schema: agentrust-io/trace-tests/report/1` for dashboards and CI. - -CLI reports add an independently versioned `obligation_accounting` member for a -bounded three-obligation pilot: `TR-APR-001`, `TR-POL-003`, and `TR-SCA-002`. -The rows and findings come from one execution snapshot, and the report refuses -an incomplete pilot matrix. This does not claim complete TRACE accounting. -The extension treats `report/1` as additively extensible; compatibility with -consumers requiring the exact historical top-level key set is not established. -See [Known limitations](LIMITATIONS.md) for the trust and replay boundary. - -A conformance report that looks authoritative and cannot be checked is the same shape of thing as a control plane writing its own log. So the report tells a reader who does not trust the sender to go and check the record instead, and gives them what they need to do it. +[Score your first record](docs/quickstart.md){ .md-button .md-button--primary } +[What this proves, and what it does not](LIMITATIONS.md){ .md-button } -## Where to start +!!! tip "TL;DR" + [agentrust-trace-tests](https://pypi.org/project/agentrust-trace-tests/) 0.5.1 (Apache-2.0) runs eight modules against a record on your machine and writes a report carrying the record digest, the suite version and the command to reproduce it. A pass describes the record and says nothing about the agent, and TR-RTE checks the shape of attestation fields without verifying a quote against AMD or Intel roots.
@@ -50,32 +25,67 @@ A conformance report that looks authoritative and cannot be checked is the same [Getting Started](docs/quickstart.md) -- __Understand the levels__ +- __What it proves, and what it does not__ --- - What each conformance level requires, and what a record has to carry to reach it. + The report is not evidence, and it says so on its face. Each result's scope is set out module by module. - [Conformance Levels](docs/levels.md) + [Limitations](LIMITATIONS.md) -- __Read the modules__ +- __Hardware evidence__ --- - The eight test modules, the `TR-*` error codes they emit, and what each one checks. + Quote verification happens outside the suite. Check a real Intel TDX quote at [agentrust-io.com/verify](https://agentrust-io.com/verify/). - [Test Modules](docs/modules.md) + [Runtime module](docs/modules/tr-rte.md) -- __Wire it into CI__ +- __The chain__ --- - Gate a pipeline on a level, and write your own conformance tests against the suite. + The suite scores TRACE records, the evidence step. The specification is at [trace.agentrust-io.com](https://trace.agentrust-io.com), and records can be anchored in the [TRACE Registry](https://agentrust-io.com/registry/). - [CI integration](docs/tutorials/ci-integration.md) + [See the chain](https://agentrust-io.com/#chain)
+The [eight modules](docs/modules.md) cover envelope, signature, runtime, policy, appraisal, transcript, transparency, and provenance checks. Read the [limitations](LIMITATIONS.md) to interpret what each result establishes. + +```bash +pip install agentrust-trace-tests +trace-tests verify --record path/to/trust-record.jwt --level 1 +``` + +## A report you can hand to someone else + +```bash +trace-tests report --record trust-record.json --html report.html --json report.json --badge trace.svg +``` + +- `verify` answers a question for the person running it. `report` produces an artifact for somebody who was not there. +- `report` runs every level up to `--max-level`, because the useful answer is the highest level a record reaches, not whether it cleared the level someone happened to pick. +- The HTML report is self-contained: no scripts, no fonts, no external CSS, no badge service, nothing fetched when it is opened. + +Use `--fail-under 1` to gate CI on a level. Without it the command always exits `0`, which is what you want when you are producing an artifact rather than enforcing a threshold. `report.json` is stable under `schema: agentrust-io/trace-tests/report/1` for dashboards and CI. + +CLI reports add an independently versioned `obligation_accounting` member for a +bounded three-obligation pilot: `TR-APR-001`, `TR-POL-003`, and `TR-SCA-002`. +The rows and findings come from one execution snapshot, and the report refuses +an incomplete pilot matrix. This does not claim complete TRACE accounting. +The extension treats `report/1` as additively extensible; compatibility with +consumers requiring the exact historical top-level key set is not established. +See [Known limitations](LIMITATIONS.md) for the trust and replay boundary. + +A conformance report that looks authoritative and cannot be checked is the same shape of thing as a control plane writing its own log. So the report tells a reader who does not trust the sender to go and check the record instead, and gives them what they need to do it. + +## Where to go next + +- [Conformance Levels](docs/levels.md): what each level requires, and what a record has to carry to reach it. +- [Test Modules](docs/modules.md): the eight modules, the `TR-*` error codes they emit, and what each one checks. +- [CI integration](docs/tutorials/ci-integration.md): gate a pipeline on a level, and write your own conformance tests against the suite. + ## Test modules | Module | ID | Tests | @@ -90,3 +100,5 @@ A conformance report that looks authoritative and cannot be checked is the same | Provenance | `TR-SCA` | SLSA level, builder URI, digest format | The suite tracks [TRACE Spec v0.2](https://trace.agentrust-io.com). See [Changelog](CHANGELOG.md) for what moved between suite versions. + +**Status:** agentrust-trace-tests 0.5.1 · Apache-2.0 · tracks TRACE Spec v0.2 · Sponsored by OPAQUE, which funds the engineering, infrastructure and confidential-computing work behind these projects.