Skip to content

Add an upload-hop verifier and a generation-scoped trust reader #138

Description

@sehkone

Context

REView (aicers/review) re-verifies every component package uploaded to its module store before it can be served, through this crate's shared verifier, as RFC-A requires: one verifier, and no second implementation in review. Its verification leaf, aicers/review#2060, is blocked on this issue.

Every package verifier entry point here takes the build's identity in advance. verify_package(src, &TrustSet, &VerifyRequest) (src/verify.rs:1410 at 19fef3f) and package::verify_contents(source, trust, request, target_arch, limits, staging_parent) (src/package/contents.rs:88) both need a VerifyRequest naming component, version and commit, and verify_contents also needs a TargetArch. An upload hop learns all four only from the package, and only after authentication. The only request-free reader, Payload::parse_unverified_manifest (src/payload.rs:1833), parses bytes nothing has authenticated, which is exactly what the signature-first order forbids.

What the upload hop needs, and what already exists at 19fef3f:

  • Identity from the authenticated manifest. Missing. authenticate (src/verify.rs:1497) runs the signature, the format floor, the typed parse and check_statements as one unit, so no caller can learn the identity between the parse and the statement checks.
  • A caller-chosen staging directory. Exists on verify_contents: staging_parent, judged by the directory trust policy (src/retain/trusted_dir.rs), with every snapshot made inside one fresh private 0700 directory created in it (RetentionScope::new, src/retain.rs).
  • An enforced staging bound with a distinct verdict. Exists as a mechanism: one RetentionScope charges every snapshot against a disk budget, a write fails at the first byte past it, and the refusal is ContentError::LimitExceeded { resource: LimitResource::RetainedDisk, .. }. What is missing is a bound a caller can compute from the package length before the call, so it can reserve that much disk first. With the default ContentLimits the budget is RetainedDisk, 512 GiB (src/package.rs:811).
  • The signing key and the manifest digest. Missing. verify_signature (src/verify.rs:1661) returns Ok(()) without saying which anchor verified, and no result carries the SHA-256 of the raw manifest block. REView records both per accepted build.
  • The registration template. Exists: it is PayloadArtifact::spec → module_spec::ModuleSpec::registration, a module_spec::RegistrationTemplate (src/module_spec.rs:266), inside the authenticated manifest. Nothing new is needed beyond returning the manifest.
  • One explicit trust generation. Missing. release_trust::active_trust_set (src/release_trust.rs:776) reads active/trust-set.json and then active/epoch through the active symlink, so an activation between the two opens can pair two generations. REView resolves active once itself and needs to decode the gen-<n> it resolved. It cannot do that with this crate's validation from outside: the refusing document reader trust_set::read_trust_set_document (src/trust_set.rs:549) is pub(crate), and read_active_generation (src/release_trust.rs:817) is private.
  • Trust queries. TrustSet::is_withdrawn (src/verify.rs:402) is private and TrustSet has no epoch accessor. REView re-checks withdrawal at admission and at serve against the same generation, and reports its epoch.

This issue adds those pieces by extending the existing pipeline. It adds no second verifier, and it changes no verdict, order or limit of verify_package, verify_contents or any other existing entry point.

Scope

1. UploadRequest (src/verify.rs)

A new exported type holding only what an upload hop knows in advance: the deployment namespace.

  • UploadRequest::new(namespace: &str) -> Result<UploadRequest, InputError> refuses a namespace that is_valid_segment rejects with the existing InputError::InvalidNamespace, exactly as VerifyRequest::for_namespaced_package does.
  • UploadRequest::namespace(&self) -> &str.

The namespace is required, not optional: a package declaring images must be verified under one, and one declaring none verifies exactly as it would without it (the for_namespaced_package rule).

2. Split authentication from the statement checks (src/verify.rs)

Split the private authenticate into:

  • a crate-private authenticate_manifest(container, trust) -> Result<Authenticated, VerifyError> running steps 2–4 of the module order: verify_signature, the format floor, the typed parse. Authenticated holds the parsed PayloadManifest, the key_id of the anchor that verified the signature, and the SHA-256 of the raw manifest block as it was read;
  • the existing check_statements(manifest, request, Some(trust)), unchanged.

authenticate becomes those two calls in sequence, so verify_package, verify_package_bounded and verify_contents keep their order and verdicts exactly. verify_signature returns the verifying anchor's key_id instead of (); its four-step order and every verdict stay as they are.

3. package::verify_upload (new file src/package/upload.rs, re-exported from package)

pub fn verify_upload<R: Read + Seek>(
    source: R,
    trust: &TrustSet,
    request: &UploadRequest,
    limits: &ContentLimits,
    staging_parent: &Path,
) -> Result<VerifiedUpload, ContentError>

It runs the verify_contents pipeline with the request and architecture taken from the authenticated manifest, in this order:

  1. Seek source to its end to learn package_len, then to its start. Open one RetentionScope in staging_parent with the budget upload_staging_bound(limits, package_len) (item 4), and snapshot source exactly as verify_contents does.
  2. Read the snapshot's container under ContainerBounds from limits, then authenticate_manifest. Nothing is parsed before the signature verifies.
  3. Derive the request. Take component, version and commit from the first artifact entry in manifest order, and build VerifyRequest::for_namespaced_package(component, version, commit, request.namespace()). Take target_arch from the same first entry.
    • A manifest with no artifact entry is refused with UploadRefusal::NoArtifacts.
    • A first entry naming TRUST_TARGET is refused with UploadRefusal::ReservedTarget: a trust generation is never an upload, and it has its own admission paths in release_trust.
  4. check_statements(&manifest, &derived, Some(trust)). Every other entry must match the first one's build, so a package mixing builds is VerifyError::TargetMismatch naming the first entry that differs, as today.
  5. The rest of verify_retained after authentication: the archive-block snapshot, then check_contents with the derived target_arch, so a package mixing architectures is ContentError::ArchitectureMismatch whose expected is the first entry's. Refactor verify_retained (src/package/contents.rs:151) so this step and verify_contents share one body rather than a copy.
  6. Build VerifiedUpload, then release every snapshot and remove the private directory (close the scope) before returning. Removal failure never changes the result: it is best effort, as RetentionScope's Drop already is, and anything left is inside the private directory under staging_parent.

VerifiedUpload owns only metadata, never retained bytes:

  • manifest(&self) -> &PayloadManifest, the authenticated manifest (its specs carry the registration templates);
  • component(), version(), commit() as &str, and target_arch() -> TargetArch, as derived in step 3;
  • key_id(&self) -> &str, the verifying anchor's key_id;
  • manifest_sha256(&self) -> &[u8; 32], over the raw manifest block;
  • package_sha256(&self) -> &[u8; 32] and package_len(&self) -> u64, of the snapshot that was verified.

Add ContentError::Upload(UploadRefusal) with UploadRefusal a new exported enum of exactly NoArtifacts and ReservedTarget. They are reachable only from verify_upload, and only after the signature has verified. VerifyError's top-level set is not touched.

4. package::upload_staging_bound (same file)

#[must_use]
pub fn upload_staging_bound(limits: &ContentLimits, package_len: u64) -> u64

Returns min(RetainedDisk, 2 × package_len + OuterUncompressedTotal), with saturating arithmetic, from limits. It is pure and does no I/O. verify_upload uses exactly this value as its scope's budget, so this crate enforces the number the caller reserved.

Why this formula: during the call the scope holds the package snapshot (package_len bytes), the archive-block copy (at most package_len), and the extracted members (at most OuterUncompressedTotal, which extraction already enforces on what it reads). Image validation retains nothing. So the budget is never the reason a package within the other limits is refused, unless the caller lowered RetainedDisk below it. The rustdoc states this derivation, and that a caller who needs a smaller reservation lowers OuterUncompressedTotal or RetainedDisk through ContentLimits::with_limit. A breach is ContentError::LimitExceeded { resource: LimitResource::RetainedDisk, limit }, where limit is the budget the call used.

5. release_trust::generation_trust_set (src/release_trust.rs)

pub fn generation_trust_set(root: &Path, generation: u64) -> Result<TrustSet, ReleaseTrustError>
pub fn parse_generation_name(name: &OsStr) -> Option<u64>
  • generation_trust_set reads trust-set.json and epoch from <root>/gen-<generation>/ only, with the same refusing reader, epoch-record grammar, epoch-agreement check and TrustSet assembly as active_trust_set. Refactor read_active_generation so both share one body that takes the generation directory. It never reads, stats or resolves active.
  • A missing generation directory or file is ReleaseTrustError::Io whose source kind is NotFound, so a caller can tell a pruned generation from a malformed one.
  • parse_generation_name exposes the generation engine's canonical-spelling rule (generation::parse_generation, src/generation.rs:427): gen-1 is Some(1), while gen-01, gen-+1, gen-1.tmp and active are None. A caller resolving active itself matches its target with this rather than a second copy of the rule.
  • active_trust_set keeps its behaviour exactly.

6. TrustSet accessors (src/verify.rs)

Make TrustSet::is_withdrawn(&self, component: &str, version: &str, commit: &str) -> bool public, unchanged, and add TrustSet::epoch(&self) -> u64. Both #[must_use], with rustdoc.

Acceptance criteria

  • verify_upload parses no manifest bytes before the signature verifies. An unsigned package, a flipped signature bit, an unknown key and a revoked key give BadSignature, BadSignature, UnknownKeyId and RevokedKey, including for a package whose manifest has no artifacts, names trust, mixes builds or is otherwise refusable.
  • For every fixture package, verify_upload gives the same verdict as verify_contents called with the fixture's true build, the namespace and its true architecture, and with ContentLimits whose RetainedDisk is upload_staging_bound for that package. On success the derived component, version, commit and target_arch equal those values.
  • key_id() is the key_id of the anchor that verified, also when the container's hint names another anchor or is unusable. manifest_sha256() is the SHA-256 of the raw manifest block. package_sha256() and package_len() describe the verified bytes.
  • A package whose entries differ in build is TargetMismatch; one whose entries differ in architecture is ArchitectureMismatch; one with no artifact is Upload(NoArtifacts); a signed trust-generation container is Upload(ReservedTarget).
  • A v6 package whose image declares another owner.namespace is Image(NamespaceMismatch). UploadRequest::new refuses an invalid namespace with InputError::InvalidNamespace.
  • The scope's high-water mark never exceeds upload_staging_bound(limits, package_len). With RetainedDisk lowered below what a fixture needs, the call is refused as LimitExceeded { resource: RetainedDisk, .. }.
  • Nothing is created outside staging_parent, and after verify_upload returns, on success and on every refusal, staging_parent holds exactly the entries it held before. An untrusted staging_parent is ContentError::Io with IoOperation::InspectStagingParent, as for verify_contents.
  • upload_staging_bound returns the formula in item 4, saturates instead of overflowing, and is pure.
  • generation_trust_set(root, n) returns generation n while active names another generation, refuses n's epoch disagreement and malformed document as active_trust_set would, and reports a missing gen-<n> as Io with kind NotFound. For the active generation it equals active_trust_set's result.
  • parse_generation_name accepts exactly the names the generation engine writes.
  • TrustSet::is_withdrawn and TrustSet::epoch are public and answer from the set's own withdrawn list and epoch.
  • Every existing test passes unchanged: verify_package, verify_contents, the preparation and finalization paths and active_trust_set keep their verdicts, order and limits. VerifyError's top-level variants are unchanged.
  • No new dependency. Everything added is documented as the crate's rustdoc rules require, including # Errors.
  • cargo fmt -- --check --config group_imports=StdExternalCrate, both clippy invocations and both cargo test invocations in AGENTS.md pass.

Constraints

  • Signature first. No step of verify_upload reads the manifest block's contents, the archive or any member in a way that decides anything before verify_signature succeeds. Never call parse_unverified_manifest from the new code.
  • Extend, never fork: one authenticate_manifest, one check_statements, one post-authentication body shared with verify_contents, one generation-directory reader shared with active_trust_set.
  • Consult no package-id registry and name no product. Deciding which components a store accepts is the caller's job; ReservedTarget refuses only the crate's own TRUST_TARGET.
  • Change no default in ContentLimits, and no existing verdict, variant name or order.
  • The directory trust policy for staging_parent is unchanged.
  • No CHANGELOG.md entry: the crate has no release.

Out of scope

  • Any change in review. aicers/review#2060 adopts this by pinning the merge revision; how review sizes its staging reservation and which limits it lowers are its decisions.
  • Retaining fewer bytes during verification (for example hashing native members without a snapshot). The bound in item 4 describes the pipeline as it is.
  • Changing active_trust_set, the generation engine or any admission path in release_trust.

Test plan

Use the crate's existing signed fixtures (src/package/contents/tests.rs, src/trust_fixture.rs, prepare_sign_finalize) and the synthetic image builder. Every test uses tempfile::tempdir().

  • Signature first. For each of the four signature failures, pair it with a manifest that would otherwise be NoArtifacts, ReservedTarget, TargetMismatch and ArchitectureMismatch; the signature verdict wins every time.
  • Parity. Table-driven over the native, v6 image and legacy-image fixtures and each refusal fixture the verify_contents tests already build: verify_upload and verify_contents (true build, namespace, true architecture, RetainedDisk set as in the criterion) agree, variant for variant.
  • Derived identity. A native and a v6 fixture per architecture: all accessors equal the fixture's build, arch, signing key, raw-manifest digest and package digest and length. A container whose hint names another anchor that does not verify, and one with an unusable hint, still report the anchor that did.
  • Upload refusals. An empty-artifact manifest, a trust-generation container signed by a trust-set anchor, a two-build manifest and a two-architecture manifest, each signed validly.
  • Namespace. An image declaring clumit-other under UploadRequest::new("clumit-security") is refused; the same package under its own namespace passes. UploadRequest::new("") and new("a/b") are refused.
  • Staging. After each success and each refusal above, list staging_parent: unchanged. Assert the scope's high-water mark against upload_staging_bound for the largest image fixture. Lower RetainedDisk to one byte below that fixture's high-water mark: LimitExceeded naming RetainedDisk. A group-writable staging_parent is refused as InspectStagingParent.
  • Bound. upload_staging_bound at package_len 0, 1 and u64::MAX, with defaults and with lowered OuterUncompressedTotal and RetainedDisk.
  • Generation reader. Seed a tree with admit_seed_generation, add one with replace_generation, then read each generation by number: both answer their own epoch and withdrawn list, whichever active names. Remove a generation: Io with NotFound. Corrupt epoch and the document in a generation: the same refusals active_trust_set gives.
  • Names. parse_generation_name over gen-1, gen-42, gen-0, gen-01, gen-+1, gen-1.tmp, gen-, active, gen-retention.tmp.
  • Accessors. is_withdrawn true for a listed triple and false when any part differs; epoch equals the constructor argument.
  • Unchanged behaviour. The full existing suite, under both feature configurations.

How this is adopted

aicers/review#2060 adds deploy-core as a git dependency at the commit that merges this issue, uses verify_upload and upload_staging_bound for its upload verifier, generation_trust_set and parse_generation_name for its trust source, and TrustSet::is_withdrawn/epoch behind its TrustGeneration. Nothing else here waits on review.

Pointers (deploy-core 19fef3fdd92856d8dc3edc9b2efe2673dcdbf507)

  • src/verify.rs: module doc (verification order); TrustSet :353, private is_withdrawn :402; VerifyRequest :426, for_namespaced_package :481; VerifyError :672; ImageVerifyError :853; verify_package :1410; BoundedVerified and verify_package_bounded :1429–:1495; authenticate :1497; check_statements :1548; verify_signature :1661; check_target :1894.
  • src/package/contents.rs: verify_contents :88; verify_retained :151; check_contents :222; ContentError :526; IoOperation :619; VerifiedContents :743.
  • src/package.rs: module doc (pipeline and resource policy); LimitResource table :750–:817 (RetainedDisk :811, OuterUncompressedTotal); ContentLimits :872, with_limit :903.
  • src/retain.rs: module doc; RetentionScope and its budget; src/retain/trusted_dir.rs (open_trusted_dir).
  • src/module_spec.rs: ModuleSpec :43, RegistrationTemplate :266, ReloadSpec :293.
  • src/release_trust.rs: module doc; active_trust_set :776; read_active_generation :817; active_generation_index :880; admit_seed_generation :1257; replace_generation :1330.
  • src/generation.rs: module doc (tree shape, canonical names); parse_generation :427; swap_active_symlink :471.
  • src/trust_set.rs: read_trust_set_document :549.
  • src/payload.rs: read_package_container, Payload::parse_unverified_manifest :1833.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions