Skip to content

Expose a read container's raw blocks for hashing (#131) - #134

Merged
sehkone merged 3 commits into
mainfrom
sehkone/issue-131
Sep 27, 2026
Merged

sehkone merged 3 commits into
mainfrom
sehkone/issue-131

Conversation

@sehkone

@sehkone sehkone commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

Summary

This PR makes the container that read_package_container returns hand out its raw blocks, so a caller can hash them. bootler#412 needs this to fingerprint the exact installer payload an install attempt ran from. With these accessors, bootler doesn't need a second footer parser of its own. The blocks are only hashed, never parsed. #72's rule that manifest decoding must go through this crate's parse still holds.

New public API on payload::UnparsedContainer

  • container_version() returns the version recorded in the footer: 1, or FORMAT_VERSION (currently 2). This is not the manifest's format_version, and no manifest parse is needed to get it. It replaces the crate-internal footer_version().
  • raw_manifest_block() returns the manifest block exactly as it sits in the container. It is unparsed, unauthenticated and never re-serialized. It replaces the crate-internal manifest_bytes(). Its rustdoc says to decode through parse_unverified_manifest or verify::verify_package, never with serde_json directly.
  • archive_block_len() returns the length of the compressed archive block as the validated footer records it.
  • raw_archive_block() seeks the source to the start of the block once and returns a RawArchiveBlock.

RawArchiveBlock is new and public. It implements Read only, with no Seek, and borrows the container's source. It yields exactly archive_block_len() bytes and then EOF. It never reads outside the block, never holds the block in memory and never decompresses it. If the source ends before the recorded length, read returns io::ErrorKind::UnexpectedEof instead of a short stream that looks successful. If the source reports reading more bytes than it was asked for, read returns an error rather than trusting it.

The accessors work whatever state the envelope blocks are in (Absent, Present or WrongLength) and whether or not the manifest would parse. Rejecting either case is left to the caller.

The rustdoc of read_package_container now states that a file with no trailer is reported as PayloadError::NoTrailer. An installer-side caller uses that error to recognise an empty payload.

Test support

payload::append_version_1_trailer writes a version-1 container fixture: base ‖ manifest ‖ archive ‖ footer, using the 41-byte version-1 footer. It is gated with #[cfg(any(test, feature = "test-support"))], the same as widen_envelope_blocks. With it, dependent crates can test version-1 input through the real reader without building footers by hand. The README and CHANGELOG mention it.

Unchanged

No existing public signature, verdict or verdict order changes. The crate-internal callers in verify.rs, including two of its tests, now call the renamed accessors, and nothing else there changes. verification_never_reads_the_archive_block passes unchanged, because only raw_archive_block reads that block.

Tests

The unit tests are in src/payload/raw_block_tests.rs:

  • A version-1 container reports 1, and append_trailer output reports 2.
  • The raw manifest block is byte-for-byte what the unsigned writer emitted. For append_trailer_signed, it is also exactly what the signer was handed.
  • The raw archive block is what the writer emitted, and it stays the same after rewrap_trailer onto a base of a different length.
  • A recording Read + Seek source shows one seek to the start of the block, then reads of exactly archive_block_len() bytes, with nothing read before or after the block.
  • A source truncated after validation yields UnexpectedEof.
  • A failed seek returns PayloadError::Io.
  • The accessors work when the envelope blocks are WrongLength and when the manifest does not parse.
  • The version-1 writer's output is accepted by read_package_container.
  • A file with no trailer is NoTrailer.

tests/raw_container_blocks.rs repeats the rewrap and version-1 cases through the public API only, the way a dependent crate would use it.

Locally, the full CI matrix passes: fmt, clippy with and without test-support, and the tests with and without test-support.

Closes #131

Deviations from the issue

None

Test plan

  • cargo fmt -- --check --config group_imports=StdExternalCrate passes
  • cargo clippy --all-targets -- -D warnings passes
  • cargo clippy --all-targets --features test-support -- -D warnings passes
  • cargo test passes
  • cargo test --features test-support passes
  • a_version_1_container_reports_1_and_a_current_one_reports_2: a version-1 fixture reports container_version() == 1, and append_trailer output reports 2
  • the_raw_manifest_block_is_the_one_the_unsigned_writer_emitted: raw_manifest_block() equals the unsigned writer's manifest bytes, byte for byte
  • the_raw_manifest_block_is_the_one_the_signer_was_handed: for append_trailer_signed output, raw_manifest_block() equals the bytes the signer was handed
  • the_raw_archive_block_is_the_one_the_writer_emitted_and_survives_a_rewrap: raw_archive_block() yields the writer's archive bytes, and the same bytes after rewrap_trailer onto a base of a different length
  • the_archive_reader_reads_exactly_the_block_and_nothing_around_it: a recording Read + Seek source logs one seek to the block's offset, then reads that cover exactly archive_block_len() bytes and nothing before or after the block
  • a_source_that_ends_inside_the_block_is_unexpected_eof: a source cut short after validation, either at the block's start or inside it, returns io::ErrorKind::UnexpectedEof after delivering only the bytes before the cut
  • a_failed_seek_to_the_archive_block_is_an_io_error: a failing seek returns PayloadError::Io from raw_archive_block()
  • the_accessors_answer_whatever_the_envelope_state: all four accessors work when the envelope blocks are WrongLength
  • the_accessors_answer_for_a_manifest_that_does_not_parse: all four accessors work when the manifest does not parse
  • the_version_1_writer_frames_a_container_the_real_readers_accept: append_version_1_trailer output is accepted by read_package_container
  • a_file_without_a_trailer_is_no_trailer: read_package_container reports a file with no trailer as PayloadError::NoTrailer
  • Integration tests in tests/raw_container_blocks.rs, which use the public API only: the raw blocks stay the same across a rewrap, and a version-1 fixture reads through the real reader
  • The existing verify.rs tests pass; verification_never_reads_the_archive_block is unchanged, and the only edits to the others rename footer_version() / manifest_bytes() calls to container_version() / raw_manifest_block()

bootler binds each recorded install attempt to a fingerprint over the
container's version and its raw manifest, archive and envelope blocks,
and must take those blocks from this crate's reader rather than grow a
second footer parser. UnparsedContainer now hands them out: the footer
version, the manifest block exactly as written, and a bounded reader
over the still-compressed archive block that never leaves the block and
reports a source ending inside it as UnexpectedEof rather than a short
stream. The manifest accessor is documented as unauthenticated and
points decoding at the crate's own parsers, keeping the concern that
kept these bytes private.

A test-support writer for version-1 containers lets a dependent test
version-1 input through the real reader without hand-building footers.

Closes #131
raw_archive_block documents PayloadError::Io when its seek fails, and
nothing exercised that path.

Part of #131
@sehkone

sehkone commented Sep 27, 2026

Copy link
Copy Markdown
Contributor Author

[Reviewer Round 1]

Review verdict: Approve. I found no blocking issues.

The change gives a caller the exact footer version and manifest bytes, plus a bounded reader for the compressed archive block. The reader seeks once, stops at the recorded length, and reports an early end as UnexpectedEof (src/payload.rs, src/payload.rs). The version 1 fixture uses the existing footer encoder, and the tests cover signed and unsigned output, rewrapping, read bounds, truncation, malformed envelopes, and unparsable manifests (raw_block_tests.rs).

The PR body has the required Closes #131 and test plan. Its “Deviations: None” claim matches the diff. I did not rerun CI, as requested.

@sehkone

sehkone commented Sep 27, 2026

Copy link
Copy Markdown
Contributor Author

[Review Verdict Round 1: APPROVED]

@sehkone

sehkone commented Sep 27, 2026

Copy link
Copy Markdown
Contributor Author

Suggested squash commit

Title

Expose a read container's raw blocks for hashing

Body

bootler binds each recorded install attempt to a fingerprint over the
container's version and its raw manifest, archive and envelope blocks,
and must take those blocks from this crate's reader rather than grow a
second footer parser. UnparsedContainer now hands them out: the footer
version, the manifest block exactly as written, and a bounded reader
over the still-compressed archive block that never leaves the block and
reports a source ending inside it as UnexpectedEof rather than a short
stream. The manifest accessor is documented as unauthenticated and
points decoding at the crate's own parsers, keeping the concern that
kept these bytes private.

A test-support writer for version-1 containers lets a dependent test
version-1 input through the real reader without hand-building footers.

Closes #131

@sehkone
sehkone merged commit 3e49e19 into main Sep 27, 2026
5 checks passed
@sehkone
sehkone deleted the sehkone/issue-131 branch September 27, 2026 11:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Expose a read container's raw blocks for hashing

1 participant