Repository navigation
feat(task): read and check a state file - #208
Draft
greenhat616 wants to merge 1 commit into
Draft
greenhat616 wants to merge 1 commit into
greenhat616 wants to merge 1 commit into
Conversation
This was referenced Nov 11, 2026
greenhat616
force-pushed
the
feat/hardening-43-resume
branch
from
November 13, 2026 07:44
a9942bb to
1bbc74c
Compare
greenhat616
marked this pull request as draft
November 13, 2026 08:55
Add the crate-internal pieces that resuming will use, without wiring them into a task yet, so nothing changes behaviour. The reader takes the state file through the locked handle. It refuses a file shorter than any header, reads and checks the header, and rejects a file longer than its header allows: one record per block twice over, plus one. Only then does it stream the records in bounded chunks. A torn tail, a bad record, a record out of bounds and conflicting records of one block are dropped with a reason, following the spec's rules. The checks compare the header with the adapter's metadata, verify each kept block's CRC against the partial file, and build the report of what was kept and what was dropped. A block that cannot be read fails the check rather than being dropped. StateFile can rewrite itself with the cleaned records: cut to the header, sync, write the records in bounded batches, sync. Everything on this path that grows with the number of records allocates fallibly, so a large state file returns an error instead of aborting.
greenhat616
force-pushed
the
feat/hardening-43-resume
branch
from
November 13, 2026 22:26
1bbc74c to
461be9b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
C20a (T21), the first of three commits of C20 (resume) and the sixth commit of PR-9d in the download strategy hardening stack. It is stacked on #206 (C33c), which is its base branch. The design notes are in §11.5.5, §11.5.10, §14.30 and §14.30.1.
The implementer proposed, and the lead agreed, to split C20 into three commits:
Resumeis still internal.What
Only tests call the new code, so nothing changes behaviour.
Report types (
task/resume.rs)ResumeReport,DroppedRecord,DropReasonandRejectReason, as in §11.5.10. They are crate-internal and#[non_exhaustive].RejectReason::Oversizedis new (the implementer's choice): no existing reason fits a file longer than its header allows.Reader (
state_file/read.rs)read_statereads through the locked handle with positional reads and opens no other handle.BadHeader, and nothing is read.BadHeaderfor:Oversized, and no record is read.Checks (
resume_check.rs)compare: UnknownSize, ContentCoded, RangeUnsupported, TotalMismatch, SourceMismatch, NoValidator, ValidatorMismatch, in this order.retrieve_metawhenever a state file exists, even when the caller passed.meta(...). This is the implementer's reading of §14.27, agreed by the lead.verify: checks cancellation before every block; a block past the partial file's end is BeyondFileLength, without a read; a bad CRC is CrcMismatch. A read error fails the check instead of dropping the block (the implementer's decision). Otherwise a transient or permission error would throw away everything already downloaded.report: merges adjacent kept blocks, totals the kept bytes and sorts the dropped records by position.StateFile::rewriteCut to the header, sync, write the records in batches of at most 2730, then sync.
endthen points past them. C32'sappendis unchanged.Allocation
Everything on the resume path that grows with a state file's record or block count allocates fallibly (
try_reserve), and the sorts are unstable and allocation-free. The header buffer, its strings and the chunk buffer have fixed bounds. Engine structures outside C20 are out of scope (a lead decision).Cached metadata (§14.30)
The adapters cache their probe. So on the same adapter instance, the comparison is against the cached probe.
If-Rangecatches a later change only when the resume still issues a range request. If every block is kept, no request is made. Checking against the current resource therefore needs a fresh adapter. C20c's README will say so.Not breaking
Every new item is crate-internal. C20c will recheck this against the final public API.
Tests
u64::MAXtotal), repeats, six conflict orders, and many records of one block keeping file order.u64::MAX.Interruptedon every other call.DestinationLockhandle.compare,verifyandreport:rewrite: the step order, batches, an empty rewrite, each step failing, and a failed batch.Mutations: 51 of 51 caught, among them:
Review
Reviewed with
/ccg:review, using the codex backend only (a single-model review).If-Rangeclaim for a fully kept resume was qualified. Suggestion: added the locked-handle test.verify,reportandrewriteallocated without being able to fail. Now they can fail, andrewriteis batched. The scope rule above is a lead decision. Minor: the wording was corrected. Suggestion:scanis now test-only.verifywith an existing dropped list.Checks
All on
+nightly-2026-11-06, with--no-fail-fast. Clippy allowsunnecessary_lazy_evaluationsuntil #183 merges.cargo fmt --all -- --check.-D warnings.cargo test --workspace --all-features --locked: 28 suites, 934 passed, 1 ignored (the benchmark).cargo test --workspace --locked: 28 suites, 863 passed, 1 ignored.RUSTDOCFLAGS=-Dwarnings cargo doc --no-deps --all-features -p bolt-load: the same 18 errors as its base.