Skip to content

feat(task): resume from a checked state file - #209

Draft
greenhat616 wants to merge 1 commit into
feat/hardening-43-resumefrom
feat/hardening-44-resume-engine
Draft

greenhat616 wants to merge 1 commit into
feat/hardening-43-resumefrom
feat/hardening-44-resume-engine

Conversation

@greenhat616

Copy link
Copy Markdown
Owner

C20b (T21), the second of three commits of C20 (resume) and the seventh commit of PR-9d in the download strategy hardening stack. It is stacked on #208 (C20a, the reader and checks), its base branch. C20c will make Resume public, with docs and end-to-end tests. The design notes are in §11.5.5, §11.5.9, §11.5.10 and §14.30.2.

What

Public result types (lead decision)

DownloadError is public, so its new ResumeRejected variant cannot carry a crate-private reason. The same holds for TaskInfo::resumed. So these are public now:

  • ResumeReport, DroppedRecord, DropReason and RejectReason, re-exported from bolt_load::task;
  • DownloadError::ResumeRejected { reason, state_file }, shown as cannot resume from <state file>: <reason>;
  • TaskInfo::{resumable, resumed};
  • IoOp::Read.

They are inert until C20c: resumed is always None, resumable is always false, and no task fails with ResumeRejected. Their docs say so. The alternative, internal stand-ins swapped out in C20c, was rejected as throwaway plumbing.

When a task resumes

An On task whose state file existed before it took the lock and is not empty enters a new phase, Checking (B2). B2 changes neither file. Any other task starts as before, and an empty state file means a new start.

Resume order (§11.5.5, §11.5.10)

Cancellation is checked between steps and is never raced against a file step.

  1. A missing partial file is ResumeRejected(MissingPartial).
  2. The state file is read through the locked handle. BadHeader and Oversized are rejections; a read error is Io { Read }.
  3. The adapter fetches its metadata again, the task checks range support, and the header is compared with the result.
    • All three race cancellation, since they are network only.
    • The adapter's metadata is used even when the caller passed .meta(...). This is the implementer's reading of §14.27, agreed by the lead.
    • A mismatch is a rejection.
  4. The task verifies every kept block's CRC against the partial file, through its own read-only handle. A read error is Io { Read } with the block's range; one warn! is logged per dropped record.
  5. The task enters phase C. Only now does it change the state file: it writes the cleaned records back (Io { Write }), then sets the partial file's size (Io { Allocate }). Making the sizing a separate step is the implementer's decision; it shows in tests that the write-back comes first.
  6. The task runs in concurrent mode, with one connection if it preferred one. The state file becomes the checkpoint receiver.

How B2 ends (§11.5.9)

Outcome in B2 Result
Rejection, failure, cancellation, panic Both files untouched, nothing reported, lock released, state file kept

Phase C is entered before the first change to the state file, so B2 needs no cut flag. Every failure from C on keeps a valid header plus a prefix of verified records, and reports kept_for_resume.

Seeding

  • The kept ranges become finished chunks in the planner and finished written ranges, so only the gaps are requested.
  • Progress starts at the kept bytes (STATUS-12).
  • A fully kept file makes no request.
  • The kept blocks are also marked as recorded in the checkpoints, as §11.5.10 asks for persisted blocks.

Info and fallback

resumable is true when the task keeps a state file, and resumed holds the report. A fallback to one connection clears both. The report moves into the published info once; snapshots share it through the existing Arc<TaskInfo> and never copy it.

Docs

task-api-spec §4.3 step 5, TASK-8 and the history notes describe B2 and the comparison against the adapter's own metadata.

Not breaking

  • The four new types are #[non_exhaustive]. DownloadError, IoOp and TaskInfo already were.
  • Outside the library, these types appear only in bolt-load's integration tests, which use matches!.
  • Resume stays crate-internal, so no user task takes any new path.

Tests

task/tests/resume_from.rs runs real tasks on real files with step hooks. It covers:

  • A normal resume: the exact step order; only the missing ranges requested; the report; progress that starts exactly at the kept bytes.
  • A fully kept pair: no request.
  • Nine rejections: the partial file missing, a bad header, an oversized file, the size, the ETag, the validator kind, no validator, content coding, no range support. Each leaves both files byte-identical, reports nothing, makes no request, and frees the lock.
  • The adapter's metadata wins over the caller's, in both directions.
  • Six spoiled cases: a CRC mismatch, a block past the end of the file, a torn tail, a bad record, a conflict, out of bounds. Each checks the written-back state, the dropped list, the re-download and the final content.
  • Failures: every resume step failing, by phase.
  • Cancellation: cancelling during the state read, during the block checks (first and last block), during the write-back, while taking the lock, and during a stalled range check.
  • Panics in B2 and in C.
  • Repeated resumes: a block added on the second attempt is kept on the third and never requested again.
  • A fallback after a resume.
  • A single-connection resume, with one range request and a peak of one connection.
  • The §11.5.5 "next On" table.
  • The partial file longer or shorter than the total.
  • The full rejection message.

Mutations: 26 run, 25 caught. The one survivor drops the marking of kept blocks as recorded, and it is equivalent at the task level: kept blocks are never written, so they can never be recorded twice. A unit test covers the method itself.

Review

Reviewed with /ccg:review, using the codex backend only (a single-model review).

  1. Round 1 (86).
    • Major: the range check after retrieve_meta did not race cancellation, so a stalled adapter could leave a cancelled task holding the lock. It is now in the same cancellable block, with a test.
    • Major: info.clone() deep-copied the report. The info now moves once.
    • Minor (an exact progress test) and the Suggestion (repeated resume adds a block) are applied.
  2. Round 2: PASS (96/100). Its Minor is fixed: the initial-progress test is now independent of scheduling, through a test handshake.

Checks

All on +nightly-2026-11-06, with --no-fail-fast. Clippy allows unnecessary_lazy_evaluations until #183 merges.

  • cargo fmt --all -- --check.
  • 4 clippy configs with -D warnings.
  • cargo test --workspace --all-features --locked: 28 suites, 954 passed, 1 ignored (the benchmark).
  • cargo test --workspace --locked: 28 suites, 883 passed, 1 ignored.
  • RUSTDOCFLAGS=-Dwarnings cargo doc --no-deps --all-features -p bolt-load: the same 18 errors as its base.

@greenhat616
greenhat616 added this pull request to stack #212 November 11, 2026 15:23
@greenhat616
greenhat616 removed this pull request from stack #212 November 11, 2026 15:25
@greenhat616
greenhat616 added this pull request to stack #190 November 11, 2026 15:25
@greenhat616
greenhat616 force-pushed the feat/hardening-44-resume-engine branch from d15eecd to 56d128a Compare November 13, 2026 07:44
@greenhat616
greenhat616 marked this pull request as draft November 13, 2026 08:55
A task with resume on, still crate-internal until the next change, now
resumes when it finds a state file it did not just create. It checks
the pair before it changes anything: the partial file must exist, the
state file must read cleanly, and the header must match the metadata
the adapter fetches again for this task. Then it verifies the CRC of
every kept block against the partial file.

A pair that cannot resume ends the task with ResumeRejected and leaves
both files exactly as they were. Otherwise the task writes the cleaned
records back, sizes the partial file, and downloads only the ranges
that are missing. Progress starts at the kept bytes, and a fully kept
file makes no request at all.

The result types are public now, because a public error and TaskInfo
cannot carry crate-private ones: ResumeReport, DroppedRecord,
DropReason, RejectReason, DownloadError::ResumeRejected,
TaskInfo::{resumable, resumed} and IoOp::Read. Every one is added to a
non-exhaustive type, and none can occur until resuming can be switched
on.
@greenhat616
greenhat616 force-pushed the feat/hardening-44-resume-engine branch from 56d128a to 789fc2c Compare November 13, 2026 22:25
@greenhat616
greenhat616 removed this pull request from stack #190 November 13, 2026 22:28
@greenhat616
greenhat616 added this pull request to stack #215 November 13, 2026 22:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant