Repository navigation
feat(task): keep a state file for resumable downloads - #205
Draft
greenhat616 wants to merge 1 commit into
Draft
greenhat616 wants to merge 1 commit into
greenhat616 wants to merge 1 commit into
Conversation
greenhat616
force-pushed
the
feat/hardening-41-resumable-state
branch
from
November 13, 2026 07:44
8d38f5e to
aee9caa
Compare
greenhat616
marked this pull request as draft
November 13, 2026 08:55
A task with resume on, which stays crate-internal until resume reads the file back, now keeps the state file of C33a next to its partial file whenever it can resume: concurrent mode, a usable If-Range validator, and a header that encodes. It needs the destination lock, and fails where the file system cannot lock files. A new start invalidates any old state, creates, sizes and syncs the partial file, writes and syncs the header, then syncs the directory. Once the header is synced the pair is resumable. Cancellation is checked between these steps, never raced against one. How the task ends depends on its phase: - Success renames the partial file, then empties, removes and syncs the state file. A failure after the rename only warns, and the task stays Finished. - A failure, a cancellation, or a failed rename keeps both files and reports the partial file as kept for resume. - A changed resource discards both files. The state is invalidated first; if that fails, the partial file is kept as a leftover. - A fallback to one connection invalidates the state first, and does not fall back if that fails. - A panic cleans up by phase. A state already cut, but not yet synced, is never reported as resumable. Tasks with resume off, and tasks that cannot resume, take none of these steps and end exactly as before.
greenhat616
force-pushed
the
feat/hardening-41-resumable-state
branch
from
November 13, 2026 22:25
aee9caa to
52ee08a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
C33b (T21), the fourth commit of PR-9d (resume) in the download strategy hardening stack. It is stacked on #204 (C33a, the state file format and writer), its base branch.
The implementer proposed, and the lead agreed, to move two parts to a small C33c:
Onwith a preferred single connection through the concurrent engine (§11.5.8);discard_partial.The design notes are in §11.5.3–§11.5.9 and §14.29.2.
What
Resume::{Off, On}stays crate-internal, and tests switch it on. It becomes public in C20, together with reading the state file back. The same holds forkept_for_resume: a lead decision keeps both internal until then.When a task keeps a state file (§11.5.3)
A task keeps a state file when all of these hold:
On;meta.validator(C34);Any other
Ontask behaves exactly likeOff, apart from the lock. A test comparesOnandOfffield by field for:Each case runs as a success and as a failure.
Onlocks withUnsupported::Fail(Q3).OffkeepsProceed.New start (§11.5.5)
The steps run in Initializing, before
infois published:set_len(total),sync_all.sync_all.StateFilebecomes C32's checkpoint receiver.kept_for_resume.End of task, by phase (§11.5.9)
Finished.kept_for_resume= the partial file.ResourceChangedset_len(0)+ sync) → remove the partial file → directory sync → remove the state file → directory sync.Invalidation failures
An invalidation is two steps, a truncate and a sync, and each can fail on its own. The result is
Done,IntactorUnsynced, and a separatekeepflag decides which files stay.Io { Lock }and keeps both files withkept_for_resume. A discard keeps the partial file.leftover_temp_file, andkept_for_resumeisNone.cutflag is recorded before the sync, so panic cleanup also reportsleftover_temp_file, neverkept_for_resume.Known limitation
Windows has no directory sync. It relies on NTFS metadata journaling (§11.5.5).
Not breaking
Offtakes none of the new steps. It keeps the same lock mode, commit and release, has no checkpoint receiver, and changes no error. A test checks thatOffconcurrent,Offsingleton,Onwithout a validator andOnwith a single connection take zero file steps and leave only the destination. Every earlier test passes unmodified.Tests
The tests run real tasks on real files with fault hooks (
StepLog), which log every §11.5.5 step and can fail it, gate it or panic at it.ResourceChanged.Ontable.ResourceChanged.InUse.Onthat cannot resume equalsOff.Mutations: 19 of 19 caught. They include:
Review
Reviewed with
/ccg:review, using the codex backend only (a single-model review).Round 1 (82). Three Majors:
All three are fixed, together with the suggestion to test panics through real tasks.
Round 2 (90). One Major: a panic between the truncate and the sync of an invalidation. Fixed with the
cutflag. The B1 tests asked for in its suggestion are added.Round 3: PASS (98/100). Its suggestion, a test of B1's retried invalidation after a panic, is applied.
Checks
All on
+nightly-2026-11-06, with--no-fail-fast. Clippy allowsunnecessary_lazy_evaluationsuntil #183 merges.cargo fmt --all -- --check.-D warnings.cargo test --workspace --all-features --locked: 28 suites, 886 passed, 1 ignored (the benchmark).cargo test --workspace --locked: 28 suites, 815 passed, 1 ignored.RUSTDOCFLAGS=-Dwarnings cargo doc --no-deps --all-features -p bolt-load: the same 18 errors as its base. It was last run in round 2; round 3 changed only a test anddesign.md.