Repository navigation
fix(task)!: lock a destination for the whole task - #201
Draft
greenhat616 wants to merge 1 commit into
Draft
greenhat616 wants to merge 1 commit into
greenhat616 wants to merge 1 commit into
Conversation
greenhat616
force-pushed
the
fix/hardening-37-destination-lock
branch
from
November 13, 2026 07:44
29a7c03 to
c115b91
Compare
greenhat616
marked this pull request as draft
November 13, 2026 08:55
Every task now takes an exclusive advisory lock (fs4) on <dest>.boltload before it touches the temporary file, and holds it until teardown is done. A second task on the same destination, in this process or another, fails with DownloadError::InUse instead of sharing the .downloading file (#152). The lock is checked against the path's file ID after locking, and a lock file left by an earlier task is emptied through the locked handle before the download starts. The lock lives outside the attempt's unwind boundary, with the temporary file and the attempt's phase. After a panic, the task cancels its workers, waits until every worker that holds the temporary file has closed it, cleans up as the phase requires, then releases the lock. The singleton file is now written on blocking threads that the panic handler can wait for, and every writer worker drops its file before its drain clone on every path. BREAKING CHANGE: every task creates <dest>.boltload while it runs and removes it when it ends. A second task on a destination in use fails with the new DownloadError::InUse, and a failure to create or lock the lock file is DownloadError::Io with the new IoOp::Lock; an unwritable directory used to report Io with IoOp::Create on the temporary file. On a file system that cannot lock files, the task warns, goes on unlocked, and leaves the empty lock file behind. Closes #152
greenhat616
force-pushed
the
fix/hardening-37-destination-lock
branch
from
November 13, 2026 22:26
c115b91 to
4517da6
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
C31 (T21), the first implementation commit of the resume work in the download strategy hardening stack. It is stacked on #200 (the resume design), which is its base branch. The design notes are in §11.5.4, §11.5.8, §11.5.9, §11.5.13 and §14.26, with task-api-spec TASK-8 and §4.3.
Fixes #152.
Why
Two tasks on the same destination shared one
.downloadingtemporary file (#152). Resume also needs one owner per destination: the state file and the partial file must not change under a task. C31 adds that lock first, for every task, before any resume code exists. Every task isResume::Offuntil C20.What
The lock (
task/instance/destination_lock.rs)fs41.1.0 (default-features = false, features = ["sync"]).FileExt::try_lockis called explicitly: std'stry_lockneeds Rust 1.89, and the MSRV is 1.88.same-fileandlibc(unix) become direct dependencies; both were already in the lock file.<dest>.boltloadis opened withcreate_new, else the existing file, andcreatedis recorded. It retries onNotFound, and on Windows on a sharing violation or a delete-pending file.ERROR_ACCESS_DENIED. That is told from a real denial by whether the path's metadata can still be read.Io { Lock }, notInUse.DownloadError::InUse { path: dest }.Io { op: Lock }.same_file::Handlechecks that the path still names the locked file. On a mismatch it unlocks and retries, at most 3 times, then reportsInUse. An error in the check itself isIo { Lock }, not contention.set_len(0)andsync_all) through the locked handle, before the temporary file is touched.Ownership outside the unwind (
task/instance/ownership.rs)drive::spawnkeeps anOwnershipoutsidecatch_unwind. It holds the lock, the temporary path, the phase (Unowned, Transfer, Published: the design's A, D, P), the completed progress once published, and a drainWaitGroup.blocking::unblockand is not raced against cancellation. A cancellation seen after the lock discards the owned temporary file, then releases.releaseruns after teardown and before the terminal status. It sets the phase to Unowned and drops the temporary path under the same mutex, then releases the lock. So a later panic cleanup cannot remove a successor's file.report_paniccancels, waits for the drain without a bound (design Q2), then cleans up by phase:leftover_temp_file;Finishedwith the stored completed progress, otherwiseFailed(Panicked).async_fs::Filehanded writes to a background worker nobody could wait for. It is replaced byDrainedFile: each operation is its own blocking task holding the file and a drain clone, and writes no longer buffer.Drained<T> { value, _drain }puts a file and its drain clone in one struct, so the file closes before the drain is told on every path: a return, a worker panic, or a queued task dropped without running. Struct fields drop in declaration order; separate closure captures do not have a specified order.StatusPublisherno longer tracks the temporary file;Ownershipdoes.Breaking change
<dest>.boltloadwhile it runs and removes it when it ends.DownloadError::InUse.DownloadError::Iowith the newIoOp::Lock.Io { Create }on the temporary file; it now reportsIo { Lock }on the lock file.#[non_exhaustive].The README,
task.md, task-api-spec and §14.26 document this.Tests
destination_lock.rs: contention; a stale file; a crash leftover; a file replaced once and every time; unsupported locking; a failed lock that must not remove the holder's file (barrier-ordered); identity errors; the Windows access-denied classification; running out on a denial.ownership.rs: the lock is held until the workers exit; cleanup by phase; cleanup after release touches nothing.drained_file.rs: a dropped in-flight write holds the drain;Draineddrop order on a panic and on a closure dropped unrun.file_writer.rs: every writer kind holds the drain untilfinalize.drive.rs: panics before and after the rename.task/tests/destination_lock.rs. The first task is held behind an explicit gate, with a one-hour stream idle timeout, so nothing depends on timing.InUsein both modes, and the first completes intact.tests/destination_lock.rs, across processes. The test binary re-spawns itself as a child that holds the destination until the parent closes its stdin.InUse, bounded at 60 s.releasekeeping the temporary file owned.Drained's fields swapped.InUse.Not covered: a real Windows ACL denial and a real delete-pending file. The tests check the classification with synthesized error codes only.
Review
Reviewed with
/ccg:review, using the codex backend only (a single-model review).InUse;Drained<T>.InUse;releasemust not run concurrently.Checks
All on
+nightly-2026-11-06, with--no-fail-fast. Clippy allowsunnecessary_lazy_evaluationsuntil #183 merges.cargo fmt --all -- --check.-D warnings.cargo test --workspace --all-features --locked: 28 suites, 812 passed.cargo test --workspace --locked: 28 suites, 744 passed.RUSTDOCFLAGS=-Dwarnings cargo doc --no-deps --all-features -p bolt-load: the same 18 errors as its base.