Repository navigation
feat(task)!: track written ranges separately from received ranges - #193
Draft
greenhat616 wants to merge 1 commit into
Draft
greenhat616 wants to merge 1 commit into
greenhat616 wants to merge 1 commit into
Conversation
This was referenced Nov 8, 2026
greenhat616
force-pushed
the
feat/hardening-31-written-offset
branch
from
November 13, 2026 07:44
48357df to
de585a4
Compare
greenhat616
marked this pull request as draft
November 13, 2026 08:55
The concurrent task now keeps the ranges whose writes completed apart from the ranges it received. A merged range set in the chunk planner grows only on successful write completions, which can arrive out of order. Every successful completion in a batch is recorded before the first failure is returned, and the writes that complete during the final flush count on every outcome. Progress.downloaded_chunks reports the written ranges, so it lags behind while writes are pending, and a failed or cancelled task reports what was written before its teardown. Progress.downloaded stays the bytes received. The single-connection task is unchanged. BREAKING CHANGE: Progress.downloaded_chunks of a concurrent task now holds the ranges whose writes completed, not the ranges received. Use Progress.downloaded for the received byte total.
greenhat616
force-pushed
the
feat/hardening-31-written-offset
branch
from
November 13, 2026 22:26
de585a4 to
2015b1c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
C18 (T19), the first commit of PR-9 (resume) in the download strategy hardening stack. It is stacked on #192, and that branch is its base. The design notes are in §11.1 and §14.21.
Why
To resume, the task must know which bytes are in the file, not just which bytes it received. Today progress runs ahead of the write:
handle_data_frameupdates progress, then the loop writes. T19 adds a written state that advances only onWriteCompletion. Thedurablestate that advances after fsync (§11.1) belongs with the state file in C20.What
WrittenSetinChunkPlanner. A sorted set of ranges that never overlap and are never adjacent.note_written(range)merges a completed write in, and empty ranges are ignored.DownloadLoop::record_completions.debug_assertchecks that written ⊆ downloaded.PendingWriterkeeps up tomax_connectionswrites in flight, so the writes of one chunk complete out of order.WriteCompletioncarries only the range.settle_completionsreplacescheck_completions. Every successful completion is handed over, and the first failure, in the order received, is returned asTaskError::Write. Successful writes after it still count.publish_progressreports the written ranges.finish()takeswrittenbefore dropping the manager and adds the completions of the finalwriter.finish()flush, whatever the outcome. A completed task still reports[0..total].RunnerManager::get_downloaded_rangesis removed, since nothing calls it any more.How the written set interacts with the rest
downloaded.end, so no written byte lies past a split point.downloaded.end. Earlier writes still in flight are recorded when they complete, and the retry does not overlap them.writtenincluded. Received bytes never shrink inside a planner, so there is no other reset. A real reset test goes with C20's state file.Breaking change
Progress.downloaded_chunksof a concurrent task changes meaning, from the ranges received to the ranges whose writes completed successfully before teardown. No signature changes, so consumers see different values without a compile error.Progress.downloadedfor the received byte total, and readdownloaded_chunksas the confirmed written ranges. This is documented onProgressand in §14.21.0..downloaded.Tests
chunk_planner.rs:completion.rs: the writes before and after a failed write are recorded, and the failed one is not.download_loop/tests/written.rs:hedge.rs: the harness checks, in every hedge test, that the planner's written set equals the recording writer's ranges.status_contract.rs:test_a_fallback_starts_with_nothing_written.test_custom_timer_survives_other_eventswrote to the writer without the planner accepting the bytes, which the new assert rejects. Its bytes now go through the planner, and its assertions are unchanged.Mutations, all caught:
A "missing reset on fallback" mutation has no code to break: the reset is structural.
Review
Reviewed with
/ccg:review, using the codex backend only (a single-model review).feat(task)!, per tasks.md §0 rule 3. The type was changed and a migration note added.settle_completionsdoc, is applied.Checks
All on
+nightly-2026-11-06. Clippy allowsunnecessary_lazy_evaluationsuntil #183 merges.cargo fmt --all -- --check: passes.-D warnings: pass.cargo test --workspace --all-features --locked: 25 suites, 747 tests pass (bolt-load lib 571).cargo test --workspace --locked: 25 suites, 697 tests pass (lib 540).test_save_into_rejects_hostile_filenames, the known flaky test. It failed in the first full runs because a.DOWNLOADING.tmpwas not yet removed, then passed 5 out of 5 on its own.splits_stay_within_the_limit_and_cover_the_file. The property test found a sequence that trips thedebug_assertinChunkPlanner::mark_failed("the last side of a pair fails, not the first"). The same seed also fails on the base feat(task)!: make doubling the default growth strategy #192 (ea69115), so this PR did not cause it. It is a test-driver bug, not a product bug, and is fixed in test(task): fail runners in the split property test as the loop does #194.RUSTDOCFLAGS=-Dwarnings cargo doc --no-deps --all-features -p bolt-load: the same 18 errors as its base.