Skip to content

feat(task)!: track written ranges separately from received ranges - #193

Draft
greenhat616 wants to merge 1 commit into
feat/hardening-30-default-doublingfrom
feat/hardening-31-written-offset
Draft

greenhat616 wants to merge 1 commit into
feat/hardening-30-default-doublingfrom
feat/hardening-31-written-offset

Conversation

@greenhat616

@greenhat616 greenhat616 commented Nov 8, 2026 •

Copy link
Copy Markdown
Owner

C18 (T19), the first commit of PR-9 (resume) in the download strategy hardening stack. It is stacked on #192, and that branch is its base. The design notes are in §11.1 and §14.21.

Why

To resume, the task must know which bytes are in the file, not just which bytes it received. Today progress runs ahead of the write: handle_data_frame updates progress, then the loop writes. T19 adds a written state that advances only on WriteCompletion. The durable state that advances after fsync (§11.1) belongs with the state file in C20.

What

  • WrittenSet in ChunkPlanner. A sorted set of ranges that never overlap and are never adjacent. note_written(range) merges a completed write in, and empty ranges are ignored.
    • It advances only in DownloadLoop::record_completions.
    • A debug_assert checks that written ⊆ downloaded.
  • A set, not the per-chunk offset of §11.1. PendingWriter keeps up to max_connections writes in flight, so the writes of one chunk complete out of order. WriteCompletion carries only the range.
  • settle_completions replaces check_completions. Every successful completion is handed over, and the first failure, in the order received, is returned as TaskError::Write. Successful writes after it still count.
  • Progress.
    • publish_progress reports the written ranges.
    • finish() takes written before dropping the manager and adds the completions of the final writer.finish() flush, whatever the outcome. A completed task still reports [0..total].
  • RunnerManager::get_downloaded_ranges is removed, since nothing calls it any more.

How the written set interacts with the rest

  • Splits happen at or after the split runner's downloaded.end, so no written byte lies past a split point.
  • Hedges (§9.6): the loser's frames, late ones included, are dropped before the planner and the writer.
  • Restarts and stops release the range from downloaded.end. Earlier writes still in flight are recorded when they complete, and the retry does not overlap them.
  • Chunk merges and cooldown claims do not affect a range set.
  • The fallback to a single connection drops the whole concurrent attempt, its planner and written included. Received bytes never shrink inside a planner, so there is no other reset. A real reset test goes with C20's state file.

Breaking change

Progress.downloaded_chunks of a concurrent task changes meaning, from the ranges received to the ranges whose writes completed successfully before teardown. No signature changes, so consumers see different values without a compile error.

  • Migration: use Progress.downloaded for the received byte total, and read downloaded_chunks as the confirmed written ranges. This is documented on Progress and in §14.21.
  • What the ranges do not promise: a failed write may leave part of its bytes in the file without reporting them. Nothing is fsynced. A failed or cancelled task deletes its temporary file.
  • Singleton: the single-connection task is unchanged, 0..downloaded.

Tests

  • chunk_planner.rs:

    • merging in any order, including duplicates and empty ranges;
    • written lagging behind downloaded, including a completion after a split, a failure and a backoff.
  • completion.rs: the writes before and after a failed write are recorded, and the failed one is not.

  • download_loop/tests/written.rs:

    • published progress shows 64 KiB received and nothing written until the write completes;
    • the final flush counts its writes;
    • a cancelled task counts its flushed writes;
    • a failed task keeps its transfer error over a flush with mixed results;
    • with two failed writes, the error is the first one to complete, not the lowest offset.
  • hedge.rs: the harness checks, in every hedge test, that the planner's written set equals the recording writer's ranges.

  • status_contract.rs: test_a_fallback_starts_with_nothing_written.

  • test_custom_timer_survives_other_events wrote to the writer without the planner accepting the bytes, which the new assert rejects. Its bytes now go through the planner, and its assertions are unchanged.

  • Mutations, all caught:

    • written advanced on a failed write;
    • written recorded at accept time;
    • the final flush not counted;
    • successful writes after a failure dropped;
    • published progress showing received bytes.

    A "missing reset on fallback" mutation has no code to break: the reset is structural.

Review

Reviewed with /ccg:review, using the codex backend only (a single-model review).

  1. Round 1 (88). It confirmed the implementation: the set, its interactions, the fallback as the only reset, and the error precedence. Its findings:
    • Major: a public field changes meaning, so the commit must be feat(task)!, per tasks.md §0 rule 3. The type was changed and a migration note added.
    • Minor: the guarantee was narrowed to "completed before teardown", and the tests for unsuccessful outcomes were added.
  2. Round 2: PASS (99/100). Its suggestion, to say "first in the order received" in the settle_completions doc, is applied.

Checks

All on +nightly-2026-11-06. Clippy allows unnecessary_lazy_evaluations until #183 merges.

  • cargo fmt --all -- --check: passes.
  • The 4 clippy configs with -D warnings: pass.
  • cargo test --workspace --all-features --locked: 25 suites, 747 tests pass (bolt-load lib 571).
  • cargo test --workspace --locked: 25 suites, 697 tests pass (lib 540).
  • Two tests were skipped in those runs:
  • RUSTDOCFLAGS=-Dwarnings cargo doc --no-deps --all-features -p bolt-load: the same 18 errors as its base.

The concurrent task now keeps the ranges whose writes completed apart
from the ranges it received. A merged range set in the chunk planner
grows only on successful write completions, which can arrive out of
order. Every successful completion in a batch is recorded before the
first failure is returned, and the writes that complete during the
final flush count on every outcome.

Progress.downloaded_chunks reports the written ranges, so it lags
behind while writes are pending, and a failed or cancelled task reports
what was written before its teardown. Progress.downloaded stays the
bytes received. The single-connection task is unchanged.

BREAKING CHANGE: Progress.downloaded_chunks of a concurrent task now
holds the ranges whose writes completed, not the ranges received. Use
Progress.downloaded for the received byte total.
@greenhat616
greenhat616 force-pushed the feat/hardening-31-written-offset branch from de585a4 to 2015b1c Compare November 13, 2026 22:26
@greenhat616
greenhat616 removed this pull request from stack #190 November 13, 2026 22:28
@greenhat616
greenhat616 added this pull request to stack #215 November 13, 2026 22:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant