Skip to content

feat(core)!: carry validators in adapter meta and send If-Range - #197

Draft
greenhat616 wants to merge 1 commit into
test/hardening-32-proptest-side-failfrom
feat/hardening-33-if-range
Draft

greenhat616 wants to merge 1 commit into
test/hardening-32-proptest-side-failfrom
feat/hardening-33-if-range

Conversation

@greenhat616

Copy link
Copy Markdown
Owner

C19 (T20), the second commit of PR-9 (resume) in the download strategy hardening stack. It is stacked on #194, which is its base branch. The design notes are in §3 (D12–D14), §11.2 and §14.23.

Why

A download must not mix two versions of a resource. Before this PR, the only check was the total in Content-Range, so a same-size replacement between two range requests went unnoticed. T20 adds HTTP validators and If-Range, so the server itself reports a change (RFC 9110 §13.1.5).

Scope: the maintainer's decision

The first version of this PR handled the change inside the engine:

  • it restarted the task once from scratch (D12);
  • it added BoltLoadAdapter::refresh() (D13);
  • it passed the validator with each request (D14).

The first review showed that this does not belong in the engine:

  • refreshing a shared adapter changes other tasks' baselines;
  • the restart path escaped cancellation;
  • whether and how often to restart is a policy decision, which IDM leaves to its UI.

On 2026-11-09 the maintainer moved restart handling above the engine, into a client or job layer. It is tracked in #195, with the full constraints and acceptance criteria. This PR now only detects a change and fails. The engine marks the place with TODO(#195).

What

Core (bolt-load-core)

  • BoltLoadAdapterMeta.etag keeps only a strong ETag, and last_modified keeps the raw value. Both have with_* setters.
  • UnretryableError::ResourceChanged is new.
  • The adapter trait is unchanged.

Validators (bolt-load-http/src/validator.rs)

  • ETag: the full entity-tag grammar (§8.8.3). A list in one field is not an entity tag.
  • Repeated fields: a repeated ETag, Last-Modified or Date counts as missing.
  • If-Range value:
    • a strong ETag;
    • otherwise a Last-Modified at least 1 s older than Date;
    • nothing when the ETag is weak, or when there is no usable date.
  • Comparison: a strong ETag comparison, and dates compared after parsing.

Classification (range.rs)

Response Validator Result
206 conflicts with the one sent ResourceChanged, checked before Content-Range
206 none accepted
200 to a conditional range request different or missing ResourceChanged
200 to a conditional range request the same RangeIgnored (the fallback, as before)
200 without If-Range — RangeIgnored

Adapters

  • ureq2 probes once: a HEAD plus an unconditional bytes=0-0 GET.
    • The validators come from the GET.
    • The total comes from the validated 206's Content-Range, otherwise from the HEAD's Content-Length.
    • The filename and coding come from the HEAD.
  • reqwest: on its HEAD fallback path, the validators come from the probe response and never from the HEAD.
  • Both send If-Range on every range request when they have a validator.

Engine

ResourceChanged, from a plain failure or from a hedge side, ends the task with DownloadError::ResourceChanged:

  • before the fallback check, so it never takes the fallback;
  • after the usual teardown, which removes the temp file.

Test server

New RangeFault::VALIDATED faults, 18 in all:

  • Date, ETag and Last-Modified come from one fixed snapshot;
  • If-Range is evaluated per RFC 9110;
  • they cover a change mid-download, a change followed by ignored ranges, 206 parts of another version, weak and recent validators, the date formats, HEAD validators or sizes that differ from the GET, and a rejected probe.

ReplacedAfterShortRange keeps no validators, so the fallback stays covered.

Breaking change

  • What changes: range requests now carry If-Range. A resource that changes during a download fails the task with DownloadError::ResourceChanged, where it used to fall back or complete.
  • Retrying: to get the new version, download again with a fresh adapter. An adapter probes once and keeps its validator, and a cloned adapter or builder shares it. So a later task on the same adapter fails again at once. a_later_task_on_the_same_adapter_keeps_the_old_validator records this behaviour, which Restart a download whose resource changed, above the task engine #195 must change.

Known limitations

  • Detection needs a probe that succeeded and carried a validator. If the bytes=0-0 probe fails or is rejected, or there is no usable validator, ranges carry no If-Range for the adapter's lifetime.
  • No detection in the single-connection fallback, which sends no If-Range, or with custom adapters that send none.
  • A same-size replacement on a server without validators is still undetected. The C20 resume work may add a content sample (§11.4, item 5).

Tests

  • Unit:

    • validator.rs: the grammar, repeated fields, the date rules (equal, exactly 1 s, future, missing Date, three HTTP-date formats), and the weak ETag;
    • range.rs: the classification;
    • both adapters: crafted responses;
    • runner.rs: a changed resource ends the mode without a fallback.
  • bolt-load-adapter/tests/if_range.rs, both adapters. It covers:

    • a strong ETag sent;
    • weak and recent validators;
    • HEAD validators that differ from the GET;
    • a 206 conflict;
    • a same-validator 200;
    • the total coming from the validated 206;
    • a rejected probe.
  • bolt-load/tests/resource_changed.rs, all three clients. It covers:

    • a change mid-download: ResourceChanged, with no file and no temp file left;
    • a change followed by ignored ranges: ResourceChanged, with no fallback;
    • parts of another version, by ETag and by date: ResourceChanged, while parts without validators complete;
    • ignored ranges with the same validator: the fallback;
    • HEAD and GET validators that differ on a stable server: completes;
    • a rejected probe: completes with unconditional ranges;
    • two tasks sharing an adapter: both send the original validator;
    • a later task on the same adapter: keeps the old validator.

    Every data range's If-Range is asserted.

  • Mutations, all 11 caught:

    • a weak ETag sent;
    • a same-validator 200 taken as a change;
    • a 206 conflict ignored;
    • ureq2 using the HEAD validators;
    • a weak ETag falling back to the date;
    • ResourceChanged taking the fallback;
    • reqwest's HEAD path using the HEAD validators;
    • a validator-less 206 taken as a conflict;
    • dates compared as strings;
    • the grammar accepting any opaque tag;
    • the first of repeated fields taken.

Moved to #195

  • the in-engine restart (D12);
  • refresh (D13);
  • the per-request validator (D14);
  • T20's "file replaced → fresh download, hash matches" verification;
  • C18's deferred "no stale written ranges after a reset" test.

Review

Reviewed with /ccg:review, using the codex backend only (a single-model review).

  1. Round 1 (64).
    • Four Majors:
      • a 206 with a conflicting validator was accepted;
      • refresh changed other tasks' baselines;
      • HEAD-only validators caused a ureq2 regression;
      • the restart path escaped cancellation.
    • Minors: the entity-tag grammar, a racy zero-progress assertion, and the clock sampling in the date test.
    • After the review, the maintainer rescoped the PR (above). Every remaining item was fixed; the restart, refresh and cancellation items no longer apply.
  2. Round 2: PASS (90/100). Its two Minors and two suggestions are applied:
    • the fresh-adapter retry guidance;
    • the failed-probe limitation and its test;
    • the probe-total, shared-adapter and later-task tests, with every If-Range asserted;
    • two wording fixes in the design notes.

Checks

All on +nightly-2026-11-06, with --no-fail-fast. Clippy allows unnecessary_lazy_evaluations until #183 merges.

  • cargo fmt --all -- --check.
  • 4 clippy configs with -D warnings.
  • cargo test --workspace --all-features --locked: 27 suites, 782 passed.
  • cargo test --workspace --locked: 27 suites, 714 passed.
  • RUSTDOCFLAGS=-Dwarnings cargo doc --no-deps --all-features -p bolt-load: the same 18 errors as its base.

The test server's existing port race (it picks a port, then binds it, giving os error 10048) failed some earlier runs, but not the final ones. It will be fixed in a separate test-infrastructure PR.

The adapter meta now carries the resource's strong ETag and its
Last-Modified, and both adapters send If-Range on every range request
when the probe gave a usable validator: a strong ETag, or else a
Last-Modified at least a second older than Date, and nothing when the
ETag is weak. The validators come from the unconditional bytes=0-0 GET,
never from a HEAD, which some servers answer with other validators.

A 200 to a conditional range request that carries another validator or
none, and a 206 whose validator conflicts with the one sent, are
ResourceChanged. A 200 with the same validator is still RangeIgnored and
falls back to one connection. A changed resource ends the task with
DownloadError::ResourceChanged after the usual teardown, without the
fallback. Restarting it belongs above the engine (#195).

BREAKING CHANGE: range requests now carry If-Range, and a resource that
changes during a download fails the task with the new
DownloadError::ResourceChanged (UnretryableError::ResourceChanged from
adapters) instead of falling back or completing. To get the new version,
download again with a fresh adapter: an adapter probes once and keeps
its validator, and its clones share it.
@greenhat616
greenhat616 force-pushed the feat/hardening-33-if-range branch from a1afe09 to a074c82 Compare November 13, 2026 22:25
@greenhat616
greenhat616 removed this pull request from stack #190 November 13, 2026 22:28
@greenhat616
greenhat616 added this pull request to stack #215 November 13, 2026 22:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant