Skip to content

feat(task): add the state file format and its writer - #204

Draft
greenhat616 wants to merge 1 commit into
feat/hardening-39-checkpointsfrom
feat/hardening-40-state-file
Draft

greenhat616 wants to merge 1 commit into
feat/hardening-39-checkpointsfrom
feat/hardening-40-state-file

Conversation

@greenhat616

Copy link
Copy Markdown
Owner

C33a (T21), the third commit of PR-9d (resume) in the download strategy hardening stack. It is stacked on #203, its base branch. C33 was too large for one reviewable commit, so it is split in two. This half adds the state file format and its writer, with no behaviour change. C33b turns it on for resumable downloads. The design notes are in §11.5.5 and §14.29.

What

The format (task/instance/state_file.rs, §11.5.5)

All fields are little-endian.

  • Header:
    • magic (BOLTLOAD), version 1, flags 0;
    • block_size (u32) and total (u64);
    • the validator: kind (1 = strong ETag, 2 = Last-Modified), a u16 length, then the bytes, at most 1024;
    • source: a u16 length (0xFFFF = none), then the bytes, at most 4096;
    • header_crc: the CRC32C of every header byte before it.
  • Records (24 bytes): index (u64), len (u32), data_crc (u32), kind 1 plus 3 zero bytes, then record_crc, the CRC32C of the 20 bytes before it.
  • Limits. An over-long validator or source is refused before anything is written. C33b treats an over-long validator as "no usable validator" (§11.5.3).
  • Reading. The real reader comes with C20. Here, a test-only decoder checks the bytes.

The writer

StateFile works through an injectable StateIo. In production that is the destination's locked handle (C31), which DestinationLock::handle() now exposes inside the crate.

Operation Steps
write_header write at offset 0, then sync_all
append write at end, sync_all, then move end
roll_back set_len(end), then sync_all
invalidate set_len(0), then sync_all
  • Checkpoints. StateFile implements C32's CheckpointReceiver, so it is the real checkpoint writer. A failed append rolls back, and a failed rollback stops the checkpoints. The file stays valid up to end.
  • The lock's handle. The docs of handle() and release() say that every state-file operation must end before the lock is released. On Windows, seek_write moves the cursor shared by all clones of the handle.
  • Parent directory. sync_parent_dir syncs the parent directory on Unix. Windows has no directory sync, which is a known limitation.

Checkpoint schedule

C32 left an open item: check its default schedule before checkpoints are switched on. Re-measured with the real StateFile as the receiver (release build, mmap writer, 5 runs per group, medians, one Windows machine, medium confidence):

Source No spacing (C32) 8 MiB / 2 s + 0.5 s spacing (new) Teardown only 32 MiB, no spacing
local, unthrottled, 256 MiB +58.1% (32 checkpoints) +15.8% (2) +13.2% (1) +24.5% (8)
100 MiB/s, 256 MiB −1.9% −1.9% +0.5% −1.4%
20 MiB/s, 64 MiB −0.4% (8) −0.3% (7) +0.2% −0.0% (2)
  • The decision (the implementer's): keep 8 MiB / 2 s, and add an internal minimum spacing of 0.5 s between the end of one checkpoint and the start of the next. The teardown checkpoint does not wait for it.
  • Why not a larger threshold: 32 MiB costs more locally, and it gives slow sources a quarter of the checkpoints, so after a crash there is up to four times as much to download again.
  • The price: a fast source may lose about 0.5 s of unrecorded data in a crash, which it downloads again quickly.

Not breaking

No public item changes. The TaskOptions change is documentation only. Production behaviour is unchanged, because only tests switch checkpoints on.

Tests

  • Byte layout. Header and record layouts with their CRCs, both validator kinds, no source and an empty source, the limits, and one byte over each.
  • Operation order and failures.
    • The order of append, rollback and invalidation.
    • A failed write or sync during an append: end is unchanged, and the next append goes to the same offset.
    • A failed rollback: the file is valid up to end, and the torn tail is dropped by the decoder.
    • A failed header write or sync.
  • Real files.
    • A real file through the locked handle. A second acquire is InUse, and release removes the file.
    • sync_parent_dir.
    • A real download with StateFile as the receiver. It leaves a valid header plus exactly one record per block, with CRCs that match the temporary file and nothing trailing.
  • Spacing. The minimum spacing, checked just before and exactly at the boundary. That test passed 20 times out of 20.
  • Mutations: 10 of 10 caught.
    • end moved before the sync.
    • A rollback that does nothing.
    • A missing sync on the header, on records, on a rollback, and on invalidation.
    • The header CRC skipping the magic.
    • The record CRC covering 16 bytes.
    • The validator limit off by one.
    • The spacing ignored.

Review

Reviewed with /ccg:review, using the codex backend only (a single-model review).

Round 1: PASS (94/100). Its two Minors and its suggestion are applied:

  • the spacing test no longer depends on the wall clock;
  • the docs say the teardown checkpoint ignores the spacing;
  • the docs of handle() and release() state the lifetime rule.

C33b adds a gated real-handle test for that rule.

Checks

All on +nightly-2026-11-06, with --no-fail-fast. Clippy allows unnecessary_lazy_evaluations until #183 merges.

  • cargo fmt --all -- --check.
  • 4 clippy configs with -D warnings.
  • cargo test --workspace --all-features --locked: 28 suites, 867 passed, 1 ignored (the benchmark).
  • cargo test --workspace --locked: 28 suites, 796 passed, 1 ignored.
  • RUSTDOCFLAGS=-Dwarnings cargo doc --no-deps --all-features -p bolt-load: the same 18 errors as its base. It was run before the round-1 fixes, which changed only docs and a test.

Add the binary state file of the resume design: a header (magic,
version, block size, total, the If-Range validator, the source, and a
header CRC32C) and fixed 24-byte records of complete blocks, each with
its own CRC32C. StateFile writes it only through the destination's
locked handle. It writes the header, appends records at the end it
keeps, rolls back to that end, and invalidates, and every step is
synced before it counts. It is the real receiver for the checkpoints:
a failed append rolls back, and a failed rollback stops them.

Checkpoints now keep at least 0.5 s between the end of one and the
start of the next; the teardown checkpoint does not wait. Measured with
the real state file, that cuts the cost of an unthrottled local
download from +58% to +16% (teardown alone is +13%), while slower
sources barely change. The 8 MiB / 2 s defaults stay.

Nothing in production writes a state file yet: resumable downloads
switch it on in the next commit, and reading it back comes with resume.
@greenhat616
greenhat616 force-pushed the feat/hardening-40-state-file branch from 4180d6f to 29412b9 Compare November 13, 2026 22:26
@greenhat616
greenhat616 removed this pull request from stack #190 November 13, 2026 22:28
@greenhat616
greenhat616 added this pull request to stack #215 November 13, 2026 22:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant