build(a2d-run): port source_hash to sha2 0.11 / digest 0.11 traits

[undeemed]

Intent

Port a2d to sha2 0.11.0, replacing dependabot PR #3 (#3), whose naive version bump fails the rust CI job. Despite the minor-looking version number, sha2 0.11 rides the breaking digest 0.11 trait release, so this is a real port rather than a lockfile bump.

Two compile breaks had to be fixed in crates/a2d-run/src/rundir.rs source_hash(), and both were reproduced locally first against the same errors CI reported on #3:

1. digest 0.11 removed its 'std' feature entirely, so hashers no longer implement io::Write and std::io::copy no longer compiles. Deliberate choice: an explicit 64 KiB read loop, NOT reading the file into memory, because model.safetensors can be multi-GB. An io::Write adapter struct was considered and rejected as more code than the loop for a single call site.
2. finalize() now returns a hybrid_array::Array instead of generic-array::GenericArray, and Array has no LowerHex impl, so format!("{:x}", ..) no longer compiles. Deliberate choice: encode the bytes inline with format!("{b:02x}") rather than adding a hex or base16ct dependency, since a hex encoder is a few characters of code and this is the only call site.

Hash-output compatibility was the risk that actually mattered here and was verified, not assumed. source_hash feeds manifest.source_hash, which is durable provenance, and the Python eval worker independently re-derives it with hashlib.sha256().hexdigest() and refuses to trust the AR baseline on mismatch (packages/a2d-worker-hf/src/a2d_core/eval/likelihood.py). Output is unchanged: still a lowercase 64-char hex digest. The pre-existing sha256("abc") test was intentionally left untouched because its passing unmodified IS the before/after compatibility proof.

One test was added: source_hash_spans_read_chunks in crates/a2d-run/src/worker.rs, feeding 200_000 bytes so the new multi-pass read loop is actually exercised, since the existing 3-byte case only ever runs one iteration. Both expected digests were computed with Python hashlib, an independent implementation, so the cross-language contract is anchored rather than self-confirming.

Also checked, per the task: only one digest major is linked (0.11.3), crypto-common is 0.2.2 only, and generic-array drops out of Cargo.lock entirely, so there is no dual-major situation to resolve.

Context on this being a re-run: an earlier run (01KYQ2DC2W4VMG5RJAJ4D3GBG0) on this exact commit was cancelled mid-flight when the shared no-mistakes daemon was terminated during a container OOM sweep - it was not a test failure, and no fix commits were produced, so the branch is unchanged at 781267f. That cancelled run's review had completed and raised two latent, non-blocking notes, recorded here so this round has the context rather than rediscovering it cold: (a) rundir.rs - the hand-rolled read loop drops the ErrorKind::Interrupted retry that std::io::copy does internally, currently unreachable because the workspace installs no signal handlers, but it would become a spurious failure on a multi-GB hash the day a Ctrl-C/cancel handler lands; (b) Cargo.toml - the bump raises the effective MSRV to 1.85 because hybrid-array, crypto-common and digest are edition 2024, which is harmless today since rust-toolchain.toml pins channel = stable and no crate declares rust-version. Treat both as open questions for this review rather than as settled, and flag them if you disagree.

Constraints the reviewer should know: the change is deliberately kept small (35 insertions, 3 deletions excluding Cargo.lock, well under a 500-line budget); no GPU or accelerated validation is possible on this box (aarch64, no Metal/MPS/CUDA); and this ships on its own branch fm/a2d-dep-sha2-t2 rather than being pushed to the dependabot branch, which is intentional - firstmate closes #3 as superseded once this lands.

What Changed

• Bumped the workspace sha2 dependency from 0.10 to 0.11, which rides the breaking digest 0.11 release: digest 0.11.3, crypto-common 0.2.2 and hybrid-array 0.4.13 enter Cargo.lock and generic-array drops out entirely.
• Rewrote source_hash in crates/a2d-run/src/rundir.rs for the new traits. digest 0.11 dropped the hasher io::Write impl, so std::io::copy is replaced by an explicit 64 KiB read loop (with an ErrorKind::Interrupted retry arm) that keeps peak memory flat on multi-GB model.safetensors; finalize() now returns a hybrid_array::Array with no LowerHex impl, so the digest is hex-encoded inline with format!("{b:02x}"). Output stays a lowercase 64-char hex digest, keeping manifest.source_hash identical to the hashlib.sha256().hexdigest() the Python eval worker re-derives.
• Added source_hash_spans_read_chunks in crates/a2d-run/src/worker.rs (200,000 bytes, so the multi-pass read loop is actually exercised - the pre-existing 3-byte case only ran one iteration), documented the cross-language digest contract on source_hash, and recorded the resulting stable-1.85 floor in CONTRIBUTING.md since the new dependency chain is edition 2024. No rust-version key was added, so the floor is prose-only and cargo does not enforce it.

Risk Assessment

✅ Low: Tightly scoped dependency port with one call site, an independently verified hash-output contract (both pinned digests reproduce under Python hashlib), a new test covering the multi-pass read path, and the only prior finding (missing EINTR retry) already fixed in 0d3ea1a.

Testing

Baseline cargo test --workspace is fully green (48 tests) and the two Python eval test files that carry source_hash pass. Beyond unit tests, I drove the actual product surface: a2d convert on a 5 MiB+1 byte weights file (so the new 64 KiB read loop makes ~81 passes including a partial tail) and captured the resulting manifest.json, then proved the durable-provenance digest is byte-identical to a locally rebuilt pre-port sha2 0.10 CLI binary, to GNU sha256sum, and to Python hashlib, and that the real downstream gate in the Python eval worker accepts the Rust hash and rejects it after a single flipped byte. The new chunk-spanning test also passes against the base sha2 0.10 code, which makes its expected digest a genuine before/after invariant. The only part I could not exercise is the ErrorKind::Interrupted retry - a regular-file read has no signal source in this environment and the workspace installs no handlers - and no GPU/accelerated path exists on this aarch64 box, neither of which this change touches. No product or test failures found.

Evidence: a2d convert CLI transcript (end-user run that writes manifest.source_hash)

$ a2d convert /tmp/a2d-sha2-e2e/model --out /tmp/a2d-sha2-e2e/run --data ./fixtures/data/tiny.jsonl --max-steps 10 --anneal-steps 5 --worker-cmd "sh /tmp/a2d-sha2-e2e/fake_worker.sh" job started (worker: fake-worker) identity gate: PASS (max_abs_diff=0.00e0, tolerance=1e-6) job completed run 285ccef1-01a0-461a-baac-e57cce25412b completed -> /tmp/a2d-sha2-e2e/run exit=0

$ a2d convert /tmp/a2d-sha2-e2e/model --out /tmp/a2d-sha2-e2e/run --data ./fixtures/data/tiny.jsonl --max-steps 10 --anneal-steps 5 --worker-cmd "sh /tmp/a2d-sha2-e2e/fake_worker.sh"
job started (worker: fake-worker)
identity gate: PASS (max_abs_diff=0.00e0, tolerance=1e-6)
job completed
run 285ccef1-01a0-461a-baac-e57cce25412b completed -> /tmp/a2d-sha2-e2e/run
exit=0

Evidence: manifest.source_hash vs coreutils sha256sum vs Python hashlib (5,242,881-byte weights)

== independent oracles over the same 5242881-byte model.safetensors == manifest.source_hash (a2d, Rust sha2 0.11) : 8bd83e68831344d036c3cd254d77fe6401fb907d4cdbbd853a2881b43557bbea sha256sum (GNU coreutils) : 8bd83e68831344d036c3cd254d77fe6401fb907d4cdbbd853a2881b43557bbea python hashlib.sha256().hexdigest() : 8bd83e68831344d036c3cd254d77fe6401fb907d4cdbbd853a2881b43557bbea len=64 lowercase-hex=yes MATCH: all three agree

== manifest.json (a2d convert output, sha2 0.11 build) ==
{
  "schema_version": "0.1.0",
  "a2d_version": "0.1.0",
  "job_id": "285ccef1-01a0-461a-baac-e57cce25412b",
  "created_at": "2026-07-29T14:22:23.088842850+00:00",
  "model_path": "/tmp/a2d-sha2-e2e/model",
  "status": "completed",
  "finished_at": "2026-07-29T14:22:23.089887393+00:00",
  "model_spec": {
    "model_type": "gpt2",
    "n_layers": 12,
    "d_model": 768,
    "vocab_size": 50257,
    "n_heads": 12,
    "n_kv_heads": 12,
    "sliding_window": null,
    "n_experts": null,
    "n_active_experts": null,
    "capabilities": [
      "paradigm.ar-transformer",
      "attn.full",
      "pos.learned",
      "ffn.dense"
    ],
    "mask_token_id": null,
    "inferred": false
  },
  "conversion_config": {
    "objective": "mdlm",
    "data": "./fixtures/data/tiny.jsonl",
    "anneal_steps": 5,
    "anneal_schedule": "linear",
    "seq_len": 512,
    "per_device_batch_size": 8,
    "grad_accum": 1,
    "lr": 0.0001,
    "max_steps": 10,
    "max_tokens": null,
    "mask_token": "grow",
    "keep_last": 3,
    "seed": 0,
    "device": "auto",
    "dtype": "float32"
  },
  "identity": {
    "passed": true,
    "max_abs_diff": 0.0,
    "tolerance": 1e-6
  },
  "data_source": "./fixtures/data/tiny.jsonl",
  "source_hash": "8bd83e68831344d036c3cd254d77fe6401fb907d4cdbbd853a2881b43557bbea"
}

== independent oracles over the same 5242881-byte model.safetensors ==
manifest.source_hash (a2d, Rust sha2 0.11) : 8bd83e68831344d036c3cd254d77fe6401fb907d4cdbbd853a2881b43557bbea
sha256sum (GNU coreutils)                  : 8bd83e68831344d036c3cd254d77fe6401fb907d4cdbbd853a2881b43557bbea
python hashlib.sha256().hexdigest()        : 8bd83e68831344d036c3cd254d77fe6401fb907d4cdbbd853a2881b43557bbea
len=64  lowercase-hex=yes
MATCH: all three agree

Evidence: Before/after A/B: same file, same CLI, sha2 0.10 vs sha2 0.11 builds

== before/after A/B: same 5 MiB model.safetensors, same CLI, two sha2 majors == a2d built against sha2 0.10 -> manifest.source_hash = 8bd83e68831344d036c3cd254d77fe6401fb907d4cdbbd853a2881b43557bbea a2d built against sha2 0.11 -> manifest.source_hash = 8bd83e68831344d036c3cd254d77fe6401fb907d4cdbbd853a2881b43557bbea IDENTICAL: durable provenance unchanged across the port (binaries verified distinct: a2d(sha2 0.10) links digest-0.10.7 generic-array-0.14.7 sha2-0.10.9; a2d(sha2 0.11) links digest-0.11.3 sha2-0.11.0)

== before/after A/B: same 5 MiB model.safetensors, same CLI, two sha2 majors ==
a2d built against sha2 0.10  ->  manifest.source_hash = 8bd83e68831344d036c3cd254d77fe6401fb907d4cdbbd853a2881b43557bbea
a2d built against sha2 0.11  ->  manifest.source_hash = 8bd83e68831344d036c3cd254d77fe6401fb907d4cdbbd853a2881b43557bbea
IDENTICAL: durable provenance unchanged across the port

Evidence: Python eval worker re-derives and gates on the Rust-written hash (with negative control)

eval worker re-derives manifest.source_hash=8bd83e68831344d036c3cd254d77fe6401fb907d4cdbbd853a2881b43557bbea trust AR baseline? reason = None negative control: one byte of model.safetensors flipped trust AR baseline? reason = 'source weights hash mismatch (source changed since conversion)' byte restored trust AR baseline? reason = None

$ uv run python check_consumer.py   # Python eval worker independently verifies the Rust-written hash
eval worker re-derives manifest.source_hash=8bd83e68831344d036c3cd254d77fe6401fb907d4cdbbd853a2881b43557bbea
  trust AR baseline?  reason = None
negative control: one byte of model.safetensors flipped
  trust AR baseline?  reason = 'source weights hash mismatch (source changed since conversion)'
byte restored
  trust AR baseline?  reason = None

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

🔧 **Review** - 2 issues found → auto-fixed ✅

• ⚠️ crates/a2d-run/src/rundir.rs:84 - The hand-rolled read loop that replaced std::io::copy drops the ErrorKind::Interrupted retry that io::copy performs internally, so an EINTR from File::read now aborts source_hash with a "hashing <path>" error instead of resuming. Unreachable today (the workspace installs no signal handlers - grep for signal/SIGINT/ctrlc across crates/ finds none), but it becomes a spurious failure on a multi-GB hash the moment a Ctrl-C/cancel handler lands, and is already reachable on EINTR-prone mounts (NFS soft mounts, FUSE). Restore parity with a Err(e) if e.kind() == std::io::ErrorKind::Interrupted => continue arm by matching on file.read(&mut buf) instead of using ?.
• ℹ️ Cargo.toml:14 - The sha2 0.10 -> 0.11 bump pulls in digest 0.11.3, crypto-common 0.2.2 and hybrid-array 0.4.13, which are edition 2024 crates, so the effective MSRV becomes 1.85. Nothing in the repo declares rust-version, rust-toolchain.toml pins channel = "stable", and .github/workflows/ci.yml uses setup-rust-toolchain@v1 with no version pin, so this is inert today - noting it so a future MSRV policy does not get set below 1.85 by accident.

🔧 Fix: retry EINTR in source_hash read loop
✅ Re-checked - no issues remain.

✅ **Test** - passed

✅ No issues found.

• cargo test --workspace (48 tests, all ok; includes worker::tests::source_hash_spans_read_chunks and source_hash_of_primary_safetensors)
• Manual E2E CLI run: target/debug/a2d convert <model> --out <run> --data ./fixtures/data/tiny.jsonl --max-steps 10 --anneal-steps 5 --worker-cmd "sh fake_worker.sh" on a 5,242,881-byte model.safetensors (80 full 64 KiB reads + 1-byte final partial chunk), then read manifest.source_hash out of the emitted manifest.json
• Cross-implementation digest check on the same file: manifest.source_hash vs sha256sum vs python3 -c 'hashlib.sha256(...).hexdigest()' - all three identical, 64 chars, lowercase hex
• Before/after A/B: temporarily checked out base Cargo.toml/Cargo.lock/rundir.rs (sha2 0.10), built a second a2d binary with cargo build --offline -p a2d-cli, restored the worktree, then ran both binaries over the same weights file - identical manifest.source_hash. Verified the binaries genuinely differ (grep -a shows sha2-0.10.9/digest-0.10.7/generic-array-0.14.7 vs sha2-0.11.0/digest-0.11.3)
• Ran the new source_hash_spans_read_chunks test against the base sha2 0.10 implementation (cargo test --offline -p a2d-run source_hash) to confirm its expected digest is a real before/after invariant rather than fitted to the new read loop
• Downstream consumer gate: uv run python calling a2d_core.eval.likelihood.source_reason(model_path, manifest.source_hash) - returns None (AR baseline trusted), returns 'source weights hash mismatch (source changed since conversion)' after flipping one byte at offset 3,000,000, returns None again after restoring it
• uv run pytest tests/test_eval_main.py tests/test_eval_report.py -q (5 passed) - the Python-side paths that carry source_hash through eval request/report
• Lockfile audit: grep -n -A1 '^name = "digest"|"crypto-common"|"generic-array"|"sha2"|"hybrid-array"' Cargo.lock - digest 0.11.3 only, crypto-common 0.2.2 only, hybrid-array 0.4.13, generic-array absent
• MSRV latent-note check: rust-toolchain.toml pins channel = "stable", no rust-version in any Cargo.toml, .github/workflows/ci.yml rust job uses that pin with cargo test --workspace
• git status --porcelain after the A/B checkout dance - worktree restored clean, scratch dir removed

⚠️ **Document** - 1 info

• ℹ️ Cargo.toml:14 - The Rust 1.85 floor is now documented only in CONTRIBUTING.md prose. Declaring rust-version = "1.85" in the workspace Cargo.toml would make cargo enforce it instead of leaving a contributor on an older stable to hit a raw edition2024 is required error. That is a manifest/config change, not documentation, so it is out of scope for this pass - a human should decide whether to codify the MSRV or keep it prose-only alongside the rust-toolchain.toml stable pin.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.