fix: refuse wrong-architecture warm-start checkpoints structurally; cap DataLoader workers (#574)

[Ultimate-Storm]

Two fixes found by running the 1.8.0 MVP deploy test on real kits. Both are product code, so they are separate from the harness work in #573.

1. The warm-start guard let an unlabelled wrong-architecture checkpoint load

The #545 guard refused a mirror whose provenance sidecar named a different model — but accepted any unlabelled mirror with a warning, reasoning that pre-provenance mirrors have no sidecar and refusing them would break warm-continue for no gain.

On 2026-09-11 that was tested. A 722 MB 1DivideAndConquer mirror left on dl2 by a failed run (no sidecar) was auto-warm-started into an MST client. The guard warned and proceeded:

WarmStart: /scratch/mediswarm_latest_global.pt carries no provenance; cannot confirm it was produced by 'MST'. … Proceeding.
FLCallback - WARNING - There were missing keys when loading the global state_dict: ['_class_weight', 'loss.weight']
…
ValueError: None of the 187 incoming model parameter(s) matched the local model's 450 parameter(s).

E2, one round late. The two name sets were disjoint from the first byte; nothing needed to be trained to know it.

Now: the checkpoint's parameter names are intersected with the model's, sidecar or not. Empty intersection → WARM_START_MODEL_MISMATCH, naming both sides' first keys and the model the checkpoint's own train_conf records (E2's prescription). Partial overlap — the Lightning wrapper's few extra buffers — passes as before.

Two contract changes fall out, both tightenings:

• A refusal now returns None from load_model instead of panicking and then calling the parent load anyway. The original sidecar-mismatch path had that fall-through too; a test caught it.
• A skipped check is logged as skipped, with the reason. The first draft swallowed read errors silently — a guard that fails open quietly is the F10 pattern in a different coat.

Keys are read with mmap=True where torch supports it (2.2.2 in the image does), so listing a multi-hundred-MB mirror's names does not pull it into memory.

2. Cap DataLoader workers at what the data can feed (#574)

16 workers on 38 volumes maximises shm churn at every epoch boundary; the file_system sharing strategy's cleanup race then kills the pin-memory thread. The FT controller retried and the client was fine next attempt — but on a 2-client run with min_clients=2 a retry is a whole round.

cap_loader_workers guarantees ≥4 samples per worker per epoch: 16 → 9 on 38 volumes. It never binds on a real site (VHIO, the smallest, has 190). num_workers=0 is untouched. Logged when it applies. Lives in an import-free module so it is unit-testable without the training image.

Tests

+5 guard tests (the 09-11 case, same-architecture-unlabelled still loads, a lying sidecar, no-model skip, string train_conf), +6 cap tests. Full suite 545 passed, 8 skipped. None use importorskip.

🤖 Generated with Claude Code