Reduce Git contention and benchmark multi-gateway Git and LFS workloads

[forhappy]

Summary

• Let fetch preparation share a repository object cache concurrently; coordinate same-object hydration and preserve atomic publication. Allocate object-write locks only for caches that actually hydrate objects.
• Receive multiple uploads for one repository concurrently while keeping push-ID checks, native Git, and durable publication serialized.
• Limit each account to half of the pending transfer queue, leaving room for other accounts.
• Add round-robin multi-ingress benchmark runs with per-gateway outcomes and latency percentiles for metadata, ref discovery, clone, cold/incremental fetch, fast-forward pull, unique-ref push, and bounded direct-basic LFS upload/download. An opt-in two-commit corpus supports incremental workloads while existing one-commit manifests remain compatible.

Verification

• cargo test --locked --lib (111 passed)
• cargo test --locked --test multi_server stock_git_history_refs_and_shallow_fetch_survive_fresh_disk_restore (passed, including four concurrent cold clones and fsck)
• cargo test --locked --test multi_server peers:: (2 passed)
• cargo test --locked --test smart_http (passed, including stalled concurrent push-upload and replay assertions)
• cargo clippy --all-targets --locked -- -D warnings (passed)
• python3 -B -m unittest discover -s scripts -p test_benchmark_repositories.py -v (6 passed, including versioned seed/verify, stock Git, LFS transfer and corruption fixtures)

Scope

This removes two hot-repository serialization points and improves transfer fairness. It adds measurement tools but does not establish the proposed 10,000-repository capacity or production Git/LFS throughput targets; those still require a representative multi-node workload and provider-backed measurements.

Local process smoke (2026-09-29)

• Started the debug Canopy binary against local RustFS S3, seeded 10 repositories (3 populated, 2-commit incremental fixtures and 128-byte LFS fixtures), then verified all 10 identities and 3 stock-Git v0/v2 samples.
• Ran each of metadata, refs, clone, cold fetch, incremental fetch, incremental pull, unique-ref push, LFS download and LFS upload at 2 offered requests/s for 2 seconds, concurrency 2: 4/4 successful arrivals per path, no driver failures. Git transfers used stock Git; LFS downloads checked size and SHA-256.
• This is a low-rate functional smoke on one macOS gateway and local storage, not evidence of the 10,000-repository target, multi-node scaling, or production latency. The checked-in Cellule revision still needs the documented idle-renewal qualification.

Dense owner renewal dependency

• Pins Cellule revision 21bed5e from PR #25, based on the scheduler change merged in PR #23. The scheduler orders due owner renewals by deadline and refills freed slots; the new commit separates explicit release admission from automatic pressure shedding.
• A 30-second in-memory diagnostic passed 1,000 active repositories (1,001/1,001 serving Cells updated, 326.830 control updates/s, zero failed PUTs) and fresh-node recovery. The checked-in example now labels memory:/// as diagnostic only.
• RustFS beta.8 with 2 CPUs/4 GiB passed 100 and 500 active repositories, then mass-fenced owners while seeding beyond 800 and stopped at 812. It did not complete a 1,000-Cell window or graceful shutdown; real-store density remains unqualified.
• Parallel CI and a full local suite reproduced a 4,096-ref mirror push exceeding Cellule's five-second SQL command deadline. Commit e57d149 skips per-ref state and namespace probes when the refs table is transactionally empty, with a regression for in-plan ancestor conflicts. The mirror test now passes focused and under full local parallelism; the Repository Cell regression passes.
• A separate 65 MiB commit-ingestion test exceeded the same SQL deadline under full local parallel load (92/93 non-ignored integration tests passed). It passes isolated and in a four-at-a-time run; the contention sensitivity remains open, not a production-capacity qualification. The bounded run also exposed a cold-eviction fixture assumption and a two-second fault-test timeout. Commit 2b51b14 makes the fixture establish cold state and leaves a five-second timeout for the non-serialization check; all affected focused tests pass. Current-head CI is pending.
• The >5 GiB RustFS gate failed again on a GitHub-hosted runner: RustFS returned write-quorum HTTP 500 during multipart staging and the runner explicitly reported No space left on device. Commit 97e9ddd adds a 40-GiB-free preflight for both scratch and provider volumes; 10 Python tests and a live negative preflight on undersized local Docker storage passed. The non-sparse size gate is still required and remains unqualified on the current pin until run on adequately provisioned storage.
• At 100 active slots, a first-pass uniform scan across 10,000 locally owned identities requires at least 9,900 releases. The previous shared two-per-second movement budget was a roughly 82.5-minute rate bound. The new Cellule pin gives requested releases an independent 32-per-second, 32-in-flight budget while preserving automatic pressure shedding at two per second. Even at the new maximum rate, about 310 one-second windows are needed before restore or Git work; this is not a measured scan or provider capacity claim. Focused Cellule budget/lifecycle/pressure tests and 111 Canopy library plus 15 residency tests passed. A representative 10,000-identity uniform-access run remains open.