Bound authentication, stabilize residency, and restore retained catalog identities

[forhappy]

Summary

Correctness status: release CI at production-equivalent head 5581d5c failed residency::faults::disconnected_admission_finishes_release_and_allows_a_later_restore with HTTP 503. Both debug workflows passed, but they do not clear this failure. The unchanged-source release diagnostic completed: 100/100 isolated runs and 2/2 full-target runs passed. Its archive, 264 source inputs, retained executable and all 102 result/log bindings were independently audited and copied to a second local filesystem. This is nonreproduction, not a fix. No root cause or resolution is claimed. Fresh CI for the documentation-only update is separate.

Follow-up to merged #17. This PR contains bounded authentication, cold-residency admission, retained-catalog startup fixes, the latest Cellule pin and three-node verification tools, with qualification status separated below. An owned old-binary RustFS upgrade and twenty scheduled critical workflows passed complete fresh-owner recovery. The affected CI test's HTTP port reservation gap is now reproduced and corrected, with separately bound release/RustFS checks. The original 10,003-Cell corpus also completed supported same-code maintenance recovery: all 301 unsettled Cells drained, all published roots and catalog identities unchanged; it subsequently passed full new-release admission and controlled activation to Ready revision 9 without changing any canonical Control or published root. Three upgraded gateways passed full remote verification: all 10,000 original identities, 100 full LFS bodies, 200 stock-Git v0/v2 clones and both critical fixtures. An independent offline audit passed, and 7,061 closed files were copied and reread. The unchanged full load campaign is running: its first ten metadata windows recorded 23,779/24,000 OK, 206 busy drops and 15 HTTP 503s; concurrent critical load closed at 19/20 OK with one busy drop. These are failed arrival gates, not capacity passes. The original 10,000-repository campaign and reference capacity remain open. Both Linux PR and push CI at documentation head 2c897c7eae1afa36049685dd35f2214082e03e23 passed formatting, lints, tests, real RustFS Git compatibility, the server build and the Python harness. All four fresh CI jobs at documentation head 085a278 passed; those historical documentation updates changed no production, test, script, Cargo or live benchmark-plan inputs.

• Bound authentication query 5 to 36 bytes of canonical input and 256 bytes of result. Preserve generic contracts, execution-time expiry, FIFO and runtime budgets. The held-worker regression admits all 16 calls with zero refusals and 4,673 retained bytes; the original refused the sixteenth in all ten pre-fix runs.
• Reobserve settling, unpinned residents within 200 ms and at most eight waits. Keep permits, pin/generation/release checks and runtime ownership safety intact. No mutation replay or forced release of busy Cells.
• Retain the exact Directory predecessor at schema 1. Admit verified existing SQL catalog identities read-only under the exact unchanged Ready release; new entries still use strict current-code provisioning.
• Reject unsupported persisted Control code/schema before acquisition, with identical canonical Control bytes after refused startup. Cover release transitions, corrupt descriptors and unsupported identities with targeted guards.
• Isolate the concurrent-fork parent-fence test and add inherited-fence coverage. Production cleanup is unchanged; temporary diagnostic logging is excluded.
• Upgrade all six Cellule packages to 191409685b001a82bd02780def45102b4fc2f164, rechecked against current upstream main at publication. Its Linux debug CI and independently audited Linux release correctness run passed; latest-head debug CI passed, but release CI failed the disconnected-admission test described above. The original-corpus campaign remains frozen on the separately qualified 0dc04a6 executable, so its recovery/performance evidence does not transfer to this pin. Only five manifest pins and six lockfile source revisions change; other dependency versions and runtime budgets are unchanged.
• Update documentation with a restore sequence diagram, concise result tables, reproducible commands and separate current versus historical artifact bindings.
• Transfer already-bound HTTP listeners through the existing startup supervisor. Reject listener/config address mismatch before workspace/storage writes; preserve readiness, fencing, cancellation and drain. All six SHA-256 test startup addresses retain reservations through startup. No retry-only fix, serialized tests or relaxed Git assertions.

Historical verification of the frozen 0dc04a6 build

At the previous PR head ec81650, all 264 production, test, script and Cargo files matched frozen tested source 3dda2b47b1cba105642a62b4ad27d7c84d0940d5. The current PR changes only two of those files: the Cellule manifest pins and lockfile sources. The results in this historical section remain bound to the frozen source, not to the current dependency pin. The executable and workspace test artifacts are retained outside Cargo targets. Publication checks verify source equivalence, all eight listener-checkpoint artifact digests, formatting, diff cleanliness, documentation links and the same-source Python harness. The earlier measured build remains bound to bbd784a40c3867646044a8c716b7ee517f9aca49; its performance results do not transfer to the new executable.

Check	Closed result
Locked release build	Passed; executable SHA-256 a61ef0f2cb977348e4e4fc45330334a1f8fd68bf8c44146cd9343b0e6676a6c8
Release workspace	244 top-level Rust tests passed, zero failed, nine ignored; nested subprocess tests counted once
Release lints	All targets passed with warnings denied
Python harness	All 84 tests passed
Directory suite	All 12 passed, including bounded authentication and predecessor descriptor admission
Catalog admission guards	All five passed
Real RustFS compatibility	All eight exact provider-only gates passed, closed October 2 at 01:18:09 UTC (October 1 Pacific)
Listener handoff	All four regressions plus admitted startup cancellation passed in each of 20 independent runs; 100 checks, not performance samples
Lifecycle	All 14 passed, including both bind paths rejecting ignored conditional writes before enrollment; actual 300-second SSH-LFS expiry preserved in full workspace
Earlier frozen regression evidence	On bbd784a4, 20 cold-activation repetitions, 20 three-case retained-startup repetitions and all 15 residency tests passed; those historical artifacts are separately retained
Evidence preservation	All 695 closed listener qualification files copied and independently reread on a different local filesystem, including failed attempts; not off-machine backups
CI	At 240203a7, one Rust job failed AddrInUse and the parallel job passed. Both later Rust jobs and both harness jobs at 0498e22 passed before the correction. The original failure is retained. Fresh Linux PR CI at eaebfc4 passed both Rust and harness jobs, including real RustFS compatibility; the separate push workflow also passed both jobs

The native retained-catalog regression uses an owned in-memory store written by the current test runtime; the eight provider compatibility gates use a disposable fresh fixture. The separate actual-old-binary run below is an upgrade/recovery proof, not a non-sparse five-GiB test. The existing 10,000-repository provider was unchanged before and after qualification; neither its corpus nor the UI preview was upgraded during that qualification. The subsequent original-corpus activation is reported separately below.

The listener reservation regression failed with AddrInUse in all three pre-fix runs. An injected competing binder reproduces the helper's gap; the exact competing binder in the original CI job was not captured. The corrected tests keep the listener bound into supervised startup and verify advertised clone URLs, refusal before writes, rebind after shutdown and cleanup after cancelled startup. Other tests still use the legacy address helper; this does not claim every test-suite port race is eliminated.

Actual old-binary upgrade and scheduled critical workflows

• The exact retained old executable produced two Git/LFS repositories on an owned RustFS fixture, verified full bytes, exited zero and entered same-code maintenance with zero writers/unsettled Cells.
• Fixture-only admission checked the actual predecessor descriptor, all 256 catalog shards, the three expected Cells, supported catalog and actual Control code/schema, zero advertised writers and repeated unchanged snapshots. Controlled activation did not rewrite catalog identity or bypass ownership fencing.
• The new qualified executable restored those repositories through three fresh gateways and a proxy, with exact Git v0/v2 refs, full fsck and full LFS bytes. Both subsequent fresh Git/LFS writes passed.
• 20/20 scheduled workflows and 340/340 critical steps passed in 300 seconds, at one workflow every 15 seconds, concurrency cap four and observed peak overlap three. No failed or dropped arrivals. Delivered throughput was 0.066667 workflows/s; whole-workflow p50/p95/p99 was 18.414/30.630/31.909 seconds, including stock-Git client work and validation. This is not saturation capacity or a matched speedup.
• Closed ledgers were preserved before SIGKILL of the exact three owned gateway PIDs. After 32.007527 seconds confirmed owner absence, three new gateways with fresh local disks verified both old repositories, both new Git/LFS write ACKs and all 40 critical repositories. Recovery gateways then drained without force, all exiting zero.
• Final evidence includes 13,292 copied and independently reread files on a different local filesystem, with failed attempts retained. Original corpus, provider and UI remained untouched.

The initial remote LFS byte mismatch was missing clone-side LFS filter setup, proven by a failing stock-Git regression and corrected without weakening the full-byte check. The earlier reserved-ref rejection remains a negative result, with its partial main/tag ACKs preserved.

Original full-corpus maintenance recovery

• Two write-disabled scans verified all 256 catalog shards / 10,003 Cells, the actual old descriptor, supported initial catalog and actual Control code/schema, zero advertised writers and all three previous owners canonically retired. Snapshots were identical; no provider write was attempted.
• The exact retained old executable entered normal fenced maintenance using a new worker NodeID, test signing identity and scratch disk. All 301 unsettled Cells drained; the command and final status both exited zero. At that checkpoint, admission remained closed in the old release's Maintenance revision 5; the subsequent activation below supersedes that status.
• Two complete post-recovery snapshots and an independent canonical JSON/receipt audit passed: 10,003 idle, unowned, rooted Controls, 9,702 previously idle Controls and ETags unchanged, every published root and Cell incarnation unchanged. The 301 recovered Controls advance fencing epoch and revision.
• Original RustFS container, volume, image, configuration, start time, resource envelope and restart count stayed unchanged. No owner erasure, force-CAS, lease widening, catalog rewrite, reseed or provider restart. All owned workers and children are absent.
• 73 closed files and exact inputs were copied and independently reread on another local filesystem, including preserved failed-tool attempts. This is not an off-machine/provider-data backup.
• The 234.461-second recovery command is maintenance wall time, not Git latency or throughput. That maintenance checkpoint alone establishes no new serving gateway, remote Git/LFS payload result, original-corpus upgrade, matched speedup or reference capacity.

See the original corpus recovery checkpoint for the sequence, counts, CLI boundaries and artifact digests.

Original full-corpus release activation

• Full rolling admission verified the actual predecessor descriptor, new release compatibility, every catalog and actual Control code/schema, all 256 shards / 10,003 Cells, and two identical read-only snapshots.
• The exact old CLI ended maintenance normally: old Ready 6 → Prepared 7 → Activating 8 → new Ready 9. The scoped controller used existing SDK release transitions, not raw record rewrites.
• Its transport gate allowed only the exact new descriptor creation and three exact conditional release updates: four forwarded writes, zero refused activation attempts. Control, catalog, node and data writes stayed prohibited.
• Eight complete activation scans matched the admitted canonical Controls, ETags, catalog and published roots. All Cells were idle and unowned during activation; RustFS configuration, volume, resource envelope and restart count stayed unchanged.
• The scoped tool passed three unit tests, locked offline metadata/release build and all-target Clippy. 369 closed files were copied and independently reread on another local filesystem; this is not an off-machine/provider-data backup.
• Activation closed October 2 at 02:32:05 UTC (October 1 Pacific). Three fresh gateways then reported ready behind the proxy with the qualified executable SHA-256 a61ef0f2cb977348e4e4fc45330334a1f8fd68bf8c44146cd9343b0e6676a6c8 and 100 active repositories per node. Readiness is not full remote-content or performance proof.

See the activation checkpoint for the sequence, write boundary, counts and immutable artifact bindings. This activation and the live load use Cellule 0dc04a6. The PR now pins 1914096, with separate CI below. These activation and performance results do not transfer to that revision.

Original-corpus remote verification and current load snapshot

Gate	Result
Full remote content	Passed: 10,000 original UUIDs, 100 complete LFS body digests, 200 v0/v2 clones with exact commits/files/fsck, and both critical fixtures with four exact ref inventories/mirrors
Independent offline audit	Passed: 10,002 identities, 100 LFS digests, 824 Git commands and all local clone/mirror bytes reconciled
Closed evidence	7,061 files and immutable inputs copied and independently reread on a different local filesystem; not off-machine/provider-data backups
Completed metadata phase / 18 windows	Failed arrival gate: 41,485 OK / 43,200 scheduled; 1,669 busy drops and 46 HTTP 503s
Completed creation phase / 6 windows	Failed arrival gate: 1,464 OK / 2,160 scheduled; 690 busy drops and 6 HTTP 503s. All positive ACK UUIDs are unique; post-owner-loss recovery remains open
Completed HTTP v2 discovery / 8 windows	Failed arrival gate: 10,309 OK / 57,600 scheduled; 9,581 busy drops, 37,706 HTTP 503s and 4 transport errors. Capability discovery, not the full stock-Git ref exchange
Completed stock-Git ls-remote / 8 windows	Failed arrival gate: 632 OK / 1,200 scheduled; 514 busy drops and 54 Git errors. Actual Git client checks the expected main ref
Concurrent critical schedule	Failed arrival gate: 19 OK / 20 scheduled, one busy drop with no writes; 323 successful steps and 38 acknowledged UUIDs in complete attempted receipts
Critical workflow timing	0.060000 workflows/s in-window, 0.060223/s including drain; p50/p95/p99 37.089/60.551/60.551 seconds including stock-Git work and validation
Preservation during load	Each sealed window's ledgers, report and resource bindings audited before copying; completed workflow receipts/local Git data preserved. Mutable outputs are excluded
Recovery / capacity	Still open: full schedule/audit, original-corpus and every-ACK recovery after owner loss, concurrent faults, matched comparisons and isolated Linux capacity

Remote verification closed October 2 at 03:02:09 UTC, independently audited at 03:02:40 UTC. The four completed phases / 40-window snapshot was checked at 04:33 UTC (October 1 Pacific). Later clone windows are excluded from this completed-phase summary. Full load remains running on the qualified 0dc04a6 executable; no rate, cap, timeout or assertion was relaxed. Completed-attempt percentiles include HTTP errors; busy drops have no completed-request latency. Delivered RPS excludes successful requests finishing during drain. Direct full LFS object downloads are verified, not clone-side hydration of uncommitted LFS pointer files.

Verification receipt SHA-256: 1447486483a253c3fad18000a73eaa449dbaa26a1223903c074497d2b431bd4a; independent audit: 2b1a256a40cec3bcf24359a21559da8b35927d697ec0e25ba5a42982cdb030d0; verified backup manifest: 783902ec100a41e7882c73478a98b912b06b6ec0293907146a60a7dd7d8b8078.

The updated full-corpus checkpoint contains the completed-phase outcome table, six creation throughput/latency windows, stock-Git ref-listing results, the historical ten-window metadata table, closed critical artifact bindings and remaining gates. The first two windows' service tails dominate dispatch delay, and proxy connection counts were balanced; these observations do not prove a Directory or RustFS bottleneck. No performance fix or matched speedup is claimed.

Open correctness and performance gates

The target remains 10,000 identities / 100 populated Git-LFS fixtures / three nodes and a proxy / 108 windows / 114,960 arrivals / 8,640 scheduled seconds. No matched speedup or isolated Linux reference capacity is claimed.

• Original-corpus pre-load remote Git/LFS verification is now closed and independently audited on the qualified executable. Full new-release admission and controlled activation closed at Ready revision 9. Fresh-owner recovery of the original corpus and every new load ACK remains open; neither the remote check nor activation is a general upgrade controller.
• Migrate remaining legacy test port reservations. Linux PR and push CI at the tested production head eaebfc4 both passed. Both Rust jobs and both Python harness jobs at 4ddc742 passed. All four Rust/harness jobs at 2c897c7 passed. All four fresh Rust/harness jobs at documentation-only head 085a278 also passed, including Linux PR and push RustFS compatibility. Production, test, script, Cargo and live benchmark inputs are unchanged. The affected SHA-256 gap is reproduced and corrected, but a later passing job does not erase the original failure.
• Explain the diagnostic fleet's terminal lease-fencing failure. All three nodes exited 1 without forced shutdown while RustFS stayed unchanged; the new passing suites do not establish its cause or resolution.
• Complete the entire scheduled matrix and audit, fresh-owner recovery of the original corpus and every newly acknowledged write, and explicit concurrent fault coverage. The current critical-load schedule is closed at 19/20 OK: all nineteen attempted receipts are complete, but its one busy drop remains a failed arrival.
• Complete independent qualification of the current 1914096 PR pin. Its pin-only candidate passed Linux debug CI: 244 top-level Rust tests (nine ignored), 84 Python tests, all eight fresh RustFS compatibility gates, formatting, all-target lints and the debug server build. The Linux release candidate passed 244 top-level Rust tests (nine ignored), 84 Python tests, all eight fresh RustFS gates, release lints and the executable build. Its retained archive, source hashes, metadata and binary checksum were independently verified and copied to a second local filesystem. Latest-head debug CI passed, but release CI failed the disconnected-admission test described above. Native qualification, retained-store recovery and matched performance remain open. Do not relabel the tested 0dc04a6 results.
• Qualify higher admission profiles, uniform/skewed active sets, independent operation rates/concurrency, ref-only/fresh-object pushes, matched comparisons, non-sparse five-GiB transfers and CPU/cost scope on an isolated Linux runner.

Historical failed performance and missing raw ledgers remain documented. New tests cannot certify missing historical full-corpus/every-ACK recovery. Retention and read-only catalog admission are not an upgrade controller.

See the listener handoff verification, actual old-binary upgrade and recovery checkpoint, latest Cellule checkpoint, retained-catalog checkpoint and full performance plan.

Latest PR update

Head 5581d5c brings the latest Cellule pin and the release-correctness workflow into this PR, without changing the running campaign's checkout, workload, binary or provider.

• Current pin: 191409685b001a82bd02780def45102b4fc2f164; all six lockfile packages resolve to it. Only five direct pins and six lockfile sources change; no other dependency versions or runtime budgets change.
• Candidate Linux debug CI passed 244 top-level Rust tests (nine ignored), 84 Python tests, all eight fresh RustFS gates, formatting, all-target lints and a debug server build.
• Candidate Linux release CI passed and its retained artifacts were independently audited: 244 top-level Rust tests (zero failed, nine ignored), 84 Python tests, eight release RustFS gates, formatting, release lints and the executable build. Linux binary SHA-256: 5ff3a4daf2c2012357b4643b896e4f89b8402f7cad9c29679ea20c12becc880d. All 267 bound source/workflow files match the tested commit; the current PR has identical production, test, script and Cargo inputs. The archive digest was verified against GitHub, and 17 files were copied and independently reread on another local filesystem. The first setup failure remains retained; native/recovery/performance proof is not inferred.
• The new workflow also runs on pushes to this PR branch and retains source hashes, dependency metadata, provider identity, logs and the release executable, including available evidence from failed runs. Fresh PR-head debug checks passed; release CI failed. The earlier candidate pass does not establish that the failure is resolved.
• Local publication checks passed: all 84 Python harness tests, formatting, workflow YAML/Bash syntax, 34 local documentation links, Markdown fences and diff whitespace. Production/test/script/Cargo inputs exactly match the pin-only debug candidate. All 638 frozen campaign input bindings remain unchanged.
• Qualification docs distinguish the new PR pin from historical 0dc04a6 release, activation and benchmark evidence. Historical failed arrivals remain unchanged.

Remaining gates: native latest-pin verification, retained-store/every-ACK recovery, the complete performance campaign, matched comparisons, non-sparse five-GiB transfers and isolated Linux reference capacity are not yet closed.

Documentation evidence refresh

Head 0ba7775 updates the three qualification/performance documents with the closed release-candidate results, the retained PR-head failure and the audited diagnostic. Production, tests, Cargo, scripts and workflows are unchanged from 5581d5c; the diagnostic-only workflow is not added to this PR.

• Local checks passed: 84 Python harness tests, 34 local documentation links, balanced Markdown fences and diff whitespace.
• All 638 frozen live-campaign bindings remain unchanged; no workload, provider or runtime budget was changed.
• Diagnostic: 100 isolated passes; each of two full-target runs passed 104 tests, zero failed, nine ignored. The original failure and skipped release gates remain recorded. No full-workspace, provider, recovery or performance pass is inferred from the diagnostic.
• Diagnostic audit SHA-256: ccd848c30d08b75e0958ff3b973e9416df027d1ebb81cd2fa685eca7f6ce018f. All 115 evidence files were copied and independently reread on a second local filesystem, not an off-machine backup.

Complete baseline campaign and owner-loss verification

The unchanged 108-window / 114,960-arrival / 8,640-scheduled-second baseline has closed on the separately qualified 0dc04a6 binary and original RustFS provider. Every report, arrival ledger and resource boundary was independently audited. It recorded 59,554 OK / 55,406 failed arrivals; this is a failed arrival gate, not capacity or a matched speedup. The separately scheduled critical work remains 19/20 OK.

• Complete positive ACK inventory: 1,464 creations / 1,023 ref-only pushes / 467 fresh-object pushes / 242 LFS uploads / 19 critical workflows (38 repositories). All 11,504 original/acknowledged repository identities are disjoint. The terminal inventory and 464 closed files were copied and independently reread; 6,493 previously closed window/workflow files were reread as well.
• The complete 1-MiB push phase recorded 150/1,200 OK, 855 busy drops, 186 Git errors and nine timeouts. All 150 positive ACKs and 157,286,400 declared payload bytes were reconciled. The two 16-client / 4-push/s windows had successful-only p95 latencies of 70.764 / 100.998 seconds, with 0.117 / 0.050 successful in-window pushes/s. Completion counts include drain; delivered in-window rates exclude it.
• Complete LFS downloads: 900/1,200 OK, 297 busy drops and three HTTP 503s. Uploads: 242/1,200 OK, 626 busy drops, 174 transport errors, 157 HTTP 503s and one HTTP 500. Upload ACK OIDs were independently reconstructed from the declared deterministic bodies. Post-loss full-body verification is separate.
• The exact three original owned gateways were removed only after closed evidence and every-ACK preflight passed. All three exits were SIGKILL without forced cleanup; 32.008191 seconds of confirmed owner absence was recorded. Original RustFS identity, start time, configuration and resource envelope were unchanged.
• The first fresh launch failed a helper readiness/metrics ordering check and drained all three nodes normally. Its failure, logs and configuration are preserved. A separate controller with nine passing guard tests waits for metrics from the same live child within the original startup deadline, without respawn or deadline widening. A new directory and new owners then reached readiness. The read-only verifier is now running against the full original corpus and every ACK; recovery is not yet established.

Full campaign audit SHA-256: 23836f6a437826a58e5157b534a17346b939e9c4e7213168b1c872c19f387f59; complete ACK inventory: 9853c5f32551980165d62e06281e163a26d435fde070984883fe8c388758a0de; owner-loss receipt: 40cb5aa9068b8f32cffb51433682af6c8fa4ce36b4a2a856543c753018f0addb.

Fresh PR checks at 0ba7775 passed, including release correctness; the exact release attempt was preserved. This does not erase the earlier disconnected-admission 503. The separate stage-context diagnostic preserved all seven declared cases: the disconnected-admission test passed in all seven; one serial full-target case instead failed ssh::sha256_ssh_push_and_clone with AddrInUse (OS error 98). Its independent audit SHA-256 is ba4d93864f31e4e507c8b04130e460cfa0c88df3c30907eefdd98b5364ffad75. This is a distinct unresolved bind failure, not reproduction or resolution of the residency 503. Diagnostic test context remains outside this PR.

The current 1914096 pin is not the baseline runtime. Its native retained-store recovery, matched performance, concurrent faults, higher admission profiles, non-sparse five-GiB transfers and isolated Linux capacity remain open. Failed arrivals, failed startup attempts and CI failures remain retained.

[forhappy]

Original-corpus remote verification passed

The admitted original RustFS corpus now passed remote verification through three fresh qualified 0dc04a6 gateways and the proxy. This is the retained original data, not a reseeded fixture.

Closed check	Result
Corpus identities	All 10,000 exact repository UUIDs
Populated Git fixtures	100 fixtures, 200 ordinary stock-Git v0/v2 clones
Git content	Exact HEAD/base commits, README/incremental hashes and strict full fsck
LFS	All 100 direct full-body responses matched declared size and SHA-256; these fixtures do not contain LFS checkout pointers
Original critical fixtures	Both exact identities; four v0/v2 mirrors, exact refs, payload, notes and strict full fsck
Independent audit	All 10,002 identities, 100 LFS digests and 824 stock-Git commands reconciled; all 200 clones and four mirrors checked locally again
Runtime boundaries	All 633 bound inputs unchanged; actual fleet processes/configurations and fresh proxy metrics checked at stage boundaries; original RustFS configuration/resource envelope unchanged
Preservation	All 7,061 closed files and immutable inputs copied and independently reread on another local filesystem

Verification closed October 2 at 03:02:09 UTC (October 1 Pacific). The full-corpus stage took 756.929 seconds, including all client clones and validation; this is verification wall time, not scheduled throughput or a latency benchmark.

• Verification receipt SHA-256: 1447486483a253c3fad18000a73eaa449dbaa26a1223903c074497d2b431bd4a
• Independent audit SHA-256: 2b1a256a40cec3bcf24359a21559da8b35927d697ec0e25ba5a42982cdb030d0
• Backup manifest SHA-256: 783902ec100a41e7882c73478a98b912b06b6ec0293907146a60a7dd7d8b8078

Receipts remain at /Users/haipingfu/.codex/canopy-original-corpus-admission-YxGodu/upgraded-original-verification; the verified copy is /Volumes/Workspace/CrabData/canopy-original-upgraded-remote-dooxekr0. This is not an off-machine or live provider-data backup.

Full load campaign is running, not qualified

The unchanged 108-window / 114,960-arrival / 8,640-second plan is now launched on the same bound fleet. Its mandatory full preflight is running. Twenty scheduled critical workflows (300 seconds, 15-second interval, cap four) will overlap the first admitted load windows. Outputs are retained at /Volumes/Workspace/CrabData/canopy-original-full-0dc04a6-j6promvd; no completed load or performance result is claimed yet.

A separate clean checkout at 63c93b0 stages Cellule 191409685b001a82bd02780def45102b4fc2f164. Locked metadata resolves all six packages to that revision, with only five manifest pins and six lockfile sources changed; formatting passed. Its release, RustFS and performance qualification remain open. No compilation is running alongside the campaign, and no result from 0dc04a6 transfers to it.

Both Rust and both Python CI jobs at PR head 2c897c7 passed. PR remains draft: every newly acknowledged load write after owner loss, concurrent fault coverage, higher profiles, matched comparisons, large transfers and isolated Linux capacity still require verification. The earlier diagnostic lease-fencing failure remains unexplained.

[forhappy]

First six full-campaign windows are audited and preserved

This is a failed diagnostic baseline so far, not a capacity or speedup pass. The unchanged 108-window campaign continues on qualified Cellule 0dc04a6; no workload rate, concurrency, assertion or runtime budget has been relaxed.

All six closed windows offered 20 requests/s for 120 seconds with concurrency 32 over 100 active identities:

Window	Successful / scheduled	Busy drops	Delivered RPS within window	Scheduled p95 / p99
Uniform 1	2,344 / 2,400	56	19.5	1,447.299 / 3,198.533 ms
Uniform 2	2,392 / 2,400	8	19.9	732.404 / 1,556.094 ms
Uniform 3	2,400 / 2,400	0	20.0	387.751 / 714.107 ms
Skewed 1	2,400 / 2,400	0	20.0	169.310 / 388.415 ms
Skewed 2	2,400 / 2,400	0	20.0	206.813 / 395.493 ms
Skewed 3	2,400 / 2,400	0	20.0	297.194 / 611.530 ms

The 300-second concurrent critical schedule is closed and independently audited: 19/20 successful workflows, one busy drop, 323 successful critical steps, 38 acknowledged repository identities. One workflow completed after its schedule window; delivered workflow throughput was 0.060000/s in-window and 0.060223/s including drain, with attempted-workflow p50/p95/p99 37.089/60.551/60.551 seconds. Times include stock-Git work and validation. The dropped workflow has no receipt or writes; all nineteen attempted workflows have complete successful receipts. Their owner-loss recovery is still required.

• Critical report SHA-256: 66a1468b8e51b2b4254813d06af12e04e802fa5598512f6bb0081cc484d16285
• Critical sample-ledger SHA-256: 1ae9c44266cb49eeffaec80406ac74f3e3a342e6ab5b07fc42761a99fdb9988a

The read-only watcher audits each sealed window's exact arrival sequences, deterministic selection, outcomes, percentiles, throughput and resource bindings before copying its report/ledger/resource files to another local filesystem. Every finished workflow attempt is retained too, including failures and its local Git data when present. In-progress outputs are not copied or qualified.

Copies and manifests are at /Users/haipingfu/.codex/canopy-original-full-window-copies-qknece2i; the separately closed critical audit is at /Users/haipingfu/.codex/canopy-original-corpus-admission-YxGodu/closed-critical-load-audit.json, with its verified final report/source copy at /Users/haipingfu/.codex/canopy-original-closed-critical-dved6eyf. These are local backups, not live provider-data or off-machine backups.

Diagnosis remains open. In the first two windows, dispatch p99 was only 12.760/20.340 ms versus service p99 3,190.791/1,550.308 ms. Proxy connection deltas were balanced (85/85/84 and 106/106/105), with zero rejections/errors. Those facts weaken timer scheduling and connection-count imbalance as dominant explanations; they do not prove a Directory or RustFS bottleneck. No fix is claimed. The previous independent lease-fencing failure also remains unexplained.

The full matrix is now proceeding to its 500-identity active sets. All running-fleet source bindings remain unchanged. Full scheduled completion, every newly acknowledged write after owner loss, explicit concurrent fault coverage, higher admission profiles, matched comparisons, large transfers and isolated Linux capacity remain open. The newer 1914096 candidate remains isolated and unqualified beyond locked metadata/format checks.

[forhappy]

Read-only failure diagnosis: first thirteen sealed windows

The unchanged full baseline is still running; this is not a passing capacity result. An offline analysis reread all 52 copied report/ledger/resource files for exactly the first thirteen windows and revalidated the immutable live-run bindings.

• 30,566 OK / 31,200 scheduled, 608 driver-busy drops, 26 HTTP 503s.
• Every one of the 608 busy arrivals coincided with 32 executing request-body intervals. These intervals exclude executor queueing and result/permit-release overhead; this is evidence of actual client-slot saturation, not a reason to increase the cap or omit dropped arrivals.
• The concurrent critical schedule drained at 03:19:45.663 UTC. Windows entirely after that point recorded 23,430 OK, 544 busy drops and all 26 HTTP 503s. Critical writes are therefore not necessary for the failure; this is not proof that they have no effect.
• Captured server-log prefixes and HTTP 503 counts agree by UTC window. The logged ServerError::Directory wraps InvocationError<Vec<SqlResultSet>> from both Directory and Repository SQL calls. The text “repository directory operation failed” does not identify the shared Directory Cell as the source or reveal the inner rejection class. Log prefixes are explicitly partial observations, not closed full-run logs or request-ID attribution.
• Client dispatch p99 remained below 25 ms in the later analyzed windows while service p99 reached roughly 2–4 seconds. Front-proxy connection-count deltas remained balanced with zero proxy errors/rejections. These observations weaken client dispatch and connection-count imbalance; they do not exclude scheduling inside service, per-request owner skew, residency churn or storage stalls.

The next separately bound replay will use the qualified binary's existing bounded request/stage tracing to distinguish repository transition time from Directory lookup/authentication time. That follows full-baseline completion and mandatory original-corpus/every-ACK recovery; no logging, source, resource budget, rate, timeout, cap or assertion was changed on the running baseline. No bottleneck fix or matched speedup is claimed.

Analysis: /Users/haipingfu/.codex/canopy-original-corpus-admission-YxGodu/sealed-metadata-13-necisxpg/analysis.json

SHA-256: 35c3c8623d825c9161399d3f13c64047fe91f57bc1c6a37bb53fe8fd3baaadda

The analysis, exact analyzer and three captured log prefixes were copied and independently reread on a different local filesystem at /Volumes/Workspace/CrabData/canopy-sealed-metadata-13-o4epm4bg; five files, manifest SHA-256 b81e83c0b419957ad24ebe1470c3925e1939b09c83ecba2e5d32982ec60ee804. This is local evidence preservation, not a provider-data/off-machine backup.

The full 108-window / 114,960-arrival plan, concurrent faults, every-ACK recovery, newer Cellule release/RustFS qualification, higher admission profiles, matched comparisons, large non-sparse transfers and isolated Linux capacity remain open.

[forhappy]

Repository creation windows and terminal evidence gate

The full baseline remains active and unchanged. 22 windows are now sealed, audited and copied: all eighteen metadata windows and the first four creation windows. Results below are creation-only observations on the shared Mac/Colima RustFS diagnostic, not isolated capacity or matched improvement.

Creation window	Offered RPS / client cap	OK / scheduled	Busy / HTTP 503	Delivered RPS in-window	Scheduled p50 / p95 / p99 (ms)
1 RPS, repetition 1	1 / 16	120 / 120	0 / 0	1.000	481.603 / 3,205.203 / 5,940.919
1 RPS, repetition 2	1 / 16	120 / 120	0 / 0	1.000	385.347 / 5,220.908 / 6,795.877
1 RPS, repetition 3	1 / 16	120 / 120	0 / 0	1.000	816.248 / 5,282.604 / 7,891.099
5 RPS, repetition 1	5 / 16	394 / 600	204 / 2	3.150	2,509.941 / 11,488.210 / 13,053.231

Each window retained its declared 120-second schedule, 30-second HTTP timeout and unchanged runtime/node budgets. Busy drops remain failed arrivals with no fabricated latency; completed-attempt percentiles include HTTP errors. Drain completions count toward ACKs, not in-window RPS.

The stock closed-ledger validator reconciled all 754 positive creation ACKs across these four windows: canonical UUIDs, exact run/sequence-derived names, global uniqueness and no collisions with the original 10,000 identities or the nineteen critical workflows' 38 UUIDs. All copied finalized report/sample/resource files were independently reread. This is ACK-input integrity, not remote verification after owner loss. HTTP failures are not asserted to mean rollback or absence of persisted state.

The new read-only terminal evidence helper is live at PID 96486, waiting for the existing supervisor, matrix, critical child and copy watcher to be absent with complete terminal receipts. Its four pure tests (18 cases including subtests) passed, and it correctly refused terminal admission against the actual live campaign. It makes no provider requests and sends no server signals. Only after all 108 windows close will it run the independent full-ledger/resource audit, reconcile the complete creation/write/critical ACK inventory and preserve finalized inputs. Incomplete or changed ledgers refuse admission; failed performance does not excuse missing correctness recovery.

Helper qualification receipt: /Users/haipingfu/.codex/canopy-original-corpus-admission-YxGodu/full-campaign-terminal-helper-checks.json, SHA-256 6e8496b1c862d1ec6d18e8b43e953184b9b5d04decb8df0ab60aef24f5a93347. Three exact helper/checker/receipt files were copied and reread on another local filesystem at /Volumes/Workspace/CrabData/canopy-terminal-evidence-helper-bkxs2_ph, manifest SHA-256 0613747784d18b53f3f8088d1b8e4ca0bddb1e9326ffe18d08777db134f1765c. This is local evidence preservation, not provider-data/off-machine backup.

All four fresh Linux PR/push Rust and harness CI jobs at 085a278 passed; the PR description now reflects that. The full campaign, original-corpus and every-ACK recovery after owner loss, concurrent faults, independent qualification of newer Cellule, higher admission profiles, matched comparisons, large non-sparse transfers and isolated Linux capacity remain open. No rate, cap, timeout or correctness assertion was relaxed.

[forhappy]

Creation and HTTP discovery phases closed; targeted observer prepared

The unchanged full campaign remains live. Its six creation windows and eight HTTP Git-v2 capability/discovery windows are now completely sealed, independently ledger-audited and copied. Stock-Git ls-remote windows have begun; those are separate actual-ref checks.

Closed phase	Scheduled	OK	Driver-busy drops	HTTP 503	Transport errors
Repository creation, six windows	2,160	1,464	690	6	0
HTTP Git-v2 capability/discovery, eight windows	57,600	10,309	9,581	37,706	4

All 1,464 positive creation ACKs were reconciled across the complete six-window ledgers: exact run/sequence names, canonical globally distinct UUIDs and no collisions with the original 10,000 identities, original critical-2 fixtures or nineteen concurrent workflows' 38 UUIDs. All 24 finalized creation report/sample/resource copy files were reread. This is input integrity, not post-owner-loss remote verification. Failed HTTP writes are not assumed to have rolled back.

Fast refusals are not a latency improvement. In the first 100-RPS uniform discovery window, all-completed p50 was 7.824 ms, while successful-only p50/p95/p99 was 1,406.853 / 4,472.563 / 6,349.508 ms. That window had 1,343 OK, 8,857 HTTP 503s and 1,800 busy drops out of 12,000 arrivals; delivered success throughput was 10.992/s. Failed attempts remain in the primary completed-attempt timing population, with successful-only timing shown separately. Busy arrivals have no fabricated latency.

The first discovery window's HTTP 503s have no corresponding routing-error logs. The transfer-admission path can refuse without such a log, so the current logs do not prove an inner SQL/storage cause. A later separately bound read-only observer now classifies exact known 503 responses into transfer admission, node readiness, repository availability, Cell availability and changing-ref snapshots; unknown bodies retain only byte count and SHA-256. It records canonical request UUIDs and static route categories, never paths, credentials, response text or exception messages. Writes/unsupported routes refuse before the client runs; it performs no retries and preserves original client outcomes.

The diagnostic observer passed seven synthetic safety/privacy/classification tests. A regression then exposed its initial 10,000-record limit as insufficient for the full 100-RPS × 120-second = 12,000-arrival declared window. The separately bound full-window adapter now passes all eight tests with every sequence retained; the workload was not shortened. The failing pre-fix source/output and passing source/output were preserved and reread on a different local filesystem. This changes diagnostic collection only—not any server, benchmark admission limit or runtime budget—and it has not been installed in the running baseline.

• Base observer qualification: /Users/haipingfu/.codex/canopy-original-corpus-admission-YxGodu/read-response-observer-checks.json, SHA-256 4904b12dc7f917a4e11a332896fe0ea8dd523fa950f289878931e14b67c2659a.
• Failed full-window check: /Users/haipingfu/.codex/canopy-original-corpus-admission-YxGodu/full-window-observer-before-1f7p_aiu/checks.json, SHA-256 9911000c755150bba86422085349edd94355a3ef57cd8d8d0b823c65e7557040.
• Passing full-window check: /Users/haipingfu/.codex/canopy-original-corpus-admission-YxGodu/full-window-observer-after-5xjc8h8y/checks.json, SHA-256 c2db734d2b0fda13faaeac93dc858fdec1a3e3df62bd4ca13135f9b99d6478f7.
• Passing source/check copy: /Volumes/Workspace/CrabData/canopy-full-window-observer-after-1x96hzad, manifest SHA-256 2a0378ea9c560002ace6153bd763d71dedf2136d671daa84796e9c01e6ef862f. Local evidence, not an off-machine/provider-data backup.

All 638 immutable launch bindings remain unchanged. The terminal evidence auditor remains live and read-only. Full 108-window completion/audit, original-corpus/every-ACK recovery, concurrent faults, newer Cellule release/RustFS qualification, higher admission profiles, matched improvements, large non-sparse transfers and isolated Linux capacity remain open. No bottleneck fix or capacity pass is claimed.

[forhappy]

Progress after the 40-window documentation snapshot: the clone and cold-fetch phases are now closed, audited and retained (56 completed windows through these phases; the full 108-window campaign continues unchanged).

Newly completed phase	OK / scheduled	Busy drops	Git errors
Stock Git clone, 8 windows	704 / 1,200	495	1
Stock Git cold fetch, 8 windows	814 / 1,200	386	0

All 64 finalized source/copy file pairs from these two phases were independently reread. Both phases vary rate (1 or 4/s) independently of concurrency (1 or 16), with two repetitions each. At concurrency 16 / 4/s, clone returned 239/240 and 240/240 OK, delivering 3.850 and 3.950 successful operations/s; cold fetch returned 240/240 in both repetitions, delivering 3.950 and 3.917/s. Busy arrivals remain failed, and successful drain completions do not count toward in-window RPS. Cold fetch means a fresh empty bare client, not a proven cold server cache. Clone checks the expected tip and README digest; full fsck remains part of separate original-corpus/recovery verification. No matched speedup or reference capacity is claimed.

The isolated latest-Cellule candidate (63c93b0, Cellule 191409685b001a82bd02780def45102b4fc2f164) was pushed to codex/cellule-main-1914096 and is running standard Linux CI: https://github.com/crabbuild/canopy/actions/runs/36965371936 . Its Python harness passed; Rust testing is in progress. Only five manifest pins and six lockfile source entries differ from its base, and 262 non-pin source files match the frozen qualified source. A six-file CI/source observation has a verified cross-filesystem copy (manifest SHA-256 182c3ce2e01874bfd5e03b504da62faac90c3f2c09fb1553ff1f73356f149675). This is debug CI, not native/release qualification or performance proof. The PR and live benchmark still use the independently qualified 0dc04a6 revision.

A separate read-only recovery preflight now guards complete terminal evidence and reconciles every future creation/Git/LFS/critical ACK inventory against its closed ledgers. A missing-owned-handle regression failed before the guard correction; all ten pure guard/identity tests now pass, and the actual live-campaign inspection correctly refuses readiness. Existing closed creation/critical ledgers reconcile 11,504 distinct UUIDs and names (10,000 originals + two original critical fixtures + 1,464 positive creation ACKs + 38 current critical UUIDs). No owner was stopped and no provider request was sent. The full post-terminal path and remote every-ACK recovery remain unverified. Failed and passing preflight evidence is retained outside the repository; verified manifests are d119704bffc96c0168a8c11bc71ae489403f6befc1f146038cbd6765271678f5 and 77abcc0216593448752be30987af9feeca58b0ecc17d88bf0e76324f5abe753a.

All 638 frozen live input bindings remain unchanged. Incremental fetch is now running; push/pull/LFS phases, complete terminal audit, original-corpus/every-ACK recovery, concurrent fault coverage, higher caps, matched comparisons, large transfers and isolated Linux reference capacity remain open.

[forhappy]

Latest Cellule candidate Linux CI is now closed and passed: https://github.com/crabbuild/canopy/actions/runs/36965371936 . Exact head 63c93b0 pins Cellule 191409685b001a82bd02780def45102b4fc2f164. Log reconciliation confirms 244 top-level Rust tests passed, zero failed, nine ignored; two nested child results are not double-counted. All 84 Python harness tests and all eight exact fresh RustFS compatibility gates passed, as did formatting, all-target Clippy and the debug server build. Native/release qualification, retained-store upgrade/recovery and performance do not follow from these debug CI results. The live fleet and PR pin remain on the separately qualified 0dc04a6 revision.

Both PR and push CI at documentation-only head ec81650 also passed all four Rust/harness jobs, including RustFS compatibility. The PR description now records those closed scopes.

Closed candidate CI/source/log evidence has nine copied and independently reread files; manifest SHA-256 880d2100bf28801b80aeaa0cd6b38bfd50abe0b263b19189b273cf03531de1c1. The independent log-accounting copy has manifest 7683fcea17f284d593cbfdea6f5975aa5e5a51c653e151c16b41a832e0f475f1. These are local cross-filesystem evidence copies, not off-machine/provider-data backups.

The original full campaign has now closed 64 windows and moved to incremental pull. The eight incremental-fetch windows finished at 771 OK / 1,200 scheduled, 418 busy drops and 11 Git errors; all 32 finalized source/copy file pairs were reread. At concurrency 16 / 4/s, the first repetition returned 211/240 OK (18 busy, 11 Git errors), delivering 3.333 successful operations/s with completed-attempt p95/p99 7,245.835/8,568.187 ms; the second returned 240/240 OK, delivering 3.883/s with p95/p99 3,847.302/4,845.643 ms. Variability and failed arrivals remain recorded, with no matched speedup claimed.

A separate complete-recovery verifier is prepared outside the repository. It gates on the full terminal audit/ACK inventory, the exact three old owners being absent for at least 32 recorded monotonic seconds, unchanged provider/deployment/budgets, distinct new owners, the same executable and fresh local-state provenance. It invokes full original 10K/100 content checks plus every creation, ref/fresh-object Git push, LFS and complete critical-workflow ACK verifier, retaining a request ledger. Eleven pure guard/accounting tests passed; ledger reconciliation also matched the actual earlier complete pre-load record (10,002 identities, 100 full LFS downloads, 824 Git commands). The real live-campaign invocation refused before reading nonexistent owner/fresh-launch inputs or creating any verifier output. No owner signal or provider request was sent. The real post-terminal/fresh-fleet path and every-ACK remote recovery have not run; this is preparation, not a correctness pass. Its four-file verified qualification copy has manifest febe3e79bc3a3be77274084b79b0d0dd5bce07224ecc6abea13a2e7092eca231.

The existing large-transfer workflow requires a dedicated self-hosted Linux runner. The repository runner inventory currently returns zero registered runners, so no unserviceable job was queued. Full schedule/audit and owner-loss recovery continue to be required before reference capacity or any improvement claim; all 638 frozen live input bindings remain unchanged.

[forhappy]

Closed verification update (no changes to the live benchmark):

• Latest Cellule 1914096 candidate Linux release CI passed. Independent artifact accounting confirms 244 top-level Rust tests / 0 failed / 9 ignored, 84 Python tests, all eight exact release RustFS gates, formatting, release lints and the release executable build. Two filtered child results were not double-counted.
• Verified the GitHub ZIP digest, all 267 bound source/workflow hashes, six locked Cellule package revisions, provider image identity, the executable TAR and binary checksum. Current PR production/test/script/Cargo inputs match the candidate. Linux executable SHA-256: 5ff3a4daf2c2012357b4643b896e4f89b8402f7cad9c29679ea20c12becc880d. Seventeen artifact/audit files have a reread-verified second-filesystem copy. This is Linux fresh-fixture correctness, not native Mac, retained-store recovery, performance or reference capacity. Current PR-head CI remains separately in progress.
• Re-audited all 8 incremental-pull windows: 743 OK / 1,200 scheduled; 457 busy drops, zero Git errors. The two concurrency-16 / 4-RPS windows each returned 240/240 OK; delivered in-window RPS was 3.900 / 3.833, with successful p95/p99 2,228.861/2,604.536 ms and 4,364.562/5,076.078 ms.
• Re-audited all 4 ref-only-push windows: 1,023 OK / 1,200 scheduled; 85 busy drops, 92 Git errors. Both 1-RPS windows returned 120/120 OK; the 4-RPS windows returned 427/480 and 356/480. Preserved all 1,023 distinct positive ref ACKs for post-owner-loss verification. Forty-eight finalized source/copy pairs, deterministic selections, arrival/timing counters and resource boundaries reconcile. Failed pushes are not assumed to have rolled back.
• The frozen driver discards Git stderr, so git_error alone cannot establish a cause. No error detail, root-cause finding or performance improvement is inferred. Instrumentation must be a separate subsequent diagnostic, not a change to the live baseline.
• Prepared the full-campaign owner-loss controller outside the repository. Six guard tests, real process/config identity checks and an actual live-load refusal passed, with six qualification files reread on a second filesystem. No gateway was signalled and no recovery performed. Execution requires all 108 windows, 114,960 arrivals, preserved terminal ledgers and every positive ACK; it will then require exact three-owner loss, unchanged provider and at least 32 seconds of confirmed absence. Concurrent-load faults remain a separate gate.

Immutable receipts: release artifact audit 1c524ecb218e2fafe2f27f98fac96673e0de2a9bbc047773d569bc0d12868c3b; pull/ref-push audit 439e4688e0043adc1918fd9333e34206b1617b74b4213c561a71352b9c38b882; owner-loss guard qualification 89ff5d1c25d971771df7f952e174e4d71af05959fa8a7fa98f1aed56a95e3ba2. All 638 frozen campaign input bindings remain unchanged. The full campaign is live and measuring fresh-object pushes; original-corpus/every-ACK recovery, matched comparisons, five-GiB transfers and isolated Linux reference capacity remain open.

[forhappy]

New negative correctness evidence and closed push accounting:

• At current PR head 5581d5c, release CI failed residency::faults::disconnected_admission_finishes_release_and_allows_a_later_restore, returning HTTP 503 from repository creation. Observed top-level results before Cargo stopped: 240 passed / 1 failed / 9 ignored; this is not a complete workspace pass. The later harness, RustFS and release-binary steps were skipped. All failed logs and the exact artifact ZIP (digest verified against GitHub) have a reread-verified second-filesystem copy. Both debug PR and debug push workflows passed: each Rust job has 244 top-level tests, nine ignored, and eight fresh RustFS gates; both 84-test harness jobs passed. These successes and the earlier candidate release success do not erase the release failure.
• Launched an unchanged-source release diagnostic, not retry-until-green: 100 isolated repetitions of the exact failing test, then two original full-target runs at four test threads. All repetitions and the exact release test executable are retained. The first diagnostic stopped before tests because the shallow checkout lacked its baseline commit; that failed setup is preserved. The corrected run passed setup and source equivalence and is executing the declared repetitions. The diagnostic branch changes only its workflow; no Canopy, test, dependency or benchmark input is changed. No root cause or repair is claimed.
• Re-audited the entire 256-KiB fresh-object-push subphase / eight windows / 1,200 arrivals: 317 OK, 736 busy drops, 147 Git errors. Preserved all 317 distinct positive ref/commit ACKs, their original parents and payload declarations: 83,099,648 acknowledged new Git payload bytes. This excludes protocol overhead and is not a remote recovery proof. Both concurrency-16 / 1-RPS windows returned 60/60 OK, but concurrency-16 / 4-RPS windows returned only 23/240 and 48/240; successful p95/p99 was 71,213.366/73,063.333 ms and 58,241.442/65,166.040 ms, with delivered in-window throughput 0.183/0.600 RPS. Thirty-two finalized source/copy pairs, deterministic selections, timing/resource and positive-payload accounting reconcile. The frozen driver still has no retained Git stderr; failed writes are not assumed rolled back.
• Prepared the fresh-fleet supervisor and immutable handoff outside the repository. Six guard tests and a real live-load refusal passed. It requires closed full-campaign/every-ACK evidence, exact three-owner loss, >=32 seconds confirmed absence, unchanged original provider/budgets, the same native executable and absent local scratch. No owner was signalled, no server started and no recovery performed. The actual post-terminal launch/recovery path remains unverified.

Immutable receipts: CI contradiction audit 1501ebf40273c369412e3c253c4a9f4acd3112acf85615d51e34f4f9606907b1; 256-KiB push audit 054b21bea943e2e6db806f9fc23fd51ba321aa36bef5568d3f855f2d98cdd934; fresh-launch guard qualification d920974678e8a4bd7defc4c62859ff8a15cb3937435fab259664abc7f91cc699. All 638 frozen campaign input bindings remain unchanged. The baseline is live on one-MiB pushes. Full 108-window closure, original-corpus/every-ACK recovery, concurrent faults, matched comparisons, five-GiB transfers and isolated Linux reference capacity remain open; no performance improvement is claimed.

[forhappy]

Updated this PR to 0ba7775 with the latest audited qualification status. All 84 local Python tests passed; documentation links/fences and diff whitespace passed. All 638 live-campaign inputs remain unchanged.

The unchanged-source release diagnostic passed 100 isolated runs and both full-target runs (104 passed, zero failed, nine ignored each). The GitHub archive, all 264 source inputs, retained ELF and every result/log binding were independently verified; 115 evidence files were copied and reread. This does not clear the original release failure, establish a fix, or qualify recovery/performance.

The update is documentation-only. No production code, assertions, dependencies, workflows, workload budgets or provider state changed. Fresh PR-head CI is separate.

[forhappy]

The complete unchanged baseline is now closed and independently audited: 108 windows / 114,960 arrivals / 59,554 OK / 55,406 failed arrivals. Every positive ACK is preserved: 1,464 creations, 1,490 Git writes, 242 LFS uploads and 19 critical workflows. This is not a performance or capacity pass.

After the closed-evidence preflight passed, only the exact three owned gateways were removed. 32.008191 seconds of confirmed owner absence was recorded, with RustFS unchanged. A first fresh-start observer race was preserved and all its nodes drained normally; a separately qualified, bounded same-child metrics wait allowed a new attempt to reach readiness. Full original-corpus and every-ACK read-only verification is running; recovery is not yet proven.

Fresh PR-head checks are green, but the earlier residency 503 remains unresolved. The seven-case stage-context diagnostic did not reproduce it: the target passed all seven cases. A different SSH SHA-256 bind test failed AddrInUse in one serial full-target run; that failure and all cases were independently audited and preserved. No production change, assertion relaxation, workload adjustment or provider restart was made.