[SO-338] Fingerprint the cached deployment-manager binary - #384
Open
greenpeppers100 wants to merge 1 commit into
Open
[SO-338] Fingerprint the cached deployment-manager binary#384greenpeppers100 wants to merge 1 commit into
greenpeppers100 wants to merge 1 commit into
Conversation
redeploy-contract downloads a prebuilt `deploy` from a fixed prerelease asset (deployment-manager-bin/deploy-linux-x64). The name is fixed and `--clobber`-ed, so nothing tied the cached binary to the source it was built from — a cache with no key. That cost us run 30772017011. The asset was built 2026-07-26; SO-337 (#383) migrated the backend off JSON-RPC on 2026-08-02. The workflow downloaded the pre-migration binary and ran it against fullnodes that had already switched JSON-RPC off: 0: building Sui client for testnet 1: MethodNotFound — JSON-RPC on public fullnodes has been deprecated while `crates/sui-tx/src/sui_client.rs` in the very same checkout was already gRPC-only. `rebuild_manager` existed the whole time, and that is the point: a cache whose only invalidation is a human remembering to tick a box is not a cache. So key the asset name on a fingerprint of the build closure — git object ids of tools/deployment-manager, crates/, Cargo.toml and Cargo.lock. A hit now provably means "built from exactly this code"; a source change misses and rebuilds itself. `services/` and `contracts/` stay out: nothing under services/ links into `deploy`, and move-publish compiles Move at run time from the checkout, so neither is baked into the binary and neither should evict the cache. Assets are pruned to the 10 newest, which also retires the old unfingerprinted one. rebuild_manager stays as an escape hatch for toolchain drift. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The failure
Redeploy run 30772017011 died 10s in at Deploy contract on-chain:
Why it is not a code bug in
stagingNeither
building Sui client fornor therpc=log field it printed exists anywhere instagingatafb124f. SO-337 (#383) replaced both:crates/sui-tx/src/sui_client.rserrors withbuilding chain client for {network}, logsgrpc_host=, and its module doc already reads "JSON-RPC (sui_sdk::SuiClient) is gone".The run summary confirms it —
Fetch prebuilt deployment-manager binary✓,Build deployment managerskipped. And the asset it fetched:Built 2026-07-26. SO-337 merged 2026-08-02. The workflow ran a pre-migration binary against fullnodes that had already turned JSON-RPC off, from a checkout whose source was already correct.
Root cause
The cache had no key.
deploy-linux-x64is a fixed asset name,--clobber-ed on every rebuild, and step 3 downloads it whenever it exists.rebuild_managerexisted the whole time — and that is exactly the problem. A cache whose only invalidation is a human remembering to tick a box will go stale, and it went stale on the highest-stakes workflow in the repo.The tempting patch is to re-run with
rebuild_manager: true. That fixes today's redeploy and leaves the trap armed for the next crate change.The fix
Content-address the cache. A new step fingerprints the binary's build closure from git object ids — no toolchain needed, exact by construction:
The asset becomes
deploy-linux-x64-<fp>. A hit provably means "built from exactly this code"; a source change misses and rebuilds automatically, then uploads under the new name.Out of the closure, deliberately:
services/— nothing under it links intodeploy, and it is where nearly all churn lives. Including it would force a ~15-minute rebuild on every redeploy and destroy the cache's value.contracts/—move-publishcompiles Move at run time from the checkout, so Move is never baked into the binary.Verified the closure claim: no
crates/*/Cargo.tomlhas a path dep outsidecrates/.Assets are pruned to the 10 newest so the prerelease does not grow without bound; that also retires the old unfingerprinted
deploy-linux-x64. Losing an asset a concurrent run wanted just costs that run a rebuild, so best-effort deletion is fine.rebuild_managerstays as an escape hatch for toolchain drift, with an updated description.Verification
yaml.safe_loadon the workflow passes.staging:541d13db69cbaa15.gh release delete-asset <tag> <name>confirmed to exist in the installedgh.deploy-linux-x64-541d13db69cbaa15exists, so the first run after merge is a deliberate cache miss — it compiles the migrated deployment-manager from source, which is what unblocks the redeploy.cargo test -p deployment-manager --test deploy_build) running locally againstafb124f; will report before re-running the redeploy.Only
.github/workflows/redeploy-contract.ymlchanges. No Rust, Move, or service code is touched.