feat: add agent-based install, update, repair, and maintenance support - #48
Open
danielsreichenbach wants to merge 35 commits into
Open
feat: add agent-based install, update, repair, and maintenance support#48danielsreichenbach wants to merge 35 commits into
danielsreichenbach wants to merge 35 commits into
Conversation
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
danielsreichenbach
changed the title
- Apply rustfmt across workspace (line wrapping, import grouping)
- Fix clippy lints: format_push_string -> write!, map_unwrap_or ->
map_or, write_with_newline -> writeln!, needless_range_loop ->
iterator, useless_conversion removal
- Fix 12 failing tests in cascette-client-storage:
- archive_file: always remap mmap after file growth (64MB threshold
prevented remaps on small test writes)
- segment/archive_file: fix key round-trip expectations (only first
9 bytes survive LocalHeader reverse+zero-pad)
- segment/archive_file: compute checksums before validation
- index/update: correct capacity assertion from 60 to 12 pages
- Update lz4_flex 0.12.0 -> 0.12.1 (RUSTSEC-2026-0041)
- Update rustls-webpki 0.103.9 -> 0.103.10 (RUSTSEC-2026-0049)
- Run cargo update for 50 semver-compatible dependency bumps
- clap 4.5 -> 4.6 - tokio 1.49 -> 1.50 (workspace + WASM-target overrides) - lz4_flex 0.12 -> 0.13 - gloo-timers 0.3 -> 0.4
- mockall 0.13 -> 0.14 - notify 6 -> 8 - sysinfo 0.32 -> 0.38 - axum-server 0.7 -> 0.8 No code changes required. All 1976 tests pass.
- BLTE: "Block Table Encoded" -> "Block Table Encoding" (canonical term) - TACT: "Tooling And Content Technology" -> "Trusted Application Content Transfer" Terminology standardized across wiki, management, and cascette repos.
Two issues found by testing cascette-agent against the WoW 1.13.2 client under WINE: 1. Historical builds with pinned build_config and cdn_config triggered concurrent BTS bootstrapper and launcher installs. The BTS pipeline resolved the current live BTS version and wrote its own .build.info with product="bts" and cdn_path="tpr/bnt004", overwriting the game's .build.info. The client then rejected the install because the version string didn't match "1.13.2.31650". Fix: skip resolve_bts_configs() entirely when custom configs are provided. 2. The V7 IDX header segment_size field was written as 1 << 30 (1 GB) instead of (1 << 30) * 256 (256 GB). The Blizzard agent writes the total addressable space (single segment size * max archive count), not just one segment. The client uses this to compute archive file indices from offsets. Fix: multiply by 256. Also updates PROMPT.md to let the user choose CDN source (local mirror, project mirror at tactic.wowemu.dev, or custom). A third issue remains: the CASC data file layout differs from the reference (data.000 starts with zeros, total size ~200 MB less). With the reference Data/ directory swapped in, the client boots to the login screen. This needs further investigation in the ArchiveManager.
…rades, remove 3 unused deps - cargo update: 146 semver-compatible patches (tokio, axum, hyper, clap, etc.) - lz4_flex 0.13 -> 0.14 (trivial, no API change for block::compress/decompress) - tower-http 0.6 -> 0.7 (no API change for CorsLayer, TraceLayer, CompressionLayer) - mockall 0.14 -> 0.15 (test-only, additive change) - Remove unused deps: salsa20, cipher, asn1 - Fix clippy: assert! -> assert_eq! in cascette-ribbit server.rs test Deferred: crypto group (digest/sha2/md-5) blocked by rsa 0.10 rc; ASN.1/PKI group (der/x509-cert/cms) blocked by cms 0.3.0 pre-release; turso 0.4->0.7 engine swap needs dedicated testing.
- cascette-crypto: salsa20_vector example emits a Salsa20 test vector for cross-verification with the C++ Salsa20Cipher - cascette-formats: gen_lz4_cross/decode_lz4_cross, gen_encrypted_blte/ decode_enc, encrypt_vector examples prove BLTE interop with C++ CascLib - tools: add wow_classic_titan 3.80.2.68943 to generate_wow_paths.sh and verify_mirror_sizes.sh (1061 builds); drop stale per-product count comments - CHANGELOG: document the new examples and tool update
danielsreichenbach
force-pushed
the
feat/new-crates
branch
from
August 4, 2026 10:11
7d982e2 to
d2e256b
Compare
- Rotate 5 builds with new CDN configs (d2a7d30a → 9a824cce) - Add wow 12.1.0.69214 and wow 12.1.0.69273 - Sync comments to match values
- New build detected in DB after upstream sync
…le headers The client's BuildLocalFileHeader stores the complete reversed encoding key (not the 9-byte truncated form cascette-rs wrote) and writes flags=0 for normal data entries (flags=1 only for segment reconstruction headers). Verified byte-identical against a client-written 1.13.2.31650 store: key field 1b856f765a48ae137418c07d2dd0ca59 for ekey 59cad02d7dc0187413ae485a766f851b, enc_size 13061932 including the 30-byte header, checksum_a/b matching. LocalHeader::new now takes an explicit flags argument (0 = data entry, 1 = reconstruction header); SegmentHeader::generate and the dynamic container reconstruction-header update pass 1.
- DB dropped wow_classic_era 1.13.x builds (42 removed total) - Rotate 6 builds with new CDN configs (cc values) - Add wow 12.1.0.69299 - Preserve script structure: only build/comment lines touched
Patched VFS shards (1.15.4+) store the CFT as a flat array of 9-byte ekeys; span cft_offsets are multiples of ekey_size (9), not the flag-derived entry size (27 with flags=0x7). resolve_path() used a linear search over flag-stride entries, dropping every rem-9/rem-18 span and capping resolution at 44k/130k paths. - TvfsFile::resolve_path now returns Option<Vec<u8>> (the ekey) via a new ContainerFileTable::ekey_at() that reads bytes at any offset - cft_stride() derives the true entry stride as the GCD of all span cft_offsets (fallback: flag-derived entry size for degenerate cases) - Added gcd() helper, test_resolve_path_flat_cft and test_gcd - Removed 4 leftover scratch tests (cft_raw_check, vfs_raw_check, vfs_resolve_check, vfs_table_check) that hardcoded /tmp/patched_vfs32.blte Workspace: 2080 tests pass, 0 failures.
Ports cascette-py dc0dfb8 + 17339be to the Rust install pipeline. - root_content::fetch_loose_build_files: fetches the build-config loose files the client reads on a fresh install but the download manifest omits — vfs-root/vfs-N TVFS manifests (1.14.4+), the patch-index manifest, and the patched VFS shards named by patch-config 'patch-entry = vfs:' records (1.15.4+). Without these the client re-fetches them on first start (149 requests on 1.14.4.51001, 5382 on 1.15.4.56738). - root_content::fetch_root_content: resolves the root manifest's content set (TVFS CFT ekeys, or MFST/TSFM content keys through the encoding table), intersects with the tag-selected download ekeys, and fetches missing content via archive range + loose fallback (skips keys already in the store). On 1.15.4.56738 the root enumerates 214k keys; without this the client re-fetches ~544 files (7200 requests, no login). Both run in the Downloading state after the CASC phase. Failures are per-file warnings, not fatal. Adds parse_vfs_shards helper + 2 unit tests. Workspace: 2082 tests, 0 failures.
- Rename gen's CDN to CDN_URL; make CDN_URL/CDN_PATH overridable in all scripts - mirror-tact-from-cdn: use MIRROR, CDN_URL, OUTDIR (was CDN_MIRROR, PATHS_DIR) - Document environment variables in all three headers
The prior format (8-byte LE decompressed-size prefix + single LZ4 block) matched no Blizzard binary: the Battle.net-Setup binary (RE via Ghidra, DecoderZ::Decode 0x4e65b0) parses a 10-byte header — format 0x01, BE32 compressed total, BE32 decompressed total, block-size exponent k with 0x400 < 2^k <= 0x10000 — followed by back-to-back raw LZ4 blocks with a 64 KiB dictionary window across blocks. The Agent binary stubs BLTE '4' (error 5), so its codec was never a valid reference. - compress: 16 KiB blocks, 64 KiB dict-tail window, header-only for empty - decompress: manual LZ4 token walk per block + decompress_with_dict - rejects bad format bytes, out-of-range exponents, size mismatches - cross-validated: gen_lz4_cross output decodes identically with the battle.net-setup C++ decoder (single- and multi-chunk) Tests: cascette-formats 739+, client-storage 234 pass.
- Add --product-config <hash> to 648 build lines that have a product config in the cascette-py DB; comment gains pc=<hash> - build_file_tree already supports --product-config and emits the config from tpr/configs/data on the official CDN; no binary change needed - 7 builds with empty-string product config left without the flag
Add #[allow(clippy::unused_async_trait_impl)] to methods in the Container trait implementations across all container types, and to the CdnClient fetch methods that implement a trait with async methods.
Remove unused BinReaderExt import and convert all trait method calls to explicit binrw::BinReaderExt qualified calls to satisfy clippy.
Use is_multiple_of() instead of manual modulo check. Move gcd() to module level before tests. Simplify cft_stride if/else expression.
Reorder imports alphabetically. Wrap long function calls and log macros for readability. Add implicit_hasher allow on HashSet param.
dump_segments: rename size_with_header -> encoded_size field. dump_shmem: improve shmem detection, add per-field round-trip check, print unmodeled client fields for debugging. local_verification: support env vars for hash overrides. build_file_tree: report missing files as MISSING instead of ignoring.
Full CAS container storage dump: ordered walk of every data.NNN file cross-referenced against the IDX index (KMT). Validates segment headers, reconstruction header checksums, key matches, contiguity, and reports per-bucket stats.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request
Summary
This PR delivers the ability to install, update, verify, repair, and maintain Blizzard CASC/NGDP game installations from Rust. It also adds a compatible Blizzard Agent HTTP API server on port 1120, so existing tooling that speaks the agent protocol can interact with cascette-rs.
New crates:
product.dbprotobuf formatExtended crates:
Type of Change
Related Issues
N/A
Testing
Unit and integration tests included per crate. CDN fixture tests cover real data from WoW Classic Era, WoW Classic, WoW Retail, and StarCraft 2.
Tested On
WoW Versions Tested
Quality Checklist
By submitting this PR, I confirm that: