Implement multiple tiers caching with fallback and backfilling by Felixoid · Pull Request #2581 · mozilla/sccache

Felixoid · 2026-01-29T10:21:01Z

This PR addresses multiple requests to have tiered caching.

Closes #30
Closes #1020
Closes #2493
Closes #2566

What is added:

multilevel configuration for the caching chain
config parameter for cache writing tolerance: ignore errors, fail on write to L0, fail on a write to any level
added stats for each caching level
documentation
tests
integration tests

codecov-commenter · 2026-01-29T10:26:20Z

Codecov Report

❌ Patch coverage is 89.54352% with 197 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.18%. Comparing base (f5edaf1) to head (7045a15).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
src/cache/multilevel.rs	86.48%	68 Missing ⚠️
src/cache/multilevel_test.rs	94.29%	60 Missing ⚠️
src/config.rs	79.91%	46 Missing ⚠️
src/cache/cache.rs	70.73%	12 Missing ⚠️
src/cache/disk.rs	81.25%	6 Missing ⚠️
src/cache/readonly.rs	0.00%	3 Missing ⚠️
src/server.rs	91.30%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2581      +/-   ##
==========================================
+ Coverage   73.38%   74.18%   +0.79%     
==========================================
  Files          68       70       +2     
  Lines       37340    39205    +1865     
==========================================
+ Hits        27402    29084    +1682     
- Misses       9938    10121     +183

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Felixoid · 2026-02-08T21:10:21Z

I've done quite a few features today, like control for write error handling, and added server stats for multilevel caching.

sylvestre · 2026-02-08T21:17:37Z

this is a huge patch, can you split a bit more? thanks

Felixoid · 2026-02-08T21:20:09Z

Probably I could take something out of cache.rs, but the rest is actually multilevel.rs, and 2/3 of 2k LoC are tests.

Felixoid · 2026-02-09T09:36:10Z

Rebased to the current main.

This PR now contains the multi-level cache feature exclusively. Most of the changes are the tests and documentation, which are around 70%.

drahnr · 2026-04-14T20:22:09Z

@drahnr another round of the review, maybe? Or even merging it? ^_^

Sorry for the delay, I'll get to it tomorrow.

drahnr

The logic is sound, however, I think another pass avoiding code duplication and making the naming easier to discover without needing to read all of the docs, might be a worthy approach. This can also be done as a follow-up PR, although I am not keen to break public "config API".

Thank you!

Felixoid · 2026-04-16T09:27:39Z

Thanks for the review!

I think all points addressed so far. 👍

Felixoid · 2026-04-16T09:50:58Z

Looks like an infrastructure error https://github.com/mozilla/sccache/actions/runs/24502299252/job/71611929011

I'll try adding retries here later in another PR. A simple rerun of docker build could help

drahnr · 2026-04-17T07:11:56Z

+
+# Level 2: S3 cache
+export SCCACHE_BUCKET="my-sccache-bucket"
+export SCCACHE_REGION="us-east-1"


I'd have expected a .._S3_.. component in the path?

Same as other configs. Most probably worth changing, but I didn't touch the existing configuration.

drahnr · 2026-04-17T07:18:00Z

+                    })?,
+                    "oss" => self.oss.clone().map(CacheType::OSS).ok_or_else(|| {
+                        anyhow!("OSS cache not configured but specified in levels")
+                    })?,


nit: Can we avoid the 8x duplication here

Probably. We can benefit from having a map &str, CacheType. It's used in multilevel.rs in a similar way, but with feature gates.

I tried to implement it in 8275713

drahnr

Just nits, let's merge and move on

….0) (#40) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [mozilla/sccache](https://github.com/mozilla/sccache) | minor | `0.14.0` → `0.15.0` | --- ### Release Notes <details> <summary>mozilla/sccache (mozilla/sccache)</summary> ### [`v0.15.0`](https://github.com/mozilla/sccache/releases/tag/v0.15.0) [Compare Source](mozilla/sccache@v0.14.0...v0.15.0) ##### sccache 0.15.0 ##### Summary sccache 0.15.0 brings several notable improvements: - **Multi-tier caching**: New support for layered caches with fallback and automatic backfilling between tiers ([#2581](mozilla/sccache#2581)). - **C++20 modules**: Initial (partial) support for compiling C++20 modules ([#2516](mozilla/sccache#2516)). - **Expanded MSVC support**: New flags handled including `d1nodatetime`, `await:strict` ([#2617](mozilla/sccache#2617)), `Y-`, `YI`, `Zf` ([#2663](mozilla/sccache#2663)), and the default `.pdb` extension for `/Fd` ([#2621](mozilla/sccache#2621)). - **New platforms**: loongarch64 support ([#2669](mozilla/sccache#2669)) and `crt-static` for riscv64 musl targets ([#2683](mozilla/sccache#2683)). - **ccache interop**: Avoid double-caching when ccache is also installed in `PATH` ([#2524](mozilla/sccache#2524)). - **Cargo integration**: `CARGO_ENCODED_RUSTFLAGS` is now excluded from the env var hash to prevent spurious cache misses ([#2651](mozilla/sccache#2651)), and dep-info handling correctly skips directories ([#2676](mozilla/sccache#2676)). - **Distribution**: cargo-binstall metadata for prebuilt binary installation ([#2647](mozilla/sccache#2647)). - **Reliability fixes**: GCS initialization ([#2637](mozilla/sccache#2637)), Windows `Win32_Security` feature ([#2627](mozilla/sccache#2627)), and retries for dist docker image builds ([#2677](mozilla/sccache#2677)). This release also includes a large round of clippy-driven cleanups and several CI fixes. Welcome to 7 new contributors! ##### Features - Avoid double-caching when ccache is installed in PATH by [@avikivity](https://github.com/avikivity) in [#2524](mozilla/sccache#2524) - partial c++20 module support by [@TroyKomodo](https://github.com/TroyKomodo) in [#2516](mozilla/sccache#2516) - Add support for `d1nodatetime` & `await:strict` MSVC flags by [@AJIOB](https://github.com/AJIOB) in [#2617](mozilla/sccache#2617) - msvc: Append the default .pdb extension for the /Fd argument by [@ikolomiko](https://github.com/ikolomiko) in [#2621](mozilla/sccache#2621) - msvc: add support for Y-, YI, Zf flags by [@AJIOB](https://github.com/AJIOB) in [#2663](mozilla/sccache#2663) - Implement multiple tiers caching with fallback and backfilling by [@Felixoid](https://github.com/Felixoid) in [#2581](mozilla/sccache#2581) - feat: Add loongarch64 support by [@wojiushixiaobai](https://github.com/wojiushixiaobai) in [#2669](mozilla/sccache#2669) - Add cargo-binstall metadata for prebuilt binary installation by [@sylvestre](https://github.com/sylvestre) in [#2647](mozilla/sccache#2647) ##### Fixes - fix: add Win32\_Security feature to windows-sys dependency by [@JulienCote](https://github.com/JulienCote) in [#2627](mozilla/sccache#2627) - Fix GCS initialization by [@msuozzo](https://github.com/msuozzo) in [#2637](mozilla/sccache#2637) - fix: exclude `CARGO_ENCODED_RUSTFLAGS` from env var hash by [@weihanglo](https://github.com/weihanglo) in [#2651](mozilla/sccache#2651) - fix: handle directories in dep-info source file hashing by [@mmastrac](https://github.com/mmastrac) in [#2676](mozilla/sccache#2676) ##### CI - Revert "actions: add security audit workflow" by [@sylvestre](https://github.com/sylvestre) in [#2603](mozilla/sccache#2603) - ci: fix artifact\_failure action when target dir does not exist by [@sylvestre](https://github.com/sylvestre) in [#2635](mozilla/sccache#2635) - ci: set crt-static for riscv64 musl targets by [@wojiushixiaobai](https://github.com/wojiushixiaobai) in [#2683](mozilla/sccache#2683) - Add retry for dists docker image build by [@Felixoid](https://github.com/Felixoid) in [#2677](mozilla/sccache#2677) ##### Clippy & cleanup - clippy: fix cloned\_ref\_to\_slice\_refs lint by [@xtqqczze](https://github.com/xtqqczze) in [#2602](mozilla/sccache#2602) - clippy: fix semicolon\_if\_nothing\_returned lint by [@xtqqczze](https://github.com/xtqqczze) in [#2601](mozilla/sccache#2601) - clippy: fix cloned\_instead\_of\_copied lint by [@xtqqczze](https://github.com/xtqqczze) in [#2605](mozilla/sccache#2605) - clippy: fix explicit\_into\_iter\_loop lint by [@xtqqczze](https://github.com/xtqqczze) in [#2616](mozilla/sccache#2616) - clippy: fix unnecessary\_semicolon lint by [@xtqqczze](https://github.com/xtqqczze) in [#2615](mozilla/sccache#2615) - clippy: fix manual\_string\_new lint by [@xtqqczze](https://github.com/xtqqczze) in [#2609](mozilla/sccache#2609) - clippy: fix ptr\_as\_ptr lint by [@xtqqczze](https://github.com/xtqqczze) in [#2611](mozilla/sccache#2611) - clippy: fix ref\_option lint by [@xtqqczze](https://github.com/xtqqczze) in [#2608](mozilla/sccache#2608) - clippy: fix from\_iter\_instead\_of\_collect lint by [@xtqqczze](https://github.com/xtqqczze) in [#2614](mozilla/sccache#2614) - Move PreprocessorCacheModeConfig to src/config.rs by [@xis19](https://github.com/xis19) in [#2604](mozilla/sccache#2604) - Remove benchmark normalize\_win\_path\_utf8 by [@sylvestre](https://github.com/sylvestre) in [#2634](mozilla/sccache#2634) - Move cmake-modules to integration tests by [@Felixoid](https://github.com/Felixoid) in [#2649](mozilla/sccache#2649) ##### Docs - docs(Rust.md): Add caveats from README by [@TheJanzap](https://github.com/TheJanzap) in [#2684](mozilla/sccache#2684) ##### Dependencies - chore: switch thirtyfour\_sync to thirtyfour by [@tottoto](https://github.com/tottoto) in [#2613](mozilla/sccache#2613) - build(deps): bump actions/upload-artifact from 4 to 7 by [@dependabot](https://github.com/dependabot)\[bot] in [#2632](mozilla/sccache#2632) - build(deps): bump actions/download-artifact from 5 to 8 by [@dependabot](https://github.com/dependabot)\[bot] in [#2631](mozilla/sccache#2631) - chore(deps): update rust crate quinn-proto to v0.11.14 by [@xtqqczze](https://github.com/xtqqczze) in [#2645](mozilla/sccache#2645) ##### New Contributors - [@ikolomiko](https://github.com/ikolomiko) made their first contribution in [#2621](mozilla/sccache#2621) - [@JulienCote](https://github.com/JulienCote) made their first contribution in [#2627](mozilla/sccache#2627) - [@msuozzo](https://github.com/msuozzo) made their first contribution in [#2637](mozilla/sccache#2637) - [@weihanglo](https://github.com/weihanglo) made their first contribution in [#2651](mozilla/sccache#2651) - [@wojiushixiaobai](https://github.com/wojiushixiaobai) made their first contribution in [#2669](mozilla/sccache#2669) - [@TheJanzap](https://github.com/TheJanzap) made their first contribution in [#2684](mozilla/sccache#2684) - [@mmastrac](https://github.com/mmastrac) made their first contribution in [#2676](mozilla/sccache#2676) **Full Changelog**: <mozilla/sccache@v0.14.0...v0.15.0> </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).  Reviewed-on: https://git.erwanleboucher.dev/eleboucher/towonel/pulls/40

Felixoid force-pushed the multiple-tiers-cache branch 5 times, most recently from 0a4503d to e069c90 Compare January 29, 2026 12:10

Felixoid mentioned this pull request Jan 29, 2026

Time sensitive: GitHub Actions Cache rate limits (from GitHub staff) #2566

Closed

Felixoid force-pushed the multiple-tiers-cache branch 2 times, most recently from 38af4ef to b70bd73 Compare January 31, 2026 09:41

xtqqczze mentioned this pull request Jan 31, 2026

Time sensitive: GitHub Actions Cache rate limits (from GitHub staff) uutils/coreutils#10347

Open

Felixoid force-pushed the multiple-tiers-cache branch 5 times, most recently from 22b9835 to 445e8b9 Compare February 8, 2026 18:04

Felixoid mentioned this pull request Feb 8, 2026

chore: Integration tests, feature gate, cache_type_name method #2596

Merged

Felixoid force-pushed the multiple-tiers-cache branch 4 times, most recently from 7e50b5c to 3b5cfe5 Compare February 8, 2026 21:08

Felixoid mentioned this pull request Feb 8, 2026

Two-tier caching / Multiple cache levels #30

Closed

Felixoid mentioned this pull request Feb 8, 2026

chore: Simplify storage_from_config #2597

Merged

Felixoid force-pushed the multiple-tiers-cache branch 4 times, most recently from 3fe062e to 1dc75d4 Compare February 9, 2026 09:16

trxcllnt reviewed Apr 14, 2026

View reviewed changes

Comment thread src/cache/cache.rs Outdated

Use Bytes in get_raw/put_raw for zero-copy

6facb61

Felixoid force-pushed the multiple-tiers-cache branch from b24fbc4 to 6facb61 Compare April 14, 2026 21:58