Add buildcache channel for Build Caching Service runtime resolution

docs/build_cache_requirements.md

@@ -0,0 +1,129 @@
+# Build Cache Service — Requirements for Crank Integration
+
+## Context
+
+Crank (the .NET benchmarking tool) has been updated to support a new `buildcache` channel that downloads pre-built runtime binaries from the Build Cache Service (BCS) instead of resolving versions from VMR/NuGet feeds. This gives per-commit granularity for performance testing and regression bisection.
+
+The crank-side changes are complete. This document describes what's needed on the BCS/dotnet-performance-infra side to make the integration work end-to-end.
+
+---
+
+## Requirement 1: Public Blob Access
+
+**Status:** Already in progress (per prior discussion).
+
+Crank's BCS client uses unauthenticated HTTP GET requests to download artifacts. The blobs in the `pvscmdupload` storage account's `$web` container need to be publicly readable.
+
+**URLs crank will hit:**
+
+```
+GET https://pvscmdupload.z22.web.core.windows.net/builds/{repoName}/latest/{branch}/latestBuilds.json
+GET https://pvscmdupload.z22.web.core.windows.net/builds/{repoName}/buildArtifacts/{commitSha}/{configKey}/{artifactFile}
+```
+
+Where:
+- `repoName` = `runtime` or `aspnetcore` (selected per-job via the `buildCacheRepo` property; defaults to the agent's `--build-cache-repo-name`)
+- `branch` = e.g., `main`, `release/10.0`
+- `configKey` = e.g., `coreclr_x64_linux`, `coreclr_arm64_windows` (runtime); `aspnetcore_x64_linux`, `aspnetcore_arm64_windows` (aspnetcore)
+- `artifactFile` = e.g., `BuildArtifacts_linux_x64_Release_coreclr.tar.gz` (runtime); `BuildArtifacts_linux_x64_Release_aspnetcore.nupkg` (aspnetcore — the verbatim runtime-pack nupkg)
+
+The aspnetcore `latestBuilds.json` lives at `builds/aspnetcore/latest/{branch}/latestBuilds.json` and contains only the 5 `aspnetcore_*` config keys plus an `all` entry and `branch_name`; it does **not** carry the runtime `coreclr_*` keys. Crank's parser enumerates keys dynamically, so it tolerates either repo's file.
+
+---
+
+## Requirement 2: Commit Index File (Not Required)
+
+~~Originally proposed as a per-branch `commitIndex.json` mapping commits to timestamps.~~
+
+**Decision:** Not needed. For the default case, `latestBuilds.json` provides the latest commit. For specific-commit runs (e.g., bisection), users will already know the SHAs — either from git history, GitHub, or a local tool that queries the GitHub API for the commit list. A separate index in BCS would be redundant.
+
+If automated bisection tooling is built in the future, it can query GitHub directly for ordered commit SHAs and then check BCS blob existence per-commit.
+
+---
+
+## Requirement 3: latestBuilds.json Compatibility
+
+**Resolved.** The actual `latestBuilds.json` uses PascalCase (`CommitSha`, `CommitTime`), not snake_case. Crank's parser has been updated to accept both casings for forward compatibility.
+
+---
+
+## Requirement 4: Artifact Layout Stability
+
+Crank extracts **runtime** artifacts using this path convention inside the archive:
+
+```
+microsoft.netcore.app.runtime.{rid}/Release/runtimes/{rid}/lib/net{X}.0/  → managed DLLs
+microsoft.netcore.app.runtime.{rid}/Release/runtimes/{rid}/native/        → native libs
+{rid}.Release/corehost/                                                    → host binaries (dotnet, libhostfxr, libhostpolicy)
+```
+
+For **aspnetcore** artifacts the stored blob is the **verbatim runtime-pack nupkg** (a zip,
+extension `.nupkg`), so the layout is the nupkg's own — `runtimes/{rid}` sits at the archive root,
+with no `microsoft.aspnetcore.app.runtime.{rid}/Release` wrapper. Crucially, the verbatim nupkg
+carries the host-resolvable framework metadata next to the managed assemblies:
+
+```
+runtimes/{rid}/lib/net{X}.0/Microsoft.AspNetCore.*.dll                  → managed assemblies
+runtimes/{rid}/lib/net{X}.0/Microsoft.AspNetCore.App.deps.json          → host-resolvable metadata (REQUIRED)
+runtimes/{rid}/lib/net{X}.0/Microsoft.AspNetCore.App.runtimeconfig.json → host-resolvable metadata (REQUIRED)
+runtimes/{rid}/native/                                                  → native libs (optional)
+```
+
+Because the nupkg is a complete framework, crank **places `Microsoft.AspNetCore.App` directly** from
+the pack into the per-job dotnet home (the whole managed set incl `deps.json`/`runtimeconfig.json`,
+no feed contribution) and **fails the job** (`BuildCacheIncompleteException`) if the pack is missing
+managed assemblies, `deps.json`, or `runtimeconfig.json` — for perf runs, erroring is preferable to
+silently running mixed/feed bits. The base runtime + host stay feed-resolved (the aspnetcore pack
+ships neither). Self-contained (SCD) publishes are the exception: the framework is co-mingled with
+the app under the app's own `.deps.json`, so for SCD crank overlays only the managed `*.dll`
+(+ native), not the framework metadata.
+
+This differs from **runtime**, whose archive is raw build output (no shared-framework
+`deps.json`/`runtimeconfig.json`), so the runtime flavour overlays BCS binaries onto a feed-installed
+runtime (reusing the feed's metadata) rather than placing directly.
+
+Where `{rid}` = `linux-x64`, `linux-arm64`, `win-x64`, `win-arm64`, `win-x86`. (aspnetcore v1 has no
+musl/osx/arm32 configs.)
+
+The runtime layout was confirmed by inspecting `BuildArtifacts_linux_x64_Release_coreclr.tar.gz` (no
+framework metadata in the pack lib). The aspnetcore contract — a verbatim runtime-pack nupkg carrying
+managed + `deps.json` + `runtimeconfig.json`, uploaded as `BuildArtifacts_{os}_{arch}_Release_aspnetcore.nupkg`
+— was confirmed against a locally-built `Microsoft.AspNetCore.App.Runtime.win-x64.nupkg` and
+dotnet/performance#5243's `stage-bcs-nupkg-aspnetcore.ps1`.
+**If either layout changes in future builds, the crank extraction will break.** Consider treating it
+as a stable contract or documenting it.
+
+---
+
+## Nice-to-Have: Artifact Manifest
+
+A `manifest.json` per commit+config that describes the archive contents would make extraction more robust:
+
+```
+builds/{repoName}/buildArtifacts/{commitSha}/{configKey}/manifest.json
+```
+
+```json
+{
+  "runtimeVersion": "10.0.0-preview.4.26120.3",
+  "commitSha": "abc123...",
+  "rid": "linux-arm64",
+  "managedPath": "microsoft.netcore.app.runtime.linux-arm64/Release/runtimes/linux-arm64/lib/net10.0",
+  "nativePath": "microsoft.netcore.app.runtime.linux-arm64/Release/runtimes/linux-arm64/native",
+  "corehostPath": "linux-arm64.Release/corehost"
+}
+```
+
+This isn't blocking — crank currently discovers paths by convention — but it would decouple crank from the internal archive layout and make future changes safe.
+
+---
+
+## Summary
+
+| # | Requirement | Priority | Blocking? |
+|---|-------------|----------|-----------|
+| 1 | Public blob access | High | Yes — crank can't download without it |
+| 2 | ~~Commit index~~ | N/A | Dropped — users provide SHAs directly or use GitHub |
+| 3 | `latestBuilds.json` field names | N/A | Resolved — crank parser updated to handle PascalCase |
+| 4 | Artifact layout stability | Medium | Not now, but breaking changes would break crank |
+| 5 | Artifact manifest.json | Low | Nice-to-have for robustness |

docs/dotnet_versions.md

@@ -55,9 +55,12 @@ When a TFM is configured, the agent will download the corresponding .NET SDK ver
 - `current`: only latest public versions, this is the default
 - `latest`: latest versions used by ASP.NET 
 - `edge`: latest nightly builds available
+- `buildcache`: runtime from the Build Cache Service (per-commit builds)
 
 The difference between `latest` and `edge` is that `latest` will pick runtimes and SDKs that are deemed compatible together. For instance a very recent .NET core runtime might be compatible with a less recent ASP.NET runtime. The `edge` is used to pick the absolute latest build for the select TFM.
 
+The `buildcache` channel uses the Build Cache Service (BCS) from `dotnet-performance-infra` to resolve runtime versions by individual commit SHA rather than from VMR feeds. This provides much finer-grained control — every cached runtime commit is available, whereas VMR feeds may have multi-day gaps between ingested commits. SDK and ASP.NET Core versions are resolved from `latest` when using `buildcache`.
+
 In order to benchmark and ASP.NET application using very recent runtimes of .NET 5, the `latest` channel is recommended:
 
 ```
@@ -115,4 +118,75 @@ The following command uses the `edge` channel but ASP.NET is fixed so it doesn't
 
 ```
 > crank --config /crank/samples/hello/hello.benchmarks.yml --scenario hello --profile local --application.framework netcoreapp5.0 --application.channel edge --application.aspnetCoreVersion 5.0.0-preview.6.20279.12
-```
+```
+
+## Using the Build Cache channel
+
+The `buildcache` channel resolves pre-built binaries for individual commits from the Build Cache Service (BCS). This is useful for performance regression bisection where VMR feed gaps make it hard to pinpoint which commit caused a regression.
+
+The channel can resolve from one of two repositories, selected per-job with the `buildCacheRepo` property:
+
+- `runtime` (default) — **overlays** the base .NET runtime (`Microsoft.NETCore.App`) with BCS bits built from a [dotnet/runtime](https://github.com/dotnet/runtime) commit. The runtime archive is raw build output (no shared-framework metadata), so BCS binaries are overlaid onto a feed-installed runtime.
+- `aspnetcore` — **places** the ASP.NET Core shared framework (`Microsoft.AspNetCore.App`) directly from a [dotnet/aspnetcore](https://github.com/dotnet/aspnetcore) commit's BCS build. The aspnetcore archive is the runtime-pack nupkg stored verbatim (carrying `deps.json` + `runtimeconfig.json`), so the framework folder is built entirely from BCS and the job **fails** if the pack is incomplete. The base runtime stays at the feed-resolved version.
+
+When no `buildCacheRepo` is supplied the job falls back to the agent-level `--build-cache-repo-name` (which itself defaults to `runtime`), so existing runtime usage is unchanged.
+
+### Basic usage (latest cached build on main)
+
+```
+# runtime (default)
+> crank --config benchmarks.yml --scenario json --profile aspnet-perf-lin --application.channel buildcache
+```
+
+### Bisecting ASP.NET Core
+
+Add `--application.buildCacheRepo aspnetcore` to resolve an ASP.NET Core commit instead. The `buildCacheCommitSha` / `buildCacheBranch` values are then interpreted as **dotnet/aspnetcore** commit/branch:
+
+```
+# latest cached aspnetcore build on main
+> crank --config benchmarks.yml --scenario json --profile aspnet-perf-lin --application.channel buildcache --application.buildCacheRepo aspnetcore
+
+# specific aspnetcore commit
+> crank --config benchmarks.yml --scenario json --profile aspnet-perf-lin --application.channel buildcache --application.buildCacheRepo aspnetcore --application.buildCacheCommitSha a1b2c3d4e5f6...
+```
+
+> **One BCS component per job (v1).** A single job overrides EITHER the base runtime OR the ASP.NET Core shared framework from BCS, not both at once. The `buildCacheRepo` selector decides which one; the other framework comes from the normal feed. If a job somehow requests both, the selected flavour wins and the other is skipped with a log warning.
+
+### Specific commit SHA
+
+```
+> crank --config benchmarks.yml --scenario json --profile aspnet-perf-lin --application.channel buildcache --application.buildCacheCommitSha a1b2c3d4e5f6...
+```
+
+If the commit is not found in the cache, crank will fail with an error rather than falling back.
+
+### Different branch
+
+```
+> crank --config benchmarks.yml --scenario json --profile aspnet-perf-lin --application.channel buildcache --application.buildCacheBranch release/10.0
+```
+
+### Mixed channels (BCS runtime + pinned ASP.NET)
+
+```
+> crank --config benchmarks.yml --scenario json --profile aspnet-perf-lin --application.channel buildcache --application.aspNetCoreVersion 10.0.0-preview.3.26115.7
+```
+
+### Build Cache properties
+
+| Property | Default | Description |
+|----------|---------|-------------|
+| `buildCacheRepo` | (agent `--build-cache-repo-name`, i.e. `runtime`) | Which BCS repository to resolve from: `runtime` (overlays `Microsoft.NETCore.App` from a feed install) or `aspnetcore` (places `Microsoft.AspNetCore.App` directly from the verbatim runtime-pack nupkg; fails loud if incomplete). |
+| `buildCacheCommitSha` | (empty) | Specific commit SHA in the selected repo (runtime or aspnetcore). If empty, uses the latest cached build for the branch. |
+| `buildCacheBranch` | `main` | Branch to query for the latest build. |
+| `buildCacheConfig` | (auto-detected) | BCS configuration key (e.g., `coreclr_x64_linux` for runtime, `aspnetcore_x64_linux` for aspnetcore). Auto-detected from agent platform and the selected repo. |
+
+### Agent configuration
+
+The agent supports these command-line options for BCS:
+
+| Option | Default | Description |
+|--------|---------|-------------|
+| `--build-cache-base-url` | `https://pvscmdupload.z22.web.core.windows.net` | Base URL for BCS blob storage. |
+| `--build-cache-repo-name` | `runtime` | Repository name in BCS. |
+| `--build-cache-disabled` | (not set) | Disables BCS integration on this agent. |