RFC: date-pinned nightly toolchain packages (e.g. rust-lang.org/nightly@YYYYMMDD) to unblock from-source builds

[tannevaled]

Problem

Several from-source recipes are blocked because the upstream pins a specific Rust nightly that pantry's stable rust-lang.org can't satisfy:

• tikv — rust-toolchain.toml → channel = "nightly-2023-12-28" (I just landed a vendored tikv.org in new(tikv.org): distributed transactional KV store (vendored) #13319 as a stopgap; from-source needs the pinned nightly).
• buck2 — also nightly-pinned (and its tags are rolling/dated).
• Generally any project whose rust-toolchain.toml pins a dated nightly.

Today the only options are "vendor the upstream binary" (works, but not from-source) or "hope stable compiles" (usually fails on nightly-only features).

Proposal (for discussion)

A date-pinned nightly toolchain package, e.g. rust-lang.org/nightly, versioned by date. static.rust-lang.org serves dated nightly components for all our platforms — confirmed live:

https://static.rust-lang.org/dist/2023-12-28/rust-nightly-aarch64-unknown-linux-gnu.tar.xz   → 200
.../cargo-nightly-aarch64-unknown-linux-gnu.tar.gz                                            → 200
.../rustc-nightly-x86_64-apple-darwin.tar.gz                                                  → 200

So a vendored recipe could pull the rust-nightly-<triple> tarball, versioned YYYYMMDD (e.g. 20231228). A from-source recipe then build-deps on the matching pin, e.g. rust-lang.org/nightly: =20231228.

Questions

1. Desirable in-pantry, or do you prefer keeping nightly-pinned projects as vendored-binary only?
2. Naming — rust-lang.org/nightly (single, date-versioned) vs per-tool toolchain packages?
3. Versioning scheme — YYYYMMDD from the dist date directory?
4. Concerns about bottle proliferation / nightly churn (we'd only build the specific dates that consumers pin)?

Floating this before implementing. cc @jhheider

[jhheider]

This would blow up storage usage to benefit very little. If you search existing package recipes for rustup, you'll find several that use nightly or specified versions in exactly the way you need.

[jhheider]

crates.io/skim for example:

    - run: ln -sf {{deps.rust-lang.org/rustup.prefix}}/bin/rustup rustup
      working-directory: $HOME/.cargo/bin
    - run: rustup default stable
      if: <1.3
    - run: rustup default nightly
      if: ">=1.3<2"
    # extremely specific version needs
    - run: rustup default "$(sed -n 's/^channel = "\(.*\)".*/\1/p' $SRCROOT/rust-toolchain.toml)"
      if: ">=2"
    - run: ln -sf $HOME/.rustup/toolchains/*/bin/* .
      working-directory: $HOME/.cargo/bin
...
  env:
    PATH: $HOME/.cargo/bin:$PATH