Crater run for cargo build-dir changes

[ranger-ross]

View all comments

PR for crater run of rust-lang/cargo#16336
Before running, we should probably get a sanity check from @epage that there is nothing I missed.

See Zulip Thread

r? @Mark-Simulacrum

[rustbot]

Some changes occurred in src/tools/cargo

cc @ehuss

[ehuss]

Do you happen to know the crater command you want to run? (The command syntax is documented at https://github.com/rust-lang/crater/blob/master/docs/bot-usage.md)

[Kivooeo]

Let's do try first and then run crater in check-only mode, at least this is how I understand this process

@bors try

[ehuss]

@Kivooeo IIUC, that's not going to be sufficient for this because the new layout is not enabled by default. There may be some special settings like environment variables that will be needed to test this properly. And as for the mode, I suspect we want to build, not check, because we'll want to see if build scripts or proc-macros are doing any shenanigans that are required for linking correctly. (Or maybe test I don't know.)

[rust-bors]

☀️ Try build successful (CI)
Build commit: 3117848 (311784882baa42a9638298cb325fa01a062c9d0f, parent: 198328ad7960b1bece0dc48496bff6c62dd5d339)

[epage]

For this, you can either set -Zbuild-dir-new-layout or CARGO_BUILD_DIR_LAYOUT_V2=true.

Note that x.py can't build with this enabled, see #t-infra/bootstrap > Preparing for the new `build-dir` layout @ 💬

I guess build-only is desirable over check-only to ensure -sys crates link?

A full build-and-test run would help uncover how much people's tests are relying on internals (e.g. I recently fixed rustfmts tests).

[Mark-Simulacrum]

@craterbot run mode=build-and-test start=master#198328ad7960b1bece0dc48496bff6c62dd5d339 end=try#311784882baa42a9638298cb325fa01a062c9d0f+cargoflags=-Zbuild-dir-new-layout crates=random-1000 p=1

Small run just to make sure it mostly works.

[craterbot]

👌 Experiment pr-149852 created and queued.
🔍 You can check out the queue and this experiment's details.

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[bors]

☔ The latest upstream changes (presumably #149934) made this pull request unmergeable. Please resolve the merge conflicts.

[craterbot]

🚧 Experiment pr-149852 is now running

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[craterbot]

🎉 Experiment pr-149852 is completed!
📊 2 regressed and 0 fixed (1000 total)
📊 0 spurious results on the retry-regessed-list.txt, consider a retry¹ if this is a significant amount.
📰 Open the summary report.

⚠️ If you notice any spurious failure please add them to the denylist!
ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

Footnotes

1. re-run the experiment with crates=https://crater-reports.s3.amazonaws.com/pr-149852/retry-regressed-list.txt ↩

[epage]

Example from https://github.com/ikorason/headr at 7f4ae60bfd5efd605d9a7e2572ba482b3ac02551

[INFO] [stdout] ---- dies_bad_lines stdout ----
[INFO] [stdout] Error: CargoError { cause: Some(NotFoundError { path: "/opt/rustwide/target/debug/build/headr/757f0ead74b5dd7c/headr" }) }

• Using assert_cmds deprecated Command::cargo_bin which I believe will fail even with a split target-dir / build-dir

Example from chute-kun 0.1.0

[INFO] [stdout] thread 'init_config_writes_default_toml_to_env_path' (150) panicked at tests/cli_init_config_test.rs:12:47:
[INFO] [stdout] called `Result::unwrap()` on an `Err` value: CargoError { cause: Some(NotFoundError { path: "/opt/rustwide/target/debug/build/chute-kun/f8f797d1537351f6/chute" }) }

• Using assert_cmds deprecated Command::cargo_bin which I believe will fail even with a split target-dir / build-dir

[epage]

Opened issues for snapbox's dependents that use the deprecated cargo_bin:

• non-nextest bin look up code for tests is broken software-mansion/scarb#2917
• Tests use deprecated snapbox::cmd::cargo_bin which will break on future Cargo versions cargo-lambda/cargo-lambda#900
• Tests rely on deprecated snapbox::cmd::cargo_bin which will break on future Cargo versions bacpop/ska.rust#103
• Tests rely on deprecated snapbox::cmd::cargo_bin which will break on future Cargo versions ffimnsr/midas-rs#1
• Tests rely on deprecated snapbox::cmd::cargo_bin which will break on future Cargo versions ffimnsr/mk-rs#2
• Tests rely on deprecated snapbox::cmd::cargo_bin which will break on future Cargo versions sbdchd/squawk#745
• Tests rely on deprecated snapbox::cmd::cargo_bin which will break on future Cargo versions bacpop/sketchlib.rust#60
• Tests rely on deprecated snapbox::cmd::cargo_bin which will break on future Cargo versions fritzrehde/asciibar#9
• 🐛 [Bug]: Tests rely on implementation details of Cargo and will break with a future version sripwoud/cza#48

[epage]

Opened issues for assert_cmd's dependents that use the deprecated cargo_bin:

• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions xiph/rav1e#3459
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions rust-db/refinery#409
• Tests rely on deprecated `assert_cmd::cargo_bin` which will break on future Cargo versions transact-rs/sqlx#4127
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions sharkdp/bat#3528
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions parcel-bundler/lightningcss#1112
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions hoxxep/rapidhash#66
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions epwalsh/rust-cached-path#87
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions mozilla/sccache#2520
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions xd009642/llvm-profparser#71
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions CodSpeedHQ/codspeed-rust#154
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions holmgr/cargo-sweep#147
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions cargo-public-api/cargo-public-api#844
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions omerbenamram/mft#132
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions dfinity/ic-wasm#101
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions ast-grep/ast-grep#2399
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions frewsxcv/cargo-all-features#71
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions chmln/sd#325
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions juan-leon/lowcharts#80
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions nushell/nushell#17205
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions sharkdp/hyperfine#844
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions sharkdp/pastel#288
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions thepacketgeek/clap-stdin#15
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions jneem/nnnoiseless#36
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions awslabs/tough#913
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions bootandy/dust#545
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions jihchi/dify#35
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions ggwpez/zepter#177
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions swc-project/swc#11385
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions JohnnyMorganz/StyLua#1056
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions holochain/lair#161
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions pksunkara/cargo-workspaces#208
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions Stranger6667/css-inline#602
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions CycloneDX/cyclonedx-rust-cargo#813
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions sourcefrog/cargo-mutants#578
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions regexident/cargo-modules#394
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions nicolaiunrein/cargo-get#107
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions cocogitto/cocogitto#518
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions sayanarijit/xplr#768
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions CosmWasm/cosmwasm#2584
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions lsd-rs/lsd#1178
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions Wilfred/difftastic#934
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions koraa/huniq#35
• https://gitlab.com/sequoia-pgp/sequoia-policy-config/-/issues/9
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions dustinblackman/cargo-run-bin#36
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions tobni/merge-junit#99
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions omerbenamram/evtx#269
• https://gitlab.com/sequoia-pgp/sequoia-sq/-/issues/608
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions use-ink/cargo-contract#2181
• https://gitlab.com/woshilapin/cargo-sonar/-/issues/16
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions esp-rs/espup#535
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions pacman82/odbcsv#97
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions epi052/feroxbuster#1284
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions Y2Z/monolith#481
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions geiger-rs/cargo-geiger#561
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions EverlastingBugstopper/billboard#14
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions ddboline/weather_util_rust#8
• https://gitlab.com/dlalic/gitlab-clippy/-/issues/5
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions deadlinks/cargo-deadlinks#165
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions risc0/risc0#3612
• Remove deprecated Command::cargo_bin pemistahl/grex#349
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions Nukesor/pueue#651
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions jfrimmel/cargo-valgrind#145
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions LukeMathWalker/bunyan#23
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions paiml/ruchy#176
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions gnostr-org/gnostr#69
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions elrnv/gltfgen#10
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions sharkdp/hexyl#254
• Tests rely on deprecated assert_cmd::cargo_bin which will break on future Cargo versions untitaker/hyperlink#191

(skipped all with download counts below 100,000)

[epage]

Realized a better route for this might be to move forward with making CARGO_BIN_EXE_* available at runtime, which I had previously proposed. cargo_bin could be updated to check that and people will just need to cargo update assert_cmd and it will just work. There will be a span of Cargo versions where CARGO_BUILD_BUILD_DIR will break the tests.

First step is rust-lang/cargo#16421

[Skgland]

making CARGO_BIN_EXE_* available at runtime, which I had previously proposed. cargo_bin could be updated to check that

Unless I'm mistaken it already does that.

Command::cargo_bin calls crate::cargo::cargo_bin_cmd¹, which calls crate::cargo::cargo_bin², which calls crate::cargo::cargo_bin_str which checks the env var³

Footnotes

1. https://docs.rs/crate/assert_cmd/latest/source/src/cmd.rs#68 ↩

2. https://docs.rs/crate/assert_cmd/latest/source/src/cargo.rs#155 ↩

3. https://docs.rs/crate/assert_cmd/latest/source/src/cargo.rs#247-248 ↩

[epage]

which checks the env var3

That is a bug and it always falls back to the unwrap_or_else case.