Reduce rustc -L args used in the new build-dir layout

[ranger-ross]

What does this PR try to resolve?

With the new build-dir enabled in rust-lang/rust there are have been a few reported issues due to long commands / PATH.
This PR attempts to mitigate this by doing the following:

1. Fix a bug where the current build unit's out dir would be added the rustc args even though it is never needed. (due to the first iteration in the recursive add_dep_arg fn).
   • This probably wont make much different for long PATHs but it should not passed.
2. Avoid passing build-script build unit's out dir which contains the build-script binary as it would very odd to link against a build script. This also has the effect of not recursively including other [build-dependencies] the build-script unit depended on.

I included this both in the PR as they are small and related but happy to split into dedicated PRs if desired :)

How to test and review this PR?

See the included tests. Best reviewed commit by commit

cc: @Kobzol

[rustbot]

r? @epage

rustbot has assigned @epage.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

• Owners of files modified in this PR: @ehuss, @epage, @weihanglo
• @ehuss, @epage, @weihanglo expanded to ehuss, epage, weihanglo
• Random selection from ehuss, epage, weihanglo

[Kobzol]

Maybe as another optimization (because each dependency with a build script will generate two -Ls even after this PR), Cargo could check if the build script's $OUT_DIR is empty, and if yes, in that case do not generate the -L for the out directory of the build script's execution? 🤔

[Kobzol]

I think that another -L that we could skip is the out directory of each direct dependency of the compiled crate. Consider a hello world crate foo that depends e.g. on log and env_logger, using the new build dir layout. It generates this final rustc call (I omitted some flags):

rustc --crate-name foo --edition=2024 src/main.rs --crate-type bin
--emit=dep-info,link -C embed-bitcode=no -C debuginfo=2
--out-dir /tmp/foo/target/debug/build/foo/ea1ee4071e58254b/out
-C incremental=/tmp/foo/target/debug/incremental
-L dependency=/tmp/foo/target/debug/build/aho-corasick/c53d038be9b9715d/out
-L dependency=/tmp/foo/target/debug/build/anstream/4bdda5cd8d2978b2/out
-L dependency=/tmp/foo/target/debug/build/log/a7c7b57a5f6aeb19/out
-L dependency=/tmp/foo/target/debug/build/env_logger/c5ba55a17507a07e/out 
--extern env_logger=/tmp/foo/target/debug/build/env_logger/c5ba55a17507a07e/out/libenv_logger-c5ba55a17507a07e.rlib
--extern log=/tmp/foo/target/debug/build/log/a7c7b57a5f6aeb19/out/liblog-a7c7b57a5f6aeb19.rlib

Notice how cargo passes both -L dependency=log and --extern log=log.rlib. Is that actually necessary? It shouldn't be for the .rlib file itself, because when you pass something via --extern, it does not actually need -L (see this comment).

So if that is the case, we could skip each -L flag for a dependency that is already passed through --extern, and only use -L for indirect dependencies. That being said, I'm not sure if there can be something else in the direct dependency's out directory that rustc might have to depend on through -L. Do you know? I'll also try to ask bjorn or Vadim.

[Kobzol]

I tested it both with depending on .rlib and dylib (.so), with and without -Zno-embed-metadata, and everything seems to work even without the -L for direct dependencies.

[rustbot]

This PR was rebased onto a different master commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

[ranger-ross]

dont merge this yet, I am still testing it and I think there might be some issues hidden in other tests since -Zbuild-dir-new-layout no longer defaults to ON.

@rustbot author

[ranger-ross]

Notice how cargo passes both -L dependency=log and --extern log=log.rlib. Is that actually necessary? It shouldn't be for the .rlib file itself, because when you pass something via --extern, it does not actually need -L (see this comment).

So if that is the case, we could skip each -L flag for a dependency that is already passed through --extern, and only use -L for indirect dependencies. That being said, I'm not sure if there can be something else in the direct dependency's out directory that rustc might have to depend on through -L. Do you know? I'll also try to ask bjorn or Vadim.

Oh interesting, I am not sure. That sounds like a reasonable optimization we could make if we indeed don't need to depend on anything through -L for direct deps

[ranger-ross]

I tried removing -L args from direct dependencies and the Cargo testsuite passed (minus a few tests due to cmd line arg snapshots)
So might be possible to do, though maybe better to do that in its own PR.

---

I finished checking the remaining tests and I believe the latest version is now correct and all of the tests pass.

@rustbot ready