Convert -Ctarget-cpu into a target-modifier for AVR, AMDGCN and NVPTX #150732
Convert -Ctarget-cpu into a target-modifier for AVR, AMDGCN and NVPTX #150732kulst wants to merge 2 commits into
-Ctarget-cpu into a target-modifier for AVR, AMDGCN and NVPTX #150732Conversation
-Ctarget-cpu a target-modifier on NVPTX
|
@rustbot label: +O-NVPTX |
-Ctarget-cpu a target-modifier on NVPTX-Ctarget-cpu into a target-modifier on NVPTX
-Ctarget-cpu into a target-modifier on NVPTX-Ctarget-cpu into a target-modifier for NVPTX
7164067 to
a8f5e96
Compare
|
Some changes occurred in src/doc/rustc/src/platform-support cc @Noratrieb These commits modify compiler targets. |
|
r? @nnethercote rustbot has assigned @nnethercote. Use |
|
r? @bjorn3 as you suggested this change. |
|
It works out really nice with -CTarget-cpu being able to be a target modifier. It would be nice to know what the plan was with the ptx isa version as well. I asked that question here |
|
I can't vouch for the implementation, but the approach sounds good. :) Cc @Darksonn |
|
I have added some explanation of why |
| pub(super) fn target_cpu( | ||
| sess: &Session, | ||
| l: &TargetModifier, | ||
| r: Option<&TargetModifier>, | ||
| ) -> bool { | ||
| if sess.target.cpu_is_target_modifier { | ||
| if let Some(r) = r { | ||
| return l.extend().tech_value == r.extend().tech_value; | ||
| } else { | ||
| return false; |
There was a problem hiding this comment.
What happens if -Ctarget-cpu appears multiple times on the command line? Please add a test.
There was a problem hiding this comment.
The value of the last appearance in matches is used. In practice this is probably the value of -Ctarget-cpus last appearance in the command line arguments of the rustc invocation.
Will add a test soon, but I am asking myself what is the intended behavior? I assume we should error if a target-modifier appears more than once, would you agree?
There was a problem hiding this comment.
At the very least we have to ensure that whatever values rustc actually uses and what is recorded as target modifier are the same.
There was a problem hiding this comment.
Last argument taking precedence is the desired functionality:
Corresponding MCP
There was a problem hiding this comment.
Behavior as per MCP is fine. Whichever value ends up being used should be tracked and verified.
There was a problem hiding this comment.
Is checked now by tests/run-make/target-cpu-is-target-modifier/rmake.rs
Does that match what you had in mind?
a8f5e96 to
e5cbf49
Compare
|
Some changes occurred to the intrinsics. Make sure the CTFE / Miri interpreter cc @rust-lang/miri, @RalfJung, @oli-obk, @lcnr The reflection data structures are tied exactly to the implementation cc @oli-obk
Some changes occurred in tests/ui/stack-protector cc @rust-lang/project-exploit-mitigations, @rcvalle Some changes occurred to the CTFE / Miri interpreter cc @rust-lang/miri Some changes occurred in src/tools/clippy cc @rust-lang/clippy Some changes occurred to the CTFE machinery |
|
I have updated the PR and rebased it on top of the current main branch. The PR is now split into two commits:
For this to work, I added a new target-spec flag, When
The reason for keeping the flags separate is to preserve the ability to compile for NVPTX with a stable compiler. Since NVPTX is a Tier 2 target, its sysroot crates are built in CI using the target’s default CPU implicitly. The consistency check treats an omitted Since
The PR adds tests covering the following cases:
Additional tests explicitly cover that
This does not fully prove that all other targets avoid treating @rustbot label -A-CI -A-meta -A-testsuite -T-infra |
Can you add a hack in |
| // modifier. To not accidentally link two crates built with different `native` | ||
| // cpus we reject `native` completely. | ||
| if sess.target.requires_consistent_cpu | ||
| && sess.opts.cg.target_cpu.as_deref().unwrap_or(sess.target.cpu.as_ref()) == "native" |
There was a problem hiding this comment.
Instead of adding a separate out-of-band check, can you insert this check in the place where we are actually handling "native"? Currently you are effectively "parsing" target_cpu twice, that's generally a bad idea.
There was a problem hiding this comment.
"native" is currently handled separately in each codegen backend. Performing the check there would result in duplication.
Instead, I moved the parsing into its own function: rustc_session::config::is_native_cpu. It is called by all codegen backends and also by the new check.
The check itself now lives in rustc_session::session::validate_commandline_args_with_session_available, which seems more appropriate.
Is that in line with what you had in mind?
There was a problem hiding this comment.
"native" is currently handled separately in each codegen backend.
Ah, bummer. Well I guess the code already is a mess so you're not making it significantly worse. ;)
3c8a444 to
e5a16e9
Compare
|
cc @bjorn3
|
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
e5a16e9 to
02d2121
Compare
|
I've made the requested changes and rebased on top of the current main branch. |
This comment has been minimized.
This comment has been minimized.
02d2121 to
6736669
Compare
|
This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed. Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers. |
This comment has been minimized.
This comment has been minimized.
| pub flag_name: String, | ||
| pub flag_name_prefixed: String, | ||
| pub extern_value: String, | ||
| pub value_len: usize, |
There was a problem hiding this comment.
Maybe do extern_value: Option<String> if fluent supports that? Otherwise has_extern_value: bool.
There was a problem hiding this comment.
Using a boolean works. I wasn't aware of this, since it does not work on the fluent playground.
| /// Returns whether `cpu_name` requests the host CPU. | ||
| pub fn is_native_cpu(cpu_name: &str) -> bool { | ||
| cpu_name == "native" | ||
| } |
There was a problem hiding this comment.
Not sure this is an improvement. If you want to avoid duplicating the "native" string, adding a const for it would also be an option.
| // must succeed. | ||
| // | ||
| // It then repeats the same compilation using a copy of that target spec with | ||
| // `requires-consistent-cpu` set to true. This time the compilation must fail. |
There was a problem hiding this comment.
This requires a //@ needs-llvm-component: x86 I think given that it forces x86_64 as target arch.
| use serde_json::{Value, json, to_string}; | ||
|
|
||
| fn main() { | ||
| let target = "x86_64-unknown-linux-gnu"; |
There was a problem hiding this comment.
Maybe using avr-none would be simpler? Would require skipping this test when AVR if LLVM is built without AVR support or if a different codegen backend without AVR support is used.
There was a problem hiding this comment.
Not sure this test as written is useful.
There was a problem hiding this comment.
The intention was to verify, that for all targets other than avr, amdgcn and nvptx -Ctarget-cpu is not interpreted as a target modifier.
tests/run-make/requires-consistent-cpu-behavior/rmake.rs verifies that requires_consistent_cpu is the flag that controls the behavior and this test verifies that the flag is only set for the mentioned targets.
I found this to be more stable than my previous attempt:
- Querying rustc for its supported targets and for their target-cpus.
- Compiling for each target using two cpus from the queried list.
- Exactly the three targets must reject
It relied on the llvm target machine to be available.
But after giving it a second thought I could compile with --emit metadata and --Zcodegen-backend=dummy.
This should trigger the target modifier check but be way more stable.
| } | ||
|
|
||
| fn is_native_cpu_line(line: &str) -> bool { | ||
| line.trim_start().starts_with("native ") |
There was a problem hiding this comment.
Maybe
| line.trim_start().starts_with("native ") | |
| line.contains("native") |
and inline it as closure?
There was a problem hiding this comment.
Inlining is a good idea, but I would like to stick to the previous string recognition: It would still work even if LLVM or other codegen backends decide to introduce a CPU name containing alternative or something similar.
There was a problem hiding this comment.
This is testing behavior of the argument parser that is independent of -Ctarget-cpu. Feels a bit unnecessary.
There was a problem hiding this comment.
The intention was for this test to verify that the last-mentioned CPU takes precedence as a target modifier, and for tests/codegen-llvm/target-cpu-precedence.rs to verify that it also takes precedence as the codegened target CPU. I was following this review, but perhaps I misinterpreted it?
6736669 to
cf747da
Compare
- Boolean target modifiers are now mentioned without a trailing `=` in the messages. - Wording improved for unset target modifiers.
…and NVPTX For AVR, AMDGCN, and NVPTX, crates built with different target CPU values are not generally link-compatible. Add a `requires_consistent_cpu` flag to the target spec and enable it for these targets. When the flag is set, treat `-Ctarget-cpu` as a target modifier and require all linked crates to agree on its value. Reject `-Ctarget-cpu=native` before codegen for targets that set `requires_consistent_cpu` to true. Also do not include `native` in the printed `target-cpus` list for such targets. Add tests covering: - which built-in targets set `requires-consistent-cpu` - cross-crate behavior with and without `requires-consistent-cpu` - that an omitted `-Ctarget-cpu` compares equal to an explicitly specified default CPU - rejection and printing behavior for `native` - precedence of repeated `-Ctarget-cpu` flags in metadata comparison and LLVM IR
cf747da to
e67d6fb
Compare
View all comments
Crates built for AVR, AMDGCN and NVPTX that specify different values for
-Ctarget-cpucannot be soundly linked together.This PR attempts to make
rustcensure that no crates with disagreeing values for-Ctarget-cpuare linked together. This is achieved by converting-Ctarget-cpuinto a target-modifier depending on--target.To do this, the consistency check for
-Ctarget-cpuconsiders mismatching values as inconsistent only for targets for which the new flagrequires_consistent_cpuis set in their target spec.Why should
-Ctarget-cpube a target-modifier fornvptx?PTX is a single-module contract
PTX requires a binary to start with .version (`ptx$$`) then .target (`sm_$$`). If the ptx contains instructions that are not supported by either .version or .target, the binary is ill-formed and will be rejected by ptxas. The concept of features that can be mixed and matched in a binary does not exist for nvptx and is therefore not supported by LLVM.
It prevents the production of bitcode that cannot be codegen'd after linking
A target modifier should prevent configurations that are not composable across crates when those crates are linked together. The most prominent example is when enabling a target feature changes the ABI, making cross-crate calls inherently unsound.
In the case of nvptx, ABI mismatch is (at least for now) not the core problem motivating target modifiers. NVIDIA’s documented PTX calling convention has remained stable since ptx20.
However, in the current state it is possible to produce bitcode that cannot be codegen'd after linking, because some operations are only lowerable for sufficiently new SM/PTX levels. In the best case this results in an LLVM error during the final llc step, but this is not something we should rely on for correctness.
nvptx has a special compilation pipeline where instead of linking the final PTX object, instead LLVM bitcode is linked. The resulting artifact is then compiled in one invocation.
Now consider crate A which is independently compiled into bitcode with the following rustc arguments:
Crate A is a dependency of crate B. In the rustc invocation of crate B
This should now ideally create an LLVM error, because the linked bitcode contains code paths that were selected under
sm_70assumptions but the final NVPTX codegen is targetingsm_60, where those operations are not lowerable. An LLVM error here is better than silent miscompilation, but it’s not a promise we should rely on.A real example where this could happen is the lowering of atomic loads and stores with non-relaxed orderings, which is known to depend on the selected SM level.
Why should
-Ctarget-cpube a target-modifier foramdgcnandavr?Previous discussions about the topic can be found here and here.
I also created a Zulip discussion.
I am unsure if a MCP is needed before proceeding. If you think so please let me know.
Creating target-modifiers for NVPTX target-features is to be done in a follow-up.
cc @kjetilkjeka as target maintainer for NVPTX
cc @Flakebi as target maintainer for amdgcn
cc @Patryk27 as target maintainer for AVR
cc @RalfJung you were very involved in the discussions so far
Target modifier tracking issue: #136966