Re-enable dotnet test tests on macOS ARM64 (#49665)#54485
Open
Evangelink wants to merge 2 commits into
Open
Conversation
The underlying infrastructure issue (Helix installing the x64 .NET runtime on arm64 macOS machines, causing 'incompatible architecture' dlopen failures when running multi-TFM tests against the previous .NET runtime) was fixed in eng/restore-toolset.sh by commits 6d4031f (Rosetta 2 detection + reinstall) and bba0cc1 (use TARGET_ARCHITECTURE for cross-arch installs). This re-enables the remaining dotnet test tests that were skipped on OSX because of that infrastructure issue: - GivenDotnetTestBuildsAndDiscoversTests.DiscoverProjectWithMSTestMetaPackageAndMultipleTFMsWithTests_ShouldReturnExitCodeSuccess - GivenDotnetTestBuildsAndRunsHelp.RunHelpOnMultipleTestProjects_ShouldReturnExitCodeSuccess - GivenDotnetTestBuildsAndRunsTests.RunningWithGlobalPropertyShouldProperlyPropagate - GivenDotnetTestBuildsAndRunsTestsForMultipleTFMs.RunMultipleProjectWithDifferentTFMs_ShouldReturnExitCodeGenericFailure - GivenDotnetTestBuildsAndRunsTestsForMultipleTFMs.RunProjectWithMultipleTFMs_ShouldReturnExitCodeGenericFailure - GivenDotnetTestBuildsAndRunsTestsForMultipleTFMs.RunProjectWithMultipleTFMs_ParallelizationTest_RunInParallelShouldFail - GivenDotnetTestBuildsAndRunsTestsForMultipleTFMs.RunProjectWithMSTestMetaPackageAndMultipleTFMs_ShouldReturnExitCodeGenericFailure Also fixes RunTestProjectSolutionWithRuntimeOption_ShouldReturnExitCodeSuccess, which hardcoded 'x64' as the runtime architecture. On arm64 macOS the x64 runtime is not installed, which produced the 'have arm64, need x86_64' mismatch in the test logs. It now derives the runtime architecture from the current process the same way the neighbouring tests do. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR re-enables previously skipped dotnet test coverage on macOS by removing the OSX exclusion for affected tests, and fixes one runtime RID selection in a test to use the current process architecture.
Changes:
- Replaces OSX-excluding
PlatformSpecificTheoryattributes with regular[Theory]. - Removes obsolete #49665 failure comments from re-enabled tests.
- Updates one
--runtimetest to derive the runtime architecture dynamically instead of hardcodingx64.
Show a summary per file
| File | Description |
|---|---|
test/dotnet.Tests/CommandTests/Test/GivenDotnetTestBuildsAndRunsTestsWithDifferentOptions.cs |
Re-enables the runtime-option test on macOS and derives the RID architecture from the current process. |
test/dotnet.Tests/CommandTests/Test/GivenDotnetTestBuildsAndRunsTestsForMultipleTFMs.cs |
Re-enables several multi-TFM dotnet test scenarios on macOS. |
test/dotnet.Tests/CommandTests/Test/GivenDotnetTestBuildsAndRunsTests.cs |
Re-enables the global-property propagation test on macOS. |
test/dotnet.Tests/CommandTests/Test/GivenDotnetTestBuildsAndRunsHelp.cs |
Re-enables multi-project help output coverage on macOS. |
test/dotnet.Tests/CommandTests/Test/GivenDotnetTestBuildsAndDiscoversTests.cs |
Re-enables MSTest multi-TFM discovery coverage on macOS. |
Copilot's findings
- Files reviewed: 5/5 changed files
- Comments generated: 0
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Re-enables the remaining
dotnet testtests that were skipped on OSX as part of #49665.Context
The underlying infrastructure issue was that Helix was installing the x64 .NET runtime on arm64 macOS machines (and vice versa for cross-arch jobs), which caused
dlopenfailures like:when running multi-TFM
dotnet testtests that target the previous .NET runtime viaDotnetVersionHelper.GetPreviousDotnetVersion().That infrastructure issue was addressed in
eng/restore-toolset.shby:Fix arm64 macOS CI installing x64 runtimes under Rosetta 2(detects Rosetta + reinstalls the SDK and removes the cached x64 runtimes)Use TARGET_ARCHITECTURE env var for cross-arch runtime installationChanges
This PR re-enables the following tests (removes the
[PlatformSpecificTheory(TestPlatforms.Any & ~TestPlatforms.OSX)]skip and the issue/error comment, swaps back to[Theory]):GivenDotnetTestBuildsAndDiscoversTests.DiscoverProjectWithMSTestMetaPackageAndMultipleTFMsWithTests_ShouldReturnExitCodeSuccessGivenDotnetTestBuildsAndRunsHelp.RunHelpOnMultipleTestProjects_ShouldReturnExitCodeSuccessGivenDotnetTestBuildsAndRunsTests.RunningWithGlobalPropertyShouldProperlyPropagateGivenDotnetTestBuildsAndRunsTestsForMultipleTFMs.RunMultipleProjectWithDifferentTFMs_ShouldReturnExitCodeGenericFailureGivenDotnetTestBuildsAndRunsTestsForMultipleTFMs.RunProjectWithMultipleTFMs_ShouldReturnExitCodeGenericFailureGivenDotnetTestBuildsAndRunsTestsForMultipleTFMs.RunProjectWithMultipleTFMs_ParallelizationTest_RunInParallelShouldFailGivenDotnetTestBuildsAndRunsTestsForMultipleTFMs.RunProjectWithMSTestMetaPackageAndMultipleTFMs_ShouldReturnExitCodeGenericFailureAlso fixes a real bug in
RunTestProjectSolutionWithRuntimeOption_ShouldReturnExitCodeSuccess: the test hardcoded"x64"as the runtime architecture, so on arm64 macOS it tried to load the x64 host (have 'arm64', need 'x86_64'in the error message). The test now derives the architecture from the current process the same way the neighbouring tests do.Relevance of #49665
Still relevant: there are additional tests skipped across publish/watch/blazor/etc. areas that reference the same issue. This PR only addresses the
dotnet testsubset that was the primary scope of the issue. The remaining tests should be tackled separately so this leg can be verified independently on CI.Validation
Local Windows build of
dotnet.Tests.csprojsucceeds with 0 warnings / 0 errors. The macOS ARM64 leg is not runnable locally, so CI is the real validation here.