Repository navigation
ci: run every macOS/Windows test binary with --no-fail-fast - #1016
Merged
Merged
Conversation
cargo test stops at the first failing test binary, so one flaky test on the macOS runner hid every binary after it. Verifying #1013 took three runs that each stopped on a different unrelated test (keyed_alpha, port_pool, thumbnail_tap_cuda) before the vision mixer FX tests were ever reached. With --no-fail-fast every binary runs and all failures show in one run; the step still fails if any test fails. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
srperens
added
ci:macos
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
cargo teststops at the first failing test binary. On the macOS runner, verifying #1013 took three runs that each stopped on a different, unrelated test before the vision mixer FX tests were reached:keyed_alpha::keyed_pads_keep_alpha_with_auto_output_format("white program input never appeared in PGM within 20s"),port_pool_test::a_port_held_by_another_process_is_skipped_and_marked_blocked(409), andthumbnail_tap_cuda_test::thumbnail_on_a_cuda_memory_tee_downloads_first_and_keeps_the_stream_flowing. One flaky test hides every binary after it.Change
--no-fail-faston thecargo test --package stromstep in the Windows and macOS jobs. Every test binary runs and all failures show in one run; the step still fails if any test fails. Linux is unchanged.Evidence
This PR carries
ci:macosandci:windows, so both jobs run with the change.Tests
CI only.
Out of scope
The flaky tests themselves (keyed_alpha on macOS, port_pool, thumbnail_tap_cuda, and
whip_slot_decoder_reuse_test::reused_slot_accepts_a_different_codecseen on main earlier) need their own issues.🤖 Generated with Claude Code