Skip to content

Media: the window-reread test times its wait from before the stop's sleep (flaky on loaded runners) - #1382

Merged
GLinnik21 merged 1 commit into
mainfrom
fix/window-reread-timing-flake
Nov 14, 2026
Merged

GLinnik21 merged 1 commit into
mainfrom
fix/window-reread-timing-flake

Conversation

@GLinnik21

Copy link
Copy Markdown
Collaborator

What failed

plx_media::route::decision::queue_predecessor_tests::the_window_reread_waits_for_the_outgoing_stop_within_its_bound (a devtriggers test inside the hostsim pass) failed in CI twice with the same message:

panicked at media/src/route/decision_queue_predecessor_tests.rs:315:5:
it waited for the stop: 249.328727ms

Cause

let started = Instant::now() was taken INSIDE the spawned reader thread, while the 250 ms std::thread::sleep that holds the stop back runs on the test thread right after scope.spawn. When the scheduler starts the reader late (a loaded 4-vCPU runner), the reader's clock starts after the test thread's sleep has already begun. A read that correctly waited for stop.finish() then measures under 250 ms, so read >= 250ms fails although the code under test behaved.

The causal facts were already asserted separately and are not affected: fx.pms.window_reads() is 0 at the 250 ms mark and 1 after the join.

Fix (test only)

started is taken before std::thread::scope, so before the reader exists and before the sleep can begin. The reader still returns started.elapsed() right after refreshed_within returns, so the assertion keeps its meaning: the read could not finish before the stop landed. The 250 ms bound is unchanged and the assertion stays.

The other half of the test (the stop that never lands) starts and stops its clock on one thread around one call, so it has no cross-thread comparison and is unchanged. The only other sleep in the file is the mock PMS's accept poll (2 ms), which is not compared against any clock.

Demonstration

A 50 ms std::thread::sleep was put at the very start of the reader closure, before started, to stand in for a reader the scheduler starts late (a smaller delay is not reliable: the reader's elapsed also includes the window read after the stop lands).

Old code, with the delay:

---- route::decision::queue_predecessor_tests::the_window_reread_waits_for_the_outgoing_stop_within_its_bound stdout ----
thread '...the_window_reread_waits_for_the_outgoing_stop_within_its_bound' panicked at media/src/route/decision_queue_predecessor_tests.rs:316:5:
it waited for the stop: 204.252334ms
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 2513 filtered out; finished in 0.29s

Fixed code, same delay still in:

test route::decision::queue_predecessor_tests::the_window_reread_waits_for_the_outgoing_stop_within_its_bound ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 2513 filtered out; finished in 0.43s

The delay was then removed (grep -c DEMO-ONLY is 0 on the committed file; the diff is only the moved clock and its comment).

Gates

make test-hostsim C=plx_media T=queue_predecessor_tests on the committed tree (git status --short empty before and after):

test result: ok. 6 passed; 0 failed; 0 ignored; 0 measured; 2508 filtered out; finished in 0.45s

Not run: make check (test-only change in one file, gated by the CI lanes), the TV (no device behavior involved).

🤖 Generated with Claude Code

…leep

`the_window_reread_waits_for_the_outgoing_stop_within_its_bound` took its
`started` clock inside the spawned reader, while the 250 ms sleep that holds the
stop back runs on the test thread right after the spawn. A reader the scheduler
starts late (a loaded 4-vCPU runner) began its clock after that sleep had
already begun, so a read that correctly waited for `stop.finish()` measured
under 250 ms and failed `read >= 250ms` (249.328727ms in CI on 2026-11-12,
PR #1355, and on 2026-11-14, PR #1380, run 43551876055).

The clock now starts before the reader is spawned, so before the sleep can
begin; the reader still returns `started.elapsed()` right after the re-read
returns, so the bound still states that the read could not finish before the
stop landed. The causal facts stay asserted by `window_reads()` being 0 at the
250 ms mark and 1 after the join. The bound is unchanged. The "stop that never
lands" half starts and stops its clock on one thread around one call and needs
no change.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@GLinnik21
GLinnik21 merged commit a3ea02c into main Nov 14, 2026
19 checks passed
@GLinnik21
GLinnik21 deleted the fix/window-reread-timing-flake branch November 14, 2026 04:10
GLinnik21 added a commit that referenced this pull request Nov 14, 2026
Brings in #1382 and #1384; nothing in them touches the hostsim suite or
the files this branch edits.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@GLinnik21 GLinnik21 added the backport release/v0.10 When merged into main, also cherry-pick onto release/v0.10 (backport.yml) label Nov 19, 2026 — with ChatGPT Codex Connector
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport release/v0.10 When merged into main, also cherry-pick onto release/v0.10 (backport.yml)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant