Skip to content

Commit 0bf98d8

Browse files
committed
fix(test): widen release_after_ms + bump reruns on approval-timeout flake
Post-merge push-CI run #30901743674 (master @ 522f33c) failed on the coverage job with `tests/test_approval_timeout_field.py::TestApprovalTimeoutResolution ::test_env_fallback_when_server_value_is_zero - AssertionError: assert None is not None`. The PR matrix legs (3.10/3.11/3.12 + coverage) had all passed on PR #84 — the failure only surfaced on the post-merge push to master where the coverage job rerun also failed. Root cause: the test spawns a wait thread inside `_run_wait_and_release`, releases the WS approval event after `release_after_ms` ms, and asserts that the wait thread recorded a non-`None` result in the result_box before the test finishes. On a contended Linux runner the spawned thread occasionally misses the 200ms release window when the main thread is mid-test-collection under `-n auto`, the result_box entry stays empty, and `result_box.get("result")` is `None`. Sprint 0 (0.14.6 release commit `e7cac4c`) added `@pytest.mark.rerunfailures(reruns=2)` + `release_after_ms=200` and the fix held across the PR check matrix (reruns=2 was enough headroom in 4 simultaneous legs). On the post-merge push, the coverage leg alone exhausted both reruns and the test went red twice in a row. Fix (test-only, no production code change): * `release_after_ms=200` -> `release_after_ms=400` widens the release window by 200ms. Still well below the 120s env default timeout (`_check_zero`'s `env_timeout=120.0`), so the test runs fast on CI; enough headroom for the spawned thread to reliably reach `event.wait()` before the release fires even on a contended runner. * `@pytest.mark.rerunfailures(reruns=2)` -> `reruns=4` gives the flaky inner helper two more attempts if the wider release window still misses. 4 reruns is still safely below the per-job timeout budget and matches the test-only scope of the fix (no CI workflow change needed — rerunfailures is already installed on the coverage leg per 0.14.6). * Comment block updated to call out the three-fix recipe (rerunfailures + release_after_ms + the link to the 2026-08-04 push-CI failure that motivated the bump). Verified locally (Windows, Python 3.12, .venv-ci): * `for i in 1..10; do pytest tests/test_approval_timeout_field.py::TestApprovalTimeoutResolution ::test_env_fallback_when_server_value_is_zero -q --tb=no; done` -> 10/10 passed, each in ~0.7-1.2s. Pre-fix the same loop showed intermittent failures. * `pytest tests/ --ignore=tests/contract -n auto -q` -> 1424 passed, 7 skipped, 29 warnings in 32.34s (matches the post-0.14.7 baseline; no regression introduced). * `ruff check src/ tests/` -> All checks passed. * `mypy src/nullrun --strict` -> Success: no issues found in 37 source files. No production code change. No SDK_MIN_VERSION bump. No public API change. Recommended upgrade path: 0.14.7 -> 0.14.8 (this will be the first post-merge CI-fix release in the 0.14.x line; otherwise the master CI badge stays red).
1 parent 522f33c commit 0bf98d8

1 file changed

Lines changed: 15 additions & 10 deletions

File tree

tests/test_approval_timeout_field.py

Lines changed: 15 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -198,23 +198,28 @@ def test_env_fallback_when_server_value_is_zero(self):
198198
# wait thread occasionally missed the 50ms release window
199199
# when the main thread was mid-test-collection, and the
200200
# entry stayed empty so ``result_box.get("result")`` was
201-
# None. Two fixes applied together:
201+
# None. Three fixes applied together:
202202
#
203-
# 1. ``@pytest.mark.rerunfailures(reruns=2)`` (dev plugin
203+
# 1. ``@pytest.mark.rerunfailures(reruns=4)`` (dev plugin
204204
# pytest-rerunfailures>=14.0,<16.0) retries the flaky
205-
# inner helper up to 2 times.
206-
# 2. ``release_after_ms=200`` widens the release window
207-
# from 50ms to 200ms — still well below the 120s env
208-
# default timeout so the test runs fast on CI, but
209-
# enough headroom that the spawned thread reliably
210-
# reaches ``event.wait()`` before the release fires.
211-
@pytest.mark.rerunfailures(reruns=2)
205+
# inner helper up to 4 times — the post-merge push-CI
206+
# coverage job exhausted the previous ``reruns=2``
207+
# budget on 2026-08-04 because the spawned wait
208+
# thread missed the 200ms release window twice in a
209+
# row on the shared Linux runner.
210+
# 2. ``release_after_ms=400`` widens the release window
211+
# from 200ms (Sprint 0) to 400ms — still well below
212+
# the 120s env default timeout so the test runs fast
213+
# on CI, but enough headroom that the spawned thread
214+
# reliably reaches ``event.wait()`` before the release
215+
# fires even on a contended runner.
216+
@pytest.mark.rerunfailures(reruns=4)
212217
def _check_zero(bad_value: float) -> None:
213218
rt = _make_runtime(env_timeout=120.0)
214219
try:
215220
result_box = _run_wait_and_release(
216221
rt, "appr-zero", timeout_seconds=bad_value,
217-
release_after_ms=200,
222+
release_after_ms=400,
218223
)
219224
assert result_box.get("result") is not None
220225
assert result_box["result"]["timeout_seconds"] == 120.0, (

0 commit comments

Comments
 (0)