Problem
A pre-existing-failure blocker halts the run even when the failing test (a) lives in a file the plan never touches and (b) is a known flake that passes on immediate rerun. Observed in a real agent-driven run: a JVM test with global-dispatcher contention failed once mid-run, the blocker fired, and a human had to intervene solely to say "passes on rerun, not caused by this plan" — the one human intervention in an otherwise autonomous multi-cycle run.
Proposal
Before raising the blocker for a failure that is outside the plan's declared file set and absent from the baseline:
- Rerun the failing test once, in isolation (single-test invocation via the adapter).
- If it passes, record a
flaky_pre_existing event with both outcomes and continue — the evidence a human would have provided is now in the ledger, without the human.
- If it fails again, raise the blocker as today (now with two consistent failures as evidence, which also strengthens the genuine case).
Scope guard: only for tests outside the declared file set — a flaky test the plan touches is the plan's problem and should still block. A config knob for rerun count (default 1) keeps it conservative.
Problem
A pre-existing-failure blocker halts the run even when the failing test (a) lives in a file the plan never touches and (b) is a known flake that passes on immediate rerun. Observed in a real agent-driven run: a JVM test with global-dispatcher contention failed once mid-run, the blocker fired, and a human had to intervene solely to say "passes on rerun, not caused by this plan" — the one human intervention in an otherwise autonomous multi-cycle run.
Proposal
Before raising the blocker for a failure that is outside the plan's declared file set and absent from the baseline:
flaky_pre_existingevent with both outcomes and continue — the evidence a human would have provided is now in the ledger, without the human.Scope guard: only for tests outside the declared file set — a flaky test the plan touches is the plan's problem and should still block. A config knob for rerun count (default 1) keeps it conservative.