Skip to content

blockers: auto-rerun an untouched pre-existing failure once before blocking #73

Description

@geuben

Problem

A pre-existing-failure blocker halts the run even when the failing test (a) lives in a file the plan never touches and (b) is a known flake that passes on immediate rerun. Observed in a real agent-driven run: a JVM test with global-dispatcher contention failed once mid-run, the blocker fired, and a human had to intervene solely to say "passes on rerun, not caused by this plan" — the one human intervention in an otherwise autonomous multi-cycle run.

Proposal

Before raising the blocker for a failure that is outside the plan's declared file set and absent from the baseline:

  1. Rerun the failing test once, in isolation (single-test invocation via the adapter).
  2. If it passes, record a flaky_pre_existing event with both outcomes and continue — the evidence a human would have provided is now in the ledger, without the human.
  3. If it fails again, raise the blocker as today (now with two consistent failures as evidence, which also strengthens the genuine case).

Scope guard: only for tests outside the declared file set — a flaky test the plan touches is the plan's problem and should still block. A config knob for rerun count (default 1) keeps it conservative.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions