Skip to content

[Feature]: CI guidance is missing its interpretation side — propose a verdict attribution doc #119

Description

@neoLsH
  • I searched existing issues for a similar request.

What problem are you trying to solve?

I went through the repo's CI-related content, and the evaluation side is
actually in good shape: software-fluency scores hosted CI L1–L5,
harness-engineering §8 checks CI/local consistency, and git-hooks.md has a CI
And Merge Stage. But all of it answers "does this project's CI exist, is it
strong, where in the lifecycle should it sit". None of it answers what an
agent should do with an actual verdict.

What I see missing:

  1. Red-verdict attribution. The failure-repair chain in agent-work-loop
    assumes the check is locally runnable and reproducible; hosted CI usually
    isn't. I've watched agents burn rounds "fixing" their own diff for
    failures that were really infra (hitting other PRs too) or upstream
    test-corpus drift (the consumed repo deleted a failing case an hour
    earlier).
  2. Fake-green defense. Nothing warns about green runs that executed zero
    cases: silent fallback when a remote fetch fails, scripts dying quietly
    under set -e, 0/0 passing.

What would success look like?

One short references doc: attribute the verdict before touching code (infra /
upstream asset drift / local), plus a few evidence-integrity guards
(empty-run must fail, fail-fast over silent fallback). I already sketched it
following the references contract — link-graph check passes — and can send a
PR.

Where would this be used?

references/project-harness/, guidance only, host-agnostic.

Examples or evidence

All from real episodes on OSS and internal CI; happy to provide redacted
write-ups.

Anything else?

The ownership call is genuinely open: separate doc, or extend
agent-verify-loop's "After a Non-Passing Verdict" (I lean separate because
that section assumes locally runnable checks). Whether the guards deserve
their own doc, whether a companion case-study is worth it — also open. This
is my reading of the repo, so if I missed existing coverage or got the
framing wrong, please correct me; happy to discuss.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions