Skip to content

ci: Fleet status runs only for a comment that can change a status - #169

Merged
askalf merged 2 commits into
masterfrom
ci/fleet-status-skips-unrelated-events
Oct 7, 2026
Merged

askalf merged 2 commits into
masterfrom
ci/fleet-status-skips-unrelated-events

Conversation

@askalf

@askalf askalf commented Oct 7, 2026

Copy link
Copy Markdown
Owner

Summary

  • Fleet status on comments. Every comment on a PR started a Fleet status run on the exec runner, though scripts/fleet-status.mjs reads only ## Verification at <sha> comments. Preview bots' comments started dozens a day (amnesia 33, plumbline 20 in 48 hours). issue_comment now runs the job only when the comment starts with ## Verification at, or did before an edit (github.event.changes.body.from). A deleted verification comment still runs it, because the event carries the deleted body.
  • fleet-status-self-test no longer runs for a PR that changes only .github/workflows/redline.yml or redline-fix.yml, which is every Redline pin bump (18 to 23 self-test runs per repo in 48 hours). It is not a required check here, and any other workflow change, the script or its test still runs it.

Tests

scripts/fleet-status.test.mjs: 220 pass, 0 fail. The if: evaluator learns startsWith, case-insensitive like GitHub's. The new checks cover:

  • a verification comment on a PR runs the job, and one on an issue does not;
  • a preview bot's comment, a person's comment and an empty one do not;
  • an edit away from a verification comment, and a deleted one, still run it.

Every PR comment started a Fleet status run, though the script reads
only "## Verification at <sha>" comments; preview bots' comments
started dozens a day. issue_comment now runs it only when the comment
starts with that heading, or did before an edit. The test's evaluator
learns startsWith and checks a verification comment, another comment,
an edit away from one and a deletion.

fleet-status-self-test no longer runs for a PR that touches only
redline.yml or redline-fix.yml, which is every Redline pin bump.
@github-actions github-actions Bot added github_actions Pull requests that update GitHub Actions code tests Test suite and CI size/S 10-49 hand-written lines labels Oct 7, 2026

@sprayberry-redline sprayberry-redline left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: request changes. The comment filter preserves verification updates, including edits away from the heading and deletions, and the tests exercise those cases. The new workflow comment needs revision to describe the job condition rather than past run volume.

1. Blocking: .github/workflows/fleet-status.yml:44

# And only a verification comment changes a status, so no other comment starts a run (preview
# bots' comments started dozens a day); an edit away from one, or its deletion, still does.

The parenthetical narrates past run volume rather than explaining the condition. The comment also says unrelated comments do not start a run, but issue_comment still triggers the workflow; the job condition only skips the status job.

Suggested fix:

Replace this with a comment explaining that the status job runs only when the current or previous comment body starts with the verification heading, including deleted comments.

rule:reads-as-generated

The comment above the job's if: now gives the condition itself: a PR
comment runs the status job only when its body, or its body before an
edit, starts with the verification heading, and a deleted comment's
event carries its body. Every comment still fires the workflow; only the
job is skipped.

@sprayberry-redline sprayberry-redline left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: approve. The comment filter preserves updates for verification comments, edits away from verification, and deletions, with tests covering those cases and unrelated comments. The self-test path exclusions are limited to the two Redline workflows and leave other workflow and script changes covered.

@askalf
askalf merged commit 8b404e7 into master Oct 7, 2026
17 checks passed
@askalf
askalf deleted the ci/fleet-status-skips-unrelated-events branch October 7, 2026 04:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

github_actions Pull requests that update GitHub Actions code size/S 10-49 hand-written lines tests Test suite and CI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants