Skip to content

verify-release-intent: opened-before-labeled race produces a spurious failing check run #93

Closed

Description

@cratis-direct

What's happening

verify-release-intent.yml correctly implements its label-extraction logic — this is not a bug in the jq filtering. But there's a race condition in how consuming repos trigger it that produces a confusing, spurious failing check run whenever a pull request is opened and then labeled a few seconds later (a very common flow: gh pr create --label patch, or a human opening a PR and immediately clicking a label).

Concrete example (Cratis/Templates PR #36)

Every repo's thin caller (verify-semver-label.yml) triggers on:

on:
  pull_request:
    types: [opened, reopened, synchronize, labeled, unlabeled]

For PR #36 (fix/26-atomic-publication):

  • 2026-09-06T09:08:12Z — PR opened (no labels yet) → run 34023786118 starts, evaluates LABELS: [], fails with "This pull request states no release intent."
  • 2026-09-06T09:08:16Z — patch label added → run 34023791826 starts (correctly sees patch), succeeds.

The concurrency: cancel-in-progress: true group (${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}) is supposed to cancel the stale "opened" run once the "labeled" run starts for the same PR, but the job body is a few lines of jq/bash that finishes in well under a second — faster than GitHub can cancel it. So instead of the older run being cleanly superseded (which does happen sometimes — see run 33147269338 on PR carrying docs/discoverability, which shows cancelled), it completes first and posts a failure.

The result: the commit ends up with both a failure and a success entry for the same check name (verify / verify) in its status-check rollup. It doesn't block merging (the latest run for the SHA is what GitHub's merge gate honors), but it's confusing — a red X shows up in the PR's check history for a run that already reflects stale, pre-label state, and it can trigger unnecessary "why did this fail?" investigation (as it did here).

This isn't unique to Cratis/Templates — every repo that adopted the reusable workflow copies the same types: [...] trigger list (checked Chronicle, Arc, Fundamentals, Components — all identical), so all of them see this whenever a PR is opened before it's labeled.

Possible directions (not prescribing a fix — this is a judgment call for the team that owns the gate's UX)

  • Drop opened from the trigger types, since it almost always precedes a deliberate label choice; labeled/unlabeled/synchronize would still guarantee the check reruns whenever the label state or code actually changes. Trade-off: a PR that's opened and genuinely never labeled would show as "expected"/pending rather than an explicit failing message telling the author what to do.
  • Or accept this as a known, non-blocking cosmetic rough edge, since it never actually blocks a merge.

Evidence

Filed after investigating Cratis/Templates#37 (a workflow-run failure report) — no code fix was made in Cratis/Templates, since its thin caller has no logic to fix and correctly delegates to this reusable workflow.

Status

State: Done — Merged e888941; natural-PR spot-check watch j521 running
Branch: stage-422-atomic-execution (#213)
Scope: PR #213: 20s label grace on unlabeled opened PRs; merge held for L0 confirmation (cross-caller behaviour)
Delivered: Nothing yet.
Remaining: To be confirmed.
Follow-ups: None.

Activity

  1. cratis-direct commented on Sep 6, 2026

    @cratis-direct
    Author

    Thanks for opening this - it has been received and will be looked at.


    Posted by Stagehand (AI) - an autonomous agent, not a person. Review accordingly.

  2. added theissue type on Sep 6, 2026
  3. cratis-direct commented on Sep 8, 2026

    @cratis-direct
    Author

    Another occurrence of the same pattern, this time in Cratis/AuthProxy PR #120 (fix/legacy-identity-empty-userid-guard):

    • 2026-09-08T08:28:03Z — PR opened (no labels yet) → run 34204646349 starts, evaluates LABELS: [], fails with the same "states no release intent" message.
    • 2026-09-08T08:28:11Z — patch label added → run 34204662347 starts (correctly sees patch), succeeds 10 seconds after the first run started.

    Same root cause as the original report: the opened event's payload snapshot genuinely had no labels at that instant (confirmed against the PR's label-added timeline via the GitHub API), and the two runs are only ~8 seconds apart in this case — too close for cancel-in-progress to supersede the first before it finishes. The PR merged successfully afterward (mergeStateStatus resolved once the passing run existed), so this remains non-blocking, just a confusing red X in the run history.

    No action taken in Cratis/AuthProxy — its verify-semver-label.yml is a thin caller with no logic to change, per the existing guidance in this issue. Filing this as evidence that the race recurs across repos (now confirmed in Templates and AuthProxy) rather than opening a duplicate issue.


    Posted by an autonomous AI agent while investigating a reported failure in Cratis/AuthProxy. Review accordingly.

  4. cratis-direct commented on Sep 14, 2026

    @cratis-direct
    Author

    Thanks for opening this - it has been received and will be looked at.


    Posted by Direct (AI) - an autonomous agent, not a person. Review accordingly.

  5. self-assigned this
    on Oct 25, 2026
  6. woksin commented on Oct 25, 2026

    @woksin
    Contributor

    Delivered in #213 (merged e888941): an unlabeled opened pull request into main now waits 20 seconds before the release-intent check reads labels, in both the policy job and verify-release-intent.yml (skipped for Dependabot). A label added in that window starts a run that cancels the waiting one through the callers' cancel-in-progress group, so no stale failure is left. A spot-check on a naturally occurring opened-then-labeled pull request follows.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions