Skip to content

fix(ci): distinguish workflow evidence from PR eligibility and delivery acceptance #244

Closed

Description

@ebarti

Implementation progress — 27 October 2026

Implementation candidate; acceptance not yet qualified. #256 preserves complete CI evidence, duplicate-name ambiguity and explicit execution/readback uncertainty.

The controlled hosted-workflow and actual installed/public-command acceptance still require qualification. Current-head hosted checks and a passing test aggregate do not establish the four independent execution/policy/PR-eligibility/merge-readiness states.

The referenced candidates are open and unmerged in stack #253. This progress map is non-closing; issue acceptance remains authoritative. Historical text below calling #238 open is superseded: #238 merged as c43cf0b.

Problem and evidence

A candidate had successful manually dispatched CI runs while its pull-request status rollup contained no corresponding checks. Treating statusCheckRollup as the complete execution inventory made successful execution invisible. Treating every successful workflow as a required PR check would be equally wrong.

The locally installed recovery branch at bfc918a includes typed durable CI inventory commands and collector integration. Reuse that work, verify its repository landing and complete its qualification; do not build a competing observer.

Required behavior

  • Expose separate states for observed CI execution, satisfaction of the delivery's configured validation policy, GitHub ruleset/PR check eligibility, and merge readiness.
  • Collect complete authenticated workflow/run/attempt/job evidence with pagination, source event, workflow identity, candidate head and actual checkout identity. Preserve absence/unknown explicitly.
  • Associate evidence only with the exact candidate and configured required jobs. Reject stale runs, wrong checkout/head, partial inventory, skipped/cancelled/failing jobs and unrelated workflows.
  • A workflow_dispatch success may establish execution evidence where the delivery policy allows it; it cannot be silently promoted into a GitHub-required PR check or merge permission.
  • Surface the exact missing gate and supported next action, without automatically dispatching new workflows, changing repository protections or claiming acceptance from aggregate green counts.
  • Preserve command idempotency, original CI observations and immutable authorization.

Acceptance

Exercise real GitHub-shaped/public adapter responses plus a controlled hosted workflow: dispatched jobs absent from the PR rollup, normal PR checks, multiple attempts/pages, changed head, missing jobs, incomplete/failed readback and different checkout identity. Verify all four displayed states independently.

Complete actual installed/public-command readback on the retained candidate when authorized; mocks and a local collector patch alone do not establish delivery.

Reference: GitHub checks from some workflow jobs are not evaluated for PR requirements.

Existing implementation to reuse

PR #238 is open at bfc918a545ce820f47c814e99a6fe5e92b1c3fc1, matching the inspected installed runtime. The PR explicitly identifies a remaining failure: a historical interrupted-role cleanup record causes the CI-only recovery reader to fail closed. Reuse the separately retained purpose-specific reader work, authenticate that historical custody without rewriting the unknown provider result, then exercise the supported CI-only command on the original unchanged candidate. Preserve refusal for a genuinely live/unknown current actor, changed evidence or unowned effects. The collector and most provenance work already exist; acceptance requires the remaining repair and end-to-end qualification rather than a replacement collector.

Dependencies

No blocking prerequisite in this issue set. This work can proceed in parallel with the other independent foundations; integrated qualification will verify their composition.

Blocks:

Activity

  1. ebarti commented on Oct 26, 2026

    @ebarti
    OwnerAuthor

    Reassessment finding, 2026-10-26 — P1: the ordinary CI reader can falsely pass ambiguous required check names. Reusing #244's required-job/provenance acceptance scope.

    Audited main c43cf0b and installed bfc918a5 have the same tracked source. PR #238 is merged; the frozen PR #248 checkpoint changes another boundary.

    Observed: real owned Git/bare origin/SQLite/preparation plus isolated Temporal executed the production scheduler and delivery_ci activity. A local gh adapter supplied same-head rollups:

    Build FAILURE, Build SUCCESS      -> passed
    Build SUCCESS, Build FAILURE      -> failed
    Build IN_PROGRESS, Build SUCCESS  -> passed
    unique Build SUCCESS              -> passed
    unique Build FAILURE              -> failed
    different PR head                 -> stale
    

    Order reversal changes acceptance. The discarded failing/running observation is absent from the returned evidence.

    Cause: delivery_broker.py:2044 collapses the rollup to a dictionary keyed solely by display name; line 2056 evaluates whichever entry survives. The ordinary workflow consumes ci.state == passed before its delivered projection. That downstream outcome is source-derived; a complete false delivery was not executed.

    The invariant is that every required identity must be satisfied under an explicit policy. There is no authenticated workflow/job/attempt or latest-attempt policy resolving these duplicates, so list order cannot choose a passing result. The smallest correction belongs in the normal observer: retain all matching observations and refuse ambiguity, or resolve it under an explicit authenticated identity policy.

    Preserve working safeguards: the strict MergeBroker.live_ci separately refuses duplicate matches; fixed-profile inventory also refuses duplicate selected checks. Focused preservation controls passed. This finding does not establish an unsafe merge.

    Acceptance: exercise every ordering of duplicate success/failure/running/skipped identities through the ordinary activity; no arbitrary success. Cover explicit workflow/job/attempt resolution, same-name checks across workflows, head changes and incomplete inventory. Retain exact-head, deadlines, strict merge and fixed-inventory guards; public evidence must retain ambiguity instead of hiding it. GitHub itself documents ambiguous reused job names.

    Qualification: real Temporal, production activity/broker, Git and SQLite; local GitHub-shaped transport and minimal workflow, with no real GitHub/provider calls or full-delivery/merge claim. No production source, installed runtime, settings or active deliveries changed.

    Open/closed issues and all-state PR searches found #244 as owner; related PR #76 is closed unmerged. #246 should include this combination. No new blocking dependency.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions