Skip to content

fix(ci-conformance): resolve the policy checkout from the OIDC job_workflow_sha claim - #75

Merged
krisarmstrong merged 7 commits into
mainfrom
fix/ci-conformance-ref-73
Sep 16, 2026
Merged

krisarmstrong merged 7 commits into
mainfrom
fix/ci-conformance-ref-73

Conversation

@krisarmstrong

@krisarmstrong krisarmstrong commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Summary

ci-conformance.yml pinned its own policy checkout with
ref: ${{ github.job_workflow_sha }}. That expression evaluates to the empty
string
, and an empty ref: is actions/checkout's default-branch path — so
every run read the deciding scripts from .github@main, whatever was there at
run time. The caller's SHA pin decided nothing.

Two consequences, both already observed: check 9 (the theme hash, #72) started
failing in all four product repos the instant it merged, even though neither
pinned commit's check-ci-conformance.py contains that check; and two runs of an
identical product commit can disagree.

The fix takes the SHA from the OIDC token instead, where the claim is
populated, and fails closed if it cannot. It also adds the verification vehicle
this repo lacked: .github now runs its own conformance gate against a fixture
and asserts the pinned ref was honoured.

Why OIDC and not the context

Measured on a run, not reasoned from the YAML (the issue asked for this
explicitly). Run
35103911603,
a throwaway caller pinned at a throwaway callee:

ctx  job_workflow_sha=[] len=0
ctx  job_workflow_ref=[] len=0
ctx  workflow_sha=[14b7016993a76e61742cde463b865fb1b736478d] len=40
oidc job_workflow_sha=[34fde546a3fbfd5587e63e0f3b62506344517106]
oidc job_workflow_ref=[MustardSeedNetworks/.github/.github/workflows/diag-callee.yml@34fde546a3fbfd5587e63e0f3b62506344517106]
oidc workflow_ref=[MustardSeedNetworks/.github/.github/workflows/diag-caller.yml@refs/heads/fix/ci-conformance-ref-73]

34fde546… is exactly the callee commit the caller pinned. So: both
job_workflow_* are absent from the github context and present in the
OIDC token
. workflow_sha is populated but is the caller's commit in another
repository, as the existing comment already warned.

actionlint was right all along — job_workflow_sha is not in its context
schema, and the -ignore 'property "job_workflow_sha" is not defined' line in
this repo's own ci.yml was the only thing keeping that from being visible. That
ignore is removed here; it was a suppression standing in front of the defect, not
a tool gap.

Caller cost

Each calling job must add id-token: write. It grants no repository access — it
mints an identity assertion and reads nothing — and it is the only mechanism that
gives a reusable workflow its own commit. A caller that omits it now gets a hard
error naming the missing permission rather than a silent fall-back to a branch.
The Call it with: block documents it.

Linked Issue

Fixes #73

Type of Change

fix — a fleet CI gate that was not enforcing what it claimed to enforce, plus
the test vehicle that would have caught it.

Risk

Medium, and deliberately fail-closed. Every product repo's next pin bump to
this workflow must add id-token: write to the calling job or that job fails
loudly. That is the intended trade: a caller that cannot prove which policy
commit it is running should not get a verdict. Support-plan row D-GH-4 already
tracks those bumps as born-red.

Scope note, so nobody waits on the wrong thing: this PR does not green the four
product repos.
They call the old workflow at the old pin, so nothing here
changes what they execute; their red is drifted msn-shared.css, cleared by the
adoption rows (UI-SEED-7 / UI-STEM-7 / UI-NIAC-9 / UI-TRL-8). This PR is
the separate defect — that a new check can go fleet-wide with no adoption window.

Testing Evidence

The gate runs against the fixture, and is not vacuous. Both RED proofs taken
locally with golangci-lint 2.13.2 (CI's pinned version):

$ cd test/fixtures/conformant-repo && python3 ../../../scripts/check-ci-conformance.py
ci-conformance: passed

# RED 1 — drift the theme copy (check 9)
::error::ui/src/theme/msn-shared.css has drifted from the canonical theme
    found:    sha256 01a671fb8415
    expected: sha256 b073e5e15bab (MustardSeedNetworks/.github ui/theme/msn-shared.css)
ci-conformance: 1 finding(s)   EXIT=1

# RED 2 — drop gosec from the declared `minimum` tier
::error::linter 'gosec' is in this repo's declared tier but not enabled
ci-conformance: 1 finding(s)   EXIT=1

# restored
ci-conformance: passed         EXIT=0

The fixture's theme copy is a symlink to the canonical file, so it cannot drift
away from it silently as the canonical file changes.

Local gates:

$ python3 scripts/test-check-ci-conformance.py
Ran 16 tests in 3.515s
OK

$ python3 -m compileall -q scripts/      # clean
$ actionlint -ignore 'SC2129' -color     # EXIT=0, no findings
$ zizmor --min-severity high .github/workflows/
No findings to report. Good job! (10 suppressed)

The pin-honoured proof is SHA equality, asserted by
conformance-self-test-result: the reusable workflow now exposes a policy-sha
output, and the job compares it to the literal SHA in its own uses:. The
issue's phrasing ("a fixture pinned to a commit whose script lacks a check does
not run that check") cannot be exercised in this PR — every commit carrying the
fix also carries check 9, so there is no such commit to pin. That form becomes
testable the first time a check 10 lands after this. The equality assertion tests
the same property without a fabricated fixture.

The CI run for this PR is the end-to-end evidence; both self-test jobs are in
ci-complete's needs:, so a regression in ref resolution blocks a merge here.

Security and Release Checklist

  • No secrets, credentials or tokens added. The OIDC token is minted per job,
    scoped to the ci-conformance-policy-ref audience, and never persisted.
  • Fails closed: no token endpoint, a missing claim, a non-SHA claim, or a
    checkout landing on any other commit each abort the job.
  • All actions remain SHA-pinned; no new third-party actions.
  • persist-credentials: false retained on both checkouts.
  • No //nolint, no biome-ignore; one actionlint ignore removed.
  • No product behaviour changes — CI policy only.
  • Follow-up (D-docs(ci): show pinned-SHA usage in reusable-workflow examples #4, tracked): each product repo bumps its pin and adds
    id-token: write to the calling job.

End-to-end run on this PR

Run 35104862421Conformance Self-Test and Conformance Self-Test Result both pass:

Conformance scripts will be read from .github@55d509d09e431c441ec068c031aba538881c5d3c
policy checkout verified at 55d509d09e431c441ec068c031aba538881c5d3c
ci-conformance: passed

pinned in uses:    55d509d09e431c441ec068c031aba538881c5d3c
policy-sha output: 55d509d09e431c441ec068c031aba538881c5d3c
The pinned ref was honoured.

This is discriminating, not a tautology: main is 75ad8d1 at the time of this
run. The old code would have landed there — it read the default branch. The run
read 55d509d0, the commit named in uses:.

…rkflow_sha claim

github.job_workflow_sha evaluates to the empty string in the github context,
so the ref: it fed to actions/checkout took the default-branch path and every
run read .github@main. The caller's SHA pin decided nothing, two runs of one
product commit could disagree, and check 9 went live in four repos the instant
it merged, with no adoption window (.github#73).

Measured, not reasoned from the YAML: run 35103911603 printed
job_workflow_sha=[] len=0 and job_workflow_ref=[] len=0 from the context while
the OIDC token for the same job carried job_workflow_sha=34fde546..., the
commit the caller pinned. Take the claim from the token, validate it is a
40-hex SHA, and assert after checkout that .conformance HEAD is that commit.
Fails closed: a caller without id-token: write gets an error naming the
missing permission, never a silent fall-back to a branch.

Also drops the actionlint ignore that excused the expression. actionlint was
right that the property is not in the context schema.
…in is honoured

Nothing exercised this repo's own conformance gate end to end — only unit
tests over the script — so a checkout landing on the wrong commit could not
have been caught here. ci.yml now calls ci-conformance.yml on a fixture repo
under test/fixtures/, and a following job asserts the workflow's policy-sha
output equals the SHA pinned in uses:. That equality is the proof the pin is
honoured; if resolution regresses to the default branch the two diverge.

Both jobs are in ci-complete's needs, so the self-test can block a merge.
@krisarmstrong
krisarmstrong merged commit fc5f4bc into main Sep 16, 2026
6 checks passed
@krisarmstrong
krisarmstrong deleted the fix/ci-conformance-ref-73 branch September 16, 2026 13:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant