Skip to content

ci: release ordering, CodeQL gate and a release-readiness check - #36

Merged
serialexperimentslainnnn merged 8 commits into
developfrom
feature/update-pipelines
Aug 6, 2026
Merged

serialexperimentslainnnn merged 8 commits into
developfrom
feature/update-pipelines

Conversation

@serialexperimentslainnnn

Copy link
Copy Markdown
Owner

Summary

Four CI changes on the road to the 5.0.0 release. No plugin code is touched.

  1. release.yml reordered to the intended sequence — merge to main → cut and sign the tag → open the GitHub Release → build and publish to the Marketplace **from that tag**. It previously
    built, published, and then stamped a tag on what had already gone out.
  2. CodeQL required on develop, not only on main. Both jobs already ran on every PR into
    develop; requiring them only decides whether anyone has to look at the result.
  3. A release-readiness gate: develop → main cannot merge while Claude or Dependabot has an
    open pull request into develop.
  4. apply-rulesets.sh fixed — it stripped only the key named exactly _comment, so a second
    annotation in the same object reached the API and got a bare 422.

Type of change

  • Docs / build / CI
  • Bug fix

Risk and rollback

Risk: confined to CI and release. The plugin artifact is unchanged. The sharp edge is #3: it is
a required check, and a required check that never reports blocks a PR forever — so the ruleset
must be applied after this merges, never before (see the ordering note below).

Rollback: revert the commits and re-run scripts/apply-rulesets.sh. Nothing here has reached a
user; nothing is published.

Ordering — this matters

scripts/apply-rulesets.sh must run after this is merged into develop. The new required
check No bot PRs pending on develop does not exist as a job until then, and applying the ruleset
first would leave PR #35 waiting on a check nothing can ever report.

Notes for reviewers

Why #3 is a status check and not a ruleset rule. GitHub rulesets speak of checks, signatures
and approvals. They have no vocabulary for "no other pull request exists", so the gate has to be a
job, required by display name like every other one.

The Dependabot cost is real and is written into the workflow rather than left to be discovered.
Dependabot's resting state in this repository is "has something open", so draining that queue
becomes a release step. That is the intended trade — nothing ships alongside an unmerged dependency
bump — but it is the kind of gate people learn to bypass if the queue is ignored for weeks. If it
starts being routinely in the way, the answer is to merge Dependabot more often, not to widen the
filter.

On #1, two things a reviewer should push back on if they disagree:

  • Cutting the tag before the build reverses a deliberate earlier decision, and the comment that
    argued for the old order was right at the time. What changed is that the whole publish job now
    sits behind the marketplace environment, so the tag cannot appear before a human approves. The
    residual case — a tag that outlives a failed publish — is real, and the asset upload carries
    --clobber so re-running is the recovery.
  • It is deliberately not two workflows chained by the tag push. A tag pushed with the
    GITHUB_TOKEN does not create a workflow run (the recursion guard), so chaining would need a PAT,
    a GitHub App or a deploy key — a long-lived write credential — to buy an ordering that one run
    already achieves by checking the tag out.

Found during review of my own change, and NOT fixed here — flagged so it is a decision rather
than an oversight:

  • release.yml, "Create and sign the release tag": the comment claims the step is a no-op when the
    tag already exists, so a failed publish can be recovered by re-running. It is not — git tag -s
    exits non-zero on an existing tag, and fetch-depth: 0 fetches tags, so the re-run aborts before
    reaching the --clobber upload. Needs an existence guard.
  • release.yml header: the justification "main is a moving ref" is inaccurate. actions/checkout
    defaults to github.sha, so the run was already pinned to the triggering commit. Building from
    the tag is still right as a provenance statement; the stated reason is not.

Both were surfaced by a security review pass over this branch, which found no security findings
of its own — the changes keep every irreversible action behind the same approval boundary.

How was this tested?

  • ./gradlew test (694, 0 failures) and npm test (84) — green before these commits; none of
    them touch plugin code.
  • Workflow YAML parsed and every required check name in .github/rulesets/*.json cross-checked
    against an actual job display name — no orphans.
  • The bot author matcher exercised against app/claude, app/dependabot, claude[bot],
    dependabot[bot] (all blocked) and serialexperimentslainnnn, claudia99 (both pass).
  • release.yml end to end. It cannot be tested without publishing; the first real run is the
    5.0.0 release.

The release ran as: build, publish to the Marketplace, then stamp a tag on what had already gone
out. It now runs as the sequence the process actually describes — merge to main, cut and sign the
tag, open the GitHub Release, then build and publish FROM that tag.

Building from the tag rather than from `main` is the part that changes behaviour: `main` is a
moving ref, so a merge landing between the `guard` job and the `publish` job was silently included
in a release named after a different tree. `publish` now checks out `refs/tags/vX.Y.Z`.

Cutting the tag first used to be unsafe, and the comment saying so was right at the time: a tag
could exist for a version that was never published, and published tags are immutable here. What
makes it safe now is that the whole job sits behind the `marketplace` environment, so nothing —
including the tag — happens before a human approves. The residual case is a publish that fails
after the tag exists; the recovery is re-running the job on that tag, which is why the asset
upload carries `--clobber`.

This is deliberately NOT two workflows chained by the tag push. A tag pushed with the GITHUB_TOKEN
does not create a workflow run (the recursion guard), so chaining would need a PAT, a GitHub App
or a deploy key — a long-lived write credential — to buy an ordering one run already achieves.

`buildPlugin signPlugin publishPlugin` stays a single Gradle invocation. `publishPlugin` uploads
the signed archive only if `signPlugin.didWork` and falls back to the UNSIGNED one otherwise, so
splitting it to fit the new ordering is exactly how an unsigned plugin ships unnoticed.

The GitHub Release is created as a draft and undrafted once the artifacts are attached: created
final and empty, its download links would 404 for the length of the build.

Out of band but part of the same path: the `marketplace` environment's deployment branch policy
allowed only `tag: v*.*.*`, and the run's ref on the primary path is `refs/heads/main` — the
deployment would have been rejected before even asking for the approval. `main` was added to it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
codeql.yml already triggers on every pull request into develop, so both jobs were running there and
nobody was obliged to read the result. Requiring them changes only that.

Affordable in a way the other main-only checks are not: no extra run is created. And a SAST finding
is the class of defect worth catching before the merge rather than at the release door, where it
arrives mixed in with everything else that landed on the branch since.

The contexts are the jobs' DISPLAY names. Renaming a job in codeql.yml does not fail this gate — it
silently stops applying it, which is why the names are duplicated in a comment beside them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
apply-rulesets.sh deleted the key named exactly `_comment`, so the moment a second annotation was
needed in one object the natural name — `_comment_codeql` — sailed through the filter and reached
the API, which rejected the whole ruleset with a bare 422 naming no property.

Observed, not hypothetical: it is how the CodeQL required check failed to apply. The failure reads
as "the ruleset is wrong" rather than "a comment leaked into the payload", which is the expensive
part. Now every key with the `_comment` prefix is stripped, so annotating a block twice is safe.

The key added in the previous commit is renamed back to the convention as well.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A release is a claim that develop is a finished state. An open pull request from Claude or from
Dependabot contradicts it: the change was meant to be in this release and sits one click away from
being in it. Merging past it does not lose the work — it ships a version whose CHANGELOG was
written as if the work had landed. For Dependabot it also means releasing with a known dependency
update unmerged, the one class of pending change an advisory gets written about.

A status check rather than a ruleset entry because it cannot be a ruleset entry: rulesets speak of
checks, signatures and approvals, and have no vocabulary for "no other pull request exists".
main.json requires this job by DISPLAY name, like every other gate.

The author match is anchored, not a substring, so a human whose username contains "claude" is not
caught by a release gate. It covers both renderings, since which one appears depends on how each
integration is installed: `app/<slug>` for an app, `<name>[bot]` for a bot user.

The cost is stated in the workflow rather than left to be discovered: Dependabot's resting state is
"has something open", so draining that queue becomes a release step. That is the intended trade,
and if the gate starts being routinely in the way the answer is to merge Dependabot more often, not
to widen the filter.

NB the check cannot be required until this job exists on develop — a required check that never
reports blocks the pull request forever. Merge first, run scripts/apply-rulesets.sh second.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
One image served every job, and an image is a cost paid PER JOB: each one pulls its own
copy onto its own runner. So `Frontend tests` — whose work is an 8-second vitest run —
spent 1m05s pulling a JDK, a Gradle distribution and 3.4 GB of extracted IntelliJ
Platform it never opened.

There are now two, named for what they carry and pinned by version, not a floating tag:

  node-test:v1.0.0   462 MB. Node, npm, warm npm cache. For `Frontend tests` and
                     `Dependency audit`.
  jvm-test:v1.0.0    8.08 GB. The above plus the JDK, Gradle and the extracted platform.
                     For every job that runs Gradle: JVM tests, Static analysis, Plugin
                     verifier, CodeQL (java-kotlin), drift, and the release gate.

jvm-test is built FROM node-test, so it is not a second copy — the registry stores the
shared layers once — and it carries Node deliberately: `Static analysis`, `drift` and the
release gate each run Gradle AND npm in one job. Splitting those would add a whole extra
image pull, which is the cost this change exists to remove.

Two jobs now pull NOTHING. `Build plugin` downloads an artifact and runs `unzip`, `grep`
and `ls` — it does not build anything despite the name, and it was pulling GB to do it.
The bot-PR gate only calls `gh`. Both run on a bare runner.

Why the pull cannot simply be cached, since it is the obvious first idea: a `container:`
job pulls in `Initialize containers`, which runs BEFORE the job's first step, so there is
no point at which an `actions/cache` step could run first — and every job starts on a
fresh runner with no shared layer cache. Restoring a tarball instead is slower, not
faster: it moves the same bytes from a store further away than ghcr and capped at 10 GB
per repository. The only lever is how much each job downloads, which is what this does.

The images also run `dnf upgrade --refresh` before installing. That makes the build
non-reproducible, which is acceptable here precisely because the tag is explicit: what CI
runs is frozen at v1.0.0, and bumping it is the deliberate act that moves it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Since `CodeQL (java-kotlin)` moved into a container, every run logs:

  ERROR: ld.so: object '.../${LIB}_${PLATFORM}_trace.so' from LD_PRELOAD
         cannot be preloaded (cannot open shared object file): ignored

`$LIB` and `$PLATFORM` are glibc dynamic string tokens that ld.so expands at load time, so one
variable covers several ABIs. Measured locally rather than assumed: Fedora 44's loader expands
them to `lib64_x86_64` and loads the file happily; Ubuntu 24.04's does not resolve that form at
all. CodeQL is built and tested on Ubuntu runners, so the shipped filename matches Ubuntu's
expansion and Fedora asks for a name the bundle does not contain.

The step symlinks the name Fedora asks for onto the 64-bit tracer that is actually shipped, and
prints the directory listing first — that `ls` is the evidence the layout still matches, and it is
deliberately not guarded with `|| true` so a future CodeQL release that moves these files fails
loudly instead of quietly reverting to the current behaviour.

Whether the message was ever more than noise is NOT established, and this commit does not claim it
was: github/codeql-action#1113 records the same line as harmless with the real failure elsewhere.
It is removed because a permanent ERROR in a security gate's log trains you to skim past the one
that matters — and this gate has just become a required check on develop. The same run will settle
it: if extraction was actually being skipped, the build step's behaviour changes with the symlink
in place.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The previous attempt globbed the tool cache:

  ls -d /__t/CodeQL/*/x64/codeql/tools/linux64

and that matched TWO directories — the runner image ships one CodeQL bundle and the action had
downloaded another, so 2.26.1 and 2.26.2 sat side by side. The variable held two newline-separated
paths and every command after it failed:

  ls: cannot access '.../2.26.1/...'$'\n''.../2.26.2/...*_trace.so': No such file or directory

Picking one by sort order would have been a nicer-looking guess. LD_PRELOAD already names the exact
file the loader will be asked for, so the whole inference disappears: the step now reads it, takes
its dirname, and substitutes the tokens the way Fedora's ld.so resolves them.

The substitution is on the value read from the ENVIRONMENT, where `${LIB}` and `${PLATFORM}` are
literal characters that a shell assignment does not re-expand. Verified with a literal env value
rather than assumed — the first test of it was wrong (it built the string with double quotes, so
bash expanded both tokens to empty before the substitution ever ran) and looked like a real failure.

It also fails loudly if LD_PRELOAD is unset, since that means tracing was never initialised and this
step is patching a problem that no longer exists in the shape it was written for.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`Initialize CodeQL` runs inside the container and writes to $GITHUB_ENV:

  LD_PRELOAD=/__t/CodeQL/<v>/x64/codeql/tools/linux64/${LIB}_${PLATFORM}_trace.so

`/__t` is the name the tool cache has INSIDE the container; on the host the same directory is
/opt/hostedtoolcache — which is precisely what this job exported back when it ran on a bare runner,
and why the message never appeared there. But $GITHUB_ENV is consumed by the runner process, which
lives on the HOST, so its helpers start with an LD_PRELOAD naming a path that does not exist from
where they stand, and ld.so logs "cannot be preloaded ... ignored" on every step.

The fix makes one string valid from both namespaces: symlink /opt/hostedtoolcache to /__t inside the
container, and rewrite the variable to use it. Verified inside the real jvm-test image before being
written here — the rewritten path resolves and the real tracer loads through it.

Three earlier hypotheses were wrong and are recorded in the workflow so nobody re-runs them:

  - Not a missing Fedora package. The bundle ships the full matrix (lib/lib64/lib32/x86_64-linux-gnu
    times x86_64/haswell/i686/xeon_phi) and lib64_x86_64_trace.so is present in the container, 0755.
  - Not a glibc difference. Fedora 44's loader expands the tokens and loads the real tracer fine in
    this exact image: AT_PLATFORM x86_64, every dependency satisfied.
  - Not a broken database. The tracing that matters happens inside the container, where the path was
    always valid — which is why the scan succeeded throughout.

This supersedes the symlink patch from df99ffe and 284233e, which the evidence showed was a no-op:
the file it created already existed.

Residual, stated rather than hidden: the ERROR still appears once at the start of the rewriting step
itself, since the new value cannot apply before the step that sets it. The steps that run the
compiler get the corrected value.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@serialexperimentslainnnn
serialexperimentslainnnn merged commit 5c35d99 into develop Aug 6, 2026
10 checks passed
@serialexperimentslainnnn
serialexperimentslainnnn deleted the feature/update-pipelines branch August 10, 2026 19:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant