Skip to content

Add a runner capability probe for the CI that depends on it - #17

Merged
MarkusPaulsen merged 2 commits into
mainfrom
chore/runner-capability-probe
Sep 11, 2026
Merged

MarkusPaulsen merged 2 commits into
mainfrom
chore/runner-capability-probe

Conversation

@MarkusPaulsen

@MarkusPaulsen MarkusPaulsen commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds a manual workflow that reports what a GitHub runner can actually do, before the CI that depends on the answer is written. Two planned jobs turn on the machine rather than on this repository: running the Landlock acceptance suites inside an ordinary container, and running the prune phase, which needs Bubblewrap. Neither answer can be read out of the source, and a runner image can withdraw either without notice.

Linked issues

No linked issues.

1. Problem

Nothing here can say whether a GitHub runner enforces a Landlock policy inside an ordinary container, or whether Bubblewrap can build a sandbox on it at all. Both are properties of the machine rather than of the code, and each decides whether a planned job is worth writing: the first governs the Landlock acceptance suites, the second the prune phase that discovers which paths a language environment needs.

Two parts of Phobos are involved, and they use different mechanisms for the same job. The run phase confines a submission with Landlock. The prune phase uses Bubblewrap, which needs unprivileged user namespaces, restricted by default on Ubuntu since 24.04 and so measured on the ubuntu-26.04 runner, not assumed.

Writing either job first and finding out afterwards means writing a job that may have to be thrown away, or worse, one that passes for a reason nobody checked. Nothing is broken today. This is the gap that has to be closed before the rest of the CI is worth writing.

2. Improvement from the user's perspective

No Improvement from the user's perspective.

3. Improvement from the maintainer's perspective

One dispatch answers both capability questions, with the kernel, the Landlock version, the AppArmor settings and the Bubblewrap version recorded beside the answer, so a later CI failure can be compared against a known-good survey instead of being re-diagnosed from scratch. Because the probe stays in the repository, the same question can be asked again whenever GitHub changes the runner image.

The survey also reports one finding about this repository rather than the machine: the prune phase asks Bubblewrap for --unshare-utc, which Bubblewrap has never had. The option is --unshare-uts. That is reported and not asserted, because it is a defect here and not a property of the runner.

4. Testing manual

Prerequisites

  1. This branch, and Docker. Nothing on the host has to be prepared, and no container needs --privileged, --cap-add or --security-opt.
  2. Optionally shellcheck 0.11.0, actionlint 1.7.12 and yamllint 1.38.0, if you would rather reproduce the lint gates locally than read them off this pull request's checks.

Steps

Not reproducible from a run. This adds a workflow and the scripts it calls, so no submission executes differently. A reviewer verifies it in three ways instead.

  1. Read the checks on this pull request. The Lint workflow covers all four new and existing file types: shellcheck over every .sh, gcc -Wall -Wextra -Werror -fanalyzer and cppcheck over every .c, actionlint over the new workflow, and yamllint over the repository.
  2. Run the survey in a container, on the image the runner and the probe both use:
    docker run --rm -v "$PWD:/mnt:ro" ubuntu:26.04 bash -c 'apt-get update -qq >/dev/null && apt-get install -y -qq gcc libc6-dev bubblewrap >/dev/null && cp -r /mnt/tests /t && bash /t/runner-capability-probe.sh --report'
    libc6-dev is named because 26.04 no longer pulls it in with gcc. Without it the probe stops at fatal error: errno.h, and the whole Landlock half of the survey goes unanswered while the Bubblewrap half still reports, which is easy to read past.
  3. Run the Landlock assertion in the same container, replacing --report with --assert-landlock.
  4. After merge, dispatch Runner Capabilities from the Actions tab and read the run summary. It cannot be dispatched before merge: GitHub only offers workflow_dispatch for a workflow that is already on the default branch, which is why this change comes before the CI that uses it.

Expected result

  1. All six Lint jobs pass.
  2. The survey prints one line per fact: kernel, architecture, effective uid, Landlock ABI, whether Landlock enforces on the host and in a plain container, the two AppArmor user-namespace sysctls, and the Bubblewrap version. The last line under Bubblewrap reads rejected: --unshare-utc is not an option, --unshare-uts is.
  3. The assertion prints ok for the ABI, for the fixture baseline, for the applied ruleset, for the permitted read, for the refused read and for the host as a whole, and then verdict: indeterminate, the probe could not answer, exit 3. That is the right answer from inside a container: there is no Docker daemon in it, so the container half cannot be asked, and an unanswered half must never be reported as available. On a GitHub runner both halves are answered and the verdict is available.
  4. The summary shows a two-row table naming each capability available, unavailable or indeterminate.

Negative case (what must still be rejected)

The probe must never report a capability as present when it is not, so both false-positive paths are checked directly.

  1. A Bubblewrap that prints the expected marker and then fails must not pass. Put a fake bwrap on PATH that runs echo sandboxed-build-ran; exit 73 and run --assert-bwrap: it must print FAIL ... (exit 73) and verdict: unavailable.
  2. A forbidden file that was already unreadable must not be mistaken for enforcement. chmod 000 the fixture's forbidden file before the probe runs and the probe must exit indeterminate with the forbidden file is unreadable before any restriction, not report enforcement.
  3. Running --assert-bwrap as root must refuse to answer, because root is exempt from the restriction being measured.
  4. An absent capability and a probe that could not answer must stay distinguishable: the first leaves the dispatch green, the second fails it.

One check cannot be made in an ordinary container. Proving that the Bubblewrap invocation is well formed needs a host where user namespaces are permitted, so that one was run once under --privileged, purely to confirm the sandbox assembles and the script inside it runs. Every other result above was produced without any security flag.

Layers exercised

No layer-specific behaviour changed.

  • Filesystem layer
  • Network layer
  • Timeout layer
  • All three together, as a run uses them by default

5. Test case coverage regarding this PR

No behaviour covered by the suites changed.

Breaking changes and migration

No breaking changes or migration.

Checklist

  • The title of this pull request describes the change, not the implementation.
  • I have self-reviewed the diff of this pull request.
  • Any weakening of the sandbox boundary is stated explicitly above, including what it now permits that it did not permit before.
  • The change was exercised in a container started without --privileged, --cap-add or --security-opt, or the manual says why that was not possible.
  • CI is green, or every remaining failure is explained above.
  • No secrets, tokens or absolute local paths are contained in the diff.

Review progress

  • Code review
  • Manual test

@MarkusPaulsen
MarkusPaulsen requested a review from a team September 9, 2026 18:39
@MarkusPaulsen
MarkusPaulsen requested review from a team and krusche as code owners September 9, 2026 18:39
Markus Paulsen added 2 commits September 9, 2026 22:12
Two planned jobs depend on the machine rather than on this repository:
running the Landlock acceptance suites inside an ordinary container, and
running the pruner, which needs Bubblewrap and therefore unprivileged
user namespaces. Neither can be read out of the source, and a runner
image can withdraw either without notice, so the question is asked in one
place and asked the same way every time.

The probe walks the whole Landlock path rather than reading the version:
it proves both fixture files are readable to begin with, applies a
ruleset, and then proves the permitted file is still readable and the
forbidden one is not. The opening baseline is what makes the answer
trustworthy, since a file some other mechanism had already made
unreadable would otherwise look exactly like Landlock working. It
requires ABI 4, which is what the helper itself demands for the network
rules the acceptance suite exercises with --connect-tcp.

The Bubblewrap assertion runs a real script in a production-shaped
sandbox rather than bwrap /bin/true, since the namespaces, the tmpfs root
and the bound work directory are the parts that fail on a restricted
host. It refuses to answer as root, which is exempt from the very
restriction being measured.

Verdicts are three-valued. A capability this repository does not yet use
being absent is a fact to record, so it leaves the dispatch green; a
probe that could not reach a verdict is not a fact about anything, and
fails the run instead.

The workflow is manual on purpose. It is a diagnostic, not a gate.
main moved its jobs to ubuntu-26.04 while this branch was open, so the
probe was left asking its questions of a different machine from the one
the rest of CI uses, which is the one thing a capability probe must not
do. The container it starts follows for the same reason.
@MarkusPaulsen
MarkusPaulsen force-pushed the chore/runner-capability-probe branch from 04213aa to b1adb01 Compare September 9, 2026 20:13
@MarkusPaulsen
MarkusPaulsen merged commit a198c6c into main Sep 11, 2026
15 of 16 checks passed
@MarkusPaulsen
MarkusPaulsen deleted the chore/runner-capability-probe branch September 11, 2026 15:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant