Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/local-cli-audit.yml
Original file line number Diff line number Diff line change
Expand Up @@ -248,6 +248,7 @@ jobs:
run: |
set -euo pipefail
export PATH="${CODE_MOWER_LOCAL_AUDIT_PATH}:${PATH}"
export PYTHONPATH="${SUPPORT_PATH}/src${PYTHONPATH:+:${PYTHONPATH}}"

if [ -z "${CODE_MOWER_CLOUD_TOKEN:-}" ]; then
echo "::notice::CODE_MOWER_CLOUD_TOKEN is not configured; skipping audit metadata upload."
Expand Down
86 changes: 84 additions & 2 deletions docs/local-audit-runner.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,89 @@ logins. Install the GitHub runner as a service only after the generated local
audit workflow passes the same smoke checks.

For service mode, set `USER`, `LOGNAME`, `SHELL`, and `LANG` in the runner
`.env`, then fully recycle the listener after edits. `svc.sh stop/start` may
`.env`, along with `CODE_MOWER_PYTHON` as described below, then fully recycle
the listener after edits. `svc.sh stop/start` may
leave an older `Runner.Listener` process alive with the previous environment.

## Stable Python environment

The source wrappers use `scripts/dev-python` and require Python 3.12+ with the
runtime dependencies declared in the trusted support checkout's `pyproject.toml`
(currently `PyYAML>=6.0` and `packaging>=23.2`). A Python executable alone is not
enough. Provision a dedicated virtual environment outside the runner's disposable
work directories, using an installed Python 3.12+ interpreter and a reviewed
Code Mower source checkout:

```bash
python3.12 -m venv "$HOME/.local/share/code-mower/audit-venv"
"$HOME/.local/share/code-mower/audit-venv/bin/python" -m pip install /absolute/path/to/reviewed/code-mower
"$HOME/.local/share/code-mower/audit-venv/bin/python" -m pip check
```

Installing that checkout installs its declared runtime dependencies. Repeat the
install when those requirements change. Never install dependencies from a PR
checkout into this trusted environment. Configure the runner `.env` with the
**literal absolute path**, for example:

```dotenv
CODE_MOWER_PYTHON=/absolute/path/to/audit-venv/bin/python
```

Replace this example with the full path to the environment created above;
`.env` does not expand `$HOME` or `~`.
Keep the interpreter path stable across support-checkout resets and recycle the
listener after configuring it. Do not rely on shell activation or an interactive
shell's Python selection. `scripts/dev-python` prefers a `.venv` in its current
working directory over `CODE_MOWER_PYTHON`, so keep that job directory free of a
shadowing `.venv`; the preflight below rejects a different selected interpreter.

After the trusted default-branch support checkout, run this preflight in an
actual runner job with `SUPPORT_PATH` set to that checkout. Run from the job
workspace, never the PR checkout. It checks the same interpreter selector and
source imports used by the wrappers and metadata upload, without invoking a
provider or uploading anything:

```bash
set -euo pipefail
export PYTHONPATH="${SUPPORT_PATH}/src${PYTHONPATH:+:${PYTHONPATH}}"
if ! {
test -n "${CODE_MOWER_PYTHON:-}" &&
test -x "${CODE_MOWER_PYTHON}" &&
"${CODE_MOWER_PYTHON}" -m pip check &&
"${SUPPORT_PATH}/scripts/dev-python" - <<'PY'
import os
import sys
from pathlib import Path
import tomllib

assert sys.version_info >= (3, 12)
assert os.path.abspath(sys.executable) == os.path.abspath(os.environ["CODE_MOWER_PYTHON"])
from importlib.metadata import version
from packaging.requirements import Requirement
import yaml
import code_mower.cli

support = Path(os.environ["SUPPORT_PATH"])
assert Path(code_mower.cli.__file__).resolve() == (support / "src/code_mower/cli.py").resolve()
project = tomllib.loads((support / "pyproject.toml").read_text())["project"]
for declared in project["dependencies"]:
requirement = Requirement(declared)
if requirement.marker is None or requirement.marker.evaluate():
assert requirement.specifier.contains(version(requirement.name))
PY
} >/dev/null 2>&1; then
echo "::error::Code Mower Python preflight failed; check CODE_MOWER_PYTHON and trusted runtime dependencies."
exit 1
fi
echo "Code Mower Python preflight passed"
```

Keep installation diagnostics and any provider stdout/stderr in private local
logs. Do not dump the runner environment, tokens, auth status payloads, or raw
provider output into Actions logs or artifacts.

## Runner account preflight

Check `~/Library/LaunchAgents/actions.runner.*.plist` after `svc.sh install`.
If it contains `SessionCreate=true`, remove that key and unload/reload the
LaunchAgent or recycle the listener. That launchd setting creates a new security
Expand All @@ -23,10 +103,12 @@ Verify from a runner job, not only from an interactive terminal:
gh auth status >/dev/null 2>&1 && echo "gh auth ok" || { echo "gh auth NOT ready"; false; }
codex --version
claude auth status >/dev/null 2>&1 && echo "claude auth ok" || { echo "claude auth NOT ready"; false; }
claude -p "Reply with exactly: ok" --output-format json
devin auth status >/dev/null 2>&1 && echo "devin auth ok" || { echo "devin auth NOT ready"; false; }
```

Check only providers enabled for this runner. These are auth readiness checks;
they do not start a provider canary or repeat an audit.

Local Claude and Codex merge-authority audits normally publish through
`.github/workflows/local-audit-publication.yml`. The self-hosted audit job uses
its short-lived `GITHUB_TOKEN` to dispatch this default-branch workflow. The
Expand Down
4 changes: 3 additions & 1 deletion src/code_mower/audit_publication.py
Original file line number Diff line number Diff line change
Expand Up @@ -246,7 +246,9 @@ def verify_run(run, repository, *, run_id=None, terminal=False):
isinstance(run.get("head_sha"), str) and SHA.fullmatch(run["head_sha"]),
"invalid workflow SHA",
)
require(run.get("display_title") == WORKFLOW_NAME, "invalid run title")
# repository_dispatch uses the event type as display_title; name identifies
# the workflow, alongside the trusted path checked above.
require(run.get("name") == WORKFLOW_NAME, "untrusted workflow name")
if terminal:
require(
run.get("status") == "completed" and run.get("conclusion") == "success",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -246,6 +246,7 @@ __LOCAL_AUDIT_TOKEN_ENV_ASSIGNMENTS__
run: |
set -euo pipefail
export PATH="${CODE_MOWER_LOCAL_AUDIT_PATH}:${PATH}"
export PYTHONPATH="${SUPPORT_PATH}/src${PYTHONPATH:+:${PYTHONPATH}}"

if [ -z "${CODE_MOWER_CLOUD_TOKEN:-}" ]; then
echo "::notice::CODE_MOWER_CLOUD_TOKEN is not configured; skipping audit metadata upload."
Expand Down
1 change: 1 addition & 0 deletions templates/workflows/self-hosted-local-audit.yml.j2
Original file line number Diff line number Diff line change
Expand Up @@ -246,6 +246,7 @@ __LOCAL_AUDIT_TOKEN_ENV_ASSIGNMENTS__
run: |
set -euo pipefail
export PATH="${CODE_MOWER_LOCAL_AUDIT_PATH}:${PATH}"
export PYTHONPATH="${SUPPORT_PATH}/src${PYTHONPATH:+:${PYTHONPATH}}"

if [ -z "${CODE_MOWER_CLOUD_TOKEN:-}" ]; then
echo "::notice::CODE_MOWER_CLOUD_TOKEN is not configured; skipping audit metadata upload."
Expand Down
76 changes: 74 additions & 2 deletions tests/test_audit_publication.py
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,8 @@ def run_for(value, *, terminal=False, comment_id=91, **changes):
head_repository=REPOSITORY,
head_branch="main",
head_sha=SOURCE,
display_title=pub.WORKFLOW_NAME,
name=pub.WORKFLOW_NAME,
display_title=pub.EVENT,
status="completed" if terminal else "in_progress",
conclusion="success" if terminal else None,
)
Expand Down Expand Up @@ -366,13 +367,15 @@ def test_run_provenance_refuses_untrusted_repository_workflow_and_ref(self):
bad = [
dict(id=900),
dict(path=".github/workflows/evil.yml"),
dict(path=None),
dict(name="Other workflow", display_title=pub.WORKFLOW_NAME),
dict(name=None),
dict(event="workflow_dispatch"),
dict(run_attempt=2),
dict(head_branch="codex/topic"),
dict(head_repository={"id": 90}),
dict(repository={"id": 1234, "full_name": "other/repo"}),
dict(head_sha="bad"),
dict(display_title="untrusted text"),
]
for change in bad:
api = MemoryGitHub()
Expand All @@ -381,6 +384,26 @@ def test_run_provenance_refuses_untrusted_repository_workflow_and_ref(self):
pub.publish(event_for(api.value), environment(), api, now=NOW)
self.assertEqual(api.writes, [])

def test_repository_dispatch_title_does_not_define_publication_identity(self):
for title in (pub.EVENT, pub.WORKFLOW_NAME, "Custom run title", None):
with self.subTest(title=title):
api = MemoryGitHub()
api.run["display_title"] = title
pub.publish(event_for(api.value), environment(), api, now=NOW)
self.assertEqual([method for method, _, _ in api.writes], ["POST", "PATCH"])
api.run.update(status="completed", conclusion="success")
self.assertTrue(
pub.attested(
body=api.comments[0]["body"],
comment_id=91,
repo=REPO,
issue_number=42,
head_sha=HEAD,
run=api.run,
repository=REPOSITORY,
)
)

def test_head_moves_at_each_publication_boundary_fail_closed(self):
for at in (1, 2, 3):
api = MemoryGitHub()
Expand Down Expand Up @@ -462,6 +485,10 @@ def check(**changes):
for changes in (
dict(status="in_progress"),
dict(conclusion="failure"),
dict(name="Other workflow", display_title=pub.WORKFLOW_NAME),
dict(name=None),
dict(path=".github/workflows/evil.yml"),
dict(path=None),
dict(run_attempt=2),
dict(publication_jobs=[]),
dict(publication_jobs=[run["publication_jobs"][0]] * 2),
Expand Down Expand Up @@ -492,6 +519,51 @@ def test_workflow_has_no_unvalidated_output_or_pr_execution(self):


class WrapperTests(unittest.TestCase):
def test_metadata_upload_imports_support_package_from_job_workspace(self):
import yaml

workflow = yaml.safe_load((ROOT / ".github/workflows/local-cli-audit.yml").read_text())
step = next(
s for s in workflow["jobs"]["audit"]["steps"]
if s.get("name") == "Upload Code Mower audit metadata"
)
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
support = root / "support"
scripts = support / "scripts"
scripts.mkdir(parents=True)
selector = scripts / "dev-python"
selector.write_bytes((ROOT / "scripts/dev-python").read_bytes())
selector.chmod(0o755)
for directory, source in (
(support / "src", "import sys; print(sys.argv[2])\n"),
(root / "ambient", "raise AssertionError('wrong package')\n"),
):
module = directory / "code_mower"
module.mkdir(parents=True)
(module / "__init__.py").write_text("")
(module / "cli.py").write_text(source)
env = dict(
os.environ,
SUPPORT_PATH=str(support),
PR_HEAD_PATH=str(root / "pr-head"),
PYTHONPATH=str(root / "ambient"),
CODE_MOWER_PYTHON=sys.executable,
CODE_MOWER_LOCAL_AUDIT_PATH=os.environ["PATH"],
CODE_MOWER_CLOUD_TOKEN="fixture-token",
CODE_MOWER_INSTALL_ID="fixture-install",
CODE_MOWER_REVIEWER_SPEND_PATH=str(root / "spend.json"),
RUNNER_TEMP=str(root),
GITHUB_REPOSITORY=REPO,
PR_NUMBER="42",
)
result = subprocess.run(
["bash", "-c", step["run"]], cwd=root, env=env, capture_output=True, text=True
)
self.assertEqual(result.returncode, 0, result.stderr)
self.assertEqual(result.stdout.splitlines(), ["reviewer-runs", "dogfood"])
self.assertNotIn("fixture-token", result.stdout + result.stderr)

def test_real_source_step_seals_blocked_without_leaking_tokens_or_output(self):
import yaml

Expand Down
4 changes: 3 additions & 1 deletion tools/audit_publication.py
Original file line number Diff line number Diff line change
Expand Up @@ -246,7 +246,9 @@ def verify_run(run, repository, *, run_id=None, terminal=False):
isinstance(run.get("head_sha"), str) and SHA.fullmatch(run["head_sha"]),
"invalid workflow SHA",
)
require(run.get("display_title") == WORKFLOW_NAME, "invalid run title")
# repository_dispatch uses the event type as display_title; name identifies
# the workflow, alongside the trusted path checked above.
require(run.get("name") == WORKFLOW_NAME, "untrusted workflow name")
if terminal:
require(
run.get("status") == "completed" and run.get("conclusion") == "success",
Expand Down
Loading