Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 11 additions & 1 deletion .github/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,21 @@ Describe the concrete trigger and resulting behavior.
## Evidence

- Failing test before implementation (or explain a documentation-only change):
- Passing commands/results:
- Focused passing commands/results while iterating:
- Stable candidate hosted CI run/checks (exact head SHA):
- Relevant fault/security/resource cases:
- Independent review and model:
- Tests not run and why:

## Incremental review ledger

- Candidate source SHA and files/risk surface:
- Findings batched before the candidate push:
- Previously resolved findings carried forward with original URL, source SHA and resolution evidence:
- Final delta sign-off against the exact candidate SHA:
- New full review needed because of changed risk/interface (or explain why not):
- Security second pass (when applicable):

## Operational effect

State migrations, cleanup ownership, rollout/rollback and remaining limitations. Do not include raw credentials or private job logs.
27 changes: 25 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,31 @@

Start with an issue in the implementation Project. Read [the plan](docs/PLAN.md), [agent execution](docs/EXECUTION.md) and [TDD strategy](docs/TEST-STRATEGY.md).

Keep changes scoped to an observable issue goal. Include a reproducing failing test for behavior changes, relevant checks, independent review of security/concurrency changes, and accurate limitations. Documentation-only changes do not require artificial tests.
Keep changes scoped to an observable issue goal. The writer follows red -> minimal
green -> refactor, then runs focused unit/negative checks while iterating;
documentation-only changes record why no artificial test is needed. Use
`make fast` only with explicit `FAST_MODULE`, `FAST_PACKAGE` and `FAST_TEST`
selectors; it fails closed for missing or no-match selectors and is not the full
gate. Batch source, documentation and finding-ledger fixes before pushing one
stable review candidate rather than launching a review for every local commit.

The public baseline is documented in [CI.md](docs/CI.md). Run `make check` before submitting a change; it uses the pinned Go toolchain and reports absent unit or fuzz suites explicitly.
Independent reviewers use the immutable candidate source and exact-source CI
evidence, adding delta/risk probes instead of repeating the complete suite. A
second independent security/recovery pass remains required when the changed
boundary warrants it. The coordinator audits the contract, ledger and evidence;
the coordinator is not a third full-suite tester. Carry resolved findings forward
with their original URL, source SHA and resolution evidence, and sign off the
final delta against the exact candidate SHA.

The public baseline is documented in [CI.md](docs/CI.md). `make check` remains the
complete public validation suite and hosted PR CI remains the required stable
candidate gate; it is not an automatic per-commit requirement. The final merge
gate still requires exact-head GitHub Codex review, required CI for that same SHA,
stale/outdated finding resolution or rebuttal, and any applicable security second
pass. Main's postmerge integration run does not replace those premerge checks.

Release, macOS, soak and other trusted/live checks run before the applicable
release or live qualification only, with an immutable reviewed commit and
explicit maintainer authorization. No mandatory security gate is deferred.

Do not run unreviewed contributions on privileged self-hosted runners. Do not submit credentials or private workflow logs. Dependencies and assets must have documented licenses compatible with distribution.
10 changes: 9 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,21 @@ GOFLAGS ?=
override GOTOOLCHAIN := go1.26.8
export GOTOOLCHAIN
export GOFLAGS
FAST_MODULE := $(value FAST_MODULE)
FAST_PACKAGE := $(value FAST_PACKAGE)
FAST_TEST := $(value FAST_TEST)
export FAST_MODULE FAST_PACKAGE FAST_TEST

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve dollar signs in environment-provided selectors

When FAST_TEST is supplied using the documented environment-prefix form and contains a nonterminal $, GNU make imports it as a recursively expanded make variable and this export expands the $ reference before the script receives it. For example, FAST_TEST='^TestA$.*' make fast passes ^TestA*, which can run and accept TestB even though the requested selector only matched TestA; GNU make's variable-reference rules treat $x as a one-character variable reference. Preserve the raw environment value or require $$ escaping for environment assignments too.

AGENTS.md reference: AGENTS.md:L7-L7

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Preserve raw FAST selector values with immediate Make assignments using $(value ...). The regression was already green on this host GNU Make3.81; no pre-fix red is claimed for this finding. Independent review will verify preservation and scope rather than infer resolution from staleness. Candidate a78792d; focused verification selected 19 top-level tests plus 32 subtests using env GOTOOLCHAIN=go1.26.8 GOWORK=off go test -count=1 ./scripts -run '^(TestFastCheck|TestToolingGOFLAGSUsesGoEnvironment|TestToolingPinsNewerSystemGo)' (PASS, about31s), helper tests PASS. Independent delta review and exact-head CI/Codex remain pending.

FUZZTIME ?= 1s
GOVULNCHECK_VERSION ?= v1.7.0

.PHONY: help build check toolchain fmt fmt-check vet test test-race fuzz-smoke deps licenses vuln experiments
.PHONY: help build check fast toolchain fmt fmt-check vet test test-race fuzz-smoke deps licenses vuln experiments

help:
@printf '%s\n' \
'gh-runnerd development commands:' \
' make fmt format Go sources in place' \
' make check run the complete public validation suite' \
' make fast run one explicit FAST_MODULE/FAST_PACKAGE/FAST_TEST selector (not the full gate)' \
' make build compile all packages' \
' make experiments test reviewed offline gate modules when present' \
' make vuln run pinned govulncheck (network access may be needed)'
Expand All @@ -25,6 +30,9 @@ build:

check: toolchain fmt-check build vet test test-race fuzz-smoke deps licenses experiments vuln

fast:
GO="$(GO)" bash scripts/fast-check.sh

toolchain:
GO="$(GO)" bash scripts/check-toolchain.sh

Expand Down
79 changes: 79 additions & 0 deletions docs/CI.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,12 +22,52 @@ disabled. The aggregator uses `always()` so a failed, cancelled, or skipped
required job is observed, then fails unless all three dependency results are
exactly `success`; it does not check out source or run a test itself.

The pull-request workflow is the premerge source gate. The identical workflow on
`main` is postmerge integration evidence; a green postmerge run cannot substitute
for the exact-head premerge gate. During local editing, use focused checks and the
opt-in `make fast` selector below. After source, documentation and finding-ledger
changes are batched into one stable candidate, hosted CI supplies the complete
matrix; reviewers and the coordinator use that exact-source evidence rather than
rerunning the full suite independently. Release, macOS, soak and other trusted
profiles run only before an applicable release/live qualification, with explicit
maintainer authorization, and are not implied by this hosted check.

### Issue #73 audit evidence

This audit was captured from `origin/main` at
`cf67d4aeb511116fee0de31a4ac38409f28fa29f` and the live PR #72 record on
2026-09-10. It records measured observations, not a forecast of savings:

| Surface | Measured observation | Policy consequence |
| --- | --- | --- |
| Main CI | `ci.yml` has three parallel test-bearing jobs (`root`, `offline`, `vuln`) plus the required `Go checks` aggregator; each remains capped at 15 minutes, with pinned actions and `cache: false`. | Preserve the current complete coverage, job/check names, cache policy and timeouts; no path classifier or cache shortcut was added. |
| Makefile | Existing `check` prerequisites remain `toolchain`, formatting, build, vet, unit/race, fuzz, dependency, license, offline-experiment and vulnerability checks. New `fast` is an opt-in target and is not a `check` prerequisite. | Keep `make check` complete and unchanged as the local public gate; focused iteration cannot silently weaken it. |
| [PR #72 review history](https://github.com/1XP-AI/gh-runnerd/pull/72) | Audit snapshot through immutable PR #72 head `116beda04dc2bf69280cdefc4de4ef2fef397ef3`, captured 2026-09-10. Review records and public checkpoints do not measure actor-side full-suite run counts; the checkpoint comments identify focused/offline or focused race regressions. | Batch findings, source and docs before one candidate push; reviewers perform delta/risk probes against shared CI evidence, and the coordinator audits rather than acting as a third tester. |
| [PR #72 hosted critical path](https://github.com/1XP-AI/gh-runnerd/actions/runs/34419651240) | `gh run view 34419651240 --json jobs` measured workflow start `00:04:06Z`, required jobs finishing by `00:15:14Z`, and aggregator completion at `00:15:18Z`; `00:15:19Z` is a workflow metadata update, not completion. Start-to-aggregator completion was 11m12s. Root ran 11m06s, offline 8m49s, vulnerability 33s, aggregator 2s. | No workflow critical-path speedup is claimed or changed; full CI remains the stable candidate gate. |
| Focused local command | Warmed direct baseline: `env GOTOOLCHAIN=go1.26.8 GOWORK=off go test -count=1 -run '^TestFixedTarget$' ./internal/scheduler/capacity` → `real 0.32s`. New entry point: `env FAST_MODULE=. FAST_PACKAGE=./internal/scheduler/capacity FAST_TEST='^TestFixedTarget$' make fast` → `real 0.34s`; both passed. | This one local pair demonstrates bounded behavior only; it does not claim a speedup or predict CI duration. |

```console
$ gh run view 34419651240 --repo 1XP-AI/gh-runnerd --json headSha,startedAt,updatedAt,jobs
# head 116beda04dc2bf69280cdefc4de4ef2fef397ef3; start 00:04:06Z; aggregator 00:15:18Z; metadata update 00:15:19Z;
# jobs: root 00:04:08Z-00:15:14Z, offline 00:04:09Z-00:12:58Z,
# vuln 00:04:10Z-00:04:43Z, checks 00:15:16Z-00:15:18Z
```

The audit commands and exact observed values are kept in this single summary
instead of accumulating per-step logs. Finding ledgers must retain original URLs,
source SHAs and resolution evidence without copying credentials or private job
logs.

Run the same checks locally with:

```console
make check
```

`make check` remains the complete local public gate, but it is not a per-commit
requirement; use it on demand when the environment supports it and rely on hosted
CI for the stable candidate gate.

Individual commands are available when iterating:

| Command | Check |
Expand All @@ -43,6 +83,45 @@ Individual commands are available when iterating:
| `make licenses` | Compare the exact runtime module/version/replacement graph with its inventory and require a top-level license file. |
| `make experiments` | Require both established G01/G02 modules, run their static-partitioned default race/vet suites, then exercise the two explicitly reviewed G01 CLI packages with `g01_live,g01_worker` tags and the reviewed `g01_pair_fixture` livecanary collection/listener and terminal partitions with tagged vet. |
| `make vuln` | Run the exact `golang.org/x/vuln/cmd/govulncheck@v1.7.0` tool. |
| `make fast FAST_MODULE=... FAST_PACKAGE=... FAST_TEST=...` | Run one explicit test selector in one selected module/package. All three selectors are required and invalid/no-match selectors fail closed; this is focused evidence only, never the complete gate. |

For example:

```console
FAST_MODULE=. FAST_PACKAGE=./internal/scheduler/capacity FAST_TEST='^TestFixedTarget$' make fast
```

The script resolves the selected module and every package matched by the Go
pattern, preserving wildcard forms such as `./.../capacity`, and rejects
selectors whose physical paths leave the current repository or selected module,
as well as absolute or lexical `..` paths, missing `go.mod`, missing selectors,
package patterns that match no package, and selectors that match no compiled
test. The focused invocation explicitly bounds Go's test execution flags:
inherited `-bench` and `-fuzz` selectors are cleared so they cannot expand work
beyond `FAST_TEST`; `-list`, `-skip`, and build-only `-c` are cleared so they
cannot suppress it; explicit `-run`/`-count` values retain the requested
selector and `-cpu=1` bounds each selected test to one execution. Inherited
`-exec` is cleared so a wrapper cannot bypass the test binary. The command forces
Go's `-test.v=test2json` framed output while disabling the outer `-json` mode, and
requires paired framed per-test `=== RUN` and terminal `--- PASS` lines whose test
name is a complete match under Go's slash-separated component and top-level
alternation semantics; possible-parent events do not count. Framed `skip` or
`fail` outcomes, including a later contradictory terminal after the first
terminal in one `RUN` lifecycle, package summaries, and ordinary unframed
`TestMain` output do not count. Each new framed `RUN` starts a new lifecycle for
that name, so a same-named test in a later package may qualify after an earlier
package skipped it. Dry-run or unsupported build modes fail closed when no
passing framed test result is observed. The frame is a Go test protocol boundary
against ordinary output, not authentication against a deliberately spoofing test
process; this remains a trusted local helper, not a hostile-code sandbox. Useful build
flags such as `-race` and `-mod=readonly` remain inherited; this is not a blanket
`GOFLAGS` removal. If selectors are passed as Make command-line variables instead
of environment assignments, escape literal `$` as `$$` so Make preserves the
regexp anchor.

`FAST_TEST` is passed as one argument and may contain whitespace in a Go
subtest name; `FAST_MODULE` and `FAST_PACKAGE` remain whitespace-free path
selectors.

No hardware, live GitHub, Docker or daemon suite is part of this public check. Those profiles remain explicit future or maintainer-controlled runs; they are not silently converted into passing tests here. G04 introduces the first application behavior contracts and should add meaningful unit and fuzz targets before claiming those forms of coverage.

Expand Down
3 changes: 3 additions & 0 deletions docs/DEPENDENCIES.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,9 @@ runtime graph contains only the local module; nested experiments remain separate
Go standard library: `go1.26.8`, BSD-3-Clause, the language/runtime baseline;
[official release history](https://go.dev/doc/devel/release).

The selector matcher helper adapted from the Go standard library is documented
in the [Go testing matcher adaptation notice](third-party/go-testing-matcher.md).

## Public CI actions

Action refs are immutable commit pins. The version labels are recorded for human review; the workflow uses the full SHA.
Expand Down
31 changes: 31 additions & 0 deletions docs/EXECUTION.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,37 @@ The repository default is Luna max, including contract-setting evidence gates su

Verify each issue's contract and dependencies before dependent implementation begins. Escalate discovered architecture, secret handling, process isolation or concurrency changes to an additional independent Luna max review; do not stretch a small issue into an unreviewed redesign. Parallelize only independent issues with non-overlapping file ownership; no simultaneous edits to shared protocol/state definitions.

## Incremental validation and review

Validation is incremental during editing and complete at the stable merge
candidate. Do not require or automatically run the whole suite for every local
commit. The writer records a meaningful red case for behavior changes, applies a
minimal green fix, and runs the explicit focused unit/negative checks relevant to
the changed surface; documentation-only changes record why no artificial test is
needed. The opt-in `make fast` entry point requires an explicit module, package,
and test selector, fails closed for missing/invalid selectors, and never stands in
for the complete gate.

| Role/tier | Required work and evidence |
|---|---|
| Writer / edit | Red -> minimal green -> refactor; focused checks and `git diff --check` while iterating. Batch source, docs and finding-ledger changes before one review candidate push. |
| Independent reviewer(s) | Review the immutable candidate source and shared exact-source CI evidence; run only delta/risk probes. Add the independent Luna max security/recovery pass when the changed boundary warrants it. Do not duplicate the full suite by default. |
| Coordinator | Audit the contract, changed surface, finding ledger and exact-source CI/review records; coordinate resolution. The coordinator is not a third full-suite tester. |
| Candidate / premerge | The pushed PR head receives the complete hosted CI matrix once stable. Preserve required job/check names, complete coverage, exact pull-request-head checkout, cache policy and timeouts. |
| Main / postmerge | The same workflow on `main` is integration evidence after merge; it does not replace the premerge candidate gate or authorize a merge. |
| Release/live qualification | Run release, macOS, soak and other trusted/live profiles only before the applicable release or live qualification, on a reviewed immutable commit with explicit maintainer authorization. Do not defer a mandatory security gate. |

Use one finding ledger per candidate: carry each previously resolved finding with
its immutable source SHA, original finding URL and resolution evidence, then record
the final delta sign-off against the exact candidate SHA. A new internal full review
is needed only when the changed diff crosses a new risk or interface boundary;
focused delta review remains required for ordinary fixes. GitHub Codex review and
the required CI gate are separate and stricter: immediately before merge, Codex
must have reviewed the exact final HEAD, stale/outdated findings must be read and
resolved or rebutted, required CI must pass for that same SHA, and a security
second pass remains required where applicable. Do not request repetitive Codex
reviews mid-edit; after a fix push, request and await the fresh exact-head review.

## Per-issue goal workflow

1. Select a Ready issue from the live GitHub Project whose dependencies are Done; read the plan, relevant ADR and current repository instructions. Do not treat `backlog.json` `status` as live Ready.
Expand Down
Loading
Loading