Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/workflows/pr-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -123,6 +123,10 @@ jobs:
- name: test E2E runtime helpers
run: go test -race -count=1 ./e2e/framework/runenv/... ./e2e/framework/dockerwsl ./e2e/cmd/devsy-ci

- name: check E2E CI label coverage
if: runner.os == 'Linux'
run: go run ./e2e/cmd/devsy-ci check-labels

- name: test Compose UID expectation matrix
shell: bash
run: |
Expand Down
23 changes: 23 additions & 0 deletions e2e/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,3 +73,26 @@ Ginkgo executes the specs directly. `devsy-ci check-report` checks its JSON
report: all six BuildKit specs must pass on their first attempt, with no
selected skips, dry run or programmatic focus. Teardown runs on success and
failure, before the report and non-secret diagnostics are uploaded.

## Label coverage guard

`devsy-ci check-labels` requires every `It` spec in the e2e suite to match at least one
CI label filter, and every filter to select at least one spec. Filters come from the
`matrix.label` rows and the literal `--ginkgo.label-filter=` values in
`.github/workflows/pr-ci.yml` and `.github/workflows/docker-buildkit-windows.yml`.
Expressions such as `${{ matrix.label }}` are skipped. Filters are evaluated with
Ginkgo's own parser.

Run it from the repository root:

```bash
go run ./e2e/cmd/devsy-ci check-labels
```

The command builds the e2e test binary and lists its specs with `-ginkgo.dry-run`. Pass
`--report <path>` to check an existing Ginkgo JSON report instead.

Spec sets depend on the host OS, because build-tagged files such as `*_linux.go` are
excluded elsewhere. Run the check on Linux to match the `unit-tests` CI job, which
gates the step on `runner.os == 'Linux'`. On macOS, Linux-only labels report as
selecting zero specs.
2 changes: 1 addition & 1 deletion e2e/cmd/devsy-ci/command.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,6 @@ func newRootCommand(deps *runenv.Dependencies) *cobra.Command {
SilenceUsage: true,
SilenceErrors: true,
}
root.AddCommand(dockerwsl.NewCommand(deps), newReportCommand(deps))
root.AddCommand(dockerwsl.NewCommand(deps), newReportCommand(deps), newCheckLabelsCommand(deps))
return root
}
2 changes: 1 addition & 1 deletion e2e/cmd/devsy-ci/command_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,5 +19,5 @@ func (s *CommandSuite) TestRootCommandSurface() {
for _, c := range root.Commands() {
names = append(names, c.Name())
}
s.ElementsMatch([]string{"docker-wsl", reportCommandName}, names)
s.ElementsMatch([]string{"docker-wsl", reportCommandName, checkLabelsCommandName}, names)
}
Loading
Loading