Skip to content

Support matrix include and exclude expansions #68

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
gjkim42 wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

Support matrix include and exclude expansions #68

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions api/v1alpha1/workflowjob_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,8 +65,9 @@ type WorkflowJobMatrix struct {
// +required
LogicalJobID string `json:"logicalJobID"`

// Values contains the scalar matrix axis values for this combination.
// Values are represented using their workflow string form.
// Values contains the scalar matrix values for this combination, including
// values added by an include transformation. Values are represented using
// their workflow string form.
// +kubebuilder:validation:MinProperties=1
// +kubebuilder:validation:MaxProperties=100
// +kubebuilder:validation:XValidation:rule="self.all(k, size(k) > 0 && size(k) <= 256 && size(self[k]) <= 1024)",message="matrix keys must contain 1 to 256 characters and values must contain at most 1024 characters"
Expand Down
63 changes: 46 additions & 17 deletions docs/reference.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -148,14 +148,41 @@ trusted. See
[`config/samples/actions_v1alpha1_docker_runner.yaml`](../config/samples/actions_v1alpha1_docker_runner.yaml)
for a Docker-enabled Runner.

A job strategy may define scalar matrix axes and an optional positive
`max-parallel`. The controller creates one `WorkflowJob` per Cartesian-product
combination in deterministic order. Each child has a unique `spec.jobID`, while
`spec.matrix.logicalJobID`, `values`, and `maxParallel` preserve its logical
identity and scheduling group. `max-parallel` limits active children in that
group independently of the number of matching Runners. A failed matrix child
makes the completed WorkflowRun fail. Strategy `fail-fast` is not supported;
remaining combinations continue to completion after a child fails.
A job strategy may define scalar matrix axes, `include` and `exclude`
transformations, and an optional positive `max-parallel`. `exclude` mappings
remove every Cartesian-product combination that partially matches their values.
The controller then applies `include` mappings in declaration order. An include
augments every compatible original combination and may overwrite values added by
an earlier include, but it does not overwrite original axis values. An include
that matches no original combination is appended as a standalone combination.
A matrix may consist only of `include` mappings.

```yaml
strategy:
max-parallel: 2
matrix:
os: [linux, darwin]
version: [1, 2]
exclude:
- os: darwin
version: 1
include:
- os: linux
coverage: true
- os: windows
version: 2
```

The controller creates one `WorkflowJob` per transformed combination. Axis and
value declaration order determines the order of Cartesian-product combinations,
followed by standalone includes in declaration order. Each child has a unique
`spec.jobID`, while `spec.matrix.logicalJobID`, `values`, and `maxParallel`
preserve its logical identity and scheduling group. Every scalar axis or include
value is available through the `matrix` expression context and persisted in
`spec.matrix.values`. `max-parallel` limits active children in that group
independently of the number of matching Runners. A failed matrix child makes the
completed WorkflowRun fail. Strategy `fail-fast` is not supported; remaining
combinations continue to completion after a child fails.

### Conditions

Expand Down Expand Up @@ -424,10 +451,13 @@ Workflow definitions must satisfy these limits:
65,535 characters.
- A `schedule` trigger may contain at most 20 cron expressions, each at most
256 characters.
- A matrix may define at most 100 axes and expand one logical job into at most
256 jobs. A workflow may expand to at most 1,000 jobs in total.
- Matrix axis names contain at most 256 characters, and scalar matrix values
contain at most 1,024 characters.
- A matrix may define at most 100 axes. Its `include` and `exclude` lists may
each contain at most 256 mappings, with at most 100 values per mapping. The
final transformed matrix must contain 1 to 256 jobs, and a workflow may expand
to at most 1,000 jobs in total.
- Matrix keys contain at most 256 characters, scalar matrix values contain at
most 1,024 characters, and each transformed combination contains at most 100
values.
- A job may contain at most 100 steps and 100,000 bytes of aggregate planned
content.
- A completed job result may contain at most 100 outputs and 4 KiB of encoded
Expand Down Expand Up @@ -470,11 +500,10 @@ than one plan version must emit the result version assigned to that plan, not
always the latest result version supported by the runner binary.

Docker and local actions, private cross-repository action authentication, job
dependencies, matrix `include` and `exclude`, strategy `fail-fast`, service
containers, repository secret and variable sources, caches, and artifacts are
not supported. Expressions outside the documented fields and runtime contexts
are rejected during planning or execution and are never interpreted as literal
values.
dependencies, strategy `fail-fast`, service containers, repository secret and
variable sources, caches, and artifacts are not supported. Expressions outside
the documented fields and runtime contexts are rejected during planning or
execution and are never interpreted as literal values.
`WorkflowJob` resources are not retried or reassigned when a Runner is removed.
Native Jobs and their Pod logs are deleted one hour after completion. Completed
WorkflowRuns are retained indefinitely unless `spec.ttlSecondsAfterFinished` is
Expand Down
6 changes: 3 additions & 3 deletions internal/controller/workflowrun_controller_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -246,7 +246,7 @@ func TestPlanWorkflowJobsExpandsArchitectureMatrix(t *testing.T) {
Revision: actionsv1alpha1.GitRevision{SHA: strings.Repeat("a", 40), Ref: "refs/heads/main"},
},
}}}
definition, err := workflow.Parse([]byte("name: Release\non: push\njobs:\n build-images:\n strategy:\n max-parallel: 1\n matrix:\n arch: [amd64, arm64]\n runs-on: ${{ matrix.arch == 'arm64' && 'ubuntu-24.04-arm' || 'ubuntu-latest' }}\n outputs:\n image: ${{ matrix.arch }}-${{ steps.build.outputs.image }}\n steps:\n - id: build\n run: make image IMAGE_PLATFORMS=linux/${{ matrix.arch }}\n"))
definition, err := workflow.Parse([]byte("name: Release\non: push\njobs:\n build-images:\n strategy:\n max-parallel: 1\n matrix:\n arch: [amd64, arm64]\n include:\n - variant: release\n runs-on: ${{ matrix.arch == 'arm64' && 'ubuntu-24.04-arm' || 'ubuntu-latest' }}\n outputs:\n image: ${{ matrix.arch }}-${{ matrix.variant }}-${{ steps.build.outputs.image }}\n steps:\n - id: build\n run: make image IMAGE_PLATFORMS=linux/${{ matrix.arch }}\n"))
if err != nil {
t.Fatal(err)
}
Expand All @@ -260,7 +260,7 @@ func TestPlanWorkflowJobsExpandsArchitectureMatrix(t *testing.T) {
}
for index, arch := range []string{"amd64", "arm64"} {
job := planned[index]
if job.id != fmt.Sprintf("build-images-matrix-%d", index+1) || job.matrix == nil || job.matrix.LogicalJobID != "build-images" || job.matrix.Values["arch"] != arch || job.matrix.MaxParallel != 1 || job.resultVersion != jobResultVersion {
if job.id != fmt.Sprintf("build-images-matrix-%d", index+1) || job.matrix == nil || job.matrix.LogicalJobID != "build-images" || job.matrix.Values["arch"] != arch || job.matrix.Values["variant"] != "release" || job.matrix.MaxParallel != 1 || job.resultVersion != jobResultVersion {
t.Errorf("planned job %d = %#v", index, job)
}
wantRunner := "ubuntu-latest"
Expand All @@ -274,7 +274,7 @@ func TestPlanWorkflowJobsExpandsArchitectureMatrix(t *testing.T) {
if err := json.Unmarshal([]byte(job.plan), plan); err != nil {
t.Fatal(err)
}
if plan.JobID != "build-images" || plan.Matrix["arch"] != arch || plan.Outputs["image"] != "${{ matrix.arch }}-${{ steps.build.outputs.image }}" {
if plan.JobID != "build-images" || plan.Matrix["arch"] != arch || plan.Matrix["variant"] != "release" || plan.Outputs["image"] != "${{ matrix.arch }}-${{ matrix.variant }}-${{ steps.build.outputs.image }}" {
t.Errorf("plan for %s = %#v", arch, plan)
}
}
Expand Down
5 changes: 3 additions & 2 deletions internal/manifests/charts/open-actions/templates/crds/actions.kelos.dev_workflowjobs.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -102,8 +102,9 @@ spec:
additionalProperties:
type: string
description: |-
Values contains the scalar matrix axis values for this combination.
Values are represented using their workflow string form.
Values contains the scalar matrix values for this combination, including
values added by an include transformation. Values are represented using
their workflow string form.
maxProperties: 100
minProperties: 1
type: object
Expand Down
Loading
Loading