Skip to content

ci: adopt the unified dotnet workflow #118

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
matt-edmondson wants to merge 1 commit into from
Closed

ci: adopt the unified dotnet workflow #118

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
303 changes: 248 additions & 55 deletions .github/workflows/dotnet.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ on:
paths-ignore:
["**.md", ".github/ISSUE_TEMPLATE/**", ".github/pull_request_template.md"]
pull_request:
paths-ignore:
["**.md", ".github/ISSUE_TEMPLATE/**", ".github/pull_request_template.md"]
schedule:
- cron: "0 23 * * *" # Daily at 11 PM UTC
workflow_dispatch: # Allow manual triggers
Expand Down Expand Up @@ -33,10 +35,198 @@ env:
DOTNET_VERSION: "10.0" # Only needed for actions/setup-dotnet

jobs:
build:
name: Build, Test & Release
discover:
name: Discover Test Projects
runs-on: ubuntu-latest
timeout-minutes: 10

outputs:
matrix: ${{ steps.discover.outputs.matrix }}
platforms: ${{ steps.discover.outputs.platforms }}
has_tests: ${{ steps.discover.outputs.has_tests }}

steps:
- name: Checkout Repository
uses: actions/checkout@v7

- name: Setup .NET SDK ${{ env.DOTNET_VERSION }}
uses: actions/setup-dotnet@v6
with:
dotnet-version: ${{ env.DOTNET_VERSION }}.x

- name: Install KtsuBuild
shell: bash
run: |
dotnet tool install ktsu.KtsuBuild.Tool --tool-path "${{ runner.temp }}/ktsubuild"
echo "${{ runner.temp }}/ktsubuild" >> "$GITHUB_PATH"

# `test list` reports every test project regardless of the host it runs on, unlike the
# filter `build` and `ci` apply, so one Linux job can enumerate cells that Windows and
# macOS runners will execute. It writes failures to stdout rather than stderr, so the
# exit code is the only reliable signal and has to be checked before parsing.
- name: Discover Test Projects
id: discover
shell: bash
run: |
set -euo pipefail

if ! projects=$(ktsubuild test list --workspace "$GITHUB_WORKSPACE"); then
echo "::error::ktsubuild test list failed:"
echo "$projects"
exit 1
fi

echo "Discovered test projects:"
echo "$projects" | jq .

# An unrecognized platform must stop the run rather than drop the project. Dropping
# it would produce a smaller matrix that still reports success, which is the failure
# this design exists to remove.
unknown=$(echo "$projects" | jq -r '[.[] | select(.platform as $p | ["neutral","windows"] | index($p) | not) | .platform] | unique | join(", ")')
if [ -n "$unknown" ]; then
echo "::error::Cannot place test project(s) on a runner. Unhandled platform(s): $unknown"
echo "::error::macOS is currently excluded from the matrix, so an ios-tied test project has nowhere to run."
exit 1
fi

# macOS is deliberately absent from this mapping. A macOS runner builds any project
# whose target frameworks are widened on that host, and in a repo with an iOS head that
# pulls in a target framework needing a workload this job does not install, so every
# macOS cell fails during its build. Restoring the workload on each cell is slow and
# macOS runner minutes are billed at a premium, so the platform is excluded until the
# underlying problem is fixed rather than papered over. An ios-tied test project now
# fails the guard above instead of silently finding no runner.
matrix=$(echo "$projects" | jq -c '
{
include: [
.[]
| . as $p
| {
neutral: ["ubuntu-latest", "windows-latest"],
windows: ["windows-latest"]
}[$p.platform][]
| {
os: .,
project: $p.project,
name: ($p.project | split("/") | last | rtrimstr(".csproj")),
slug: ($p.project | rtrimstr(".csproj") | gsub("[^A-Za-z0-9]"; "-"))
}
]
}')

count=$(echo "$matrix" | jq '.include | length')
echo "Matrix has $count cell(s)."
echo "$matrix" | jq .

# The distinct hosts the cells land on. One test job runs per platform and builds once,
# so this is what that job fans out over, while `matrix` tells each job which projects
# are its own.
platforms=$(echo "$matrix" | jq -c '[.include[].os] | unique')
echo "Platforms: $platforms"

echo "matrix=$matrix" >> "$GITHUB_OUTPUT"
echo "platforms=$platforms" >> "$GITHUB_OUTPUT"
if [ "$count" -gt 0 ]; then
echo "has_tests=true" >> "$GITHUB_OUTPUT"
else
echo "has_tests=false" >> "$GITHUB_OUTPUT"
fi

test:
name: Test on ${{ matrix.os }}
needs: discover
if: needs.discover.outputs.has_tests == 'true'
runs-on: ${{ matrix.os }}
timeout-minutes: 45

strategy:
# One platform's failure must not cancel the others. Knowing that a project fails on one
# host only is the point of testing on more than one.
fail-fast: false
matrix:
os: ${{ fromJson(needs.discover.outputs.platforms) }}

steps:
- name: Checkout Repository
uses: actions/checkout@v7
with:
lfs: true
submodules: recursive

- name: Setup .NET SDK ${{ env.DOTNET_VERSION }}
uses: actions/setup-dotnet@v6
with:
dotnet-version: ${{ env.DOTNET_VERSION }}.x
cache: true
cache-dependency-path: |
**/*.csproj
**/Directory.Packages.props
**/global.json

- name: Install KtsuBuild
shell: bash
run: |
dotnet tool install ktsu.KtsuBuild.Tool --tool-path "${{ runner.temp }}/ktsubuild"
echo "${{ runner.temp }}/ktsubuild" >> "$GITHUB_PATH"

# `test all` restores, builds, and tests every test project this host can build, pinned to
# the host's runtime identifier. The pin is what makes this cheap: without it a project's
# output carries native assets for every runtime its packages ship, sixteen of them here,
# and copying that dominates the job on Windows where file writes are several times slower
# than on Linux. Measured at 115 MB against 39 MB for the smallest test project.
#
# Deliberately not `ci --no-release`: `ci` commits and pushes the metadata files when the
# build is official and on main, so with one job per platform both jobs would race to
# commit on every push to main. `test all` does no metadata, version, or release work.
#
# A project the host cannot build is skipped and named before anything is built, and a
# project that fails does not stop the ones after it, so one run reports everything broken.
#
# The five UI suites are effectively the whole cost of this job. They run in parallel, so the
# slowest of them sets the job's duration, while all nine other test projects finish in about
# thirty-four seconds combined. They are therefore run on Linux only:
#
# * measured per assembly on Windows: 17m28s, 14m32s, 10m05s, 8m20s, 1m01s against ~34s for
# everything else, in a job whose test phase took 17m45s.
# * what they exercise is a pure managed CPU rasterizer with no window, GPU or driver. Timed
# on one machine, self-contained for each runtime, it renders the same workload in 482.6ms
# on Windows against 476.0ms on Linux, so running it on one platform covers the same ground.
# * Linux is the faster host for this work, so it keeps them.
#
# Only the UI test projects are excluded. The example applications they drive stay in the
# build on both platforms, so a change that breaks one still fails here.
- name: Test
shell: bash
run: |
set -euo pipefail
if [ "${{ runner.os }}" = "Windows" ]; then
ktsubuild test all --workspace "$GITHUB_WORKSPACE" --verbose --exclude "**/*.UITests/*"
else
ktsubuild test all --workspace "$GITHUB_WORKSPACE" --verbose
fi

- name: Upload Coverage
uses: actions/upload-artifact@v7
if: always()
with:
name: coverage-${{ matrix.os }}
path: ./coverage/*
retention-days: 7
if-no-files-found: warn

release:
name: Analyze & Release
needs: [discover, test]
# `!cancelled()` is required because `test` is skipped when a repo has no test projects, and
# a skipped dependency would otherwise skip this job too. It also stops a run that
# `concurrency.cancel-in-progress` superseded from reaching `Release` and racing the newer
# run. The explicit result checks are what keep a genuine test failure from releasing anyway.
if: |
!cancelled()
&& needs.discover.result == 'success'
&& (needs.test.result == 'success' || needs.test.result == 'skipped')
runs-on: windows-latest
timeout-minutes: 20
timeout-minutes: 30
permissions:
contents: write # For creating releases and committing metadata
packages: write # For publishing packages
Expand Down Expand Up @@ -114,15 +304,48 @@ jobs:
if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
"${{ runner.temp }}/ktsubuild" >> $env:GITHUB_PATH

# Every cell wrote a file named coverage.xml, so the downloads must stay in their own
# per-artifact directories. Merging them would leave one file and report the coverage of
# a single test project as though it were the whole repository's.
- name: Download Coverage
if: needs.discover.outputs.has_tests == 'true'
uses: actions/download-artifact@v7
with:
pattern: coverage-*
path: coverage

# SonarCloud's "previous version" new-code period needs recorded version boundaries to
# anchor to. Without /v: the scanner reports the version as "not provided", so the period
# has nothing to anchor against and widens to the whole history, which makes the new-code
# coverage condition measure the entire codebase instead of what this change touched.
# `version bump` prints the computed version as its only bare semver line.
- name: Resolve Version for Analysis
id: analysis_version
shell: pwsh
run: |
$output = & ktsubuild version bump --workspace "${{ github.workspace }}" 2>&1
if ($LASTEXITCODE -ne 0) { $output; exit $LASTEXITCODE }
$matches = @($output | Where-Object { $_ -match '^\d+\.\d+\.\d+' })
if ($matches.Count -ne 1) {
$output
Write-Error "Expected exactly one bare version line from 'version bump', got $($matches.Count)."
exit 1
}
"version=$($matches[0].Trim())" >> $env:GITHUB_OUTPUT

- name: Begin SonarQube
if: ${{ env.SONAR_TOKEN != '' }}
env:
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
shell: pwsh
run: |
.\.sonar\scanner\dotnet-sonarscanner begin /k:"${{ github.repository_owner }}_${{ github.event.repository.name }}" /o:"${{ github.repository_owner }}" /d:sonar.token="$env:SONAR_TOKEN" /d:sonar.host.url="https://sonarcloud.io" /d:sonar.projectBaseDir="${{ github.workspace }}" /d:sonar.cs.vscoveragexml.reportsPaths="coverage/coverage.xml" /d:sonar.coverage.exclusions="**/*Test*.cs,**/*.Tests.cs,**/*.Tests/**/*,**/obj/**/*,**/*.dll,**/NativeExports.cs" /d:sonar.cs.vstest.reportsPaths="coverage/TestResults/**/*.trx" /d:sonar.exclusions="**/NativeExports.cs"
.\.sonar\scanner\dotnet-sonarscanner begin /k:"${{ github.repository_owner }}_${{ github.event.repository.name }}" /o:"${{ github.repository_owner }}" /v:"${{ steps.analysis_version.outputs.version }}" /d:sonar.token="$env:SONAR_TOKEN" /d:sonar.host.url="https://sonarcloud.io" /d:sonar.projectBaseDir="${{ github.workspace }}" /d:sonar.qualitygate.wait=true /d:sonar.cs.vscoveragexml.reportsPaths="coverage/**/coverage.xml" /d:sonar.coverage.exclusions="**/*Test*.cs,**/*.Tests.cs,**/*.Tests/**/*,**/obj/**/*,**/*.dll,**/NativeExports.cs" /d:sonar.cs.vstest.reportsPaths="coverage/**/*.trx" /d:sonar.exclusions="**/NativeExports.cs"

- name: Run KtsuBuild CI Pipeline
# `ci` rather than restore and build directly, because it is the only place that updates
# and commits the metadata files, updates the repository topics, applies the version gate
# behind `[skip ci]`, and writes the step outputs the security job reads.
# The tests already ran in the matrix, and the release waits for the quality gate below.
- name: Run KtsuBuild Pipeline
id: pipeline
shell: pwsh
env:
Expand All @@ -131,11 +354,9 @@ jobs:
KTSU_PACKAGE_KEY: ${{ secrets.KTSU_PACKAGE_KEY }}
EXPECTED_OWNER: ktsu-dev
run: |
# Run the CI pipeline
$versionBump = "${{ github.event.inputs.version-bump }}"

# Build arguments array - only add --version-bump if explicitly set (for backward compatibility during bootstrap)
$args = @("ci", "--workspace", "${{ github.workspace }}", "--verbose")
$args = @("ci", "--workspace", "${{ github.workspace }}", "--no-test", "--no-release", "--verbose")
if (![string]::IsNullOrEmpty($versionBump) -and $versionBump -ne "auto") {
$args += @("--version-bump", $versionBump)
}
Expand All @@ -151,63 +372,35 @@ jobs:
run: |
.\.sonar\scanner\dotnet-sonarscanner end /d:sonar.token="$env:SONAR_TOKEN"

# Gated by the step above. `sonar.qualitygate.wait=true` makes a failed gate fail that
# step, and a step whose `if:` names no status function is implicitly gated on success, so
# a release cannot proceed past a gate the project did not pass.
- name: Release
if: steps.pipeline.outputs.should_release == 'true'
shell: pwsh
env:
GH_TOKEN: ${{ github.token }}
NUGET_API_KEY: ${{ secrets.NUGET_KEY }}
KTSU_PACKAGE_KEY: ${{ secrets.KTSU_PACKAGE_KEY }}
EXPECTED_OWNER: ktsu-dev
run: |
ktsubuild release --workspace "${{ github.workspace }}" --verbose
if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }

- name: Upload Coverage Report
uses: actions/upload-artifact@v7
if: always()
with:
name: coverage-report
name: analysis-coverage-report
path: |
./coverage/*
retention-days: 7
if-no-files-found: ignore

winget:
name: Update Winget Manifests
needs: build
if: needs.build.outputs.should_release == 'true'
runs-on: windows-latest
timeout-minutes: 10
permissions:
contents: write

steps:
- name: Checkout Release Commit
uses: actions/checkout@v7
with:
ref: ${{ needs.build.outputs.release_hash }}
fetch-depth: 0 # Full history for better auto-detection

- name: Setup .NET SDK ${{ env.DOTNET_VERSION }}
uses: actions/setup-dotnet@v6
with:
dotnet-version: ${{ env.DOTNET_VERSION }}.x

# PATH is not shared between jobs, so this job installs the tool for itself.
- name: Install KtsuBuild
shell: pwsh
run: |
dotnet tool install ktsu.KtsuBuild.Tool --tool-path "${{ runner.temp }}/ktsubuild"
if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
"${{ runner.temp }}/ktsubuild" >> $env:GITHUB_PATH

- name: Update Winget Manifests
shell: pwsh
env:
GH_TOKEN: ${{ github.token }}
run: |
ktsubuild winget generate --version "${{ needs.build.outputs.version }}" --workspace "${{ github.workspace }}" --verbose

- name: Upload Updated Manifests
uses: actions/upload-artifact@v7
with:
name: winget-manifests-${{ needs.build.outputs.version }}
path: winget/*.yaml
retention-days: 30

security:
name: Security Scanning
needs: build
if: needs.build.outputs.should_release == 'true'
needs: release
if: needs.release.outputs.should_release == 'true'
runs-on: windows-latest
timeout-minutes: 10
permissions:
Expand All @@ -218,6 +411,6 @@ jobs:
- name: Checkout Release Commit
uses: actions/checkout@v7
with:
ref: ${{ needs.build.outputs.release_hash }}
ref: ${{ needs.release.outputs.release_hash }}
- name: Detect Dependencies
uses: advanced-security/component-detection-dependency-submission-action@31f25a8de68ae5ce2ca274bc28546a78683c15ce # v0.1.4