Skip to content

ci: cut the macOS UI suite from 14.5 to 3.4 minutes, skip Rust jobs for prose-only PRs - #189

Merged
DWAA1660 merged 3 commits into
AlienTechSoftware:mainfrom
DWAA1660:ci/speed-up
Nov 12, 2026
Merged

DWAA1660 merged 3 commits into
AlienTechSoftware:mainfrom
DWAA1660:ci/speed-up

Conversation

@DWAA1660

Copy link
Copy Markdown
Collaborator

Why

The macOS build/test job took ~20 min of every CI run, and 14.5 min of that was one test binary, crates/ui-egui/tests/ui.rs (870 s on macOS vs 143 s on Linux and 178 s on Windows).

A CPU sample of the slowest UI test on a macos-15 runner showed the test thread spending 11,190 of 11,342 samples (98.7%) asleep in settle() (ui.rs:121). settle() slept 20 ms twenty-five times to give background work half a second. On the hosted macOS VMs a 20 ms sleep lasts about 100 ms, so each of the ~640 settle() calls took 2.5 s instead of 0.5 s. It wasn't the GPU: with a shared wgpu device the slowest test went from 74 s to 73 s, which ruled that out.

What

  1. settle() waits by the clock (crates/ui-egui/tests/ui.rs). Background work still gets 500 ms of wall-clock time and the window still steps the same 26 frames; only sleeping past the 500 ms is gone. Nothing a test waits for gets less time on any platform, so test coverage is unchanged.
  2. One shared offscreen GPU device per UI test binary (tests/support/gpu.rs), the same pattern icons_visual.rs already uses. Each window keeps its own renderer and textures. Small win (Windows local: 53.5 s → 49.2 s), mainly saves device setup on Linux's software renderer.
  3. Prose-only PRs skip the Rust jobs (ci.yml). A changes job lists the PR's files; if every path is top-level Markdown, SKILLS/, work/, or docs/ other than what tests read (docs/demo/, docs/commands.md, docs/coverage.json), then build/test, clippy and coverage are skipped. fmt and the static checks (links, text, PII) still run, and pushes to main always run everything.
  4. The CI status job is stricter. It fails if the change filter fails or if a required job is cancelled, so a skip can never pass by accident.
  5. macOS and Windows compile with all their cores (3 and 4). CARGO_BUILD_JOBS=2 stays on Linux, where runner pressure was the original reason for it.
  6. docs/testing.md gains a short "Continuous integration" section.

Measurements

before after
macOS UI suite (macos-15, 3 threads) 870.7 s 205.2 s (208 passed)
macOS build/test job (estimate) ~20 min ~8 to 9 min
Windows UI suite, local (20 cores) 53.5 s 46.7 s

The macOS numbers come from probe runs on a fork with the same runner image. This PR's own CI run gives the real end-to-end figure.

Considered and not done

  • Sharding the macOS tests across runners. With the suite at ~3.4 min, macOS is no longer the long pole. Each extra shard costs another macOS runner plus a duplicate compile, and the org-wide macOS concurrency limit is what makes PRs queue now (several recent runs waited 30 to 45 min in total for 20 min of work).
  • More test threads than cores. Some unit tests assert frame timings (for example the_autosave_does_not_hold_up_the_window); oversubscribing could make them flaky.
  • A shorter settle budget. That would trade test robustness for speed, so it stays at 500 ms.

Checks run

  • cargo test -p notecraft-ui-egui --test ui on Windows: 208 passed, 1 ignored. On macos-15: 208 passed, 1 ignored.
  • cargo fmt --all -- --check, cargo clippy -p notecraft-ui-egui --tests -- -D warnings: clean.
  • cargo xtask text (930 files, 0 problems), cargo xtask docs (627 links, 0 broken), cargo xtask layers: pass.
  • The docs-only classifier was exercised on sample path lists (prose → skip; demo notebook, commands.md, coverage.json, crate fixture READMEs, Cargo.lock, workflows → full run).

🤖 Generated with Claude Code

DWAA1660 and others added 3 commits November 12, 2026 15:42
settle() slept 20 ms twenty-five times to give background work half a
second. On the hosted macOS runners a 20 ms sleep lasts about 100 ms, so
each call took 2.5 s and the UI suite spent 98% of its time asleep (a
sampled run of the slowest test showed 11,190 of 11,342 samples in that
sleep). It now gives background work 500 ms measured with a clock and
still steps the same 26 frames, so nothing a test waits for gets less
time on any platform.

macOS UI suite: 870.7 s -> 205.2 s, 208 passed.

The UI tests also share one offscreen wgpu device per test binary, as
icons_visual.rs already does; each window keeps its own renderer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- A changes job lists the PR's files; when every path is prose
  (top-level Markdown, SKILLS/, work/, docs/ apart from the demo
  notebooks, commands.md and coverage.json, which tests read), the
  build/test, clippy and coverage jobs are skipped. fmt and the static
  checks still run. Pushes to main always run everything.
- The CI status job now fails when the filter fails, and treats a
  cancelled required job as a failure, so a skip cannot pass by accident.
- macOS (3 cores) and Windows (4 cores) compile with all their cores;
  Linux stays at two jobs because of its disk and memory history.
- docs/testing.md describes the CI behaviour.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@DWAA1660
DWAA1660 merged commit 399d503 into AlienTechSoftware:main Nov 12, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant