Skip to content

Publish specification timing reports and shard browser specs in CI - #271

Merged
woksin merged 5 commits into
mainfrom
l44-scene262-spec-timings
Oct 25, 2026
Merged

woksin merged 5 commits into
mainfrom
l44-scene262-spec-timings

Conversation

@woksin

@woksin woksin commented Oct 25, 2026

Copy link
Copy Markdown
Contributor

Added

  • CI now publishes JUnit and TRX specification reports as build artifacts, kept for 3 days, and writes a per-project job summary with test counts, durations and the ten slowest files or tests. The summary step fails on a missing, malformed or empty report. (Measure spec timings and share browsers in blueprint specs #262)

Changed

Vitest writes JUnit reports only when SPEC_REPORTS_DIR is set, so local
runs are unchanged. .NET specification results are now uploaded on every
run with a 3-day retention and summarized in the job summary.
Shards are assigned from the discovered browser workspaces and run
against the built artifacts, uploading their JUnit reports for the
summary.
A zero-test JUnit suite or TRX counter, or a non-numeric count, used to
summarize successfully and let a shard report green without evidence.
Counts must now be non-negative integers, durations finite, and every
report must record at least one test.
@woksin woksin self-assigned this Oct 25, 2026
@woksin

woksin commented Oct 25, 2026

Copy link
Copy Markdown
Contributor Author

Reviewer notes (not release notes)

  • Origin: lane L44 built this, and it is shipped by the Scene owner lane. Base: main 13cb55f; head: e4e6786.
  • Whole-change review: round 1 found one blocker. The summary script accepted zero-test reports and non-numeric counts. Commit e4e6786 fixed it with regression tests, and round 2 passed with 0 blocking findings.
  • Local checks:
    • summarize-spec-reports tests: 9/9 passed.
    • browser-spec-shards tests: 6/6 passed.
    • All 17 browser workspaces, run in three shards: 2,306 tests, all passed.
    • verify-packages, verify-blueprints, build-storybook, blueprint scaffold build/lint/test: all passed.
    • Artifact round trip (tar the dist folders, then run the browser specs without rebuilding): passed.
    • Summaries of real browser JUnit reports and Release TRX reports: both succeeded.
  • Not green: root yarn ci failed locally on one test: primereact when_sanitizing_hostile_html exceeded its 5 s timeout (5.9 s). Main's own config fails the same way on the same machine (5.28 s). An independent review agreed this diff cannot slow that path, because it only adds an opt-in JUnit reporter. How often it fails in CI is not verified. Raising that test's time budget is a separate fix.
  • Sharing one Chromium per worker was measured and not kept. primereact saved 1.6 s of wall time, and the default workspace got slower. Blueprint app specs are dominated by a 300 ms settle wait instead. That is left as a possible follow-up; no issue is filed for it.

@woksin
woksin merged commit 4666ceb into main Oct 25, 2026
10 of 11 checks passed
@woksin
woksin deleted the l44-scene262-spec-timings branch October 25, 2026 12:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant