Repository navigation
Conversation
The report only counted the failed jobs without a test annotation. In one month, these were about half of the failed develop runs: failed setup steps, and jobs that reached their time limit, which GitHub reports as cancelled. The report now lists them by job and step. The gate job is excluded, because it only repeats the failure of another job.
Contributor
size-limit report 📦Compared: 18fd907 (PR test merge using develop at f9f479a) Against: 9b7265f (develop) · base run #66464
|
s1gr1d
marked this pull request as ready for review
November 17, 2026 12:51
javascript-sdk-gitflow
Bot
requested review from
RulaKhaled,
msonnb and
timfish
November 17, 2026 12:51
timfish
approved these changes
Nov 17, 2026
nicohrubec
approved these changes
Nov 17, 2026
The tests that stay cover the logic that is easy to break: a failed job without a test annotation is listed with its failed step, a job that reached its time limit is listed as timed out, and a job that was cancelled for another reason is not listed. The tests for the gate job, the E2E job grouping, the fallback step name, and the rendered table go away.
Member
Author
|
I reduced the number of tests and only kept the most important |
Contributor
Optional E2E testsAll optional checks were successful. 2 successful. See the run. |
51 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The weekly flaky test report has a second table. It lists the jobs that failed without a test failure, by job and step, with the result "failed" or "timed out":
Why
The report only printed a count for these jobs, for example "155 failed job attempts; 129 without test annotations". In the last month, they were about half of the failed
developruns: hung Playwright installs, lockfile checks, job timeouts. GitHub reports a job that reachedtimeout-minutesas cancelled, so the report did not see those jobs at all.The gate job is excluded, because it fails when any other job fails. No workflow runs
pnpm test:scripts, so the new test file only runs locally.Third PR of a stack of four.