Repository navigation
test: drop platform-dependent exit status assertion in shell test - #15
Merged
Merged
Conversation
The `expect(result.status).toBe(0)` assertion tested the host OS rather than runShell: whether an overflowing maxBuffer leaves the child at status 0 or kills it varies by platform and by coverage instrumentation. CI runs the tests collector under `vitest run --coverage` on ubuntu, where the status differs, so the test was red there while passing on macOS. The stderr assertion is retained: it is the behaviour the test protects (the spawn error surfaces in stderr, signalling that stdout is incomplete). The misleading comment claiming the status stays 0 is corrected. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
✅ Quality gate passedcommit
|
| Check | Baseline | Now | Verdict | |
|---|---|---|---|---|
| TypeScript errors | — | n/a | collector not configured — see docs/loopwright/quality-gate.md | |
| High advisories | 0 | 2 | regressed 2 (tolerance 0) |
High advisories
brace-expansion— high: brace-expansion: Quadratic-time expansion of the{a},b}rewrite causes CPU denial of service · fix available — upgrade itjs-yaml— high: js-yaml: maxTotalMergeKeys does not limit CPU use for empty merge sources · fix available — upgrade it
📊 All metrics
| Check | Baseline | Now | Verdict | |
|---|---|---|---|---|
| ✅ | Failing tests | 0 | 0 | holding |
| ✅ | Failing test suites | 0 | 0 | holding |
| TypeScript errors | — | n/a | collector not configured — see docs/loopwright/quality-gate.md | |
| ✅ | Lint errors | 0 | 0 | holding |
| ✅ | Lint warnings | 0 | 0 | holding |
| ✅ | Critical advisories | 0 | 0 | holding |
| High advisories | 0 | 2 | regressed 2 (tolerance 0) | |
| ✅ | Suppressed advisories | 0 | 0 | holding |
| 📈 | Line coverage | 84.67% | 85.01% | improved 0.34% |
| 📈 | Branch coverage | 78.93% | 79.44% | improved 0.51% |
| 📈 | Function coverage | 89.65% | 89.71% | improved 0.06% |
| 📈 | Statement coverage | 85.01% | 85.41% | improved 0.40% |
| 📈 | Duplicated code | 0.90% | 0.88% | improved 0.02% |
| ✅ | Highest function complexity | 14 | 14 | holding |
| 📈 | Average function complexity | 1.81 | 1.80 | improved 0.01 |
| ✅ | Oversized files | 0 | 0 | holding |
| ✅ | Skipped tests | 0 | 0 | holding |
| ✅ | Focused tests (.only) | 0 | 0 | holding |
| ✅ | Tests with no assertion | 0 | 0 | holding |
| ✅ | Coverage-ignore hints | 1 | 1 | holding |
| ✅ | Type suppressions (@ts-ignore etc.) | 2 | 2 | holding |
| ✅ | Inline lint suppressions | 2 | 2 | holding |
| ✅ | Empty catch blocks | 0 | 0 | holding |
📈 This PR improves 6 metric(s). Run
node .loopwright/scripts/quality-gate.mjs --update-baselineand commit.loopwright/baseline.jsonto lock the gain in.
Generated by .loopwright/scripts/quality-gate.mjs · reproduce locally with node .loopwright/scripts/run-report.mjs --all && node .loopwright/scripts/quality-gate.mjs · full reports are in the workflow artifacts.
SamuelDenani
marked this pull request as ready for review
October 7, 2026 20:41
Code reviewNo issues found. Checked for bugs and CLAUDE.md compliance. |
SamuelDenani
added a commit
that referenced
this pull request
Oct 7, 2026
Brings in the platform-dependent test fix from #15 so this PR's gate can run against a green base. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
main's quality gate has been red since 2026-08-31 (the merge of #4), and noPR in this repo can go green until this lands. It currently blocks #11, #12,
#13 and anything opened after them.
The cause is one assertion in
.loopwright/tests/shell.test.mjsthat tests thehost OS rather than the code under test:
Whether an overflowing
maxBufferleaves the child's exit status at 0 or killsthe process varies by platform and by whether coverage instrumentation is
active. The assertion holds on macOS and fails on
ubuntu-latestunder--coverage.Why the CI log is misleading about this
The workflow runs vitest twice, and only the second run feeds the gate:
Engine unit testsnpx vitest run152 passed✅Collect reportsvitest run --coverage(fromconfig.json)tests: exit 1, 1 failing assertion❌So a run shows
Test Files 14 passednear the top and a red gate at the bottom,which reads like a gate bug rather than a test failure. The coverage run is the
one
collect-metrics.mjsreads, and when it fails,coverage-summary.jsonisnever written — so the
testscollector is recorded as failed and the fourcoverage metrics block in cascade. Six blockers, one cause. That cascade is the
gate working correctly: an infrastructure failure blocks, and is not the same as
an unconfigured tool.
What changed
The status assertion is removed and the comment above it rewritten — it
previously stated the false premise ("Output overflowing maxBuffer still exits
0"). The assertion the test exists for, that the spawn error surfaces in
stderrinstead of being dropped, is retained unchanged.This is not weakening a test to make a build green.
CLAUDE.mdforbidsskipping, focusing, deleting or weakening a failing test, and none of those
happened: no
.skip, no.only, notry/catch, noprocess.platformconditional, and no change to
.loopwright/scripts/lib/shell.mjs.runShellpropagating the error into
stderris correct and is still asserted. What wasremoved asserted a property of the operating system.
Verification
npx vitest run tests/shell.test.mjsnpx vitest run tests/shell.test.mjs --coveragenpx vitest runrun-report.mjs --all && quality-gate.mjsLocal runs cannot reproduce the Linux failure, so this PR's own CI run is the
real proof.
Left alone deliberately
Two warnings remain and are not in scope here:
typecheckcollector unconfigured — recorded as such in the baseline.audit.highat 2 —brace-expansionandjs-yaml, both with a fixavailable upstream. No
audit.ignoreentry was added;CLAUDE.mdforbidsignoring an advisory that has a fix, so this wants a dependency upgrade in its
own PR.
.loopwright/baseline.jsonand.loopwright/config.jsonare untouched.🤖 Generated with Claude Code