Repository navigation
fix(audit): harvest pnpm/yarn and vitest/jest claims - #190
Conversation
Claim harvesting and claim trimming were two hand-written lists that had drifted apart. The npm family matched inside "pnpm run test" and won as the leftmost match, so the replay ran "npm run test" instead of the command the PR quoted. "yarn run test", "vitest run" and "jest" were not recognised at all, which turned a PR that named its run into a no_claims audit. Both lists now derive from one command-family table, and each tail is anchored on a token edge so a match can only start at the command. vitest and jest count only inside backticks, and their summary lines are parsed so an overstated count is still a mismatch.
326b4df to
6b89452
Compare
|
CI caught one thing worth calling out: the first push went up through the Git I re-created the commit with mode |
|
Merged — thank you. Verified independently before merging: checked out the PR head, module-level repro of all six cases from #189 passes (plus |
Fixes #189.
What this changes
xflow auditno longer keeps two hand-written lists of test commands. Therecogniser (
TEST_COMMAND_PATTERNS, whichisTestCommandconsults) and theharvester (the alternation that cuts the command out of a sentence) are now
derived from one command-family table, where each row carries the shape a claim
may take in prose and the runnable slice to harvest. Because there is one
source, a family cannot be recognised by one list and truncated by the other.
The harvest is anchored on a token edge, so a match can only start where the
command starts:
pnpm run testis no longer cut tonpm run test.vitest and jest joined the table as families that count only inside backticks,
and their summary lines (
Tests 18 passed (18),Tests: 2 failed, 16 passed, 20 total) are parsed so a claimed count is still compared against the replay.The audit help text now lists the commands it recognises.
Why
The two lists had already drifted.
pnpm run testwas replayed asnpm run test— a different package manager, in a workspace whose lockfile npm cannotread — while
yarn run test,vitest runandyarn jestwere dropped on thefloor, so a change that quoted its run audited as
no_claims. The audit thentells the reader the change named no test run at all, which is the opposite of
what the change says.
I did not fix this by adding four more patterns to each list. That is how the
last two regressions of this kind were fixed (the
unittestcase, run20261109-231900-3c6b2a; thebuncase in M6 slice 1, #132), and each fix leftthe two lists one edit away from disagreeing again. I also did not reach for a
separate registry module: the table stays in
src/core/audit.js, next to theharvest that reads it, and the exported
TEST_COMMAND_PATTERNSkeeps its nameand shape so nothing downstream has to change.
Out of scope on purpose:
pnpm -r testand other recursive invocations (aworkspace-wide run is not the same command per package), dotnet and rspec, and
the TS test-file detection in
isJavaScriptTestFile, which is a separate issueabout which files count rather than which commands do.
Tests
npm testequivalent run locally, with the pre-existing failures notednpm run drift:scan— not run: no workflow or atom changedThe four audit suites (
xflow-audit,xflow-audit-go-rust,xflow-audit-batch,xflow-sarif): 126 tests, 92 passed, 19 failed, 15 skipped. The same four filesagainst the unmodified tree give 123 tests, 89 passed, 19 failed, 15 skipped —
the same failure set, test for test. All 19 are environment failures on this
Windows machine (
EPERMwhilermSyncclears a read-only.gitobject under%TEMP%, plus gh- and seal-related cases that need a POSIX checkout); theyreproduce on
mainwith the change stashed.Three new cases, in
test/xflow-audit.test.js:keeps pnpm/yarn script runs whole instead of trimming them to npm— assertsthe harvested commands are
pnpm run testandyarn run test, with thecounts read from the same sentence.
reads vitest and jest claims from backticks, never from prose— prose abouttooling harvests nothing;
`vitest run`and`yarn jest --coverage`do, prefix included.
reads vitest and jest summary lines, and leaves a multi-project run unparsed— both count orders, both total forms, a two-block monorepo output and a
marker without counts.
Behaviour before and after, from the same module:
`pnpm run test`npm run testpnpm run test`yarn run test`yarn run test`vitest run`vitest run`yarn jest`yarn jestWe use vitest and jest for unit tests.And end to end on a two-commit fixture whose
claims.mdnames`yarn run test`:no_claims, exit 3, before; a harvested claim that isreplayed, after.
AI assistance
Yes — written with DeepSeek Harness. I read the full diff before submitting it.