Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion .githooks/pre-commit
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,15 @@ fail() { echo "pre-commit: $1" >&2; exit 1; }
# merging main into a branch trips the hook on files nobody edited. Both
# exclude forms are needed: a git pathspec's leading `**/` requires at least
# one directory component, so it alone would miss a root-level `fixtures/`.
added=$(git diff --cached -U0 -- '*.ts' '*.tsx' '*.js' '*.jsx' '*.mjs' ':(exclude)fixtures/**' ':(exclude)**/fixtures/**' | grep '^+[^+]' || true)
#
# .loopwright/ is excluded wholesale. In a host repo it is vendored code
# nobody hand-edits, so policing it only ever fires on a re-vendor. In this
# repo it is the engine, whose detector definitions necessarily spell out
# every pattern they look for — grepping for those literals in the file that
# declares them is a false positive by construction. Either way the gate
# still scans it in CI, and the gate is the authority; this hook only exists
# to deliver the same answer sooner.
added=$(git diff --cached -U0 -- '*.ts' '*.tsx' '*.js' '*.jsx' '*.mjs' ':(exclude)fixtures/**' ':(exclude)**/fixtures/**' ':(exclude).loopwright/**' | grep '^+[^+]' || true)
if [ -n "$added" ]; then
declare -a patterns=(
'\.only *\(' 'a focused test (.only)'
Expand Down
20 changes: 18 additions & 2 deletions .github/workflows/quality-gate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,12 @@ jobs:
with:
fetch-depth: 0

# corepack activates the pnpm/yarn version pinned by the host's
# `packageManager` field. It is a no-op on an npm host, so it is
# unconditional rather than guarded on a lockfile.
- name: Enable corepack
run: corepack enable

# cache: npm with a two-path cache-dependency-path stays valid even on a
# host with no package-lock.json (e.g. a yarn/pnpm project): the second
# path, .loopwright/package-lock.json, is always vendored and always
Expand All @@ -43,12 +49,22 @@ jobs:
package-lock.json
.loopwright/package-lock.json

# The collectors run the host's own binaries out of node_modules/.bin, so
# this step is what makes them resolve at all. Skipping it does not fail
# here — it fails later, as every collector reporting 'n/a', which reads
# like a broken gate rather than a missing install. Each branch uses the
# lockfile-respecting install for its package manager so CI matches the
# committed tree.
- name: Install host dependencies
run: |
if [ -f package-lock.json ]; then
if [ -f pnpm-lock.yaml ]; then
pnpm install --frozen-lockfile
elif [ -f yarn.lock ]; then
yarn install --immutable || yarn install --frozen-lockfile
elif [ -f package-lock.json ]; then
npm ci
else
echo "no package-lock.json — skipping host npm ci; install host deps in a preceding step if needed"
echo "no lockfile — skipping host install; install host deps in a preceding step if needed"
fi

- name: Install engine dependencies
Expand Down
6 changes: 3 additions & 3 deletions .loopwright/config.default.json
Original file line number Diff line number Diff line change
Expand Up @@ -41,14 +41,14 @@
"onRegression": "block"
},
"lint.errors": {
"label": "ESLint errors",
"label": "Lint errors",
"direction": "lower-better",
"hardMax": 0,
"tolerance": 0,
"onRegression": "block"
},
"lint.warnings": {
"label": "ESLint warnings",
"label": "Lint warnings",
"direction": "lower-better",
"tolerance": 0,
"onRegression": "block"
Expand Down Expand Up @@ -166,7 +166,7 @@
"onRegression": "block"
},
"integrity.lintSuppressions": {
"label": "Inline eslint-disable",
"label": "Inline lint suppressions",
"direction": "lower-better",
"tolerance": 0,
"onRegression": "warn"
Expand Down
8 changes: 4 additions & 4 deletions .loopwright/config.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
"collectors": {
"typecheck": { "adapter": "unconfigured" },
"lint": { "adapter": "eslint" },
"tests": { "adapter": "vitest", "command": "npx vitest run --coverage", "cwd": ".loopwright" },
"tests": { "adapter": "vitest", "command": "vitest run --coverage", "cwd": ".loopwright" },
"audit": { "adapter": "npm-audit" },
"duplication": { "adapter": "jscpd" }
},
Expand Down Expand Up @@ -47,14 +47,14 @@
"onRegression": "block"
},
"lint.errors": {
"label": "ESLint errors",
"label": "Lint errors",
"direction": "lower-better",
"hardMax": 0,
"tolerance": 0,
"onRegression": "block"
},
"lint.warnings": {
"label": "ESLint warnings",
"label": "Lint warnings",
"direction": "lower-better",
"tolerance": 0,
"onRegression": "block"
Expand Down Expand Up @@ -172,7 +172,7 @@
"onRegression": "block"
},
"integrity.lintSuppressions": {
"label": "Inline eslint-disable",
"label": "Inline lint suppressions",
"direction": "lower-better",
"tolerance": 0,
"onRegression": "warn"
Expand Down
2 changes: 1 addition & 1 deletion .loopwright/scripts/adapters/biome.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ function writeLintReport(reportsDir, payload) {
export default {
name: 'biome',
collector: 'lint',
defaultCommand: 'npx biome check --reporter=json .',
defaultCommand: 'biome check --reporter=json .',
collect(ctx) {
const { command, cwd, reportsDir, hostRoot } = ctx;
const result = runShell(command, cwd);
Expand Down
2 changes: 1 addition & 1 deletion .loopwright/scripts/adapters/eslint.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ export default {
// --ignore-pattern excludes the vendored .loopwright/ layer: it ships its
// own fixtures and reports that are not the host's code, so ESLint must
// never traverse into it when run from the host's config.
defaultCommand: "npx eslint . --format json --ignore-pattern '.loopwright/**'",
defaultCommand: "eslint . --format json --ignore-pattern '.loopwright/**'",
collect(ctx) {
const { command, cwd, reportsDir, hostRoot } = ctx;
const result = runShell(command, cwd);
Expand Down
2 changes: 1 addition & 1 deletion .loopwright/scripts/adapters/jest.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ export default {
name: 'jest',
collector: 'tests',
defaultCommand:
'npx jest --ci --json --outputFile=.loopwright/reports/test-results.json --coverage --coverageReporters=json-summary --coverageDirectory=.loopwright/reports/coverage',
'jest --ci --json --outputFile=.loopwright/reports/test-results.json --coverage --coverageReporters=json-summary --coverageDirectory=.loopwright/reports/coverage',
collect(ctx) {
const { command, cwd, reportsDir, hostRoot } = ctx;
const result = runShell(command, cwd);
Expand Down
2 changes: 1 addition & 1 deletion .loopwright/scripts/adapters/tsc.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ function writeTypecheckReport(reportsDir, payload) {
export default {
name: 'tsc',
collector: 'typecheck',
defaultCommand: 'npx tsc --noEmit --pretty false',
defaultCommand: 'tsc --noEmit --pretty false',
collect(ctx) {
const { command, cwd, reportsDir } = ctx;
const result = runShell(command, cwd);
Expand Down
2 changes: 1 addition & 1 deletion .loopwright/scripts/adapters/vitest.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ export default {
name: 'vitest',
collector: 'tests',
defaultCommand:
'npx vitest run --coverage --coverage.reporter=json-summary --coverage.reportsDirectory=.loopwright/reports/coverage --reporter=default --reporter=json --outputFile.json=.loopwright/reports/test-results.json',
'vitest run --coverage --coverage.reporter=json-summary --coverage.reportsDirectory=.loopwright/reports/coverage --reporter=default --reporter=json --outputFile.json=.loopwright/reports/test-results.json',
collect(ctx) {
const { command, cwd, reportsDir, hostRoot } = ctx;
const result = runShell(command, cwd);
Expand Down
7 changes: 5 additions & 2 deletions .loopwright/scripts/lib/analyze-source.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,10 @@ const SUITE_CALLEES = new Set(['describe', 'xdescribe', 'fdescribe', 'suite']);

const COVERAGE_IGNORE = /\b(?:istanbul|v8|c8|node:coverage)\s+ignore\b/;
const TS_SUPPRESSION = /@ts-(?:ignore|nocheck|expect-error)\b/;
const ESLINT_SUPPRESSION = /eslint-disable(?:-next-line|-line)?\b/;
// Both linters loopwright supports, in one rule: an integrity metric that
// only knows eslint reports a clean zero on a biome repo, which reads as
// 'nobody suppressed anything' rather than 'this check cannot see anything'.
const LINT_SUPPRESSION = /eslint-disable(?:-next-line|-line)?\b|biome-ignore(?:-start|-end)?\b/;
const ASSERTION = /\b(?:expect|expectTypeOf|assert|should)\s*[.(]/;

function isFunctionLike(node) {
Expand Down Expand Up @@ -157,7 +160,7 @@ function scanComments(text, filePath, findings) {
const location = { file: filePath, line: index + 1, snippet: line.trim().slice(0, 120) };
if (COVERAGE_IGNORE.test(line)) findings.coverageIgnores.push(location);
if (TS_SUPPRESSION.test(line)) findings.typeSuppressions.push(location);
if (ESLINT_SUPPRESSION.test(line)) findings.lintSuppressions.push(location);
if (LINT_SUPPRESSION.test(line)) findings.lintSuppressions.push(location);
});
}

Expand Down
27 changes: 25 additions & 2 deletions .loopwright/scripts/lib/shell.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,8 @@
*/
import { spawnSync } from 'node:child_process';
import { mkdirSync, writeFileSync } from 'node:fs';
import { dirname, resolve } from 'node:path';
import { delimiter, dirname, resolve } from 'node:path';
import { HOST_ROOT } from './paths.mjs';

export const REPORT_FILES = {
typecheck: ['typecheck.json'],
Expand All @@ -23,8 +24,30 @@ export const REPORT_FILES = {
duplication: ['jscpd/jscpd-report.json'],
};

/**
* Adapter commands name a bare binary (`tsc`, `vitest`, …) and rely on this
* PATH, rather than going through `npx`. npx falls through to the registry
* when a binary is not installed locally, so on a repo whose dependencies were
* never installed — or installed by a package manager the workflow did not
* recognise — `npx tsc` silently downloads `tsc`, an unrelated abandoned
* package that is not TypeScript, and the collector reports its nonsense as
* fact. Every package manager (npm, pnpm, yarn) populates node_modules/.bin,
* so resolving through it works for all three, and a genuinely missing tool
* fails with 'command not found' — which each adapter's TOOL_MISSING check
* already reports honestly.
*
* Both the host root and `cwd` contribute a bin directory: a collector may set
* `cwd` to a subdirectory with its own install (loopwright's own config points
* the tests collector at .loopwright/).
*/
function binPath(cwd) {
const dirs = [resolve(cwd, 'node_modules/.bin'), resolve(HOST_ROOT, 'node_modules/.bin')];
return [...new Set(dirs), process.env.PATH ?? ''].join(delimiter);
}

export function runShell(command, cwd, { maxBuffer = 64 * 1024 * 1024 } = {}) {
const result = spawnSync(command, { shell: true, cwd, encoding: 'utf8', maxBuffer });
const env = { ...process.env, PATH: binPath(cwd) };
const result = spawnSync(command, { shell: true, cwd, encoding: 'utf8', maxBuffer, env });
// spawnSync reports a failure to run the command at all — the shell missing,
// or output overflowing maxBuffer — on `error` rather than through the exit
// status. Dropping it would leave the caller with a bare status 1 and empty
Expand Down
33 changes: 32 additions & 1 deletion .loopwright/tests/analyze-source-extra.test.mjs
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import { describe, it, expect } from 'vitest';
import { mkdtempSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { dirname, join } from 'node:path';
import { analyzeSources } from '../scripts/lib/analyze-source.mjs';

function writeFixture(content) {
Expand Down Expand Up @@ -147,3 +147,34 @@ describe('empty catch detection', () => {
expect(findings.emptyCatches).toHaveLength(1);
});
});

// The bait lives in tests/fixtures/, which `sources.ignore` and the hook both
// exclude: sources.roots covers this directory, so a suppression written out
// in a test would be counted as a real one by the scanner under test.
function suppressionsIn(name) {
const file = join(import.meta.dirname, 'fixtures', 'suppressions', name);
return analyzeSources([file], dirname(file)).findings.lintSuppressions;
}

describe('lint suppressions — both supported linters', () => {
it('counts an eslint suppression in each of its comment forms', () => {
expect(suppressionsIn('eslint-forms.mjs')).toHaveLength(3);
});

// Regression: this matched only one of the two linters loopwright supports,
// so a biome repo reported zero suppressions however many it carried — a
// check that cannot see anything reads exactly like nothing to see.
it('counts a biome suppression in each of its comment forms', () => {
expect(suppressionsIn('biome-forms.mjs')).toHaveLength(3);
});

it('reports the file and line of each suppression so the gate can cite it', () => {
const found = suppressionsIn('biome-forms.mjs');
expect(found.map((entry) => entry.line)).toEqual([1, 3, 5]);
expect(found[0].file).toBe('biome-forms.mjs');
});

it('leaves a file with no suppressions at zero', () => {
expect(suppressionsIn('clean.mjs')).toHaveLength(0);
});
});
5 changes: 5 additions & 0 deletions .loopwright/tests/fixtures/suppressions/biome-forms.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
// biome-ignore lint/suspicious/noExplicitAny: fixture
export const a = 1;
// biome-ignore-start lint/style/useConst: fixture
export const b = 2;
// biome-ignore-end lint/style/useConst: fixture
1 change: 1 addition & 0 deletions .loopwright/tests/fixtures/suppressions/clean.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
export const a = 1; // an ordinary comment
4 changes: 4 additions & 0 deletions .loopwright/tests/fixtures/suppressions/eslint-forms.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
/* eslint-disable no-console */
// eslint-disable-next-line no-alert
const a = 1; // eslint-disable-line no-unused-vars
export const b = a;
26 changes: 25 additions & 1 deletion .loopwright/tests/shell.test.mjs
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { describe, it, expect } from 'vitest';
import { mkdtempSync, readFileSync } from 'node:fs';
import { chmodSync, mkdirSync, mkdtempSync, readFileSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { runShell, writeReport } from '../scripts/lib/shell.mjs';
Expand Down Expand Up @@ -31,6 +31,30 @@ describe('runShell', () => {
});
});

describe('runShell binary resolution', () => {
// Regression: the adapters name bare binaries and rely on runShell putting
// node_modules/.bin on PATH. With npx, a repo whose deps were not installed
// resolved 'tsc' to an unrelated package off the registry instead of failing.
it('resolves a binary from the cwd node_modules/.bin', () => {
const dir = mkdtempSync(join(tmpdir(), 'lw-bin-'));
mkdirSync(join(dir, 'node_modules/.bin'), { recursive: true });
const bin = join(dir, 'node_modules/.bin/lw-fake-tool');
writeFileSync(bin, '#!/bin/sh\necho resolved-locally\n');
chmodSync(bin, 0o755);

const result = runShell('lw-fake-tool', dir);
expect(result.status).toBe(0);
expect(result.stdout.trim()).toBe('resolved-locally');
});

it('fails loudly when a binary is absent instead of resolving it elsewhere', () => {
const dir = mkdtempSync(join(tmpdir(), 'lw-bin-'));
const result = runShell('lw-definitely-not-installed', dir);
expect(result.status).not.toBe(0);
expect(result.stderr).toMatch(/not found/i);
});
});

describe('writeReport', () => {
it('creates nested directories and writes pretty JSON with a trailing newline', () => {
const dir = mkdtempSync(join(tmpdir(), 'lw-shell-'));
Expand Down
19 changes: 17 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ they're absent, and never touches your `config.json` or `baseline.json`. Your
stack stays yours: loopwright owns `.loopwright/` and nothing else.

**Or use this repo as a template** if you'd rather own the layer itself — you
get the engine, its 146 tests and its docs as a starting point to modify. Point
get the engine, its 148 tests and its docs as a starting point to modify. Point
`sources.roots` in `.loopwright/config.json` at your code and re-baseline.

## Your stack, not loopwright's
Expand All @@ -108,6 +108,20 @@ is not a way to pass the gate.
A tool that's configured but can't run writes `{ok: false, error}` and **blocks**.
Infrastructure failure must never look like success.

### Package managers

npm, pnpm and yarn all work. Adapters invoke bare binaries (`tsc`, `vitest`, …)
and the engine resolves them from your `node_modules/.bin` — never through
`npx`, which falls back to the registry and will happily download an unrelated
package of the same name when your dependencies aren't installed. The CI
workflow installs host dependencies with whichever package manager your lockfile
names.

`audit` is the one collector that is npm-only: `pnpm audit --json` and
`yarn npm audit` emit a different report shape than `npm-audit` parses, so on a
pnpm or yarn host the detector leaves `audit` **unconfigured** rather than
wiring an adapter that would silently report zero advisories.

## The loop

The gate is the enforcement half. The other half is how work reaches it:
Expand Down Expand Up @@ -157,6 +171,7 @@ authoritative verdict.
## Requirements

- Node ≥ 20.11 and a `package.json` (loopwright targets JS/TS repos)
- npm, pnpm or yarn — the engine and its CI workflow detect which from your lockfile
- [`gh`](https://cli.github.com), authenticated
- [Claude Code](https://claude.com/claude-code) locally, and the
[Claude GitHub App](https://github.com/apps/claude) on the repo for the
Expand All @@ -165,7 +180,7 @@ authoritative verdict.
## This repo runs on itself

The engine that ships to your repo lives here under `.loopwright/scripts/`,
covered by 146 tests in `.loopwright/tests/`, gated by the same workflow that
covered by 148 tests in `.loopwright/tests/`, gated by the same workflow that
will gate your PRs — baseline, integrity metrics and all. If the gate is wrong,
it's wrong here first.

Expand Down
Loading