test: cull duplicate/tautological tests, fix env-brittle checks, split unit and packaging tiers - #155
Merged
Merged
Conversation
…trees - stock-pi: assert the Node major meets SUPPORT_MATRIX.node.minimum instead of an exact patch version, so a 24.x patch bump no longer fails the suite. - extension: drop assertCurrentNode (same exact-patch probe). - doctor AR-14: repoRoot resolved one level above the checkout, so the nested `bun test` ran whatever tests lived in the parent directory (fails inside worktrees). Resolve from the test dir, assert the target suites exist, and drop handoff.test.ts/sdd.test.ts, which were deleted long ago. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
phase-9-traceability.test.ts (485 lines) mapped POST-xx/AR-xx/CA-xx rows of the archived reliability-overhaul plan to "file::exact test name" strings and required the stale branch feature/workit-reliability-overhaul. It asserted document cross-references, not product behavior: it failed on test renames or plan edits and forced a head_ref checkout in CI. Every behavior it pointed at keeps its own test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The phase-0, release-candidate, package-contents and packed-runtime gates re-asserted the same facts about the same cached pack: - phase-0 "final release candidate is byte-stable with the phase-0 pack": same check as release-candidate "a fresh repack yields byte-identical sha256". - phase-0 "packed adapter core dependency equals the packed core version" and release-candidate "packed release metadata is synchronized": identical loops; folded into "packed internal dependencies rewrite to the same release version", which now also requires the core dependency to exist. - phase-0 "expected entry files ship in each packed tarball": every entry is asserted by the per-tarball tests in package-contents.test.ts. - package-contents "no packed package.json carries a workspace:, file: or git: protocol" and the protocol lines of release-candidate's self-contained test: subsumed by phase-0's CA-03 test, which also checks every range is a valid semver range. - packed-runtime "the runtime gate exercises the same final candidate artifacts": tautology; both calls return the same cached array. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… from the manifest Byte-copy tests: - methods "host skill copies stay byte-identical to every canonical core skill" and package-contents "tracked CLI template mirrors stay byte-identical" compared working-tree files that `bun run build` regenerates, so after the CI build step they passed by construction. Replaced by generated-copies.test.ts, which compares committed git trees (HEAD:packages/workit-core/skills vs each host copy, core templates vs the CLI mirror). A canonical edit committed without regenerated copies fails. Fourteen-skill list (restated in 9 places): - methods "method manifest lists exactly fourteen core skills" now checks the manifest against the canonical skill directories, without a literal. - Pi extension "Pi package ships the fourteen canonical method skills" deleted: the committed-tree check plus the manifest check cover it. - package-contents opencode tarball skill list deleted: identical to opencode-skill-contract "opencode packed tarball ships exactly the canonical method skills". - `expect(WORKIT).toHaveLength(14)` pins in opencode-skill-contract and cursor-install-invariants deleted (restated the constant). - Codex plugin, OpenCode plugin, opencode-v2 shell/matrix, manifests, packed-runtime and stock Pi now derive expected skill and wk- alias sets from WORKIT_METHOD_SKILLS / WORKIT_SKILL_ALIASES; stock Pi checks the discovered skill names, not only their count. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…entences methods.test.ts asserted ~35 literal sentences of the bootstrap and SKILL.md prose, so any rewording failed the suite without a behavior change. Kept only the keywords agents depend on: - bootstrap: native allow/ask/deny stays authoritative, never evade a denial, no task.start/policy.assess preamble, TDD/review left to policy, routed skill names, and every operation family name. - steer stays lifecycle-free, babysit does not authorize merge, challenge never fabricates a native permission. - all method skills: the four negative preamble/policy-gating patterns are now checked across every skill (previously two of them for two skills). Deleted "planning keeps documentation proportional..." (pure prose of workit-plan/workit-implement, no contract keyword) and the sentence-level assertions of the steer/babysit/challenge tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ep check typescript-parity.test.ts described its cases as parity with shell scripts that no longer exist, but every case exercises the TS context generators against a real fixture repo, so the behavior coverage stays. Renamed to repo-context.test.ts and named the cases after the behavior they check. Deleted "PATH scanning uses path.delimiter (Windows-safe) not a literal colon": it grepped pr-create.ts for the text `split(path.delimiter)`; the missing-CLI guard in workspaces-scripts.test.ts runs whichOnPath over a path.delimiter-joined PATH, which is the behavioral check. doctor AR-14 now targets the renamed file. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
workit-opencode/route-denial-scope.test.ts and workit-pi/route-denial-scope .test.ts ran the same five commands, with and without a live task, against the same adapter functions that core/route-denial-parity.test.ts already drives for OpenCode, Pi and Codex. The parity table now also asserts what only the per-host files checked: the denial carries the protected_ref reason (OpenCode error, Pi block reason with block: true), plus the `git checkout -b feature/raw` and `git status --short` rows. The Pi extension's own tool_call tests stay as the thin adapter-wiring check. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…l dist
The packed Codex launcher tests and the stock Pi tests ran `npm pack` on
packages/workit-{codex,pi}, which ships whatever dist/ is on disk: they
failed on a fresh checkout and silently tested stale code after a source
edit. They now take their tarball from packWorkspacePackages(), which
builds every adapter into a sandbox from current source (cached per run,
the same artifact the release gates check), so they need no prior
`bun run build`.
- The two packed Codex launcher tests move from desktop.test.ts to
packed-launcher.test.ts (packaging tier); desktop.test.ts keeps its
source-level unit tests.
- "npm installs the packed package without workspace protocol
dependencies" moves from extension.test.ts to stock-pi.test.ts and now
installs the release-shaped tarball (workspace deps rewritten).
- Deleted extension "stock Pi discovers the package manifest through its
local package manager": same pack/install/get_commands flow as stock-pi
"stock Pi discovers the packed Workit extension and skills", which
asserts a superset (worker command, settings, reconcile prompt).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ersion - `bun run test` runs the domain/unit tier; `bun run test:packaging` runs the suites that pack tarballs, run npm/Pi installs, drive doctor against installed artifacts, or need docker. The packaging list lives once in scripts/test.ts (the unit tier ignores the same patterns); plain `bun test` still runs everything. - scripts/test.ts drops node_modules/.bin from PATH before spawning `bun test`: `bun run` prepends it, which exposed the repo's own npm dependency to host-install code and failed two packed-CLI tests (and slowed an opencode-v2 shell test into a timeout) only under `bun run`. - test/shared/node-guard.ts, preloaded from bunfig.toml, stops any `bun test` run with one message when the PATH node is older than SUPPORT_MATRIX.node.minimum, instead of ~30 confusing doctor/setup failures. - `check` runs both tiers through the scripts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…kout ref - New `remaining` job runs test/workit-codex, test/workit-mcp, test/opencode-v2 and test/acceptance (~120 tests), which no job ran. They need no prior build (packed suites build their own artifacts). - The artifacts job no longer checks out `ref: head_ref`; only the deleted phase-9 traceability test needed it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cursor-install-mcp.test.ts installs packages/workit-cursor from the dev checkout and its doctor step requires dist/mcp-server.js. It only passed when `bun run build` or another suite (install-scripts) had built that dist earlier in the same run. It now runs the Cursor build script (~0.1 s) in beforeAll and sits in the packaging tier with the other install flows. project-config pins the new `check` script string. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
BrainerVirus
force-pushed
the
chore/test-suite-cleanup
branch
from
October 3, 2026 19:01
7591265 to
e3a7b8e
Compare
…uesses findNpmCli only tried install layouts relative to the `npm` it found on PATH, so a package-bin symlink (node_modules/.bin/npm -> ../npm/bin/ npm-cli.js), which `bun run` puts first on PATH, failed Cursor setup with "npm CLI could not be resolved". It now realpaths `npm` first and accepts the target when it is npm-cli.js; the layout candidates remain the fallback. Covered by a host-install test with a symlinked .bin/npm (fails without the fix). scripts/test.ts no longer strips node_modules/.bin from PATH; that workaround only hid this bug. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ount Review mutation-tested the keyword trim: inverting any of ten rules left the suite green. methods.test.ts now pins one meaning-bearing phrase per rule (whitespace-flexible, wording-exact): bootstrap prefers native host Git, a local-commit endpoint does not imply PR readiness, reconcile every requested item, a local commit alone is not push evidence; babysit is not started by PR creation and stops at PR-ready; steer does not silently resume; plan asks for no separate approval; implement reconciles every named deliverable; behavioral-tdd leaves GREEN without RED unsatisfied. Each inversion was applied and fails the test. The manifest test also pins WORKIT_METHOD_SKILLS to 14, so dropping a skill from both the manifest and its directory fails; skill-set changes update it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cursor-install-mcp.test.ts now copies packages/workit-cursor into a temp "checkout", builds it there with the Cursor build script's target argument, and installs from that sandbox, so it no longer writes dist/ into the working checkout. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
It ran test/workit-core/handoff.test.ts and sdd.test.ts, which no longer exist, so a dispatch would fail. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Adds "Write one vertical RED slice that fails" for workit-behavioral-tdd:
the existing pin only covered the close gate, so rewriting step 3 to
implement first stayed green. That inversion now fails the test.
- phrase() escapes regex metacharacters in each word, so a pinned phrase
matches literally ("a.b" no longer matches "axb").
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
|
🎉 This PR is included in version 2.1.4 🎉 The release is available on:
Your semantic-release bot 📦🚀 |
BrainerVirus
added a commit
that referenced
this pull request
Oct 3, 2026
Conflicts resolved keeping both intents: - package.json: config-driven lint/format scripts and pinned tooling from this branch; #152's knip reachability step and #155's test tiers (`test` = unit, `test:packaging`) and `check` from main; ink devDependency from main. - ci.yml: the new fast/test/portability layout; the Linux test job runs both tiers, which covers #155's `remaining` job (codex, mcp, opencode-v2, acceptance) and every other directory. - Source/test files main rewrote or deleted (#149, #151, #152, #155): main's version taken; lint fixes are re-applied in a follow-up commit. - bun.lock regenerated with bun 1.4.1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
BrainerVirus
added a commit
that referenced
this pull request
Oct 3, 2026
Safe autofixes (toSorted, unnecessary assertions) plus manual no-shadow renames and typed sort keys on code that arrived with #149/#151/#152/#155. The boolean-compare, type-conversion, template-expression and map-spread rules stay off, so no strict comparison or coercion was rewritten (checked: no removed `=== true`/`!== true`/`=== false`/`!== false`, String(), Boolean() or new Object.assign in the diff). The confirmation-gate test drops its migrateLegacyDocs case: #152 deleted docs-migration. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
BrainerVirus
added a commit
that referenced
this pull request
Oct 3, 2026
…, dependency bumps) (#153) * build(deps): bump zod, MCP SDK, OpenCode SDK, Oxc tooling, knip, release plugins - zod 4.5.4 -> 4.6.5 (core, mcp) - @modelcontextprotocol/sdk 1.30.0 -> 1.32.0 - @opencode-ai/plugin 1.18.30 -> 1.18.34 (support-matrix current + CI env; the 1.18.30 floor is unchanged) - oxlint 1.81.0 -> 1.86.0, oxfmt 0.66.0 -> 0.71.0, knip 6.35.1 -> 6.39.0 - @semantic-release/npm 13.2.0, @semantic-release/github 12.0.10 - conventional-changelog-conventionalcommits 8.0.0 -> 9.3.1: 10.x needs conventional-changelog-writer@9, but @semantic-release/release-notes-generator 14.1.1 (latest) still ships writer@8 and fails at render time; 9.3.1 renders identical notes. knip 6.39 now reports the `npm exec -c workit-cursor-session-start` call in the packed-runtime test as an unlisted binary; it is the workit-cursor bin installed into a temp project, so it is ignored like glab. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore(lint): config-driven oxlint (correctness/suspicious/perf, type-aware) and oxfmt - Add .oxlintrc.json: correctness=error, suspicious+perf=warn with denyWarnings, explicit eslint/typescript/unicorn/oxc plugins, and type-aware rules through oxlint-tsgolint 7.0.2003 (bundles typescript-go, works with TS 7; whole repo lints in ~2 s). - Add .oxfmtrc.json. Lint and format now walk the repo root and skip gitignored paths plus ignorePatterns, so package.json no longer repeats the path list four times, and previously unchecked scripts/ and packages/*/scripts are covered. - Fix findings instead of silencing them: sort/reverse -> toSorted/ toReversed (tsconfig target ES2023), no-shadow renames (e.g. locals that shadowed the `path` module), filter()[0] -> find/findLast, unbound runtime methods wrapped, needless awaits/assertions/conversions removed, `${array}` -> join(","). - Rules turned off carry a justification in the config: no-await-in-loop (ordered subprocess/lock steps), consistent-function-scoping (style), no-unsafe-type-assertion (~1k JSON-boundary casts, ratchet later), no-base-to-string (tsgolint ignores checkUnknown), consistent-return (tsc already enforces it); in tests, await-thenable (bun-types type `.resolves/.rejects` as void) and unbound-method (monkeypatch restore). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore(hooks): add opt-in lefthook pre-commit and commitlint commit-msg hooks pre-commit formats (oxfmt, re-staged) and lints (oxlint, type-aware) only the staged files in parallel: ~0.8 s measured. commit-msg runs commitlint with the conventional config (~0.5 s), since semantic-release and analyze-release-scope read Conventional Commits. Install is opt-in via `bun run hooks:install` and `no_auto_install` is set, because git hooks are shared by every worktree of a clone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore(ci): pin react-doctor 0.9.14 as a devDependency and delete probe workflows `npx -y react-doctor@latest` resolved a new version on every run, and since 0.9.x it prints "React Doctor is not installed in this project" and exits 0 when react-doctor is not a dependency, so the CI gate scanned nothing. As a pinned devDependency it runs the real scan (~2 s; warnings only, no errors). sync-token-probe.yml was a dispatch-only check to delete "once the token is stable"; the last probe passed on 2026-08-28 and the v2.1.0 manifest-sync PR (#147) was opened with RELEASE_SYNC_TOKEN. windows-inner-suite-probe.yml runs test/workit-core/handoff.test.ts and sdd.test.ts, which no longer exist. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore(ci): one fast parallel job, every test suite, SHA-pinned actions, CI-gated release CI: - `fast` job: one install, then lint (type-aware), format, knip, typecheck, react-doctor, actionlint and zizmor run concurrently (~5 s locally). Typecheck ran in 6 jobs and lint/format hid in a job named `shared`. - `test` job runs a plain `bun test` on Linux after one build, so test/workit-codex, test/workit-mcp, test/opencode-v2 and test/acceptance (never run in CI before) are covered, and new directories cannot escape. - `portability`: core + artifacts on macOS/Windows with one build each. The separate candidate job duplicated test/artifacts/phase-0-candidate. - Builds drop from 5 to 3 (one per OS); typecheck from 6 to 1. - Actions pinned to commit SHAs, persist-credentials: false, permissions default to none per job; actionlint and zizmor (offline) are clean. - PR runs cancel superseded runs; main runs are grouped per commit. Release: - release.yml is now a reusable workflow called by the `release` job in ci.yml with needs: [fast, test, portability], so a commit only publishes after its own CI passed. Secrets are passed explicitly (no inherit). - concurrency group with cancel-in-progress: false, so two merges cannot race on tags. - semantic-release runs from the pinned devDependency (`bun run release`) instead of an unpinned `npx`. - npm provenance: id-token: write plus NPM_CONFIG_PROVENANCE=true; npm still authenticates with NPM_TOKEN. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore(lint): restore strict confirmation gates and non-mutating maps loosened by autofix The type-aware autofixes trusted declared types at untyped JSON/env/host boundaries: - no-unnecessary-boolean-literal-compare rewrote `confirmed !== true` to truthiness in requireConfirmed (OpenCode init apply), migrateLegacyDocs and YouTrack postUpdate, so `confirmed: "false"`, "no" or 1 passed the gate. - no-unnecessary-type-conversion / -template-expression dropped String(), Boolean() and `${}` coercions (hostingApiHostMatches returned undefined instead of false). - the no-map-spread suggestion turned `({ ...entry, ... })` inside map() into Object.assign(entry, ...), mutating the source decisions/fixtures. Every rewritten `=== true`/`!== true`/`=== false`/`!== false` comparison and coercion is restored as it was on main (46 lines plus vcs-config and the four Object.assign sites). The four rules are turned off with the reason in .oxlintrc.json. Regression tests: confirmation gates reject "false", "no", "true", 1 and {} (all three fail against the loosened code). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore(ci): validate the Cursor marketplace before building and gate the release candidate pre-merge The CA-21 marketplace check must run on a clean checkout, so it now runs before `bun run build`. The test job also runs `verify:release-candidate` (the same pack-only gate release.yml runs before publishing, ~1.5 s locally), so a broken candidate fails the PR instead of the release. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore(hooks): keep lefthook's postinstall from installing hooks on dependency install The npm package's postinstall runs `lefthook install`, which ignores no_auto_install and would write into the .git/hooks shared by every worktree. bun already blocks it (lefthook is not in trustedDependencies); pnpm (`pnpm.neverBuiltDependencies`) and yarn (`dependenciesMeta.built: false`) are now opted out explicitly, and npm cannot install this workspace at all (EUNSUPPORTEDPROTOCOL on workspace:*, verified). A project-config test keeps lefthook untrusted and hooks opt-in via `bun run hooks:install`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore(lint): restore the locale and YouTrack date String() coercions Two coercions dropped by no-unnecessary-type-conversion were missed by the earlier restore (multi-line on main): config locale validation and the YouTrack logTime date argument now match main exactly. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore(lint): apply lint fixes to code merged from main Safe autofixes (toSorted, unnecessary assertions) plus manual no-shadow renames and typed sort keys on code that arrived with #149/#151/#152/#155. The boolean-compare, type-conversion, template-expression and map-spread rules stay off, so no strict comparison or coercion was rewritten (checked: no removed `=== true`/`!== true`/`=== false`/`!== false`, String(), Boolean() or new Object.assign in the diff). The confirmation-gate test drops its migrateLegacyDocs case: #152 deleted docs-migration. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Slice S6 (test-suite cleanup), based on main (after #152). Removes tests that restate config or duplicate other tests, fixes tests that break on environment details, and splits the suite into a fast unit tier and a packaging tier. One product fix:
findNpmClinow follows a symlinkednpm(for examplenode_modules/.bin/npm) tonpm-cli.js. Previously Cursor setup failed with "npm CLI could not be resolved" when that symlink came first on PATH, which is the case underbun run.bun run buildregenerates, so in CI they could never fail.WORKIT_METHOD_SKILLS/WORKIT_SKILL_ALIASES.typescript-parityis renamed torepo-context; its source-grep test is dropped.protected_refreason.repoRootpointed one level above the checkout, which broke it in worktrees. It also listed test files that no longer exist.bun run buildis required.bun run test(unit) andbun run test:packaging. A preloaded guard stopsbun testwith one clear message when thenodeon PATH is older than 24.remainingjob runstest/workit-codex,test/workit-mcp,test/opencode-v2andtest/acceptance, which no job ran before. The phase-9-onlyref: head_refcheckout is removed.Measurements (Node 24.20.0, bun 1.3.14, this worktree)
bun test(after build)bun testwithout dist/bun run test(unit)bun run test:packagingbun test(all)lint, format:check, typecheck and knip pass.
Removed or replaced tests
generated-copies.test.ts, which compares committed git trees (checked: a committed drift fails)expect(WORKIT).toHaveLength(14)toContainon bootstrap and steer/babysit/challenge/plan/implement prose; "planning keeps documentation proportional…"SUPPORT_MATRIX.node.minimumcheckMoved without changing assertions:
workit-codex/packed-launcher.test.ts.stock-pi.test.ts.Review follow-ups
WORKIT_METHOD_SKILLSis pinned to 14 skills.cursor-install-mcpnow builds into a sandbox checkout instead of the repo.windows-inner-suite-probe.yml.scripts/test.ts; the npm bug it hid is now fixed infindNpmCli.Not done
Risks
package.json: thetest,test:packagingandchecklines change;project-config.test.tspins thecheckstring.bunfig.toml: new[test]preload.ci.yml: newremainingjob; the artifacts checkoutrefis removed.generated-copiescompares HEAD trees. Uncommitted local drift is caught only once it is committed, and in CI.npmcomes first on PATH: it now resolves instead of failing.🤖 Generated with Claude Code