Skip to content

test(scaffold): fail Validate when the repo's own .taskless is behind the latest migration - #376

Merged
theCodeDrift merged 1 commit into
mainfrom
test/dogfood-scaffold-current
Sep 22, 2026
Merged

theCodeDrift merged 1 commit into
mainfrom
test/dogfood-scaffold-current

Conversation

@theCodeDrift

Copy link
Copy Markdown
Member

What

One vitest file, packages/cli/test/dogfood-scaffold-current.test.ts: this repository's .taskless/taskless.json must record LATEST_SCHEMA_VERSION. Runs under Validate, so the pull request that adds a migration goes red until it also runs pnpm build && pnpm cli init and commits the rewritten .taskless/. The failure message names that command.

Why

Migrations run on init, demo, onboard, and rule delivery — and on nothing the development loop invokes. pnpm lint builds and runs check, which by design refuses a stale scaffold rather than migrating it (requireCurrentSchema, migrate.ts). So a migration PR could pass every check while leaving the dogfood tree behind, and the gap only surfaced when check hit the wall: install.cliVersion sat at 0.11.0 from 2026-08-29 until migration 0008 (#368) made check refuse on 2026-09-21.

Deliberately not automated into lint. A tracked file rewritten as a side effect of a lint run is the surprise the check wall exists to prevent; a red test naming the command is the visible version.

Only the schema version is pinned. install.cliVersion is the nightly the installed skill was reconciled against and moves on its own schedule.

Verified

  • Passes on main (7 = 7).
  • With the manifest edited to "version": 6 it fails: .taskless/taskless.json records scaffold version 6 but this CLI's latest migration is 7. Run \pnpm build && pnpm cli init` and commit the rewritten .taskless/.`
  • prettier, eslint, pnpm --filter @taskless/cli typecheck clean.

Ordering

If this lands before #368, it turns #368 red until that branch is rebased — #368 already carries version: 8 and the migrated tree, so it goes green on rebase with no other change. Landing #368 first avoids the interaction entirely.

… the latest migration

Migrations run on init, demo, onboard, and rule delivery, and on nothing
the development loop invokes: pnpm lint builds and runs check, which
refuses a stale scaffold rather than migrating it. A pull request that
adds a migration could therefore pass every check while leaving the
dogfood tree behind; install.cliVersion sat at 0.11.0 from 2026-08-29
until migration 0008 made check refuse.

Assert the scaffold version against LATEST_SCHEMA_VERSION so the pull
request that adds a migration is the one that applies it. Not automated
into lint on purpose: a tracked file rewritten as a side effect is the
surprise the check wall exists to prevent.
@theCodeDrift theCodeDrift added the skip-changeset PR intentionally ships no release note (bypasses the changeset requirement) label Sep 22, 2026
@theCodeDrift

Copy link
Copy Markdown
Member Author

@claude /review

@claude

claude Bot commented Sep 22, 2026 •

Copy link
Copy Markdown

Claude finished @theCodeDrift's task in 1m 50s —— View job


Review: test(scaffold): fail Validate when the repo's own .taskless is behind the latest migration

Review mode: incremental — read 0 prior review thread(s) before reviewing. .prior-review.json contains no reviewThreads, reviews, or comments, so this is the first review of this PR; nothing to classify as addressed/still-open.

  • Read .prior-review.json (empty)
  • Read the diff (gh pr diff 376) and PR description
  • Verify path resolution, LATEST_SCHEMA_VERSION semantics, and the actual .taskless/taskless.json state
  • Check conventions (root CLAUDE.md, STYLEGUIDE-CODE.md testing section)
  • Post final assessment

Summary: This is a single new vitest file, packages/cli/test/dogfood-scaffold-current.test.ts. It asserts this repo's own .taskless/taskless.json version field equals LATEST_SCHEMA_VERSION exported from packages/cli/src/filesystem/migrate.ts, with a failure message naming the fix (pnpm build && pnpm cli init).

I checked the things most likely to be wrong in a test like this, and none of them are:

  • Path resolution — join(import.meta.dirname, "..", "..", "..", ".taskless", "taskless.json") from packages/cli/test/ resolves to <repo-root>/.taskless/taskless.json, which exists and is tracked (only .taskless/skills/ and .taskless/commands/ are gitignored, not the manifest itself).
  • Values line up — LATEST_SCHEMA_VERSION in migrate.ts:357 is derived from the migration map (sortedMigrations(migrations).at(-1)?.[0] ?? 0) specifically so tests don't hardcode a number, and the repo's checked-in manifest currently reads "version": 7, matching. Consistent with the PR's own verification (7 = 7 on main, fails when edited to 6).
  • Ordering in Validate — pnpm lint (which builds) and pnpm build both run before pnpm test in .github/workflows/validate.yml, but this test imports ../src/filesystem/migrate directly (TS source), not dist/, so build-ordering doesn't affect it either way.
  • Style precedent — the join(import.meta.dirname, "..", …) climb and the expect(value, "message").toBe(...) custom-message form are both already established patterns elsewhere in packages/cli/test/ (e.g. engine-version-consistency.test.ts, demo-command.test.ts, ast-grep-vendor-contract.test.ts), so this file is consistent with the suite rather than inventing a new idiom.
  • STYLEGUIDE-CODE.md "Verify Build Output In The Build, Not By Parsing It" — doesn't apply here: the test reads one explicit JSON field via JSON.parse (structured, not a regex reconstruction of generated code), the same way other tests in this suite read package.json. It's not re-deriving a fact the build already knew from unstructured output.
  • Import style — direct import from ../src/filesystem/migrate, no barrel import, matches STYLEGUIDE-CODE.md.

Nothing else in the diff (a new 49-line file, no other files touched) has a plausible failure path beyond the one edge case the file itself doesn't handle: if .taskless/taskless.json were ever missing or malformed, readFileSync/JSON.parse would throw an unhandled error instead of the crafted assertion message. That's a real gap in message quality but not a correctness bug — the file is guaranteed to exist and be valid JSON in this repo, and a raw parse/read error still fails Validate loudly, which is the actual goal.

No findings. I found nothing new to raise, and there's nothing from a prior review to carry forward since none exists yet.

@theCodeDrift
theCodeDrift merged commit fd896f9 into main Sep 22, 2026
8 checks passed
@theCodeDrift
theCodeDrift deleted the test/dogfood-scaffold-current branch September 22, 2026 00:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skip-changeset PR intentionally ships no release note (bypasses the changeset requirement)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant