Make the README witness check capable of failing - #360
Merged
Conversation
`sourceFiles` collected every `.ts` under `src/`, tests included — and the
witness table is a set of string literals in this very file. The corpus
therefore contained the strings it was searching for, so every witness matched
itself and the check could not fail for any reason.
Three changes, each verified by breaking the tree and watching the right test go
red rather than by reading the code:
Test files are out of the corpus, and a self-check holds them out. Excluding
them is one line, and without an assertion the next refactor of `sourceFiles`
restores the loop with nothing failing. Reverting that line now fails with
`expected [ …(836) ] to not include 'src/exampleCoverage.test.ts'`. It also
asserts the corpus is non-empty, since an empty one passes everything below it.
A witness must appear on an IMPORT, not anywhere in the file. `RateLimiter`
occurs eleven times outside tests: once as a real
`import type { RateLimiter } from "workglow"`, and ten times in prose or as a
substring of a longer local name like `secFetchRateLimiterTableNames`. Ten of
the eleven prove nothing and a substring search cannot tell them apart. The
extraction strips comments first and matches whole statements rather than
lines — these imports are routinely multi-line, and `createStandardKbStrategy`
sits on a continuation line of one, which a line-wise filter reported as a
missing package.
The README's commands are resolved against the registered command tree, for the
reason `commandsBoot.test.ts` gives: help text is prose, and a name appearing in
some description satisfies a substring check without being registered. It walks
the leading bare words of each backticked `sec …` and stops at the first flag or
placeholder. Fifteen invocations resolve today; adding one for a command that
does not exist fails it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LWp6Z6wvAPDaDCjAFcTSj6
sroussey
added a commit
that referenced
this pull request
Sep 8, 2026
### Features - implement ITabularStorage.join for @WorkGlow 0.5.0 (#362) - implement CLI signal teardown and resource shutdown ### Bug Fixes - declare the embedding width beside the model instead of assuming it (#359) #### release - declare the runtime floor, and derive the release bump (#357) #### kb - stop --dry-run creating the index tables, and report them in db stats - bound `ask`'s implicit index, pin the embedding model, and let `db reset` drop the index #### ask - refuse rather than answer from model memory when nothing is retrieved #### adv - scope each Form ADV archive to its own folder and make ingest idempotent ### Performance #### status - read the newest ADV snapshot with one indexed row ### Tests #### coverage - make the README witness check capable of failing (#360) ### Chores - update deps - update dependencies to latest versions - migrate from Prettier to oxfmt for code formatting #### deps - upgrade Vitest to 5 (#342) ### Updated Dependencies - `@workglow/cli`: 0.5.0 - `typebox`: 1.3.29 - `workglow`: 0.5.0 - `@types/bun`: 1.4.2 - `bunset`: 1.1.1 - `oxlint`: ^1.82.0 - `vitest`: ^5.0.0
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #350.
The problem
sourceFilescollected every.tsundersrc/, tests included — and the witness table is a set of string literals in this very file. The corpus therefore contained the strings it was searching for, so every witness matched itself and the check could not fail for any reason.Three changes, each verified by breaking the tree
I checked each guard by making the regression it exists to catch and confirming the right test went red, rather than by reading the code.
1. Test files are out of the corpus, and a self-check holds them out. Excluding them is one line in
sourceFiles; without an assertion the next refactor of that function restores the loop with nothing failing. Reverting the line now fails:It also asserts the corpus is non-empty — an empty one passes every check below it.
2. A witness must appear on an import, not anywhere in the file.
RateLimiteroccurs eleven times outside tests:import type { RateLimiter } from "workglow"secFetchRateLimiterTableNamesTen of the eleven prove nothing, and a substring search cannot tell them from the one that does. Verified by putting a witness string in a comment only and watching the package be reported unproven.
The extraction strips comments first and matches whole statements, not lines. That detail is load-bearing: these imports are routinely multi-line, and
createStandardKbStrategysits on a continuation line of one — a line-wise filter reported@workglow/aias missing, which is how I found it.3. The README's commands resolve against the registered command tree. For the reason
commandsBoot.test.tsgives: help output is prose, and a name appearing in some description satisfies a substring check without being registered at all. It walks the leading bare words of each backticked`sec …`and stops at the first flag or placeholder, since only the subcommand path is resolvable.One correction to the issue
The issue says this check "has already found one defect (
sec show companiesdoes not exist) before being written". That string is not in the README today —grepfinds nothing, and all fifteen backticked invocations resolve. So the check lands green rather than red; it is a ratchet against the next one, not a fix for a live break. Adding`sec show companies`to the README does fail it, which is how I confirmed it is not vacuous.Verification
format,lint,typecheckall clean.exampleCoverage.test.ts4/4 andcommandsBoot.test.ts2/2.🤖 Generated with Claude Code
https://claude.ai/code/session_01LWp6Z6wvAPDaDCjAFcTSj6
Generated by Claude Code