Repository navigation
Conversation
Adds the optional test_double and test_case node kinds and the doubles edge to the scanner contribution schema (protocol stays 1.0), plus typed PHP readers that return null on malformed facts.
A test module's mocks, spies and stubs (Vitest and Jest, as Momus recognises them) become test_double nodes: their pattern, target, configured members with the values they pin and whether those fit the replaced member's declared return type, and how often the file reaches them. A doubles edge links each to the class or module it stands in for. Fixtures copied from Momus and Nemesis live in tests/Fixtures/test-fidelity.
…pe as written An implementation that declares no return type has its object literals widened by inference (`status: 'open'` reads as `string`), so a healthy mockImplementation read as not assignable. Each returned expression is now checked against the declared type as written. Also pins that every TypeScript worker module is an analysis input, so contributions cached before the test-double collector miss and are re-read.
…ace spies by specifier A pinned value with no framework types to give it context widened its literals, so healthy mockReturnValue and mockResolvedValue pins read as definite drift. Values are now held to the declared type as written, through Promise.resolve and await, across every part of an intersection, and a generic call's widened refusal is unknown rather than false. vi.spyOn on an `import * as ns` namespace named the module by its absolute file path. It now targets the module by its specifier, with a doubles edge to that module.
Add the test-fidelity seam: FindingCategory, FindingConfidence, TestFinding, the TestFidelityRule interface, TestFidelityGraph over a scan's contributions (cached ones included) and TestFidelityAnalysis, run from ScanAnalysisPipeline. Findings resolve onto node ids as classifications do and are stored in the new test_findings table (migration 027), replaced wholesale by every scan that rebuilds the graph and left in place by the no-change fast path. No real rules yet.
…embers, rebuild on rule changes An unresolvable finding reference now drops the finding (or detaches its target) with a warning instead of failing the scan. TestFidelityGraph::member() walks uses_trait/extends/implements breadth first with cycle protection. The registered test-fidelity rule ids are folded into the scanner-set hash, so adding or revising a rule declines the no-change fast path.
Ghost member, arity mismatch, return drift and visibility breach rules over test doubles, registered in the scan pipeline. A throwing rule now loses only its own findings for a scan and is named in a warning. Co-Authored-By: Claude <noreply@anthropic.com>
The TypeScript worker keeps one node per overloaded method, holding only the first signature. It now marks that node overloaded, and the arity and return drift rules read nothing from such a node. Co-Authored-By: Claude <noreply@anthropic.com>
The TypeScript worker now lists every export of a module and every member of a class or interface, so a name with no node of its own (a constant, an alias, an accessor, a parameter property) is no longer a ghost. Also stops missing_module on resolved specifiers, counts only the double's required parameters, and keeps return drift silent without a whole signature. Co-Authored-By: Claude <noreply@anthropic.com>
Rename the class list to member_names (members is the test double's own attribute), and build the module node's attributes in module-entry.js. Co-Authored-By: Claude <noreply@anthropic.com>
… queries A global rename had also renamed the test double's own members property in the schema, so stubbed members went unvalidated. A specifier with a loader query names no node and is no longer reported as a missing module. Co-Authored-By: Claude <noreply@anthropic.com>
TAUT-001..006 (self_comparison, mock_echo, constant definite; unconfigured_spy, mock_only, zero_reach warning), one finding per assertion by precedence. MOCKS_SUBJECT (mock_of_self, warning) for whole-module doubles of a test's subject, never a __mocks__ stand-in. Both registered in the scan pipeline.
Lists the test-fidelity findings a scan stored (ghost members, arity and return drift, visibility breaches, tautologies, mocked subjects) by file and line, filterable by path, confidence and category, with per-language counts. Languages whose workers emit no test facts yet say so.
Code outside the project, which a test's production count never sees, may call what replaces a package export, so the evidence for zero reach runs out there (Nemesis packageMock.spec.ts). Co-Authored-By: Claude <noreply@anthropic.com>
{ id: '1' } as any still says what it holds; it is held to the declared return type property by property and, refused, reads as untyped, so the drift is a warning (Nemesis UserService.spec.ts:17).
Co-Authored-By: Claude <noreply@anthropic.com>
A real scan of the Momus golden and Nemesis TypeScript fixtures stores what those tools expect, mapped onto Knossos's categories; the disagreements are listed at the top of TypeScriptAcceptanceTest. An untouched test file is re-judged against its changed production file. The benchmark gains the test_fidelity_rescan_one_file scenario, budgeted at 110% of the base commit, and src/TestFidelity counts toward the query-analysis coverage component. Co-Authored-By: Claude <noreply@anthropic.com>
…ed target The TypeScript read set reaches a test's target through imports, barrels and base classes alike, so a scan never reuses such a test file; the acceptance tests now say they prove findings follow an edit in both directions, and a pipeline test feeds a cached contribution with a changed production one. The scenario budget is 2.07s, within 110% of base. Co-Authored-By: Claude <noreply@anthropic.com>
A test double's and a test case's identity carries its line, so one comment at the top of a test file read as 101 changed facts in snapshot_diff and in the release notes architecture_trends builds from it. They are derived per scan, as findings are: the archive, the active-graph reads a diff, gate or trend compares against, and the counted figures leave them out, with the doubles edges and the contains edges, roles and memberships that touch them. Co-Authored-By: Claude <noreply@anthropic.com>
search_architecture matched every text atom of a node's attributes, so a test double or test case matched as a component, and a class or module matched each name in its member_names or exports, inherited ones included, so every subclass matched its parent's members. Those lists match no term now, and the two test kinds are found only when a kind filter asks for them; architecture_summary counts neither. Co-Authored-By: Claude <noreply@anthropic.com>
…s alone The doubles and test-case passes ran inside the per-file catch, so a throw in either returned no nodes for the test file and took its module, imports and calls with it. They run in their own catch now: a throw takes back every node and edge they added, the test names written onto the doubles with them, and the file reports one TS_TEST_FACTS_FAILED warning. Co-Authored-By: Claude <noreply@anthropic.com>
DoubleTests walked every test case for each double, once per module double in the mocks-subject rule and once per zero-reach candidate in the tautology rule: 2,000 doubles over 15,000 tests took 22.9s of rule time. The graph indexes its test cases by canonical name and by file when it is built (0.04s for the same facts, findings identical), and a scan holding no test fact runs no rule and builds no graph. Co-Authored-By: Claude <noreply@anthropic.com>
vi.mock('./virtual-thing', factory, { virtual: true }) and its jest form mock a module that exists only as the mock, so its specifier resolving to nothing proved nothing, yet the double was a definite GHOST_MEMBER/missing_module. A virtual mock's target keeps no specifier and no doubles edge.
Co-Authored-By: Claude <noreply@anthropic.com>
A code span broken across lines left operands} at column 0 and ended the list item in the protocol reference; the negated() helper had landed between matcherCall and its JSDoc. Co-Authored-By: Claude <noreply@anthropic.com>
isExternal read any specifier starting with ~ as inside the project, so a mock of a package named ~pkg that resolved to nothing was a definite missing_module; the documented alias is ~/. Co-Authored-By: Claude <noreply@anthropic.com>
A stubbed name a type lacks but whose ancestor is an external_* stand-in stays silent; a name found two re-exports away is no ghost, and a name a barrel's complete list and its re-export chain both lack is. Co-Authored-By: Claude <noreply@anthropic.com>
VISIBILITY_BREACH said definitely that vi.spyOn(s as any, 'secret') cannot replace a private method, but TypeScript's private and protected hold at compile time only and the spy works. Keyword visibility is now a warning that the double bypasses the public interface; an ECMAScript #name member, which no double can reach, is definite, whether the worker gave it a node or only listed its name. The acceptance list and its disagreements record the deviation from the spec's mapping. Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
…rces it The scenario's 2.07s was 110% of one development machine's measurement, but the release benchmark runs in docker on a shared runner. Its enforced limit now takes the small profile's headroom over that machine (10s against 2.57s, 3.89x) onto the scenario's 1.91s, rounded up as the sizes are: 8s. The 10% comparison stays in the doc as the one-off acceptance it was, with the medians it is made of spelled out, and a round that did not parse the production file and its five test files again fails the run. Co-Authored-By: Claude <noreply@anthropic.com>
…ir length The search exclusions move to SearchExclusions and the virtual-mock test to its own file, so neither file grows past its listed length; the lint list names the files this wave added. Co-Authored-By: Claude <noreply@anthropic.com>
Two use lines added to GraphSummaryQuery moved its bindValue(':limit', $limit + 1) from line 81 to 83.
Co-Authored-By: Claude <noreply@anthropic.com>
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars.
|
fits, resolvedFits and objectFits called one another, a three-member dependency cycle the quality gate counts. The helpers now take the judging function as a parameter, so the recursion over an expression's shape no longer closes a loop of functions.
…scan The worker's JS coverage counts only what PHPUnit runs the worker over; the vitest suite is not measured. The returns, test-case flow, stubGlobal, mocked-instance and proxy readers were exercised only there, which left the TypeScript floors unmet. Scan the same inputs through the worker as the core does and assert what each pinned value and operand resolves to.
This was referenced Nov 20, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Phase 1 of folding the Momus and Nemesis test checks into Knossos, for TypeScript from start to finish. The TypeScript worker now reports test doubles and test cases as facts. New rules in PHP core judge those facts and store findings for each scan. The findings are available through the
audit_testsMCP tool and theknossos audit-testsCLI command. Knossos stays read-only and never runs project code.test_doubleandtest_casefacts, plus completeexports/member_namesname lists with*_openmarkers. The scanner protocol and output schema stay at1.0; everything new is optional.definiteorwarning:GHOST_MEMBER: a mock stubs a member or export the real code doesn't have.ARITY_MISMATCH: a stub requires more arguments than the real function accepts.RETURN_DRIFT: a stub returns something the real type can't return.VISIBILITY_BREACH: a test reaches into a private or protected member.TAUTOLOGY: an assertion that can't fail (six sub-rules).MOCKS_SUBJECT: the test mocks the very code it is supposed to test.027_test_findings. Rule ids are part of the scanner-set hash, so existing databases rebuild once. One bad fact or a rule that throws never fails a scan; it becomes a warning diagnostic instead.snapshot_diffor trend notes.search_architectureandarchitecture_summaryleave them out unless you ask for those kinds.Acceptance
TypeScriptAcceptanceTestscans copies of the Momus and Nemesis TypeScript fixtures:dogfood-cleanfixtures produce no findings.Where Knossos differs, on purpose or because something is not ported yet:
ledger.test.ts:39: Momus reports both mock_only and unconfigured_spy. Knossos gives one finding per assertion, the narrower one:unconfigured_spy.assignability.test.ts:30: the stub declares no return type, so the comparison rests on an inferred type and the finding is awarning.*.fixture.tsfiles are worker fixtures that its golden list doesn't cover, so the comparison uses only the*.test.tsfiles.UserService.spec.ts:32expect(true).toBe(true): Momus reports TAUT-001 and TAUT-003, Nemesis reports nothing. Knossos reports one finding,self_comparison.UserService.spec.ts:17and:23: Nemesis's outcome is the one expected. Nemesis reports one return drift per missing field; Knossos reports one per pinned value.UserService.spec.ts:29, Nemesis reports ARITY_MISMATCH from(spy as any).toHaveBeenCalledWith(...). Matching it needs a call-matcher arity fact; this is deferred to phase 2.packageMock.spec.ts:11: a spy onnode:fsstays silent in all three tools.dogfood-clean/tests/catalog.test.ts:1: Momus reports MOCK-001 and Nemesis reports nothing. Nemesis's silence is the outcome expected.private/protectedis awarning, notdefinite. TypeScript visibility is checked only at compile time, so the spy does work. ECMAScript#privatemembers staydefinite.Performance
test_fidelity_rescan_one_file, base e9e156cRules index test cases once per scan. A case with 2,000 mocks and 15,000 tests went from 22.9s to 0.04s, with identical findings.
Test plan
vendor/bin/phpunit: 6,741 tests. The only failures are 3 Rust-worker tests that also fail onmain(ProjectUnitVerificationTest×2,ReturnedReceiverTest×1).vitest: 510 of 510 pass.src/TestFidelity: covered MSI 95% (floor 63).generate-reference --check.as anymocks and the performance on a large suite.Follow-ups for phase 2
testChecksandquality_gate.console,Date.now) as external.test_doubleandtest_casenodes.