feat: ship a working test suite in generated SDK projects - #32
Merged
Merged
Conversation
The previous PR replaced the greet example with a notes domain chosen so its contract has something to assert. This emits the tests that assert it, so a scaffolded project starts with a green suite and a pattern to copy rather than an empty src/ and good intentions. Nine tests, deliberately. The suite is a template people copy, so its size sets an expectation: every test here pins something a tool's description promises and nothing else checks, and none of them exercise the example's own data structures. A test proving a Map stores things would teach nothing about tool design while implying a scaffolded project owes one per function. Two layers, because neither subsumes the other: - notes-store.test.ts covers the two store guarantees a tool contract depends on: that the store reports the full match count rather than the page size, and that an unknown tag is distinguishable from a tag with no notes. - server.test.ts drives a real Client over InMemoryTransport.createLinkedPair() and asserts on the parsed tool payload - the surface an agent actually reads. A tool can be internally correct and still return something misleading. What the payload tests pin: truncation is signalled and the hint names how to narrow; a complete result does not claim truncation; an unknown tag is distinguishable from an empty one; an unknown id returns isError naming the next tool to call; create returns an id that get accepts; a ResourceTemplate has both halves wired, since list and read are easy to implement singly. Plus a generic check that every tool and parameter carries a description, which keeps holding after the example is replaced. Three things worth knowing: - No vitest.config.ts is emitted and none is needed - vitest discovers src/**/*.test.ts and handles TypeScript unaided. - tsconfig gains src/**/*.test.ts in exclude, or the tests compile into dist/ and ship in the production Docker image. getTsconfigTemplate now takes withTests, on for SDK projects only. - The emitted test code uses string concatenation rather than template literals. Nesting a template literal inside the one that generates it needs triple-escaped backticks, and getting it wrong emits a stray backslash that fails to parse - which is exactly what happened first time round. A test now asserts no escaped template-literal syntax reaches the output. The template source files are named store-test.ts and server-test.ts, not *.test.ts, so this repo's own vitest run does not try to execute them as suites. Adds vitest and @modelcontextprotocol/client to TEMPLATE_PACKAGES. The client must track @modelcontextprotocol/server's major; the script cannot enforce that pairing, so it is called out where the list is documented. FastMCP gets no tests this round: no test script, no vitest dependency, and no test exclusion in its tsconfig. Also refreshes AGENTS.md and the repo README, which still described the greet example and the pre-split single-file server.
IBJunior
force-pushed
the
feat/generated-tests
branch
from
September 19, 2026 15:53
75cf6bb to
3dca895
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Generated SDK projects now ship a working test suite.
npm testpasses immediately, with no config to add.Nine tests, deliberately. The suite is a template people copy, so its size sets an expectation. Every test pins something a tool's description promises and nothing else checks; none exercise the example's own data structures. A test proving a
Mapstores things would teach nothing about tool design while implying a scaffolded project owes one test per function. The emitted code is ~370 lines against ~200 of tests — a ratio that reads as "here is the pattern", not "here is the bar".Fourth of the sequence, and the point of the previous one: PR #31 replaced
greetwith a notes domain chosen so its contract has something to assert. This emits the tests that assert it.Two layers
Neither subsumes the other, so both are emitted:
notes-store.test.tscovers the two store guarantees a tool contract depends on — nothing else. Fast, no transport.server.test.tsdrives a realClientoverInMemoryTransport.createLinkedPair()and asserts on the parsed tool payload, which is the surface an agent actually reads. A tool can be internally correct and still return something misleading.What the payload tests pin:
returned: 5,matched: 30,truncated: true, hint contains "5 of 30"truncated: false,hintundefinedisErrorand names the tags that existisErrorand mentionslist_notescreate_notereturns an id thatget_noteacceptslistenumerates andreadresolves the same uriThat last one is deliberately generic — it keeps holding after a user replaces the example with their own tools, and an undescribed parameter is a common reason an agent calls a tool wrongly.
Three things worth a reviewer's attention
No
vitest.config.tsis emitted, and none is needed. Vitest discoverssrc/**/*.test.tsand handles TypeScript unaided. Verified by running the generated suite, not assumed.tsconfiggainssrc/**/*.test.tsinexclude. Without it the tests compile intodist/and ship in the production Docker image.getTsconfigTemplatenow takeswithTests, on for SDK projects only. Confirmeddist/contains 12 files and 0 test files after a build.The emitted test code uses string concatenation, not template literals. Nesting a template literal inside the template literal that generates it needs triple-escaped backticks, and getting it wrong emits a stray backslash that fails to parse — which is what happened on the first attempt here, caught by running the generated suite. A test now asserts no escaped template-literal syntax reaches the output, and I verified that guard actually fires by injecting the real bug shape (a single-escaped backtick emits correctly, so only the triple form is a defect).
Relatedly, the template source files are named
store-test.tsandserver-test.tsrather than*.test.ts, so this repo's own vitest run doesn't try to execute them as suites.Dependency tracking
vitestand@modelcontextprotocol/clientare added toTEMPLATE_PACKAGES. The client must track@modelcontextprotocol/server's major — the script can't enforce that pairing, so it's called out where the list is documented.Verification
Every variant generated, installed, tested and built:
FastMCP is correctly untouched: no
testscript, no vitest dependency, no test exclusion in its tsconfig.npm testin this repo — 422 tests across 16 files (up from 410)npm run lintclean; all changed files passprettier --check.dockerignorealready excludes*.test.ts, so tests never enter the image contextNot verified: the Docker image build. The Docker CLI is installed here but the engine isn't running, so I couldn't execute
docker build. The.dockerignoreexclusion and the cleandist/make it very likely fine, but it's untested and worth a check if you have an engine running.Docs
AGENTS.mdand the repo README still described thegreetexample and the pre-split single-file server, so both are refreshed.AGENTS.mdalso gains a section on the generated test setup covering the three gotchas above, plus the rule that the emitted suite stays small and exemplary — adding a test should mean retiring one.FastMCP still has no tests and keeps its own example. Worth a tracked follow-up so the gap doesn't go quiet.