feat: complete unit + integration test-suite generator (EP-TESTGEN #153)#159
Open
ilijevicdenis wants to merge 4 commits into
Open
feat: complete unit + integration test-suite generator (EP-TESTGEN #153)#159ilijevicdenis wants to merge 4 commits into
ilijevicdenis wants to merge 4 commits into
Conversation
…153) EP-TESTGEN — extends the single-object `test scaffold` (#139) into a complete, auto-asserted suite for the whole project plus a DB-backed workflow. Core: - SuiteScaffolder walks the built model → one .test.sql per (object, category): NOT NULL/UNIQUE/PK negatives (@expect-sqlstate 23502/23505), FK orphan (23503), CRUD round-trip, view/matview queryability, fn/proc/trigger stubs (via TestScaffolder), and catalog existence smoke tests for the other ObjectKinds. - BaselineRowSynthesizer builds a minimal valid INSERT (deterministic literal per TypeNormalizer spelling; skips identity/serial/generated/default/nullable; seeds depth-1 FK parents with OVERRIDING SYSTEM VALUE), downgrading to an inconclusive stub rather than emit a false assertion (enum/UDT column, non-trivial CHECK, FK outside the model). Deterministic, sentinel-tagged output. CLI: `pgproj test generate <proj> [--connection] [--mode preserved|wipeout] [--allow-wipeout] [--categories] [-o] [--run]`. preserved brings the DB up to date via PublishService (data kept); wipeout DROP+recreates then greenfield-deploys. Regeneration replaces only its own sentinel-bearing files (hand-promoted tests survive). VS: classic-extension "Generate Tests (PgProj)…" project-context command shells the bundled CLI; UiTests gains a registration scenario. Verified: SuiteScaffolderTests 12/12; live SuiteScaffolderIntegrationTests on PG18 (constraint/FK/CRUD/view/existence PASS, enum/CHECK INCONCLUSIVE, 0 fail); full Core suite 25,204/0 with live PG18; CLI e2e wipeout --run 12 pass/1 inc/0 fail, idempotent regen + no-clobber proven; HyperV VS UI E2E 4/4 in the installed VS 2026. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ision) Found by running the full generated suite against the rich extracted sample-db (not just an inline-constraint toy schema): - Function/trigger unit stubs ran a `<expected-value>` placeholder assert (and tried to call trigger-returning functions directly), so they FAILED instead of being pending work. They now raise pgproj_inconclusive FIRST; the TestScaffolder template stays below as authoring guidance (dead code after the raise). - ObjectKind.Comment carries no schema/name, so every comment mapped to _gen._._.exists.test.sql and silently overwrote the others. Comments (and any nameless object) are now skipped; added a DisambiguateFileNames safety net so two objects can never silently share a file. Tests: two offline cases for the above. SuiteScaffolder suite 15/15 (incl. the live round-trip). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…e model `pgproj extract` (and many real projects) emit FK/PK/UNIQUE/CHECK as standalone ALTER TABLE ADD CONSTRAINT statements, not inline. The parser kept only the verb "ADD CONSTRAINT" and ModelBuilder had no AlterStatement case, so those constraints never reached TableDefinition — the comparer/deploy AND the #153 test generator were blind to them (no FK tests; CRUD/PK tests failed on a clean deploy because the hidden FK parent was never seeded). - Parser: capture ParseTableConstraint's result into AlterStatement.AddedConstraints (no accept/reject change — it already parsed and discarded the constraint). - ModelBuilder: new AlterStatement case folds the captured constraints into the target table, reusing AddTable's logic via a shared ApplyTableConstraint. Same-file CREATE-before-ALTER (what extract emits) means the table is present when applied. - TableDesigner: drop its now-redundant manual FK fold (its table already comes through ModelBuilder) — skip any ALTER it folded. Without this the FK double-counted (183 tests failed with Assert.Single -> 2 items). Removed the dead FK-reparse helpers. Verified: full generated suite against the rich EXTRACTED sample-db now 0-fail on a clean deploy (FK tests generated; CRUD/PK seed parents); full Core suite 25,208/0 with live PG18. New offline tests cover FK/PK/UNIQUE/CHECK folding. Lab note added. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
11 tasks
Update the delivery-log row to the as-shipped state: the 3 commits + PR #159, the ALTER-TABLE-ADD-CONSTRAINT model fold and the 3 defects found by verifying against the rich extracted sample-db, final numbers (Core 25,208/0; rich suite 0-fail clean deploy; VS UI E2E 4/4), documented limitations, and the spun-off deploy-ordering issue #160. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #153 (and children #154–#158).
What
A whole-project, auto-asserted unit + integration test-suite generator for
.pgproj, plus the model fix that makes it correct for real (extracted) projects.Core
SuiteScaffolderwalks the built model → one.test.sqlper (object, category): NOT NULL/UNIQUE/PK negatives (@expect-sqlstate23502/23505), FK orphan (23503), CRUD round-trip, view/matview queryability, fn/proc/trigger stubs, and catalog existence smoke tests for the otherObjectKinds.BaselineRowSynthesizerbuilds a minimal valid INSERT (deterministic literal perTypeNormalizerspelling; skips identity/serial/generated/default/nullable; seeds depth-1 FK parents withOVERRIDING SYSTEM VALUE), downgrading to an inconclusive stub rather than emitting a false assertion (enum/UDT column, non-trivial CHECK, FK outside the model).CLI
pgproj test generate <proj> [--connection] [--mode preserved|wipeout] [--allow-wipeout] [--categories] [-o] [--run]— preserved brings the DB up to date viaPublishService(data kept); wipeout DROP+recreates + greenfield-deploys;--runexecutes the suite.VS
Classic-extension "Generate Tests (PgProj)…" project-context command (shells the bundled CLI).
Model fix (found by testing the complete feature)
pgproj extractemits FKs as standaloneALTER TABLE ADD CONSTRAINT. The parser discarded the detail andModelBuilderignored it, so the comparer/deploy and the generator were blind to those FKs. This PR captures the parsed constraint and folds FK/PK/UNIQUE/CHECK into the table model (reconciling the now-redundantTableDesignerfold).Verification
--run): 12 pass / 1 inconclusive / 0 fail; idempotent regen + no-clobber proven.SuiteScaffolderTests(15),ConstraintAndColumnTestsALTER-fold cases; liveSuiteScaffolderIntegrationTestsround-trip.Known limitations (documented, not silently shipped)
ALTER(a different file than itsCREATE TABLE) is best-effort; same-file (whatextractemits) always works.42883: a view/matview depending on a function is deployed before it) blocks--mode wipeouton function-heavy extracted projects — candidate for its own issue.🤖 Generated with Claude Code