Conversation
Measurement only — zero selection/gating behavior changes. Each replayed point now records the Selection facts it already computed (widened flag, skippable count, rebuild_forced_by_nonresolution, resolution reasons) and the report aggregates them: widen-rate %, median skippable models, forced rebuild total, and the ranked widening/forcing-reason breakdown. Rendered as one Selection line (+reasons) in the table/markdown output and as a typed selection_stats block in JSON; omitted (None) when no point carried a selection so an absent surface is never read as a 0% rate. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Under on_missing_meta/on_error fail_closed, an UNKNOWN leaf only fires a BLOCKING rule; a warn rule silently ALLOWs, so an all-warn pilot policy systematically understates what block mode will do (issue #124, option 2). Record each suppressed firing as an UnprovenPolicyHit (rule, subject, unknown cause, unproven leaf conditions) carried on PolicyVerdict as unproven/unproven_count. Surfaced in the JSON verdict, and as an additive 'N unproven warn-rule conditions' section (list folded) in the Markdown report / sticky PR comment. Report-only: decision, hits, and exit codes are byte-identical to before. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add defaults.warn_rules_fire_on_unknown (bool, default OFF): when set, fail_closed applies to non-blocking rules too, so a warn rule whose predicate stays UNKNOWN fires at its declared severity as a normal warn hit marked fired_on_unknown, instead of landing in the unproven telemetry. It never escalates severity (warn stays warn, never block), so --fail-on policy exit behavior is untouched; default-off behavior is byte-identical apart from the unproven telemetry. Option 1 of issue #124, behind the knob the issue proposed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ature The model-level logic gate compared compiled SQL after a regex comment strip (strip_sql_comments), which is not string-literal-aware: a -- or /* inside a quoted string swallowed the real tokens around it, so a genuine change (e.g. amount * 1 -> amount * 100 on the same line as a '-- in a string') compared equal and was silently reported SAFE. Gate on the already-sound, lru-cached comment_free_token_signature instead — the tokenizer preserves string contents, so equal signatures prove a comment/whitespace-only edit and anything unprovable (untokenizable side) is treated as changed (fail-safe). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A MODEL whose compiled SQL was unavailable on either side silently reported 'no logic change' — the edit was invisible and the model would not even rebuild itself. The model-level logic diff is now three-state (LogicDiffStatus in models/schema.py): sources/seeds/snapshots keep the legitimate no-op, but a model node with missing SQL emits fail-safe LOGIC_CHANGED/INDETERMINATE changes for every head column (a '*' sentinel when the model is column-less) carrying logic_diff_status='indeterminate'. The marker flows into the JSON report, into a new confidence.indeterminate_logic(_models) block that degrades the impact confidence to 'partial', and — through the existing widen branch — empties skippable_models so nothing downstream of an un-diffable model is skipped. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…catalogs
structural_diff_available() only checks catalog PRESENCE, so with one prod
catalog.json mounted on both diff sides (the real deployment shape) the
structural checks nominally ran while being structurally blind: identical
catalogs can never show a removed/retyped column, provable_break_count is
stuck at 0, --fail-on tests can never fire, and nothing warned.
detect_structural_degradation(base, head) now compares catalog content
fingerprints (sha256 over canonical JSON — byte-formatting-insensitive) and
the head catalog/manifest generated_at + invocation_id stamps: identical
catalogs, or a head catalog predating the head manifest from a different dbt
invocation, stamp the report with structural_diff {status: degraded, reason}
plus summary.structural_diff, and the markdown/PR-comment renderer prints a
visible honesty line. Advisory only — exit codes and verdicts unchanged; the
key is absent when nothing is provable (stub providers make no claim).
Also adds report_version: 1 and parrant_version to the JSON report top level
so machine consumers can pin the shape they parse.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
build_selection folded reach from RESOLVED changes only: a change whose impact fan-out raised (unresolved) contributed nothing to breaking_reached, so its unknown downstream cone landed in skippable_models — an unprovable state classifying models safe to skip. Any unresolved change in by_change now forces the widen branch (rebuild = whole reachable universe, skippable empty), exactly like non-full confidence or a truncated list. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
get_model_upstream handled model/source/snapshot dep prefixes but not seed: a model ref()ing a seed recorded no upstream edge, so the seed's consumers were invisible to reachability — and therefore to impact and the rebuild selection — whenever a seed changed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The rebuild selector selects models, and dbt only runs tests attached to selected models: a relationships test attached to a skipped model that references a rebuilt one will not run in a selective CI build. Document the blind spot honestly on the selective-builds section, with concrete mitigations (--indirect-selection=buildable, selector+1, scheduled full dbt test). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ed to models --scope-git intersected the two-manifest changeset with models mapped from changed *.sql files only. A diff whose vehicle is not a model .sql file — a macro, a Python model, dbt_project.yml vars, packages.yml, a seed .csv — was silently dropped from the scope set, emptying the changeset into a SAFE verdict and skipping every CI rebuild (fail-open). resolve_git_scope now classifies the FULL git diff: model files map via resource_path (.py models and seeds included), macro files map via the backend's macro-dependents capability when available, and any remaining logic-bearing file is UNMAPPABLE — scoping is then disabled for the run (full changeset kept) with the reason surfaced on stderr, in the report source line, and as the pydantic-shaped report["scope_git"] block. Docs/property-only files (schema.yml, *.md) cannot change compiled SQL and still narrow legitimately. Exit-code semantics unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…-git ManifestReader.get_macro_dependents builds macro-file -> dependent-node map from manifest macros[*].original_file_path, nodes[*].depends_on.macros and the transitive (cycle-safe) macros[*].depends_on.macros graph; ModelRegistry exposes it as a cached capability method. resolve_git_scope picks it up via its capability lookup, so a changed macro file scopes the changeset to the models it (transitively) feeds instead of disabling scoping outright. An unknown macro file still falls back to the fail-safe floor. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes a set of fail-open gaps in the offline impact gate — cases where a genuinely-affected change could be classified
SAFEor skipped from a selective build with no visible warning. Every fix moves an unprovable state to widen / degrade / surface, never to a silent pass, and keeps the gate's exit codes unchanged.What was broken and fixed
--or/* */inside a quoted string literal made two different compiled SQLs compare equal, so a real value change was reportedSAFE. The gate now uses the already-sound, string-literal-preserving token signature; anything untokenizable is treated as changed (fail-safe).partialand widens the rebuild selection.catalog.jsonmounted on both diff sides (a common CI shape), removed/retyped columns are invisible and--fail-on testscan never fire — with no warning. The report now stampsstructural_diff: degraded(with a reason and a visible honesty line) when catalogs are content-identical or the head catalog predates the head manifest. Advisory only; verdicts/exit codes unchanged.skippable_models. Any unresolved change now forces the widen branch.get_model_upstreamignoredseed.*deps, so a model built off a dbt seed had no upstream edge and a seed change reached nothing. Fixed.--scope-gitsilently dropped non-model changes. A macro / Python-model /dbt_project.yml/ seed change mapped to no model.sql, so scoping emptied the changeset to aSAFEverdict and skipped every rebuild. Scoping now classifies the full diff: known macro files scope to their (transitive) dependent models, and any remaining logic-bearing file that can't be mapped disables scoping for the run with the reason surfaced on stderr, in the report source, and in a typedscope_gitreport block.Machine-output changes (additive)
New/extended fields on the JSON report and pydantic schema:
logic_diff_status,confidence.indeterminate_logic(_models),structural_diff {status, reason},scope_gitblock, policy verdictunproven/unproven_count, backtestselection_stats, and a top-levelreport_version+parrant_versionstamp. A new defaultwarn_rules_fire_on_unknown(off) lets warn rules fire on an unknown leaf; default-off behavior is byte-identical apart from the new telemetry.Testing
Every fix is covered by a test that fails without it. Full suite green on this branch: 669 unit + 116 integration + 31 e2e. No new type-check errors versus the current baseline (the pre-existing mypy errors are addressed separately in the quality-gate PR).
🤖 Generated with Claude Code