Skip to content

feat(review): close remaining review gaps: action dbt_prepare, data-test adequacy, parse-only column changes - #1726

Draft
RaghwendraSingh wants to merge 3 commits into
feat/review-incremental-semanticsfrom
feat/review-ac-gaps
Draft

RaghwendraSingh wants to merge 3 commits into
feat/review-incremental-semanticsfrom
feat/review-ac-gaps

Conversation

@RaghwendraSingh

@RaghwendraSingh RaghwendraSingh commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Issue for this PR

Closes #

Stacked on #1725 (← #1724 ← #1723 ← … ← #1719); merge in that order. Draft.

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

This closes the remaining acceptance-criteria gaps in the review stack. Two of the fixes came from replaying the PRs behind known escaped defects.

  • Optional artifact preparation in the action (dbt_prepare). Default none, so nothing changes for existing users.
    • compile runs dbt compile on the head, then compiles the PR's merge-base into target-base/ in a temporary worktree. A shallow checkout is fetched and deepened when needed.
    • full also runs dbt docs generate.
    • A failed base compile is a warning, and those models are marked unverified.
    • Inputs reach the script only through env.
    • The manual docs snippet now compiles the merge-base instead of the base branch's tip.
    • The example workflow gets a generic name.
  • Data tests that can't do their job (yml only, any adapter):
    • a test comparing a model with itself (equality / equal_rowcount against its own ref, or a relationships test to its own column) passes by construction;
    • new tests that inherit severity: warn or an error_if threshold from dbt_project.yml are reported;
    • a dbt_project.yml change that loosens those thresholds for a folder is reported.
  • Column-set rules work in dbt parse-only CI. Without compiled SQL, output columns come from the raw model's final SELECT; a Jinja expression in the list makes it unknown. In a parse-only replay, on_schema_change='fail' with new columns now fires on the two real escapes it targets.
  • Dedup windows split across lines are read whole. "Dedup has no ORDER BY" no longer fires falsely when order by is on its own line, and the load-timestamp rule sees such windows.

How did you verify your code works?

  • New tests:
    • review-action.test.ts: dbt_prepare is opt-in, reads inputs only from env, and builds into target-base/.
    • review-test-adequacy.test.ts: self-comparison, inherited warn/error_if, loosened thresholds, dbt 1.8+ arguments:.
    • Additions to review-incremental-semantics.test.ts: multi-line windows, raw-SQL column delta.
  • Suites: 506 review tests and 1,367 skill and release-gate tests pass, bun typecheck is clean, and the real-world corpus is unchanged (15/15 caught, 0/5 false positives).
  • dbt_prepare run locally against a real dbt 1.10 DuckDB project. The head compiled into target/, the merge-base compiled into target-base/ (the base still has the removed column), and the temporary worktree was cleaned up.
  • Replay of the PRs behind six known escapes in a private dbt monorepo (parse-only, as their CI runs):
    • on_schema_change now catches two of them (two escapes each).
    • One escape lived in Python loader code, outside dbt.
    • One targeted the removed microbatch rule.
    • One needs the opt-in slice var.
    • One is pre-existing code the PR didn't touch, so no rule reports it by design.
  • The earlier 7-PR replay is unchanged: 0 findings on files the PR didn't change.

Cost side effects

dbt_prepare: compile adds one dbt compile per side of the PR in CI, and full adds dbt docs generate (warehouse queries for the catalog). Both are opt-in; the default none adds nothing. No model calls.

Screenshots / recordings

N/A (CLI / action).

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR
  • I checked for cost side effects (model calls, queries or cloud resources that grow with user data) and described them, or wrote "none"

🤖 Generated with Claude Code


Summary by cubic

Closes the remaining acceptance-criteria gaps in the dbt review stack: adds an opt-in dbt_prepare action input for building the artifacts the full review needs, and fixes two classes of missed findings — data tests that can't do their job, and column-set rules that skipped work in parse-only CI.

New dbt_prepare action input

  • Default none; compile runs dbt compile on the head and the PR's merge-base into target-base/, and full also runs dbt docs generate.
  • Inputs reach the script only through env; a failed base compile is a warning that marks affected models unverified.
  • Docs now compile the merge-base instead of the base branch's tip, and the example workflow gets a generic name.

Review gap fixes

  • Reports data tests that pass by construction (self-comparisons with equality/equal_rowcount/relationships) or that inherit severity: warn or an error_if threshold from dbt_project.yml, plus dbt_project.yml changes that loosen those thresholds.
  • Without compiled SQL, output columns come from the raw model's final SELECT, so on_schema_change='fail' fires in parse-only CI; a Jinja expression in the list leaves it unknown.
  • Dedup windows are read from the whole file, so a split ORDER BY no longer triggers a false "no ORDER BY" and the load-timestamp rule sees such windows.

Written for commit 7da6850. Summary will update on new commits.

View guided diff Turn on auto-fix

RaghwendraSingh and others added 3 commits November 13, 2026 13:23
…erge-base and catalog artifacts

- new action inputs: `dbt_prepare` (`none` default | `compile` | `full`), `dbt_command`, `dbt_profiles_dir`
- `compile` runs `dbt compile` on the head and compiles the PR's merge-base into `target-base/` in a temporary worktree (fetching and deepening a shallow checkout when needed); `full` adds `dbt docs generate`
- a base compile that fails is a warning: the review still runs and marks those models unverified
- inputs reach the script only through env, never interpolated
- docs: the manual base-compile snippet now compiles the merge-base, not the base branch's tip
- the example workflow is renamed `examples/dbt-pr-review.yml` (was named after one internal repo)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ows split across lines are read whole

Found by replaying the PRs behind known escaped defects:

- without compiled SQL, output columns come from the raw model's final SELECT (Jinja blanked; a Jinja expression in the list makes it unknown). `on_schema_change='fail'` with new columns now fires on the two real escapes it targets, in a parse-only CI setup
- `row_number() over (…)` windows are read from the whole file, so an `ORDER BY` on its own line is seen: the "dedup has no ORDER BY" rule no longer fires falsely on multi-line windows, and the load-timestamp rule sees them. A window counts only when an added line falls inside it

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nherited warn/thresholds, loosened project thresholds

- a test added or changed in the PR that compares a model with itself (`dbt_utils.equality`/`equal_rowcount`/… against its own `ref`, or a `relationships` test to its own column) passes by construction (warning)
- new tests that inherit `severity: warn` or an `error_if` threshold from `dbt_project.yml` (walked from the top through the project name and folders, deeper wins) are reported once per yml (suggestion)
- a `dbt_project.yml` change that turns a folder's tests to `warn` or sets an `error_if` threshold is reported (warning)
- reads dbt 1.8+ `arguments:` too; all findings are catalog origin (never block)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Nov 13, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@github-actions

Copy link
Copy Markdown
- - - - - - - - - - - - - - - - - - - - - - - - -
                    AIRECEIPTS                    
             1 session behind this PR             

claude-opus-5-5 100% · &lt;s…...221,036,213 tokens
  SUBAGENTS (19).................63,001,251 tokens
--------------------------------------------------
TOTAL unpriced................≥ 284,037,464 tokens
  counted: 1 session + 19 subagents
  cache served 97% of input tokens

1 candidate session not attributed
(in repo + branch window, no branch commit)
  full receipts + session ids: section below
- - - - - - - - - - - - - - - - - - - - - - - - -
                npx aireceipts-cli                
         github.com/anandgupta42/receipts         
- - - - - - - - - - - - - - - - - - - - - - - - -
full receipts (1 session)
session id scope turns time tokens in / out cached
orchestrator 6fe8be4d entire session (slice unavailable) 700 189h 52m 1.6k / 696k 98%

orchestrator · 6fe8be4d

- - - - - - - - - - - - - - - - - - - - - - - - -
                    AIRECEIPTS                    
         “data-eng-code-review-landscape”         
  Claude Code · Nov 05 2026 10:10 UTC · 189h 52m  
      claude-opus-5-5 100% · <synthetic> 0%       
         cache served 98% of input tokens         

pre-edit: 1% of tokens (12/698 turns)
  (share before the first named edit tool)

Bash..................164,495,680 tok  (495 calls)
(thinking/reply)........19,821,384 tok  (67 turns)
Write....................8,521,992 tok  (31 calls)
ToolSearch...............5,032,526 tok  (25 calls)
Read.....................4,036,974 tok  (29 calls)
mcp__claude_ai_Atlass…...3,292,204 tok  (28 calls)
Edit.....................3,095,292 tok  (28 calls)
mcp__claude_ai_Atlass…...2,143,864 tok  (12 calls)
mcp__claude_ai_Atlass…...1,725,016 tok  (17 calls)
Agent....................1,704,960 tok  (22 calls)
mcp__claude_ai_Atlassian…...947,323 tok  (7 calls)
ExitPlanMode................758,820 tok  (4 calls)
mcp__notion__notion-upda…...607,506 tok  (4 calls)
mcp__claude_ai_Atlassian…...489,926 tok  (5 calls)
mcp__claude_ai_Atlassia…...489,318 tok  (13 calls)
mcp__claude_ai_Atlassian_…...485,105 tok  (1 call)
mcp__claude_ai_Atlassia…...479,160 tok  (17 calls)
mcp__atlassian__createJi…...419,313 tok  (2 calls)
WebSearch...................384,482 tok  (4 calls)
AskUserQuestion.............366,689 tok  (2 calls)
SendMessage.................358,309 tok  (7 calls)
mcp__notion__notion-fetch...327,408 tok  (2 calls)
mcp__notion__notion-creat…...248,897 tok  (1 call)
mcp__atlassian__searchJir…...222,754 tok  (1 call)
WebFetch.....................217,960 tok  (1 call)
mcp__notion__notion-search...106,809 tok  (1 call)
mcp__atlassian__atlassianU…...94,821 tok  (1 call)
mcp__atlassian__getVisible…...94,821 tok  (1 call)
TaskStop......................66,901 tok  (1 call)
--------------------------------------------------
TOTAL..............................221,036,213 tok
no price table matched
- - - - - - - - - - - - - - - - - - - - - - - - -
                npx aireceipts-cli                
         github.com/anandgupta42/receipts         
- - - - - - - - - - - - - - - - - - - - - - - - -
subagents (19)
subagent cost
Goal: an evidence-backed, cited landscape of (a) dbt PR review tools, (b) data-… 2,134,743 tokens
Goal: determine what gaps altimate-code's dbt PR review has, by comparing what… 5,303,299 tokens
Label dbt PR review comments by reading each one. Instructions: read /private/t… 2,794,842 tokens
Label dbt PR review comments by reading each one. Instructions: read /private/t… 3,244,430 tokens
Label escaped-defect candidates by READING them (no keyword bucketing). Directo… 2,570,116 tokens
Repo: /Users/raghwendrasingh/code/altimateai/altimate-code (current branch feat… 4,836,719 tokens
You are doing an ADVERSARIAL review of a set of Jira tickets that plan improvem… 763,761 tokens
Label dbt PR review comments by reading each one. Instructions: read /private/t… 3,351,198 tokens
Label escaped-defect candidates by READING them (no keyword bucketing). Directo… 4,214,327 tokens
Label dbt PR review comments by reading each one. Instructions: read /private/t… 2,821,134 tokens
Label dbt PR review comments by reading each one. Instructions: read /private/t… 1,795,004 tokens
You are doing an ADVERSARIAL review of a stack of changes to an open-source dbt… 4,084,798 tokens
Repo: /Users/raghwendrasingh/code/altimateai/altimate-code. Module: packages/op… 8,815,483 tokens
Label dbt PR review comments by reading each one. Instructions: read /private/t… 2,592,769 tokens
Research task, read-only (do not push, comment, or open PRs anywhere).Goal: dec… 1,198,872 tokens
Judge each of altimate-code's own inline PR review comments by reading it. Inpu… 1,673,212 tokens
Label escaped-defect candidates by READING them (no keyword bucketing). Directo… 2,862,492 tokens
Goal: read a Notion doc tree about an "agentic data engineering software factor… 1,335,636 tokens
Goal: test whether the bug classes that real reviewers catch on dbt PRs at one… 6,608,416 tokens

Generated by aireceipts

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant