Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 29 additions & 3 deletions docs/docs/usage/dbt-pr-review.md
Original file line number Diff line number Diff line change
Expand Up @@ -209,12 +209,30 @@ jobs:
| `target-base/compiled/…` | Compile the base ref into `target-base/` (below) | Equivalence proofs and column-breakage for modified models. |
| `target/catalog.json` | `dbt docs generate` | Complete warehouse columns for column-lineage breakage. |

Compiling the base (optional, for equivalence and column-breakage):
The action can build these for you with `dbt_prepare` (off by default):

```yaml
- uses: AltimateAI/altimate-code/github/review@main
with:
dbt_prepare: compile # none | compile | full
dbt_command: dbt # or "uv run dbt", "poetry run dbt"
dbt_profiles_dir: . # relative to project_dir
```

- **`compile`:** runs `dbt compile` on the head, then compiles the PR's merge-base
into `target-base/` in a temporary worktree. This enables equivalence and
column-breakage proofs. It usually works offline.
- **`full`:** also runs `dbt docs generate` for `catalog.json` (complete column
types). That needs warehouse credentials in the profile.
- A base compile that fails is a warning: the review still runs, and the
affected models are listed as not verified.

To build them yourself, compile the **merge-base**, not the base branch's tip:

```yaml
- name: Compile the base into target-base/
run: |
git worktree add ../base "origin/${{ github.base_ref }}"
git worktree add ../base "$(git merge-base origin/${{ github.base_ref }} HEAD)"
(cd ../base && dbt deps && dbt compile --target-path "$GITHUB_WORKSPACE/target-base")
```

Expand Down Expand Up @@ -242,7 +260,7 @@ ways, or neither (it self-disables, leaving a deterministic-only review):
Always pass keys as repo **secrets**. Warehouse credentials are consumed by the
`dbt compile` step (via your `profiles.yml`), **not** by the review step. A
complete, copy-paste workflow lives at
[`github/review/examples/altimate-ingestion.yml`](https://github.com/AltimateAI/altimate-code/blob/main/github/review/examples/altimate-ingestion.yml).
[`github/review/examples/dbt-pr-review.yml`](https://github.com/AltimateAI/altimate-code/blob/main/github/review/examples/dbt-pr-review.yml).

Re-pushing commits updates the same summary comment in place; fixed findings are
dropped on the next run. `--post` targets **GitHub** PRs (it reads `GITHUB_TOKEN` /
Expand Down Expand Up @@ -487,6 +505,14 @@ The deterministic rules group into four families:
- **dbt unit tests.** A unit test whose `given` inputs are all `rows: []` is
reported: it either never tests the model's logic, or expects rows the model has nothing to
build from.
- **Data tests that can't do their job.** These are checked on tests added or
changed in the PR:
- a test that compares a model with itself (`dbt_utils.equality` or
`equal_rowcount` against its own `ref`, or a `relationships` test pointing
at its own column) passes by construction;
- new tests that inherit `severity: warn` or an `error_if` threshold from
`dbt_project.yml` report failures without failing the build;
- a `dbt_project.yml` change that loosens those thresholds for a whole folder.
- **Incremental and snapshot semantics.** Config is read from the manifest
(project, yml and inline, as dbt merges them) with the head's inline
`{{ config() }}` on top. Formatting-only edits never count as a change.
Expand Down
59 changes: 59 additions & 0 deletions github/review/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,17 @@ inputs:
description: "Post the verdict to the PR (summary comment + inline review)."
required: false
default: "true"
dbt_prepare:
description: "Build the dbt artifacts the full review uses before reviewing. `none` (default): you build them in earlier steps. `compile`: `dbt compile` on the head, and on the PR's merge-base into `target-base/` (equivalence and lineage proofs). `full`: also `dbt docs generate` for `catalog.json` (complete column types). `compile` usually works offline; `full` needs warehouse credentials in the profile."
required: false
default: "none"
dbt_command:
description: "How to invoke dbt for `dbt_prepare`, e.g. `dbt`, `uv run dbt`, `poetry run dbt`."
required: false
default: "dbt"
dbt_profiles_dir:
description: "`--profiles-dir` for `dbt_prepare`, relative to `project_dir`. Empty = dbt's default lookup."
required: false
signing_key:
description: "Optional HMAC key to sign the verdict envelope (ALTIMATE_REVIEW_SIGNING_KEY)."
required: false
Expand Down Expand Up @@ -167,6 +178,54 @@ runs:
echo "Advisory lane: no model/credentials provided — running deterministic-only."
fi

# Optional: build the head and base artifacts the full-tier review needs.
# Inputs are passed as ENV, never interpolated into the script.
- name: Prepare dbt artifacts
if: inputs.dbt_prepare != 'none'
shell: bash
env:
IN_PREPARE: ${{ inputs.dbt_prepare }}
IN_DBT: ${{ inputs.dbt_command }}
IN_PROFILES: ${{ inputs.dbt_profiles_dir }}
IN_PROJECT_DIR: ${{ inputs.project_dir }}
IN_BASE: ${{ inputs.base }}
EVENT_BASE: ${{ github.event.pull_request.base.ref }}
run: |
set -euo pipefail
case "$IN_PREPARE" in compile|full) ;; *) echo "::error::dbt_prepare must be none, compile or full (got '$IN_PREPARE')"; exit 1 ;; esac
read -ra DBT <<< "$IN_DBT"
PROJECT="$(cd "${IN_PROJECT_DIR:-.}" && pwd)"
REL="$(git -C "$PROJECT" rev-parse --show-prefix)"
PROFILES=()
[[ -n "${IN_PROFILES:-}" ]] && PROFILES=(--profiles-dir "$(cd "$PROJECT" && cd "$IN_PROFILES" && pwd)")
build() { # $1 = project dir, $2 = target path
( cd "$1"
[[ -f packages.yml || -f dependencies.yml ]] && "${DBT[@]}" deps ${PROFILES[@]+"${PROFILES[@]}"}
"${DBT[@]}" compile --target-path "$2" ${PROFILES[@]+"${PROFILES[@]}"} )
}
echo "Compiling the head…"
build "$PROJECT" "$PROJECT/target"
if [[ "$IN_PREPARE" == "full" ]]; then
echo "Generating catalog.json…"
( cd "$PROJECT" && "${DBT[@]}" docs generate --target-path "$PROJECT/target" ${PROFILES[@]+"${PROFILES[@]}"} )
fi
# The base side: the PR's merge-base, compiled into target-base/.
BASE="${IN_BASE:-$EVENT_BASE}"
BASE="${BASE#origin/}"
if [[ -z "$BASE" ]]; then echo "::warning::No base branch known; skipping the base compile."; exit 0; fi
git fetch --no-tags origin "+refs/heads/$BASE:refs/remotes/origin/$BASE" >/dev/null 2>&1 || true
MB="$(git merge-base "origin/$BASE" HEAD 2>/dev/null || git merge-base "$BASE" HEAD 2>/dev/null || true)"
if [[ -z "$MB" && "$(git rev-parse --is-shallow-repository)" == "true" ]]; then
git fetch --no-tags --deepen=1000 origin "+refs/heads/$BASE:refs/remotes/origin/$BASE" "$(git rev-parse HEAD)" >/dev/null 2>&1 || true
MB="$(git merge-base "origin/$BASE" HEAD 2>/dev/null || true)"
fi
if [[ -z "$MB" ]]; then echo "::warning::Could not find the merge-base with $BASE; skipping the base compile."; exit 0; fi
WT="$RUNNER_TEMP/altimate-review-base"
git worktree add --detach "$WT" "$MB" >/dev/null
trap 'git worktree remove --force "$WT" >/dev/null 2>&1 || true' EXIT
echo "Compiling the base ($BASE @ ${MB:0:12})…"
build "$WT/$REL" "$PROJECT/target-base" || echo "::warning::The base compile failed; equivalence will be marked unverified."

- name: Run dbt PR review
shell: bash
# Inputs are passed as ENV (data), never interpolated into the command —
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Example: drop-in dbt PR review for a dbt repo (e.g. altimate-ingestion).
# Example: drop-in dbt PR review for a dbt repo.
# Copy to the target repo as `.github/workflows/dbt-pr-review.yml` and adjust the
# dbt adapter + profiles/secrets to match the warehouse.
#
Expand Down
44 changes: 34 additions & 10 deletions packages/opencode/src/altimate/review/dbt-patterns.ts
Original file line number Diff line number Diff line change
Expand Up @@ -225,15 +225,42 @@ const detectNotIn: Detector = (c) => {
})
}

/**
* `row_number() over (…)` windows in the model, read from the whole file so a
* window split across lines is seen whole. A window counts as part of the
* change when one of the PR's added lines falls inside it.
*/
function dedupWindows(c: Ctx): Array<{ spec: string; text: string }> {
const sql = stripComments(c.newSql)
const added = c.added.map((l) => stripComments(l).trim()).filter((l) => l.length >= 3)
const out: Array<{ spec: string; text: string }> = []
for (const m of sql.matchAll(/row_number\s*\(\s*\)\s*over\s*\(/gi)) {
const open = m.index! + m[0].length - 1
let depth = 0
let close = -1
for (let i = open; i < sql.length; i++) {
if (sql[i] === "(") depth++
else if (sql[i] === ")" && --depth === 0) {
close = i
break
}
}
if (close < 0) continue
const text = sql.slice(m.index!, close + 1)
if (!added.some((l) => text.includes(l) || l.includes(text.replace(/\s+/g, " ")))) continue
out.push({ spec: sql.slice(open + 1, close).replace(/\s+/g, " ").trim(), text: text.replace(/\s+/g, " ") })
}
return out
}

// 9. Dedup via ROW_NUMBER()/QUALIFY whose ORDER BY has no unique tiebreaker.
const detectDedupTie: Detector = (c) => {
const hit = c.added.find((l) => /row_number\s*\(\s*\)\s*over\s*\(/i.test(stripComments(l)))
if (!hit) return null
// Only flag dedup with NO ORDER BY — that is unambiguously non-deterministic.
// A present ORDER BY (even single-column) is the developer's deterministic
// choice and the dbt-recommended pattern; warning "it might not be unique"
// on every dedup is noise (false positives on the common, correct case).
if (/order\s+by/i.test(stripComments(hit))) return null
const hit = dedupWindows(c).find((w) => !/order\s+by/i.test(w.spec))
if (!hit) return null
return makeFinding({
severity: "warning",
category: "dedup",
Expand All @@ -244,7 +271,7 @@ const detectDedupTie: Detector = (c) => {
file: c.file.path,
model: c.model,
confidence: "high",
evidence: { tool: "dbt-patterns", result: { rule: "dedup_no_tiebreaker", line: hit.trim() } },
evidence: { tool: "dbt-patterns", result: { rule: "dedup_no_tiebreaker", line: hit.text } },
ruleKey: "dedup:no-tiebreaker",
})
}
Expand All @@ -257,11 +284,8 @@ const detectDedupTie: Detector = (c) => {
const LOAD_TS_RE =
/^(_?(ingest(ion|ed)?|load(ed)?|etl|extract(ed)?|batch)(_at|_ts|_time|_timestamp|_date|_dt)|_?(ingestion|load|etl|batch)_timestamp|_loaded_at)$/i
const detectDedupLoadTimestamp: Detector = (c) => {
for (const line of c.added) {
const s = stripComments(line)
const m = /row_number\s*\(\s*\)\s*over\s*\(([^)]*)\)/i.exec(s)
if (!m) continue
const order = /order\s+by\s+(.+)$/i.exec(m[1])?.[1]
for (const w of dedupWindows(c)) {
const order = /order\s+by\s+(.+)$/i.exec(w.spec)?.[1]
if (!order) continue
const keys = order.split(",").map((k) => k.trim().replace(/\s+(asc|desc)\b.*$/i, "").replace(/\s+nulls\s+(first|last)$/i, ""))
if (keys.length !== 1) continue
Expand All @@ -277,7 +301,7 @@ const detectDedupLoadTimestamp: Detector = (c) => {
file: c.file.path,
model: c.model,
confidence: "high",
evidence: { tool: "dbt-patterns", result: { rule: "dedup_load_timestamp_only", line: line.trim() } },
evidence: { tool: "dbt-patterns", result: { rule: "dedup_load_timestamp_only", line: w.text } },
ruleKey: "dedup:load-timestamp-only",
})
}
Expand Down
57 changes: 45 additions & 12 deletions packages/opencode/src/altimate/review/orchestrate.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ import { detectModelPatterns, detectSchemaYmlPatterns, splitDiff } from "./dbt-p
import { listMirrorsModel, type ConsumerColumnList } from "./consumer-lists"
import { detectUnitTestFixtures } from "./unit-tests"
import { selectListNames } from "./select-list"
import { detectTestAdequacy, detectThresholdLoosening } from "./test-adequacy"
import { incrementalFindings, snapshotFindings, snapshotYmlFindings } from "./incremental-semantics"
import { type AiLaneStatus, type AiReviewInput, type AiReviewResult } from "./ai-review"

Expand Down Expand Up @@ -648,19 +649,41 @@ async function columnDelta(
dialect: string,
): Promise<{ added: string[]; removed: string[]; current: Set<string> }> {
const none = { added: [], removed: [], current: new Set<string>() }
const { file, engineNewSql, engineOldSql } = ctx
if (!engineNewSql || file.status === "deleted") return none
const current = await outputColumns(runner, engineNewSql, dialect)
if (!current.size) return none
if (!hasOldSide(file)) return { added: [...current], removed: [], current }
if (!engineOldSql) return none
const before = await outputColumns(runner, engineOldSql, dialect)
if (!before.size) return none
return {
added: [...current].filter((c) => !before.has(c)),
removed: [...before].filter((c) => !current.has(c)),
current,
const { file, engineNewSql, engineOldSql, newSql, oldSql } = ctx
if (file.status === "deleted") return none
const withOld = hasOldSide(file)
const delta = (current: Set<string>, before?: Set<string>) =>
!current.size || (withOld && !before?.size)
? none
: {
added: [...current].filter((c) => !before?.has(c)),
removed: before ? [...before].filter((c) => !current.has(c)) : [],
current,
}
// Compiled SQL on both sides: column lineage plus the final SELECT's names.
if (engineNewSql && (!withOld || engineOldSql)) {
const current = await outputColumns(runner, engineNewSql, dialect)
const before = withOld ? await outputColumns(runner, engineOldSql!, dialect) : undefined
return delta(current, before)
}
// No compiled SQL (CI that only runs `dbt parse`): the raw model's own SELECT
// list, when it names its columns. Columns produced inside a macro stay unknown.
if (!newSql) return none
const current = rawSelectList(newSql)
const before = withOld && oldSql ? rawSelectList(oldSql) : undefined
return delta(current, before)
}

/** Output names of a raw (Jinja) model's final SELECT; empty when a Jinja expression is in the list. */
function rawSelectList(sql: string): Set<string> {
const plain = sql
.replace(/\{#[\s\S]*?#\}/g, " ")
.replace(/\{%[\s\S]*?%\}/g, " ")
.replace(/\{\{[\s\S]*?\}\}/g, " __jinja__ ")
const names = selectListNames(plain)
if (!names || names.includes("__jinja__")) return new Set()
// A Jinja expression inside an item (e.g. `{{ col }} as x`) leaves its alias intact.
return new Set(names)
}

const ticks = (xs: string[]) => xs.map((x) => `\`${x}\``).join(", ")
Expand Down Expand Up @@ -1643,6 +1666,9 @@ export async function runReview(input: OrchestrateInput): Promise<VerdictEnvelop
// Fetches run in parallel across schema files (a schema-heavy PR could touch
// dozens of yml files; serial `git show` per file adds up).
const schemaFiles = reviewable.filter((f) => f.kind === "schema_yml")
// The head's dbt_project.yml (test thresholds inherited by new tests), read once.
const projectYmlPath = `${input.layout?.root ? `${input.layout.root.replace(/\/$/, "")}/` : ""}dbt_project.yml`
const projectYml = schemaFiles.length ? await getContent?.(projectYmlPath, "new").catch(() => undefined) : undefined
if (schemaFiles.length) {
const schemaFindingSets = await Promise.all(
schemaFiles.map(async (file) => {
Expand All @@ -1655,11 +1681,18 @@ export async function runReview(input: OrchestrateInput): Promise<VerdictEnvelop
...detectSchemaYmlPatterns(file, input.rubric, { oldContent, newContent }),
...detectUnitTestFixtures(file, oldContent, newContent),
...snapshotYmlFindings(file, oldContent, newContent),
...detectTestAdequacy({ file, oldContent, newContent, projectYml, layout: input.layout }),
]
}),
)
for (const findings of schemaFindingSets) all.push(findings)
}
// A dbt_project.yml change that loosens test thresholds for a whole path.
for (const file of reviewable.filter((f) => f.kind === "project_config" && /(^|\/)dbt_project\.ya?ml$/.test(f.path))) {
if (file.status !== "modified") continue
const [oldContent, newContent] = await Promise.all([getContent?.(file.path, "old"), getContent?.(file.path, "new")])
all.push(detectThresholdLoosening(file, oldContent, newContent))
}

// Architectural dedup: the regex `dbt-patterns`/`rule-catalog` layer is a
// diff-aware, Jinja-aware, offline fallback. For the STRUCTURAL SQL checks
Expand Down
Loading
Loading