Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 31 additions & 2 deletions docs/docs/usage/dbt-pr-review.md
Original file line number Diff line number Diff line change
Expand Up @@ -199,6 +199,34 @@ jobs:
# model_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
```

### What the review needs (and what each input unlocks)

| Input | How to produce it | Unlocks |
|---|---|---|
| Full git history of the PR's target branch | `actions/checkout` with `fetch-depth: 0` | Correct base: the diff starts at the merge-base with the PR's target branch. A target branch that is not fetched is an error, never a silent fallback to `main`. |
| `target/manifest.json` | `dbt compile` (or `dbt parse` for lint + lineage only) | Blast radius, lineage breakage, tiering. Without it the run is lint-only. |
| `target/compiled/…` | `dbt compile` | Engine checks on the head (lint, grade, PII, grain). |
| `target-base/compiled/…` | Compile the base ref into `target-base/` (below) | Equivalence proofs and column-breakage for modified models. |
| `target/catalog.json` | `dbt docs generate` | Complete warehouse columns for column-lineage breakage. |

Compiling the base (optional, for equivalence and column-breakage):

```yaml
- name: Compile the base into target-base/
run: |
git worktree add ../base "origin/${{ github.base_ref }}"
(cd ../base && dbt deps && dbt compile --target-path "$GITHUB_WORKSPACE/target-base")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: In a monorepo the base-compile example writes target-base/ at the repo root and runs dbt from the repo root, but the reviewer reads target-base/ relative to project_dir. Equivalence and column-breakage checks would silently fall back to not verified. Note that the base compile must target <project_dir>/target-base and run inside the project folder, or the Monorepos paragraph should say so.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At docs/docs/usage/dbt-pr-review.md, line 218:

<comment>In a monorepo the base-compile example writes `target-base/` at the repo root and runs `dbt` from the repo root, but the reviewer reads `target-base/` relative to `project_dir`. Equivalence and column-breakage checks would silently fall back to not verified. Note that the base compile must target `<project_dir>/target-base` and run inside the project folder, or the Monorepos paragraph should say so.</comment>

<file context>
@@ -199,6 +199,34 @@ jobs:
+      - name: Compile the base into target-base/
+        run: |
+          git worktree add ../base "origin/${{ github.base_ref }}"
+          (cd ../base && dbt deps && dbt compile --target-path "$GITHUB_WORKSPACE/target-base")
+```
+
</file context>

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: This snippet does not work in the monorepo layout the paragraph below documents.

It writes target-base/ to the repository root and runs dbt compile from the worktree root. The resolver reads <dbt root>/target-base/compiled/<project>/… (compiled.ts:117, with opts.cwd = dbtRoot from run.ts:363), and dbt compile needs dbt_project.yml, so with project_dir: dbt both steps fail or land where the review never looks — equivalence and column-breakage silently stay off. Add a monorepo variant, e.g. (cd ../base/dbt && dbt deps && dbt compile --target-path "$GITHUB_WORKSPACE/dbt/target-base"), and note the path must sit next to the project's manifest.json.

```

**Monorepos:** set `project_dir` to the folder holding `dbt_project.yml`. Only
changes inside it are reviewed, `manifest_path` is resolved relative to it, and
changes elsewhere are counted in the comment.

**What the comment tells you it could not check:** findings the engine could not
decide, models without compiled SQL, files outside the project, and whether the
advisory AI lane ran, was skipped, timed out or failed (with the reason). The
same facts are in the envelope (`scope`, `ai`) and the `review_run` telemetry.

### Model & credentials for the advisory lane

The deterministic engine (lineage, equivalence, PII, grade, lint — the only layer
Expand Down Expand Up @@ -356,8 +384,9 @@ reviewer does **not** re-implement Jinja — it consumes dbt's own compiled outp
from `target/compiled/<project>/…` (HEAD) and `target-base/compiled/…` (BASE),
produced by `dbt compile`. To enable full equivalence verdicts, compile both
the base and head refs in CI (the base into `target-base/`, the Recce
convention). Without compiled SQL the engine lanes fall back to raw and stay
*undecidable* (never fabricated) — the `dbt-patterns` lane still runs.
convention). A model without compiled SQL is never sent to the engine as raw
Jinja: it is listed as **not verified** (with the reason) in the comment and in
the envelope's `scope.unverified`, and the `dbt-patterns` lane still runs.

## What it checks — deterministic rule catalog

Expand Down
8 changes: 7 additions & 1 deletion github/review/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,12 @@ inputs:
description: "comment (never blocks) | gate (blocks the merge on REQUEST_CHANGES)"
required: false
default: "comment"
project_dir:
description: "Directory of the dbt project (the one holding dbt_project.yml), relative to the repository root. Only changes inside it are reviewed — set it in a monorepo. `manifest_path` is resolved relative to it."
required: false
default: "."
manifest_path:
description: "Path to the compiled dbt manifest.json. Run `dbt compile` before this step for the full (non-degraded) verdict."
description: "Path to the compiled dbt manifest.json, relative to `project_dir`. Run `dbt compile` before this step for the full (non-degraded) verdict; `dbt docs generate` (catalog.json) and a base compile into `target-base/` enable column-lineage breakage and equivalence proofs."
required: false
default: "target/manifest.json"
severity_threshold:
Expand Down Expand Up @@ -170,6 +174,7 @@ runs:
# from `${{ }}` would be a shell-injection vector. Build an arg array instead.
env:
IN_MODE: ${{ inputs.mode }}
IN_PROJECT_DIR: ${{ inputs.project_dir }}
IN_MANIFEST: ${{ inputs.manifest_path }}
IN_SEVERITY: ${{ inputs.severity_threshold }}
IN_BASE: ${{ inputs.base }}
Expand All @@ -181,6 +186,7 @@ runs:
ALTIMATE_REVIEW_SIGNING_KEY: ${{ inputs.signing_key }}
run: |
args=(--mode "$IN_MODE" --manifest "$IN_MANIFEST" --severity "$IN_SEVERITY")
[[ -n "$IN_PROJECT_DIR" && "$IN_PROJECT_DIR" != "." ]] && args+=(--cwd "$IN_PROJECT_DIR")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: A misspelled project_dir silently broadens a monorepo review to every changed file. Validate that the directory contains dbt_project.yml/.yaml before adding --cwd, or make the review fail closed when project discovery fails.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At github/review/action.yml, line 189:

<comment>A misspelled `project_dir` silently broadens a monorepo review to every changed file. Validate that the directory contains `dbt_project.yml`/`.yaml` before adding `--cwd`, or make the review fail closed when project discovery fails.</comment>

<file context>
@@ -181,6 +186,7 @@ runs:
         ALTIMATE_REVIEW_SIGNING_KEY: ${{ inputs.signing_key }}
       run: |
         args=(--mode "$IN_MODE" --manifest "$IN_MANIFEST" --severity "$IN_SEVERITY")
+        [[ -n "$IN_PROJECT_DIR" && "$IN_PROJECT_DIR" != "." ]] && args+=(--cwd "$IN_PROJECT_DIR")
         [[ -n "$IN_BASE" ]] && args+=(--base "$IN_BASE")
         [[ -n "$IN_HEAD" ]] && args+=(--head "$IN_HEAD")
</file context>
Suggested change
[[ -n "$IN_PROJECT_DIR" && "$IN_PROJECT_DIR" != "." ]] && args+=(--cwd "$IN_PROJECT_DIR")
if [[ -n "$IN_PROJECT_DIR" && "$IN_PROJECT_DIR" != "." ]]; then
project_path="${GITHUB_WORKSPACE:-$PWD}/$IN_PROJECT_DIR"
if [[ ! -f "$project_path/dbt_project.yml" && ! -f "$project_path/dbt_project.yaml" ]]; then
echo "::error::project_dir must contain dbt_project.yml or dbt_project.yaml"
exit 1
fi
args+=(--cwd "$IN_PROJECT_DIR")
fi

[[ -n "$IN_BASE" ]] && args+=(--base "$IN_BASE")
[[ -n "$IN_HEAD" ]] && args+=(--head "$IN_HEAD")
[[ "$IN_POST" == "true" ]] && args+=(--post)
Expand Down
82 changes: 68 additions & 14 deletions packages/opencode/src/altimate/review/ai-review.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,8 @@ import { readStreamText } from "../llm/stream-text"

const log = Log.create({ service: "ai-review" })

const AI_TIMEOUT_MS = 60_000
const AI_TIMEOUT_BASE_MS = 60_000
const AI_TIMEOUT_MAX_MS = 300_000
const MAX_DIFF_CHARS = 6_000 // per file, keep the prompt bounded
const MAX_FILES = 20

Expand All @@ -31,6 +32,39 @@ export interface AiReviewInput {
grounding: Finding[]
prTitle?: string
prBody?: string
/** Explicit `provider/model`; omit to use the configured default model. */
model?: string
}

/** What happened in the advisory lane, reported on the envelope and in the comment. */
export type AiLaneState = "ok" | "skipped" | "timeout" | "error"
export interface AiLaneStatus {
state: AiLaneState
/** Why the lane was skipped or failed (never contains credentials). */
reason?: string
/** `provider/model` the lane ran with. */
model?: string
durationMs?: number
}
export interface AiReviewResult {
findings: Finding[]
status: AiLaneStatus
}

/** Timeout scaled to the prompt: 60 s plus 1 s per 2,000 characters, capped at 5 minutes. */
export function aiTimeoutMs(promptChars: number): number {
return Math.min(AI_TIMEOUT_MAX_MS, AI_TIMEOUT_BASE_MS + Math.ceil(promptChars / 2_000) * 1_000)
}

/** Error text safe to show in a PR comment: one line, bounded, with anything
* that looks like a key or bearer token masked. */
function safeReason(err: unknown): string {
const raw = err instanceof Error ? err.message : String(err)
return raw
.replace(/\s+/g, " ")
.replace(/(bearer\s+)[^\s]+/gi, "$1***")
.replace(/\b(sk|pk|key|token)[-_][A-Za-z0-9_-]{8,}/gi, "***")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: safeReason still exposes credential formats that do not use the four dashed prefixes, so provider failures can disclose secrets in PR comments. Reuse the repository's credential/error redactor or mask credential-labeled values and credential-bearing URLs before truncating the reason.

(Based on your team's feedback about masking credential-stem error values.) [a5c5a10f-6a4a-4e42-8c2e-f8e01b48683d]

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/ai-review.ts, line 66:

<comment>`safeReason` still exposes credential formats that do not use the four dashed prefixes, so provider failures can disclose secrets in PR comments. Reuse the repository's credential/error redactor or mask credential-labeled values and credential-bearing URLs before truncating the reason.

(Based on your team's feedback about masking credential-stem error values.) [a5c5a10f-6a4a-4e42-8c2e-f8e01b48683d]</comment>

<file context>
@@ -31,6 +32,39 @@ export interface AiReviewInput {
+  return raw
+    .replace(/\s+/g, " ")
+    .replace(/(bearer\s+)[^\s]+/gi, "$1***")
+    .replace(/\b(sk|pk|key|token)[-_][A-Za-z0-9_-]{8,}/gi, "***")
+    .slice(0, 200)
 }
</file context>

.slice(0, 200)
}
Comment on lines +61 to 68

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[🟠 MEDIUM] While this effectively sanitizes bearer tokens and typical API key patterns, it may not cover other sensitive credentials occasionally found in network errors, such as Basic <base64> auth headers or credentials embedded within URLs (e.g., https://user:pass@host/...).

Consider expanding the regex to cover these patterns for more robust redaction.

Suggested change:

Suggested change
function safeReason(err: unknown): string {
const raw = err instanceof Error ? err.message : String(err)
return raw
.replace(/\s+/g, " ")
.replace(/(bearer\s+)[^\s]+/gi, "$1***")
.replace(/\b(sk|pk|key|token)[-_][A-Za-z0-9_-]{8,}/gi, "***")
.slice(0, 200)
}
function safeReason(err: unknown): string {
const raw = err instanceof Error ? err.message : String(err)
return raw
.replace(/\s+/g, " ")
.replace(/(bearer\s+|basic\s+)[^\s]+/gi, "$1***")
.replace(/:\/\/[^/]*@[^/]+/gi, "://***@")
.replace(/\b(sk|pk|key|token)[-_][A-Za-z0-9_-]{8,}/gi, "***")
.slice(0, 200)
}


/**
Expand Down Expand Up @@ -73,23 +107,39 @@ function buildUserMessage(input: AiReviewInput): string {

/**
* Run the LLM reviewer lane. Returns advisory findings (severity ≤ warning,
* clamped by core), or [] if no model / core is available or the call fails —
* a review must never crash because the AI layer is unavailable.
* clamped by core) and a status saying whether the lane ran, was skipped, timed
* out or failed. Never throws — a review must not crash because the AI layer is
* unavailable — but never fails silently either: the status reaches the
* envelope and the PR comment.
*/
export async function runAiReview(input: AiReviewInput): Promise<Finding[]> {
export async function runAiReview(input: AiReviewInput): Promise<AiReviewResult> {
const startedAt = Date.now()
const done = (findings: Finding[], status: Omit<AiLaneStatus, "durationMs">): AiReviewResult => ({
findings,
status: { ...status, durationMs: Date.now() - startedAt },
})
const files = input.files.filter((f) => f.status !== "deleted" && (f.diff || f.sql))
if (!files.length) return []
if (!files.length) return done([], { state: "skipped", reason: "no changed model SQL to review" })

const message = buildUserMessage({ ...input, files })
const controller = new AbortController()
const timeout = setTimeout(() => controller.abort(), AI_TIMEOUT_MS)
const timeoutMs = aiTimeoutMs(message.length)
const timeout = setTimeout(() => controller.abort(), timeoutMs)
let modelLabel: string | undefined
try {
Comment on lines +124 to 129

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[🔴 HIGH] Moving buildUserMessage outside the try...catch block introduces the risk of unhandled exceptions. If buildUserMessage throws an error (e.g., due to an unexpected value or formatting error), it will crash the function, violating the JSDoc contract that "a review must not crash because the AI layer is unavailable".

Consider moving the message generation and timeout setup inside the try block to ensure all potential errors are caught and a proper failure status is returned.

Suggested change:

Suggested change
const message = buildUserMessage({ ...input, files })
const controller = new AbortController()
const timeout = setTimeout(() => controller.abort(), AI_TIMEOUT_MS)
const timeoutMs = aiTimeoutMs(message.length)
const timeout = setTimeout(() => controller.abort(), timeoutMs)
let modelLabel: string | undefined
try {
const controller = new AbortController()
let timeout: ReturnType<typeof setTimeout> | undefined
let timeoutMs = 0
let modelLabel: string | undefined
try {
const message = buildUserMessage({ ...input, files })
timeoutMs = aiTimeoutMs(message.length)
timeout = setTimeout(() => controller.abort(), timeoutMs)

// Prompt comes from the compiled core, not this file.
const promptRes = await Dispatcher.call("altimate_core.review_ai_prompt", {})
const system = ((promptRes.data ?? {}) as Record<string, unknown>).prompt as string | undefined
if (!system) return []

const defaultModel = await Provider.defaultModel()
const model = await Provider.getModel(defaultModel.providerID, defaultModel.modelID)
if (!system) return done([], { state: "error", reason: "the core reviewer prompt is unavailable" })

let model: Awaited<ReturnType<typeof Provider.getModel>>
try {
const chosen = input.model ? Provider.parseModel(input.model) : await Provider.defaultModel()
model = await Provider.getModel(chosen.providerID, chosen.modelID)
} catch (err) {
return done([], { state: "skipped", reason: `no usable model: ${safeReason(err)}`, model: input.model })
}
modelLabel = `${model.providerID}/${model.id}`

const agent: Agent.Info = {
name: "dbt-ai-reviewer",
Expand Down Expand Up @@ -119,10 +169,12 @@ export async function runAiReview(input: AiReviewInput): Promise<Finding[]> {
abort: controller.signal,
sessionID: user.sessionID,
retries: 1,
messages: [{ role: "user", content: buildUserMessage({ ...input, files }) }],
messages: [{ role: "user", content: message }],
})
const text = await readStreamText(stream)
if (!text) return []
if (controller.signal.aborted)
return done([], { state: "timeout", reason: `no answer within ${Math.round(timeoutMs / 1000)} s`, model: modelLabel })
if (!text) return done([], { state: "error", reason: "the model returned no text", model: modelLabel })

// Parse + clamp in core (the prompt-injection-resistant, advisory-only
// contract). Returns already-validated, severity-clamped, file-checked items.
Expand Down Expand Up @@ -165,10 +217,12 @@ export async function runAiReview(input: AiReviewInput): Promise<Finding[]> {
}
}
log.info("ai review complete", { findings: out.length })
return out
return done(out, { state: "ok", model: modelLabel })
} catch (err) {
log.error("ai review failed", { error: err })
return []
if (controller.signal.aborted)
return done([], { state: "timeout", reason: `no answer within ${Math.round(timeoutMs / 1000)} s`, model: modelLabel })
return done([], { state: "error", reason: safeReason(err), model: modelLabel })
} finally {
clearTimeout(timeout)
}
Expand Down
33 changes: 32 additions & 1 deletion packages/opencode/src/altimate/review/format.ts
Original file line number Diff line number Diff line change
Expand Up @@ -42,11 +42,12 @@ export function renderSummary(env: VerdictEnvelope): string {

if (env.summary.degraded) {
lines.push(
"> ⚙️ **Lint-only run** — no dbt manifest/warehouse was available, so lineage, equivalence and",
"> ⚙️ **Lint-only run** — no changed model resolved against a dbt manifest, so lineage, equivalence and",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: The banner now asserts a cause that is only one of the two ways summary.degraded becomes true.

orchestrate.ts:1120 sets runDegraded = modelFiles.length > 0 ? !anyManifest : reviewable.length === 0, so an empty reviewable set also renders this banner. That happens for a PR whose only changed files are non-dbt (.md, .json, … are dropped by filterChangedFiles → shouldReview), and there the advice "Wire manifest_path" cannot change anything. The comment then misstates why checks were skipped, which is the class of problem this PR is fixing. Branch the wording on the actual cause, or give the empty-reviewable case its own note.

"> data-impact checks were skipped. Wire `manifest_path` (and optionally warehouse creds) for the full verdict.",
"",
)
}
lines.push(...coverageNotes(env))

// G1 (Round 18) — surface classifier reasons when --explain-tier populated
// them, so a customer can see why the review ran at this tier. Also surfaces
Expand Down Expand Up @@ -110,11 +111,41 @@ export function renderSummary(env: VerdictEnvelope): string {
`<sub>altimate dbt-pr-review · verdict \`${env.verdict}\`` +
(env.signature ? ` · signed \`${env.signature.slice(0, 18)}…\`` : "") +
(env.manifestHash ? ` · manifest \`${env.manifestHash.slice(0, 10)}\`` : "") +
(env.scope?.baseRef ? ` · base \`${env.scope.baseRef}\`` : "") +
(env.ai?.state === "ok" && env.ai.model ? ` · AI \`${env.ai.model}\`` : "") +
"</sub>",
)
return lines.join("\n")
}

/**
* What the review could not fully check, each on its own line: undecidable
* findings, models without compiled SQL, files outside the dbt project, and
* the advisory AI lane when it did not run. Kept separate from the lint-only
* banner so one undecidable finding never mislabels a manifest-backed run.
*/
export function coverageNotes(env: VerdictEnvelope): string[] {
const notes: string[] = []
const undecided = env.findings.filter((f) => f.degraded).length
if (undecided) notes.push(`${undecided} finding(s) could not be decided by the engine and are marked _unverified_.`)
const unverified = env.scope?.unverified ?? []
if (unverified.length) {
const head = unverified.slice(0, 5).map((u) => `\`${u.path}\``).join(", ")
const more = unverified.length > 5 ? ` and ${unverified.length - 5} more` : ""
notes.push(
`${unverified.length} model(s) not verified by the engine — ${unverified[0].reason}: ${head}${more}. ` +

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Mixed head/base compile failures are rendered with the first model’s reason for every model, which misstates review coverage. Group the entries by reason or render each model with its own reason.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/format.ts, line 136:

<comment>Mixed head/base compile failures are rendered with the first model’s reason for every model, which misstates review coverage. Group the entries by reason or render each model with its own reason.</comment>

<file context>
@@ -110,11 +111,41 @@ export function renderSummary(env: VerdictEnvelope): string {
+    const head = unverified.slice(0, 5).map((u) => `\`${u.path}\``).join(", ")
+    const more = unverified.length > 5 ? ` and ${unverified.length - 5} more` : ""
+    notes.push(
+      `${unverified.length} model(s) not verified by the engine — ${unverified[0].reason}: ${head}${more}. ` +
+        "Run `dbt compile` (and compile the base into `target-base/`) for equivalence and lineage checks.",
+    )
</file context>

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: Only the first model's reason is reported for the whole set.

unverified entries carry two distinct reasons (orchestrate.ts:1108 head-side vs 1110 base-side), so a PR that mixes gaps gets every path labelled with unverified[0].reason. With unverified[0] a base-side entry, the line even tells the user to run dbt compile for models that were only missing target-base. Group by reason or use a neutral phrase ("no compiled SQL for the head or base").

"Run `dbt compile` (and compile the base into `target-base/`) for equivalence and lineage checks.",
)
}
if (env.scope?.outOfScopeFiles)
notes.push(`${env.scope.outOfScopeFiles} changed file(s) outside the dbt project were not reviewed.`)
if (env.ai && env.ai.state !== "ok") {
const what = { skipped: "skipped", timeout: "timed out", error: "failed" }[env.ai.state]
notes.push(`Advisory AI review ${what}${env.ai.reason ? `: ${env.ai.reason}` : ""}.`)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: The PR comment does not report the AI lane’s available model and duration: successful runs show only the model, while failed or timed-out runs show only state and reason. Include model and duration in the rendered AI status.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/format.ts, line 144:

<comment>The PR comment does not report the AI lane’s available model and duration: successful runs show only the model, while failed or timed-out runs show only state and reason. Include model and duration in the rendered AI status.</comment>

<file context>
@@ -110,11 +111,41 @@ export function renderSummary(env: VerdictEnvelope): string {
+    notes.push(`${env.scope.outOfScopeFiles} changed file(s) outside the dbt project were not reviewed.`)
+  if (env.ai && env.ai.state !== "ok") {
+    const what = { skipped: "skipped", timeout: "timed out", error: "failed" }[env.ai.state]
+    notes.push(`Advisory AI review ${what}${env.ai.reason ? `: ${env.ai.reason}` : ""}.`)
+  }
+  return notes.length ? [...notes.map((n) => `> ℹ️ ${n}`), ""] : []
</file context>

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: env.ai.reason is interpolated into a > ℹ️ blockquote verbatim.

safeReason in ai-review.ts collapses whitespace, but the orchestrate catch path does not (see the comment there), so a multi-line error message breaks out of the blockquote and can inject arbitrary markdown into the posted PR comment. The footer at lines 114-115 has the same shape: env.scope.baseRef and env.ai.model are placed in single-backtick spans without the fence-length logic this file already applies to tierReasons (lines 64-74, added for exactly this reason). Reuse that escaping for these values.

}
return notes.length ? [...notes.map((n) => `> ℹ️ ${n}`), ""] : []
}

/** GitHub `pulls.createReview` inline comments — only findings with a line. */
export interface InlineComment {
path: string
Expand Down
31 changes: 24 additions & 7 deletions packages/opencode/src/altimate/review/orchestrate.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ import { type Rubric, exclusionReason, clampSeverity } from "./rubric"
import { type ReviewConfig } from "./config"
import { type ReviewMode, type ReviewScope, type VerdictEnvelope, buildEnvelope, signEnvelope } from "./verdict"
import { detectModelPatterns, detectSchemaYmlPatterns, splitDiff } from "./dbt-patterns"
import { type AiReviewInput } from "./ai-review"
import { type AiLaneStatus, type AiReviewInput, type AiReviewResult } from "./ai-review"

/**
* The deterministic review recipe.
Expand Down Expand Up @@ -188,7 +188,11 @@ export interface OrchestrateInput {
* tests pass a fake or omit it (the lane is then skipped). Receives the
* deterministic findings as grounding and returns ADVISORY findings only.
*/
aiReview?: (input: AiReviewInput) => Promise<Finding[]>
aiReview?: (input: AiReviewInput) => Promise<AiReviewResult | Finding[]>
/** Why the AI lane is off when `aiReview` is not injected (shown in the comment). */
aiDisabledReason?: string
/** Explicit `provider/model` for the AI lane. */
aiModel?: string
/** PR metadata passed to the AI reviewer for intent checking. */
prTitle?: string
prBody?: string
Expand Down Expand Up @@ -1410,7 +1414,12 @@ export async function runReview(input: OrchestrateInput): Promise<VerdictEnvelop
// findings are clamped to ≤ warning upstream, so they enrich the review but
// never change what blocks. Skipped when no aiReview fn is injected (tests /
// headless-without-model) — the review degrades to deterministic-only.
if (lanes.has("ai_review") && input.aiReview) {
let aiStatus: AiLaneStatus | undefined
if (!lanes.has("ai_review")) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Empty-scope reviews now emit an AI skipped status and coverage note even though there is no review scope or AI lane to skip. Guard the AI-status block on a non-empty review scope so empty reviews leave ai undefined.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/orchestrate.ts, line 1418:

<comment>Empty-scope reviews now emit an AI `skipped` status and coverage note even though there is no review scope or AI lane to skip. Guard the AI-status block on a non-empty review scope so empty reviews leave `ai` undefined.</comment>

<file context>
@@ -1410,7 +1414,12 @@ export async function runReview(input: OrchestrateInput): Promise<VerdictEnvelop
   // headless-without-model) — the review degrades to deterministic-only.
-  if (lanes.has("ai_review") && input.aiReview) {
+  let aiStatus: AiLaneStatus | undefined
+  if (!lanes.has("ai_review")) {
+    aiStatus = { state: "skipped", reason: `not part of the ${tier} tier` }
+  } else if (!input.aiReview) {
</file context>

aiStatus = { state: "skipped", reason: `not part of the ${tier} tier` }

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: This reason can be false.

lanes is input.config.reviewers.length ? input.config.reviewers : TIER_LANES[tier] (line 1175), so the branch also fires when a user restricts lanes with reviewers: in .altimate/review.yml. A full-tier run with reviewers: ["sql_quality"] then reports "not part of the full tier" in the PR comment, which is a misstatement of the same kind the PR set out to remove. Distinguish "excluded by the reviewers config" from "not part of the tier".

} else if (!input.aiReview) {
aiStatus = { state: "skipped", reason: input.aiDisabledReason ?? "disabled" }
} else {
const aiFiles = [...ctxByPath.values()].map((ctx) => ({
path: ctx.file.path,
status: ctx.file.status,
Expand All @@ -1419,16 +1428,20 @@ export async function runReview(input: OrchestrateInput): Promise<VerdictEnvelop
sql: ctx.engineNewSql ?? ctx.newSql,
}))
try {
const aiFindings = await input.aiReview({
const res = await input.aiReview({
files: aiFiles,
grounding: merged,
prTitle: input.prTitle,
prBody: input.prBody,
model: input.aiModel,
})
const aiFindings = Array.isArray(res) ? res : res.findings
aiStatus = Array.isArray(res) ? { state: "ok" } : res.status
// Defense in depth: the AI never blocks, regardless of what it returns.
for (const f of aiFindings) merged.push(f.severity === "critical" ? { ...f, severity: "warning" } : f)
} catch {
// A failed AI lane must never fail the review.
} catch (err) {
// A failed AI lane must never fail the review — but it must say so.
aiStatus = { state: "error", reason: String((err as Error)?.message ?? err).slice(0, 200) }

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: AI callback failures bypass the redaction promised by AiLaneStatus and can publish raw error text in the signed envelope and PR comment. Apply the same credential-safe error formatter before storing ai.reason.

(Based on your team's feedback about masking credential-bearing error text.) [173edeff-f795-42ca-9d7d-86524f63e58a]

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/opencode/src/altimate/review/orchestrate.ts, line 1444:

<comment>AI callback failures bypass the redaction promised by `AiLaneStatus` and can publish raw error text in the signed envelope and PR comment. Apply the same credential-safe error formatter before storing `ai.reason`. 

(Based on your team's feedback about masking credential-bearing error text.) [173edeff-f795-42ca-9d7d-86524f63e58a]</comment>

<file context>
@@ -1419,16 +1428,20 @@ export async function runReview(input: OrchestrateInput): Promise<VerdictEnvelop
-      // A failed AI lane must never fail the review.
+    } catch (err) {
+      // A failed AI lane must never fail the review — but it must say so.
+      aiStatus = { state: "error", reason: String((err as Error)?.message ?? err).slice(0, 200) }
     }
   }
</file context>

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: This reason reaches the signed envelope and the PR comment unmasked.

Unlike safeReason in ai-review.ts, it is not whitespace-collapsed, not credential-masked, and only sliced to 200 chars — while AiLaneStatus.reason is documented as "never contains credentials". runAiReview does not throw today, so this is the fallback for any injected lane, but the contract should hold on every path: reuse the masking helper (or sanitize the message here) rather than embedding the raw error.

}
}

Expand All @@ -1440,8 +1453,12 @@ export async function runReview(input: OrchestrateInput): Promise<VerdictEnvelop
// Sort by severity desc, then file.
findings.sort((a, b) => SEVERITY_ORDER[b.severity] - SEVERITY_ORDER[a.severity] || a.file.localeCompare(b.file))

const degraded = runDegraded || findings.some((f) => f.degraded)
// Lint-only means no changed model resolved against a manifest. A single
// undecidable finding does not make the whole run lint-only; those findings
// carry their own `degraded` flag and are counted separately in the summary.
const degraded = runDegraded
const envelope = buildEnvelope({
ai: aiStatus,
findings,
tier,
mode: input.mode,
Expand Down
30 changes: 30 additions & 0 deletions packages/opencode/src/altimate/review/run.ts
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,8 @@ export interface ReviewPullRequestOptions {
/** PR metadata for the AI reviewer's intent check. */
prTitle?: string
prBody?: string
/** Explicit `provider/model` for the advisory AI lane. */
aiModel?: string
/** G1 — surface the tier classifier's reasons on the verdict envelope. */
explainTier?: boolean
/** G2 — [EXPERIMENTAL] bypass the tier classifier with the supplied tier.
Expand Down Expand Up @@ -379,6 +381,8 @@ export async function reviewPullRequest(opts: ReviewPullRequestOptions): Promise
// path verdicts (cubic-review PR #1041).
cliVersion: opts.cliVersion ?? Installation.VERSION,
aiReview: opts.noAi || config.ai === false ? undefined : runAiReview,
aiDisabledReason: opts.noAi ? "disabled by --no-ai" : config.ai === false ? "disabled by `ai: false` in .altimate/review.yml" : undefined,
aiModel: opts.aiModel,
Comment on lines 382 to +385

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[🔵 LOW] The checklist prohibits the use of nested ternary expressions. Please refactor this to avoid nesting, perhaps by computing the disabled reason in a separate variable before using it here.

Suggested change:

Suggested change
cliVersion: opts.cliVersion ?? Installation.VERSION,
aiReview: opts.noAi || config.ai === false ? undefined : runAiReview,
aiDisabledReason: opts.noAi ? "disabled by --no-ai" : config.ai === false ? "disabled by `ai: false` in .altimate/review.yml" : undefined,
aiModel: opts.aiModel,
cliVersion: opts.cliVersion ?? Installation.VERSION,
aiReview: opts.noAi || config.ai === false ? undefined : runAiReview,
aiDisabledReason: (function() {
if (opts.noAi) return "disabled by --no-ai"
if (config.ai === false) return "disabled by `ai: false` in .altimate/review.yml"
return undefined
})(),
aiModel: opts.aiModel,

prTitle: opts.prTitle,
prBody: opts.prBody,
explainTier: opts.explainTier,
Expand All @@ -396,6 +400,32 @@ export async function reviewPullRequest(opts: ReviewPullRequestOptions): Promise
})
}

/**
* The pull/merge request's title and description from the CI environment, so
* the AI lane can check the change against its stated intent. GitHub: the
* `pull_request` object in GITHUB_EVENT_PATH. GitLab: CI_MERGE_REQUEST_TITLE /
* CI_MERGE_REQUEST_DESCRIPTION. Empty outside a PR/MR run.
*/
export async function prContextFromEnv(env: NodeJS.ProcessEnv = process.env): Promise<{ prTitle?: string; prBody?: string }> {
if (env.GITHUB_EVENT_PATH) {
try {
const pr = JSON.parse(await readFile(env.GITHUB_EVENT_PATH, "utf8"))?.pull_request
if (pr) {
return {
prTitle: typeof pr.title === "string" && pr.title ? pr.title : undefined,
prBody: typeof pr.body === "string" && pr.body ? pr.body : undefined,
}
}
} catch {
/* unreadable event file: try GitLab */
}
}
return {
prTitle: env.CI_MERGE_REQUEST_TITLE || undefined,
prBody: env.CI_MERGE_REQUEST_DESCRIPTION || undefined,
}
}

/**
* The dbt project being reviewed: the nearest directory at or above `cwd`
* (not above the repo root) holding dbt_project.yml; else the project that
Expand Down
6 changes: 6 additions & 0 deletions packages/opencode/src/altimate/review/telemetry.ts
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,12 @@ export function emitReviewRun(input: {
// Optional in the schema and explicitly invalid as `false`, so normalise rather than copy.
tier_forced: env.tierForced === true,
degraded: env.summary.degraded,
undecided: env.findings.filter((f) => f.degraded).length,
unverified_models: env.scope?.unverified?.length ?? 0,
ai_state: env.ai?.state,
ai_duration_ms: env.ai?.durationMs,
base_source: env.scope?.baseSource,
out_of_scope_files: env.scope?.outOfScopeFiles ?? 0,
stale_manifest: env.staleManifest === true,
critical: env.summary.critical,
warning: env.summary.warning,
Expand Down
Loading
Loading