diff --git a/PRIVACY.md b/PRIVACY.md index b35aadc..0fba74f 100644 --- a/PRIVACY.md +++ b/PRIVACY.md @@ -78,6 +78,18 @@ Turn off recording and drafting with `altimate learn disable` in the project, or The delegation path below runs only when the delegation skill or the `/altimate-code:altimate` command fires and invokes the locally installed altimate-code CLI. +## The lineage report + +The column impact report is one HTML file that the plugin writes to the project's `target/.altimate-lineage/` and opens in your browser. The page loads nothing from the network: its content-security policy allows only its own inline script and style. + +To learn whether the report gets used, the plugin asks the local altimate-code CLI (`altimate-code tool report-event`) to send one usage event, `lineage_report`, at each of these steps: +- the summary was shown; +- **Open report** was pressed; +- `/lineage-report` was run; +- the report was built, failed to build, or found no browser. + +Each event carries the step and the Claude Code session id (`claude:`) only: no model, column, file or project names. It goes through Altimate's usage telemetry, described in [Usage telemetry](#usage-telemetry), and that section's opt-out turns it off. With an altimate-code that predates the command, nothing is sent. + ## Data flows when a task is delegated | Data | Where it goes | Retention | @@ -128,6 +140,8 @@ The plugin and the altimate-code CLI are professional developer tools, not direc We will update this document in this repository and revise the effective date when our practices change. Material changes to Altimate-wide practices are governed by the master [Privacy Policy](https://www.altimate.ai/privacy). +**Next release:** added [The lineage report](#the-lineage-report): the offline report page, and the usage events the altimate-code CLI sends for it. + **November 10, 2026:** added [Workspace rules](#workspace-rules) (the workspace-rules hooks and `/altimate-code:rules` in plugin 1.5.0), and the rules telemetry the altimate-code CLI sends. ## Contact diff --git a/plugins/altimate-code/CHANGELOG.md b/plugins/altimate-code/CHANGELOG.md index 8e0150d..c26f101 100644 --- a/plugins/altimate-code/CHANGELOG.md +++ b/plugins/altimate-code/CHANGELOG.md @@ -2,6 +2,20 @@ All notable changes to this plugin will be documented in this file. +## [Unreleased] + +### Added +- End-of-task column impact: after a task that changed a dbt model, a band sums up each changed column's downstream reach from `impact_analysis`. The task's start copies `target/manifest.json` aside, so the answer reflects the SQL before the task. +- An offline HTML report, built when **Open report** or `/lineage-report` asks for it: what changed and what to expect downstream in plain words, who consumes it from dbt exposures, then upstream and downstream lineage, two levels by default, and click-to-highlight paths. +- `/lineage-report .` opens the same report for any column, with no task behind it: where the column comes from first, then where its value goes. It reads the project's manifest as it is now, writes its own file beside the task's report, and an unknown model or column gets a plain error. +- When a plan goes up for approval, a line above the dialog gives the downstream models, tests and exposures of each model the plan names. +- Usage events for the report, through altimate-code's telemetry (`tool report-event`): the summary shown, **Open report** pressed, `/lineage-report` run, and the report built, failed or with no browser. The step and the session id only, off under the telemetry opt-out. See PRIVACY.md. +- The report draws the column lineage as a graph beside the summary: upstream on the left, downstream on the right, opened to fit, with zoom and pan. A click lights a column's path, "more levels" adds one, and a level of more than 12 models folds into one box. The page allows only its own inline script and style. An edge carries a label only when the engine told how the value changes (same, renamed, calculated, aggregated); the rest are dashed and unlabelled, and a note under the graph says so. A click on a column counts even when the hand drifts a few pixels (up to 12) before letting go; a longer drag still pans. + +### Changed +- The per-edit impact note no longer shows as a toast and band; it still goes to Claude's context with the edit's result. A lookup that missed its time or failed still shows a toast. +- The per-edit column lookup goes through the altimate CLI instead of the MCP server, so it raises no permission prompt in accept-edits mode and a large column answer arrives whole. + ## [1.5.0] — 2026-11-10 ### Added diff --git a/plugins/altimate-code/README.md b/plugins/altimate-code/README.md index dd8976e..a77e22b 100644 --- a/plugins/altimate-code/README.md +++ b/plugins/altimate-code/README.md @@ -100,14 +100,17 @@ The side pane and band need a Claude Code build with hooks modules and `altimate ## Lineage beside the chat -The plugin's lineage pane follows the dbt model Claude reads or edits (by Read, Edit, a shell command that names or finds its `.sql` file, or a `dbt_lineage` call), read from `target/manifest.json` with no model calls. A project with no manifest still gets its graph, read from the models' `ref()` and `source()` calls; columns, dashboards and column reach then need `dbt parse` or `dbt compile`. From a monorepo's root, `/lineage ` finds the dbt project below it that has the model: +The plugin's lineage pane follows the dbt model Claude reads or edits (by Read, Edit, a shell command that names or finds its `.sql` file, or a `dbt_lineage` call), read from `target/manifest.json` with no model calls. The end-of-task summary's **Open lineage** opens it too. A project with no manifest still gets its graph, read from the models' `ref()` and `source()` calls; columns, dashboards and column reach then need `dbt parse` or `dbt compile`. From a monorepo's root, `/lineage ` finds the dbt project below it that has the model: - **The graph.** The model's whole lineage as boxes and arrows, like dbt-lineage-viewer: every level upstream on the left and downstream on the right, every model, every name in full, dashboards (exposures) marked ◆. The pane is a window onto the drawing. A graph too big for Claude Code to draw in a pane is shown as folders instead, and the header says so; `v` switches to the other views. Scroll it up and down with the mouse, touchpad or arrow keys, pan it with `h` and `l` (or ◀ ▶), and `f` brings the model back to the middle. Links between models on the same level, or that skip a level, are not drawn. Press a model to follow it. With a column picked, the models and dashboards that read it are highlighted and the rest dimmed. - **Views for big projects.** `v` cycles the graph, **ends only** (the sources and dashboards the model sits between; "via N" counts the models between them on the shortest path), **folders** (sources, staging, intermediate, marts and dashboards as one box each, with counts, in dbt layer order: a source-to-mart chain on one screen) and the text **list** of upstream and downstream. A pane narrower than 48 columns shows the list. The pane keeps up to 500 models upstream, and through altimate-code (a manifest over 4 MiB) up to 500 downstream too; past that the header says which side was cut. - **What the answer is about.** Once the session has seen a dbt project, after each answer the models Claude named are listed under the header (IN THE ANSWER), each a press away. When Claude answered without reading any model that turn (from what it read earlier), the pane follows the first model it named, so the lineage shown matches the answer. - **Columns.** Each output column with its source columns and how it is derived, from altimate-code's `dbt_lineage`, read from the compiled manifest (`dbt compile`). The plugin's calls are read-only, so altimate-code never starts the project's dbt adapter for them. - **Column reach.** `c` picks a column; the pane shows the downstream models whose SQL reads it and the dashboards on them, from `impact_analysis` (it traces each downstream model's compiled SQL, renames included). Models that name the column only in a filter or join, or that could not be traced (no compiled SQL), are listed apart, and the count is then a lower bound. -- **Impact first.** Before an edit to a model's SQL lands, the plugin checks what reads the model and the reach of every column the edit touches (up to three, merged). It tells a touched column from the model's own lineage (what the pane already read, or `dbt_lineage` through the CLI), so a column `schema.yml` leaves out still counts. It shows the result as a toast, adds it to Claude's context with the edit's result, and leaves a band above the prompt with the severity (impact_analysis' thresholds) and counts. +- **Impact first.** Before an edit to a model's SQL lands, the plugin checks what reads the model and the reach of every column the edit touches (up to three, merged). It tells a touched column from the model's own lineage (what the pane already read, or `dbt_lineage` through the CLI), so a column `schema.yml` leaves out still counts. It adds the result to Claude's context with the edit's result. You see the summary at the end of the task, not a note per edit; a toast says so when the lookup missed its time or failed and the edit went ahead without it. +- **End-of-task summary.** When a task (one prompt and Claude's work on it) changed a model, by an edit or a shell command git sees, a band above the prompt says what changed and how far it reaches: each changed column (up to three per model) with the columns, models and end columns downstream that carry its value, the models that may carry it and those cleared, next to the model-level count, and a line when some models could not be read. The trace reads a copy of `target/manifest.json` taken when the task started, so a recompile mid-task does not change the answer. These lookups go through the CLI, so they raise no permission prompt. +- **The report, on demand.** **Open report** or `/lineage-report` builds an offline HTML report of the last task that changed a model and opens it in the browser (`target/.altimate-lineage/report.html`). It leads with what changed, before and after, what to expect downstream in plain words, and who consumes it: the dbt exposures on the model or downstream, grouped by whether they read a model the value reaches, and a plain statement when the project declares none. Only declared exposures appear: other BI tools and backend services do not. Under that are two levels upstream and downstream (more on request), and a click on any column lights up its path. Nothing is written until you ask. What it reads was kept when the task ended (`task-manifest..json`, `last-task.json`), so asking later, after other turns or a restart, still describes that task, and it says if a file has changed since. Each session keeps its own copy of the manifest, and copies a day old are removed. A later task that changes a model replaces it. The band's button goes when the next turn starts; `/lineage-report` keeps working. +- **Before a plan.** When Claude puts a plan up for approval, the plugin looks for the models the plan names and adds one line above the dialog with each one's downstream models, tests and exposures, at the model level. It holds nothing back and spends no model tokens. The band shows the same once the dialog closes, until the task's summary takes over. It goes by the names in the plan's text, so a model it mentions may not be one it changes, and "the staging models" finds nothing. Leaving plan mode with shift+tab shows nothing. - **Warehouse lineage.** When a connected MCP server offers Altimate's `table_lineage` (Snowflake) or `databricks_table_lineage`, the pane adds what the warehouse's query history shows for the model's table. The plugin calls a tool of that name on whichever connected server offers it, itself and without a permission prompt, and sends it the model's `database.schema.table`. It does so only when you ask for the model (`/lineage`, a press in the pane, or `r`), never when the pane follows one of Claude's reads. - **Claude's own lineage calls.** A `dbt_lineage` result is drawn as a list of edges colored by how each column is derived; ctrl+o keeps the original output under it. diff --git a/plugins/altimate-code/hooks/impact-report.test.ts b/plugins/altimate-code/hooks/impact-report.test.ts new file mode 100644 index 0000000..a351e85 --- /dev/null +++ b/plugins/altimate-code/hooks/impact-report.test.ts @@ -0,0 +1,402 @@ +import { describe, expect, test } from 'claude-code/testing' + +import { + answerOf, + changedColumns, + columnReportData, + type ColumnAnswer, + type ColumnTrace, + consumersOf, + DEFAULT_LEVELS, + fingerprint, + focusOf, + impactWords, + mergeConsumers, + type ModelChange, + openersFor, + planLines, + reachOf, + reportData, + reportHtml, + sectionOf, + summaryLines, + summaryOf, + upstreamWords, + WINDOWS_EXPLORER, +} from './impact-report' + +// A synthetic project: orders → fct_orders.revenue → three hops of readers. +export const DOWN: ColumnAnswer = { + version: 1, + focus: { model: 'model.shop.fct_orders', column: 'revenue' }, + direction: 'downstream', + inputs: { manifest: { generated_at: '2026-01-01', older_than_sql: false }, sql: 'compiled_code', catalog: null, engine: 'core 1.0', workarounds: [] }, + counts: { columns: 4, models: 3, hops: 3, contracts: 1, ends: 2, may_carry_models: 1, cleared_models: 2, model_level: 6 }, + nodes: [ + { id: 'model.shop.fct_orders', name: 'fct_orders', folder: 'marts', materialized: 'table', contract: false, tests: 1, exposures: [] }, + { id: 'model.shop.fct_revenue_daily', name: 'fct_revenue_daily', folder: 'marts', materialized: 'table', contract: true, tests: 2, exposures: ['exec_revenue'] }, + { id: 'model.shop.rpt_finance', name: 'rpt_finance', folder: 'reports', materialized: 'view', contract: false, tests: 0, exposures: [] }, + { id: 'model.shop.rpt_board', name: 'rpt_board', folder: 'reports', materialized: 'view', contract: false, tests: 0, exposures: [] }, + { id: 'model.shop.rpt_misc', name: 'rpt_misc', folder: 'reports', materialized: 'view', contract: false, tests: 0, exposures: [] }, + { id: 'model.shop.rpt_broken', name: 'rpt_broken', folder: 'reports', materialized: 'view', contract: false, tests: 0, exposures: [] }, + ], + columns: [ + { model: 0, column: 'revenue', hop: 0, how: null as any, how_from: null as any }, + { model: 1, column: 'daily_revenue', hop: 1, type: 'number' }, + { model: 1, column: 'revenue', hop: 1, end: true, how: '=', how_from: 'name' }, + { model: 2, column: 'weekly_revenue', hop: 2 }, + { model: 3, column: 'board_revenue', hop: 3, end: true }, + ], + edges: [ + [0, 1, 'Σ'], + [0, 2, '=', 'name'], + [1, 3, 'ƒ'], + [3, 4, '~', 'name'], + ], + also_reads: [{ model: 4, columns: ['revenue'], downstream: 2 }], + unresolved: [{ model: 5, reason: 'parse error' }], + flags: [{ model: 2, flag: 'filter_rewritten' }], + cleared: [ + { id: 'model.shop.dim_dates', name: 'dim_dates' }, + { id: 'model.shop.rpt_ops', name: 'rpt_ops' }, + ], + truncated: false, +} + +export const UP: ColumnAnswer = { + version: 1, + focus: { model: 'model.shop.fct_orders', column: 'revenue' }, + direction: 'upstream', + nodes: [ + { id: 'model.shop.fct_orders', name: 'fct_orders' }, + { id: 'model.shop.int_orders', name: 'int_orders' }, + { id: 'source.shop.raw.orders', name: 'orders' }, + ], + columns: [ + { model: 0, column: 'revenue', hop: 0, how: null as any, how_from: null as any }, + { model: 1, column: 'amount', hop: 1 }, + { model: 2, column: 'amount_cents', hop: 2, end: true }, + ], + edges: [ + // Two edges into the focus: the traced aggregate is its how, not the name match. + [1, 0, 'Σ'], + [2, 0, 'Σ', 'name'], + [2, 1, 'ƒ'], + ], +} + +export const CHANGE: ModelChange = { + model: 'fct_orders', + file: 'models/fct_orders.sql', + root: '/repo', + columns: [{ name: 'revenue', before: 'sum(amount) as revenue', after: 'sum(amount) - sum(refunds) as revenue' }], +} +const TRACE: ColumnTrace = { model: 'fct_orders', column: 'revenue', down: DOWN, up: UP, modelLevel: 6, failed: false } +const TASK = { finishedAt: '2026-01-01 00:00 UTC', builtAt: '2026-01-01 00:05 UTC', manifest: 'the manifest as it was when the task started', movedOn: [] as string[] } +/** Words as the page reads them, with names in backticks. */ +const plain = (lines: (string | { code: string })[][]) => lines.map(line => line.map(one => (typeof one === 'string' ? one : `\`${one.code}\``)).join('')) + +/** The data a report embeds, read back out of its page. */ +export function embedded(html: string): any { + const match = /' } + const html = reportHtml(reportData([hostile], [{ ...TRACE, model: hostile.model }], TASK)) + expect(html).not.toMatch(/(src|href)="(https?:)?\/\//) + expect(html).not.toContain(' + + + +` +} + +const REPORT_CSS = ` +:root{ + --bg:#f7f7f5;--panel:#ffffff;--ink:#1d1d1b;--muted:#6b6b66;--faint:#9a9a93;--line:#e4e3de;--line-strong:#cfcdc6; + --accent:#3b5bdb;--accent-soft:#e8edff;--focus:#fff4d6;--focus-line:#e6b84a; + --same:#2f7d4f;--same-bg:#e3f3e9;--ren:#2c62b8;--ren-bg:#e3ecfb;--calc:#a35c00;--calc-bg:#fbefdc;--agg:#7a3fb0;--agg-bg:#f0e6fa;--unk:#6b6b66;--unk-bg:#eeeeea; + --warn:#9a5b00;--warn-bg:#fff3dc;--del:#b42318;--del-bg:#fdecea;--add:#1f7a43;--add-bg:#e6f5ec; + --shadow:0 1px 2px rgba(0,0,0,.05),0 2px 8px rgba(0,0,0,.04); +} +@media (prefers-color-scheme:dark){:root{ + --bg:#151515;--panel:#1e1e1d;--ink:#ecebe6;--muted:#a3a29b;--faint:#76756f;--line:#2f2f2d;--line-strong:#454540; + --accent:#8ea6ff;--accent-soft:#25293b;--focus:#3a3220;--focus-line:#b48a2c; + --same:#6fcf97;--same-bg:#1c3326;--ren:#8fb6ff;--ren-bg:#1d2a40;--calc:#f0b35c;--calc-bg:#3a2a12;--agg:#c79bf2;--agg-bg:#2f2240;--unk:#a3a29b;--unk-bg:#2a2a28; + --warn:#f0b35c;--warn-bg:#352a15;--del:#ff8a80;--del-bg:#3a1f1d;--add:#7ddc9f;--add-bg:#1b3325; + --shadow:none; +}} +*{box-sizing:border-box} +html{-webkit-text-size-adjust:100%} +body{margin:0;background:var(--bg);color:var(--ink);font:14px/1.45 -apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,"Helvetica Neue",Arial,sans-serif} +code,.mono{font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,"Liberation Mono",monospace;font-size:12.5px} +.wrap{max-width:1040px;margin:0 auto;padding:20px 16px 64px} +header.top{display:flex;flex-wrap:wrap;align-items:baseline;gap:6px 14px;margin-bottom:14px} +header.top h1{font-size:20px;margin:0;font-weight:650;letter-spacing:-.01em} +header.top .when{color:var(--muted);font-size:12.5px} +.panel{background:var(--panel);border:1px solid var(--line);border-radius:10px;box-shadow:var(--shadow)} +.changes,.impact,.consumers{padding:14px 16px;margin-bottom:14px} +.changes h2,.impact h2,.consumers h2,.section h2{font-size:12px;text-transform:uppercase;letter-spacing:.06em;color:var(--muted);margin:0 0 8px;font-weight:600} +.change-model{margin:8px 0 2px;font-weight:600;overflow-wrap:anywhere} +.change-model .file{color:var(--faint);font-weight:400;margin-left:6px;font-size:12px} +.diff{display:grid;grid-template-columns:auto 1fr;gap:3px 10px;margin:6px 0 10px;align-items:start} +.diff .col-name{font-weight:600;grid-row:span 2;padding-top:2px} +.diff .was,.diff .now{border-radius:6px;padding:3px 8px;overflow-wrap:anywhere;white-space:pre-wrap} +.diff .was{background:var(--del-bg);color:var(--del)} +.diff .now{background:var(--add-bg);color:var(--add)} +.diff .none{opacity:.7;font-style:italic} +.diff .same{grid-column:2;color:var(--muted);font-size:12.5px} +.impact h3{font-size:13px;font-weight:600;margin:10px 0 4px;overflow-wrap:anywhere} +.impact ul{margin:0;padding-left:18px} +.impact li{margin:5px 0} +.impact code,.consumers code{overflow-wrap:anywhere} +.consumers .tier{font-size:12.5px;font-weight:600;margin:10px 0 2px} +.consumers ul{margin:0 0 4px;padding-left:18px} +.consumers li{margin:3px 0;overflow-wrap:anywhere} +.consumers .why{color:var(--muted)} +.consumers .foot{color:var(--muted);font-size:12.5px;margin:10px 0 0} +.consumers a{color:var(--accent)} +.deep{margin:26px 0 12px;padding-top:16px;border-top:1px solid var(--line-strong)} +.deep h2{font-size:15px;font-weight:650;margin:0 0 2px} +.deep p{margin:0;color:var(--muted);font-size:13px} +.tabs{display:flex;flex-wrap:wrap;gap:6px;margin:0 0 12px} +.tab{appearance:none;border:1px solid var(--line-strong);background:var(--panel);color:var(--ink);border-radius:999px;padding:5px 12px;font:inherit;cursor:pointer} +.tab[aria-selected=true]{background:var(--accent);border-color:var(--accent);color:#fff} +@media (prefers-color-scheme:dark){.tab[aria-selected=true]{color:#111}} +.stats{display:flex;flex-wrap:wrap;gap:8px;margin:0 0 10px} +.stat{background:var(--panel);border:1px solid var(--line);border-radius:8px;padding:6px 10px;min-width:92px} +.stat b{display:block;font-size:18px;line-height:1.2} +.stat span{color:var(--muted);font-size:12px} +.stat.dim b{color:var(--muted)} +.note{border-radius:8px;padding:8px 12px;margin:0 0 12px;font-size:13px} +.note.warn{background:var(--warn-bg);color:var(--warn)} +.note.ok{background:var(--same-bg);color:var(--same)} +.inputs{color:var(--muted);font-size:12px;margin:-4px 0 14px} +.legend{display:flex;flex-wrap:wrap;gap:4px 12px;color:var(--muted);font-size:12px;margin:0 0 14px} +.legend .g{margin-right:4px} +.legend .sw{display:inline-block;width:16px;height:12px;border-radius:3px;background:var(--focus);border:1.5px solid var(--focus-line);vertical-align:-2px} +.section{margin:0 0 12px} +.section-head{display:flex;align-items:center;flex-wrap:wrap;gap:8px;margin:18px 0 8px} +.section-head h2{flex:1} +.section-head h2{margin:0} +.more{appearance:none;border:1px dashed var(--line-strong);background:transparent;color:var(--accent);border-radius:8px;padding:4px 10px;font:inherit;font-size:12.5px;cursor:pointer} +.more:hover{background:var(--accent-soft)} +.hop{margin:0 0 10px} +.hop-label{font-size:12px;color:var(--muted);margin:0 0 6px;display:flex;align-items:center;gap:8px} +.hop-label::after{content:"";flex:1;border-top:1px solid var(--line)} +.models{display:grid;grid-template-columns:repeat(auto-fill,minmax(270px,1fr));gap:8px} +.model{padding:9px 10px 6px;transition:opacity .15s} +.model-head{display:flex;flex-wrap:wrap;align-items:center;gap:4px 6px;margin-bottom:4px} +.model-name{font-weight:600;overflow-wrap:anywhere} +.badge{font-size:11px;border-radius:4px;padding:0 5px;border:1px solid var(--line-strong);color:var(--muted);line-height:17px} +.badge.c{border-color:var(--accent);color:var(--accent);font-weight:700} +.badge.x{border-color:var(--agg);color:var(--agg)} +.badge.f{border-color:var(--warn);color:var(--warn);background:var(--warn-bg)} +.model-meta{color:var(--faint);font-size:11.5px;margin-bottom:4px} +.col{display:flex;align-items:center;gap:7px;width:100%;appearance:none;border:0;background:transparent;color:inherit;font:inherit;text-align:left;padding:3px 4px;border-radius:6px;cursor:pointer} +.col:hover{background:var(--accent-soft)} +.col .name{overflow-wrap:anywhere;flex:1} +.col .type{color:var(--faint);font-size:11.5px} +.col .end,.legend .end{font-size:10.5px;color:var(--muted);border:1px solid var(--line-strong);border-radius:4px;padding:0 4px} +.g{display:inline-flex;align-items:center;justify-content:center;min-width:20px;height:20px;border-radius:5px;font-weight:700;font-size:12.5px;flex:none} +.g.same{color:var(--same);background:var(--same-bg)} +.g.ren{color:var(--ren);background:var(--ren-bg)} +.g.calc{color:var(--calc);background:var(--calc-bg)} +.g.agg{color:var(--agg);background:var(--agg-bg)} +.g.unk{color:var(--unk);background:var(--unk-bg)} +.g.byname{outline:1.5px dashed currentColor;outline-offset:-1.5px;background:transparent;opacity:.7} +.col .g.byname+.name{opacity:.8} +.focus{padding:12px 14px;border:1.5px solid var(--focus-line);background:var(--focus);margin:6px 0} +.focus .label{font-size:11px;text-transform:uppercase;letter-spacing:.06em;color:var(--muted)} +.focus .title{font-size:17px;font-weight:650;margin:2px 0;display:flex;align-items:center;gap:8px;flex-wrap:wrap} +.focus .col{width:auto;padding:2px 6px;display:inline-flex} +.empty{color:var(--muted);font-size:13px;padding:6px 2px} +details.fold{margin:8px 0;padding:0 12px} +details.fold summary{cursor:pointer;padding:10px 0;font-weight:600} +details.fold summary span{color:var(--muted);font-weight:400} +details.fold ul{margin:0 0 10px;padding-left:18px} +details.fold li{margin:3px 0} +details.fold .why{color:var(--muted)} +/* The path bar floats over the page: shown in the flow, it would push the page down under the pointer that clicked. */ +.pathbar{position:fixed;top:8px;left:16px;right:16px;max-width:1648px;margin:0 auto;z-index:20;display:none;align-items:center;gap:10px;flex-wrap:wrap;background:var(--accent);color:#fff;padding:8px 14px;border-radius:10px;box-shadow:0 4px 16px rgba(0,0,0,.2);font-size:13px;pointer-events:none} +@media (prefers-color-scheme:dark){.pathbar{color:#111}} +.pathbar button{appearance:none;border:1px solid currentColor;background:transparent;color:inherit;border-radius:6px;padding:2px 8px;font:inherit;cursor:pointer;pointer-events:auto} +body.hl .pathbar{display:flex} +body.hl .col:not(.on){opacity:.28} +body.hl .model:not(.has-on){opacity:.45} +body.hl .col.on{background:var(--accent-soft)} +body.hl .col.sel{outline:2px solid var(--accent)} +.noscript{padding:16px} +@media (max-width:560px){ + .wrap{padding:14px 12px 48px} + header.top h1{font-size:17px} + .models{grid-template-columns:1fr} + .stat{min-width:0;flex:1 1 28%;padding:5px 8px} +} +` + +const REPORT_JS = ` +(function(){ +var DATA=JSON.parse(document.getElementById('report-data').textContent); +var app=document.getElementById('app'); +var GLYPH={'=':'same','~':'ren','\\u0192':'calc','\\u03a3':'agg','?':'unk'}; +var HOW={'=':'same value','~':'renamed','\\u0192':'calculated','\\u03a3':'aggregated','?':'unknown'}; +var state={focus:0,levels:[],expanded:[],views:[],list:false,selected:null}; +DATA.focuses.forEach(function(){state.levels.push({up:DATA.defaultLevels,down:DATA.defaultLevels});state.expanded.push({});state.views.push(null)}); +function el(tag,cls,text){var n=document.createElement(tag);if(cls)n.className=cls;if(text!=null)n.textContent=String(text);return n} +function plural(n,w){return n+' '+w+(n===1?'':'s')} +function glyph(g,byName){var s=el('span','g '+(GLYPH[g]||'unk')+(byName?' byname':''),g||'?');s.title=(HOW[g]||'unknown')+(byName?' (matched by name, not traced through SQL)':'');return s} +function graphOf(f){var fwd={},back={};f.edges.forEach(function(e){(fwd[e[0]]=fwd[e[0]]||[]).push(e[1]);(back[e[1]]=back[e[1]]||[]).push(e[0])});return{fwd:fwd,back:back}} +function walk(start,adj){var seen={},q=[start];seen[start]=1;while(q.length){var k=q.shift();(adj[k]||[]).forEach(function(n){if(!seen[n]){seen[n]=1;q.push(n)}})}return seen} +function pathOf(f,key){var g=graphOf(f),up=walk(key,g.back),down=walk(key,g.fwd),all={};Object.keys(up).forEach(function(k){all[k]=1});Object.keys(down).forEach(function(k){all[k]=1});return{all:all,up:Object.keys(up).length-1,down:Object.keys(down).length-1}} +function colButton(c){var b=el('button','col');b.type='button';b.dataset.key=c.key;b.appendChild(glyph(c.glyph,c.byName));b.appendChild(el('span','name mono',c.column));if(c.type)b.appendChild(el('span','type',c.type));if(c.end){var e=el('span','end','end');e.title='Nothing further downstream reads it';b.appendChild(e)}b.addEventListener('click',function(ev){ev.stopPropagation();select(c.key)});return b} +function modelCard(m){var card=el('div','panel model');var head=el('div','model-head');head.appendChild(el('span','model-name mono',m.name));if(m.contract){var c=el('span','badge c','C');c.title='Contract enforced: a change to this column can fail its build';head.appendChild(c)}if(m.tests){head.appendChild(el('span','badge',plural(m.tests,'test')))}m.exposures.forEach(function(x){var b=el('span','badge x','\\u25a3 '+x);b.title='Exposure (dashboard or app) on this model';head.appendChild(b)});(m.flags||[]).forEach(function(x){var b=el('span','badge f',x);b.title='Noted by the engine: the trace through this model is less certain';head.appendChild(b)});card.appendChild(head);var meta=[m.folder,m.materialized].filter(Boolean).join(' \\u00b7 ');if(meta)card.appendChild(el('div','model-meta',meta));m.columns.forEach(function(c){card.appendChild(colButton(c))});return card} +function hopBlock(h,dir){var d=el('div','hop');var cols=h.models.reduce(function(n,m){return n+m.columns.length},0);d.appendChild(el('div','hop-label',(dir==='up'?'Reads from \\u00b7 ':'')+plural(h.hop,'hop')+' '+(dir==='up'?'up':'down')+' \\u00b7 '+plural(h.models.length,'model')+' \\u00b7 '+plural(cols,'column')));var grid=el('div','models');h.models.forEach(function(m){grid.appendChild(modelCard(m))});d.appendChild(grid);return d} +function section(f,i,dir){var sec=f[dir],lv=state.levels[i][dir],s=el('section','section');var head=el('div','section-head');head.appendChild(el('h2',null,dir==='up'?'Upstream: where it comes from':'Downstream: where it goes'));var total=sec.hops.length,hidden=sec.hops.filter(function(h){return h.hop>lv}).length;if(hidden>0){var more=el('button','more',hidden===1?'Show 1 more level':'Show more levels ('+hidden+' hidden)');more.type='button';more.addEventListener('click',function(){state.levels[i][dir]=lv+1;render()});head.appendChild(more);if(hidden>1){var all=el('button','more','Show all');all.type='button';all.addEventListener('click',function(){state.levels[i][dir]=Infinity;render()});head.appendChild(all)}}else if(total>DATA.defaultLevels&&lv>DATA.defaultLevels){var less=el('button','more','Back to '+DATA.defaultLevels+' levels');less.type='button';less.addEventListener('click',function(){state.levels[i][dir]=DATA.defaultLevels;render()});head.appendChild(less)}s.appendChild(head);if(sec.error){s.appendChild(el('div','empty',sec.error));return s}var shown=sec.hops.filter(function(h){return h.hop<=lv});if(!shown.length)s.appendChild(el('div','empty',dir==='up'?'Nothing upstream was traced for this column.':'No downstream model carries this column\\u2019s value.'));if(dir==='up')shown=shown.slice().reverse();shown.forEach(function(h){s.appendChild(hopBlock(h,dir))});if(sec.truncated)s.appendChild(el('div','note warn','The trace stopped at its limit: more may sit beyond the last level.'));return s} +function stat(n,label,dim,tip){var d=el('div','stat'+(dim?' dim':''));if(tip)d.title=tip;d.appendChild(el('b',null,n==null?'\\u2013':n));d.appendChild(el('span',null,label));return d} +function fold(title,count,items){var d=el('details','panel fold');var s=el('summary',null,title+' ');s.appendChild(el('span',null,'('+(count==null?'not reported':count)+')'));d.appendChild(s);var ul=el('ul');items.forEach(function(it){ul.appendChild(it)});if(!items.length)ul.appendChild(el('li','why',count==null?'Not reported by this altimate-code build.':count?'Only the count is known.':'None.'));d.appendChild(ul);return d} +function li(name,rest){var l=el('li');l.appendChild(el('span','mono',name));if(rest){l.appendChild(document.createTextNode(' '));l.appendChild(el('span','why',rest))}return l} +function words(parent,line){line.forEach(function(w){if(typeof w==='string')parent.appendChild(document.createTextNode(w));else parent.appendChild(el('code',null,w.code))});return parent} +function upstreamPanel(){var f=DATA.focuses[state.focus],p=el('div','panel impact upstream');p.appendChild(el('h2',null,'Where it comes from'));var ul=el('ul');f.upstream.forEach(function(line){ul.appendChild(words(el('li'),line))});p.appendChild(ul);return p} +function impactPanel(){var p=el('div','panel impact');p.appendChild(el('h2',null,DATA.lookup?'Where it goes':'What to expect downstream'));DATA.focuses.forEach(function(g){if(DATA.focuses.length>1)p.appendChild(el('h3','mono',g.column?g.model+'.'+g.column:g.model));var ul=el('ul');g.impact.forEach(function(line){ul.appendChild(words(el('li'),line))});p.appendChild(ul)});return p} +function consumerItem(c){var l=el('li');if(c.url){var a=el('a',null,c.label);a.href=c.url;a.rel='noopener noreferrer';a.target='_blank';l.appendChild(a)}else l.appendChild(document.createTextNode(c.label));var bits=[c.type];if(c.via.length)bits.push('reads '+c.via.join(', '));if(c.owner)bits.push('owner: '+c.owner);if(DATA.focuses.length>1&&c.columns.length)bits.push('for '+c.columns.join(', '));l.appendChild(el('span','why',' \\u00b7 '+bits.join(' \\u00b7 ')));return l} +var TIERS=[['carries','Read a model the value reaches'],['may_carry','Read a model that may carry it'],['cleared',"Read only models the value doesn't reach"],[null,'Read this model or one downstream of it']]; +function consumersPanel(){var c=DATA.consumers,p=el('div','panel consumers');p.appendChild(el('h2',null,'Who consumes it')); + if(c.error)p.appendChild(el('div','empty','The project\\u2019s exposures could not be read: '+c.error)); + else if(c.declared===null)p.appendChild(el('div','empty','This version of altimate-code does not report exposures, so consumers are unknown.')); + else if(c.declared===0)p.appendChild(el('div','empty','This dbt project declares no exposures, so no dashboards or other consumers can be named. That does not mean nothing reads these tables.')); + else if(!c.list.length)p.appendChild(el('div','empty','This dbt project declares '+plural(c.declared,'exposure')+', but none reads this model or anything downstream of it.')); + TIERS.forEach(function(t){var mine=c.list.filter(function(x){return x.tier===t[0]});if(!mine.length)return;p.appendChild(el('div','tier',t[1]+' ('+mine.length+')'));var ul=el('ul');mine.forEach(function(x){ul.appendChild(consumerItem(x))});p.appendChild(ul)}); + p.appendChild(el('p','foot',(c.list.length?'dbt records which models an exposure reads, not which columns, so each may or may not use this one. ':'')+'Only exposures a dbt project declares can appear here: other BI tools and backend services do not.'));return p} +function render(){ + var f=DATA.focuses[state.focus],i=state.focus;app.textContent=''; + var bar=el('div','pathbar');bar.id='pathbar';var wrap=el('div','wrap');wrap.appendChild(bar); + var top=el('header','top');top.appendChild(el('h1',null,DATA.title));top.appendChild(el('span','when',(DATA.lookup?'Report built '+DATA.builtAt:'Task finished '+DATA.finishedAt+' \\u00b7 report built '+DATA.builtAt)+' \\u00b7 read from '+DATA.manifest));wrap.appendChild(top); + if(DATA.movedOn.length)wrap.appendChild(el('div','note warn','\\u26a0 Since this task ended, '+DATA.movedOn.join(', ')+(DATA.movedOn.length===1?' has':' have')+' changed. This report describes the task, not the files as they are now.')); + var ch=el('div','panel changes');ch.appendChild(el('h2',null,'What this task changed'));DATA.changes.forEach(function(c){var m=el('div','change-model mono',c.model);m.appendChild(el('span','file',c.file));ch.appendChild(m);if(!c.columns.length){ch.appendChild(el('div','empty','No output column changed (a filter or a join): only the model-level count applies.'));return}var d=el('div','diff');c.columns.forEach(function(col){d.appendChild(el('div','col-name mono',col.name));var was=el('div','was mono'+(col.before?'':' none'),col.before||'(not in the model before)');var now=el('div','now mono'+(col.after?'':' none'),col.after||'(removed)');if(col.before&&col.after&&col.before===col.after){d.appendChild(el('div','same','Expression unchanged; the column is named on a changed line.'));d.appendChild(el('div'))}else{d.appendChild(was);d.appendChild(now)}});ch.appendChild(d)}); + var page=el('div','page'),side=el('div','side'),main=el('div','main');page.appendChild(side);page.appendChild(main); + side.appendChild(DATA.lookup?upstreamPanel():ch);side.appendChild(impactPanel());side.appendChild(consumersPanel()); + var deep=el('div','deep');deep.appendChild(el('h2',null,'Column by column'));deep.appendChild(el('p',null,'Two levels up and down, with more on request. Click any column to light up its path; the rest dims and stays on the page.'));main.appendChild(deep); + if(DATA.focuses.length>1){var tabs=el('div','tabs');tabs.setAttribute('role','tablist');DATA.focuses.forEach(function(g,j){var t=el('button','tab',g.column?g.model+'.'+g.column:g.model+' (model)');t.type='button';t.setAttribute('role','tab');t.setAttribute('aria-selected',j===i?'true':'false');t.addEventListener('click',function(){state.focus=j;state.selected=null;render()});tabs.appendChild(t)});main.appendChild(tabs)} + var r=f.reach,stats=el('div','stats'); + if(DATA.lookup){var uc=0,um={},uh=0,ue=!!f.up.error;f.up.hops.forEach(function(h){uh=Math.max(uh,h.hop);h.models.forEach(function(m){um[m.id]=1;uc+=m.columns.length})});stats.appendChild(stat(ue?null:uc,'columns upstream',false,'Columns the value is made from, followed all the way back'));stats.appendChild(stat(ue?null:Object.keys(um).length,'models upstream',false,'Models those columns sit in'));stats.appendChild(stat(ue?null:uh,'hops up',false,'How many models back the trace goes'))} + if(r){stats.appendChild(stat(r.columns,'columns downstream',false,'Columns the value flows into, at every level'));stats.appendChild(stat(r.models,'models carry it',false,'Downstream models with a column the value flows into'));stats.appendChild(stat(r.ends,'end columns',false,'Columns the value reaches that nothing further downstream reads'));stats.appendChild(stat(r.mayCarry,'models may carry it',!r.mayCarry,'Models that use the value in a way the trace can\\u2019t follow (a filter or a join, say), whose SQL couldn\\u2019t be read, or that read from a model like that'));stats.appendChild(stat(r.cleared,'models cleared',r.cleared==null,'Downstream of the model, but the value doesn\\u2019t reach them'))} + stats.appendChild(stat(f.modelLevel,'models downstream (model level)',true,'Every model downstream of this one, whether or not the value reaches it'));main.appendChild(stats); + if(r&&(r.unresolved||r.truncated))main.appendChild(el('div','note warn','\\u26a0 Lower bound: '+[r.unresolved?plural(r.unresolved,'model')+' could not be read':'',r.truncated?'the trace hit its limit':''].filter(Boolean).join('; ')+'. See \\u201cNot resolved\\u201d below.')); + else if(r)main.appendChild(el('div','note ok','\\u2713 Every downstream model was read.')); + var inp=[];if(f.inputs.engine)inp.push('engine '+f.inputs.engine);inp.push(f.inputs.catalog?'with catalog types':'no catalog: types from the manifest');if(f.inputs.olderThanSql)inp.push('the manifest is older than the SQL');main.appendChild(el('div','inputs',inp.join(' \\u00b7 '))); + var lg=el('div','legend');['=','~','\\u0192','\\u03a3','?'].forEach(function(g){var s=el('span');s.appendChild(glyph(g,false));s.appendChild(document.createTextNode(HOW[g]));lg.appendChild(s)});var bn=el('span');bn.appendChild(glyph('=',true));bn.appendChild(document.createTextNode('matched by name'));lg.appendChild(bn);var cb=el('span');cb.appendChild(el('span','badge c','C'));cb.appendChild(document.createTextNode(' contract enforced'));lg.appendChild(cb); + [['badge x','\u25a3','exposures on the model'],['badge f','!','noted by the engine'],['end','end','nothing further downstream reads it'],['sw','',DATA.lookup?'the column':'the changed column']].forEach(function(k){var s=el('span');s.appendChild(el('span',k[0],k[1]));s.appendChild(document.createTextNode(' '+k[2]));lg.appendChild(s)});main.appendChild(lg); + var graph=null;if(!state.list){graph=graphPanel(f,i);main.appendChild(graph)}else{var lb=el('div','g-bar list-bar');lb.appendChild(el('span','g-title','Column list'));var gb=el('button','g-btn','Graph');gb.type='button';gb.addEventListener('click',function(){state.list=false;render()});lb.appendChild(gb);main.appendChild(lb); + main.appendChild(section(f,i,'up')); + var fc=el('div','panel focus');fc.appendChild(el('div','label',DATA.lookup?'The column':'The changed column'));var ti=el('div','title');if(f.key){ti.appendChild(colButton({key:f.key,column:f.model+'.'+(f.column||''),glyph:f.glyph||'?',byName:f.byName,end:false,type:null}))}else ti.appendChild(el('span','mono',f.column?f.model+'.'+f.column:f.model));fc.appendChild(ti);var hw=f.glyph?HOW[f.glyph]||'unknown':'';fc.appendChild(el('div','empty',(hw?hw.charAt(0).toUpperCase()+hw.slice(1)+' from what it reads'+(f.byName?' (matched by name)':'')+'. ':'')+(f.key?'Click any column to light up its path.':'')));main.appendChild(fc); + main.appendChild(section(f,i,'down'))} + wrap.appendChild(page); + wrap.appendChild(fold('Also reads it, outside the select',f.alsoReads.length,f.alsoReads.map(function(a){return li(a.name,'reads '+a.columns.join(', ')+(a.downstream!=null?' \\u00b7 '+plural(a.downstream,'model')+' downstream of it':''))}))); + wrap.appendChild(fold('Not resolved',f.unresolved.length,f.unresolved.map(function(u){return li(u.name,u.reason+(u.downstream!=null?' \\u00b7 '+plural(u.downstream,'model')+' downstream of it':''))}))); + wrap.appendChild(fold('Cleared: downstream, but does not carry it',f.cleared.count,f.cleared.names.map(function(n){return li(n,'')}))); + if(f.inputs.workarounds.length)wrap.appendChild(fold('How the engine read the SQL',f.inputs.workarounds.length,f.inputs.workarounds.map(function(w){var l=el('li','why',w);return l}))); + app.appendChild(wrap);if(graph)graph.afterMount();applyHighlight(); +} +function select(key){state.selected=state.selected===key?null:key;applyHighlight()} +function applyHighlight(){var f=DATA.focuses[state.focus],bar=document.getElementById('pathbar');var cols=document.querySelectorAll('.col,.g-row');var edges=document.querySelectorAll('.g-edge');if(!state.selected){document.body.classList.remove('hl');cols.forEach(function(c){c.classList.remove('on','sel')});edges.forEach(function(e){e.classList.remove('on')});document.querySelectorAll('.model,.g-box').forEach(function(m){m.classList.remove('has-on')});bar.textContent='';return} + var p=pathOf(f,state.selected);document.body.classList.add('hl');cols.forEach(function(c){var on=!!p.all[c.dataset.key];c.classList.toggle('on',on);c.classList.toggle('sel',c.dataset.key===state.selected)});document.querySelectorAll('.model,.g-box').forEach(function(m){m.classList.toggle('has-on',!!m.querySelector('.col.on,.g-row.on'))}); + edges.forEach(function(e){var on=(e.getAttribute('data-keys')||'').split(' ').some(function(k){var ab=k.split('>');return !!(p.all[ab[0]]&&p.all[ab[1]])});e.classList.toggle('on',on)}); + var shown={};cols.forEach(function(c){shown[c.dataset.key]=1});var hidden=Object.keys(p.all).filter(function(k){return!shown[k]}).length; + bar.textContent='';bar.appendChild(el('span',null,'Path of '+state.selected.split('|')[0].split('.').pop()+'.'+state.selected.split('|').pop()+': '+plural(p.up,'column')+' upstream, '+plural(p.down,'column')+' downstream'+(hidden?' \\u00b7 '+hidden+' more on hidden levels or folded models':'')));var clr=el('button',null,'Clear');clr.type='button';clr.addEventListener('click',function(){select(state.selected)});bar.appendChild(clr)} +${graphLayout.toString()} +${graphView.toString()} +${pressIsClick.toString()} +${GRAPH_VIEW_JS} +document.addEventListener('keydown',function(e){if(e.key==='Escape'&&state.selected)select(state.selected)}); +render(); +})(); +` + +/** The commands that may open a file in the person's browser, per platform, in the order to try them. */ +export type Platform = 'mac' | 'linux' | 'wsl' | 'windows' +export type Opener = { argv: string[]; okExits: number[] } +export function openersFor(platform: Platform, path: string, wsl: { windowsPath?: string | null; explorerAtC?: boolean } = {}): Opener[] { + if (platform === 'mac') return [{ argv: ['open', path], okExits: [0] }] + if (platform === 'linux') return [{ argv: ['xdg-open', path], okExits: [0] }] + if (platform === 'windows') return [{ argv: ['cmd', '/c', 'start', '', path], okExits: [0] }] + // WSL: wslview when installed; else Windows' explorer.exe on the Windows form of the path, by name + // (on PATH when Windows' folders are appended) and then where Windows keeps it; xdg-open last. + // explorer.exe exits 1 even when it opened the file. + const explorer = (bin: string): Opener => ({ argv: [bin, wsl.windowsPath!], okExits: [0, 1] }) + return [ + { argv: ['wslview', path], okExits: [0] }, + ...(wsl.windowsPath ? [explorer('explorer.exe'), ...(wsl.explorerAtC ? [explorer(WINDOWS_EXPLORER)] : [])] : []), + { argv: ['xdg-open', path], okExits: [0] }, + ] +} +export const WINDOWS_EXPLORER = '/mnt/c/Windows/explorer.exe' diff --git a/plugins/altimate-code/hooks/integrations.tsx b/plugins/altimate-code/hooks/integrations.tsx index 256131b..198038a 100644 --- a/plugins/altimate-code/hooks/integrations.tsx +++ b/plugins/altimate-code/hooks/integrations.tsx @@ -3,7 +3,8 @@ import type { EngineInterface, Register } from 'claude-code' import type { Busy, Expanded, Form, IntegrationsStatus, Provider, ProviderId, WorkspaceChoice } from '../types' import { MISSING_BINARY, later, parseJson, runAltimate } from './altimate-cli' -import { lineageBand, registerLineage, startLineage } from './lineage' +import { registerLineage, startLineage } from './lineage' +import { registerTaskImpact, startTaskImpact, taskBand } from './task-impact' const PANE = 'altimate-integrations' const PANE_TITLE = 'Altimate integrations' @@ -560,6 +561,7 @@ export const register: Register = on => { }) // The lineage half failing to start must not stop this pane's own setup below. await startLineage($).catch(error => $.ui.log(`lineage-pane · could not start: ${String(error)}`)) + await startTaskImpact($).catch(error => $.ui.log(`lineage-summary · could not start: ${String(error)}`)) // A press that was running when an earlier session ended can never finish, so it must not block this one. await update($, busy, () => null) // A `-p` run or the SDK has nobody to show the band to, so it does not scan the machine. @@ -580,9 +582,9 @@ export const register: Register = on => { }) on('ui.render', { component: 'AbovePrompt' }, async ($, e, next) => { - // One band per plugin: the lineage impact line rides under the integrations one, or alone. + // One band per plugin: the end-of-task lineage summary rides under the integrations one, or alone. if (e.props.hasSurvey) return next(e) - const lineage = await lineageBand($, $.ui.resolve(e)) + const lineage = await taskBand($, $.ui.resolve(e)) const pass = async () => lineage ?? next(e) const current = await read($, status) if (!current || (await read($, isBandHidden))) return pass() @@ -796,6 +798,7 @@ export const register: Register = on => { }) registerLineage(on) + registerTaskImpact(on) } // One row of buttons per provider, chosen by its state. Credentials are added diff --git a/plugins/altimate-code/hooks/lineage.test.tsx b/plugins/altimate-code/hooks/lineage.test.tsx index 2d3012d..938a74e 100644 --- a/plugins/altimate-code/hooks/lineage.test.tsx +++ b/plugins/altimate-code/hooks/lineage.test.tsx @@ -18,7 +18,7 @@ const PANE = { const BAND = { plugin: 'altimate-code', component: 'AbovePrompt', - props: { hasSurvey: false, isWorking: false, maxRows: 10, bodyColumns: 120, scroll: { offset: 0, bodyRows: 10 }, view: {} }, + props: { hasSurvey: false, isCollapsed: false, isWorking: false, maxRows: 10, bodyColumns: 120, scroll: { offset: 0, bodyRows: 10 }, view: {} }, } as const const LINEAGE = { @@ -67,8 +67,12 @@ async function start($: any, on: any, options: Options = {}) { const clock = mock.clock(on) const calls: string[] = [] const toasts: string[] = [] + // The per-edit impact note: logged for the record, never a toast (the person sees the end-of-task summary). + const notes: string[] = [] const logs: string[] = [] + const debug: string[] = [] const lists: string[] = [] + const opens: string[] = [] on('command.register', () => ({ value: { command: 'lineage' } })) // A turn's start and end beneath the plugin: the engine's answer is the model's text. on('turn.start', (_: unknown, e: { turnId: string }) => ({ turnId: e.turnId })) @@ -77,13 +81,18 @@ async function start($: any, on: any, options: Options = {}) { on('session.start', (_: unknown, e: { cwd: string }) => ({ cwd: e.cwd })) on('session.root', () => ({ value: ROOT })) // A narrow terminal: Claude Code leaves a pane the plugin opens on its own undrawn, and says why. - on('ui.open', (_: unknown, e: { focus?: boolean }) => ({ value: options.narrow && !e.focus ? { isPlaced: false, reason: options.narrow } : {} })) + on('ui.open', (_: unknown, e: { id: string; focus?: boolean }) => { + opens.push(e.id) + return { value: options.narrow && !e.focus ? { isPlaced: false, reason: options.narrow } : {} } + }) on('ui.toast', (_: unknown, e: { text: string }) => { toasts.push(e.text) return { value: undefined } }) - on('ui.log', (_: unknown, e: { text: string }) => { - logs.push(e.text) + // The transcript's lines, and the debug log's apart. + on('ui.log', (_: unknown, e: { text: string; to?: string }) => { + ;(e.to === 'debug' ? debug : logs).push(e.text) + if (e.to === 'debug' && e.text.startsWith('◆ impact-first · ')) notes.push(e.text) return { value: undefined } }) on('ui.render', ($$: any, e: any) => { @@ -186,7 +195,13 @@ async function start($: any, on: any, options: Options = {}) { // The integrations half of the plugin asks the CLI for its status at session start. if (command !== 'tool') return { value: { exitCode: 1, stdout: '', stderr: 'not this test' } } const args = JSON.parse(String(e.argv[e.argv.indexOf('--args') + 1] ?? '{}')) - calls.push(`cli:${tool}${args.include_graph ? ':graph' : ''}`) + calls.push(`cli:${tool}${args.include_graph ? ':graph' : ''}${args.column ? `:${args.column}` : ''}`) + // A column that fails, or answers late, as the MCP server does in the tests below. + if (tool === 'impact_analysis' && args.column === options.failColumn) return { value: { exitCode: 1, stdout: JSON.stringify({ status: 'error', output: 'boom', metadata: {} }), stderr: '' } } + if (tool === 'impact_analysis' && options.holdReach && args.column === options.holdReach.column) { + await options.holdReach.until + return { value: { exitCode: 0, stdout: JSON.stringify({ status: 'ok', output: 'x', metadata: { severity: 'LOW', column_reach: { models: [{ name: 'stale_reader', columns: [] }], exposures: [] } } }), stderr: '' } } + } if (options.cliFailure) return { value: { exitCode: 1, stdout: JSON.stringify({ status: 'error', output: 'the dbt project could not start', metadata: {} }), stderr: '' } } if (options.cliTooLarge) return { value: { exitCode: 0, stdout: '{"status":"ok","out', stderr: '', isStdoutTruncated: true } } if (options.cliCutOff?.(tool)) return { value: { exitCode: options.cliCutOffExit ?? 0, stdout: '{"status":"ok","out', stderr: '' } } @@ -209,7 +224,7 @@ async function start($: any, on: any, options: Options = {}) { }) await $.session.start({ cwd: ROOT, surface: 'terminal', isInteractive: true }) await clock.settle() - return { clock, calls, toasts, logs, lists } + return { clock, calls, toasts, notes, logs, debug, lists, opens } } test('reading a model points the pane at it: the manifest graph, then its columns from dbt_lineage', async ($, on) => { @@ -230,26 +245,49 @@ test('reading a model points the pane at it: the manifest graph, then its column await ui.unmount() }) -test('an edit naming a column is held until its reach is known, then the reach goes to Claude and the band', async ($, on) => { - const { clock, calls, toasts, logs } = await start($, on) +test('the pane opens by itself once a conversation; later reads, edits, shell commands and answers only move what it shows', async ($, on) => { + const { clock, opens } = await start($, on) + await $.tool.call({ tool: 'Read', file_path: FILE }) + await clock.settle() + expect(opens).toEqual(['altimate-lineage']) + await $.tool.call({ tool: 'Edit', file_path: FILE, old_string: 'sum(amount) as revenue', new_string: 'sum(amount) * 2 as revenue' }) + await clock.settle() + await $.tool.call({ tool: 'Bash', command: `cat ${FILE}` }) + await clock.settle() + await $.turn.start({ text: 'which model?', turnId: 't2' }) + await $.turn.complete({ answer: 'Look at `fct_orders`.', durationMs: 5, isAborted: false, reason: 'answer', turnId: 't2' } as any) + await clock.settle() + expect(opens).toEqual(['altimate-lineage']) + + // /lineage opens it again, on the model it followed. + await $.command.run({ command: 'lineage', args: '' } as any) + await clock.settle() + expect(opens).toEqual(['altimate-lineage', 'altimate-lineage']) + const ui = await $.ui.mount({ ...PANE, surface: 'terminal' }) + expect(await ui.find({ type: 'Text', text: /lineage · fct_orders/ })).toBeDefined() + await ui.unmount() +}) + +test('an edit naming a column is held until its reach is known, then the reach goes to Claude and the log, not the person', async ($, on) => { + const { clock, calls, toasts, notes, logs, debug } = await start($, on) const edit = { tool: 'Edit' as const, file_path: FILE, old_string: 'sum(amount) as revenue', new_string: 'sum(amount) - sum(refunds) as revenue' } const result: any = await $.tool.call(edit) await clock.settle() // The reach is looked up before the edit runs. - expect(calls.indexOf('mcp:impact_analysis:revenue')).toBeLessThan(calls.indexOf('tool:Edit')) - expect(toasts[0]).toBe('◆ impact-first · fct_orders.revenue is read by 2 models and 1 dashboard · Exec Revenue') + expect(calls.indexOf('cli:impact_analysis:revenue')).toBeGreaterThan(-1) + expect(calls.indexOf('cli:impact_analysis:revenue')).toBeLessThan(calls.indexOf('tool:Edit')) + expect(notes[0]).toBe('◆ impact-first · fct_orders.revenue is read by 2 models and 1 dashboard · Exec Revenue') + expect(toasts).toEqual([]) expect(result.context?.[0]).toContain('read by 2 downstream models: fct_revenue_daily (revenue), rpt_finance_weekly (revenue)') expect(result.context?.[0]).toContain('dashboards on them: "Exec Revenue"') - expect(logs).toContain("impact-first · added to Claude's context: lineage for fct_orders.revenue") + expect(debug).toContain("impact-first · added to Claude's context: lineage for fct_orders.revenue") + // Nothing per edit reaches the transcript: the person's one line is the end-of-task summary. + expect(logs.filter(line => line.includes('impact-first'))).toEqual([]) + // The band waits for the end of the task (task-impact.tsx): nothing per edit. const band = await $.ui.mount({ ...BAND, surface: 'terminal' }) - expect(await band.find({ type: 'Text', text: 'fct_orders.revenue' })).toBeDefined() - expect(await band.find({ type: 'Text', text: 'MEDIUM' })).toBeDefined() - expect(await band.find({ type: 'Text', text: /direct 5 · transitive 1 · tests 1 · read by 2 models · 1 dashboard(?!s)/ })).toBeDefined() - await band.press({ key: 'lineage-dismiss' }) - await clock.settle() - expect(await band.find({ key: 'lineage-open' })).toBeUndefined() + expect(await band.find({ type: 'Text', text: /fct_orders\.revenue/ })).toBeUndefined() await band.unmount() // The same column again adds nothing new to Claude's context. @@ -258,6 +296,44 @@ test('an edit naming a column is held until its reach is known, then the reach g expect(again.context ?? []).toEqual([]) }) +test('with a column answer, the note Claude gets has the numbers the end-of-task summary shows', async ($, on) => { + // The same call carries the older column reach too: the column answer wins. + const answer = { + version: 1, + focus: { model: 'model.shop.fct_orders', column: 'revenue' }, + direction: 'downstream', + counts: { columns: 3, models: 2, hops: 2, contracts: 0, ends: 2, may_carry_models: 4, cleared_models: 1, model_level: 6 }, + nodes: [ + { id: 'model.shop.fct_orders', name: 'fct_orders', exposures: [] }, + { id: 'model.shop.fct_revenue_daily', name: 'fct_revenue_daily', exposures: ['exec_revenue'] }, + { id: 'model.shop.rpt_finance_weekly', name: 'rpt_finance_weekly', exposures: [] }, + { id: 'model.shop.rpt_broken', name: 'rpt_broken', exposures: [] }, + ], + columns: [ + { model: 0, column: 'revenue', hop: 0, how: null, how_from: null }, + { model: 1, column: 'revenue', hop: 1, end: false, how: '=', how_from: 'engine' }, + { model: 1, column: 'daily_revenue', hop: 1, end: true, how: '\u03a3', how_from: 'engine' }, + { model: 2, column: 'weekly_revenue', hop: 2, end: true, how: '\u03a3', how_from: 'engine' }, + ], + edges: [[0, 1, '='], [0, 2, '\u03a3'], [1, 3, '\u03a3']], + also_reads: [], + unresolved: [{ model: 3, reason: 'parse error' }], + cleared: [{ id: 'model.shop.dim_dates', name: 'dim_dates' }], + truncated: false, + } + const { clock, notes } = await start($, on, { reach: { ...REACH, column_answer: answer } }) + const result: any = await $.tool.call({ tool: 'Edit', file_path: FILE, old_string: 'x as revenue', new_string: 'y as revenue' }) + await clock.settle() + const note = result.context?.[0] as string + expect(note).toContain('- revenue: its value reaches 3 columns in 2 downstream models (2 end columns): fct_revenue_daily (revenue, daily_revenue), rpt_finance_weekly (weekly_revenue)') + expect(note).toContain(' dashboards on them: exec_revenue') + expect(note).toContain(' 4 more models may carry it (they read it outside the select, or sit below a model that could not be read); 1 other downstream model does not carry it; 6 downstream models at model level') + expect(note).toContain(' lower bound: 1 model could not be read: rpt_broken ("parse error")') + // The older reach numbers are not given alongside. + expect(note).not.toContain('read by 2 downstream models') + expect(notes[0]).toBe('◆ impact-first · fct_orders.revenue reaches 3 columns in 2 models, 4 may carry it · exec_revenue') +}) + test('[c] picks a column and shows its reach; [d] unfolds downstream; a model in the graph is followed on press', async ($, on) => { const { clock, calls } = await start($, on) await $.tool.call({ tool: 'Read', file_path: FILE }) @@ -450,7 +526,7 @@ test('a model whose SQL changed after the manifest was written says so; r keeps test('a shell command that changes the shown model\'s file brings the warning up, and the edit note says the manifest is older', async ($, on) => { let fileMtime = 1 - const { clock, toasts } = await start($, on, { fileMtime: path => (path === FILE ? fileMtime : undefined) }) + const { clock, notes } = await start($, on, { fileMtime: path => (path === FILE ? fileMtime : undefined) }) await $.tool.call({ tool: 'Read', file_path: FILE }) await clock.settle() const ui = await $.ui.mount({ ...PANE, surface: 'terminal' }) @@ -464,7 +540,7 @@ test('a shell command that changes the shown model\'s file brings the warning up // An edit on top of it: Claude is told the lineage may be out of date. const result: any = await $.tool.call({ tool: 'Edit', file_path: FILE, old_string: 'sum(amount) as revenue', new_string: 'sum(amount) / 100 as revenue' }) await clock.settle() - expect(toasts.at(-1)).toMatch(/ · the manifest is older than this file$/) + expect(notes.at(-1)).toMatch(/ · the manifest is older than this file$/) expect(result.context?.[0]).toContain('- the manifest is older than this file (it changed after the last `dbt compile`)') }) @@ -507,15 +583,15 @@ test('a graph too big for Claude Code to draw is shown as folders, saying why, n test("an edit of a column the project doesn't declare is traced from the model's own columns, wherever the pane is", async ($, on) => { // net_revenue is in fct_orders' SQL but not in its schema.yml, and the pane shows another model. const cliLineage = { model_name: 'fct_orders', confidence: 'high', columns: { order_id: ['"int_orders"."order_id"'], net_revenue: ['"int_orders"."revenue"'] }, edges: [] } - const { clock, calls, toasts } = await start($, on, { cliLineage }) + const { clock, calls, notes } = await start($, on, { cliLineage }) await $.tool.call({ tool: 'Read', file_path: `${ROOT}/models/int_orders.sql` }) await clock.settle() const result: any = await $.tool.call({ tool: 'Edit', file_path: FILE, old_string: 'sum(amount) as net_revenue', new_string: 'sum(amount) - sum(fees) as net_revenue' }) await clock.settle() // Its columns come through the CLI (no permission dialog), before the reach. expect(calls.indexOf('cli:dbt_lineage')).toBeGreaterThan(-1) - expect(calls.indexOf('cli:dbt_lineage')).toBeLessThan(calls.indexOf('mcp:impact_analysis:net_revenue')) - expect(toasts.at(-1)).toMatch(/^◆ impact-first · fct_orders\.net_revenue is read by 2 models/) + expect(calls.indexOf('cli:dbt_lineage')).toBeLessThan(calls.indexOf('cli:impact_analysis:net_revenue')) + expect(notes.at(-1)).toMatch(/^◆ impact-first · fct_orders\.net_revenue is read by 2 models/) expect(result.context?.[0]).toContain('Altimate lineage for fct_orders.net_revenue') // The next edit of the same model uses the columns it already has. await $.tool.call({ tool: 'Edit', file_path: FILE, old_string: 'sum(fees) as net_revenue', new_string: 'sum(fees) + 1 as net_revenue' }) @@ -524,30 +600,41 @@ test("an edit of a column the project doesn't declare is traced from the model's }) test('an edit whose declared columns already show what it touches does not wait for the CLI', async ($, on) => { - const { clock, calls, toasts } = await start($, on) + const { clock, calls, notes } = await start($, on) await $.tool.call({ tool: 'Edit', file_path: FILE, old_string: 'sum(amount) as revenue', new_string: 'sum(amount) / 100 as revenue' }) await clock.settle() // revenue is declared: traced at once, while the model's columns are fetched for later edits. - expect(calls.indexOf('mcp:impact_analysis:revenue')).toBeLessThan(calls.indexOf('tool:Edit')) - expect(toasts[0]).toMatch(/^◆ impact-first · fct_orders\.revenue is read by 2 models/) + expect(calls.indexOf('cli:impact_analysis:revenue')).toBeGreaterThan(-1) + expect(calls.indexOf('cli:impact_analysis:revenue')).toBeLessThan(calls.indexOf('tool:Edit')) + expect(notes[0]).toMatch(/^◆ impact-first · fct_orders\.revenue is read by 2 models/) }) -test("a column reach Claude Code saved to a file is read from it, with no CLI call", async ($, on) => { - const { clock, calls, toasts } = await start($, on, { persisted: { readable: true } }) - await $.tool.call({ tool: 'Edit', file_path: FILE, old_string: 'sum(amount) as revenue', new_string: 'sum(amount) / 100 as revenue' }) +test("the pane's column reach Claude Code saved to a file is read from it, with no CLI call", async ($, on) => { + const { clock, calls, logs } = await start($, on, { persisted: { readable: true } }) + await $.command.run({ command: 'lineage', args: 'fct_orders.revenue', origin: { kind: 'composer' }, presentation: { isFullscreen: false, columns: 120 } } as any) await clock.settle() - expect(toasts[0]).toBe('◆ impact-first · fct_orders.revenue is read by 2 models and 1 dashboard · Exec Revenue') - expect(calls).not.toContain('cli:impact_analysis') + expect(calls).toContain('mcp:impact_analysis:revenue') + expect(calls).not.toContain('cli:impact_analysis:revenue') + expect(logs.some(line => line.includes('impact_analysis(fct_orders, column: revenue) · read by 2 models'))).toBe(true) }) -test('a saved long answer the plugin cannot read is asked of the CLI, not taken as no answer', async ($, on) => { - const { clock, calls, toasts } = await start($, on, { persisted: { readable: false } }) - await $.tool.call({ tool: 'Edit', file_path: FILE, old_string: 'sum(amount) as revenue', new_string: 'sum(amount) / 100 as revenue' }) +test("the pane's saved long answer the plugin cannot read is asked of the CLI, not taken as no answer", async ($, on) => { + const { clock, calls, logs } = await start($, on, { persisted: { readable: false } }) + await $.command.run({ command: 'lineage', args: 'fct_orders.revenue', origin: { kind: 'composer' }, presentation: { isFullscreen: false, columns: 120 } } as any) await clock.settle() - expect(calls).toContain('cli:impact_analysis') - expect(toasts[0]).toBe('◆ impact-first · fct_orders.revenue is read by 2 models and 1 dashboard · Exec Revenue') + expect(calls.indexOf('mcp:impact_analysis:revenue')).toBeLessThan(calls.indexOf('cli:impact_analysis:revenue')) + expect(calls.indexOf('mcp:impact_analysis:revenue')).toBeGreaterThan(-1) + expect(logs.some(line => line.includes('impact_analysis(fct_orders, column: revenue) · read by 2 models'))).toBe(true) }) +test('an edit asks the CLI for its column trace, never MCP, even with the MCP server up', async ($, on) => { + const { clock, calls, notes } = await start($, on, { persisted: { readable: true } }) + await $.tool.call({ tool: 'Edit', file_path: FILE, old_string: 'sum(amount) as revenue', new_string: 'sum(amount) / 100 as revenue' }) + await clock.settle() + expect(calls).toContain('cli:impact_analysis:revenue') + expect(calls.filter(call => call.startsWith('mcp:impact_analysis'))).toEqual([]) + expect(notes[0]).toBe('◆ impact-first · fct_orders.revenue is read by 2 models and 1 dashboard · Exec Revenue') +}) test("a model's columns the CLI could not give are not asked again for the same manifest", async ($, on) => { // Denied edits: the pane follows no model, so nothing but the CLI could give the columns. const { clock, calls } = await start($, on, { cliFailure: true, toolResult: e => (e.tool === 'Edit' ? { deny: 'not now' } : undefined) }) @@ -575,7 +662,7 @@ test("a CLI column answer that fails late never replaces the columns the pane re // An edit of net_revenue (undeclared, known from the pane's read) is still traced. await $.tool.call({ tool: 'Edit', file_path: FILE, old_string: 'sum(amount) as net_revenue', new_string: 'sum(amount) - 1 as net_revenue' }) await clock.settle() - expect(calls).toContain('mcp:impact_analysis:net_revenue') + expect(calls).toContain('cli:impact_analysis:net_revenue') }) test('a graph found too big is not drawn again when the pane goes back to a width it had', async ($, on) => { @@ -584,10 +671,10 @@ test('a graph found too big is not drawn again when the pane goes back to a widt const name = `order_slice_${String(i).padStart(4, '0')}` nodes[`model.shop.${name}`] = { resource_type: 'model', name, original_file_path: `models/fan/${name}.sql`, depends_on: { nodes: ['model.shop.fct_orders'] }, config: { materialized: 'view' } } } - const { clock, logs } = await start($, on, { manifest: { ...MANIFEST, nodes } }) + const { clock, debug } = await start($, on, { manifest: { ...MANIFEST, nodes } }) await $.command.run({ command: 'lineage', args: 'fct_orders', origin: { kind: 'composer' }, presentation: { isFullscreen: true, columns: 200 } }) await clock.settle() - const measured = () => logs.filter(line => line.includes('too big to draw')).length + const measured = () => debug.filter(line => line.includes('too big to draw')).length for (const bodyColumns of [90, 91, 90, 91]) { const ui = await $.ui.mount({ ...PANE, props: { ...PANE.props, bodyColumns }, surface: 'terminal' }) expect(await ui.find({ type: 'Text', text: /^FOLDERS · manifest/ })).toBeDefined() @@ -619,19 +706,16 @@ test('an edit whose column no model reads for certain never says "at least 0", a truncated: false, }, } - const { clock, toasts } = await start($, on, { reach }) + const { clock, notes } = await start($, on, { reach }) const result: any = await $.tool.call({ tool: 'Edit', file_path: FILE, old_string: 'sum(amount) as revenue', new_string: 'sum(amount) / 100 as revenue' }) await clock.settle() - expect(toasts[0]).toBe('◆ impact-first · fct_orders.revenue is read by no model for certain (1 may); 1 dashboard may read it · Exec Revenue') + expect(notes[0]).toBe('◆ impact-first · fct_orders.revenue is read by no model for certain (1 may); 1 dashboard may read it · Exec Revenue') expect(result.context?.[0]).toContain('- read by no downstream model for certain (1 may)') - const band = await $.ui.mount({ ...BAND, surface: 'terminal' }) - expect(await band.find({ type: 'Text', text: /read by no model for certain \(1 may\) · no dashboard for certain \(1 may\)/ })).toBeDefined() - expect(await band.find({ type: 'Text', text: /at least 0|dashboards/ })).toBeUndefined() - await band.unmount() + expect(`${notes[0]}\n${result.context?.[0]}`).not.toMatch(/at least 0/) }) test('a manifest over 4 MiB is read through altimate-code: the graph comes from dbt_lineage with include_graph', async ($, on) => { - const { clock, calls, toasts } = await start($, on, { size: 6 * 1024 * 1024 }) + const { clock, calls, notes } = await start($, on, { size: 6 * 1024 * 1024 }) await $.tool.call({ tool: 'Read', file_path: FILE }) await clock.settle() expect(calls).toEqual(['tool:Read', 'cli:dbt_lineage:graph']) @@ -644,7 +728,7 @@ test('a manifest over 4 MiB is read through altimate-code: the graph comes from // Impact-first works on the same path. await $.tool.call({ tool: 'Edit', file_path: FILE, old_string: 'x as revenue', new_string: 'y as revenue' }) await clock.settle() - expect(toasts[0]).toContain('fct_orders.revenue is read by 2 models') + expect(notes[0]).toContain('fct_orders.revenue is read by 2 models') }) test('a recompiled manifest is a new focus: the column lineage is read again', async ($, on) => { @@ -663,17 +747,14 @@ test('a recompiled manifest is a new focus: the column lineage is read again', a test('a subagent gets the lineage context the main agent already got, and an untraced model keeps the count a lower bound', async ($, on) => { const reach = { ...REACH, column_reach: { ...REACH.column_reach, untraced: [{ name: 'mart_cohort_ltv', reason: 'No compiled SQL found' }], uncertain: [] } } - const { clock, toasts } = await start($, on, { reach }) + const { clock, notes } = await start($, on, { reach }) const edit = { tool: 'Edit' as const, file_path: FILE, old_string: 'x as revenue', new_string: 'y as revenue' } const main: any = await $.tool.call(edit) const sub: any = await $.tool.call({ ...edit, agentId: 'sub-1' } as any) await clock.settle() - expect(toasts[0]).toContain('is read by 2 models (1 more may)') + expect(notes[0]).toContain('is read by 2 models (1 more may)') expect(main.context?.[0]).toContain('could not be traced: mart_cohort_ltv ("No compiled SQL found")') expect(sub.context?.[0]).toContain('read by 2 downstream models (1 more may)') - const band = await $.ui.mount({ ...BAND, surface: 'terminal' }) - expect(await band.find({ type: 'Text', text: /read by 2 models \(1 more may\)/ })).toBeDefined() - await band.unmount() }) test('a read Claude makes outside a dbt project stays quiet', async ($, on) => { @@ -693,7 +774,7 @@ test('a result the MCP server left out for size is fetched again through the CLI await run('fct_orders.revenue') await clock.settle() expect(calls).toContain('mcp:impact_analysis:revenue') - expect(calls).toContain('cli:impact_analysis') + expect(calls).toContain('cli:impact_analysis:revenue') const ui = await $.ui.mount({ ...PANE, surface: 'terminal' }) expect(await ui.find({ type: 'Text', text: /read by 2 models · 1 dashboard(?!s)/ })).toBeDefined() await ui.unmount() @@ -723,10 +804,10 @@ test("a dbt_lineage result Claude's call got cut for size leaves the pane to fet test('dashboards on models that may read the column are counted and named', async ($, on) => { const reach = { ...REACH, column_reach: { ...REACH.column_reach, uncertain: [{ name: 'rpt_ops', reason: 'names it only outside its select list' }], uncertain_exposures: [{ name: 'ops', label: 'Ops Board' }] } } - const { clock, toasts } = await start($, on, { reach }) + const { clock, notes } = await start($, on, { reach }) const edit: any = await $.tool.call({ tool: 'Edit', file_path: FILE, old_string: 'x as revenue', new_string: 'y as revenue' }) await clock.settle() - expect(toasts[0]).toBe('◆ impact-first · fct_orders.revenue is read by 2 models (1 more may) and 1 dashboard; 1 dashboard may read it · Exec Revenue · Ops Board') + expect(notes[0]).toBe('◆ impact-first · fct_orders.revenue is read by 2 models (1 more may) and 1 dashboard; 1 dashboard may read it · Exec Revenue · Ops Board') expect(edit.context?.[0]).toContain('dashboards on models that may read it: "Ops Board"') }) @@ -744,28 +825,29 @@ test('/lineage on the same model with another column traces that column', async await ui.unmount() }) -test('the impact toast shows before the edit runs', async ($, on) => { - const { clock, toasts } = await start($, on, { toolResult: e => (e.tool === 'Edit' ? { toastsSoFar: toasts.length } : undefined) }) +test('the impact note is logged before the edit runs', async ($, on) => { + const { clock, notes } = await start($, on, { toolResult: e => (e.tool === 'Edit' ? { notesSoFar: notes.length } : undefined) }) const result: any = await $.tool.call({ tool: 'Edit', file_path: FILE, old_string: 'x as revenue', new_string: 'y as revenue' }) await clock.settle() - expect(result.result).toEqual({ toastsSoFar: 1 }) + expect(result.result).toEqual({ notesSoFar: 1 }) }) test('an edit touching two columns traces both and reports their combined reach', async ($, on) => { - const { clock, calls, toasts } = await start($, on) + const { clock, calls, notes } = await start($, on) await $.tool.call({ tool: 'Edit', file_path: FILE, old_string: 'i.revenue, i.order_id', new_string: 'coalesce(i.revenue, 0) as revenue, i.order_id + 0 as order_id' }) await clock.settle() - expect(calls).toContain('mcp:impact_analysis:revenue') - expect(calls).toContain('mcp:impact_analysis:order_id') - expect(toasts[0]).toContain('fct_orders.') - expect(toasts[0]).toContain('is read by 2 models and 1 dashboard') + expect(calls).toContain('cli:impact_analysis:revenue') + expect(calls).toContain('cli:impact_analysis:order_id') + expect(calls.filter(call => call.startsWith('mcp:impact_analysis'))).toEqual([]) + expect(notes[0]).toContain('fct_orders.') + expect(notes[0]).toContain('is read by 2 models and 1 dashboard') }) test('a touched column whose check fails keeps the combined count a lower bound', async ($, on) => { - const { clock, toasts } = await start($, on, { failColumn: 'order_id' }) + const { clock, notes } = await start($, on, { failColumn: 'order_id' }) const edit: any = await $.tool.call({ tool: 'Edit', file_path: FILE, old_string: 'i.revenue, i.order_id', new_string: 'coalesce(i.revenue, 0) as revenue, i.order_id + 0 as order_id' }) await clock.settle() - expect(toasts[0]).toContain('fct_orders.revenue is read by at least 2 models') + expect(notes[0]).toContain('fct_orders.revenue is read by at least 2 models') expect(edit.context?.[0]).toContain('columns not checked (over the limit or no answer in time): order_id') }) @@ -779,7 +861,7 @@ test('a new unchecked column in the same turn still reaches Claude', async ($, o }) test('columns beyond the cap are named as unchecked and make the count a lower bound', async ($, on) => { - const { clock, calls, toasts } = await start($, on) + const { clock, calls, notes } = await start($, on) const edit: any = await $.tool.call({ tool: 'Edit', file_path: FILE, @@ -787,8 +869,8 @@ test('columns beyond the cap are named as unchecked and make the count a lower b new_string: 'select order_id + 0 as order_id, revenue + 0 as revenue, gross_revenue + 0 as gross_revenue, status + 0 as status', }) await clock.settle() - expect(calls.filter(call => call.startsWith('mcp:impact_analysis'))).toHaveLength(3) - expect(toasts[0]).toContain('is read by at least 2 models') + expect(calls.filter(call => call.startsWith('cli:impact_analysis'))).toHaveLength(3) + expect(notes[0]).toContain('is read by at least 2 models') expect(edit.context?.[0]).toMatch(/columns not checked \(over the limit or no answer in time\): \w+;/) }) @@ -1436,12 +1518,13 @@ test('when the whole lookup runs out of time, a toast says the edit went ahead w expect(toasts).toContain('◆ impact-first · the lineage lookup did not finish in time; the edit went ahead without it') }) -test('a .sql file outside any dbt project gets no timeout toast', async ($, on) => { - const { clock, toasts } = await start($, on) +test('a .sql file outside any dbt project gets no timeout note', async ($, on) => { + const { clock, notes, toasts } = await start($, on) const call = $.tool.call({ tool: 'Edit', file_path: '/elsewhere/migrations/001.sql', old_string: 'a', new_string: 'b' }) await clock.advance(9_000) await call await clock.settle() + expect(notes).toEqual([]) expect(toasts).toEqual([]) }) @@ -1525,14 +1608,11 @@ test("project text in the impact note is one line per name, quoted, capped, and }) test('an unreadable dashboards section keeps the dashboard count a lower bound', async ($, on) => { - const { clock, toasts } = await start($, on, { reach: { ...REACH, exposures_error: 'exposures: bad yaml' } }) + const { clock, notes } = await start($, on, { reach: { ...REACH, exposures_error: 'exposures: bad yaml' } }) const edit: any = await $.tool.call({ tool: 'Edit', file_path: FILE, old_string: 'x as revenue', new_string: 'y as revenue' }) await clock.settle() - expect(toasts[0]).toBe('◆ impact-first · fct_orders.revenue is read by 2 models and at least 1 dashboard · Exec Revenue') + expect(notes[0]).toBe('◆ impact-first · fct_orders.revenue is read by 2 models and at least 1 dashboard · Exec Revenue') expect(edit.context?.[0]).toContain('the dashboards could not be read ("exposures: bad yaml")') - const band = await $.ui.mount({ ...BAND, surface: 'terminal' }) - expect(await band.find({ type: 'Text', text: /at least 1 dashboard(?!s)/ })).toBeDefined() - await band.unmount() }) test('no column trace starts once the hold has run out', async ($, on) => { @@ -1543,7 +1623,7 @@ test('no column trace starts once the hold has run out', async ($, on) => { await late release() await clock.settle() - expect(calls.filter(one => one.startsWith('mcp:impact_analysis'))).toEqual([]) + expect(calls.filter(one => one.includes('impact_analysis'))).toEqual([]) }) test('two edits of one column while its trace runs share that trace', async ($, on) => { @@ -1555,7 +1635,7 @@ test('two edits of one column while its trace runs share that trace', async ($, answer() await Promise.all([first, second]) await both.clock.settle() - expect(both.calls.filter(one => one === 'mcp:impact_analysis:revenue')).toEqual(['mcp:impact_analysis:revenue']) + expect(both.calls.filter(one => one === 'cli:impact_analysis:revenue')).toEqual(['cli:impact_analysis:revenue']) }) test('a model folder outside the project (`../..`) is never walked', async ($, on) => { @@ -1632,15 +1712,13 @@ test('bare /lineage asks for warehouse lineage on the model shown', async ($, on expect(calls).toContain('mcp:table_lineage') }) -test('an MCP trace that never answers is given up on, so the next edit of that column traces again', async ($, on) => { - const { clock, calls } = await start($, on, { mcpHang: 'impact_analysis' }) - const first = $.tool.call({ tool: 'Edit', file_path: FILE, old_string: 'x as revenue', new_string: 'y as revenue' }) - await clock.advance(9_000) - await first - await clock.settle() - const second = $.tool.call({ tool: 'Edit', file_path: FILE, old_string: 'y as revenue', new_string: 'z as revenue' }) - await clock.advance(9_000) - await second +test("the pane's MCP column trace that never answers is given up on, and the CLI answers instead", async ($, on) => { + const { clock, calls, logs } = await start($, on, { mcpHang: 'impact_analysis' }) + const asked = $.command.run({ command: 'lineage', args: 'fct_orders.revenue', origin: { kind: 'composer' }, presentation: { isFullscreen: false, columns: 120 } } as any) + await clock.advance(61_000) + await asked await clock.settle() - expect(calls.filter(one => one === 'mcp:impact_analysis:revenue')).toEqual(['mcp:impact_analysis:revenue', 'mcp:impact_analysis:revenue']) + expect(calls.indexOf('mcp:impact_analysis:revenue')).toBeGreaterThan(-1) + expect(calls.indexOf('mcp:impact_analysis:revenue')).toBeLessThan(calls.indexOf('cli:impact_analysis:revenue')) + expect(logs.some(line => line.includes('impact_analysis(fct_orders, column: revenue) · read by 2 models'))).toBe(true) }) diff --git a/plugins/altimate-code/hooks/lineage.tsx b/plugins/altimate-code/hooks/lineage.tsx index eb41d73..fca1ac5 100644 --- a/plugins/altimate-code/hooks/lineage.tsx +++ b/plugins/altimate-code/hooks/lineage.tsx @@ -1,5 +1,5 @@ import { atom, read, update } from 'claude-code' -import type { EngineInterface, On, RenderElement } from 'claude-code' +import type { EngineInterface, On } from 'claude-code' import type { AnswerModel, @@ -36,6 +36,8 @@ import { upstreamArt, warehouseEdges, } from './lineage-graph' +import { answerOf, carriersOf, type ColumnAnswer, reachOf } from './impact-report' +import { taskBeforeTool, taskTurnComplete, taskTurnStart } from './task-impact' const PANE = 'altimate-lineage' const PANE_TITLE = 'Altimate lineage' @@ -77,14 +79,10 @@ let revealFor: string | null = null const answerAtom = atom({ plugin: 'altimate-code', key: 'lineageAnswer' } as const, [] as AnswerModel[]) // Where the graph canvas is panned to: its first visible column; null centres it on the model. const panAtom = atom({ plugin: 'altimate-code', key: 'lineagePan' } as const, null as number | null) -// lineage-impact-2 (1.4.2): `exposures` holds the may-read dashboards too, counted by `maybeExposures`. A band kept -// from 1.4.1 can't tell them apart, so it reads as absent. -const impactAtom = atom({ plugin: 'altimate-code', key: 'lineageImpact' } as const, null, { shape: 'lineage-impact-2' }) const messageAtom = atom({ plugin: 'altimate-code', key: 'lineageMessage' } as const, null as string | null) // The shown model's SQL file changed after the manifest was written: what the pane shows may be out of date. const staleAtom = atom({ plugin: 'altimate-code', key: 'lineageStale' } as const, null as string | null) const loadingAtom = atom({ plugin: 'altimate-code', key: 'lineageLoading' } as const, false) -const bandHiddenAtom = atom({ plugin: 'altimate-code', key: 'isLineageBandHidden' } as const, false) // Caches, rebuilt from the manifest file after a reload. let manifestCache: { path: string; mtimeMs: number; graph: LineageGraph } | null = null @@ -154,7 +152,7 @@ const SEVERITY_COLOR: Record = { } /** Resolves with the work's value, or null once `ms` pass or it fails. */ -function within($: EngineInterface, ms: number, work: Promise): Promise { +export function within($: EngineInterface, ms: number, work: Promise): Promise { return new Promise(resolve => { let settled = false const settle = (value: T | null) => { @@ -186,7 +184,7 @@ const dirname = (path: string) => path.replace(/[\\/][^\\/]*$/, '') || path const relative = (root: string, path: string) => path.slice(root.length).replace(/^[\\/]+/, '').replace(/\\/g, '/') /** The dbt project above a file or folder: the closest folder holding dbt_project.yml. */ -async function projectRoot($: EngineInterface, start: string, isFile: boolean): Promise { +export async function projectRoot($: EngineInterface, start: string, isFile: boolean): Promise { let dir = isFile ? dirname(start) : start for (let i = 0; i < 40; i++) { if (await $.fs.exists(`${dir}/dbt_project.yml`).catch(() => false)) return dir @@ -257,7 +255,7 @@ async function projectWith($: EngineInterface, start: string, model: string, pre * backticks (a bare `orders` in prose is too often just a word). Read from the project shown, else the * session's. */ -async function modelsNamed($: EngineInterface, text: string, root: string, shownManifest?: string): Promise { +export async function modelsNamed($: EngineInterface, text: string, root: string, shownManifest?: string): Promise { const manifest = shownManifest ?? `${root}/target/manifest.json` const fromManifest = await readGraph($, manifest).catch(() => null) // The manifest an entry keeps: the shown graph's (read here, or through altimate-code when over @@ -451,7 +449,7 @@ function fileUriPath(uri: string): string { * `cliOnly` goes straight to the CLI, whose --json carries the whole metadata; a structured result * the server left out for size is fetched that way too. */ -async function runTool($: EngineInterface, name: string, args: Record, cliOnly = false, until?: number) { +export async function runTool($: EngineInterface, name: string, args: Record, cliOnly = false, until?: number) { // With a deadline, nothing starts once it has passed, and the CLI is stopped at it. const left = async () => { const ms = until === undefined ? TOOL_TIMEOUT_MS : until - (await $.clock.now()) @@ -888,7 +886,7 @@ async function narrowHint($: EngineInterface, reason: string) { $.ui.toast(`◆ lineage · /lineage opens the pane (it opens by itself ${floor ? `from ${floor} columns` : 'in a wider terminal'})`, { timeoutMs: 10000 }) } -async function openPane($: EngineInterface) { +export async function openPane($: EngineInterface) { paneOpened = true await $.ui.open({ id: PANE, title: PANE_TITLE, ...PANE_SIZE, focus: true, closeOnEscape: true }) } @@ -954,7 +952,7 @@ function cut(text: string, max: number): string { /** * Before an edit to a model's SQL: what reads the model, and, when the edit changes one of its - * columns, which downstream models and dashboards read that column. Shown as a toast, then added to + * columns, which downstream models and dashboards read that column. Logged, then added to * Claude's context with the edit's result. */ async function impactFirst($: EngineInterface, e: any, file: string, deadline: number, progress: { project: boolean; failed?: boolean }): Promise { @@ -1004,6 +1002,7 @@ async function impactFirst($: EngineInterface, e: any, file: string, deadline: n const touched = everyTouched.slice(0, MAX_EDIT_COLUMNS) let reach: ColumnReachView | null = null + const answered: { column: string; answer: ColumnAnswer }[] = [] const traced: string[] = [] // Touched columns whose check failed or ran out of time: what they reach is unknown. // Columns beyond the cap are never traced either. @@ -1019,31 +1018,49 @@ async function impactFirst($: EngineInterface, e: any, file: string, deadline: n const left = Math.min(HOLD_MS, deadline - (await $.clock.now())) if (left <= 0) return null const args = { manifest_path: manifest, model: id, column: col, change_type: 'modify' } - return within($, left, shared(`impact\u0000${manifest}\u0000${stat.mtimeMs}\u0000${id}\u0000${col}`, () => runTool($, 'impact_analysis', args, false, deadline))) + // Through the CLI, not MCP: no permission prompt in accept-edits mode, and no size cap on the answer. + return within($, left, shared(`impact\u0000${manifest}\u0000${stat.mtimeMs}\u0000${id}\u0000${col}`, () => runTool($, 'impact_analysis', args, true, deadline))) }), ) - const reaches = results.map((result, i) => (result?.ok ? reachFromMetadata(result.metadata, node.name, touched[i]!) : null)) - reaches.forEach((one, i) => (one ? traced : unchecked).push(touched[i]!)) - reach = mergeReach(reaches.filter((one): one is ColumnReachView => !!one)) + // The column answer, as the end-of-task summary reads it, so Claude and the person see one set of + // numbers; the older column reach only from an altimate-code that sends no column answer. + const answers = results.map(result => (result?.ok ? answerOf(result.metadata) : null)) + if (answers.some(Boolean)) { + answers.forEach((one, i) => { + ;(one ? traced : unchecked).push(touched[i]!) + if (one) answered.push({ column: touched[i]!, answer: one }) + }) + } else { + const reaches = results.map((result, i) => (result?.ok ? reachFromMetadata(result.metadata, node.name, touched[i]!) : null)) + reaches.forEach((one, i) => (one ? traced : unchecked).push(touched[i]!)) + reach = mergeReach(reaches.filter((one): one is ColumnReachView => !!one)) + } const ms = Math.round((await $.clock.now()) - began) - if (reach) $.ui.log(`impact-first · impact_analysis(${node.name}.${traced.join(', ')}) · ${elapsed(ms)}`) + if (traced.length) $.ui.log(`impact-first · impact_analysis(${node.name}.${traced.join(', ')}) · ${elapsed(ms)}`, { to: 'debug' }) } const column = traced.length ? traced.join(', ') : null - const target = column && reach ? `${node.name}.${column}` : node.name + const target = column && (reach || answered.length) ? `${node.name}.${column}` : node.name // What the engine could not settle keeps the count a lower bound, never a confident zero. const maybe = reach ? reach.uncertain.length + reach.untraced.length : 0 const incomplete = !!reach && (reach.truncated || unchecked.length > 0) const unsettled = reach ? maybe + (reach.truncated ? 1 : 0) + unchecked.length : 0 + const carried = answered.map(one => ({ ...one, reach: reachOf(one.answer), carriers: carriersOf(one.answer) })) // Dashboards on models that may read the column are listed too, and said to be a maybe. const maybeDashboards = reach ? reach.uncertainExposures : [] - const dashboards = reach ? [...reach.exposures, ...maybeDashboards] : counts.exposures + const dashboards = reach + ? [...reach.exposures, ...maybeDashboards] + : carried.length + ? [...new Set(carried.flatMap(one => one.carriers.flatMap(model => model.exposures)))] + : counts.exposures const toast = reach ? `${target} is read by ${countText(reach.models.length, maybe, incomplete)}` + (reach.exposures.length ? ` and ${reach.exposuresError ? 'at least ' : ''}${plural(reach.exposures.length, 'dashboard')}` : '') + (maybeDashboards.length ? `; ${plural(maybeDashboards.length, 'dashboard')} may read it` : '') + (reach.exposuresError && !reach.exposures.length ? '; its dashboards could not be read' : '') - : `${target} feeds ${graph.truncated ? 'at least ' : ''}${plural(downstream, 'model')}${dashboards.length ? ` and ${plural(dashboards.length, 'dashboard')}` : ''}` + : carried.length + ? carried.map(one => `${node.name}.${one.column} reaches ${plural(one.reach.columns, 'column')} in ${plural(one.reach.models, 'model')}, ${one.reach.mayCarry} may carry it`).join('; ') + : `${target} feeds ${graph.truncated ? 'at least ' : ''}${plural(downstream, 'model')}${dashboards.length ? ` and ${plural(dashboards.length, 'dashboard')}` : ''}` const toastText = `◆ impact-first · ${toast}${dashboards.length ? ` · ${dashboards.slice(0, 3).join(' · ')}` : ''}${stale ? ' · the manifest is older than this file' : ''}`.replace(CONTROL, ' ').slice(0, 300) // Names and labels come from the project's files, which anyone who wrote the repo controls: each is @@ -1059,6 +1076,21 @@ async function impactFirst($: EngineInterface, e: any, file: string, deadline: n if (reach.uncertain.length) lines.push(`- may also read it: ${capped(reach.uncertain.map(one => `${quoted(one.name)} (${quoted(one.reason)})`), 10)}`) if (reach.untraced.length) lines.push(`- could not be traced: ${capped(reach.untraced.map(one => `${quoted(one.name)} (${quoted(one.reason)})`), 10)}`) } + for (const one of carried) { + const r = one.reach + const named = one.carriers.map(model => `${quoted(model.name)} (${listed(model.columns, 8)})`) + lines.push(`- ${quoted(one.column)}: its value reaches ${plural(r.columns, 'column')} in ${plural(r.models, 'downstream model')} (${plural(r.ends, 'end column')})${named.length ? `: ${capped(named, 15)}` : ''}`) + const exposures = [...new Set(one.carriers.flatMap(model => model.exposures))] + if (exposures.length) lines.push(` dashboards on them: ${listed(exposures)}`) + lines.push( + ` ${plural(r.mayCarry, 'more model')} may carry it (they read it outside the select, or sit below a model that could not be read); ${r.cleared === null ? 'the other downstream models do' : `${plural(r.cleared, 'other downstream model')} ${r.cleared === 1 ? 'does' : 'do'}`} not carry it${r.modelLevel === null ? '' : `; ${plural(r.modelLevel, 'downstream model')} at model level`}`, + ) + const unresolved = one.answer.unresolved ?? [] + if (unresolved.length || r.truncated) { + const why = unresolved.map(entry => `${quoted(String(one.answer.nodes[entry.model]?.name ?? entry.model))} (${quoted(String(entry.reason ?? 'not read'))})`) + lines.push(` lower bound: ${plural(unresolved.length, 'model')} could not be read${why.length ? `: ${capped(why, 10)}` : ''}${r.truncated ? '; the trace stopped at its limit' : ''}`) + } + } lines.push( `- ${quoted(node.name)} feeds ${graph.truncated ? 'at least ' : ''}${plural(downstream, 'downstream model')} in all (${counts.direct} direct, ${counts.transitive} transitive), ${plural(counts.tests, 'test')}${counts.exposures.length ? ` and ${plural(counts.exposures.length, 'exposure')} (${listed(counts.exposures)})` : ''}.`, ) @@ -1074,15 +1106,15 @@ async function impactFirst($: EngineInterface, e: any, file: string, deadline: n context: `${body.length > MAX_NOTE ? `${cut(body, MAX_NOTE)}...` : body}\n${close}`, view: { model: node.name, - column: reach ? column : null, + column: reach || carried.length ? column : null, severity: counts.severity, direct: counts.direct, transitive: counts.transitive, tests: counts.tests, exposures: dashboards, - readers: reach ? reach.models.length : null, - unsettled, - maybeReaders: maybe, + readers: reach ? reach.models.length : carried.length ? new Set(carried.flatMap(one => one.carriers.map(model => model.name))).size : null, + unsettled: unsettled + carried.reduce((n, one) => n + one.reach.unresolved + one.reach.mayCarry, 0), + maybeReaders: maybe + carried.reduce((n, one) => n + one.reach.mayCarry, 0), maybeExposures: maybeDashboards.length, ...(reach?.exposuresError && { dashboardsUnsettled: true }), truncated: !!graph.truncated, @@ -1164,39 +1196,6 @@ export async function startLineage($: EngineInterface) { type Ui = { Box: any; Button: any; Text: any } -/** The band line after Claude edits a model; drawn by the plugin's one band hook. */ -export async function lineageBand($: EngineInterface, ui: Ui): Promise { - const impact = await read($, impactAtom) - if (!impact || (await read($, bandHiddenAtom))) return null - const { Box, Button, Text } = ui - const target = impact.column ? `${impact.model}.${impact.column}` : impact.model - const readers = impact.readers !== null ? countText(impact.readers, impact.maybeReaders, impact.unsettled > impact.maybeReaders) : null - const dashboards = - impact.exposures.length || impact.dashboardsUnsettled ? countText(impact.exposures.length - impact.maybeExposures, impact.maybeExposures, !!impact.dashboardsUnsettled, 'dashboard') : null - // One Text, so a narrow band wraps it as a line, not word by word in each piece. - return ( - - - ⚠ Impact - {target} - (modify) - - {impact.severity} - - - {' '}direct {impact.direct} · transitive {impact.transitive}{impact.truncated ? '+' : ''} · tests {impact.tests} - {readers !== null ? ` · read by ${readers}` : ''} - {dashboards !== null ? ` · ${dashboards}` : ''} - - - -