diff --git a/AGENTS.md b/AGENTS.md index f1bcd005f..c4207f381 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -276,6 +276,14 @@ and are not a leftover score. Complete-case coverage (ADR 0168) persists to `report_leftover_map_coverage` and captions the pair list with how many scored posts entered the map. +Period leftover pairs (ADR 0017 / 0018 / 0048 / 0049 / 0149) are computed in +`lineageweave/leftover_pairs.py` from the residual after a real +GRM/GPCM score, never invented. Missing cells stay out of the +Gabriel factorization. Closest and farthest post–criterion pairs +persist to `report_leftover_pair` and sit above the member list so +a click opens that post. The grouping comparison strip reuses that +authorized leftover store; a leftover pair for a hidden post is omitted. + `frontend/` has its own toolchain (Node pinned via `frontend/mise.toml`, pnpm via Corepack -- do not add a second Node package manager or a floating Node version): diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 88d645c90..f25a29b24 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -607,7 +607,8 @@ many scored posts entered the factorization. Results persist to `report_period_score` / `report_member_score`. `GET /api/reports/{grouping}` lists the trend; `GET /api/reports/{grouping}/{period}` is ABAC-filtered; -`GET /api/reports/compare/{period}` is the home-page grouping strip; +`GET /api/reports/compare/{period}` is the home-page grouping strip +and carries the same ABAC-filtered leftover pairs (ADR 0149); `POST .../rebuild` scores every grouping kind (post_admin). `make seed` folds A-100/B-200 Event Lineage fixtures (and the Riverbend calendar post) that already have constructed IRT cells into the same shared @@ -620,6 +621,9 @@ closest/farthest pairs (signed residual `R`, observed `Y`, expected effects) above the member list, leftover-map axis share for residual SVD axes 1 and 2, and complete-case coverage captions (map used N of M scored posts), plus the + +closest/farthest pairs above the member list, leftover pairs on the +grouping comparison strip, and the PU / corp / thread comparison -- never a placeholder. TEPP is unchanged. ## Phase 6b: Knowledge Graph as a real Ontology + Semantic Layer diff --git a/CHANGELOG.d/2.12.17-leftover-pairs-comparison-strip.md b/CHANGELOG.d/2.12.17-leftover-pairs-comparison-strip.md new file mode 100644 index 000000000..c66acd3e7 --- /dev/null +++ b/CHANGELOG.d/2.12.17-leftover-pairs-comparison-strip.md @@ -0,0 +1,8 @@ +# 2.12.17 — Leftover pairs on the grouping comparison strip + +## Added + +- `GET /api/reports/compare/{period}` carries ABAC-filtered leftover + pairs (ADR 0149). After `make seed`, the A-100 comparison row names + leftover pairs; click opens that post. Hidden leftover posts stay + hidden. Do not invent leftover numbers. diff --git a/CHANGELOG.md b/CHANGELOG.md index 87383d244..17e9eb1fc 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -43,6 +43,13 @@ All notable changes to this project are documented here. Format follows (complete-case)”; incomplete rows stay excluded, never filled with zero. +- The grouping comparison strip now names leftover post–criterion + pairs on each visible row (ADR 0149). After `make seed`, open a + leftover pair on A-100 from the strip to read that post. A leftover + pair for a hidden post is omitted. Distances come from the same + authorized leftover store as the period-report list; they do not + invent a leftover score. + ### Fixed - Event Lineage's DAG no longer leaves a linear (no-branch) reconstruct diff --git a/CLAUDE.md b/CLAUDE.md index f32325abc..9b5f4d049 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -54,3 +54,60 @@ migration replay (ADR 0166), docstring coverage, and the measurement boundary are all stated in [AGENTS.md](AGENTS.md) -- read it before changing code, tests, or runtime policy rather than restating anything here. +======= +period-report run on the same snapshot (ADR 0013 / ADR 0024). The TEPP path goes through `tepp_client`. A missing +transport or an unused accepted envelope is Failed +(`tepp_not_available` / `tepp_result_not_persisted`). Do not invent a +theta or a local psychometric substitute. The home list caption stays +`kind · status · entity`; the machine failure code is detail-only +(ADR 0014). Open a Failed TEPP row, then connect a live TEPP +transport. A failed lineage row retries reconstruction -- it does not +mention TEPP. A failed period-report row rebuilds the report. A +pending TEPP row does not claim a calibrated measurement. A pending +lineage row says reconstruction has not started yet. +Digest prefixes stay audible; hover a prefix to read the full digest. +Opening a cutoff title shows the live post. Titles marked updated +after cutoff were rewritten after the run; the opened body names +both clocks and shows **Body this run knew** beside the live +rewrite. Compare those two texts before treating the live body as +reconstructed evidence (ADR 0016 / 0025). +`POST /api/analysis-runs` records Pending lineage only on an +authorized cutoff capture (ADR 0017). TEPP and period-report kinds +are 422. The Request button waits until affiliated corps load; choose +a corp if the token walks more than one. `POST /api/analysis-runs/{id}/start` +commits Running plus a durable outbox row, then reconstructs that +frozen cutoff bag (ADR 0021 / ADR 0023) or submits TEPP through +`tepp_client` (ADR 0022). A missing transport or unused accepted +envelope is Failed. Failed TEPP is terminal — connect a TEPP +transport from that Failed row. Create does not invent a Pending +TEPP row. Do not invent a theta. Hover the Result prefix to read +the parent-choice digest. +After `make seed`, open **Period report · Succeeded · Demo Corp**, +then **Open period report 2026-W02**. The home week is already +2026-W02, so the grouping comparison strip lands on Demo Corp. Report +grouping is Corporate entity and Demo Corp is current. The focused +chip name contains `Corporate entity: Demo Corp` and the persisted +mean θ. The period-report panel says Demo Corp is the opened grouping +and to read its mean θ and member posts, then open a post. Those +members land immediately under that next action, ahead of Other Corp +and the week strip. After `make seed`, leftover closest/farthest pairs +sit above the member list with leftover-map rank (rank 0 names no +leftover structure) and unexplained leftover `U` next to leftover-map +distance `d`. Leftover-map axis share badges name Gabriel inertia +of axes 1 and 2; open a leftover pair to read the post–criterion cell. +The shares do not invent a leftover score. Opening Public post names the next action: read +Event Lineage, Keyman, and evaluation on that post. The popup Event +Lineage DAG marks that post current. After that current node, the +popup names Keyman and evaluation as the next read. After landed +evaluation, the popup names the first Keyman as the next read. After +landed Ada West related, the popup names the first related node as +the next read. After that next action, the popup lands Priya Nair +related nodes. After those related nodes land, the popup names Ask +about this lineage as the next read. After that next action, the +popup lands Ask about this lineage. After landed chat, the popup names +the first Ask. After that next action, the popup lands the first Ask +answer. After landed first Ask answer, the popup names the first +cited source. After that next action, the popup lands the first cited +evidence. Changing the week first still +focuses the report period field. Mean θ stays on the period-report +panel. diff --git a/backend/app/main.py b/backend/app/main.py index 8aec031d1..4b72dd568 100644 --- a/backend/app/main.py +++ b/backend/app/main.py @@ -2430,7 +2430,24 @@ async def compare_period_groupings( ] if not members: continue - visible.append({**row, "members": [], "post_count": len(members)}) + leftover_pairs = [ + pair + for pair in row.get("leftover_pairs", []) + if _can_see_post(account, pair) + and not _is_synthetic_demo_member(pair, demo_entity_ids) + ] + leftover_pairs = [ + {key: value for key, value in pair.items() if key != "has_real_source_context"} + for pair in leftover_pairs + ] + visible.append( + { + **row, + "members": [], + "leftover_pairs": leftover_pairs, + "post_count": len(members), + } + ) return {"period_code": period_code, "groupings": visible} diff --git a/backend/app/report_ingestion.py b/backend/app/report_ingestion.py index a00350c81..ac65e6af2 100644 --- a/backend/app/report_ingestion.py +++ b/backend/app/report_ingestion.py @@ -978,6 +978,28 @@ async def fetch_period_comparison( members_by_key: dict[tuple[str, str], list[asyncpg.Record]] = defaultdict(list) for row in members: members_by_key[(row["grouping_kind"], row["grouping_key"])].append(row) + # Safe SQL: the source-context expression is an immutable schema fragment; grouping filters are bound. + leftover = await conn.fetch( # nosemgrep: python.lang.security.audit.sqli.asyncpg-sqli.asyncpg-sqli + f""" + select lp.grouping_kind, lp.grouping_key, lp.pair_kind, lp.post_id, + lp.criterion_code, lp.leftover_distance, lp.leftover_residual, + p.post_title, p.visibility_code, p.corporate_entity_id, + ({_SOURCE_CONTEXT_PRESENT_SQL}) as has_real_source_context + from report_leftover_pair lp + join source_post p on p.post_id = lp.post_id + where lp.period_code = $1 and lp.rubric_version = $2 + and lp.grouping_kind = any($3::text[]) + order by lp.grouping_kind, lp.grouping_key, + case lp.pair_kind when 'closest' then 0 else 1 end, + p.post_title + """, + period_code, + RUBRIC_VERSION, + list(GROUPING_KINDS), + ) + leftover_by_key: dict[tuple[str, str], list[asyncpg.Record]] = defaultdict(list) + for row in leftover: + leftover_by_key[(row["grouping_kind"], row["grouping_key"])].append(row) payload: list[dict[str, Any]] = [] for row in rows: label = await resolve_grouping_label(conn, row["grouping_kind"], row["grouping_key"]) @@ -997,6 +1019,20 @@ async def fetch_period_comparison( } for member in members_by_key.get((row["grouping_kind"], row["grouping_key"]), []) ], + "leftover_pairs": [ + { + "pair_kind": str(pair["pair_kind"]), + "post_id": str(pair["post_id"]), + "post_title": pair["post_title"], + "criterion_code": str(pair["criterion_code"]), + "leftover_distance": float(pair["leftover_distance"]), + "leftover_residual": float(pair["leftover_residual"]), + "visibility_code": pair["visibility_code"], + "corporate_entity_id": str(pair["corporate_entity_id"]), + "has_real_source_context": bool(pair["has_real_source_context"]), + } + for pair in leftover_by_key.get((row["grouping_kind"], row["grouping_key"]), []) + ], } ) return payload diff --git a/backend/tests/test_api.py b/backend/tests/test_api.py index 4cdf0729b..ebfef143f 100644 --- a/backend/tests/test_api.py +++ b/backend/tests/test_api.py @@ -5064,6 +5064,15 @@ def test_seed_period_report_surfaces_on_get_reports(client, demo_analyst_token, if row["grouping_kind"] == "thread_group" } assert threads["A-100"] > threads["B-200"] + leftover_thread = next( + row + for row in compare.json()["groupings"] + if row["grouping_kind"] == "thread_group" and row["grouping_label"] == "A-100" + ) + leftover_kinds = {pair["pair_kind"] for pair in leftover_thread.get("leftover_pairs", [])} + assert leftover_kinds <= {"closest", "farthest"} + assert all(pair["post_title"] for pair in leftover_thread.get("leftover_pairs", [])) + assert all(pair["leftover_distance"] >= 0 for pair in leftover_thread.get("leftover_pairs", [])) def test_seed_period_report_includes_fixture_event_lineage_posts( diff --git a/docs/adr/0049-leftover-pair-report-ui.md b/docs/adr/0049-leftover-pair-report-ui.md index 729857277..37beb78cb 100644 --- a/docs/adr/0049-leftover-pair-report-ui.md +++ b/docs/adr/0049-leftover-pair-report-ui.md @@ -55,3 +55,6 @@ next action, not only the distance. Depends on [ADR 0048](0048-persist-lsirm-leftover-pairs.md) and [ADR 0003](0003-fast-mlsirm-report-integration.md). Complete-case coverage of the leftover map is [ADR 0168](0168-leftover-map-complete-case-coverage.md). + +[ADR 0003](0003-fast-mlsirm-report-integration.md). The grouping +comparison strip reuses this leftover store ([ADR 0149](0149-leftover-pairs-on-comparison-strip.md)). diff --git a/docs/adr/0149-leftover-pairs-on-comparison-strip.md b/docs/adr/0149-leftover-pairs-on-comparison-strip.md new file mode 100644 index 000000000..61eddda9c --- /dev/null +++ b/docs/adr/0149-leftover-pairs-on-comparison-strip.md @@ -0,0 +1,42 @@ +# ADR 0149 — Leftover pairs on the grouping comparison strip + +**Decision status:** Accepted +**Date:** 2026-08-24 + +## Context + +ADR 0048 persists closest and farthest leftover post–criterion pairs. +ADR 0049 shows those pairs above the period-report member list. The +home-page grouping comparison strip (`GET /api/reports/compare/{period}`) +already names mean θ per PU / corp / thread so a buyer can switch +grouping without opening the period-report list first. That strip +does not yet name leftover pairs, so the buyer still has to switch +grouping before they can open a leftover post. + +Leftover pairs are already authorized rows. Denormalizing them onto +the comparison strip would invent a second leftover store. + +## Decision + +The comparison payload carries the same ABAC-filtered `leftover_pairs` +as the period-report payload. Each visible comparison row may name +closest and farthest leftover pairs. Clicking a leftover pair on the +strip opens that post with the same handler as a leftover pair on the +period-report list. A leftover pair for a hidden post is omitted the +same way a hidden member is. + +Do not invent leftover numbers or a second theta. Missing leftover +rows render nothing. + +## Consequences + +After `make seed`, the A-100 comparison row names leftover pairs. +Open a leftover pair from the strip to read the post–criterion cell. +The Period reports member list still shows leftover pairs above +members (ADR 0049). This slice only adds the same authorized leftover +store to the comparison strip. + +## Related + +Depends on [ADR 0048](0048-persist-lsirm-leftover-pairs.md) and +[ADR 0049](0049-leftover-pair-report-ui.md). diff --git a/frontend/src/App.test.tsx b/frontend/src/App.test.tsx index d0b0bce1d..58499682e 100644 --- a/frontend/src/App.test.tsx +++ b/frontend/src/App.test.tsx @@ -846,6 +846,16 @@ describe("App, authenticated", () => { mean_theta: 0.81, post_count: 4, link_method: "fipc", + leftover_pairs: [ + { + pair_kind: "closest", + post_id: "post-1", + post_title: "Public post", + criterion_code: "sales_lead_specificity", + leftover_distance: 0.12, + leftover_residual: 0.4, + }, + ], }, ], }), @@ -3765,6 +3775,11 @@ describe("App, authenticated", () => { expect(screen.getByRole("status")).toHaveTextContent( "A-100 is the opened grouping. Read its mean θ and member posts below, then open a post.", ); + expect( + screen.getByRole("button", { + name: /open leftover closest pair from comparison: public post/i, + }), + ).toHaveTextContent("Closest leftover: Public post · sales-lead"); await waitFor(() => expect(fetchMock).toHaveBeenCalledWith( expect.stringContaining("/api/reports/thread_group/2026-W02"), @@ -3782,6 +3797,18 @@ describe("App, authenticated", () => { expect(periodInput).toHaveValue("2026-W03"); }); + it("opens a leftover pair post from the comparison strip", async () => { + stubBackend(); + render(); + + await userEvent.click( + await screen.findByRole("button", { + name: /open leftover closest pair from comparison: public post/i, + }), + ); + await waitFor(() => expect(screen.getByText("The full body text.")).toBeInTheDocument()); + }); + it("opens a leftover pair post from the report panel", async () => { stubBackend(); render(); diff --git a/frontend/src/App.tsx b/frontend/src/App.tsx index 22f002775..90d0d4b5b 100644 --- a/frontend/src/App.tsx +++ b/frontend/src/App.tsx @@ -3588,6 +3588,37 @@ function ReportsPanel({ mean θ {row.mean_theta.toFixed(2)} {row.post_count} posts + {row.leftover_pairs && row.leftover_pairs.length > 0 && ( + + )} ))} diff --git a/frontend/src/api.ts b/frontend/src/api.ts index a951207b9..cb8bdd628 100644 --- a/frontend/src/api.ts +++ b/frontend/src/api.ts @@ -945,6 +945,7 @@ export interface GroupingComparisonRow { mean_theta: number; post_count: number; link_method: string; + leftover_pairs?: LeftoverPair[]; } export interface PeriodComparison { diff --git a/tests/test_static_sql_review_contracts.py b/tests/test_static_sql_review_contracts.py index 7269e6acf..31c7896bc 100644 --- a/tests/test_static_sql_review_contracts.py +++ b/tests/test_static_sql_review_contracts.py @@ -28,7 +28,7 @@ ) ASYNC_STATEMENT_METHODS = {"execute", "fetch", "fetchrow", "fetchval"} SQL_REVIEW_RULE = "python.lang.security.audit.sqli.asyncpg-sqli.asyncpg-sqli" -EXPECTED_SQL_SUPPRESSION_COUNT = 35 +EXPECTED_SQL_SUPPRESSION_COUNT = 36 @pytest.mark.parametrize("relative_path", SQL_REVIEW_PATHS)