diff --git a/AGENTS.md b/AGENTS.md index 0bb84cfac..b4c64b52b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -193,17 +193,20 @@ against a live local stack (`make up`) and self-skip without one -- see [README.md](README.md#local-product-stack-docker-compose). Period leftover pairs (ADR 0017 / 0018 / 0048 / 0049 / 0119 / 0162 / 0163 / -0164) are computed in `lineageweave/leftover_pairs.py` from the residual -after a real GRM/GPCM score, never invented. Distances are Euclidean on -the two-dimensional Gabriel leftover map; missing cells stay out of the -factorization. Closest and farthest post–criterion pairs persist to -`report_leftover_pair` with signed residual `R`, observed `Y`, and -expected `E[Y|θ, item]` so `R = Y − E` remains auditable, plus -leftover-map rank so rank 0 is not read as structure. They sit above the -member list so a click opens that post. Leftover-map axis share -(ADR 0148) is Gabriel inertia of residual SVD axes 1 and 2 and persists -to `report_leftover_map_axis`. Rank-0 residuals emit two zero-share -axes; the shares are report-level and are not a leftover score. +0164 / 0182) are computed in `lineageweave/leftover_pairs.py` from the +residual after a real GRM/GPCM score, never invented. Distances are +Euclidean on the two-dimensional Gabriel leftover map; missing cells stay +out of the factorization. Closest and farthest post–criterion pairs +persist to `report_leftover_pair` with signed residual `R`, observed +`Y`, and expected `E[Y|θ, item]` so `R = Y − E` remains auditable, +plus leftover-map rank so rank 0 is not read as structure, and +unexplained leftover `U = R − R̂` next to leftover-map distance `d` +after two-axis Gabriel reconstruction. They sit above the member +list so a click opens that post. Two-axis reconstruction `R̂` is not +persisted. Leftover-map axis share (ADR 0148) is Gabriel inertia of +residual SVD axes 1 and 2 and persists to `report_leftover_map_axis`. +Rank-0 residuals emit two zero-share axes; the shares are report-level +and are not a leftover score. `frontend/` has its own toolchain (Node pinned via `frontend/mise.toml`, pnpm via Corepack -- do not add a second Node package manager or a diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 9dbf19f31..b682cadee 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -594,10 +594,11 @@ on those same fixed parameters (Kim, 2006 FIPC). After scoring, information at the group's mean θ (Lord, 1980 max-info CAT). Rankings persist to `report_item_information`. After those IRT main effects, residual SVD leftover pairs on two Gabriel axes (Jeon et al., 2021; -ADR 0017 / 0048 / 0119 / 0162 / 0163 / 0164) persist to +ADR 0017 / 0048 / 0119 / 0162 / 0163 / 0164 / 0182) persist to `report_leftover_pair` with signed residual `R`, observed `Y`, expected -`E[Y|θ, item]`, and full leftover-map rank. Leftover-map axis share -(Gabriel inertia of residual SVD axes 1 and 2; ADR 0148) persists to +`E[Y|θ, item]`, full leftover-map rank, and unexplained leftover +`U = R − R̂` named on the pair row. Leftover-map axis share (Gabriel +inertia of residual SVD axes 1 and 2; ADR 0148) persists to `report_leftover_map_axis`. Results persist to `report_period_score` / `report_member_score`. `GET /api/reports/{grouping}` lists the trend; diff --git a/CHANGELOG.d/2.12.26-leftover-map-unexplained.md b/CHANGELOG.d/2.12.26-leftover-map-unexplained.md new file mode 100644 index 000000000..a64e67997 --- /dev/null +++ b/CHANGELOG.d/2.12.26-leftover-map-unexplained.md @@ -0,0 +1,8 @@ +## 2.12.26 — Leftover-map unexplained leftover + +- Persist unexplained leftover `U = R − R̂` on leftover post–criterion + pairs (ADR 0182). After `make seed`, closest and farthest leftover + pairs sit above the member list with `U` next to leftover-map + distance `d`; click opens that post. Omit the badge when unexplained + leftover is missing. Never invent a leftover score. Do not persist + leftover-map reconstruction `R̂`. diff --git a/CHANGELOG.md b/CHANGELOG.md index 5d67eb0e5..393125f3c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -40,6 +40,16 @@ All notable changes to this project are documented here. Format follows environment, so local OIDC and synthetic-data workflows resolve the same pinned dependencies as CI. +## [2.12.26] - 2026-08-24 + +### Added + +- Period leftover pair rows now name unexplained leftover `U = R − R̂` + next to leftover-map distance `d`, then open that post (Gabriel, 1971; + Jeon et al., 2021, eq. 3; ADR 0182). A missing unexplained leftover + omits the badge rather than inventing a leftover score. Two-axis + reconstruction `R̂` stays internal and is not persisted. + ## [2.12.19] - 2026-08-24 ### Added diff --git a/CLAUDE.md b/CLAUDE.md index 6f1af23b5..992753de8 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -55,8 +55,9 @@ mean θ. The period-report panel says Demo Corp is the opened grouping and to read its mean θ and member posts, then open a post. Those members land immediately under that next action, ahead of Other Corp and the week strip. After `make seed`, leftover closest/farthest pairs -sit above the member list with leftover-map rank; rank 0 names no -leftover structure. Leftover-map axis share badges name Gabriel inertia +sit above the member list with leftover-map rank (rank 0 names no +leftover structure) and unexplained leftover `U` next to leftover-map +distance `d`. Leftover-map axis share badges name Gabriel inertia of axes 1 and 2; open a leftover pair to read the post–criterion cell. The shares do not invent a leftover score. Opening Public post names the next action: read Event Lineage, Keyman, and evaluation on that post. The popup Event diff --git a/backend/app/report_ingestion.py b/backend/app/report_ingestion.py index 022daed3e..deca8d3d7 100644 --- a/backend/app/report_ingestion.py +++ b/backend/app/report_ingestion.py @@ -444,8 +444,9 @@ async def persist_period_report( insert into report_leftover_pair ( grouping_kind, grouping_key, period_code, rubric_version, pair_kind, post_id, criterion_code, leftover_distance, leftover_residual, - observed_response, expected_response, leftover_map_rank - ) values ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12) + observed_response, expected_response, leftover_map_rank, + leftover_map_unexplained + ) values ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13) """, grouping_kind, grouping_key, @@ -459,6 +460,7 @@ async def persist_period_report( pair.observed_response, pair.expected_response, pair.leftover_map_rank, + pair.leftover_map_unexplained, ) for axis in report.leftover_map_axes: await conn.execute( @@ -622,7 +624,8 @@ async def fetch_period_reports( f""" select lp.grouping_key, lp.pair_kind, lp.post_id, lp.criterion_code, lp.leftover_distance, lp.leftover_residual, - lp.observed_response, lp.expected_response, lp.leftover_map_rank, p.post_title, + lp.observed_response, lp.expected_response, lp.leftover_map_rank, + lp.leftover_map_unexplained, p.post_title, p.visibility_code, p.corporate_entity_id, ({_SOURCE_CONTEXT_PRESENT_SQL}) as has_real_source_context from report_leftover_pair lp @@ -748,6 +751,11 @@ async def fetch_period_reports( if row["leftover_map_rank"] is None else int(row["leftover_map_rank"]) ), + "leftover_map_unexplained": ( + None + if row["leftover_map_unexplained"] is None + else float(row["leftover_map_unexplained"]) + ), "visibility_code": row["visibility_code"], "corporate_entity_id": str(row["corporate_entity_id"]), "has_real_source_context": bool(row["has_real_source_context"]), diff --git a/backend/tests/test_api.py b/backend/tests/test_api.py index db59bdea1..2b1eda9ca 100644 --- a/backend/tests/test_api.py +++ b/backend/tests/test_api.py @@ -137,6 +137,11 @@ / "migrations" / "0169_report_leftover_map_axis.sql" ) +_LEFTOVER_MAP_UNEXPLAINED_MIGRATION = ( + Path(__file__).resolve().parents[2] + / "migrations" + / "0182_report_leftover_map_unexplained.sql" +) def _postgres_available() -> bool: @@ -255,6 +260,7 @@ def seeded_db(demo_analyst_token): cur.execute(_LEFTOVER_OBSERVED_EXPECTED_MIGRATION.read_text()) cur.execute(_LEFTOVER_MAP_RANK_MIGRATION.read_text()) cur.execute(_LEFTOVER_MAP_AXIS_MIGRATION.read_text()) + cur.execute(_LEFTOVER_MAP_UNEXPLAINED_MIGRATION.read_text()) cur.execute( "insert into common_lookup_value (lookup_category, lookup_code, lookup_label) values " "('corporate_entity_level', 'group', 'Group'), " @@ -4740,6 +4746,9 @@ def test_seed_period_report_surfaces_on_get_reports(client, demo_analyst_token, assert all(pair["leftover_distance"] >= 0 for pair in high_report.get("leftover_pairs", [])) for pair in high_report.get("leftover_pairs", []): assert pair["leftover_map_rank"] >= 0 + unexplained = pair.get("leftover_map_unexplained") + assert unexplained is None or isinstance(unexplained, (int, float)) + assert "leftover_map_reconstruction" not in pair observed = pair.get("observed_response") expected = pair.get("expected_response") if observed is None or expected is None: diff --git a/docs/adr/0003-fast-mlsirm-report-integration.md b/docs/adr/0003-fast-mlsirm-report-integration.md index 439380d00..ed4ca22c8 100644 --- a/docs/adr/0003-fast-mlsirm-report-integration.md +++ b/docs/adr/0003-fast-mlsirm-report-integration.md @@ -100,9 +100,13 @@ than one large PR: `information_polytomous` (Lord, 1980 max-info). Persist the ranking (`report_item_information`) and show the rank-1 item on the Period reports panel. Do not reimplement an information function here. -7. **Leftover-pair slice** (shipped in 0.71.2; ADR 0017 / 0018 / 0048 / 0049): after +7. **Leftover-pair slice** (shipped in 0.71.2; ADR 0017 / 0018 / 0048 / 0049 / 0182): after IRT main effects, persist closest and farthest post–criterion pairs - from the residual leftover map. Do not fork LSIRM; do not invent a + from the residual leftover map, and name unexplained leftover + `U = R − R̂` when Gabriel coordinates exist so the leftover cell the + two-axis map does not reconstruct is not read as leftover residual + or leftover-map distance. Do not persist two-axis reconstruction + `R̂`. Do not fork LSIRM; do not invent a leftover-pair API inside `fast-mlsirm` in this slice. 8. **Leftover-map axis-share slice** (ADR 0148): persist Gabriel inertia `σ_k² / Σ_j σ_j²` of leftover-map axes 1 and 2 on the same residual diff --git a/docs/adr/0048-persist-lsirm-leftover-pairs.md b/docs/adr/0048-persist-lsirm-leftover-pairs.md index bc79cb17e..fddc028e2 100644 --- a/docs/adr/0048-persist-lsirm-leftover-pairs.md +++ b/docs/adr/0048-persist-lsirm-leftover-pairs.md @@ -4,7 +4,8 @@ **Date:** 2026-08-17 **Amended by:** [ADR 0119](0119-leftover-map-two-dimensional-distance.md) (two leftover-map axes); [ADR 0163](0163-leftover-observed-expected.md) (observed Y and expected E); -[ADR 0164](0164-leftover-map-rank.md) (full map rank) +[ADR 0164](0164-leftover-map-rank.md) (full map rank); +[ADR 0182](0182-leftover-map-unexplained.md) (unexplained leftover U) ## Context @@ -32,12 +33,16 @@ they are never filled with zero. Persist exactly one `closest` and one `farthest` observed cell per period report in `report_leftover_pair` (3NF, two-or-more-word `snake_case`). -The biplot lives in `lineageweave/leftover_pairs.py` so leftover tests do not import `period_report` or `fast_mlsirm`. Distances are Euclidean on the two leftover-map axes (ADR 0119). Each leftover row also names observed `Y` and expected `E[Y|θ, item]` so residual reconciles to `Y − E` (ADR 0163), and names the full singular-value -rank while distance remains on the first two axes (ADR 0164). +rank while distance remains on the first two axes (ADR 0164). Each +leftover row also names unexplained leftover `U = R − R̂` when +Gabriel coordinates exist so the leftover cell the two-axis map does +not reconstruct is not read as leftover residual `R` or leftover-map +distance `d` (ADR 0182). Two-axis reconstruction `R̂` is computed +internally and is not persisted. Cascade the rows with `report_period_score`. A leftover post must also be a `report_member_score` row, and the leftover criterion diff --git a/docs/adr/0049-leftover-pair-report-ui.md b/docs/adr/0049-leftover-pair-report-ui.md index 3b6e4ce43..fdae8f2ee 100644 --- a/docs/adr/0049-leftover-pair-report-ui.md +++ b/docs/adr/0049-leftover-pair-report-ui.md @@ -4,7 +4,8 @@ **Date:** 2026-08-17 **Amended by:** [ADR 0162](0162-leftover-residual-disclosure.md) (signed residual R); [ADR 0163](0163-leftover-observed-expected.md) (observed Y and expected E); -[ADR 0164](0164-leftover-map-rank.md) (full map rank) +[ADR 0164](0164-leftover-map-rank.md) (full map rank); +[ADR 0182](0182-leftover-map-unexplained.md) (unexplained leftover U) ## Context @@ -21,14 +22,19 @@ second navigation surface. On each period-report group, render leftover pairs **above** the member list. Each pair is a button: closest or farthest label, post title, criterion short label, signed residual `R`, two-axis leftover-map -distance, full map rank, observed `Y`, and expected `E` when finite. +distance, full map rank, observed `Y`, expected `E` when finite, and +unexplained leftover `U` when finite. The next action names every available measurement before opening the -post; no amendment hides another, and rank 0 explicitly names no -leftover structure. +post; no amendment hides another, rank 0 explicitly names no +leftover structure, and unexplained leftover names "leftover map leaves +unexplained `U` after IRT main effects; open this post to read the +named criterion" when present. A missing unexplained leftover keeps +the existing next action. Clicking the button opens that post with the same handler as a member row. Residual naming is [ADR 0162](0162-leftover-residual-disclosure.md), observed/expected naming is [ADR 0163](0163-leftover-observed-expected.md), -rank naming is [ADR 0164](0164-leftover-map-rank.md). +rank naming is [ADR 0164](0164-leftover-map-rank.md), unexplained +leftover naming is [ADR 0182](0182-leftover-map-unexplained.md). After `make seed`, closest and farthest leftover pairs sit above the member list. Click a pair to open that post. diff --git a/docs/adr/0182-leftover-map-unexplained.md b/docs/adr/0182-leftover-map-unexplained.md new file mode 100644 index 000000000..e7cc46fca --- /dev/null +++ b/docs/adr/0182-leftover-map-unexplained.md @@ -0,0 +1,92 @@ +# ADR 0182 — Name unexplained leftover on period-report pair rows + +**Decision status:** Accepted +**Date:** 2026-08-24 + +Amends [ADR 0048](0048-persist-lsirm-leftover-pairs.md) and +[ADR 0049](0049-leftover-pair-report-ui.md). + +## Context + +ADR 0048 already persists leftover-map distance `d = ‖ξ_p − ζ_i‖` and +leftover residual `R = Y − E[Y|θ, item]` on `report_leftover_pair`. +ADR 0049 already renders closest and farthest pairs above the member +list and opens the named post. Distance is the Jeon et al. (2021, +eq. 3) map gap. Gabriel (1971) reconstructs a matrix from the biplot +as the inner product of person and item coordinates. The leftover map +buyers read is two-axis: unused axes pad with zero, and hidden SVD +axes after the second are dropped. Two-axis reconstruction +`R̂ = ξ_{1:2} · ζ_{1:2}` is therefore the leftover cell the map +shows. Hiding unexplained leftover `U = R − R̂` lets a buyer read +leftover residual `R` or leftover-map distance `d` as the leftover +the two-axis map does not reconstruct. + +This increment does not persist leftover-map reconstruction `R̂`, does +not persist leftover-map coordinates, does not name leftover-map inner +product as a separate full-rank column, does not name leftover-map +cosine, does not name leftover-map length, does not name observed `Y` / +expected `E`, does not name leftover-map rank, does not split leftover-map +distance onto two axes, and does not land Post quality on the leftover +criterion. Leftover-map distance stays full-rank Euclidean. +Reconstruction `R̂` is computed internally so `U` is honest, then +discarded. + +The unprotected-stack reconstructions for neighbouring leftover facts +use 0162–0181. This protected-main increment uses **0182** so it does +not collide with leftover-map reconstruction (0181), leftover-map +length (0181 on the length stack), leftover-map cosine (0180), +leftover-map inner product (0179), leftover residual disclosure +(0178), leftover observed `Y` / expected `E` (0177), leftover-map +rank (0172), two-axis leftover-map distance (0166), leftover coverage +(0168), leftover-map axis share (0148), or leftover interaction-map +persistence (0121). + +## Decision + +Each leftover pair names `leftover_map_unexplained` — unexplained +leftover `U = R − R̂` after two-axis Gabriel reconstruction +`R̂ = ξ_{1:2} · ζ_{1:2}`. Migration `0182` is the single source of +the column on every install path, fresh or existing -- shipped +migrations (`0001` / `0012`) are never edited after the fact. The +column is nullable so older leftover rows keep distance and residual +without fabricating unexplained leftover. Fallback pairs that have no +complete-case leftover map omit the value rather than inventing one. +A rank-0 origin map stores `0.0` (`R = 0` and `R̂ = 0`), not a +missing value. A non-finite unexplained leftover stores null rather +than inventing a leftover score. Do not persist +`leftover_map_reconstruction`. + +The pair button shows `U {signed}` next to leftover-map distance `d` +when the value is finite. Next action: leftover map leaves unexplained +`U` after IRT main effects; open this post to read the named criterion. +A missing or non-finite unexplained leftover omits the badge and keeps +the existing closest/farthest next action. Do not invent a leftover +score. Do not invent a theta. + +## Consequences + +`GET /api/reports/{grouping}/{period}` returns +`leftover_map_unexplained`. After `make seed`, closest and farthest +leftover pairs sit above the member list with named `U` next to `d`; +click opens that post. Hidden posts stay hidden. + +## Related + +Independent of leftover interaction-map persistence, leftover-criterion +evaluation landing, leftover residual disclosure, leftover observed +`Y` / expected `E`, leftover-map complete-case coverage, leftover-map +axis share, leftover pairs on the grouping comparison strip, two-axis +leftover-map distance, leftover-map rank, leftover-map inner product, +leftover-map cosine, leftover-map length, and leftover-map +reconstruction. + +## References + +Gabriel, K. R. (1971). The biplot graphic display of matrices with +application to principal component analysis. *Biometrika, 58*(3), +453–467. https://doi.org/10.1093/biomet/58.3.453 + +Jeon, M., Jin, I. H., Schweinberger, M., & Baugh, S. (2021). Mapping +unobserved item–respondent interactions: A latent space item response +model with interaction map. *Psychometrika, 86*(2), 378–403. +https://doi.org/10.1007/s11336-021-09762-5 diff --git a/frontend/package.json b/frontend/package.json index 844794550..64cf88197 100644 --- a/frontend/package.json +++ b/frontend/package.json @@ -1,7 +1,7 @@ { "name": "frontend", "private": true, - "version": "2.12.19", + "version": "2.12.26", "type": "module", "scripts": { "dev": "vite", diff --git a/frontend/src/App.test.tsx b/frontend/src/App.test.tsx index b738e142e..bc40dcdaf 100644 --- a/frontend/src/App.test.tsx +++ b/frontend/src/App.test.tsx @@ -950,6 +950,7 @@ describe("App, authenticated", () => { criterion_code: "sales_lead_specificity", leftover_distance: 0.12, leftover_residual: 0.4, + leftover_map_unexplained: 0.05, observed_response: 2.4, expected_response: 2.0, leftover_map_rank: 1, @@ -961,6 +962,7 @@ describe("App, authenticated", () => { criterion_code: "general_sentiment_negative", leftover_distance: 1.84, leftover_residual: -1.1, + leftover_map_unexplained: -0.25, observed_response: 0.9, expected_response: 2.0, leftover_map_rank: 1, @@ -3565,19 +3567,21 @@ describe("App, authenticated", () => { }); expect(closestPair).toHaveTextContent("Closest leftover: Public post · sales-lead"); expect(closestPair).toHaveTextContent( - "Read leftover map rank 1, observed Y 2.40, and expected E 2.00 after IRT main effects, then open this post.", + "Leftover map leaves unexplained U +0.05 after IRT main effects. Open this post to read sales-lead.", ); expect(closestPair).toHaveTextContent("R +0.40"); expect(closestPair).toHaveTextContent("Y 2.40 · E 2.00"); expect(closestPair).toHaveTextContent("rank 1"); + expect(closestPair).toHaveTextContent("U +0.05"); expect(closestPair).toHaveTextContent("d 0.12"); expect(farthestPair).toHaveTextContent("Farthest leftover: Specification revision requested · negative"); expect(farthestPair).toHaveTextContent( - "Read leftover map rank 1, observed Y 0.90, and expected E 2.00 after IRT main effects, then open this post.", + "Leftover map leaves unexplained U −0.25 after IRT main effects. Open this post to read negative.", ); expect(farthestPair).toHaveTextContent("R −1.10"); expect(farthestPair).toHaveTextContent("Y 0.90 · E 2.00"); expect(farthestPair).toHaveTextContent("rank 1"); + expect(farthestPair).toHaveTextContent("U −0.25"); expect(farthestPair).toHaveTextContent("d 1.84"); const memberButton = screen.getByRole("button", { name: /open report post: public post/i }); expect(closestPair.compareDocumentPosition(memberButton) & Node.DOCUMENT_POSITION_FOLLOWING).toBeTruthy(); diff --git a/frontend/src/api.ts b/frontend/src/api.ts index 98c7bb91a..5532bb3fa 100644 --- a/frontend/src/api.ts +++ b/frontend/src/api.ts @@ -774,6 +774,7 @@ export interface LeftoverPair { observed_response?: number | null; expected_response?: number | null; leftover_map_rank?: number | null; + leftover_map_unexplained?: number | null; } export interface LeftoverMapAxis { diff --git a/frontend/src/components/LeftoverPairList.tsx b/frontend/src/components/LeftoverPairList.tsx index f718b6369..d31fbc05b 100644 --- a/frontend/src/components/LeftoverPairList.tsx +++ b/frontend/src/components/LeftoverPairList.tsx @@ -7,6 +7,11 @@ import { } from "../leftoverMapRank"; import { formatLeftoverObservedExpected } from "../leftoverObservedExpected"; import { formatLeftoverResidual } from "../leftoverResidual"; +import { + formatLeftoverMapUnexplained, + formatSignedLeftoverValue, + LEFTOVER_MAP_UNEXPLAINED_ACTION, +} from "../leftoverMapUnexplained"; export type LeftoverPairListProps = { pairs: LeftoverPair[]; @@ -18,9 +23,11 @@ export type LeftoverPairListProps = { * Closest and farthest leftover post–criterion pairs after IRT main effects. * * Distance is the two-axis leftover-map Euclidean gap. Residual is - * ``R = Y − E[Y|θ, item]`` (Jeon et al., 2021, eq. 3 input). - * The action preserves residual, observed/expected, and full-rank - * amendments together before opening the named post. + * ``R = Y − E[Y|θ, item]`` (Jeon et al., 2021, eq. 3 input). Unexplained + * leftover ``U = R − R̂`` after two-axis Gabriel reconstruction (ADR 0182) + * takes priority over the residual/observed-expected/rank next action + * when finite; every badge still renders together before opening the + * named post. */ export function LeftoverPairList({ pairs, @@ -42,8 +49,16 @@ export function LeftoverPairList({ pair.expected_response, ); const rankBadge = formatLeftoverMapRank(pair.leftover_map_rank); + const unexplained = formatLeftoverMapUnexplained(pair.leftover_map_unexplained); let nextAction: string; - if (rankBadge !== null && observedExpected !== null) { + if (unexplained !== null) { + const signedUnexplained = + formatSignedLeftoverValue(pair.leftover_map_unexplained ?? Number.NaN) ?? "—"; + nextAction = tf(LEFTOVER_MAP_UNEXPLAINED_ACTION, { + value: signedUnexplained, + criterion, + }); + } else if (rankBadge !== null && observedExpected !== null) { nextAction = pair.leftover_map_rank === 0 ? tf( @@ -104,6 +119,7 @@ export function LeftoverPairList({ R {residual} {observedExpected ? {observedExpected} : null} {rankBadge ? {rankBadge} : null} + {unexplained ? {unexplained} : null} d {pair.leftover_distance.toFixed(2)} diff --git a/frontend/src/i18n.test.ts b/frontend/src/i18n.test.ts index 97f3c0c85..2ab5aa3b6 100644 --- a/frontend/src/i18n.test.ts +++ b/frontend/src/i18n.test.ts @@ -45,6 +45,7 @@ describe("i18n", () => { "Open leftover {kind} pair: {title} · {criterion}", "Open this post to read the criterion it sat closest to after main effects.", "Open this post to read the criterion it sat farthest from after main effects.", + "Leftover map leaves unexplained U {value} after IRT main effects. Open this post to read {criterion}.", "Read observed Y {observed} and expected E {expected} after IRT main effects, then open this post.", "Leftover map has no leftover structure after IRT main effects. Open this post.", "Leftover map rank {rank} after IRT main effects. Open this post.", @@ -149,6 +150,33 @@ describe("i18n", () => { ), ).toBe(expected); }); + + it.each([ + [ + "ko", + "잔여 지도가 IRT 주효과 이후 설명되지 않은 U +0.05을(를) 남깁니다. sales-lead 기준을 읽으려면 이 글을 여세요.", + ], + [ + "zh", + "残差图在 IRT 主效应后留下未解释的 U +0.05。打开这篇帖子阅读 sales-lead。", + ], + [ + "ja", + "残差マップはIRT主効果後の未説明 U +0.05 を残します。この投稿を開いて sales-lead を読んでください。", + ], + [ + "vi", + "Bản đồ phần dư để lại U +0.05 chưa giải thích sau hiệu ứng chính IRT. Mở bài viết này để đọc sales-lead.", + ], + ] as const)("formats leftover-map unexplained next action in %s", (locale, expected) => { + setLocale(locale); + expect( + tf( + "Leftover map leaves unexplained U {value} after IRT main effects. Open this post to read {criterion}.", + { value: "+0.05", criterion: "sales-lead" }, + ), + ).toBe(expected); + }); }); describe("locale-aware source labels", () => { diff --git a/frontend/src/i18n.ts b/frontend/src/i18n.ts index ef451f253..f4d93d9ed 100644 --- a/frontend/src/i18n.ts +++ b/frontend/src/i18n.ts @@ -389,6 +389,8 @@ const TRANSLATIONS: Partial>> = { "주효과 이후 가장 가깝게 앉은 기준을 읽으려면 이 글을 여세요.", "Open this post to read the criterion it sat farthest from after main effects.": "주효과 이후 가장 멀리 앉은 기준을 읽으려면 이 글을 여세요.", + "Leftover map leaves unexplained U {value} after IRT main effects. Open this post to read {criterion}.": + "잔여 지도가 IRT 주효과 이후 설명되지 않은 U {value}을(를) 남깁니다. {criterion} 기준을 읽으려면 이 글을 여세요.", "Read observed Y {observed} and expected E {expected} after IRT main effects, then open this post.": "IRT 주효과 이후 관측 Y {observed}와 기대 E {expected}를 읽은 다음, 이 글을 여세요.", "Leftover map has no leftover structure after IRT main effects. Open this post.": @@ -770,6 +772,8 @@ const TRANSLATIONS: Partial>> = { "打开这篇帖子,阅读主效应后距离最近的准则。", "Open this post to read the criterion it sat farthest from after main effects.": "打开这篇帖子,阅读主效应后距离最远的准则。", + "Leftover map leaves unexplained U {value} after IRT main effects. Open this post to read {criterion}.": + "残差图在 IRT 主效应后留下未解释的 U {value}。打开这篇帖子阅读 {criterion}。", "Read observed Y {observed} and expected E {expected} after IRT main effects, then open this post.": "阅读 IRT 主效应后的观测 Y {observed} 与期望 E {expected},然后打开这篇帖子。", "Leftover map has no leftover structure after IRT main effects. Open this post.": @@ -1151,6 +1155,8 @@ const TRANSLATIONS: Partial>> = { "主効果後に最も近くなった基準を読むには、この投稿を開いてください。", "Open this post to read the criterion it sat farthest from after main effects.": "主効果後に最も遠くなった基準を読むには、この投稿を開いてください。", + "Leftover map leaves unexplained U {value} after IRT main effects. Open this post to read {criterion}.": + "残差マップはIRT主効果後の未説明 U {value} を残します。この投稿を開いて {criterion} を読んでください。", "Read observed Y {observed} and expected E {expected} after IRT main effects, then open this post.": "IRT主効果後の観測 Y {observed} と期待 E {expected} を読んでから、この投稿を開いてください。", "Leftover map has no leftover structure after IRT main effects. Open this post.": @@ -1532,6 +1538,8 @@ const TRANSLATIONS: Partial>> = { "Mở bài viết này để đọc tiêu chí nằm gần nhất sau hiệu ứng chính.", "Open this post to read the criterion it sat farthest from after main effects.": "Mở bài viết này để đọc tiêu chí nằm xa nhất sau hiệu ứng chính.", + "Leftover map leaves unexplained U {value} after IRT main effects. Open this post to read {criterion}.": + "Bản đồ phần dư để lại U {value} chưa giải thích sau hiệu ứng chính IRT. Mở bài viết này để đọc {criterion}.", "Read observed Y {observed} and expected E {expected} after IRT main effects, then open this post.": "Đọc Y quan sát {observed} và E kỳ vọng {expected} sau hiệu ứng chính IRT, rồi mở bài viết này.", "Leftover map has no leftover structure after IRT main effects. Open this post.": diff --git a/frontend/src/leftoverMapUnexplained.test.ts b/frontend/src/leftoverMapUnexplained.test.ts new file mode 100644 index 000000000..aa23a125b --- /dev/null +++ b/frontend/src/leftoverMapUnexplained.test.ts @@ -0,0 +1,22 @@ +import { describe, expect, it } from "vitest"; +import { + formatLeftoverMapUnexplained, + formatSignedLeftoverValue, +} from "./leftoverMapUnexplained"; + +describe("formatLeftoverMapUnexplained", () => { + it("names unexplained leftover without inventing a leftover score", () => { + expect(formatLeftoverMapUnexplained(0.05)).toBe("U +0.05"); + expect(formatLeftoverMapUnexplained(-0.25)).toBe("U \u22120.25"); + expect(formatLeftoverMapUnexplained(0)).toBe("U 0.00"); + expect(formatSignedLeftoverValue(0.05)).toBe("+0.05"); + expect(formatSignedLeftoverValue(-0.25)).toBe("\u22120.25"); + }); + + it("omits the badge when unexplained leftover is missing or non-finite", () => { + expect(formatLeftoverMapUnexplained(null)).toBeNull(); + expect(formatLeftoverMapUnexplained(undefined)).toBeNull(); + expect(formatLeftoverMapUnexplained(Number.NaN)).toBeNull(); + expect(formatLeftoverMapUnexplained(Number.POSITIVE_INFINITY)).toBeNull(); + }); +}); diff --git a/frontend/src/leftoverMapUnexplained.ts b/frontend/src/leftoverMapUnexplained.ts new file mode 100644 index 000000000..4fa23c935 --- /dev/null +++ b/frontend/src/leftoverMapUnexplained.ts @@ -0,0 +1,26 @@ +/** Unexplained leftover ``U = R − R̂`` after two-axis reconstruction. */ + +export const LEFTOVER_MAP_UNEXPLAINED_ACTION = + "Leftover map leaves unexplained U {value} after IRT main effects. Open this post to read {criterion}."; + +export function formatSignedLeftoverValue(value: number): string | null { + if (!Number.isFinite(value)) { + return null; + } + const magnitude = Math.abs(value).toFixed(2); + if (value > 0) { + return `+${magnitude}`; + } + if (value < 0) { + return `\u2212${magnitude}`; + } + return magnitude; +} + +export function formatLeftoverMapUnexplained(value: number | null | undefined): string | null { + if (value == null) { + return null; + } + const signed = formatSignedLeftoverValue(value); + return signed === null ? null : `U ${signed}`; +} diff --git a/lineageweave/leftover_pairs.py b/lineageweave/leftover_pairs.py index 06a85d715..7531607e9 100644 --- a/lineageweave/leftover_pairs.py +++ b/lineageweave/leftover_pairs.py @@ -1,6 +1,6 @@ """Jeon leftover post–criterion pairs after a main-effect IRT. -Implements ADR 0048 as amended by ADR 0119, ADR 0163, and ADR 0164. +Implements ADR 0048 as amended by ADR 0119, ADR 0163, ADR 0164, and ADR 0182. Does not import ``fast_mlsirm`` or ``period_report``. A Gabriel biplot of the residual ``R = Y − E[Y|θ, item]`` supplies person and item @@ -13,6 +13,11 @@ after the second are dropped. Each pair also names the full leftover-map rank so a rank-0 collapse is not read as leftover structure. Axis share is the Gabriel inertia of the first two leftover-map axes (ADR 0148). +Each pair also names unexplained leftover ``U = R − R̂`` after two-axis +Gabriel reconstruction ``R̂ = ξ_{1:2} · ζ_{1:2}`` so the leftover cell +the map does not reconstruct is not confused with leftover residual +``R`` or leftover-map distance ``d``. Reconstruction is computed +internally and is not persisted. """ from __future__ import annotations @@ -40,6 +45,7 @@ class LeftoverPair: observed_response: float expected_response: float leftover_map_rank: int + leftover_map_unexplained: float | None = None @dataclass(frozen=True) @@ -66,7 +72,11 @@ def leftover_pairs_from_residual( stable closest/farthest pair so seed is not empty and does not invent a leftover score. Stored residual equals observed ``Y`` minus expected ``E[Y|θ, item]``. Stored leftover-map rank is the number - of Gabriel singular values above the floor. + of Gabriel singular values above the floor. When Gabriel coordinates + exist, unexplained leftover ``U = R − R̂`` names the leftover cell + the two-axis map does not reconstruct; ``R̂`` stays internal and is + never persisted. Fallback pairs (no complete-case map) omit + unexplained leftover rather than fabricating one. """ pairs, _axes = leftover_map_from_residual(post_ids, item_codes, matrix, expected) return pairs @@ -114,7 +124,7 @@ def leftover_map_from_residual( ) axes = leftover_map_axes_from_singular(singular) leftover_map_rank = int(singular.size) - candidates: list[tuple[float, str, str, float, float, float]] = [] + candidates: list[tuple[float, str, str, float, float, float, float | None]] = [] if person_pos is not None and item_pos is not None: person_index = np.flatnonzero(keep_person) item_index = np.flatnonzero(keep_item) @@ -130,8 +140,22 @@ def leftover_map_from_residual( ) if not np.isfinite(distance): continue + reconstruction = float( + np.dot(person_xy[local_person[person]], item_xy[local_item[item]]) + ) + unexplained = _unexplained_leftover(float(residual[person, item]), reconstruction) candidates.append( - _candidate_row(post_ids, item_codes, matrix, expected, residual, person, item, distance) + _candidate_row( + post_ids, + item_codes, + matrix, + expected, + residual, + person, + item, + distance, + unexplained, + ) ) if not candidates: leftover_map_rank = 0 @@ -147,6 +171,7 @@ def leftover_map_from_residual( person, item, max(distance, 0.0), + None, ) ) closest = min(candidates, key=lambda row: (row[0], row[1], row[2])) @@ -158,6 +183,16 @@ def leftover_map_from_residual( return pairs, axes +def _unexplained_leftover(residual: float, reconstruction: float) -> float | None: + """Return ``U = R − R̂`` when both terms are finite; otherwise omit.""" + if not np.isfinite(reconstruction): + return None + unexplained = residual - reconstruction + if not np.isfinite(unexplained): + return None + return float(unexplained) + + def _candidate_row( post_ids: list[str], item_codes: tuple[str, ...], @@ -167,8 +202,9 @@ def _candidate_row( person: int, item: int, distance: float, -) -> tuple[float, str, str, float, float, float]: - """One observed leftover cell: distance, ids, residual, Y, E.""" + leftover_map_unexplained: float | None, +) -> tuple[float, str, str, float, float, float, float | None]: + """One observed leftover cell: distance, ids, residual, Y, E, unexplained U.""" leftover_residual = float(residual[person, item]) observed_response = float(matrix[person, item]) expected_response = float(expected[person, item]) @@ -181,12 +217,13 @@ def _candidate_row( leftover_residual, observed_response, expected_response, + leftover_map_unexplained, ) def _pair_from_candidate( pair_kind: str, - row: tuple[float, str, str, float, float, float], + row: tuple[float, str, str, float, float, float, float | None], leftover_map_rank: int, ) -> LeftoverPair: """Build a leftover pair from a candidate row.""" @@ -201,6 +238,7 @@ def _pair_from_candidate( observed_response=row[4], expected_response=row[5], leftover_map_rank=leftover_map_rank, + leftover_map_unexplained=row[6], ) @@ -289,7 +327,13 @@ def _leftover_map_positions( def _pad_map_axes(positions: np.ndarray) -> np.ndarray: - """Pad or truncate Gabriel coordinates to two leftover-map axes.""" + """Pad or truncate Gabriel coordinates to two leftover-map axes. + + Unused axes pad with zero rather than inventing a second component. + Hidden SVD axes after the second are dropped so reconstruction is + ``ξ_{1:2} · ζ_{1:2}``, not the full-rank inner product. That + reconstruction stays internal; only unexplained leftover is named. + """ padded = np.zeros((positions.shape[0], _LEFTOVER_MAP_AXES), dtype=np.float64) width = min(_LEFTOVER_MAP_AXES, positions.shape[1]) padded[:, :width] = positions[:, :width] diff --git a/migrations/0182_report_leftover_map_unexplained.sql b/migrations/0182_report_leftover_map_unexplained.sql new file mode 100644 index 000000000..9dfc250ed --- /dev/null +++ b/migrations/0182_report_leftover_map_unexplained.sql @@ -0,0 +1,10 @@ +-- ADR 0182: persist unexplained leftover U = R − R̂ after two-axis +-- leftover-map reconstruction R̂ = ξ_{1:2} · ζ_{1:2}. Distance stays +-- Euclidean leftover-map d. Reconstruction is computed internally and +-- is not persisted. Upgrade column is nullable so older leftover rows +-- keep distance and residual without fabricating unexplained leftover. +-- This migration is the single source of the column on fresh and +-- existing installations. + +alter table report_leftover_pair + add column if not exists leftover_map_unexplained numeric; diff --git a/migrations/rollback/0182_report_leftover_map_unexplained.sql b/migrations/rollback/0182_report_leftover_map_unexplained.sql new file mode 100644 index 000000000..2352e3054 --- /dev/null +++ b/migrations/rollback/0182_report_leftover_map_unexplained.sql @@ -0,0 +1,4 @@ +-- Reverse 0182. Leftover distance and residual stay on the pair row. + +alter table report_leftover_pair + drop column if exists leftover_map_unexplained; diff --git a/pyproject.toml b/pyproject.toml index 8b01c9850..c5ffda1bf 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "lineageweave" -version = "2.12.19" +version = "2.12.26" description = "Reconstructs git-branch-style lineage DAGs from scattered short records using multi-channel score fusion and LLM adjudication." readme = "README.md" license = { text = "MIT" } diff --git a/scripts/seed_demo_data.py b/scripts/seed_demo_data.py index 514ae2768..62249b9e8 100644 --- a/scripts/seed_demo_data.py +++ b/scripts/seed_demo_data.py @@ -119,6 +119,7 @@ def seed( cur.execute((migrations / "0012_report_leftover_pair.sql").read_text()) cur.execute((migrations / "0163_report_leftover_observed_expected.sql").read_text()) cur.execute((migrations / "0164_report_leftover_map_rank.sql").read_text()) + cur.execute((migrations / "0182_report_leftover_map_unexplained.sql").read_text()) cur.execute((migrations / "0060_role_responsibility_agent_type.sql").read_text()) cur.execute((migrations / "0013_person_job_title.sql").read_text()) cur.execute((migrations / "0014_role_responsibility_team_actor_type.sql").read_text()) @@ -1198,8 +1199,9 @@ def _persist_seed_period_report( "insert into report_leftover_pair (" "grouping_kind, grouping_key, period_code, rubric_version, " "pair_kind, post_id, criterion_code, leftover_distance, leftover_residual, " - "observed_response, expected_response, leftover_map_rank" - ") values (%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s)", + "observed_response, expected_response, leftover_map_rank, " + "leftover_map_unexplained" + ") values (%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s)", ( grouping_kind, grouping_key, @@ -1213,6 +1215,7 @@ def _persist_seed_period_report( pair.observed_response, pair.expected_response, pair.leftover_map_rank, + pair.leftover_map_unexplained, ), ) for axis in report.leftover_map_axes: diff --git a/tests/test_leftover_pairs.py b/tests/test_leftover_pairs.py index 2e6881801..135ddcd2b 100644 --- a/tests/test_leftover_pairs.py +++ b/tests/test_leftover_pairs.py @@ -1,6 +1,7 @@ """Leftover post–criterion pairs after the main-effect IRT. -Covers ADR 0048 as amended by ADR 0119, ADR 0148, ADR 0163, and ADR 0164. +Covers ADR 0048 as amended by ADR 0119, ADR 0148, ADR 0163, ADR 0164, +and ADR 0182. Uses a constructed residual matrix so the closest and farthest pair are known without calling ``fit_polytomous``. Loads @@ -99,6 +100,9 @@ def test_leftover_residual_biplot_separates_aligned_and_opposed_cells() -> None: assert farthest.observed_response == pytest.approx(-2.0) assert farthest.expected_response == pytest.approx(0.0) assert farthest.leftover_distance == pytest.approx(2.0 * np.sqrt(2.0), rel=1e-6) + assert closest.leftover_map_unexplained == pytest.approx(0.0, abs=1e-6) + assert farthest.leftover_map_unexplained == pytest.approx(0.0, abs=1e-6) + assert not hasattr(closest, "leftover_map_reconstruction") for pair in pairs: _assert_residual_reconciles(pair) assert pair.leftover_map_rank == 1 @@ -120,6 +124,8 @@ def test_zero_residual_still_emits_stable_leftover_pairs() -> None: assert pairs[0].expected_response == pytest.approx(1.0) assert pairs[1].post_id == "beta-post" assert pairs[1].criterion_code == "item_two" + assert pairs[0].leftover_map_unexplained == pytest.approx(0.0) + assert pairs[1].leftover_map_unexplained == pytest.approx(0.0) for pair in pairs: _assert_residual_reconciles(pair) assert pair.leftover_map_rank == 0 @@ -152,6 +158,7 @@ def test_partial_observation_does_not_treat_missing_as_zero_residual() -> None: for pair in pairs: _assert_residual_reconciles(pair) assert pair.leftover_map_rank == 1 + assert pair.leftover_map_unexplained == pytest.approx(0.0, abs=1e-6) def test_leftover_is_empty_without_observed_cells() -> None: @@ -208,6 +215,7 @@ def test_leftover_residual_rejects_database_tolerance_boundary() -> None: 0, 0, 0.0, + None, ) @@ -308,6 +316,56 @@ def test_rank_four_pair_distances_match_two_dimensional_gabriel_coords() -> None assert (post_index[farthest.post_id], item_index[farthest.criterion_code]) == farthest_map +def test_unexplained_equals_residual_minus_two_axis_reconstruction() -> None: + """Unexplained leftover U is R − R̂, not leftover residual R, not leftover-map distance d. + + Uses the same rank-4 matrix as the two-axis distance proof above: R̂ is + the two-axis Gabriel reconstruction ``person_map @ item_map.T``, built + from the same padded coordinates ``leftover_distance`` already uses. + """ + post_ids = ["post-a", "post-b", "post-c", "post-d"] + item_codes = ("item-a", "item-b", "item-c", "item-d") + matrix = np.array( + [ + [4.0, 1.0, 0.0, -1.0], + [0.0, 3.0, 1.0, -2.0], + [-2.0, 0.0, 2.0, 1.0], + [1.0, -1.0, 0.0, 4.0], + ], + dtype=np.float64, + ) + expected = np.zeros_like(matrix) + filled = matrix - float(np.mean(matrix)) + person_full, item_full = _gabriel_positions(filled) + assert person_full.shape[1] == 4 + person_map = _pad_map_axes(person_full) + item_map = _pad_map_axes(item_full) + reconstruction = person_map @ item_map.T + full_inner = person_full @ item_full.T + assert float(np.max(np.abs(reconstruction - filled))) > 1e-6 + assert float(np.max(np.abs(reconstruction - full_inner))) > 1e-6 + + pairs = leftover_pairs_from_residual(post_ids, item_codes, matrix, expected) + assert [pair.pair_kind for pair in pairs] == [PAIR_KIND_CLOSEST, PAIR_KIND_FARTHEST] + post_index = {post_id: index for index, post_id in enumerate(post_ids)} + item_index = {code: index for index, code in enumerate(item_codes)} + for pair in pairs: + person = post_index[pair.post_id] + item = item_index[pair.criterion_code] + expected_unexplained = float(pair.leftover_residual) - float(reconstruction[person, item]) + assert pair.leftover_map_unexplained == pytest.approx(expected_unexplained) + assert pair.leftover_map_unexplained != pytest.approx(pair.leftover_residual) + assert pair.leftover_map_unexplained != pytest.approx(pair.leftover_distance) + assert not hasattr(pair, "leftover_map_reconstruction") + + +def test_pad_map_axes_truncates_hidden_svd_components() -> None: + """Axes after the second leftover-map axis do not enter reconstruction.""" + padded = leftover._pad_map_axes(np.array([[1.0, 2.0, 9.0]], dtype=np.float64)) + assert padded.shape == (1, 2) + assert padded[0].tolist() == pytest.approx([1.0, 2.0]) + + def test_rejects_response_and_expectation_shape_mismatches() -> None: """Scientific inputs must match their declared post and criterion axes.""" with pytest.raises(ValueError, match="matrix shape"): @@ -343,6 +401,27 @@ def test_sparse_residual_uses_only_observed_cells_for_fallback_distance() -> Non assert [pair.leftover_map_rank for pair in pairs] == [0, 0] +def test_leftover_fallback_omits_unexplained_without_complete_case_map() -> None: + """No complete-case rectangle: persist distance from |R − center|, omit U.""" + post_ids = ["sparse-a", "sparse-b"] + item_codes = ("item_near", "item_far") + matrix = np.array( + [ + [2.0, np.nan], + [np.nan, -2.0], + ], + dtype=np.float64, + ) + expected = np.zeros_like(matrix) + pairs = leftover_pairs_from_residual(post_ids, item_codes, matrix, expected) + assert [pair.pair_kind for pair in pairs] == [PAIR_KIND_CLOSEST, PAIR_KIND_FARTHEST] + assert {pair.post_id for pair in pairs} == {"sparse-a", "sparse-b"} + for pair in pairs: + assert pair.leftover_map_unexplained is None + assert pair.leftover_distance >= 0.0 + assert not hasattr(pair, "leftover_map_reconstruction") + + def test_nonfinite_map_distance_falls_back_to_centered_residual( monkeypatch: pytest.MonkeyPatch, ) -> None: @@ -388,6 +467,6 @@ def test_leftover_map_rank_rejects_negative_rank() -> None: with pytest.raises(ValueError, match="non-negative integer"): leftover._pair_from_candidate( PAIR_KIND_CLOSEST, - (0.0, "public-post", "sales_lead_specificity", 0.0, 1.0, 1.0), + (0.0, "public-post", "sales_lead_specificity", 0.0, 1.0, 1.0, None), -1, ) diff --git a/tests/test_period_report.py b/tests/test_period_report.py index f5cee497b..943321394 100644 --- a/tests/test_period_report.py +++ b/tests/test_period_report.py @@ -270,6 +270,9 @@ def test_calibrated_report_attaches_leftover_pairs() -> None: pair.observed_response - pair.expected_response, abs=1e-6 ) assert pair.leftover_map_rank >= 0 + if pair.leftover_map_unexplained is not None: + assert np.isfinite(pair.leftover_map_unexplained) + assert not hasattr(pair, "leftover_map_reconstruction") assert [axis.axis_index for axis in report.leftover_map_axes] == [1, 2] for axis in report.leftover_map_axes: assert axis.leftover_singular_value >= 0.0 diff --git a/tests/test_schema.py b/tests/test_schema.py index 253ec3a84..f2a295cc6 100644 --- a/tests/test_schema.py +++ b/tests/test_schema.py @@ -58,6 +58,11 @@ / "migrations" / "0169_report_leftover_map_axis.sql" ) +_LEFTOVER_MAP_UNEXPLAINED_MIGRATION = ( + Path(__file__).resolve().parents[1] + / "migrations" + / "0182_report_leftover_map_unexplained.sql" +) def _postgres_available() -> bool: @@ -97,6 +102,7 @@ def schema_db(): cur.execute(_LEFTOVER_OBSERVED_EXPECTED_MIGRATION.read_text()) cur.execute(_LEFTOVER_MAP_RANK_MIGRATION.read_text()) cur.execute(_LEFTOVER_MAP_AXIS_MIGRATION.read_text()) + cur.execute(_LEFTOVER_MAP_UNEXPLAINED_MIGRATION.read_text()) conn.commit() yield conn finally: @@ -227,6 +233,23 @@ def test_leftover_pair_names_leftover_map_rank_column(schema_db) -> None: assert columns["leftover_map_rank"] == "YES" +def test_leftover_pair_names_nullable_unexplained_column(schema_db) -> None: + """Every install path preserves legacy pairs while naming unexplained leftover.""" + with schema_db.cursor() as cur: + cur.execute( + """ + select column_name, is_nullable + from information_schema.columns + where table_name = 'report_leftover_pair' + """ + ) + columns = dict(cur.fetchall()) + assert columns["leftover_map_unexplained"] == "YES" + assert columns["leftover_residual"] == "NO" + assert columns["leftover_distance"] == "NO" + assert "leftover_map_reconstruction" not in columns + + def test_leftover_map_axis_references_period_score(schema_db) -> None: """Axis share is report-level; it must cascade with the period score.""" with schema_db.cursor() as cur: diff --git a/uv.lock b/uv.lock index 095fa6696..8bf26bf75 100644 --- a/uv.lock +++ b/uv.lock @@ -454,7 +454,7 @@ wheels = [ [[package]] name = "lineageweave" -version = "2.12.19" +version = "2.12.26" source = { editable = "." } dependencies = [ { name = "certifi" },