feat: name leftover-map unexplained leftover on leftover pairs (v2.12.26)

AGENTS.md

@@ -193,17 +193,20 @@ against a live local stack (`make up`) and self-skip without one -- see
 [README.md](README.md#local-product-stack-docker-compose).
 
 Period leftover pairs (ADR 0017 / 0018 / 0048 / 0049 / 0119 / 0162 / 0163 /
-0164) are computed in `lineageweave/leftover_pairs.py` from the residual
-after a real GRM/GPCM score, never invented. Distances are Euclidean on
-the two-dimensional Gabriel leftover map; missing cells stay out of the
-factorization. Closest and farthest post–criterion pairs persist to
-`report_leftover_pair` with signed residual `R`, observed `Y`, and
-expected `E[Y|θ, item]` so `R = Y − E` remains auditable, plus
-leftover-map rank so rank 0 is not read as structure. They sit above the
-member list so a click opens that post. Leftover-map axis share
-(ADR 0148) is Gabriel inertia of residual SVD axes 1 and 2 and persists
-to `report_leftover_map_axis`. Rank-0 residuals emit two zero-share
-axes; the shares are report-level and are not a leftover score.
+0164 / 0182) are computed in `lineageweave/leftover_pairs.py` from the
+residual after a real GRM/GPCM score, never invented. Distances are
+Euclidean on the two-dimensional Gabriel leftover map; missing cells stay
+out of the factorization. Closest and farthest post–criterion pairs
+persist to `report_leftover_pair` with signed residual `R`, observed
+`Y`, and expected `E[Y|θ, item]` so `R = Y − E` remains auditable,
+plus leftover-map rank so rank 0 is not read as structure, and
+unexplained leftover `U = R − R̂` next to leftover-map distance `d`
+after two-axis Gabriel reconstruction. They sit above the member
+list so a click opens that post. Two-axis reconstruction `R̂` is not
+persisted. Leftover-map axis share (ADR 0148) is Gabriel inertia of
+residual SVD axes 1 and 2 and persists to `report_leftover_map_axis`.
+Rank-0 residuals emit two zero-share axes; the shares are report-level
+and are not a leftover score.
 
 `frontend/` has its own toolchain (Node pinned via `frontend/mise.toml`,
 pnpm via Corepack -- do not add a second Node package manager or a

ARCHITECTURE.md

@@ -594,10 +594,11 @@ on those same fixed parameters (Kim, 2006 FIPC). After scoring,
 information at the group's mean θ (Lord, 1980 max-info CAT). Rankings
 persist to `report_item_information`. After those IRT main effects,
 residual SVD leftover pairs on two Gabriel axes (Jeon et al., 2021;
-ADR 0017 / 0048 / 0119 / 0162 / 0163 / 0164) persist to
+ADR 0017 / 0048 / 0119 / 0162 / 0163 / 0164 / 0182) persist to
 `report_leftover_pair` with signed residual `R`, observed `Y`, expected
-`E[Y|θ, item]`, and full leftover-map rank. Leftover-map axis share
-(Gabriel inertia of residual SVD axes 1 and 2; ADR 0148) persists to
+`E[Y|θ, item]`, full leftover-map rank, and unexplained leftover
+`U = R − R̂` named on the pair row. Leftover-map axis share (Gabriel
+inertia of residual SVD axes 1 and 2; ADR 0148) persists to
 `report_leftover_map_axis`. Results persist to
 `report_period_score` / `report_member_score`.
 `GET /api/reports/{grouping}` lists the trend;

CHANGELOG.d/2.12.26-leftover-map-unexplained.md

@@ -0,0 +1,8 @@
+## 2.12.26 — Leftover-map unexplained leftover
+
+- Persist unexplained leftover `U = R − R̂` on leftover post–criterion
+  pairs (ADR 0182). After `make seed`, closest and farthest leftover
+  pairs sit above the member list with `U` next to leftover-map
+  distance `d`; click opens that post. Omit the badge when unexplained
+  leftover is missing. Never invent a leftover score. Do not persist
+  leftover-map reconstruction `R̂`.

CHANGELOG.md

@@ -40,6 +40,16 @@ All notable changes to this project are documented here. Format follows
   environment, so local OIDC and synthetic-data workflows resolve the same
   pinned dependencies as CI.
 
+## [2.12.26] - 2026-08-24
+
+### Added
+
+- Period leftover pair rows now name unexplained leftover `U = R − R̂`
+  next to leftover-map distance `d`, then open that post (Gabriel, 1971;
+  Jeon et al., 2021, eq. 3; ADR 0182). A missing unexplained leftover
+  omits the badge rather than inventing a leftover score. Two-axis
+  reconstruction `R̂` stays internal and is not persisted.
+
 ## [2.12.19] - 2026-08-24
 
 ### Added

CLAUDE.md

@@ -55,8 +55,9 @@ mean θ. The period-report panel says Demo Corp is the opened grouping
 and to read its mean θ and member posts, then open a post. Those
 members land immediately under that next action, ahead of Other Corp
 and the week strip. After `make seed`, leftover closest/farthest pairs
-sit above the member list with leftover-map rank; rank 0 names no
-leftover structure. Leftover-map axis share badges name Gabriel inertia
+sit above the member list with leftover-map rank (rank 0 names no
+leftover structure) and unexplained leftover `U` next to leftover-map
+distance `d`. Leftover-map axis share badges name Gabriel inertia
 of axes 1 and 2; open a leftover pair to read the post–criterion cell.
 The shares do not invent a leftover score. Opening Public post names the next action: read
 Event Lineage, Keyman, and evaluation on that post. The popup Event

backend/app/report_ingestion.py

@@ -444,8 +444,9 @@ async def persist_period_report(
             insert into report_leftover_pair (
                 grouping_kind, grouping_key, period_code, rubric_version,
                 pair_kind, post_id, criterion_code, leftover_distance, leftover_residual,
-                observed_response, expected_response, leftover_map_rank
-            ) values ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12)
+                observed_response, expected_response, leftover_map_rank,
+                leftover_map_unexplained
+            ) values ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13)
             """,
             grouping_kind,
             grouping_key,
@@ -459,6 +460,7 @@ async def persist_period_report(
             pair.observed_response,
             pair.expected_response,
             pair.leftover_map_rank,
+            pair.leftover_map_unexplained,
         )
     for axis in report.leftover_map_axes:
         await conn.execute(
@@ -622,7 +624,8 @@ async def fetch_period_reports(
         f"""
         select lp.grouping_key, lp.pair_kind, lp.post_id, lp.criterion_code,
                lp.leftover_distance, lp.leftover_residual,
-               lp.observed_response, lp.expected_response, lp.leftover_map_rank, p.post_title,
+               lp.observed_response, lp.expected_response, lp.leftover_map_rank,
+               lp.leftover_map_unexplained, p.post_title,
                p.visibility_code, p.corporate_entity_id,
                ({_SOURCE_CONTEXT_PRESENT_SQL}) as has_real_source_context
         from report_leftover_pair lp
@@ -748,6 +751,11 @@ async def fetch_period_reports(
                             if row["leftover_map_rank"] is None
                             else int(row["leftover_map_rank"])
                         ),
+                        "leftover_map_unexplained": (
+                            None
+                            if row["leftover_map_unexplained"] is None
+                            else float(row["leftover_map_unexplained"])
+                        ),
                         "visibility_code": row["visibility_code"],
                         "corporate_entity_id": str(row["corporate_entity_id"]),
                         "has_real_source_context": bool(row["has_real_source_context"]),

backend/tests/test_api.py

@@ -137,6 +137,11 @@
     / "migrations"
     / "0169_report_leftover_map_axis.sql"
 )
+_LEFTOVER_MAP_UNEXPLAINED_MIGRATION = (
+    Path(__file__).resolve().parents[2]
+    / "migrations"
+    / "0182_report_leftover_map_unexplained.sql"
+)
 
 
 def _postgres_available() -> bool:
@@ -255,6 +260,7 @@ def seeded_db(demo_analyst_token):
             cur.execute(_LEFTOVER_OBSERVED_EXPECTED_MIGRATION.read_text())
             cur.execute(_LEFTOVER_MAP_RANK_MIGRATION.read_text())
             cur.execute(_LEFTOVER_MAP_AXIS_MIGRATION.read_text())
+            cur.execute(_LEFTOVER_MAP_UNEXPLAINED_MIGRATION.read_text())
             cur.execute(
                 "insert into common_lookup_value (lookup_category, lookup_code, lookup_label) values "
                 "('corporate_entity_level', 'group', 'Group'), "
@@ -4740,6 +4746,9 @@ def test_seed_period_report_surfaces_on_get_reports(client, demo_analyst_token,
     assert all(pair["leftover_distance"] >= 0 for pair in high_report.get("leftover_pairs", []))
     for pair in high_report.get("leftover_pairs", []):
         assert pair["leftover_map_rank"] >= 0
+        unexplained = pair.get("leftover_map_unexplained")
+        assert unexplained is None or isinstance(unexplained, (int, float))
+        assert "leftover_map_reconstruction" not in pair
         observed = pair.get("observed_response")
         expected = pair.get("expected_response")
         if observed is None or expected is None:

docs/adr/0003-fast-mlsirm-report-integration.md

@@ -100,9 +100,13 @@ than one large PR:
    `information_polytomous` (Lord, 1980 max-info). Persist the ranking
    (`report_item_information`) and show the rank-1 item on the Period
    reports panel. Do not reimplement an information function here.
-7. **Leftover-pair slice** (shipped in 0.71.2; ADR 0017 / 0018 / 0048 / 0049): after
+7. **Leftover-pair slice** (shipped in 0.71.2; ADR 0017 / 0018 / 0048 / 0049 / 0182): after
    IRT main effects, persist closest and farthest post–criterion pairs
-   from the residual leftover map. Do not fork LSIRM; do not invent a
+   from the residual leftover map, and name unexplained leftover
+   `U = R − R̂` when Gabriel coordinates exist so the leftover cell the
+   two-axis map does not reconstruct is not read as leftover residual
+   or leftover-map distance. Do not persist two-axis reconstruction
+   `R̂`. Do not fork LSIRM; do not invent a
    leftover-pair API inside `fast-mlsirm` in this slice.
 8. **Leftover-map axis-share slice** (ADR 0148): persist Gabriel inertia
    `σ_k² / Σ_j σ_j²` of leftover-map axes 1 and 2 on the same residual

docs/adr/0048-persist-lsirm-leftover-pairs.md

@@ -4,7 +4,8 @@
 **Date:** 2026-08-17
 **Amended by:** [ADR 0119](0119-leftover-map-two-dimensional-distance.md) (two leftover-map axes);
 [ADR 0163](0163-leftover-observed-expected.md) (observed Y and expected E);
-[ADR 0164](0164-leftover-map-rank.md) (full map rank)
+[ADR 0164](0164-leftover-map-rank.md) (full map rank);
+[ADR 0182](0182-leftover-map-unexplained.md) (unexplained leftover U)
 
 ## Context
 
@@ -32,12 +33,16 @@ they are never filled with zero. Persist exactly one `closest`
 and one `farthest` observed cell per period report in
 `report_leftover_pair` (3NF, two-or-more-word `snake_case`).
 
-The biplot lives in `lineageweave/leftover_pairs.py` so leftover
 tests do not import `period_report` or `fast_mlsirm`. Distances are
 Euclidean on the two leftover-map axes (ADR 0119). Each leftover row
 also names observed `Y` and expected `E[Y|θ, item]` so residual
 reconciles to `Y − E` (ADR 0163), and names the full singular-value
-rank while distance remains on the first two axes (ADR 0164).
+rank while distance remains on the first two axes (ADR 0164). Each
+leftover row also names unexplained leftover `U = R − R̂` when
+Gabriel coordinates exist so the leftover cell the two-axis map does
+not reconstruct is not read as leftover residual `R` or leftover-map
+distance `d` (ADR 0182). Two-axis reconstruction `R̂` is computed
+internally and is not persisted.
 
 Cascade the rows with `report_period_score`. A leftover post must
 also be a `report_member_score` row, and the leftover criterion

docs/adr/0049-leftover-pair-report-ui.md

@@ -4,7 +4,8 @@
 **Date:** 2026-08-17
 **Amended by:** [ADR 0162](0162-leftover-residual-disclosure.md) (signed residual R);
 [ADR 0163](0163-leftover-observed-expected.md) (observed Y and expected E);
-[ADR 0164](0164-leftover-map-rank.md) (full map rank)
+[ADR 0164](0164-leftover-map-rank.md) (full map rank);
+[ADR 0182](0182-leftover-map-unexplained.md) (unexplained leftover U)
 
 ## Context
 
@@ -21,14 +22,19 @@ second navigation surface.
 On each period-report group, render leftover pairs **above** the
 member list. Each pair is a button: closest or farthest label, post
 title, criterion short label, signed residual `R`, two-axis leftover-map
-distance, full map rank, observed `Y`, and expected `E` when finite.
+distance, full map rank, observed `Y`, expected `E` when finite, and
+unexplained leftover `U` when finite.
 The next action names every available measurement before opening the
-post; no amendment hides another, and rank 0 explicitly names no
-leftover structure.
+post; no amendment hides another, rank 0 explicitly names no
+leftover structure, and unexplained leftover names "leftover map leaves
+unexplained `U` after IRT main effects; open this post to read the
+named criterion" when present. A missing unexplained leftover keeps
+the existing next action.
 Clicking the button opens that post with the same handler as a member
 row. Residual naming is [ADR 0162](0162-leftover-residual-disclosure.md),
 observed/expected naming is [ADR 0163](0163-leftover-observed-expected.md),
-rank naming is [ADR 0164](0164-leftover-map-rank.md).
+rank naming is [ADR 0164](0164-leftover-map-rank.md), unexplained
+leftover naming is [ADR 0182](0182-leftover-map-unexplained.md).
 
 After `make seed`, closest and farthest leftover pairs sit above the
 member list. Click a pair to open that post.

docs/adr/0182-leftover-map-unexplained.md

@@ -0,0 +1,92 @@
+# ADR 0182 — Name unexplained leftover on period-report pair rows
+
+**Decision status:** Accepted
+**Date:** 2026-08-24
+
+Amends [ADR 0048](0048-persist-lsirm-leftover-pairs.md) and
+[ADR 0049](0049-leftover-pair-report-ui.md).
+
+## Context
+
+ADR 0048 already persists leftover-map distance `d = ‖ξ_p − ζ_i‖` and
+leftover residual `R = Y − E[Y|θ, item]` on `report_leftover_pair`.
+ADR 0049 already renders closest and farthest pairs above the member
+list and opens the named post. Distance is the Jeon et al. (2021,
+eq. 3) map gap. Gabriel (1971) reconstructs a matrix from the biplot
+as the inner product of person and item coordinates. The leftover map
+buyers read is two-axis: unused axes pad with zero, and hidden SVD
+axes after the second are dropped. Two-axis reconstruction
+`R̂ = ξ_{1:2} · ζ_{1:2}` is therefore the leftover cell the map
+shows. Hiding unexplained leftover `U = R − R̂` lets a buyer read
+leftover residual `R` or leftover-map distance `d` as the leftover
+the two-axis map does not reconstruct.
+
+This increment does not persist leftover-map reconstruction `R̂`, does
+not persist leftover-map coordinates, does not name leftover-map inner
+product as a separate full-rank column, does not name leftover-map
+cosine, does not name leftover-map length, does not name observed `Y` /
+expected `E`, does not name leftover-map rank, does not split leftover-map
+distance onto two axes, and does not land Post quality on the leftover
+criterion. Leftover-map distance stays full-rank Euclidean.
+Reconstruction `R̂` is computed internally so `U` is honest, then
+discarded.
+
+The unprotected-stack reconstructions for neighbouring leftover facts
+use 0162–0181. This protected-main increment uses **0182** so it does
+not collide with leftover-map reconstruction (0181), leftover-map
+length (0181 on the length stack), leftover-map cosine (0180),
+leftover-map inner product (0179), leftover residual disclosure
+(0178), leftover observed `Y` / expected `E` (0177), leftover-map
+rank (0172), two-axis leftover-map distance (0166), leftover coverage
+(0168), leftover-map axis share (0148), or leftover interaction-map
+persistence (0121).
+
+## Decision
+
+Each leftover pair names `leftover_map_unexplained` — unexplained
+leftover `U = R − R̂` after two-axis Gabriel reconstruction
+`R̂ = ξ_{1:2} · ζ_{1:2}`. Migration `0182` is the single source of
+the column on every install path, fresh or existing -- shipped
+migrations (`0001` / `0012`) are never edited after the fact. The
+column is nullable so older leftover rows keep distance and residual
+without fabricating unexplained leftover. Fallback pairs that have no
+complete-case leftover map omit the value rather than inventing one.
+A rank-0 origin map stores `0.0` (`R = 0` and `R̂ = 0`), not a
+missing value. A non-finite unexplained leftover stores null rather
+than inventing a leftover score. Do not persist
+`leftover_map_reconstruction`.
+
+The pair button shows `U {signed}` next to leftover-map distance `d`
+when the value is finite. Next action: leftover map leaves unexplained
+`U` after IRT main effects; open this post to read the named criterion.
+A missing or non-finite unexplained leftover omits the badge and keeps
+the existing closest/farthest next action. Do not invent a leftover
+score. Do not invent a theta.
+
+## Consequences
+
+`GET /api/reports/{grouping}/{period}` returns
+`leftover_map_unexplained`. After `make seed`, closest and farthest
+leftover pairs sit above the member list with named `U` next to `d`;
+click opens that post. Hidden posts stay hidden.
+
+## Related
+
+Independent of leftover interaction-map persistence, leftover-criterion
+evaluation landing, leftover residual disclosure, leftover observed
+`Y` / expected `E`, leftover-map complete-case coverage, leftover-map
+axis share, leftover pairs on the grouping comparison strip, two-axis
+leftover-map distance, leftover-map rank, leftover-map inner product,
+leftover-map cosine, leftover-map length, and leftover-map
+reconstruction.
+
+## References
+
+Gabriel, K. R. (1971). The biplot graphic display of matrices with
+application to principal component analysis. *Biometrika, 58*(3),
+453–467. https://doi.org/10.1093/biomet/58.3.453
+
+Jeon, M., Jin, I. H., Schweinberger, M., & Baugh, S. (2021). Mapping
+unobserved item–respondent interactions: A latent space item response
+model with interaction map. *Psychometrika, 86*(2), 378–403.
+https://doi.org/10.1007/s11336-021-09762-5

frontend/package.json

@@ -1,7 +1,7 @@
 {
   "name": "frontend",
   "private": true,
-  "version": "2.12.19",
+  "version": "2.12.26",
   "type": "module",
   "scripts": {
     "dev": "vite",

frontend/src/App.test.tsx

@@ -950,6 +950,7 @@ describe("App, authenticated", () => {
                     criterion_code: "sales_lead_specificity",
                     leftover_distance: 0.12,
                     leftover_residual: 0.4,
+                    leftover_map_unexplained: 0.05,
                     observed_response: 2.4,
                     expected_response: 2.0,
                     leftover_map_rank: 1,
@@ -961,6 +962,7 @@ describe("App, authenticated", () => {
                     criterion_code: "general_sentiment_negative",
                     leftover_distance: 1.84,
                     leftover_residual: -1.1,
+                    leftover_map_unexplained: -0.25,
                     observed_response: 0.9,
                     expected_response: 2.0,
                     leftover_map_rank: 1,
@@ -3565,19 +3567,21 @@ describe("App, authenticated", () => {
     });
     expect(closestPair).toHaveTextContent("Closest leftover: Public post · sales-lead");
     expect(closestPair).toHaveTextContent(
-      "Read leftover map rank 1, observed Y 2.40, and expected E 2.00 after IRT main effects, then open this post.",
+      "Leftover map leaves unexplained U +0.05 after IRT main effects. Open this post to read sales-lead.",
     );
     expect(closestPair).toHaveTextContent("R +0.40");
     expect(closestPair).toHaveTextContent("Y 2.40 · E 2.00");
     expect(closestPair).toHaveTextContent("rank 1");
+    expect(closestPair).toHaveTextContent("U +0.05");
     expect(closestPair).toHaveTextContent("d 0.12");
     expect(farthestPair).toHaveTextContent("Farthest leftover: Specification revision requested · negative");
     expect(farthestPair).toHaveTextContent(
-      "Read leftover map rank 1, observed Y 0.90, and expected E 2.00 after IRT main effects, then open this post.",
+      "Leftover map leaves unexplained U −0.25 after IRT main effects. Open this post to read negative.",
     );
     expect(farthestPair).toHaveTextContent("R −1.10");
     expect(farthestPair).toHaveTextContent("Y 0.90 · E 2.00");
     expect(farthestPair).toHaveTextContent("rank 1");
+    expect(farthestPair).toHaveTextContent("U −0.25");
     expect(farthestPair).toHaveTextContent("d 1.84");
     const memberButton = screen.getByRole("button", { name: /open report post: public post/i });
     expect(closestPair.compareDocumentPosition(memberButton) & Node.DOCUMENT_POSITION_FOLLOWING).toBeTruthy();