feat: name leftover-map unexplained share on leftover pairs (v2.12.27)

AGENTS.md

@@ -259,21 +259,23 @@ stops startup instead of leaving a healthy-looking partial schema, and
 application code must not compensate for a missing table.
 
 Period leftover pairs (ADR 0017 / 0018 / 0048 / 0049 / 0119 / 0158 / 0162 /
-0163 / 0164 / 0182) are computed in `lineageweave/leftover_pairs.py` from the
+0163 / 0164 / 0182 / 0203) are computed in `lineageweave/leftover_pairs.py` from the
 residual after a real GRM/GPCM score, never invented. Distances are
 Euclidean on the two-dimensional Gabriel leftover map; missing cells stay
 out of the factorization. Closest and farthest post–criterion pairs
 persist to `report_leftover_pair` with signed residual `R`, observed
 `Y`, and expected `E[Y|θ, item]` so `R = Y − E` remains auditable,
 plus leftover-map rank so rank 0 is not read as structure, and
 unexplained leftover `U = R − R̂` next to leftover-map distance `d`
-after two-axis Gabriel reconstruction. They sit above the member
-list so a click opens that post with the leftover criterion current
-in Post quality (ADR 0158). Two-axis reconstruction `R̂` is not
-persisted. Leftover-map axis share (ADR 0148) is Gabriel inertia of
-residual SVD axes 1 and 2 and persists to `report_leftover_map_axis`.
-Rank-0 residuals emit two zero-share axes; the shares are report-level
-and are not a leftover score. Complete-case coverage (ADR 0168) persists to
+after two-axis Gabriel reconstruction, plus unexplained leftover share
+`s = U_c² / R̃²` of centered leftover ([ADR 0203](docs/adr/0203-leftover-map-unexplained-share.md)).
+Two-axis reconstruction `R̂` / `R̂_c` and centered leftover `U_c` are not
+persisted. They sit above the member list so a click opens that post with
+the leftover criterion current in Post quality (ADR 0158). Leftover-map
+axis share (ADR 0148) is Gabriel inertia of residual SVD axes 1 and 2 and
+persists to `report_leftover_map_axis`. Rank-0 residuals emit two
+zero-share axes; the shares are report-level and are not a leftover score.
+Complete-case coverage (ADR 0168) persists to
 `report_leftover_map_coverage` and captions the pair list with how
 many scored posts entered the map.
 

ARCHITECTURE.md

@@ -596,11 +596,13 @@ on those same fixed parameters (Kim, 2006 FIPC). After scoring,
 information at the group's mean θ (Lord, 1980 max-info CAT). Rankings
 persist to `report_item_information`. After those IRT main effects,
 residual SVD leftover pairs on two Gabriel axes (Jeon et al., 2021;
-ADR 0017 / 0048 / 0049 / 0119 / 0158 / 0162 / 0163 / 0164 / 0182) persist to
+ADR 0017 / 0048 / 0049 / 0119 / 0158 / 0162 / 0163 / 0164 / 0182 /
+0203) persist to
 `report_leftover_pair` with signed residual `R`, observed `Y`, expected
-`E[Y|θ, item]`, full leftover-map rank, and unexplained leftover
-`U = R − R̂` named on the pair row. Leftover-map axis share (Gabriel
-inertia of residual SVD axes 1 and 2; ADR 0148) persists to
+`E[Y|θ, item]`, full leftover-map rank, unexplained leftover
+`U = R − R̂`, and unexplained leftover share `s = U_c² / R̃²` of
+centered leftover named on the pair row. Leftover-map axis share
+(Gabriel inertia of residual SVD axes 1 and 2; ADR 0148) persists to
 `report_leftover_map_axis`. Complete-case leftover-map coverage (ADR
 0168) persists to `report_leftover_map_coverage` so readers see how
 many scored posts entered the factorization. Results persist to

CHANGELOG.d/2.12.27-leftover-map-unexplained-share.md

@@ -0,0 +1,9 @@
+## 2.12.27 — Leftover-map unexplained leftover share
+
+- Persist unexplained leftover share `s = U_c² / R̃²` of centered
+  leftover on leftover post–criterion pairs (ADR 0203). After
+  `make seed`, closest and farthest leftover pairs sit above the
+  member list with `U²/R̃²` next to leftover-map distance `d`; click
+  opens that post. Omit the badge when the share is missing. Never
+  invent a leftover score. Do not persist leftover-map unexplained
+  leftover `U` or reconstruction `R̂`.

CHANGELOG.md

@@ -42,6 +42,12 @@ All notable changes to this project are documented here. Format follows
   list is captioned “Leftover map used N of M scored posts
   (complete-case)”; incomplete rows stay excluded, never filled with
   zero.
+- Period leftover pair rows now name unexplained leftover share
+  `s = U_c² / R̃²` of centered leftover next to leftover-map distance
+  `d`, then open that post (Gabriel, 1971; Jeon et al., 2021, eq. 3;
+  ADR 0202). A missing share omits the badge rather than inventing a
+  leftover score. Centered leftover `U_c` and two-axis reconstruction
+  `R̂_c` stay internal and are not persisted.
 
 - The grouping comparison strip now names leftover post–criterion
   pairs on each visible row (ADR 0149). After `make seed`, open a

backend/app/report_ingestion.py

@@ -445,8 +445,8 @@ async def persist_period_report(
                 grouping_kind, grouping_key, period_code, rubric_version,
                 pair_kind, post_id, criterion_code, leftover_distance, leftover_residual,
                 observed_response, expected_response, leftover_map_rank,
-                leftover_map_unexplained
-            ) values ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13)
+                leftover_map_unexplained, leftover_map_unexplained_share
+            ) values ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14)
             """,
             grouping_kind,
             grouping_key,
@@ -461,6 +461,7 @@ async def persist_period_report(
             pair.expected_response,
             pair.leftover_map_rank,
             pair.leftover_map_unexplained,
+            pair.leftover_map_unexplained_share,
         )
     for axis in report.leftover_map_axes:
         await conn.execute(
@@ -646,7 +647,7 @@ async def fetch_period_reports(
         select lp.grouping_key, lp.pair_kind, lp.post_id, lp.criterion_code,
                lp.leftover_distance, lp.leftover_residual,
                lp.observed_response, lp.expected_response, lp.leftover_map_rank,
-               lp.leftover_map_unexplained, p.post_title,
+               lp.leftover_map_unexplained, lp.leftover_map_unexplained_share, p.post_title,
                p.visibility_code, p.corporate_entity_id,
                ({_SOURCE_CONTEXT_PRESENT_SQL}) as has_real_source_context
         from report_leftover_pair lp
@@ -790,6 +791,11 @@ async def fetch_period_reports(
                             if row["leftover_map_unexplained"] is None
                             else float(row["leftover_map_unexplained"])
                         ),
+                        "leftover_map_unexplained_share": (
+                            None
+                            if row["leftover_map_unexplained_share"] is None
+                            else float(row["leftover_map_unexplained_share"])
+                        ),
                         "visibility_code": row["visibility_code"],
                         "corporate_entity_id": str(row["corporate_entity_id"]),
                         "has_real_source_context": bool(row["has_real_source_context"]),

backend/tests/test_api.py

@@ -162,6 +162,11 @@
     / "migrations"
     / "0182_report_leftover_map_unexplained.sql"
 )
+_LEFTOVER_MAP_UNEXPLAINED_SHARE_MIGRATION = (
+    Path(__file__).resolve().parents[2]
+    / "migrations"
+    / "0202_report_leftover_map_unexplained_share.sql"
+)
 
 
 def _postgres_available() -> bool:
@@ -305,6 +310,7 @@ def seeded_db(demo_analyst_token):
             cur.execute(_GLOBAL_ASK_JOB_MIGRATION.read_text())
             cur.execute(_LEFTOVER_MAP_AXIS_MIGRATION.read_text())
             cur.execute(_LEFTOVER_MAP_UNEXPLAINED_MIGRATION.read_text())
+            cur.execute(_LEFTOVER_MAP_UNEXPLAINED_SHARE_MIGRATION.read_text())
             cur.execute(
                 "insert into common_lookup_value (lookup_category, lookup_code, lookup_label) values "
                 "('corporate_entity_level', 'group', 'Group'), "
@@ -5046,6 +5052,10 @@ def test_seed_period_report_surfaces_on_get_reports(client, demo_analyst_token,
         assert pair["leftover_map_rank"] >= 0
         unexplained = pair.get("leftover_map_unexplained")
         assert unexplained is None or isinstance(unexplained, (int, float))
+        share = pair.get("leftover_map_unexplained_share")
+        assert share is None or isinstance(share, (int, float))
+        if share is not None:
+            assert share >= 0
         assert "leftover_map_reconstruction" not in pair
         observed = pair.get("observed_response")
         expected = pair.get("expected_response")

docs/adr/0003-fast-mlsirm-report-integration.md

@@ -100,13 +100,14 @@ than one large PR:
    `information_polytomous` (Lord, 1980 max-info). Persist the ranking
    (`report_item_information`) and show the rank-1 item on the Period
    reports panel. Do not reimplement an information function here.
-7. **Leftover-pair slice** (shipped in 0.71.2; ADR 0017 / 0018 / 0048 / 0049 / 0182): after
+7. **Leftover-pair slice** (shipped in 0.71.2; ADR 0017 / 0018 / 0048 / 0049 / 0182 / 0202): after
    IRT main effects, persist closest and farthest post–criterion pairs
    from the residual leftover map, and name unexplained leftover
-   `U = R − R̂` when Gabriel coordinates exist so the leftover cell the
-   two-axis map does not reconstruct is not read as leftover residual
-   or leftover-map distance. Do not persist two-axis reconstruction
-   `R̂`. Do not fork LSIRM; do not invent a
+   `U = R − R̂` and unexplained leftover share `s = U_c² / R̃²` of
+   centered leftover when Gabriel coordinates exist so the leftover
+   cell the two-axis map does not reconstruct is not read as leftover
+   residual or leftover-map distance. Do not persist two-axis
+   reconstruction `R̂` / `R̂_c` or centered leftover `U_c`. Do not fork LSIRM; do not invent a
    leftover-pair API inside `fast-mlsirm` in this slice.
 8. **Leftover-map axis-share slice** (ADR 0148): persist Gabriel inertia
    `σ_k² / Σ_j σ_j²` of leftover-map axes 1 and 2 on the same residual

docs/adr/0048-persist-lsirm-leftover-pairs.md

@@ -5,7 +5,8 @@
 **Amended by:** [ADR 0119](0119-leftover-map-two-dimensional-distance.md) (two leftover-map axes);
 [ADR 0163](0163-leftover-observed-expected.md) (observed Y and expected E);
 [ADR 0164](0164-leftover-map-rank.md) (full map rank);
-[ADR 0182](0182-leftover-map-unexplained.md) (unexplained leftover U)
+[ADR 0182](0182-leftover-map-unexplained.md) (unexplained leftover U);
+[ADR 0202](0202-leftover-map-unexplained-share.md) (unexplained leftover share s)
 
 ## Context
 
@@ -41,8 +42,10 @@ rank while distance remains on the first two axes (ADR 0164). Each
 leftover row also names unexplained leftover `U = R − R̂` when
 Gabriel coordinates exist so the leftover cell the two-axis map does
 not reconstruct is not read as leftover residual `R` or leftover-map
-distance `d` (ADR 0182). Two-axis reconstruction `R̂` is computed
-internally and is not persisted.
+distance `d` (ADR 0182), and names unexplained leftover share
+`s = U_c² / R̃²` of centered leftover the same way (ADR 0202).
+Two-axis reconstruction `R̂` / `R̂_c` and centered leftover `U_c` are
+computed internally and are not persisted.
 
 Cascade the rows with `report_period_score`. A leftover post must
 also be a `report_member_score` row, and the leftover criterion

docs/adr/0049-leftover-pair-report-ui.md

@@ -6,7 +6,8 @@
 [ADR 0163](0163-leftover-observed-expected.md) (observed Y and expected E);
 [ADR 0164](0164-leftover-map-rank.md) (full map rank);
 [ADR 0182](0182-leftover-map-unexplained.md) (unexplained leftover U);
-[ADR 0158](0158-leftover-criterion-evaluation-landing.md) (criterion evaluation landing)
+[ADR 0158](0158-leftover-criterion-evaluation-landing.md) (criterion evaluation landing);
+[ADR 0203](0203-leftover-map-unexplained-share.md) (unexplained leftover share s)
 
 ## Context
 
@@ -24,19 +25,23 @@ On each period-report group, render leftover pairs **above** the
 member list. Each pair is a button: closest or farthest label, post
 title, criterion short label, signed residual `R`, two-axis leftover-map
 distance, full map rank, observed `Y`, expected `E` when finite, and
-unexplained leftover `U` when finite.
+unexplained leftover `U` when finite, and unexplained leftover share
+`s` of centered leftover after IRT main effects when Gabriel
+coordinates exist.
 The next action names every available measurement before opening the
 post; no amendment hides another, rank 0 explicitly names no
 leftover structure, and unexplained leftover names "leftover map leaves
 unexplained `U` after IRT main effects; open this post to read the
-named criterion" when present. A missing unexplained leftover keeps
-the existing next action.
+named criterion" when present. A missing unexplained leftover or a
+missing share keeps the existing next action.
 Clicking the button opens that post with leftover focus so Post
 quality marks the named criterion current (ADR 0158). Residual naming
 is [ADR 0162](0162-leftover-residual-disclosure.md), observed/expected
 naming is [ADR 0163](0163-leftover-observed-expected.md), rank naming
 is [ADR 0164](0164-leftover-map-rank.md), unexplained leftover naming
-is [ADR 0182](0182-leftover-map-unexplained.md).
+is [ADR 0182](0182-leftover-map-unexplained.md), and unexplained
+leftover share naming is
+[ADR 0203](0203-leftover-map-unexplained-share.md).
 
 After `make seed`, closest and farthest leftover pairs sit above the
 member list. Click a pair to open that post with the leftover

docs/adr/0203-leftover-map-unexplained-share.md

@@ -0,0 +1,105 @@
+# ADR 0203 — Name unexplained leftover share on period-report pair rows
+
+**Decision status:** Accepted
+**Date:** 2026-08-24
+
+Amends [ADR 0048](0048-persist-lsirm-leftover-pairs.md) and
+[ADR 0049](0049-leftover-pair-report-ui.md).
+
+## Context
+
+ADR 0048 already persists leftover-map distance `d = ‖ξ_p − ζ_i‖` and
+leftover residual `R = Y − E[Y|θ, item]` on `report_leftover_pair`.
+ADR 0049 already renders closest and farthest pairs above the member
+list and opens the named post. Distance is the Jeon et al. (2021,
+eq. 3) map gap. Gabriel (1971) reconstructs a *centered* matrix from
+the biplot as the inner product of person and item coordinates. The
+leftover map buyers read is two-axis: unused axes pad with zero, and
+hidden SVD axes after the second are dropped. Two-axis reconstruction
+`R̂_c = ξ_{1:2} · ζ_{1:2}` therefore recovers centered leftover
+`R̃ = R − center`, not raw residual `R`. Hiding unexplained leftover
+share `s = U_c² / R̃²` (`U_c = R̃ − R̂_c`) lets a buyer read leftover
+residual `R` or leftover-map distance `d` as the leftover the two-axis
+map does not reconstruct. Subtracting reconstruction from raw `R`
+(or dividing by `R²`) leaves the grand mean inside the named leftover,
+so a fully reconstructed rank-1 cell would look unexplained whenever
+`center ≠ 0`.
+
+This increment does not persist leftover-map reconstruction `R̂` or
+`R̂_c`, does not persist leftover-map unexplained leftover `U` or
+`U_c`, does not persist leftover-map coordinates, does not name
+leftover-map inner product, cosine, or length, does not name observed
+`Y` / expected `E`, does not name leftover-map rank, does not split
+leftover-map distance onto two axes, and does not land Post quality on
+the leftover criterion. Leftover-map distance stays full-rank
+Euclidean. Centered leftover `U_c` and reconstruction `R̂_c` are
+computed internally so `s` is honest, then discarded.
+
+The unprotected-stack reconstructions for neighbouring leftover facts
+use 0162–0182. This increment originally claimed **0183**, but that
+number collided with an unrelated, already-merged GNB navigation ADR
+(`0183-gnb-four-korean-chrome.md`) by the time this branch reached
+`main`, so it renumbers to **0203** at merge -- above every ADR number
+in use on `main` (≤ 0201) so it cannot collide again. It does not
+collide with leftover-map unexplained leftover (0182), leftover-map
+reconstruction (0181), leftover-map length (0181 on the length stack),
+leftover-map cosine (0180), leftover-map inner product (0179), leftover
+residual disclosure (0178), leftover observed `Y` / expected `E`
+(0170), leftover-map rank (0172), two-axis leftover-map distance
+(0166), leftover coverage (0165 / 0168), leftover-map axis share
+(0148), or leftover interaction-map persistence (0121).
+
+## Decision
+
+Each leftover pair names `leftover_map_unexplained_share` — unexplained
+leftover share `s = U_c² / R̃²` of centered leftover after two-axis
+Gabriel reconstruction `R̂_c = ξ_{1:2} · ζ_{1:2}`. Migration `0202`
+is the single source of the column on every install path, fresh or
+existing -- shipped migrations (`0001` / `0012`) are never edited after
+the fact. The column is nullable so older leftover rows keep distance
+and residual without fabricating a share. A leftover map with no
+complete-case rectangle names no pair at all (ADR 0168), so there is
+no row to fabricate a share on. A rank-0 origin map that still has a
+complete-case rectangle stores `0.0` (`R̃ = 0` and `U_c = 0`), not a
+missing value. A fully reconstructed rank-1 cell stores `0.0` even when
+the grand mean of `R` is nonzero. A non-finite or negative share stores
+null rather than inventing a leftover score. Named check
+`leftover_pair_unexplained_share_nonnegative_chk` rejects a stored
+negative share. Do not persist `leftover_map_unexplained` or
+`leftover_map_reconstruction`.
+
+The pair button shows `U²/R̃² {share}` next to leftover-map distance
+`d` when the value is a finite non-negative number. Next action:
+leftover map leaves unexplained share `s` of centered leftover after
+IRT main effects; open this post to read the named criterion. A
+missing or non-finite share omits the badge and keeps the existing
+closest/farthest next action. Do not invent a leftover score. Do not
+invent a theta.
+
+## Consequences
+
+`GET /api/reports/{grouping}/{period}` returns
+`leftover_map_unexplained_share`. After `make seed`, closest and
+farthest leftover pairs sit above the member list with named
+`U²/R̃²` next to `d`; click opens that post. Hidden posts stay hidden.
+
+## Related
+
+Independent of leftover interaction-map persistence, leftover-criterion
+evaluation landing, leftover residual disclosure, leftover observed
+`Y` / expected `E`, leftover-map complete-case coverage, leftover-map
+axis share, leftover pairs on the grouping comparison strip, two-axis
+leftover-map distance, leftover-map rank, leftover-map inner product,
+leftover-map cosine, leftover-map length, leftover-map reconstruction,
+and leftover-map unexplained leftover.
+
+## References
+
+Gabriel, K. R. (1971). The biplot graphic display of matrices with
+application to principal component analysis. *Biometrika, 58*(3),
+453–467. https://doi.org/10.1093/biomet/58.3.453
+
+Jeon, M., Jin, I. H., Schweinberger, M., & Baugh, S. (2021). Mapping
+unobserved item–respondent interactions: A latent space item response
+model with interaction map. *Psychometrika, 86*(2), 378–403.
+https://doi.org/10.1007/s11336-021-09762-5