Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
7cce59d
feat: name leftover-map cross share on leftover pairs (v2.12.29)
seonghobae Aug 24, 2026
364f02b
test: use explicit NaN and infinity predicates in cross-share assertions
seonghobae Aug 24, 2026
468826b
Merge origin/main into feat/leftover-map-cross-share-v21229
seonghobae Aug 24, 2026
740d50f
fix(frontend): restore missing oidcReturnUrl import lost in merge
seonghobae Aug 24, 2026
b2f0f44
Merge branch 'main' into feat/leftover-map-cross-share-v21229
seonghobae Aug 24, 2026
12eaa2b
fix(lineage): floor the cross share on absolute magnitudes
seonghobae Aug 24, 2026
26e7d64
Merge remote-tracking branch 'origin/main' into HEAD
seonghobae Aug 24, 2026
abc0106
fix: wire the leftover-map axis migration into make seed
seonghobae Aug 24, 2026
482de93
fix: update leftover-map rank test for the merged singular-value return
seonghobae Aug 24, 2026
39986e8
Merge remote-tracking branch 'origin/main' into HEAD
seonghobae Aug 24, 2026
ae74349
fix(tests): reconcile leftover-pair tests with merged main state
seonghobae Aug 24, 2026
4438ec6
Merge remote-tracking branch 'origin/main' into HEAD
seonghobae Aug 24, 2026
0b599ad
Merge remote-tracking branch 'origin/main' into feat/leftover-map-cro…
seonghobae Aug 24, 2026
b41f044
fix(docs): fold out-of-sequence 2.12.29 changelog entry into Unreleased
seonghobae Aug 25, 2026
5bc5e5e
Merge remote-tracking branch 'origin/main' into HEAD
seonghobae Aug 25, 2026
01da223
docs: remove leaked merge-conflict tail
seonghobae Aug 25, 2026
1a16e80
fix(leftover): reconcile cross share with raw residual
seonghobae Aug 25, 2026
47c1f59
Merge remote-tracking branch 'origin/feat/leftover-map-cross-share-v2…
seonghobae Aug 25, 2026
353b747
Merge commit 'ec706f350d1a3870c47430cede32b48a7240b4e1' into HEAD
seonghobae Aug 25, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -596,11 +596,12 @@ on those same fixed parameters (Kim, 2006 FIPC). After scoring,
information at the group's mean θ (Lord, 1980 max-info CAT). Rankings
persist to `report_item_information`. After those IRT main effects,
residual SVD leftover pairs on two Gabriel axes (Jeon et al., 2021;
ADR 0017 / 0048 / 0049 / 0119 / 0158 / 0162 / 0163 / 0164 / 0182) persist to
`report_leftover_pair` with signed residual `R`, observed `Y`, expected
`E[Y|θ, item]`, full leftover-map rank, and unexplained leftover
`U = R − R̂` named on the pair row. Leftover-map axis share (Gabriel
inertia of residual SVD axes 1 and 2; ADR 0148) persists to
ADR 0017 / 0048 / 0049 / 0119 / 0148 / 0158 / 0162 / 0163 / 0164 / 0168 /
0182 / 0185) persist to `report_leftover_pair` with signed residual `R`,
observed `Y`, expected `E[Y|θ, item]`, full leftover-map rank, unexplained
leftover `U = R − R̂` named on the pair row, and leftover-map cross share
`x = 2 R̂ U / R²` of raw residual. Leftover-map axis share
(Gabriel inertia of residual SVD axes 1 and 2; ADR 0148) persists to
`report_leftover_map_axis`. Complete-case leftover-map coverage (ADR
0168) persists to `report_leftover_map_coverage` so readers see how
many scored posts entered the factorization. Results persist to
Expand Down
10 changes: 10 additions & 0 deletions CHANGELOG.d/2.12.29-leftover-map-cross-share.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
## 2.12.29 — Leftover-map cross share

- Persist leftover-map cross share `x = 2 R̂ U / R²` of raw residual
on leftover post–criterion pairs (ADR 0185). After
`make seed`, closest and farthest leftover pairs sit above the
member list with `2R̂U/R²` next to leftover-map distance `d`; click
opens that post. Omit the badge when the share is missing. A signed
remainder is shown, never clamped. Never invent a leftover score. Do
not introduce leftover-map explained share `e`, unexplained share
`s`, or reconstruction `R̂`; ADR 0182 remains authoritative for `U`.
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,14 @@ All notable changes to this project are documented here. Format follows
list is captioned “Leftover map used N of M scored posts
(complete-case)”; incomplete rows stay excluded, never filled with
zero.
- Period leftover pair rows now name leftover-map cross share
`x = 2 R̂ U / R²` of raw residual next to leftover-map
distance `d`, then open that post (Gabriel, 1971; Jeon et al., 2021,
eq. 3; ADR 0185). A missing share omits the badge rather than
inventing a leftover score. A signed remainder is shown, never
clamped. Two-axis reconstruction `R̂` stays internal; unexplained
leftover remains the ADR 0182 value `U = R − R̂`. Explained leftover share
`e` and unexplained leftover share `s` are not persisted here.

- The grouping comparison strip now names leftover post–criterion
pairs on each visible row (ADR 0149). After `make seed`, open a
Expand Down
12 changes: 9 additions & 3 deletions backend/app/report_ingestion.py
Original file line number Diff line number Diff line change
Expand Up @@ -445,8 +445,8 @@ async def persist_period_report(
grouping_kind, grouping_key, period_code, rubric_version,
pair_kind, post_id, criterion_code, leftover_distance, leftover_residual,
observed_response, expected_response, leftover_map_rank,
leftover_map_unexplained
) values ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13)
leftover_map_unexplained, leftover_map_cross_share
) values ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14)
""",
grouping_kind,
grouping_key,
Expand All @@ -461,6 +461,7 @@ async def persist_period_report(
pair.expected_response,
pair.leftover_map_rank,
pair.leftover_map_unexplained,
pair.leftover_map_cross_share,
Comment thread
seonghobae marked this conversation as resolved.
)
for axis in report.leftover_map_axes:
await conn.execute(
Expand Down Expand Up @@ -646,7 +647,7 @@ async def fetch_period_reports(
select lp.grouping_key, lp.pair_kind, lp.post_id, lp.criterion_code,
lp.leftover_distance, lp.leftover_residual,
lp.observed_response, lp.expected_response, lp.leftover_map_rank,
lp.leftover_map_unexplained, p.post_title,
lp.leftover_map_unexplained, lp.leftover_map_cross_share, p.post_title,
p.visibility_code, p.corporate_entity_id,
({_SOURCE_CONTEXT_PRESENT_SQL}) as has_real_source_context
from report_leftover_pair lp
Expand Down Expand Up @@ -790,6 +791,11 @@ async def fetch_period_reports(
if row["leftover_map_unexplained"] is None
else float(row["leftover_map_unexplained"])
),
"leftover_map_cross_share": (
None
if row["leftover_map_cross_share"] is None
else float(row["leftover_map_cross_share"])
),
"visibility_code": row["visibility_code"],
"corporate_entity_id": str(row["corporate_entity_id"]),
"has_real_source_context": bool(row["has_real_source_context"]),
Expand Down
20 changes: 17 additions & 3 deletions backend/tests/test_api.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@

from __future__ import annotations

import math
import os
import uuid
from contextlib import closing
Expand Down Expand Up @@ -142,6 +143,11 @@
/ "migrations"
/ "0164_report_leftover_map_rank.sql"
)
_LEFTOVER_MAP_CROSS_SHARE_MIGRATION = (
Path(__file__).resolve().parents[2]
/ "migrations"
/ "0185_report_leftover_map_cross_share.sql"
)
_GLOBAL_ASK_JOB_MIGRATION = (
Path(__file__).resolve().parents[2]
/ "migrations"
Expand Down Expand Up @@ -305,6 +311,7 @@ def seeded_db(demo_analyst_token):
cur.execute(_GLOBAL_ASK_JOB_MIGRATION.read_text())
cur.execute(_LEFTOVER_MAP_AXIS_MIGRATION.read_text())
cur.execute(_LEFTOVER_MAP_UNEXPLAINED_MIGRATION.read_text())
cur.execute(_LEFTOVER_MAP_CROSS_SHARE_MIGRATION.read_text())
cur.execute(
"insert into common_lookup_value (lookup_category, lookup_code, lookup_label) values "
"('corporate_entity_level', 'group', 'Group'), "
Expand Down Expand Up @@ -5049,9 +5056,16 @@ def test_seed_period_report_surfaces_on_get_reports(client, demo_analyst_token,
assert "leftover_map_reconstruction" not in pair
observed = pair.get("observed_response")
expected = pair.get("expected_response")
if observed is None or expected is None:
continue
assert abs(pair["leftover_residual"] - (observed - expected)) < 1e-6
if observed is not None and expected is not None:
assert abs(pair["leftover_residual"] - (observed - expected)) < 1e-6
share = pair.get("leftover_map_cross_share")
assert share is None or isinstance(share, (int, float))
if share is not None:
assert not math.isnan(share)
assert not math.isinf(share)
assert "leftover_map_explained_share" not in pair
assert "leftover_map_unexplained_share" not in pair
assert "leftover_map_reconstruction" not in pair
leftover_axes = high_report.get("leftover_map_axes", [])
assert [axis["axis_index"] for axis in leftover_axes] == [1, 2]
assert all(axis["leftover_singular_value"] >= 0 for axis in leftover_axes)
Expand Down
17 changes: 10 additions & 7 deletions docs/adr/0003-fast-mlsirm-report-integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,13 +100,16 @@ than one large PR:
`information_polytomous` (Lord, 1980 max-info). Persist the ranking
(`report_item_information`) and show the rank-1 item on the Period
reports panel. Do not reimplement an information function here.
7. **Leftover-pair slice** (shipped in 0.71.2; ADR 0017 / 0018 / 0048 / 0049 / 0182): after
IRT main effects, persist closest and farthest post–criterion pairs
from the residual leftover map, and name unexplained leftover
`U = R − R̂` when Gabriel coordinates exist so the leftover cell the
two-axis map does not reconstruct is not read as leftover residual
or leftover-map distance. Do not persist two-axis reconstruction
`R̂`. Do not fork LSIRM; do not invent a
7. **Leftover-pair slice** (shipped in 0.71.2; ADR 0017 / 0018 / 0048 / 0049 /
0182 / 0185): after IRT main effects, persist closest and farthest
post–criterion pairs from the residual leftover map, and name
unexplained leftover `U = R − R̂` and leftover-map cross share
`x = 2 R̂_c U_c / R̃²` of centered leftover when Gabriel coordinates
exist so the identity remainder after two-axis reconstruction is not
read as leftover residual, leftover-map distance, explained leftover
share, or unexplained leftover share. Do not persist leftover-map
explained leftover share `e`, unexplained leftover share `s`, or
two-axis reconstruction `R̂`. Do not fork LSIRM; do not invent a
leftover-pair API inside `fast-mlsirm` in this slice.
8. **Leftover-map axis-share slice** (ADR 0148): persist Gabriel inertia
`σ_k² / Σ_j σ_j²` of leftover-map axes 1 and 2 on the same residual
Expand Down
12 changes: 9 additions & 3 deletions docs/adr/0048-persist-lsirm-leftover-pairs.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@
**Amended by:** [ADR 0119](0119-leftover-map-two-dimensional-distance.md) (two leftover-map axes);
[ADR 0163](0163-leftover-observed-expected.md) (observed Y and expected E);
[ADR 0164](0164-leftover-map-rank.md) (full map rank);
[ADR 0182](0182-leftover-map-unexplained.md) (unexplained leftover U)
[ADR 0182](0182-leftover-map-unexplained.md) (unexplained leftover U);
[ADR 0185](0185-leftover-map-cross-share.md) (leftover-map cross share)

## Context

Expand Down Expand Up @@ -41,8 +42,13 @@ rank while distance remains on the first two axes (ADR 0164). Each
leftover row also names unexplained leftover `U = R − R̂` when
Gabriel coordinates exist so the leftover cell the two-axis map does
not reconstruct is not read as leftover residual `R` or leftover-map
distance `d` (ADR 0182). Two-axis reconstruction `R̂` is computed
internally and is not persisted.
distance `d` (ADR 0182), and names leftover-map cross share
`x = 2 R̂_c U_c / R̃²` of centered leftover when Gabriel coordinates
exist so the identity remainder after two-axis reconstruction is not
read as leftover residual `R`, leftover-map distance `d`, explained
leftover share `e`, or unexplained leftover share `s` (ADR 0185).
Two-axis reconstruction `R̂` / `R̂_c` and centered unexplained leftover
`U_c` are computed internally and are not persisted.

Cascade the rows with `report_period_score`. A leftover post must
also be a `report_member_score` row, and the leftover criterion
Expand Down
32 changes: 19 additions & 13 deletions docs/adr/0049-leftover-pair-report-ui.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@
[ADR 0163](0163-leftover-observed-expected.md) (observed Y and expected E);
[ADR 0164](0164-leftover-map-rank.md) (full map rank);
[ADR 0182](0182-leftover-map-unexplained.md) (unexplained leftover U);
[ADR 0158](0158-leftover-criterion-evaluation-landing.md) (criterion evaluation landing)
[ADR 0158](0158-leftover-criterion-evaluation-landing.md) (criterion evaluation landing);
[ADR 0185](0185-leftover-map-cross-share.md) (leftover-map cross share)

## Context

Expand All @@ -23,20 +24,25 @@ second navigation surface.
On each period-report group, render leftover pairs **above** the
member list. Each pair is a button: closest or farthest label, post
title, criterion short label, signed residual `R`, two-axis leftover-map
distance, full map rank, observed `Y`, expected `E` when finite, and
unexplained leftover `U` when finite.
The next action names every available measurement before opening the
post; no amendment hides another, rank 0 explicitly names no
leftover structure, and unexplained leftover names "leftover map leaves
unexplained `U` after IRT main effects; open this post to read the
named criterion" when present. A missing unexplained leftover keeps
the existing next action.
Clicking the button opens that post with leftover focus so Post
quality marks the named criterion current (ADR 0158). Residual naming
is [ADR 0162](0162-leftover-residual-disclosure.md), observed/expected
distance, full map rank, observed `Y`, expected `E` when finite,
unexplained leftover `U` when finite, and leftover-map cross share next
to distance when finite. The next action names every available
measurement before opening the post; no amendment hides another, rank 0
explicitly names no leftover structure, and unexplained leftover names
"leftover map leaves unexplained `U` after IRT main effects; open this
post to read the named criterion" when present. When leftover-map cross
share is also present, the next action instead names the identity
remainder `x` two leftover-map axes leave of centered leftover after
IRT main effects. A missing or non-finite value falls back in order —
cross share, then unexplained leftover, then the existing
closest/farthest next action. Clicking the button opens that post with
leftover focus so Post quality marks the named criterion current
(ADR 0158). Residual naming is
[ADR 0162](0162-leftover-residual-disclosure.md), observed/expected
naming is [ADR 0163](0163-leftover-observed-expected.md), rank naming
is [ADR 0164](0164-leftover-map-rank.md), unexplained leftover naming
is [ADR 0182](0182-leftover-map-unexplained.md).
is [ADR 0182](0182-leftover-map-unexplained.md), leftover-map cross
share naming is [ADR 0185](0185-leftover-map-cross-share.md).

After `make seed`, closest and farthest leftover pairs sit above the
member list. Click a pair to open that post with the leftover
Expand Down
108 changes: 108 additions & 0 deletions docs/adr/0185-leftover-map-cross-share.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
# ADR 0185 — Name leftover-map cross share on period-report pair rows

**Decision status:** Draft
**Date:** 2026-08-24

Amends [ADR 0048](0048-persist-lsirm-leftover-pairs.md) and
[ADR 0049](0049-leftover-pair-report-ui.md).

## Context

ADR 0048 already persists leftover-map distance `d = ‖ξ_p − ζ_i‖` and
leftover residual `R = Y − E[Y|θ, item]` on `report_leftover_pair`.
ADR 0049 already renders closest and farthest pairs above the member
list and opens the named post. Distance is the Jeon et al. (2021,
eq. 3) map gap. Gabriel (1971) derives coordinates from the centered
matrix, while ADR 0182 defines the auditable cell reconstruction as
`R̂ = ξ_{1:2} · ζ_{1:2}` and unexplained leftover as `U = R − R̂`.
The leftover map is two-axis: unused axes pad with zero, and hidden SVD
axes after the second are dropped. The raw-residual cell identity
`R² = R̂² + U² + 2 R̂ U` therefore yields
`e + s + x = 1` with explained leftover share `e = R̂² / R²`,
unexplained leftover share `s = U² / R²`, and leftover-map cross
share `x = 2 R̂ U / R²`. Hiding `x` lets a buyer read `e + s`
as a complete leftover partition even though the truncated map leaves
an identity remainder. `x` may be negative when reconstruction and
unexplained leftover have opposite signs; a nonnegative CHECK would
reject a mathematically honest cell.

This increment does not persist leftover-map reconstruction `R̂`, does
not add another unexplained-leftover column, does not persist
leftover-map unexplained leftover share `s`,
does not persist leftover-map explained leftover share `e`, does not
persist leftover-map coordinates, does not name leftover-map inner
product, cosine, or length, does not name observed `Y` / expected
`E`, does not name leftover-map rank, does not split leftover-map
distance onto two axes, and does not land Post quality on the leftover
criterion. Leftover-map distance stays full-rank Euclidean.
Reconstruction `R̂` is computed internally so `x` reconciles with the
persisted raw residual and ADR 0182 unexplained leftover.

The unprotected-stack reconstructions for neighbouring leftover facts
use 0162–0184. This protected-main increment uses **0185** so it does
not collide with leftover-map explained leftover share (0184),
leftover-map unexplained leftover share (0183), leftover-map
unexplained leftover (0182), leftover-map reconstruction (0181),
leftover-map length (0181 on the length stack), leftover-map cosine
(0180), leftover-map inner product (0179), leftover residual
disclosure (0178), leftover observed `Y` / expected `E` (0170),
leftover-map rank (0172), two-axis leftover-map distance (0166),
leftover coverage (0165 / 0168), leftover-map axis share (0148), or
leftover interaction-map persistence (0121).

## Decision

Each leftover pair names `leftover_map_cross_share` — leftover-map
cross share `x = 2 R̂ U / R²` of raw residual after
two-axis Gabriel reconstruction `R̂ = ξ_{1:2} · ζ_{1:2}` and
unexplained leftover `U = R − R̂`. Migration `0185` is the
single source of the column on every install path, fresh or existing
-- shipped migrations (`0001` / `0012`) are never edited after the
fact. The column is nullable so older leftover rows keep distance and
residual without fabricating a share. Fallback pairs that have no
complete-case leftover map omit the value rather than inventing one.
A rank-0 origin cell stores `0.0` when `R = R̂ = U = 0`, not a missing
value. A rank-1 cell may retain a nonzero raw-residual cross term when
centering removed a nonzero mean. A non-finite share stores null rather than
inventing a leftover score. A finite negative share is stored; do not
add a nonnegative CHECK. This increment does not introduce
`leftover_map_explained_share`, `leftover_map_unexplained_share`, or
`leftover_map_reconstruction`; ADR 0182 remains authoritative for
`leftover_map_unexplained`.

The pair button shows `2R̂U/R² {share}` next to leftover-map
distance `d` when the value is a finite number, including a signed
negative remainder. Next action: two leftover-map axes leave identity
remainder `x` of raw residual after IRT main effects; open this
post to read the named criterion. A missing or non-finite share omits
the badge and keeps the existing closest/farthest next action. Do not
invent a leftover score. Do not invent a theta.

## Consequences

`GET /api/reports/{grouping}/{period}` returns
`leftover_map_cross_share`. After `make seed`, closest and farthest
leftover pairs sit above the member list with named `2R̂U/R²` next
to `d`; click opens that post. Hidden posts stay hidden.

## Related

Independent of leftover interaction-map persistence, leftover-criterion
evaluation landing, leftover residual disclosure, leftover observed
`Y` / expected `E`, leftover-map complete-case coverage, leftover-map
axis share, leftover pairs on the grouping comparison strip, two-axis
leftover-map distance, leftover-map rank, leftover-map inner product,
leftover-map cosine, leftover-map length, leftover-map reconstruction,
leftover-map unexplained leftover, leftover-map unexplained leftover
share, and leftover-map explained leftover share.

## References

Gabriel, K. R. (1971). The biplot graphic display of matrices with
application to principal component analysis. *Biometrika, 58*(3),
453–467. https://doi.org/10.1093/biomet/58.3.453

Jeon, M., Jin, I. H., Schweinberger, M., & Baugh, S. (2021). Mapping
unobserved item–respondent interactions: A latent space item response
model with interaction map. *Psychometrika, 86*(2), 378–403.
https://doi.org/10.1007/s11336-021-09762-5
Loading
Loading