Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 12 additions & 9 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -192,15 +192,18 @@ in the same spirit) -- never against real data, per the hard rule above.
against a live local stack (`make up`) and self-skip without one -- see
[README.md](README.md#local-product-stack-docker-compose).

Period leftover pairs (ADR 0048 / 0049 / 0119 / 0162 / 0163 / 0164) are computed in
`lineageweave/leftover_pairs.py` from the residual after a real
GRM/GPCM score, never invented. Distances are Euclidean on the
two-dimensional Gabriel leftover map; missing cells stay out of
the factorization. Closest and farthest post–criterion pairs
persist to `report_leftover_pair` with signed residual `R`, observed
`Y`, and expected `E[Y|θ, item]` so `R = Y − E` remains auditable,
plus leftover-map rank so rank 0 is not read as structure. They sit
above the member list so a click opens that post.
Period leftover pairs (ADR 0017 / 0018 / 0048 / 0049 / 0119 / 0162 / 0163 /
0164) are computed in `lineageweave/leftover_pairs.py` from the residual
after a real GRM/GPCM score, never invented. Distances are Euclidean on
the two-dimensional Gabriel leftover map; missing cells stay out of the
factorization. Closest and farthest post–criterion pairs persist to
`report_leftover_pair` with signed residual `R`, observed `Y`, and
expected `E[Y|θ, item]` so `R = Y − E` remains auditable, plus
leftover-map rank so rank 0 is not read as structure. They sit above the
member list so a click opens that post. Leftover-map axis share
(ADR 0148) is Gabriel inertia of residual SVD axes 1 and 2 and persists
to `report_leftover_map_axis`. Rank-0 residuals emit two zero-share
axes; the shares are report-level and are not a leftover score.

`frontend/` has its own toolchain (Node pinned via `frontend/mise.toml`,
pnpm via Corepack -- do not add a second Node package manager or a
Expand Down
11 changes: 7 additions & 4 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -594,9 +594,11 @@ on those same fixed parameters (Kim, 2006 FIPC). After scoring,
information at the group's mean θ (Lord, 1980 max-info CAT). Rankings
persist to `report_item_information`. After those IRT main effects,
residual SVD leftover pairs on two Gabriel axes (Jeon et al., 2021;
ADR 0048 / 0119 / 0162 / 0163 / 0164) persist to `report_leftover_pair` with
signed residual `R`, observed `Y`, expected `E[Y|θ, item]`, and full
leftover-map rank. Results persist to
ADR 0017 / 0048 / 0119 / 0162 / 0163 / 0164) persist to
`report_leftover_pair` with signed residual `R`, observed `Y`, expected
`E[Y|θ, item]`, and full leftover-map rank. Leftover-map axis share
(Gabriel inertia of residual SVD axes 1 and 2; ADR 0148) persists to
`report_leftover_map_axis`. Results persist to
`report_period_score` / `report_member_score`.
`GET /api/reports/{grouping}` lists the trend;
`GET /api/reports/{grouping}/{period}` is ABAC-filtered;
Expand All @@ -610,7 +612,8 @@ open ticket title, status lookup label, and due date when one exists. The home p
the actual mean θ, the FIPC delta, the CAT-selected item, leftover
closest/farthest pairs (signed residual `R`, observed `Y`, expected
`E`, full rank, and two-axis leftover-map distance `d` after IRT main
effects) above the member list, and the
effects) above the member list, leftover-map axis share for residual
SVD axes 1 and 2, and the
PU / corp / thread comparison -- never a placeholder. TEPP is unchanged.

## Phase 6b: Knowledge Graph as a real Ontology + Semantic Layer
Expand Down
11 changes: 11 additions & 0 deletions CHANGELOG.d/2.12.19-leftover-map-axis-share.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# 2.12.19 — Leftover-map axis share

## Added

- Persist leftover-map axis share (Gabriel inertia of residual SVD
axes 1 and 2, `σ²/Σσ²`) on each period report (ADR 0148). Rank-0
residuals emit two zero-share axes; missing cells stay out of the
factorization. After `make seed`, leftover-axis badges sit with the
leftover pairs; the caption tells the buyer to open a leftover pair.
Axis share is report-level 3NF and is not hidden when leftover pairs
are ABAC-filtered.
11 changes: 11 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,17 @@ All notable changes to this project are documented here. Format follows
environment, so local OIDC and synthetic-data workflows resolve the same
pinned dependencies as CI.

## [2.12.19] - 2026-08-24

### Added

- Period reports now persist leftover-map axis share (Gabriel inertia of
residual SVD axes 1 and 2) next to leftover pairs (ADR 0148). Rank-0
residuals emit two zero-share axes. After `make seed`, leftover-axis
badges sit with the leftover pairs; the caption tells the buyer to open
a leftover pair. Axis share is report-level 3NF and is not hidden when
leftover pairs are ABAC-filtered.

## [2.12.18] - 2026-08-24

### Fixed
Expand Down
4 changes: 3 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,9 @@ and to read its mean θ and member posts, then open a post. Those
members land immediately under that next action, ahead of Other Corp
and the week strip. After `make seed`, leftover closest/farthest pairs
sit above the member list with leftover-map rank; rank 0 names no
leftover structure. Opening Public post names the next action: read
leftover structure. Leftover-map axis share badges name Gabriel inertia
of axes 1 and 2; open a leftover pair to read the post–criterion cell.
The shares do not invent a leftover score. Opening Public post names the next action: read
Event Lineage, Keyman, and evaluation on that post. The popup Event
Lineage DAG marks that post current. After that current node, the
popup names Keyman and evaluation as the next read. After landed
Expand Down
9 changes: 8 additions & 1 deletion backend/app/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -2379,8 +2379,15 @@ async def read_period_reports(
{key: value for key, value in pair.items() if key != "has_real_source_context"}
for pair in leftover_pairs
]
leftover_map_axes = list(report.get("leftover_map_axes", []))
visible.append(
{**report, "members": members, "leftover_pairs": leftover_pairs, "post_count": len(members)}
{
**report,
"members": members,
"leftover_pairs": leftover_pairs,
"leftover_map_axes": leftover_map_axes,
"post_count": len(members),
}
)
Comment thread
seonghobae marked this conversation as resolved.
return {"grouping_kind": grouping_kind, "period_code": period_code, "reports": visible}

Expand Down
40 changes: 39 additions & 1 deletion backend/app/report_ingestion.py
Original file line number Diff line number Diff line change
Expand Up @@ -351,7 +351,7 @@ async def persist_period_report(
period_code: str,
report: PeriodReport,
) -> None:
"""Replace the stored report, member scores, leftover pairs, and item bank."""
"""Replace the stored report, member scores, leftover pairs, leftover-map axes, and item bank."""
await conn.execute(
"""
delete from report_period_score
Expand Down Expand Up @@ -460,6 +460,22 @@ async def persist_period_report(
pair.expected_response,
pair.leftover_map_rank,
)
for axis in report.leftover_map_axes:
await conn.execute(
"""
insert into report_leftover_map_axis (
grouping_kind, grouping_key, period_code, rubric_version,
axis_index, leftover_singular_value, leftover_share
) values ($1,$2,$3,$4,$5,$6,$7)
""",
grouping_kind,
grouping_key,
period_code,
RUBRIC_VERSION,
axis.axis_index,
axis.leftover_singular_value,
axis.leftover_share,
)
Comment on lines +463 to +478

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 Info: Axis-share replace relies on cascade delete

Neither persist_period_report nor the seed's _persist_seed_period_report explicitly deletes report_leftover_map_axis rows before re-inserting; replacement depends on ON DELETE CASCADE from the leading report_period_score delete (migration 0169 FK). This mirrors the existing report_leftover_pair pattern and is correct as long as that period-score delete always precedes the inserts.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.



def _groups_from_rows(
Expand Down Expand Up @@ -620,6 +636,17 @@ async def fetch_period_reports(
period_code,
RUBRIC_VERSION,
)
leftover_axes = await conn.fetch(
"""
select grouping_key, axis_index, leftover_singular_value, leftover_share
from report_leftover_map_axis
where grouping_kind = $1 and period_code = $2 and rubric_version = $3
order by grouping_key, axis_index
""",
grouping_kind,
period_code,
RUBRIC_VERSION,
)
status_labels = await labels_for_codes(
conn,
[row["ticket_status_code"] for row in members if row["ticket_status_code"]],
Expand All @@ -633,6 +660,9 @@ async def fetch_period_reports(
leftover_by_group: dict[str, list[asyncpg.Record]] = defaultdict(list)
for row in leftover:
leftover_by_group[row["grouping_key"]].append(row)
leftover_axes_by_group: dict[str, list[asyncpg.Record]] = defaultdict(list)
for row in leftover_axes:
leftover_axes_by_group[row["grouping_key"]].append(row)
payload: list[dict[str, Any]] = []
for header in headers:
grouping_key = header["grouping_key"]
Expand Down Expand Up @@ -724,6 +754,14 @@ async def fetch_period_reports(
}
for row in leftover_by_group.get(header["grouping_key"], [])
],
"leftover_map_axes": [
{
"axis_index": int(row["axis_index"]),
"leftover_singular_value": float(row["leftover_singular_value"]),
"leftover_share": float(row["leftover_share"]),
}
for row in leftover_axes_by_group.get(header["grouping_key"], [])
],
}
)
return payload
Expand Down
10 changes: 10 additions & 0 deletions backend/tests/test_api.py
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,11 @@
/ "migrations"
/ "0164_report_leftover_map_rank.sql"
)
_LEFTOVER_MAP_AXIS_MIGRATION = (
Path(__file__).resolve().parents[2]
/ "migrations"
/ "0169_report_leftover_map_axis.sql"
)


def _postgres_available() -> bool:
Expand Down Expand Up @@ -249,6 +254,7 @@ def seeded_db(demo_analyst_token):
cur.execute(_CHANNEL_WEIGHT_MIGRATION.read_text())
cur.execute(_LEFTOVER_OBSERVED_EXPECTED_MIGRATION.read_text())
cur.execute(_LEFTOVER_MAP_RANK_MIGRATION.read_text())
cur.execute(_LEFTOVER_MAP_AXIS_MIGRATION.read_text())
cur.execute(
"insert into common_lookup_value (lookup_category, lookup_code, lookup_label) values "
"('corporate_entity_level', 'group', 'Group'), "
Expand Down Expand Up @@ -4739,6 +4745,10 @@ def test_seed_period_report_surfaces_on_get_reports(client, demo_analyst_token,
if observed is None or expected is None:
continue
assert abs(pair["leftover_residual"] - (observed - expected)) < 1e-6
leftover_axes = high_report.get("leftover_map_axes", [])
assert [axis["axis_index"] for axis in leftover_axes] == [1, 2]
assert all(axis["leftover_singular_value"] >= 0 for axis in leftover_axes)
assert all(0.0 <= axis["leftover_share"] <= 1.0 for axis in leftover_axes)

week3 = client.get(
"/api/reports/process_unit/2026-W03",
Expand Down
6 changes: 5 additions & 1 deletion docs/adr/0003-fast-mlsirm-report-integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,10 +100,14 @@ than one large PR:
`information_polytomous` (Lord, 1980 max-info). Persist the ranking
(`report_item_information`) and show the rank-1 item on the Period
reports panel. Do not reimplement an information function here.
7. **Leftover-pair slice** (shipped in 0.71.2; ADR 0017 / 0018): after
7. **Leftover-pair slice** (shipped in 0.71.2; ADR 0017 / 0018 / 0048 / 0049): after
IRT main effects, persist closest and farthest post–criterion pairs
from the residual leftover map. Do not fork LSIRM; do not invent a
leftover-pair API inside `fast-mlsirm` in this slice.
8. **Leftover-map axis-share slice** (ADR 0148): persist Gabriel inertia
`σ_k² / Σ_j σ_j²` of leftover-map axes 1 and 2 on the same residual
SVD. Rank-0 residuals emit two zero-share axes. Do not invent a
leftover score.

**TEPP boundary.** [ARCHITECTURE.md](../../ARCHITECTURE.md) already
assigns calibrated temporal/event measurement to
Expand Down
60 changes: 60 additions & 0 deletions docs/adr/0148-leftover-map-axis-share.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# ADR 0148 — Persist leftover-map axis share

**Decision status:** Accepted
**Date:** 2026-08-24

## Context

ADR 0048 persists closest and farthest leftover post–criterion pairs
from a Gabriel (1971) biplot of the residual `R = Y − E[Y|θ, item]`
after a real GRM/GPCM score (Jeon et al., 2021, eq. 3). Buyers can
open those pairs (ADR 0049) but cannot yet read how much leftover-map
structure sits on axis 1 versus axis 2.

Gabriel inertia of leftover-map axis `k` is `σ_k² / Σ_j σ_j²`. That
share is a report-level property of the residual SVD, not a
post-identifying leftover score and not a second theta. Denormalizing
it onto each leftover pair would violate 3NF.

`fast-mlsirm` still exposes no leftover-pair or leftover-map API.
LineageWeave must not fork LSIRM or invent leftover numbers when the
residual is rank-0.

## Decision

After the same residual SVD that produces leftover pairs, persist
exactly two leftover-map axes (axis 1 and axis 2) per period report in
`report_leftover_map_axis` (3NF, two-or-more-word `snake_case`).

Share is `σ_k² / Σ_j σ_j²` from the leftover singular values that
survive the leftover singular floor. Rank-0 residuals emit two
zero-share axes so `make seed` can name leftover-map structure without
inventing a leftover score. Missing response cells stay out of the
factorization.

Cascade the rows with `report_period_score`. Axes are aggregate and
non-identifying: ABAC that hides leftover pairs does not hide axis
share. Do not store a second theta. Do not invent leftover numbers.

The biplot lives in `lineageweave/leftover_pairs.py` so leftover tests
do not import `period_report` or `fast_mlsirm`.

## Consequences

Rebuild and seed write leftover-map axes in the same transaction as
leftover pairs. `GET /api/reports/{grouping}/{period}` returns
`leftover_map_axes` next to `leftover_pairs`. The Period reports panel
shows leftover-axis share badges and a caption that tells the buyer to
open a leftover pair. Migration `0169_report_leftover_map_axis.sql`
upgrades volumes that already applied `0001`.

## References

Gabriel, K. R. (1971). The biplot graphic display of matrices with
application to principal component analysis. *Biometrika, 58*(3),
453–467. https://doi.org/10.1093/biomet/58.3.453

Jeon, M., Jin, I. H., Schweinberger, M., & Baugh, S. (2021). Mapping
unobserved item–respondent interactions: A latent space item response
model with interaction map. *Psychometrika, 86*(2), 378–403.
https://doi.org/10.1007/s11336-021-09762-5
2 changes: 1 addition & 1 deletion frontend/package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "frontend",
"private": true,
"version": "2.12.18",
"version": "2.12.19",
"type": "module",
"scripts": {
"dev": "vite",
Expand Down
17 changes: 17 additions & 0 deletions frontend/src/App.test.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -966,6 +966,18 @@ describe("App, authenticated", () => {
leftover_map_rank: 1,
},
],
leftover_map_axes: [
{
axis_index: 1,
leftover_singular_value: 1.84,
leftover_share: 0.82,
},
{
axis_index: 2,
leftover_singular_value: 0.86,
leftover_share: 0.18,
},
],
members: [
{
post_id: "post-1",
Expand Down Expand Up @@ -3531,6 +3543,11 @@ describe("App, authenticated", () => {
expect(screen.getByText(/TEST-PU-REPORT/)).toBeInTheDocument();
expect(screen.getAllByText("shared metric").length).toBeGreaterThan(0);
expect(screen.getAllByText(/CAT: sales-lead I=0\.70/).length).toBeGreaterThan(0);
expect(screen.getAllByText(/leftover axis 1 82%/).length).toBeGreaterThan(0);
expect(screen.getAllByText(/leftover axis 2 18%/).length).toBeGreaterThan(0);
expect(screen.getByLabelText("Leftover-map axis share")).toHaveTextContent(
"Open a leftover pair to read the post–criterion cell",
);
expect(screen.getByRole("button", { name: /open report period 2026-W03/i })).toHaveTextContent(
"vs 2026-W02: +0.92",
);
Expand Down
15 changes: 15 additions & 0 deletions frontend/src/App.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -3409,6 +3409,21 @@ function ReportsPanel({
{report.selected_items[0].information.toFixed(2)}
</span>
)}
{report.leftover_map_axes?.map((axis) => (
<span key={axis.axis_index} className="post-badge">
{tf("leftover axis {axis} {share}%", {
axis: axis.axis_index,
share: (axis.leftover_share * 100).toFixed(0),
})}
</span>
))}
{report.leftover_map_axes && report.leftover_map_axes.length > 0 && (
<p aria-label={t("Leftover-map axis share")}>
{t(
"Leftover-map axis share is Gabriel inertia of residual SVD axes 1 and 2. Open a leftover pair to read the post–criterion cell. The shares do not invent a leftover score.",
)}
</p>
)}
{report.leftover_pairs && report.leftover_pairs.length > 0 && (
<LeftoverPairList
pairs={report.leftover_pairs}
Expand Down
7 changes: 7 additions & 0 deletions frontend/src/api.ts
Original file line number Diff line number Diff line change
Expand Up @@ -776,6 +776,12 @@ export interface LeftoverPair {
leftover_map_rank?: number | null;
}

export interface LeftoverMapAxis {
axis_index: number;
leftover_singular_value: number;
leftover_share: number;
}

export interface PeriodGroupReport {
grouping_key: string;
grouping_label?: string;
Expand All @@ -791,6 +797,7 @@ export interface PeriodGroupReport {
members: ReportMember[];
selected_items: SelectedReportItem[];
leftover_pairs: LeftoverPair[];
leftover_map_axes?: LeftoverMapAxis[];
}

export interface PeriodReports {
Expand Down
3 changes: 3 additions & 0 deletions frontend/src/i18n.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,9 @@ describe("i18n", () => {
"Search",
"Page",
"Answer",
"leftover axis {axis} {share}%",
"Leftover-map axis share",
"Leftover-map axis share is Gabriel inertia of residual SVD axes 1 and 2. Open a leftover pair to read the post–criterion cell. The shares do not invent a leftover score.",
"Leftover pairs",
"Closest leftover",
"Farthest leftover",
Expand Down
Loading
Loading