diff --git a/AGENTS.md b/AGENTS.md index 1728f9e61..1fa4b1274 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -186,12 +186,14 @@ in the same spirit) -- never against real data, per the hard rule above. against a live local stack (`make up`) and self-skip without one -- see [README.md](README.md#local-product-stack-docker-compose). -Period leftover pairs (ADR 0017 / 0018) are computed in +Period leftover pairs (ADR 0048 / 0049 / 0168) are computed in `lineageweave/leftover_pairs.py` from the residual after a real GRM/GPCM score, never invented. Missing cells stay out of the Gabriel factorization. Closest and farthest post–criterion pairs -persist to `report_leftover_pair` and sit above the member list so -a click opens that post. +persist `leftover_observed_score`, `leftover_expected_score`, and +`leftover_residual` (`R = Y − E`) to `report_leftover_pair` and sit +above the member list so a click opens that post. Do not invent a +leftover score or a second theta. `frontend/` has its own toolchain (Node pinned via `frontend/mise.toml`, pnpm via Corepack -- do not add a second Node package manager or a diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index d0280ff97..ddf463753 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -588,8 +588,9 @@ on those same fixed parameters (Kim, 2006 FIPC). After scoring, `information_polytomous` ranks the shared-bank items by Fisher information at the group's mean θ (Lord, 1980 max-info CAT). Rankings persist to `report_item_information`. After those IRT main effects, -residual SVD leftover pairs (Jeon et al., 2021; ADR 0017) persist to -`report_leftover_pair`. Results persist to +residual SVD leftover pairs (Jeon et al., 2021; ADR 0048 / 0168) persist to +`report_leftover_pair` with observed Y, expected E[Y|θ, item], and +leftover residual R = Y − E. Results persist to `report_period_score` / `report_member_score`. `GET /api/reports/{grouping}` lists the trend; `GET /api/reports/{grouping}/{period}` is ABAC-filtered; @@ -601,7 +602,8 @@ bank as the dummy high/low band rows, so comparison-strip click through opens those DAG posts. Report members include the earliest open ticket title, status lookup label, and due date when one exists. The home page renders the actual mean θ, the FIPC delta, the CAT-selected item, leftover -closest/farthest pairs above the member list, and the +closest/farthest pairs (observed, expected, leftover residual) above +the member list, and the PU / corp / thread comparison -- never a placeholder. TEPP is unchanged. ## Phase 6b: Knowledge Graph as a real Ontology + Semantic Layer diff --git a/CHANGELOG.d/2.12.20-leftover-observed-expected.md b/CHANGELOG.d/2.12.20-leftover-observed-expected.md new file mode 100644 index 000000000..d2c8e266d --- /dev/null +++ b/CHANGELOG.d/2.12.20-leftover-observed-expected.md @@ -0,0 +1,13 @@ +# 2.12.20 — Leftover observed Y and expected E + +## Added + +- Persist leftover observed score `Y` and expected score `E[Y|θ, item]` + on each closest/farthest leftover pair so leftover residual + `R = Y − E` is checkable (ADR 0168). Missing cells stay out of the + factorization. Existing leftover rows are derived; migration `0108` + deletes incomplete rows rather than inventing `Y` or `E`. +- After `make seed`, leftover pair badges name observed, expected, and + leftover residual above the member list. Click still opens that post. + Copy tells the buyer to check leftover residual equals observed minus + expected. Do not invent a leftover score or a second theta. diff --git a/CHANGELOG.md b/CHANGELOG.md index c8ed1a099..d26808de0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,6 +12,18 @@ All notable changes to this project are documented here. Format follows environment, so local OIDC and synthetic-data workflows resolve the same pinned dependencies as CI. +## [2.12.20] - 2026-08-24 + +### Added + +- Leftover closest/farthest pairs now persist observed score `Y` and + expected score `E[Y|θ, item]` beside leftover residual `R = Y − E` + (ADR 0168). After `make seed`, leftover pair badges name observed, + expected, and leftover residual above the member list; click still + opens that post. Check leftover residual equals observed minus + expected. Missing cells stay out of the factorization. Do not invent + a leftover score or a second theta. + ## [2.12.6] - 2026-08-20 ### Added diff --git a/backend/app/report_ingestion.py b/backend/app/report_ingestion.py index 50614b0ad..b2f0e89f6 100644 --- a/backend/app/report_ingestion.py +++ b/backend/app/report_ingestion.py @@ -443,8 +443,9 @@ async def persist_period_report( """ insert into report_leftover_pair ( grouping_kind, grouping_key, period_code, rubric_version, - pair_kind, post_id, criterion_code, leftover_distance, leftover_residual - ) values ($1,$2,$3,$4,$5,$6,$7,$8,$9) + pair_kind, post_id, criterion_code, leftover_distance, leftover_residual, + leftover_observed_score, leftover_expected_score + ) values ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11) """, grouping_kind, grouping_key, @@ -455,6 +456,8 @@ async def persist_period_report( pair.criterion_code, pair.leftover_distance, pair.leftover_residual, + pair.leftover_observed_score, + pair.leftover_expected_score, ) @@ -601,7 +604,8 @@ async def fetch_period_reports( leftover = await conn.fetch( # nosemgrep: python.lang.security.audit.sqli.asyncpg-sqli.asyncpg-sqli f""" select lp.grouping_key, lp.pair_kind, lp.post_id, lp.criterion_code, - lp.leftover_distance, lp.leftover_residual, p.post_title, + lp.leftover_distance, lp.leftover_residual, + lp.leftover_observed_score, lp.leftover_expected_score, p.post_title, p.visibility_code, p.corporate_entity_id, ({_SOURCE_CONTEXT_PRESENT_SQL}) as has_real_source_context from report_leftover_pair lp @@ -698,6 +702,8 @@ async def fetch_period_reports( "criterion_code": str(row["criterion_code"]), "leftover_distance": float(row["leftover_distance"]), "leftover_residual": float(row["leftover_residual"]), + "leftover_observed_score": float(row["leftover_observed_score"]), + "leftover_expected_score": float(row["leftover_expected_score"]), "visibility_code": row["visibility_code"], "corporate_entity_id": str(row["corporate_entity_id"]), "has_real_source_context": bool(row["has_real_source_context"]), diff --git a/backend/tests/test_api.py b/backend/tests/test_api.py index 438b4786a..3038eba5a 100644 --- a/backend/tests/test_api.py +++ b/backend/tests/test_api.py @@ -113,6 +113,11 @@ / "migrations" / "0102_project_bound_summary_event.sql" ) +_LEFTOVER_OBSERVED_EXPECTED_MIGRATION = ( + Path(__file__).resolve().parents[2] + / "migrations" + / "0108_report_leftover_observed_expected.sql" +) def _postgres_available() -> bool: @@ -226,6 +231,7 @@ def seeded_db(demo_analyst_token): cur.execute(_MAJOR_EVENT_ACTION_MIGRATION.read_text()) cur.execute(_PROJECT_BOUND_ACTION_MIGRATION.read_text()) cur.execute(_PROJECT_BOUND_EVENT_MIGRATION.read_text()) + cur.execute(_LEFTOVER_OBSERVED_EXPECTED_MIGRATION.read_text()) cur.execute( "insert into common_lookup_value (lookup_category, lookup_code, lookup_label) values " "('corporate_entity_level', 'group', 'Group'), " @@ -4518,6 +4524,14 @@ def test_seed_period_report_surfaces_on_get_reports(client, demo_analyst_token, assert leftover_kinds <= {"closest", "farthest"} assert all(pair["post_title"] for pair in high_report.get("leftover_pairs", [])) assert all(pair["leftover_distance"] >= 0 for pair in high_report.get("leftover_pairs", [])) + assert all( + abs( + pair["leftover_residual"] + - (pair["leftover_observed_score"] - pair["leftover_expected_score"]) + ) + < 1e-6 + for pair in high_report.get("leftover_pairs", []) + ) week3 = client.get( "/api/reports/process_unit/2026-W03", diff --git a/docker/postgres-init/migrate.sh b/docker/postgres-init/migrate.sh index f329117d6..01beb7b78 100644 --- a/docker/postgres-init/migrate.sh +++ b/docker/postgres-init/migrate.sh @@ -18,7 +18,7 @@ for migration in /opt/lineageweave/migrations/*.sql; do migration_name=${migration##*/} case "$migration_name" in 0012_*|0013_*|0014_*|0015_*|0016_*|0017_*|0018_*|0019_*|0020_*|0021_*|0022_*|0023_*|0024_*|0025_*|0026_*|0027_*|0028_*|0029_*|0030_*|0031_*|0032_*|0033_*|0034_*|0035_*|0036_*|0037_*|0038_*|0039_*|0040_*|0041_*|0042_*|0043_*|0044_*|0045_*|0046_*|0047_*|0048_*|0049_*|0050_*) ;; - 0060_*|0100_*|0101_*|0102_*) ;; + 0060_*|0100_*|0101_*|0102_*|0108_*) ;; *) continue ;; esac printf 'Applying %s\n' "$migration_name" diff --git a/docs/adr/0168-leftover-observed-expected.md b/docs/adr/0168-leftover-observed-expected.md new file mode 100644 index 000000000..4ef1d844d --- /dev/null +++ b/docs/adr/0168-leftover-observed-expected.md @@ -0,0 +1,66 @@ +# ADR 0168 — Persist leftover observed Y and expected E[Y|θ, item] + +**Decision status:** Accepted +**Date:** 2026-08-24 + +Amends [ADR 0048](0048-persist-lsirm-leftover-pairs.md) and +[ADR 0049](0049-leftover-pair-report-ui.md). Independent of leftover-map +coordinates, leftover criterion landing, leftover complete-case +coverage, leftover-map axis share, leftover comparison-strip reuse, +and two-axis leftover-map distance. + +## Context + +ADR 0048 already computes leftover residual `R = Y − E[Y|θ, item]` +after IRT main effects (Jeon et al., 2021, eq. 3; Gabriel, 1971) and +persists closest/farthest pairs with `leftover_distance` and +`leftover_residual`. The buyer could see distance `d` but could not +check that residual identity: observed `Y` and expected `E` were +computed, then dropped. + +Backfilling `Y = R` and `E = 0` would invent an expected score. +Persisting only `R` leaves the identity uncheckable. + +## Decision + +Each leftover pair persists `leftover_observed_score` (`Y`) and +`leftover_expected_score` (`E[Y|θ, item]`) next to +`leftover_residual`. The check +`abs(R − (Y − E)) < 1e-9` is named +`leftover_pair_residual_identity`. Missing or non-finite cells never +become pairs. Existing leftover rows are derived: migration `0108` +deletes incomplete rows so rebuild/seed rewrites honest `Y` and `E` +instead of fabricating them. + +Do not invent a leftover score. Do not invent a second theta. + +After `make seed`, leftover pair badges name observed, expected, and +leftover residual above the member list. Click still opens that post. +Copy tells the buyer to check leftover residual equals observed minus +expected. + +## Consequences + +`GET /api/reports/{grouping}/{period}` returns `leftover_observed_score` +and `leftover_expected_score` on each pair. Hidden posts stay hidden +through the same ABAC join as members. Migration `0108` upgrades +volumes that already applied `0012`. Fresh `0001` schema includes the +same columns and identity check. + +## Related + +Does not mix into leftover persist-map, leftover criterion landing, +leftover complete-case coverage, leftover-map axis share, leftover +comparison-strip reuse, or two-axis leftover-map distance. Issues #79 +and #87 stay open. + +## References + +Gabriel, K. R. (1971). The biplot graphic display of matrices with +application to principal component analysis. *Biometrika, 58*(3), +453–467. https://doi.org/10.1093/biomet/58.3.453 + +Jeon, M., Jin, I. H., Schweinberger, M., & Baugh, S. (2021). Mapping +unobserved item–respondent interactions: A latent space item response +model with interaction map. *Psychometrika, 86*(2), 378–403. +https://doi.org/10.1007/s11336-021-09762-5 diff --git a/frontend/package.json b/frontend/package.json index e2e996bbe..a82321845 100644 --- a/frontend/package.json +++ b/frontend/package.json @@ -1,7 +1,7 @@ { "name": "frontend", "private": true, - "version": "2.12.6", + "version": "2.12.20", "type": "module", "scripts": { "dev": "vite", diff --git a/frontend/src/App.test.tsx b/frontend/src/App.test.tsx index 7462abd2c..9447b546b 100644 --- a/frontend/src/App.test.tsx +++ b/frontend/src/App.test.tsx @@ -41,6 +41,9 @@ describe("App, unauthenticated", () => { state: expect.objectContaining({ returnUrl: expect.stringMatching(/^\//) }), }), ); + // Persisted as a fallback in case the OIDC state round-trip is dropped + // (see oidcReturnUrl.ts's restoreOidcReturnUrl, consumed in main.tsx). + expect(window.sessionStorage.getItem("lineageweave.oidc.returnUrl")).toMatch(/^\//); }); }); @@ -932,6 +935,8 @@ describe("App, authenticated", () => { criterion_code: "sales_lead_specificity", leftover_distance: 0.12, leftover_residual: 0.4, + leftover_observed_score: 3.4, + leftover_expected_score: 3.0, }, { pair_kind: "farthest", @@ -940,6 +945,8 @@ describe("App, authenticated", () => { criterion_code: "general_sentiment_negative", leftover_distance: 1.84, leftover_residual: -1.1, + leftover_observed_score: 1.0, + leftover_expected_score: 2.1, }, ], members: [ @@ -3357,13 +3364,19 @@ describe("App, authenticated", () => { }); expect(closestPair).toHaveTextContent("Closest leftover: Public post · sales-lead"); expect(closestPair).toHaveTextContent( - "Open this post to read the criterion it sat closest to after main effects.", + "Open this post to read the closest leftover criterion. Check leftover residual equals observed minus expected.", ); + expect(closestPair).toHaveTextContent("observed 3.40"); + expect(closestPair).toHaveTextContent("expected 3.00"); + expect(closestPair).toHaveTextContent("leftover 0.40"); expect(closestPair).toHaveTextContent("d 0.12"); expect(farthestPair).toHaveTextContent("Farthest leftover: Specification revision requested · negative"); expect(farthestPair).toHaveTextContent( - "Open this post to read the criterion it sat farthest from after main effects.", + "Open this post to read the farthest leftover criterion. Check leftover residual equals observed minus expected.", ); + expect(farthestPair).toHaveTextContent("observed 1.00"); + expect(farthestPair).toHaveTextContent("expected 2.10"); + expect(farthestPair).toHaveTextContent("leftover -1.10"); expect(farthestPair).toHaveTextContent("d 1.84"); const memberButton = screen.getByRole("button", { name: /open report post: public post/i }); expect(closestPair.compareDocumentPosition(memberButton) & Node.DOCUMENT_POSITION_FOLLOWING).toBeTruthy(); diff --git a/frontend/src/App.tsx b/frontend/src/App.tsx index 6fba0dd41..c84486fcd 100644 --- a/frontend/src/App.tsx +++ b/frontend/src/App.tsx @@ -3396,8 +3396,8 @@ function ReportsPanel({ pair.pair_kind === "farthest" ? "Farthest leftover" : "Closest leftover"; const nextAction = pair.pair_kind === "farthest" - ? "Open this post to read the criterion it sat farthest from after main effects." - : "Open this post to read the criterion it sat closest to after main effects."; + ? "Open this post to read the farthest leftover criterion. Check leftover residual equals observed minus expected." + : "Open this post to read the closest leftover criterion. Check leftover residual equals observed minus expected."; const criterion = criterionShortLabel(pair.criterion_code); return (
  • {nextAction} + + observed {pair.leftover_observed_score.toFixed(2)} + + + expected {pair.leftover_expected_score.toFixed(2)} + + + leftover {pair.leftover_residual.toFixed(2)} + d {pair.leftover_distance.toFixed(2)}
    • @@ -4610,7 +4619,8 @@ export default function App({ showLabPanels = false }: { showLabPanels?: boolean
    - {destination === "admin" ? : null}