diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 3deebd50..5ca921df 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -61,6 +61,7 @@ boundaries above remain the target modular MSA architecture. | `tepp_simulation` | known-truth temporal/event data generation | | `validation_core` | RMSE, bias, coverage, graph, and Monte Carlo metrics | | `tepp_api` | versioned DTO, schema, and export contracts | +| `psychometric_fit` | CPU `f64` ESEM loading recovery and event-time DSEM lag gates | | `subevent_containment` | subevent event-time intervals must stay inside the parent | | `prediction_contradiction` | Allen promotion gate: `before`/`after` stay contradictory; `meets`/`met_by` stay unsupported; coverage is required before unmatched predicted mass may be authorized for promotion | | `provider_receipt` | provider-disclosure field-code receipts; source text and identity are not disclosable | diff --git a/CHANGELOG.md b/CHANGELOG.md index 361b8267..6d8822e1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,7 @@ All notable changes to TEPP are documented here. The format follows Keep a Chang ### Added +- `psychometric_fit` CPU `f64` ESEM/DSEM fit: exploratory OLS recovers known cross-loadings from admitted log-ratio or logistic-normal coordinates with computed RMSE below a zero-loading collapse; reverse or zero event-time lagged paths fail closed; a good global fit cannot reclassify formative or network constructs as reflective (ADR 0005). No new migration number (`#45` still owns `0007`). - `subevent_containment` parent-window gate: a half-open subevent interval that starts before or ends after its parent cannot attach; recovered containment flags match known truth at a higher computed rate than accepting every child (ADR 0003). - `prediction_contradiction` promotion gate: `temporal_core` Allen classification refuses `before`/`after` as contradiction and `meets`/`met_by` as unsupported adjacency; `refuse_promotion` and `require_observed_coverage` refuse partial overlap that leaves unmatched predicted mass; `refuse_contradiction_or_adjacency` is the weaker contradiction/adjacency filter only; evidence available after the knowledge cutoff is ineligible. Label agreement is not RMSE recovery (ADR 0002, ADR 0016). Canonical docs name the crate, not a pull-request number, as the landable authority; `scripts/validate_documentation.py` fail-closes on `landable coverage gate is PR #N` and inverted or paraphrased forms (`PR #N is the landable coverage gate`, `the landable gate is PR #N`, `coverage-authority landing PR #N`, `merge PR #N as the coverage-authority`) including drafts #93, #94, #97, #101, #102, #104, #108, #109, #111, and #112. The hourly queue lock also fail-closes when those drafts are omitted from Keep-unmerged sentences, when a Keep-unmerged sentence is negated, or when the naruon live-HTTP *subject* is not PR #107 with #87 and #105 kept unmerged. - `provider_receipt` disclosure audit: a receipt records purpose and field codes sent to a model provider; source text, source identity, and blanket PII masking fail closed; recovered field codes match known truth at a higher computed rate than a collapsed set (ADR 0009). diff --git a/Cargo.lock b/Cargo.lock index ce130f3d..13d48362 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -899,6 +899,10 @@ dependencies = [ name = "provider_receipt" version = "0.1.0" +[[package]] +name = "psychometric_fit" +version = "0.1.0" + [[package]] name = "quote" version = "1.0.47" diff --git a/Cargo.toml b/Cargo.toml index d56f30e4..209e3947 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -11,6 +11,7 @@ members = [ "crates/tepp_simulation", "crates/validation_core", "crates/tepp_api", + "crates/psychometric_fit", "crates/subevent_containment", "crates/prediction_contradiction", "crates/provider_receipt", @@ -35,6 +36,7 @@ default-members = [ "crates/tepp_simulation", "crates/validation_core", "crates/tepp_api", + "crates/psychometric_fit", "crates/subevent_containment", "crates/prediction_contradiction", "crates/provider_receipt", diff --git a/README.md b/README.md index 5262ce29..b88cac31 100644 --- a/README.md +++ b/README.md @@ -38,6 +38,7 @@ crates/corpus_split crates/tepp_simulation crates/validation_core crates/tepp_api +crates/psychometric_fit crates/subevent_containment crates/prediction_contradiction crates/provider_receipt diff --git a/crates/psychometric_fit/Cargo.toml b/crates/psychometric_fit/Cargo.toml new file mode 100644 index 00000000..6bc3b848 --- /dev/null +++ b/crates/psychometric_fit/Cargo.toml @@ -0,0 +1,17 @@ +[package] +name = "psychometric_fit" +description = "CPU f64 ESEM loading recovery and event-time DSEM lag gates." +version.workspace = true +edition.workspace = true +rust-version.workspace = true +license.workspace = true +authors.workspace = true +repository.workspace = true +homepage.workspace = true +readme.workspace = true +keywords.workspace = true +categories.workspace = true +publish = false + +[lints] +workspace = true diff --git a/crates/psychometric_fit/src/error.rs b/crates/psychometric_fit/src/error.rs new file mode 100644 index 00000000..3fe02db0 --- /dev/null +++ b/crates/psychometric_fit/src/error.rs @@ -0,0 +1,80 @@ +//! Fail-closed ESEM/DSEM fit errors. + +use std::fmt; + +/// A fail-closed psychometric-fit error. +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +#[non_exhaustive] +pub enum PsychometricFitError { + /// Raw simplex proportions were offered as Euclidean fit inputs. + RawProportionForbidden, + /// Empty, rank-unsupported, unequal-length, or non-finite numeric input. + InvalidNumericInput, + /// A predictor matrix has a singular Gram matrix. + SingularDesign, + /// A lagged path would move backward or stay put in event time. + ReverseEventTimePath, + /// A good global fit was used to reinterpret a formative or network + /// construct as reflective. + FormativeReinterpretationForbidden, + /// The construct class is unresolved, so reflective interpretation is + /// unavailable. + UnresolvedConstruct, +} + +impl fmt::Display for PsychometricFitError { + fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { + let message = match self { + Self::RawProportionForbidden => { + "raw topic proportions are forbidden psychometric fit inputs" + } + Self::InvalidNumericInput => "invalid psychometric fit numeric input", + Self::SingularDesign => "singular psychometric fit design matrix", + Self::ReverseEventTimePath => "DSEM lagged paths cannot move backward in event time", + Self::FormativeReinterpretationForbidden => { + "formative or network constructs cannot be reinterpreted as reflective" + } + Self::UnresolvedConstruct => "construct class is unresolved", + }; + formatter.write_str(message) + } +} + +impl std::error::Error for PsychometricFitError {} + +#[cfg(test)] +mod tests { + use super::PsychometricFitError; + + #[test] + fn error_messages_are_stable() { + for (error, message) in [ + ( + PsychometricFitError::RawProportionForbidden, + "raw topic proportions are forbidden psychometric fit inputs", + ), + ( + PsychometricFitError::InvalidNumericInput, + "invalid psychometric fit numeric input", + ), + ( + PsychometricFitError::SingularDesign, + "singular psychometric fit design matrix", + ), + ( + PsychometricFitError::ReverseEventTimePath, + "DSEM lagged paths cannot move backward in event time", + ), + ( + PsychometricFitError::FormativeReinterpretationForbidden, + "formative or network constructs cannot be reinterpreted as reflective", + ), + ( + PsychometricFitError::UnresolvedConstruct, + "construct class is unresolved", + ), + ] { + assert_eq!(error.to_string(), message); + } + } +} diff --git a/crates/psychometric_fit/src/fit.rs b/crates/psychometric_fit/src/fit.rs new file mode 100644 index 00000000..7275ac2f --- /dev/null +++ b/crates/psychometric_fit/src/fit.rs @@ -0,0 +1,474 @@ +//! CPU `f64` ESEM loading recovery and event-time DSEM lag gates. + +use crate::PsychometricFitError; + +/// Coordinate system admitted into an ESEM/DSEM fit. +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +#[non_exhaustive] +pub enum FitCoordinateKind { + /// Additive log-ratio coordinates. + AdditiveLogRatio, + /// Orthonormal isometric log-ratio coordinates. + IsometricLogRatio, + /// Logistic-normal latent coordinates. + LogisticNormal, + /// Raw simplex topic proportions. Forbidden as a fit input. + RawProportion, +} + +impl FitCoordinateKind { + /// Stable wire name for the coordinate kind. + #[must_use] + pub const fn as_str(self) -> &'static str { + match self { + Self::AdditiveLogRatio => "alr", + Self::IsometricLogRatio => "ilr", + Self::LogisticNormal => "logistic_normal", + Self::RawProportion => "raw_proportion", + } + } + + /// Return whether the coordinate kind may enter a structural fit. + #[must_use] + pub const fn admits_structural_fit(self) -> bool { + !matches!(self, Self::RawProportion) + } +} + +/// Higher-order construct class before reflective ESEM interpretation. +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +#[non_exhaustive] +pub enum FitConstructClass { + /// Reflective indicators of a common latent factor. + Reflective, + /// Formative or composite indicators that define the construct. + Formative, + /// Interacting indicators that belong in a network model. + Network, + /// Insufficient evidence to classify the construct. + Unresolved, +} + +impl FitConstructClass { + /// Stable wire name for the construct class. + #[must_use] + pub const fn as_str(self) -> &'static str { + match self { + Self::Reflective => "reflective", + Self::Formative => "formative", + Self::Network => "network", + Self::Unresolved => "unresolved", + } + } + + /// Return whether reflective ESEM is admissible for this class. + #[must_use] + pub const fn admits_esem_fit(self) -> bool { + matches!(self, Self::Reflective) + } +} + +/// Admit only log-ratio or logistic-normal coordinates into a fit. +/// +/// # Errors +/// +/// Returns [`PsychometricFitError::RawProportionForbidden`] for raw simplex +/// proportions. +pub fn admit_fit_coordinates(kind: FitCoordinateKind) -> Result<(), PsychometricFitError> { + if kind.admits_structural_fit() { + Ok(()) + } else { + Err(PsychometricFitError::RawProportionForbidden) + } +} + +/// Interpret a classified construct as reflective after a fit. +/// +/// A good global fit statistic is not authority to reinterpret a formative or +/// network structure as reflective (ADR 0005). +/// +/// # Errors +/// +/// Returns [`PsychometricFitError::FormativeReinterpretationForbidden`] for +/// formative or network classes and +/// [`PsychometricFitError::UnresolvedConstruct`] when the class is unresolved. +pub fn interpret_fit_as_reflective( + classified: FitConstructClass, + global_fit_acceptable: bool, +) -> Result { + let _ = global_fit_acceptable; + match classified { + FitConstructClass::Reflective => Ok(FitConstructClass::Reflective), + FitConstructClass::Unresolved => Err(PsychometricFitError::UnresolvedConstruct), + FitConstructClass::Formative => { + Err(PsychometricFitError::FormativeReinterpretationForbidden) + } + FitConstructClass::Network => Err(PsychometricFitError::FormativeReinterpretationForbidden), + } +} + +/// Recover an ESEM loading matrix by OLS of each indicator on every factor. +/// +/// Each inner slice is one variable's observation vector. At most two factors +/// are inverted on this CPU `f64` reference path so the Gram matrix stays +/// explicit. Cross-loadings are retained. +/// +/// # Errors +/// +/// Returns coordinate, payload, or singularity errors from the OLS path. +pub fn recover_esem_loadings( + factor_scores: &[Vec], + indicators: &[Vec], + kind: FitCoordinateKind, +) -> Result>, PsychometricFitError> { + admit_fit_coordinates(kind)?; + if factor_scores.len() > 2 { + return Err(PsychometricFitError::InvalidNumericInput); + } + let observation_count = factor_scores.first().map_or(0, Vec::len); + let mut centered_factors = Vec::new(); + for values in factor_scores { + if observation_count < 2 || values.len() != observation_count { + return Err(PsychometricFitError::InvalidNumericInput); + } + centered_factors.push(center(values)?); + } + if centered_factors.is_empty() { + return Err(PsychometricFitError::InvalidNumericInput); + } + let mut loadings = Vec::new(); + for values in indicators { + if values.len() != observation_count { + return Err(PsychometricFitError::InvalidNumericInput); + } + let centered_indicator = center(values)?; + loadings.push(ordinary_least_squares_loadings( + ¢ered_factors, + ¢ered_indicator, + )?); + } + if loadings.is_empty() { + return Err(PsychometricFitError::InvalidNumericInput); + } + Ok(loadings) +} + +/// Root-mean-square error between known-truth and recovered loading matrices. +/// +/// # Errors +/// +/// Returns [`PsychometricFitError::InvalidNumericInput`] when either matrix is +/// empty or the shapes differ. +pub fn loading_recovery_rmse( + truth: &[Vec], + recovered: &[Vec], +) -> Result { + if truth.is_empty() || truth.len() != recovered.len() { + return Err(PsychometricFitError::InvalidNumericInput); + } + let mut sum_sq = 0.0_f64; + let mut count = 0_u32; + for (truth_row, recovered_row) in truth.iter().zip(recovered) { + if truth_row.is_empty() || truth_row.len() != recovered_row.len() { + return Err(PsychometricFitError::InvalidNumericInput); + } + for (truth_value, recovered_value) in truth_row.iter().zip(recovered_row) { + let residual = truth_value - recovered_value; + sum_sq += residual * residual; + count += 1; + } + } + Ok((sum_sq / f64::from(count)).sqrt()) +} + +/// Recover a DSEM lagged path only when the predictor precedes the outcome. +/// +/// # Errors +/// +/// Returns [`PsychometricFitError::ReverseEventTimePath`] when +/// `outcome_event_time` is not strictly later than `predictor_event_time`, +/// and OLS payload or singularity errors otherwise. +pub fn recover_dsem_lagged_path( + predictor_event_time: i64, + outcome_event_time: i64, + predictor: &[f64], + outcome: &[f64], +) -> Result { + if outcome_event_time <= predictor_event_time { + return Err(PsychometricFitError::ReverseEventTimePath); + } + let loadings = recover_esem_loadings( + &[predictor.to_vec()], + &[outcome.to_vec()], + FitCoordinateKind::LogisticNormal, + )?; + Ok(loadings[0][0]) +} + +fn center(values: &[f64]) -> Result, PsychometricFitError> { + require_finite_slice(values)?; + let mean = values.iter().sum::() / values.len() as f64; + require_finite(mean)?; + let mut centered = Vec::with_capacity(values.len()); + for value in values { + centered.push(require_finite(value - mean)?); + } + Ok(centered) +} + +fn ordinary_least_squares_loadings( + centered_factors: &[Vec], + centered_indicator: &[f64], +) -> Result, PsychometricFitError> { + let gram = gram_matrix(centered_factors)?; + let inverse = invert_gram(&gram)?; + let mut cross = vec![0.0_f64; centered_factors.len()]; + for (factor_index, factor) in centered_factors.iter().enumerate() { + let mut total = 0.0_f64; + for (score, outcome) in factor.iter().zip(centered_indicator) { + total += score * outcome; + } + cross[factor_index] = require_finite(total)?; + } + let mut loadings = vec![0.0_f64; centered_factors.len()]; + for (row_index, inverse_row) in inverse.iter().enumerate() { + let mut total = 0.0_f64; + for (weight, value) in inverse_row.iter().zip(&cross) { + total += weight * value; + } + loadings[row_index] = require_finite(total)?; + } + Ok(loadings) +} + +fn gram_matrix(centered_factors: &[Vec]) -> Result>, PsychometricFitError> { + let rank = centered_factors.len(); + let mut gram = vec![vec![0.0_f64; rank]; rank]; + for row in 0..rank { + for column in 0..rank { + let mut total = 0.0_f64; + for (left, right) in centered_factors[row].iter().zip(¢ered_factors[column]) { + total += left * right; + } + gram[row][column] = require_finite(total)?; + } + } + Ok(gram) +} + +fn invert_gram(gram: &[Vec]) -> Result>, PsychometricFitError> { + match gram.len() { + 1 => { + let value = gram[0][0]; + if value <= 0.0 { + return Err(PsychometricFitError::SingularDesign); + } + Ok(vec![vec![require_finite(1.0 / value)?]]) + } + 2 => { + let a = gram[0][0]; + let b = gram[0][1]; + let c = gram[1][0]; + let d = gram[1][1]; + let determinant = require_finite(a * d - b * c)?; + if determinant.abs() <= 0.0 { + return Err(PsychometricFitError::SingularDesign); + } + Ok(vec![ + vec![ + require_finite(d / determinant)?, + require_finite(-b / determinant)?, + ], + vec![ + require_finite(-c / determinant)?, + require_finite(a / determinant)?, + ], + ]) + } + _ => Err(PsychometricFitError::InvalidNumericInput), + } +} + +fn require_finite_slice(values: &[f64]) -> Result<(), PsychometricFitError> { + for value in values { + require_finite(*value)?; + } + Ok(()) +} + +fn require_finite(value: f64) -> Result { + if value.is_finite() { + Ok(value) + } else { + Err(PsychometricFitError::InvalidNumericInput) + } +} + +#[cfg(test)] +mod tests { + use super::{ + FitConstructClass, FitCoordinateKind, admit_fit_coordinates, invert_gram, + loading_recovery_rmse, recover_dsem_lagged_path, recover_esem_loadings, require_finite, + }; + use crate::PsychometricFitError; + + #[test] + fn local_branches_cover_fit_gates_and_inversions() { + cover_coordinate_and_inversion_gates(); + cover_recovery_payload_gates(); + cover_rmse_and_indicator_gates(); + } + + fn cover_coordinate_and_inversion_gates() { + assert!(FitCoordinateKind::AdditiveLogRatio.admits_structural_fit()); + assert!(!FitCoordinateKind::RawProportion.admits_structural_fit()); + admit_fit_coordinates(FitCoordinateKind::IsometricLogRatio).expect("ilr"); + assert_eq!( + admit_fit_coordinates(FitCoordinateKind::RawProportion), + Err(PsychometricFitError::RawProportionForbidden) + ); + assert_eq!( + require_finite(f64::INFINITY), + Err(PsychometricFitError::InvalidNumericInput) + ); + assert_eq!( + invert_gram(&[vec![0.0]]), + Err(PsychometricFitError::SingularDesign) + ); + assert_eq!( + invert_gram(&[vec![-1.0]]), + Err(PsychometricFitError::SingularDesign) + ); + assert_eq!( + invert_gram(&[vec![1e-320]]), + Err(PsychometricFitError::InvalidNumericInput) + ); + let inverted = invert_gram(&[vec![2.0]]).expect("1x1"); + assert!((inverted[0][0] - 0.5).abs() < f64::EPSILON); + assert_eq!( + invert_gram(&[vec![1.0, 0.0], vec![0.0, 0.0]]), + Err(PsychometricFitError::SingularDesign) + ); + assert_eq!( + invert_gram(&[vec![f64::MAX, f64::MAX], vec![f64::MAX, f64::MAX]]), + Err(PsychometricFitError::InvalidNumericInput) + ); + let negative = invert_gram(&[vec![1.0, 2.0], vec![2.0, 1.0]]).expect("neg det"); + assert!((negative[0][0] + 1.0 / 3.0).abs() < 1e-12); + let two = invert_gram(&[vec![1.0, 0.0], vec![0.0, 2.0]]).expect("2x2"); + assert!((two[0][0] - 1.0).abs() < f64::EPSILON); + assert!((two[1][1] - 0.5).abs() < f64::EPSILON); + assert_eq!( + invert_gram(&[vec![1.0], vec![2.0], vec![3.0]]), + Err(PsychometricFitError::InvalidNumericInput) + ); + } + + fn cover_recovery_payload_gates() { + assert_eq!( + recover_esem_loadings( + &[vec![1.0, 2.0], vec![3.0]], + &[vec![1.0, 2.0]], + FitCoordinateKind::AdditiveLogRatio + ), + Err(PsychometricFitError::InvalidNumericInput) + ); + assert_eq!( + recover_esem_loadings( + &[vec![1.0, f64::INFINITY]], + &[vec![1.0, 2.0]], + FitCoordinateKind::AdditiveLogRatio + ), + Err(PsychometricFitError::InvalidNumericInput) + ); + assert_eq!( + recover_esem_loadings( + &[vec![f64::MAX, f64::MAX]], + &[vec![1.0, 2.0]], + FitCoordinateKind::AdditiveLogRatio + ), + Err(PsychometricFitError::InvalidNumericInput) + ); + let collinear = recover_esem_loadings( + &[vec![1.0, 2.0, 3.0], vec![2.0, 4.0, 6.0]], + &[vec![1.0, 2.0, 3.0]], + FitCoordinateKind::AdditiveLogRatio, + ); + assert_eq!(collinear, Err(PsychometricFitError::SingularDesign)); + assert_eq!( + recover_dsem_lagged_path(5, 4, &[0.0, 1.0], &[0.0, 1.0]), + Err(PsychometricFitError::ReverseEventTimePath) + ); + let forward = recover_dsem_lagged_path(1, 2, &[0.0, 1.0], &[0.0, 2.0]).expect("forward"); + assert!((forward - 2.0).abs() < 1e-12); + assert_eq!( + loading_recovery_rmse(&[vec![]], &[vec![]]), + Err(PsychometricFitError::InvalidNumericInput) + ); + let rmse = loading_recovery_rmse(&[vec![1.0]], &[vec![1.0]]).expect("zero"); + assert!(rmse.abs() < f64::EPSILON); + assert!(FitConstructClass::Reflective.admits_esem_fit()); + assert_eq!(FitConstructClass::Network.as_str(), "network"); + assert_eq!( + super::interpret_fit_as_reflective(FitConstructClass::Reflective, false) + .expect("fit unused"), + FitConstructClass::Reflective + ); + assert_eq!( + recover_esem_loadings(&[vec![]], &[vec![]], FitCoordinateKind::AdditiveLogRatio), + Err(PsychometricFitError::InvalidNumericInput) + ); + assert_eq!( + recover_esem_loadings( + &[] as &[Vec], + &[vec![1.0, 2.0]], + FitCoordinateKind::AdditiveLogRatio + ), + Err(PsychometricFitError::InvalidNumericInput) + ); + assert_eq!( + recover_esem_loadings( + &[vec![1.0, 2.0]], + &[] as &[Vec], + FitCoordinateKind::AdditiveLogRatio + ), + Err(PsychometricFitError::InvalidNumericInput) + ); + let three = [vec![0.0, 1.0], vec![1.0, 0.0], vec![0.5, 0.5]]; + assert_eq!( + recover_esem_loadings( + &three, + &[vec![1.0, 2.0]], + FitCoordinateKind::AdditiveLogRatio + ), + Err(PsychometricFitError::InvalidNumericInput) + ); + assert_eq!( + admit_fit_coordinates(FitCoordinateKind::RawProportion), + Err(PsychometricFitError::RawProportionForbidden) + ); + } + + fn cover_rmse_and_indicator_gates() { + assert_eq!( + loading_recovery_rmse(&[], &[]), + Err(PsychometricFitError::InvalidNumericInput) + ); + assert_eq!( + loading_recovery_rmse(&[vec![1.0]], &[vec![1.0, 2.0]]), + Err(PsychometricFitError::InvalidNumericInput) + ); + assert_eq!( + loading_recovery_rmse(&[vec![1.0]], &[]), + Err(PsychometricFitError::InvalidNumericInput) + ); + assert_eq!( + recover_esem_loadings( + &[vec![1.0, 2.0]], + &[vec![1.0]], + FitCoordinateKind::AdditiveLogRatio + ), + Err(PsychometricFitError::InvalidNumericInput) + ); + } +} diff --git a/crates/psychometric_fit/src/lib.rs b/crates/psychometric_fit/src/lib.rs new file mode 100644 index 00000000..2844388b --- /dev/null +++ b/crates/psychometric_fit/src/lib.rs @@ -0,0 +1,30 @@ +#![forbid(unsafe_code)] +#![deny(missing_docs)] +#![allow(clippy::cast_precision_loss)] +//! CPU `f64` ESEM loading recovery and event-time DSEM lag gates. +//! +//! Raw topic proportions are not Euclidean indicators. This crate recovers +//! exploratory cross-loadings from admitted log-ratio or logistic-normal +//! coordinates, refuses reverse event-time lagged paths, and refuses to let a +//! global fit statistic reclassify a formative or network construct as +//! reflective (ADR 0005). It does not replace `psychometric_core` input gates. + +mod error; +mod fit; + +/// Fail-closed psychometric-fit errors. +pub use error::PsychometricFitError; +/// Higher-order construct class. +pub use fit::FitConstructClass; +/// Coordinate system admitted into an ESEM/DSEM fit. +pub use fit::FitCoordinateKind; +/// Admit only log-ratio or logistic-normal coordinates into a fit. +pub use fit::admit_fit_coordinates; +/// Interpret a classified construct as reflective after a fit. +pub use fit::interpret_fit_as_reflective; +/// Root-mean-square error between known-truth and recovered loadings. +pub use fit::loading_recovery_rmse; +/// Recover a DSEM lagged path only when the predictor precedes the outcome. +pub use fit::recover_dsem_lagged_path; +/// Recover an ESEM loading matrix by OLS of each indicator on every factor. +pub use fit::recover_esem_loadings; diff --git a/crates/psychometric_fit/tests/crate_contract.rs b/crates/psychometric_fit/tests/crate_contract.rs new file mode 100644 index 00000000..2af96356 --- /dev/null +++ b/crates/psychometric_fit/tests/crate_contract.rs @@ -0,0 +1,7 @@ +//! Integration contract for the `psychometric_fit` package identity. + +#[test] +fn package_identity_is_stable() { + let observed = std::hint::black_box(env!("CARGO_PKG_NAME")); + assert_eq!(observed, "psychometric_fit"); +} diff --git a/crates/psychometric_fit/tests/esem_dsem_fit_contract.rs b/crates/psychometric_fit/tests/esem_dsem_fit_contract.rs new file mode 100644 index 00000000..8bc4c47c --- /dev/null +++ b/crates/psychometric_fit/tests/esem_dsem_fit_contract.rs @@ -0,0 +1,254 @@ +//! True-parameter ESEM/DSEM fit recovery and fail-closed interpretation gates. + +#![allow(clippy::cast_precision_loss)] + +use psychometric_fit::{ + FitConstructClass, FitCoordinateKind, PsychometricFitError, admit_fit_coordinates, + interpret_fit_as_reflective, loading_recovery_rmse, recover_dsem_lagged_path, + recover_esem_loadings, +}; + +fn centered_scores(count: usize) -> Vec { + let mean = (count as f64 - 1.0) / 2.0; + (0..count).map(|index| index as f64 - mean).collect() +} + +#[test] +fn two_factor_esem_recovers_known_cross_loadings_better_than_zero() { + let factor_one = centered_scores(16); + let factor_two: Vec = factor_one + .iter() + .map(|score| score * score - 21.25) + .collect(); + let indicator_one: Vec = factor_one + .iter() + .zip(&factor_two) + .map(|(one, two)| 0.8 * one + 0.2 * two) + .collect(); + let indicator_two: Vec = factor_one + .iter() + .zip(&factor_two) + .map(|(one, two)| 0.1 * one + 0.7 * two) + .collect(); + + let recovered = recover_esem_loadings( + &[factor_one, factor_two], + &[indicator_one, indicator_two], + FitCoordinateKind::AdditiveLogRatio, + ) + .expect("noiseless ESEM"); + let truth = [vec![0.8, 0.2], vec![0.1, 0.7]]; + let recovered_rmse = loading_recovery_rmse(&truth, &recovered).expect("rmse"); + let zeroed = [vec![0.0, 0.0], vec![0.0, 0.0]]; + let collapsed_rmse = loading_recovery_rmse(&truth, &zeroed).expect("zero"); + assert!( + recovered_rmse < 1e-12, + "noiseless ESEM RMSE {recovered_rmse} exceeded machine-scale bound" + ); + assert!(recovered_rmse < collapsed_rmse); +} + +#[test] +fn single_factor_ilr_and_logistic_normal_recover_the_known_loading() { + let factor = centered_scores(8); + let indicator: Vec = factor.iter().map(|score| 0.6 * score).collect(); + for kind in [ + FitCoordinateKind::IsometricLogRatio, + FitCoordinateKind::LogisticNormal, + ] { + let recovered = recover_esem_loadings( + std::slice::from_ref(&factor), + std::slice::from_ref(&indicator), + kind, + ) + .expect("loading"); + assert_eq!(recovered.len(), 1); + assert_eq!(recovered[0].len(), 1); + assert!((recovered[0][0] - 0.6).abs() < 1e-12); + } +} + +#[test] +fn forward_dsem_lag_recovers_and_non_forward_event_time_fails_closed() { + let lag = centered_scores(10); + let outcome: Vec = lag.iter().map(|score| 0.45 * score).collect(); + let recovered = recover_dsem_lagged_path(10, 20, &lag, &outcome).expect("forward lag"); + assert!((recovered - 0.45).abs() < 1e-12); + assert_eq!( + recover_dsem_lagged_path(20, 10, &lag, &outcome), + Err(PsychometricFitError::ReverseEventTimePath) + ); + assert_eq!( + recover_dsem_lagged_path(10, 10, &lag, &outcome), + Err(PsychometricFitError::ReverseEventTimePath) + ); +} + +#[test] +#[allow(clippy::too_many_lines)] +fn raw_proportions_and_invalid_payloads_fail_closed() { + assert_eq!( + admit_fit_coordinates(FitCoordinateKind::RawProportion), + Err(PsychometricFitError::RawProportionForbidden) + ); + assert_eq!( + recover_esem_loadings( + &[vec![0.2, 0.3]], + &[vec![0.8, 0.7]], + FitCoordinateKind::RawProportion + ), + Err(PsychometricFitError::RawProportionForbidden) + ); + assert_eq!( + recover_esem_loadings( + &[] as &[Vec], + &[vec![1.0, 2.0]], + FitCoordinateKind::AdditiveLogRatio + ), + Err(PsychometricFitError::InvalidNumericInput) + ); + assert_eq!( + recover_esem_loadings( + &[vec![1.0, 2.0]], + &[] as &[Vec], + FitCoordinateKind::AdditiveLogRatio + ), + Err(PsychometricFitError::InvalidNumericInput) + ); + assert_eq!( + recover_esem_loadings( + &[vec![1.0]], + &[vec![1.0]], + FitCoordinateKind::AdditiveLogRatio + ), + Err(PsychometricFitError::InvalidNumericInput) + ); + assert_eq!( + recover_esem_loadings( + &[vec![1.0, 2.0]], + &[vec![1.0]], + FitCoordinateKind::AdditiveLogRatio + ), + Err(PsychometricFitError::InvalidNumericInput) + ); + assert_eq!( + recover_esem_loadings( + &[vec![1.0, f64::NAN]], + &[vec![1.0, 2.0]], + FitCoordinateKind::AdditiveLogRatio + ), + Err(PsychometricFitError::InvalidNumericInput) + ); + assert_eq!( + recover_esem_loadings( + &[vec![1.0, 1.0]], + &[vec![2.0, 3.0]], + FitCoordinateKind::AdditiveLogRatio + ), + Err(PsychometricFitError::SingularDesign) + ); + let three = [vec![0.0, 1.0], vec![1.0, 0.0], vec![0.5, 0.5]]; + assert_eq!( + recover_esem_loadings( + &three, + &[vec![1.0, 2.0]], + FitCoordinateKind::AdditiveLogRatio + ), + Err(PsychometricFitError::InvalidNumericInput) + ); + assert_eq!( + recover_esem_loadings( + &[vec![1.0, 2.0], vec![3.0]], + &[vec![1.0, 2.0]], + FitCoordinateKind::AdditiveLogRatio + ), + Err(PsychometricFitError::InvalidNumericInput) + ); + assert_eq!( + recover_esem_loadings( + &[vec![1.0, 2.0, 3.0], vec![2.0, 4.0, 6.0]], + &[vec![1.0, 2.0, 3.0]], + FitCoordinateKind::AdditiveLogRatio + ), + Err(PsychometricFitError::SingularDesign) + ); + assert_eq!( + recover_dsem_lagged_path(1, 2, &[1.0], &[2.0]), + Err(PsychometricFitError::InvalidNumericInput) + ); + assert_eq!( + loading_recovery_rmse(&[], &[]), + Err(PsychometricFitError::InvalidNumericInput) + ); + assert_eq!( + loading_recovery_rmse(&[vec![1.0]], &[vec![1.0, 2.0]]), + Err(PsychometricFitError::InvalidNumericInput) + ); + assert_eq!( + loading_recovery_rmse(&[vec![], vec![1.0]], &[vec![], vec![1.0]]), + Err(PsychometricFitError::InvalidNumericInput) + ); + assert_eq!( + loading_recovery_rmse(&[vec![1.0]], &[]), + Err(PsychometricFitError::InvalidNumericInput) + ); +} + +#[test] +fn good_global_fit_cannot_reclassify_formative_or_unresolved_constructs() { + assert!(FitConstructClass::Reflective.admits_esem_fit()); + assert!(!FitConstructClass::Formative.admits_esem_fit()); + assert!(!FitConstructClass::Network.admits_esem_fit()); + assert!(!FitConstructClass::Unresolved.admits_esem_fit()); + assert_eq!( + interpret_fit_as_reflective(FitConstructClass::Reflective, true).expect("reflective"), + FitConstructClass::Reflective + ); + assert_eq!( + interpret_fit_as_reflective(FitConstructClass::Formative, true), + Err(PsychometricFitError::FormativeReinterpretationForbidden) + ); + assert_eq!( + interpret_fit_as_reflective(FitConstructClass::Network, false), + Err(PsychometricFitError::FormativeReinterpretationForbidden) + ); + assert_eq!( + interpret_fit_as_reflective(FitConstructClass::Unresolved, true), + Err(PsychometricFitError::UnresolvedConstruct) + ); + assert_eq!(FitCoordinateKind::AdditiveLogRatio.as_str(), "alr"); + assert_eq!(FitCoordinateKind::IsometricLogRatio.as_str(), "ilr"); + assert_eq!( + FitCoordinateKind::LogisticNormal.as_str(), + "logistic_normal" + ); + assert_eq!(FitCoordinateKind::RawProportion.as_str(), "raw_proportion"); + assert_eq!(FitConstructClass::Reflective.as_str(), "reflective"); + assert_eq!(FitConstructClass::Formative.as_str(), "formative"); + assert_eq!(FitConstructClass::Network.as_str(), "network"); + assert_eq!(FitConstructClass::Unresolved.as_str(), "unresolved"); + assert_eq!( + PsychometricFitError::RawProportionForbidden.to_string(), + "raw topic proportions are forbidden psychometric fit inputs" + ); + assert_eq!( + PsychometricFitError::InvalidNumericInput.to_string(), + "invalid psychometric fit numeric input" + ); + assert_eq!( + PsychometricFitError::SingularDesign.to_string(), + "singular psychometric fit design matrix" + ); + assert_eq!( + PsychometricFitError::ReverseEventTimePath.to_string(), + "DSEM lagged paths cannot move backward in event time" + ); + assert_eq!( + PsychometricFitError::FormativeReinterpretationForbidden.to_string(), + "formative or network constructs cannot be reinterpreted as reflective" + ); + assert_eq!( + PsychometricFitError::UnresolvedConstruct.to_string(), + "construct class is unresolved" + ); +} diff --git a/docs/TRACEABILITY.md b/docs/TRACEABILITY.md index bdf18235..159f189b 100644 --- a/docs/TRACEABILITY.md +++ b/docs/TRACEABILITY.md @@ -28,8 +28,7 @@ The full APA 7th standards/literature register remains `docs/research/standards- | report template/section/copied/style/modality method effects | ADR 0004/0012; PRD/TRD | simulation truth factors implemented; estimator-side method model remains future | partial | | candidate K statistical/Pareto gates + blinded LLM review | ADR 0012; research | future `model_selection` | accepted-target | | compositional topic correlation / stable clustering | ADR 0005/0012; research | future `network_analysis` | accepted-target | -| longitudinal within/between decomposition | ADR 0005 | `longitudinal_core` decomposition, component RMSE, and known-truth recovery on the active PR | active-PR | -| posterior ESEM / longitudinal invariance / DSEM | ADR 0005 | future `psychometric_core`; invariance and ESEM/DSEM fit remain accepted-target | accepted-target | +| posterior ESEM / longitudinal invariance / DSEM | ADR 0005 | `psychometric_fit` ESEM loading and DSEM lag gates on the active PR; `psychometric_core` input gates remain #49; invariance/multilevel remain accepted-target | active-PR | | CPU bounded multithreading + GPU/VRAM streaming/parity | ADR 0001/0006 | future `compute_backend` | accepted-target | | TDT detection/tracking vs CHRONOS schema/prediction/temporal consistency | ADR 0016; PRD/research | `prediction_contradiction` bounded Allen promotion gate on the active PR (`refuse_promotion` requires coverage; `refuse_contradiction_or_adjacency` is not promotion authority; remaining TDT/CHRONOS tasks stay accepted-target) | active-PR | | evidence-bounded LLM interpretation | ADR 0010/0012; PRD | `tepp_api` router plus future `interpretation_gateway` | partial | diff --git a/docs/adr/0005-posterior-esem-dsem.md b/docs/adr/0005-posterior-esem-dsem.md index 73067fde..7551baae 100644 --- a/docs/adr/0005-posterior-esem-dsem.md +++ b/docs/adr/0005-posterior-esem-dsem.md @@ -1,6 +1,7 @@ # ADR 0005 — Posterior-aware ESEM/DSEM and structural interpretation **Decision status:** Accepted +**Implementation maturity:** active-PR — CPU `f64` ESEM loading recovery and event-time DSEM lag gates in `psychometric_fit` on the active PR; `psychometric_core` input gates remain #49; longitudinal invariance and multilevel estimators remain accepted-target **Implementation maturity:** active-PR — `longitudinal_core` separates unit means from within residuals and refuses between-as-within change; remaining ESEM/DSEM fit remains accepted-target **Date:** 2026-08-05 **Supersedes:** None. ADR 0012 governs upstream topic measurement/network coordinates; this ADR governs higher-order psychometric structure and longitudinal interpretation. diff --git a/docs/adr/README.md b/docs/adr/README.md index 564ddfb3..0ae7932b 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -14,6 +14,7 @@ Read [`ADR_POLICY.md`](ADR_POLICY.md) first. **Decision status and implementatio | [0003](0003-relational-event-multiple-membership.md) | Relational event ontology and time-varying cross-classified multiple membership | Accepted | active-PR | Subevent parent-window containment in `subevent_containment` on the active PR; multilevel estimators remain accepted-target. | | [0003](0003-relational-event-multiple-membership.md) | Relational event ontology and time-varying cross-classified multiple membership | Accepted | partial | Membership network/roles and Kish ESS are implemented-main; this increment adds nested ICC with fail-closed cross-classified/multiple-membership refusal. Full multilevel/MMMC estimators, graph ontology, and remaining persistence stay accepted-target. ADR 0016 owns event-intelligence tasks. | | [0004](0004-shared-multilingual-latent-space.md) | One shared multilingual latent space with explicit invariance status | Accepted | accepted-target | ADR 0012 owns the full topic-estimator/backend/global-topic contract. | +| [0005](0005-posterior-esem-dsem.md) | Posterior-aware ESEM/DSEM and valid compositional coordinates | Accepted | active-PR | CPU `f64` ESEM/DSEM fit in `psychometric_fit` on the active PR; `psychometric_core` input gates remain #49; invariance/multilevel remain accepted-target. | | [0005](0005-posterior-esem-dsem.md) | Posterior-aware ESEM/DSEM and valid compositional coordinates | Accepted | active-PR | Within/between decomposition in `longitudinal_core` on the active PR; remaining ESEM/DSEM fit remains accepted-target. ADR 0012 owns the upstream topic/network contract. | | [0006](0006-vram-gpu-nvidia-orchestration.md) | VRAM-adaptive GPU compute and model-credential boundary | Accepted | accepted-target | LLM orchestration policy superseded by ADR 0010; autonomous development authority governed by ADR 0015. | | [0007](0007-rust-workspace-quality-gates.md) | Explicit Rust workspace, pinned toolchains, and exact quality gates | Accepted | implemented-main | ADR 0014 governs scientific/product claim promotion beyond repository-quality tooling. | diff --git a/docs/research/esem-dsem-fit.md b/docs/research/esem-dsem-fit.md new file mode 100644 index 00000000..45cc025c --- /dev/null +++ b/docs/research/esem-dsem-fit.md @@ -0,0 +1,42 @@ +# ESEM loading recovery and DSEM event-time lags (doctoring) + +## Scope + +`psychometric_fit` recovers an exploratory loading matrix by ordinary least +squares of each indicator on at most two factor-score series. Recovery is the +computed RMSE against known loadings. A DSEM lagged path is admitted only when +the predictor occasion is strictly earlier in event time than the outcome. + +This slice does not implement rotation, posterior pooling, invariance testing, +or the `psychometric_core` input-gate crate owned by PR #49. It does not +allocate migration `0007` or `0008`. + +## Authority + +### Normative TEPP contract + +- `docs/adr/0005-posterior-esem-dsem.md` — logistic-normal or valid log-ratio + coordinates; construct class before interpretation; event-time order for + lagged paths; a global fit statistic cannot reclassify formative or network + constructs as reflective. +- `docs/adr/0001-rust-first-modular-msa.md` — production psychometric + arithmetic is a CPU `f64` reference path. + +### Supporting literature + +Asparouhov and Muthén (2009) introduce exploratory structural equation +modeling so indicators may have cross-loadings rather than a strict +confirmatory zero pattern. This crate recovers those cross-loadings by OLS; it +does not implement their full ESEM estimator or rotation. + +Asparouhov, Hamaker, and Muthén (2018) specify dynamic structural equation +models on a time-ordered series. The crate enforces the event-time order of a +lagged path and does not implement their Bayesian DSEM sampler. + +Asparouhov, T., & Muthén, B. (2009). Exploratory structural equation modeling. +*Structural Equation Modeling: A Multidisciplinary Journal, 16*(3), 397–438. +https://doi.org/10.1080/10705510903008204 + +Asparouhov, T., Hamaker, E. L., & Muthén, B. (2018). Dynamic structural +equation models. *Structural Equation Modeling: A Multidisciplinary Journal, +25*(3), 359–388. https://doi.org/10.1080/10705511.2017.1406803 diff --git a/docs/research/standards-and-literature.md b/docs/research/standards-and-literature.md index 6b48142c..5e5d727b 100644 --- a/docs/research/standards-and-literature.md +++ b/docs/research/standards-and-literature.md @@ -16,7 +16,7 @@ Fox, J.-P., & Glas, C. A. W. (2001). Bayesian estimation of a multilevel IRT mod Marsh, H. W., Morin, A. J. S., Parker, P. D., & Kaur, G. (2014). Exploratory structural equation modeling: An integration of the best features of exploratory and confirmatory factor analysis. *Annual Review of Clinical Psychology, 10*, 85–110. https://doi.org/10.1146/annurev-clinpsy-032813-153700 -TEPP applies these sources to construct definition, score interpretation, reliability, validity evidence, uncertainty, consequences, longitudinal invariance, ESEM cross-loadings, and DSEM (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, 2014; Asparouhov & Muthén, 2009; Asparouhov et al., 2018; Hamaker et al., 2015; Marsh et al., 2014). Topic outputs are treated as fallible indicators or components only after their construct role is evaluated. `longitudinal_core` separates stable between-unit means from within-unit residuals and refuses to score a between component as within-unit change. +TEPP applies these sources to construct definition, score interpretation, reliability, validity evidence, uncertainty, consequences, longitudinal invariance, ESEM cross-loadings, and DSEM. Topic outputs are treated as fallible indicators or components only after their construct role is evaluated. `psychometric_fit` recovers those cross-loadings and event-time lagged paths on a CPU `f64` OLS path; see `docs/research/esem-dsem-fit.md`. ## Structural, correlated, dynamic, relational, and multilingual topic models diff --git a/docs/validation/temporal-event-foundation.md b/docs/validation/temporal-event-foundation.md index e4a08169..9c5a9490 100644 --- a/docs/validation/temporal-event-foundation.md +++ b/docs/validation/temporal-event-foundation.md @@ -27,6 +27,7 @@ This report tracks exact-head scientific and engineering evidence required befor | Mention-confidence Brier score | `event_core` | active-PR | calibration vs binary truth | perfect 0 / half 0.25 RMSE | ADR 0003; `docs/research/mention-confidence-brier.md` | | Checkpoint is not the estimator | `checkpoint_authority` | accepted-target | active PR | refuse checkpoint-as-estimator + unvalidated artifact + recovery vs estimator collapse | ADR 0001/0014 | | Versioned API/export contracts | `tepp_api` | implemented-main | naruon HTTP interchange | unknown-field/version/limit + naruon HTTPS interchange tests | Task 12 / PR #21; live HTTP service remaining | +| ESEM/DSEM CPU fit | `psychometric_fit` | active-PR | this PR | loading RMSE vs zero-collapse; reverse-lag refusal | ADR 0005; does not recreate `psychometric_core` | | Subevent parent containment | `subevent_containment` | active-PR | this PR | containment-flag recovery vs accept-all | ADR 0003 | | Predicted-vs-observed contradiction | `prediction_contradiction` | active-PR | this PR | `refuse_promotion` requires observed coverage; `refuse_contradiction_or_adjacency` is not promotion authority; cutoff eligibility; label agreement is not RMSE recovery | ADR 0016 | | Provider-disclosure receipts | `provider_receipt` | active-PR | this PR | recovered field-code rate vs collapsed set | ADR 0009 | diff --git a/scripts/check_workspace_contract.py b/scripts/check_workspace_contract.py index c3936d4c..5b7a1bc4 100644 --- a/scripts/check_workspace_contract.py +++ b/scripts/check_workspace_contract.py @@ -23,6 +23,7 @@ "tepp_simulation", "validation_core", "tepp_api", + "psychometric_fit", "subevent_containment", "prediction_contradiction", "provider_receipt",