Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ boundaries above remain the target modular MSA architecture.
| `temporal_core` | typed clocks, intervals, and temporal reasoning |
| `event_core` | event instances, mentions, roles, and provenance |
| `relation_graph` | typed relations and forward-transition validation |
| `membership_core` | time-varying cross-classified multiple membership |
| `membership_core` | time-varying cross-classified multiple membership, Kish ESS, nested ICC with non-nested refusal |
| `persistence_postgres` | PostgreSQL repositories and migrations |
| `corpus_split` | cutoff-safe, relation-aware partitioning |
| `tepp_simulation` | known-truth temporal/event data generation |
Expand Down
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ All notable changes to TEPP are documented here. The format follows Keep a Chang

### Added

- `membership_core` nested ICC: CPU `f64` unbalanced ANOVA recovers a known cluster ICC and refuses to treat cross-classified or multiple-membership designs as a single hierarchy (ADR 0003).
- `persistence_postgres` typed `text_segment` SQL: insert/lookup of exact UTF-8 half-open byte spans on the existing `0006` table, cutoff-eligible document reads (`available_time <= knowledge_cutoff`), and live recovery of a known `hello` span. No new migration number (`#45` still owns `0007`).
- Hourly contextual-orchestrator discovery records all provider models but routes OpenCode only through general-chat candidates, excluding embedding, image, reranker, transcription, moderation, safety, and other endpoint-only identifiers before price selection.
- Live `docs/product-technical-gap-baseline.md` mapping operator-visible gaps to
Expand Down
35 changes: 35 additions & 0 deletions crates/membership_core/src/error.rs
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,16 @@ pub enum MembershipError {
UnknownMembershipRole,
/// A duplicate assignment key was rejected by the membership network.
DuplicateMembershipAssignment,
/// Nested ICC is undefined for cross-classified or multiple-membership designs.
NestedIccInapplicable,
/// Clusters or within-group residual degrees of freedom are insufficient.
InsufficientClusterStructure,
/// An outcome value was non-finite.
InvalidOutcome,
/// The same member contributed more than one nested ICC outcome.
DuplicateOutcomeMember,
/// An outcome member has no active nested membership at the requested time.
UnknownOutcomeMember,
}

impl fmt::Display for MembershipError {
Expand All @@ -30,6 +40,11 @@ impl fmt::Display for MembershipError {
Self::UnsupportedWireVersion => "unsupported membership wire version",
Self::UnknownMembershipRole => "unknown membership role",
Self::DuplicateMembershipAssignment => "duplicate membership assignment",
Self::NestedIccInapplicable => "nested ICC is inapplicable to this membership design",
Self::InsufficientClusterStructure => "insufficient cluster structure for nested ICC",
Self::InvalidOutcome => "invalid nested ICC outcome",
Self::DuplicateOutcomeMember => "duplicate nested ICC outcome member",
Self::UnknownOutcomeMember => "unknown nested ICC outcome member",
};
formatter.write_str(message)
}
Expand Down Expand Up @@ -68,6 +83,26 @@ mod tests {
MembershipError::DuplicateMembershipAssignment,
"duplicate membership assignment",
),
(
MembershipError::NestedIccInapplicable,
"nested ICC is inapplicable to this membership design",
),
(
MembershipError::InsufficientClusterStructure,
"insufficient cluster structure for nested ICC",
),
(
MembershipError::InvalidOutcome,
"invalid nested ICC outcome",
),
(
MembershipError::DuplicateOutcomeMember,
"duplicate nested ICC outcome member",
),
(
MembershipError::UnknownOutcomeMember,
"unknown nested ICC outcome member",
),
] {
assert_eq!(error.to_string(), message);
}
Expand Down
246 changes: 246 additions & 0 deletions crates/membership_core/src/icc.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,246 @@
//! Nested ICC with fail-closed cross-classified and multiple-membership gates.

use crate::{MemberId, MembershipError, MembershipNetwork, MembershipRole};
use std::collections::{BTreeMap, BTreeSet};
use temporal_core::EventTime;

/// Membership design implied by active assignments at one event time.
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
#[non_exhaustive]
pub enum MembershipDesign {
/// Each active member belongs to exactly one group in one role.
Nested,
/// At least one member is active in two or more roles.
CrossClassified,
/// At least one member is active in two or more groups of the same role.
MultipleMembership,
}

impl MembershipDesign {
/// Return whether a one-way nested ICC is identified for this design.
#[must_use]
pub const fn allows_nested_icc(self) -> bool {
match self {
Self::Nested => true,
Self::CrossClassified | Self::MultipleMembership => false,
}
}
}

/// One finite outcome attached to an opaque member for nested ICC recovery.
#[derive(Clone, Copy, Debug, PartialEq)]
pub struct NestedOutcome {
member_id: MemberId,
value: f64,
}

impl NestedOutcome {
/// Construct a finite nested ICC outcome.
///
/// # Errors
///
/// Returns [`MembershipError::InvalidOutcome`] when `value` is not finite.
pub fn new(member_id: MemberId, value: f64) -> Result<Self, MembershipError> {
if value.is_finite() {
Ok(Self { member_id, value })
} else {
Err(MembershipError::InvalidOutcome)
}
}

/// Return the outcome member.
#[must_use]
pub const fn member_id(self) -> MemberId {
self.member_id
}

/// Return the finite outcome value.
#[must_use]
pub const fn value(self) -> f64 {
self.value
}
}

/// Classify active memberships at `instant` without collapsing structure.
///
/// Multiple membership is reported before cross-classification so a member
/// who occupies two groups in one role is not misread as a nested hierarchy.
///
/// # Errors
///
/// Returns [`MembershipError::InsufficientClusterStructure`] when no assignment
/// is active at `instant`.
pub fn classify_membership_design(
network: &MembershipNetwork,
instant: EventTime,
) -> Result<MembershipDesign, MembershipError> {
let mut members = BTreeSet::new();
for assignment in network.assignments() {
members.insert(assignment.member_id());
}
classify_members(network, instant, members.iter().copied())
}

/// Recover the one-way random-intercept ICC for a nested membership design.
///
/// The CPU `f64` estimator is the unbalanced ANOVA (Snijders & Bosker)
/// estimator
/// `σ²_u / (σ²_u + σ²_e)` with
/// `σ²_e = MSW` and `σ²_u = max(0, (MSB − MSW) / n₀)`.
/// Cross-classified and multiple-membership designs fail closed: a nested
/// ICC is not a substitute for an MMMC model.
///
/// # Errors
///
/// Returns a membership error when outcomes are empty, duplicated, unknown,
/// inactive, non-nested, or when cluster residual structure is insufficient.
pub fn nested_intraclass_correlation(
network: &MembershipNetwork,
instant: EventTime,
outcomes: &[NestedOutcome],
) -> Result<f64, MembershipError> {
if outcomes.is_empty() {
return Err(MembershipError::InsufficientClusterStructure);
}
let mut seen = BTreeSet::new();
let mut groups: BTreeMap<crate::GroupId, Vec<f64>> = BTreeMap::new();
let mut outcome_members = BTreeSet::new();
for outcome in outcomes {
if !seen.insert(outcome.member_id()) {
return Err(MembershipError::DuplicateOutcomeMember);
}
let active = network.active_memberships_for(outcome.member_id(), instant);
if active.is_empty() {
return Err(MembershipError::UnknownOutcomeMember);
}
outcome_members.insert(outcome.member_id());
let group = active[0].group_id();
groups.entry(group).or_default().push(outcome.value());
}
match classify_members(network, instant, outcome_members.iter().copied())? {
MembershipDesign::Nested => {}
MembershipDesign::CrossClassified | MembershipDesign::MultipleMembership => {
return Err(MembershipError::NestedIccInapplicable);
}
}
anova_nested_icc(&groups)
Comment on lines +120 to +126

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 Info: Design classification differs in scope between public and ICC paths

classify_membership_design at icc.rs classifies over ALL members in the network, whereas the internal gate in nested_intraclass_correlation (icc.rs) classifies only over the members that supplied outcomes. This is intentional (the ICC only concerns members with outcomes), but it means a network the public classifier reports as CrossClassified can still yield a valid nested ICC if the supplied outcome members happen to each be singly nested. Reviewers relying on the public classifier as a pre-check should be aware the two answers can legitimately diverge.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

}
Comment on lines +108 to +127

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 Info: Nested ICC gate builds cluster groups before design verification

In nested_intraclass_correlation (icc.rs), the per-group value map is populated using active[0].group_id() for each outcome member before classify_members confirms the design is nested. This is safe: if the design is cross-classified or multiple-membership, the function returns NestedIccInapplicable and the partially-built groups map is discarded. And when the design is Nested, each member has exactly one active membership (single role, single group), so active[0] is unambiguous. No ordering dependence on active_memberships_for's return order affects the result in the nested case.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.


fn classify_members<I>(
network: &MembershipNetwork,
instant: EventTime,
members: I,
) -> Result<MembershipDesign, MembershipError>
where
I: IntoIterator<Item = MemberId>,
{
let mut saw_active = false;
let mut saw_cross = false;
for member_id in members {
let active = network.active_memberships_for(member_id, instant);
if active.is_empty() {
continue;
}
saw_active = true;
let mut groups_by_role: BTreeMap<MembershipRole, BTreeSet<crate::GroupId>> =
BTreeMap::new();
for assignment in active {
groups_by_role
.entry(assignment.role())
.or_default()
.insert(assignment.group_id());
}
for groups in groups_by_role.values() {
if groups.len() >= 2 {
return Ok(MembershipDesign::MultipleMembership);
}
}
if groups_by_role.len() >= 2 {
saw_cross = true;
}
}
if !saw_active {
return Err(MembershipError::InsufficientClusterStructure);
}
if saw_cross {
Ok(MembershipDesign::CrossClassified)
} else {
Ok(MembershipDesign::Nested)
}
Comment on lines +145 to +169

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 Info: Nested members in distinct role types treated as nested

classify_members at icc.rs only marks a design cross-classified when a single member holds two or more roles. If outcome members each hold exactly one membership but in different role types (e.g. one Author, one Department), the design is reported Nested and a nested ICC is computed by grouping on group_id. This is a degenerate population-level cross-classification that is not detected because cross-classification is defined per-member. Given the estimator groups purely by group id and each member is singly nested, this is defensible, but it is a subtle definitional choice worth confirming against ADR 0003 intent.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

}

fn anova_nested_icc(groups: &BTreeMap<crate::GroupId, Vec<f64>>) -> Result<f64, MembershipError> {
let cluster_count = groups.len();
if cluster_count < 2 {
return Err(MembershipError::InsufficientClusterStructure);
}
let mut sample_size = 0_usize;
let mut total = 0.0;
for values in groups.values() {
sample_size += values.len();
for &value in values {
total += value;
}
}
if sample_size <= cluster_count {
return Err(MembershipError::InsufficientClusterStructure);
}
let n = sample_size as f64;
let j = cluster_count as f64;
let grand_mean = total / n;
let mut sum_of_squares_between = 0.0;
let mut sum_of_squares_within = 0.0;
let mut sum_cluster_size_squared = 0.0;
for values in groups.values() {
let cluster_size = values.len() as f64;
sum_cluster_size_squared += cluster_size * cluster_size;
let mut cluster_total = 0.0;
for &value in values {
cluster_total += value;
}
let cluster_mean = cluster_total / cluster_size;
let between = cluster_mean - grand_mean;
sum_of_squares_between += cluster_size * between * between;
for &value in values {
let within = value - cluster_mean;
sum_of_squares_within += within * within;
}
}
if sum_of_squares_between + sum_of_squares_within == 0.0 {
return Err(MembershipError::InsufficientClusterStructure);
}
let mean_square_between = sum_of_squares_between / (j - 1.0);
let mean_square_within = sum_of_squares_within / (n - j);
let harmonic_cluster_size = (n - sum_cluster_size_squared / n) / (j - 1.0);
let cluster_variance =
((mean_square_between - mean_square_within) / harmonic_cluster_size).max(0.0);
Ok(cluster_variance / (cluster_variance + mean_square_within))
Comment on lines +209 to +217

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 Info: Unbalanced ANOVA n₀ factor matches Snijders–Bosker and recovers balanced ICC exactly

harmonic_cluster_size = (n - sum_cluster_size_squared / n) / (j - 1.0) (icc.rs) is the Snijders–Bosker unbalanced cluster-size factor ñ = (M − Σn_j²/M)/(N−1). For the balanced 4×2 test it reduces to the per-cluster size (2), giving σ²_u=2/3, σ²_e=2, ICC=1/4, matching the contract test. All divisors are guarded: j-1 ≥ 1 (cluster_count ≥ 2), n-j > 0 (sample_size > cluster_count), and the denominator cluster_variance + mean_square_within can only be zero when both SSB and SSW are zero, which is caught by the earlier sum_of_squares_between + sum_of_squares_within == 0.0 guard. No division-by-zero or out-of-range ICC is reachable.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

}

#[cfg(test)]
mod tests {
use super::{MembershipDesign, NestedOutcome, anova_nested_icc};
use crate::{GroupId, MemberId, MembershipError};
use std::collections::BTreeMap;

#[test]
fn design_gate_and_outcome_accessors_cover_local_branches() {
assert!(MembershipDesign::Nested.allows_nested_icc());
assert!(!MembershipDesign::CrossClassified.allows_nested_icc());
assert!(!MembershipDesign::MultipleMembership.allows_nested_icc());
let member = MemberId::new();
let outcome = NestedOutcome::new(member, 1.5).expect("finite");
assert_eq!(outcome.member_id(), member);
assert!((outcome.value() - 1.5).abs() < 1e-12);
assert_eq!(
NestedOutcome::new(member, f64::NEG_INFINITY),
Err(MembershipError::InvalidOutcome)
);
let mut one = BTreeMap::new();
one.insert(GroupId::new(), vec![1.0, 2.0]);
assert_eq!(
anova_nested_icc(&one),
Err(MembershipError::InsufficientClusterStructure)
);
}
}
9 changes: 9 additions & 0 deletions crates/membership_core/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
mod assignment;
mod error;
mod ess;
mod icc;
mod identifier;
mod network;
mod role;
Expand All @@ -39,3 +40,11 @@ pub use ess::design_effect;
pub use ess::group_normalized_kish_ess;
/// Kish effective sample size for membership weights.
pub use ess::kish_effective_sample_size;
/// Membership design implied by active assignments.
pub use icc::MembershipDesign;
/// Finite outcome used by the nested ICC estimator.
pub use icc::NestedOutcome;
/// Classify nested versus cross-classified versus multiple-membership designs.
pub use icc::classify_membership_design;
/// CPU `f64` nested ICC that refuses non-nested membership designs.
pub use icc::nested_intraclass_correlation;
Loading
Loading