diff --git a/CHANGELOG.md b/CHANGELOG.md index c45ef29d..3e462e00 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -190,6 +190,7 @@ All notable changes to TEPP are documented here. The format follows Keep a Chang - `prompt_source` identity gate: instruction and prompt boilerplate is not unique latent content and is not erased by a stopword list; `identity_recovery_rate` reports exact kind matches, with a contract test comparing correct recovery with an all-unique collapse on a mixed known-truth fixture (ADR 0004/0012). - `location_membership` identity gate: geographic and market assignments are time-varying memberships, not permanent entity identity and not language channels; recovered location kinds match known truth at a higher computed rate than collapsing every assignment to entity identity (ADR 0003). - `membership_target` identity gate: language, episode, template, department, and opportunity-pool memberships cannot collapse into the entity/project pair stored by migration `0006`; comparison-contract tests record recovered target kinds against an entity-collapse baseline (ADR 0003). +- `event_core` TDT link-detection contracts: undirected mention-pair hypotheses, fail-closed self-links, refusal to treat a detected link as an instance or state transition, and computed precision/recall plus RMSE against known-truth pairs. - `event_core` first-story detection gate: first-story versus follow-up labels stay distinct from promoted instances, false-alarm and miss rates are computed from known truth, and calibrated detection scores recover the binary first-story target with lower RMSE than an always-first detector. - `tepp_api` naruon live loopback HTTP/1.1 listener: `serve_one` installs a read/write deadline, requires a loopback `Host`, refuses `Transfer-Encoding` and NIM/proxy credential headers, parses `knowledge_cutoff` as RFC 3339 and refuses a future cutoff, keys analysis-run idempotency by tenant plus key, and proves both analysis-run and export POSTs over a real `TcpStream`. Not a production TLS/`$PORT` service (ADR 0011). - `tepp_api` adaptive orchestration router (ADR 0010): versioned `direct`/`verify`/`committee`/`conductor`/`abstain` selection from CPU `f64` risk, ambiguity, evidence, and token-budget inputs; recorded stages, recursion, decomposition, access lists, and role-specific reasoning effort; fail-closed document-controlled policy/access/credentials; LLM plans remain proposals under deterministic statistical authority; comparable-budget ablation requires a direct baseline; credential-free contextual-orchestrator binding. Live NIM HTTP remains accepted-target. diff --git a/DOCUMENTATION.md b/DOCUMENTATION.md index 6a9cbca8..5021cf3b 100644 --- a/DOCUMENTATION.md +++ b/DOCUMENTATION.md @@ -42,6 +42,7 @@ TEPP's approved PRD v0.4 and implementation plan are the primary product baselin | Strong-invariance latent-mean doctoring | [`docs/research/strong-invariance-latent-means.md`](docs/research/strong-invariance-latent-means.md) | | Mention-confidence Brier doctoring | [`docs/research/mention-confidence-brier.md`](docs/research/mention-confidence-brier.md) | | Event-intelligence status-gate doctoring | [`docs/research/event-intelligence-status-gates.md`](docs/research/event-intelligence-status-gates.md) | +| TDT link-detection precision/recall doctoring | [`docs/research/event-link-detection-calibration.md`](docs/research/event-link-detection-calibration.md) | | First-story detection FAR/miss doctoring | [`docs/research/first-story-detection-calibration.md`](docs/research/first-story-detection-calibration.md) | | VRAM budget / GPU fallback doctoring | [`docs/research/vram-budget-types.md`](docs/research/vram-budget-types.md) | | Causal-identification gate doctoring | [`docs/research/causal-identification-gate.md`](docs/research/causal-identification-gate.md) | diff --git a/crates/event_core/src/error.rs b/crates/event_core/src/error.rs index 1ab853f8..7ada37de 100644 --- a/crates/event_core/src/error.rs +++ b/crates/event_core/src/error.rs @@ -24,6 +24,12 @@ pub enum EventError { DetectionIsNotTransition, /// A CHRONOS prediction was treated as an observed or promoted fact. PredictionIsNotFact, + /// A TDT link detection was treated as an event instance. + EventLinkIsNotEventInstance, + /// A TDT link detection was treated as a state transition. + EventLinkIsNotStateTransition, + /// An unknown event-link label was supplied. + UnknownEventLinkLabel, /// A first-story detection was treated as an event instance. FirstStoryIsNotEventInstance, /// An unknown first-story label name was supplied. @@ -64,6 +70,9 @@ impl fmt::Display for EventError { Self::UnknownEventRole => "unknown event role", Self::DetectionIsNotTransition => "detection is not a state transition", Self::PredictionIsNotFact => "prediction is not an observed fact", + Self::EventLinkIsNotEventInstance => "event link is not an event instance", + Self::EventLinkIsNotStateTransition => "event link is not a state transition", + Self::UnknownEventLinkLabel => "unknown event link label", Self::FirstStoryIsNotEventInstance => "first-story detection is not an event instance", Self::UnknownFirstStoryLabel => "unknown first-story label", Self::EventTrackIsNotEventInstance => "event track is not an event instance", @@ -127,6 +136,16 @@ mod tests { "prediction is not an observed fact", ), ( + EventError::EventLinkIsNotEventInstance, + "event link is not an event instance", + ), + ( + EventError::EventLinkIsNotStateTransition, + "event link is not a state transition", + ), + ( + EventError::UnknownEventLinkLabel, + "unknown event link label", EventError::FirstStoryIsNotEventInstance, "first-story detection is not an event instance", ), diff --git a/crates/event_core/src/lib.rs b/crates/event_core/src/lib.rs index 4b64b908..4c8eb716 100644 --- a/crates/event_core/src/lib.rs +++ b/crates/event_core/src/lib.rs @@ -20,6 +20,7 @@ mod first_story; mod identifier; mod instance; mod intelligence; +mod link; mod mention; mod prediction; mod registry; @@ -66,6 +67,20 @@ pub use intelligence::admit_state_transition; pub use intelligence::classify_tdt_story; /// Score first-story detections against a known stream. pub use intelligence::first_story_detection_rates; +/// TDT same-event versus distinct-event link label. +pub use link::EventLinkLabel; +/// Undirected TDT link hypothesis between two mentions. +pub use link::EventLinkPair; +/// Threshold a link probability into a detection label. +pub use link::decide_event_link; +/// Precision of recovered TDT links against known-truth pairs. +pub use link::event_link_precision; +/// Recall of recovered TDT links against known-truth pairs. +pub use link::event_link_recall; +/// Explicit refusal to treat a TDT link as an event instance. +pub use link::refuse_event_link_as_instance; +/// Explicit refusal to treat a TDT link as a state transition. +pub use link::refuse_event_link_as_transition; /// Fallible textual event mention. pub use mention::EventMention; /// One CHRONOS occurrence forecast that remains hypothetical. diff --git a/crates/event_core/src/link.rs b/crates/event_core/src/link.rs new file mode 100644 index 00000000..8d507c69 --- /dev/null +++ b/crates/event_core/src/link.rs @@ -0,0 +1,223 @@ +//! TDT link-detection scores stay distinct from instances and transitions. + +use crate::{EventConfidence, EventError, EventInstanceId, EventMentionId}; +use std::collections::BTreeSet; + +/// TDT same-event versus distinct-event link label. +/// +/// A link decision is detection evidence. It is never a promoted event instance +/// and cannot create a forward state transition by itself. +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub enum EventLinkLabel { + /// The mention pair is scored as the same event or story. + Linked, + /// The mention pair is scored as distinct events. + Unlinked, +} + +impl EventLinkLabel { + /// Return the stable wire label name. + #[must_use] + pub const fn wire_name(self) -> &'static str { + match self { + Self::Linked => "linked", + Self::Unlinked => "unlinked", + } + } + + /// Parse a stable wire link label. + /// + /// # Errors + /// + /// Returns [`EventError::UnknownEventLinkLabel`] for unrecognized names. + pub fn from_wire_name(name: &str) -> Result { + match name { + "linked" => Ok(Self::Linked), + "unlinked" => Ok(Self::Unlinked), + _ => Err(EventError::UnknownEventLinkLabel), + } + } + + /// Return whether this label is a positive link detection. + #[must_use] + pub const fn is_linked(self) -> bool { + matches!(self, Self::Linked) + } + + /// Return the binary probability target used for RMSE. + /// + /// Linked truth is `1.0`; unlinked truth is `0.0`. + #[must_use] + pub const fn as_probability_target(self) -> f64 { + match self { + Self::Linked => 1.0, + Self::Unlinked => 0.0, + } + } +} + +/// An undirected TDT link hypothesis between two mentions. +#[derive(Clone, Copy, Debug, Eq, Hash, Ord, PartialEq, PartialOrd)] +pub struct EventLinkPair { + left: EventMentionId, + right: EventMentionId, +} + +impl EventLinkPair { + /// Construct a normalized undirected mention pair. + /// + /// # Errors + /// + /// Returns [`EventError::InvalidWirePayload`] when both mentions are the + /// same identity. A mention cannot link to itself. + pub fn new(left: EventMentionId, right: EventMentionId) -> Result { + if left == right { + return Err(EventError::InvalidWirePayload); + } + if left <= right { + Ok(Self { left, right }) + } else { + Ok(Self { + left: right, + right: left, + }) + } + } + + /// Return the lexicographically smaller mention identifier. + #[must_use] + pub const fn left(self) -> EventMentionId { + self.left + } + + /// Return the lexicographically larger mention identifier. + #[must_use] + pub const fn right(self) -> EventMentionId { + self.right + } +} + +/// Threshold a link probability into a detection label. +/// +/// The threshold is inclusive: `probability >= threshold` is linked. +#[must_use] +pub fn decide_event_link( + probability: EventConfidence, + threshold: EventConfidence, +) -> EventLinkLabel { + if probability.value() >= threshold.value() { + EventLinkLabel::Linked + } else { + EventLinkLabel::Unlinked + } +} + +/// Explicit refusal to treat a TDT link as an event instance. +/// +/// # Errors +/// +/// Always returns [`EventError::EventLinkIsNotEventInstance`]. +pub fn refuse_event_link_as_instance(_link: EventLinkPair) -> Result { + Err(EventError::EventLinkIsNotEventInstance) +} + +/// Explicit refusal to treat a TDT link as a state transition. +/// +/// # Errors +/// +/// Always returns [`EventError::EventLinkIsNotStateTransition`]. +pub fn refuse_event_link_as_transition(_link: EventLinkPair) -> Result<(), EventError> { + Err(EventError::EventLinkIsNotStateTransition) +} + +/// Precision of recovered TDT links against the known-truth pair set. +/// +/// # Errors +/// +/// Returns [`EventError::InvalidWirePayload`] when the recovered set is empty. +pub fn event_link_precision( + truth: &[EventLinkPair], + recovered: &[EventLinkPair], +) -> Result { + let truth_set: BTreeSet<_> = truth.iter().copied().collect(); + let recovered_set: BTreeSet<_> = recovered.iter().copied().collect(); + counted_rate( + recovered_set.intersection(&truth_set).count(), + recovered_set.len(), + ) +} + +/// Recall of recovered TDT links against the known-truth pair set. +/// +/// # Errors +/// +/// Returns [`EventError::InvalidWirePayload`] when the truth set is empty. +pub fn event_link_recall( + truth: &[EventLinkPair], + recovered: &[EventLinkPair], +) -> Result { + let truth_set: BTreeSet<_> = truth.iter().copied().collect(); + let recovered_set: BTreeSet<_> = recovered.iter().copied().collect(); + counted_rate( + recovered_set.intersection(&truth_set).count(), + truth_set.len(), + ) +} + +fn counted_rate(numerator: usize, denominator: usize) -> Result { + let numerator = u32::try_from(numerator).map_err(|_| EventError::InvalidWirePayload)?; + let denominator = u32::try_from(denominator).map_err(|_| EventError::InvalidWirePayload)?; + if denominator == 0 { + return Err(EventError::InvalidWirePayload); + } + Ok(f64::from(numerator) / f64::from(denominator)) +} + +#[cfg(test)] +mod tests { + use super::{ + EventLinkLabel, EventLinkPair, counted_rate, decide_event_link, event_link_precision, + event_link_recall, refuse_event_link_as_instance, refuse_event_link_as_transition, + }; + use crate::{EventConfidence, EventError, EventMentionId}; + + #[test] + fn link_helpers_cover_local_branches() { + let left = EventMentionId::new(); + let right = EventMentionId::new(); + let link = EventLinkPair::new(left, right).expect("pair"); + assert_eq!( + refuse_event_link_as_instance(link), + Err(EventError::EventLinkIsNotEventInstance) + ); + assert_eq!( + refuse_event_link_as_transition(link), + Err(EventError::EventLinkIsNotStateTransition) + ); + let high = EventConfidence::new(0.8).expect("high"); + let low = EventConfidence::new(0.2).expect("low"); + assert_eq!(decide_event_link(high, low), EventLinkLabel::Linked); + assert_eq!(decide_event_link(low, high), EventLinkLabel::Unlinked); + let truth = [link]; + let recovered = [link]; + assert!((event_link_precision(&truth, &recovered).expect("p") - 1.0).abs() < f64::EPSILON); + assert!((event_link_recall(&truth, &recovered).expect("r") - 1.0).abs() < f64::EPSILON); + assert_eq!( + EventLinkPair::new(left, left), + Err(EventError::InvalidWirePayload) + ); + assert_eq!( + event_link_precision(&truth, &[]), + Err(EventError::InvalidWirePayload) + ); + assert_eq!( + event_link_recall(&[], &recovered), + Err(EventError::InvalidWirePayload) + ); + assert_eq!( + counted_rate(0, usize::MAX), + Err(EventError::InvalidWirePayload) + ); + assert_eq!(counted_rate(1, 0), Err(EventError::InvalidWirePayload)); + } +} diff --git a/crates/event_core/tests/link_detection_contract.rs b/crates/event_core/tests/link_detection_contract.rs new file mode 100644 index 00000000..83389c1c --- /dev/null +++ b/crates/event_core/tests/link_detection_contract.rs @@ -0,0 +1,132 @@ +//! TDT link detections are not instances; precision/recall come from truth. + +use event_core::{ + EventConfidence, EventError, EventLinkLabel, EventLinkPair, EventMentionId, decide_event_link, + event_link_precision, event_link_recall, refuse_event_link_as_instance, + refuse_event_link_as_transition, +}; + +fn computed_rmse(truth: &[f64], recovered: &[f64]) -> f64 { + assert_eq!(truth.len(), recovered.len()); + let n = f64::from(u32::try_from(truth.len()).expect("tiny fixture")); + let sse: f64 = truth + .iter() + .zip(recovered) + .map(|(truth_value, recovered_value)| { + let residual = truth_value - recovered_value; + residual * residual + }) + .sum(); + (sse / n).sqrt() +} + +fn pair(left: EventMentionId, right: EventMentionId) -> EventLinkPair { + EventLinkPair::new(left, right).expect("distinct mentions") +} + +#[test] +fn event_link_detection_cannot_be_cast_to_an_instance_or_transition() { + let left = EventMentionId::new(); + let right = EventMentionId::new(); + let link = pair(left, right); + assert_eq!( + refuse_event_link_as_instance(link), + Err(EventError::EventLinkIsNotEventInstance) + ); + assert_eq!( + refuse_event_link_as_transition(link), + Err(EventError::EventLinkIsNotStateTransition) + ); +} + +#[test] +fn precision_and_recall_are_computed_from_known_truth_pairs() { + let a = EventMentionId::new(); + let b = EventMentionId::new(); + let c = EventMentionId::new(); + let d = EventMentionId::new(); + let truth = [pair(a, b), pair(b, c)]; + let calibrated = [pair(a, b)]; + let always_link = [pair(a, b), pair(b, c), pair(c, d), pair(a, d)]; + + let calibrated_precision = event_link_precision(&truth, &calibrated).expect("precision"); + let naive_precision = event_link_precision(&truth, &always_link).expect("naive precision"); + let calibrated_recall = event_link_recall(&truth, &calibrated).expect("recall"); + let naive_recall = event_link_recall(&truth, &always_link).expect("naive recall"); + + assert!( + calibrated_precision > naive_precision, + "computed precision {calibrated_precision} must exceed always-link precision {naive_precision}" + ); + assert!( + calibrated_recall < naive_recall, + "computed recall {calibrated_recall} must stay below the always-link recall {naive_recall}" + ); +} + +#[test] +fn calibrated_link_scores_have_lower_rmse_than_always_link() { + let truth = [1.0_f64, 0.0, 0.0, 1.0, 0.0, 0.0]; + let calibrated = [0.90_f64, 0.10, 0.15, 0.85, 0.20, 0.05]; + let always_link = [1.0_f64, 1.0, 1.0, 1.0, 1.0, 1.0]; + let calibrated_rmse = computed_rmse(&truth, &calibrated); + let naive_rmse = computed_rmse(&truth, &always_link); + assert!( + calibrated_rmse < naive_rmse, + "computed calibrated RMSE {calibrated_rmse} must be below always-link RMSE {naive_rmse}" + ); +} + +#[test] +fn pair_helpers_fail_closed_on_self_links_empty_and_missing_sets() { + let mention = EventMentionId::new(); + assert_eq!( + EventLinkPair::new(mention, mention), + Err(EventError::InvalidWirePayload) + ); + let a = EventMentionId::new(); + let b = EventMentionId::new(); + let truth = [pair(a, b)]; + assert_eq!( + event_link_precision(&truth, &[]), + Err(EventError::InvalidWirePayload) + ); + assert_eq!( + event_link_recall(&[], &truth), + Err(EventError::InvalidWirePayload) + ); +} + +#[test] +fn labels_round_trip_and_threshold_is_inclusive() { + assert_eq!(EventLinkLabel::Linked.wire_name(), "linked"); + assert_eq!(EventLinkLabel::Unlinked.wire_name(), "unlinked"); + assert_eq!( + EventLinkLabel::from_wire_name("linked").expect("parse"), + EventLinkLabel::Linked + ); + assert_eq!( + EventLinkLabel::from_wire_name("unlinked").expect("parse"), + EventLinkLabel::Unlinked + ); + assert_eq!( + EventLinkLabel::from_wire_name("same_event"), + Err(EventError::UnknownEventLinkLabel) + ); + assert!(EventLinkLabel::Linked.is_linked()); + assert!(!EventLinkLabel::Unlinked.is_linked()); + assert!((EventLinkLabel::Linked.as_probability_target() - 1.0).abs() < f64::EPSILON); + assert!((EventLinkLabel::Unlinked.as_probability_target() - 0.0).abs() < f64::EPSILON); + + let half = EventConfidence::new(0.5).expect("half"); + assert_eq!(decide_event_link(half, half), EventLinkLabel::Linked); + assert_eq!( + decide_event_link(EventConfidence::new(0.49).expect("below"), half), + EventLinkLabel::Unlinked + ); + + let left = EventMentionId::new(); + let right = EventMentionId::new(); + assert_eq!(pair(left, right), pair(right, left)); + assert_ne!(pair(left, right).left(), pair(left, right).right()); +} diff --git a/docs/TRACEABILITY.md b/docs/TRACEABILITY.md index 9f864156..283fab68 100644 --- a/docs/TRACEABILITY.md +++ b/docs/TRACEABILITY.md @@ -68,6 +68,8 @@ The full APA 7th standards/literature register remains `docs/research/standards- | CPU bounded multithreading + GPU/VRAM streaming/parity | ADR 0001/0006 | future `compute_backend` | accepted-target | | posterior ESEM / longitudinal invariance / DSEM | ADR 0005 | `psychometric_fit` ESEM loading and DSEM lag gates on the active PR; `psychometric_core` input gates remain #49; invariance/multilevel remain accepted-target | active-PR | | CPU bounded multithreading + GPU/VRAM streaming/parity | ADR 0001/0006 | future `compute_backend` | accepted-target | +| TDT detection/tracking vs CHRONOS schema/prediction/temporal consistency | ADR 0016; PRD/research | `event_core` TDT link precision/recall on the active PR; remaining TDT/CHRONOS stack and any future `event_intelligence` crate remain accepted-target | active-PR | +| evidence-bounded LLM interpretation | ADR 0010/0012; PRD | `tepp_api` router plus future `interpretation_gateway` | partial | | TDT detection/tracking vs CHRONOS schema/prediction/temporal consistency | ADR 0016; PRD/research | `event_core` first-story FAR/miss and mention-confidence Brier scoring on the active PR; remaining TDT/CHRONOS stack and any future `event_intelligence` crate remain accepted-target | active-PR | | evidence-bounded LLM interpretation | ADR 0010/0012; PRD | `tepp_api` router implemented-main plus future `interpretation_gateway` | partial | | CPU bounded multithreading + GPU/VRAM streaming/parity | ADR 0001/0006 | `compute_backend` CPU `f64` reference, bounded planning, and VRAM-budget refusal are active; full GPU streaming and CPU/GPU parity remain future | partial | diff --git a/docs/adr/0016-tdt-chronos-event-intelligence-boundary.md b/docs/adr/0016-tdt-chronos-event-intelligence-boundary.md index f9f9590b..8f4976cf 100644 --- a/docs/adr/0016-tdt-chronos-event-intelligence-boundary.md +++ b/docs/adr/0016-tdt-chronos-event-intelligence-boundary.md @@ -1,6 +1,7 @@ # ADR 0016 — TDT, CHRONOS, and Event Ontology intelligence boundary **Decision status:** Accepted +**Implementation maturity:** active-PR — evidence-layer admission, TDT link precision/recall, and detection-versus-instance refusal live in existing `event_core`, alongside the bounded predicted-vs-observed Allen promotion gate whose coverage authorization precedes any unmatched predicted mass; full TDT tracking/calibration and CHRONOS schema extraction/prediction layers remain accepted-target **Implementation maturity:** active-PR — evidence-layer admission, first-story false-alarm/miss rates, and detection-versus-instance refusal live in existing `event_core`, alongside the bounded predicted-vs-observed Allen promotion gate whose coverage authorization precedes any unmatched predicted mass; full TDT tracking/calibration and CHRONOS schema extraction/prediction layers remain accepted-target **Date:** 2026-08-12 **Decision status:** Accepted diff --git a/docs/adr/README.md b/docs/adr/README.md index c20b6c41..1a742559 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -57,6 +57,7 @@ Read [`ADR_POLICY.md`](ADR_POLICY.md) first. **Decision status and implementatio | [0013](0013-bitemporal-persistence-reproducibility-and-split-authority.md) | Bitemporal persistence, reproducibility manifests, and relation-aware split authority | Accepted | partial | Owns PostgreSQL adapter semantics, immutable run/split manifests, leakage-safe partitions, and recovery identity; optional `live-sqlx` `PgPool`, live PG CI, tenant RLS, and `0006` membership implemented-main; `0007` retention/deletion/legal-hold on the active PR; remaining physical ERD/backup accepted-target. | | [0014](0014-scientific-claim-promotion-and-release-evidence.md) | Scientific claim promotion and release evidence authority | Accepted | partial | Separates design, implementation, scientific/product claim, and release authority; repository SBOM/provenance generator implemented; checkpoint-versus-estimator refusal is `checkpoint_authority` on the active PR; full release bundle remaining. | | [0015](0015-autonomous-development-review-and-merge-authority.md) | Autonomous development, review, and merge authority separation | Accepted | active-PR | Separates model proposal, deterministic verification, publication, independent review, and merge/release authority. | +| [0016](0016-tdt-chronos-event-intelligence-boundary.md) | TDT, CHRONOS, and Event Ontology intelligence boundary | Accepted | active-PR | TDT link precision/recall in existing `event_core`; remaining TDT/CHRONOS stack remains accepted-target. | | [0016](0016-tdt-chronos-event-intelligence-boundary.md) | TDT, CHRONOS, and Event Ontology intelligence boundary | Accepted | accepted-target | Separates observed evidence, detection/tracking, prediction/schema inference, temporal consistency, and promoted transition authority. | | [0016](0016-tdt-chronos-event-intelligence-boundary.md) | TDT, CHRONOS, and Event Ontology intelligence boundary | Accepted | active-PR | First-story FAR/miss in existing `event_core`; remaining TDT/CHRONOS stack remains accepted-target. | | [0020](0020-span-grounded-semantic-units.md) | Span-grounded semantic units; language tags are not identity | Accepted | active-PR | First ADR 0004 production slice. Does not claim concept alignment, invariance, or a topic estimator. | diff --git a/docs/research/event-link-detection-calibration.md b/docs/research/event-link-detection-calibration.md new file mode 100644 index 00000000..696da812 --- /dev/null +++ b/docs/research/event-link-detection-calibration.md @@ -0,0 +1,31 @@ +# Event-link detection calibration + +## Scope + +This note doctors the `event_core` gate that keeps TDT link detection distinct from event-instance promotion and state-transition authority: + +1. a linked versus unlinked mention pair is detection evidence, not a promoted instance or transition; +2. precision and recall are computed from known-truth pair sets; +3. calibrated link probabilities recover the binary same-event target with lower RMSE than an always-link detector. + +No database migration is allocated. A later TDT tracker may consume these scores as measurement evidence only. + +## Authoritative sources + +Allan, J., Carbonell, J., Doddington, G., Yamron, J., & Yang, Y. (1998). Topic detection and tracking pilot study: Final report. In *Proceedings of the DARPA Broadcast News Transcription and Understanding Workshop* (pp. 194–218). + +Allan, J. (Ed.). (2002). *Topic detection and tracking: Event-based information organization*. Kluwer Academic Publishers. + +Fiscus, J. G., & Doddington, G. R. (2002). Topic detection and tracking evaluation overview. In J. Allan (Ed.), *Topic detection and tracking: Event-based information organization* (pp. 17–31). Kluwer Academic Publishers. + +## Application + +Allan et al. (1998) and Allan (2002) define link detection as a *same-event / same-story* decision over story pairs. Fiscus and Doddington (2002) keep official TDT link scores in the measurement layer and report miss and false-alarm trade-offs rather than instance identity. TEPP therefore refuses to cast a detected link as an event instance or a forward state transition and requires computed precision, recall, and RMSE against known truth (Allan et al., 1998; Allan, 2002; Fiscus & Doddington, 2002). + +## Verification + +- `refuse_event_link_as_instance` always returns `EventLinkIsNotEventInstance`; +- `refuse_event_link_as_transition` always returns `EventLinkIsNotStateTransition`; +- `EventLinkPair::new` refuses a self-link and normalizes pair order; +- `event_link_precision` and `event_link_recall` fail closed on empty recovered or truth sets; +- computed RMSE of known link targets is lower under calibrated probabilities than under an always-link detector. diff --git a/docs/validation/temporal-event-foundation.md b/docs/validation/temporal-event-foundation.md index 0e93dd24..77665e40 100644 --- a/docs/validation/temporal-event-foundation.md +++ b/docs/validation/temporal-event-foundation.md @@ -20,6 +20,7 @@ This report tracks exact-head scientific and engineering evidence required befor | Knowledge-cutoff identity | `cutoff_clock` | active-PR | this PR | recovered cutoff flags vs availability-time stand-in | ADR 0002 | | Allen path-consistency | `temporal_core` | implemented-main | — | unit + budget tests | Task 4 / PR #9 | | Event mention/instance | `event_core` | partial | — | unit + fail-closed promotion | Task 5 / PR #13 | +| TDT link precision/recall | `event_core` | active-PR | this PR | computed precision/recall + RMSE vs always-link | ADR 0016; `docs/research/event-link-detection-calibration.md` | | First-story FAR/miss | `event_core` | active-PR | this PR | computed FAR/miss + RMSE vs always-first | ADR 0016; `docs/research/first-story-detection-calibration.md` | | Multiple membership | `membership_core` | partial | nested ICC + non-nested refusal | unit + ESS + nested ICC recovery | Task 7 / PR #12 + #25 + this increment | | TDT tracking stability | `event_core` | active-PR | this PR | pair P/R + switch rate + RMSE vs always-one-track | ADR 0016; `docs/research/event-tracking-calibration.md` |