feat(method): refuse prompt boilerplate as unique content

ARCHITECTURE.md

@@ -61,6 +61,7 @@ boundaries above remain the target modular MSA architecture.
 | `tepp_simulation` | known-truth temporal/event data generation |
 | `validation_core` | RMSE, bias, coverage, graph, and Monte Carlo metrics |
 | `tepp_api` | versioned DTO, schema, and export contracts |
+| `prompt_source` | prompt boilerplate is not unique latent content and not stopword deletion |
 | `corpus_background` | corpus-background wording is not unique latent content and not stopword deletion |
 | `modality_source` | non-lexical modality is not unique latent content and not stopword deletion |
 | `copied_text` | copied-text residue is not unique latent content and not stopword deletion |

CHANGELOG.md

@@ -6,6 +6,7 @@ All notable changes to TEPP are documented here. The format follows Keep a Chang
 
 ### Added
 
+- `prompt_source` identity gate: instruction and prompt boilerplate is not unique latent content and is not erased by a stopword list; `identity_recovery_rate` reports exact kind matches, with a contract test comparing correct recovery with an all-unique collapse on a mixed known-truth fixture (ADR 0004/0012).
 - `corpus_background` identity gate: corpus-level background wording is not unique latent content and is not erased by a stopword list; recovered background kinds match known truth at a higher computed rate than collapsing every token to unique content (ADR 0004/0012).
 - `modality_source` identity gate: non-lexical modality is not unique latent content and is not erased by a stopword list; recovered modality kinds match known truth at a higher computed rate than collapsing every token to unique content (ADR 0004/0012).
 - `copied_text` identity gate: copied and boilerplate residue is not unique latent content and is not erased by a stopword list; recovered copied-text kinds match known truth at a higher computed rate than collapsing every token to unique content (ADR 0004/0012).

Cargo.toml

@@ -11,6 +11,7 @@ members = [
     "crates/tepp_simulation",
     "crates/validation_core",
     "crates/tepp_api",
+    "crates/prompt_source",
     "crates/corpus_background",
     "crates/modality_source",
     "crates/copied_text",
@@ -59,6 +60,7 @@ default-members = [
     "crates/tepp_simulation",
     "crates/validation_core",
     "crates/tepp_api",
+    "crates/prompt_source",
     "crates/corpus_background",
     "crates/modality_source",
     "crates/copied_text",

README.md

@@ -43,6 +43,7 @@ crates/corpus_split
 crates/tepp_simulation
 crates/validation_core
 crates/tepp_api
+crates/prompt_source
 crates/corpus_background
 crates/modality_source
 crates/copied_text

crates/prompt_source/Cargo.toml

@@ -0,0 +1,17 @@
+[package]
+name = "prompt_source"
+description = "Prompt boilerplate is not unique content and not stopword deletion."
+version.workspace = true
+edition.workspace = true
+rust-version.workspace = true
+license.workspace = true
+authors.workspace = true
+repository.workspace = true
+homepage.workspace = true
+readme.workspace = true
+keywords.workspace = true
+categories.workspace = true
+publish = false
+
+[lints]
+workspace = true

crates/prompt_source/src/error.rs

@@ -0,0 +1,53 @@
+//! Fail-closed prompt-source errors.
+
+use std::fmt;
+
+/// A fail-closed prompt-source error.
+#[derive(Clone, Copy, Debug, Eq, PartialEq)]
+#[non_exhaustive]
+pub enum PromptSourceError {
+    /// Prompt boilerplate was treated as unique latent content.
+    PromptIsNotUniqueContent,
+    /// Prompt boilerplate was treated as stopword deletion.
+    PromptIsNotStopwordDeletion,
+    /// A recovery slice was empty or length-mismatched.
+    InvalidPromptPayload,
+}
+
+impl fmt::Display for PromptSourceError {
+    fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
+        let message = match self {
+            Self::PromptIsNotUniqueContent => "prompt boilerplate is not unique latent content",
+            Self::PromptIsNotStopwordDeletion => "prompt boilerplate is not stopword deletion",
+            Self::InvalidPromptPayload => "invalid prompt-source payload",
+        };
+        formatter.write_str(message)
+    }
+}
+
+impl std::error::Error for PromptSourceError {}
+
+#[cfg(test)]
+mod tests {
+    use super::PromptSourceError;
+
+    #[test]
+    fn error_messages_are_stable() {
+        for (error, message) in [
+            (
+                PromptSourceError::PromptIsNotUniqueContent,
+                "prompt boilerplate is not unique latent content",
+            ),
+            (
+                PromptSourceError::PromptIsNotStopwordDeletion,
+                "prompt boilerplate is not stopword deletion",
+            ),
+            (
+                PromptSourceError::InvalidPromptPayload,
+                "invalid prompt-source payload",
+            ),
+        ] {
+            assert_eq!(error.to_string(), message);
+        }
+    }
+}

crates/prompt_source/src/kind.rs

@@ -0,0 +1,132 @@
+//! Prompt boilerplate versus unique latent content.
+
+use crate::PromptSourceError;
+
+/// Closed vocabulary of prompt-related token treatments.
+#[derive(Clone, Copy, Debug, Eq, PartialEq)]
+pub enum PromptKind {
+    /// Instruction or prompt boilerplate, not unique document meaning.
+    PromptBoilerplate,
+    /// Token treatment reserved for unique latent content.
+    UniqueContent,
+}
+
+impl PromptKind {
+    /// Return the stable wire kind name.
+    #[must_use]
+    pub const fn wire_name(self) -> &'static str {
+        match self {
+            Self::PromptBoilerplate => "prompt_boilerplate",
+            Self::UniqueContent => "unique_content",
+        }
+    }
+
+    /// Parse a stable wire kind name.
+    ///
+    /// # Errors
+    ///
+    /// Returns [`PromptSourceError::InvalidPromptPayload`] for unrecognized
+    /// names.
+    pub fn from_wire_name(name: &str) -> Result<Self, PromptSourceError> {
+        match name {
+            "prompt_boilerplate" => Ok(Self::PromptBoilerplate),
+            "unique_content" => Ok(Self::UniqueContent),
+            _ => Err(PromptSourceError::InvalidPromptPayload),
+        }
+    }
+}
+
+/// Refuse to treat prompt boilerplate as unique latent content.
+///
+/// # Errors
+///
+/// Returns [`PromptSourceError::PromptIsNotUniqueContent`] when `kind` is
+/// [`PromptKind::PromptBoilerplate`].
+pub fn refuse_prompt_as_unique_content(kind: PromptKind) -> Result<(), PromptSourceError> {
+    match kind {
+        PromptKind::PromptBoilerplate => Err(PromptSourceError::PromptIsNotUniqueContent),
+        PromptKind::UniqueContent => Ok(()),
+    }
+}
+
+/// Refuse to treat prompt boilerplate as stopword deletion.
+///
+/// # Errors
+///
+/// Returns [`PromptSourceError::PromptIsNotStopwordDeletion`] when `kind` is
+/// [`PromptKind::PromptBoilerplate`].
+pub fn refuse_prompt_as_stopword_deletion(kind: PromptKind) -> Result<(), PromptSourceError> {
+    match kind {
+        PromptKind::PromptBoilerplate => Err(PromptSourceError::PromptIsNotStopwordDeletion),
+        PromptKind::UniqueContent => Ok(()),
+    }
+}
+
+/// Fraction of recovered prompt kinds that match known truth.
+///
+/// # Errors
+///
+/// Returns [`PromptSourceError::InvalidPromptPayload`] when either slice is
+/// empty or the lengths differ.
+pub fn identity_recovery_rate(
+    truth: &[PromptKind],
+    decided: &[PromptKind],
+) -> Result<f64, PromptSourceError> {
+    if truth.is_empty() || truth.len() != decided.len() {
+        return Err(PromptSourceError::InvalidPromptPayload);
+    }
+    let mut matches = 0_u32;
+    for (truth_kind, decided_kind) in truth.iter().zip(decided) {
+        if truth_kind == decided_kind {
+            matches += 1;
+        }
+    }
+    Ok(f64::from(matches) / truth.len() as f64)
+}
+
+#[cfg(test)]
+mod tests {
+    use super::{
+        PromptKind, identity_recovery_rate, refuse_prompt_as_stopword_deletion,
+        refuse_prompt_as_unique_content,
+    };
+    use crate::PromptSourceError;
+
+    #[test]
+    fn local_branches_cover_kinds_payloads_and_wire_names() {
+        assert_eq!(
+            refuse_prompt_as_unique_content(PromptKind::PromptBoilerplate),
+            Err(PromptSourceError::PromptIsNotUniqueContent)
+        );
+        assert_eq!(
+            refuse_prompt_as_stopword_deletion(PromptKind::PromptBoilerplate),
+            Err(PromptSourceError::PromptIsNotStopwordDeletion)
+        );
+        refuse_prompt_as_unique_content(PromptKind::UniqueContent).expect("unique");
+        refuse_prompt_as_stopword_deletion(PromptKind::UniqueContent).expect("unique");
+        for kind in [PromptKind::PromptBoilerplate, PromptKind::UniqueContent] {
+            assert_eq!(
+                PromptKind::from_wire_name(kind.wire_name()).expect("round-trip"),
+                kind
+            );
+        }
+        assert_eq!(
+            PromptKind::from_wire_name("template"),
+            Err(PromptSourceError::InvalidPromptPayload)
+        );
+        let matched = identity_recovery_rate(
+            &[PromptKind::PromptBoilerplate],
+            &[PromptKind::PromptBoilerplate],
+        )
+        .expect("rate");
+        assert!((matched - 1.0).abs() < f64::EPSILON);
+        assert_eq!(
+            identity_recovery_rate(&[], &[]),
+            Err(PromptSourceError::InvalidPromptPayload)
+        );
+        assert_eq!(
+            identity_recovery_rate(&[PromptKind::PromptBoilerplate], &[]),
+            Err(PromptSourceError::InvalidPromptPayload)
+        );
+    }
+}

crates/prompt_source/src/lib.rs

@@ -0,0 +1,22 @@
+#![forbid(unsafe_code)]
+#![deny(missing_docs)]
+#![allow(clippy::cast_precision_loss)]
+//! Prompt boilerplate is not unique latent content.
+//!
+//! Instruction and prompt text stays explicit method structure. It is not
+//! unique document meaning and is not erased by a stopword list
+//! (ADR 0004/0012).
+
+mod error;
+mod kind;
+
+/// Fail-closed prompt-source errors.
+pub use error::PromptSourceError;
+/// Closed vocabulary of prompt-related token treatments.
+pub use kind::PromptKind;
+/// Fraction of recovered prompt kinds that match known truth.
+pub use kind::identity_recovery_rate;
+/// Refuse to treat prompt boilerplate as stopword deletion.
+pub use kind::refuse_prompt_as_stopword_deletion;
+/// Refuse to treat prompt boilerplate as unique latent content.
+pub use kind::refuse_prompt_as_unique_content;

crates/prompt_source/tests/crate_contract.rs

@@ -0,0 +1,7 @@
+//! Integration contract for the `prompt_source` package identity.
+
+#[test]
+fn package_identity_is_stable() {
+    let observed = std::hint::black_box(env!("CARGO_PKG_NAME"));
+    assert_eq!(observed, "prompt_source");
+}

crates/prompt_source/tests/prompt_source_contract.rs

@@ -0,0 +1,67 @@
+//! Prompt boilerplate is not unique content and not stopword deletion.
+
+use prompt_source::{
+    PromptKind, PromptSourceError, identity_recovery_rate, refuse_prompt_as_stopword_deletion,
+    refuse_prompt_as_unique_content,
+};
+
+#[test]
+fn prompt_boilerplate_cannot_become_unique_content_or_stopword_deletion() {
+    assert_eq!(
+        refuse_prompt_as_unique_content(PromptKind::PromptBoilerplate),
+        Err(PromptSourceError::PromptIsNotUniqueContent)
+    );
+    assert_eq!(
+        refuse_prompt_as_stopword_deletion(PromptKind::PromptBoilerplate),
+        Err(PromptSourceError::PromptIsNotStopwordDeletion)
+    );
+    refuse_prompt_as_unique_content(PromptKind::UniqueContent).expect("unique");
+    refuse_prompt_as_stopword_deletion(PromptKind::UniqueContent).expect("unique");
+}
+
+#[test]
+fn recovered_kinds_match_known_truth_better_than_a_unique_content_collapse() {
+    let truth = [
+        PromptKind::PromptBoilerplate,
+        PromptKind::UniqueContent,
+        PromptKind::PromptBoilerplate,
+    ];
+    let recovered = truth;
+    let collapsed = [
+        PromptKind::UniqueContent,
+        PromptKind::UniqueContent,
+        PromptKind::UniqueContent,
+    ];
+    let recovered_rate = identity_recovery_rate(&truth, &recovered).expect("recovered");
+    let collapsed_rate = identity_recovery_rate(&truth, &collapsed).expect("collapsed");
+    let expected = {
+        let mut matches = 0_u32;
+        for (truth_kind, decided_kind) in truth.iter().zip(recovered.iter()) {
+            if truth_kind == decided_kind {
+                matches += 1;
+            }
+        }
+        f64::from(matches) / f64::from(u32::try_from(truth.len()).expect("len"))
+    };
+    assert!((recovered_rate - expected).abs() < f64::EPSILON);
+    assert!(recovered_rate > collapsed_rate);
+}
+
+#[test]
+fn empty_or_mismatched_kind_payloads_fail_closed() {
+    assert_eq!(
+        identity_recovery_rate(&[], &[]),
+        Err(PromptSourceError::InvalidPromptPayload)
+    );
+    assert_eq!(
+        identity_recovery_rate(&[PromptKind::PromptBoilerplate], &[]),
+        Err(PromptSourceError::InvalidPromptPayload)
+    );
+    assert_eq!(
+        identity_recovery_rate(
+            &[PromptKind::PromptBoilerplate, PromptKind::UniqueContent],
+            &[PromptKind::PromptBoilerplate]
+        ),
+        Err(PromptSourceError::InvalidPromptPayload)
+    );
+}

docs/TRACEABILITY.md

@@ -25,7 +25,7 @@ The full APA 7th standards/literature register remains `docs/research/standards-
 | TRSL-TM temporal/relational topic posterior and backend compatibility | ADR 0012; ADR 0004 | future `topic_measurement` | accepted-target |
 | global P0 topic identity with activity/dormancy/reactivation | ADR 0012 | future topic lineage/activity state | accepted-target |
 | no default stopword deletion / no TF-IDF-BM25 inferential weighting | ADR 0004/0012; PRD/TRD | future semantic/method-source model | accepted-target |
-| report template/section/copied/style/modality method effects | ADR 0004/0012; PRD/TRD | simulation truth factors implemented; `corpus_background` background-versus-unique-content identity on the active PR; estimator-side method model remains future | partial |
+| report template/section/copied/style/modality method effects | ADR 0004/0012; PRD/TRD | simulation truth factors implemented; `prompt_source` prompt-versus-unique-content identity on the active PR; estimator-side method model remains future | partial |
 | candidate K statistical/Pareto gates + blinded LLM review | ADR 0012; research | future `model_selection` | accepted-target |
 | compositional topic correlation / stable clustering | ADR 0005/0012; research | future `network_analysis` | accepted-target |
 | posterior ESEM / longitudinal invariance / DSEM | ADR 0005 | `psychometric_fit` ESEM loading and DSEM lag gates on the active PR; `psychometric_core` input gates remain #49; invariance/multilevel remain accepted-target | active-PR |

docs/adr/0004-shared-multilingual-latent-space.md

@@ -1,6 +1,7 @@
 # ADR 0004 — Shared multilingual latent semantic space
 
 **Decision status:** Accepted  
+**Implementation maturity:** accepted-target — prompt-versus-unique-content identity in `prompt_source` on the active PR; shared-space estimators remain accepted-target  
 **Implementation maturity:** accepted-target — corpus-background-versus-unique-content identity in `corpus_background` on the active PR; shared-space estimators remain accepted-target  
 **Implementation maturity:** accepted-target — modality-versus-unique-content identity in `modality_source` on the active PR; shared-space estimators remain accepted-target  
 **Implementation maturity:** accepted-target — copied-versus-unique-content identity in `copied_text` on the active PR; shared-space estimators remain accepted-target  

docs/adr/0012-temporal-relational-shared-latent-topic-measurement.md

@@ -1,6 +1,7 @@
 # ADR 0012 — Temporal Relational Shared-Latent Topic Measurement
 
 **Decision status:** Accepted  
+**Implementation maturity:** accepted-target — prompt-versus-unique-content identity in `prompt_source` on the active PR; estimator-side method model remains accepted-target  
 **Implementation maturity:** accepted-target — corpus-background-versus-unique-content identity in `corpus_background` on the active PR; estimator-side method model remains accepted-target  
 **Implementation maturity:** accepted-target — modality-versus-unique-content identity in `modality_source` on the active PR; estimator-side method model remains accepted-target  
 **Implementation maturity:** accepted-target — copied-versus-unique-content identity in `copied_text` on the active PR; estimator-side method model remains accepted-target