diff --git a/CHANGELOG.md b/CHANGELOG.md index 3fe6d08d..93891a27 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,8 +6,8 @@ All notable changes to TEPP are documented here. The format follows Keep a Chang ### Added +- `tepp_api` adaptive orchestration router (ADR 0010): versioned `direct`/`verify`/`committee`/`conductor`/`abstain` selection from CPU `f64` risk, ambiguity, evidence, and token-budget inputs; recorded stages, recursion, decomposition, access lists, and role-specific reasoning effort; fail-closed document-controlled policy/access/credentials; LLM plans remain proposals under deterministic statistical authority; comparable-budget ablation requires a direct baseline; credential-free contextual-orchestrator binding. Live NIM HTTP remains accepted-target. - `tepp_api` purpose-bound provider-payload minimization: time-bounded `PurposeGrant` evaluation, fail-closed expired/not-yet-valid/inverted/cross-tenant/impossible-calendar denial, semantic UTC calendar validation, refusal to copy identity mappings into model-provider payloads or ordinary logs, preservation of opaque analytical identifiers and membership roles (no blanket PII mask), a separately authorized scientific re-identification path, and an internally bound FIPS 180-4 SHA-256 audit digest appended through `ReidentificationAuditSink` before disclosure. -- `persistence_postgres` retention/deletion/legal-hold (migration `0007`): policy rows, legal holds that block completed deletion, evidence tombstones without raw-source restore, analysis exclusion only for `logical_revocation`/`identity_tombstone` (not `cache_export_removal`), and deletion requests bound to the cited retention policy's tenant/class/purpose. - `persistence_postgres` backup/restore integrity: restored snapshots stay unusable until tenant, canonical `SHA-256`, knowledge-cutoff eligibility, temporal window order, and append-only triggers revalidate; SQL probes raise `restore integrity failed` (ADR 0013). - `persistence_postgres` concurrent document-write stress: atomic revise `DO` block that requires exactly one open `system_to` close, SQLSTATE mapping onto `ConcurrentWriteConflict` / `DuplicateDocumentRecord`, and live multi-session insert/revise/append-only proofs. No new migration number. - `tepp_api` naruon HTTP interchange: versioned `https` POST contracts for analysis-run create and modular export authorization that refuse table-access URLs, review/Copilot credential headers, reserved standard-header redefinition, principal-only export idempotency keys, and lexical inference claims (ADR 0011). diff --git a/DOCUMENTATION.md b/DOCUMENTATION.md index f64cf32b..3f094947 100644 --- a/DOCUMENTATION.md +++ b/DOCUMENTATION.md @@ -35,6 +35,7 @@ TEPP's approved PRD v0.4 and implementation plan are the primary product baselin | Actions fleet research doctoring | [`docs/research/actions-workflow-fleet.md`](docs/research/actions-workflow-fleet.md) | | Retention/deletion/legal-hold doctoring | [`docs/research/retention-deletion-legal-hold.md`](docs/research/retention-deletion-legal-hold.md) | | Provider-payload minimization doctoring | [`docs/research/provider-payload-minimization.md`](docs/research/provider-payload-minimization.md) | +| Adaptive orchestration router doctoring | [`docs/research/adaptive-orchestration-router.md`](docs/research/adaptive-orchestration-router.md) | | Hourly NIM OpenCode doctoring | [`docs/doctoring/hourly-nim-opencode-development.md`](docs/doctoring/hourly-nim-opencode-development.md) | | Change history | [`CHANGELOG.md`](CHANGELOG.md) | diff --git a/crates/tepp_api/src/lib.rs b/crates/tepp_api/src/lib.rs index 7be66c1b..411da83f 100644 --- a/crates/tepp_api/src/lib.rs +++ b/crates/tepp_api/src/lib.rs @@ -15,6 +15,7 @@ mod envelope; mod error; mod export; mod naruon_http; +mod orchestration; mod provider_payload; mod wire; @@ -67,6 +68,42 @@ pub use naruon_http::naruon_analysis_run_exchange_with_headers; pub use naruon_http::naruon_export_exchange; /// Refuse lexical heuristics as TEPP inference claims. pub use naruon_http::naruon_may_claim_tepp_inference; +/// Comparable-budget ablation record. +pub use orchestration::BudgetAblationRecord; +/// Credential-free contextual-orchestrator binding. +pub use orchestration::ContextualOrchestratorBinding; +/// Document attempt to override TEPP orchestration authority. +pub use orchestration::DocumentControlAttempt; +/// Bounded interpretation task kind. +pub use orchestration::InterpretationTaskKind; +/// Maximum access capabilities on one orchestration request. +pub use orchestration::MAX_ORCHESTRATION_ACCESS_ENTRIES; +/// Maximum UTF-8 bytes in one orchestration access token. +pub use orchestration::MAX_ORCHESTRATION_ACCESS_TOKEN_BYTES; +/// Maximum billable token budget on one orchestration request. +pub use orchestration::MAX_ORCHESTRATION_TOKEN_BUDGET; +/// Orchestration contract version for contextual-orchestrator bindings. +pub use orchestration::ORCHESTRATION_CONTRACT_VERSION; +/// Versioned TEPP orchestration policy identity. +pub use orchestration::ORCHESTRATION_POLICY_VERSION; +/// Versioned orchestration mode. +pub use orchestration::OrchestrationMode; +/// Governed orchestration plan. +pub use orchestration::OrchestrationPlan; +/// Orchestration router request. +pub use orchestration::OrchestrationRequest; +/// Orchestration role identity. +pub use orchestration::OrchestrationRole; +/// Role-specific reasoning effort. +pub use orchestration::ReasoningEffort; +/// Role plus recorded reasoning effort. +pub use orchestration::RoleAssignment; +/// Bind a plan for contextual-orchestrator execution. +pub use orchestration::bind_contextual_orchestrator; +/// Record a comparable-budget ablation against a direct baseline. +pub use orchestration::record_budget_ablation; +/// Route a task onto a versioned orchestration plan. +pub use orchestration::route_orchestration; /// Elevated re-identification result. pub use provider_payload::DisclosedIdentityMapping; /// Separately protected identity mapping. diff --git a/crates/tepp_api/src/orchestration.rs b/crates/tepp_api/src/orchestration.rs new file mode 100644 index 00000000..ff9de681 --- /dev/null +++ b/crates/tepp_api/src/orchestration.rs @@ -0,0 +1,1088 @@ +//! Deterministic ADR 0010 orchestration routing and orchestrator binding. + +use crate::ApiError; +use crate::wire::require_nonempty; +use std::fmt; + +/// Versioned TEPP orchestration policy identity. +pub const ORCHESTRATION_POLICY_VERSION: &str = "tepp.orchestration.v1"; + +/// Contract version for a contextual-orchestrator binding. +pub const ORCHESTRATION_CONTRACT_VERSION: u16 = 1; + +/// Maximum billable token budget accepted by one orchestration request. +pub const MAX_ORCHESTRATION_TOKEN_BUDGET: u64 = 1_000_000; +/// Maximum number of TEPP-owned access capabilities on one request. +pub const MAX_ORCHESTRATION_ACCESS_ENTRIES: usize = 64; +/// Maximum UTF-8 byte length of one access capability token. +pub const MAX_ORCHESTRATION_ACCESS_TOKEN_BYTES: usize = 128; + +const EVIDENCE_FLOOR: f64 = 0.35; +const LOW_COMPLEXITY: f64 = 0.35; +const HIGH_COMPLEXITY: f64 = 0.50; +const COMPARABLE_BUDGET_NUMERATOR: u64 = 10; + +/// Versioned orchestration mode selected by the governed router. +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub enum OrchestrationMode { + /// One model call for low-ambiguity schema-constrained work. + Direct, + /// Producer plus an independent verifier. + Verify, + /// Blinded parallel raters plus adjudication. + Committee, + /// Adaptive roles and topology under an explicit budget. + Conductor, + /// No forced answer when evidence, budget, or gates are insufficient. + Abstain, +} + +impl OrchestrationMode { + /// Return the stable wire name for this mode. + #[must_use] + pub const fn wire_name(self) -> &'static str { + match self { + Self::Direct => "direct", + Self::Verify => "verify", + Self::Committee => "committee", + Self::Conductor => "conductor", + Self::Abstain => "abstain", + } + } + + /// Minimum token budget required to execute this mode. + #[must_use] + pub const fn minimum_token_budget(self) -> u64 { + match self { + Self::Direct => 4_000, + Self::Verify => 8_000, + Self::Committee => 16_000, + Self::Conductor => 24_000, + Self::Abstain => 0, + } + } + + /// Default workflow stage count recorded for this mode. + #[must_use] + pub const fn stage_count(self) -> u8 { + match self { + Self::Direct => 1, + Self::Verify => 2, + Self::Committee => 3, + Self::Conductor => 4, + Self::Abstain => 0, + } + } + + /// Default recursion depth recorded for this mode. + #[must_use] + pub const fn recursion_depth(self) -> u8 { + match self { + Self::Conductor => 2, + Self::Direct | Self::Verify | Self::Committee | Self::Abstain => 0, + } + } + + /// Cheaper bounded fallback when this mode cannot complete. + /// + /// Keeping this match at one function boundary prevents callers from + /// duplicating unreachable enum branches during coverage instrumentation. + #[must_use] + #[inline(never)] + pub const fn fallback_mode(self) -> Self { + match self { + Self::Conductor => Self::Committee, + Self::Committee => Self::Verify, + Self::Verify => Self::Direct, + Self::Direct | Self::Abstain => Self::Abstain, + } + } + + /// Decomposition code recorded on the plan. + #[must_use] + pub const fn decomposition_code(self) -> &'static str { + match self { + Self::Direct => "single_call", + Self::Verify => "producer_then_verifier", + Self::Committee => "blinded_parallel_then_adjudicate", + Self::Conductor => "adaptive_roles_under_budget", + Self::Abstain => "no_forced_answer", + } + } +} + +/// Role-specific reasoning effort recorded for ablation. +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub enum ReasoningEffort { + /// Schema conversion or formatting. + Minimal, + /// Low-ambiguity span classification. + Low, + /// Concept alignment or narrative synthesis. + Medium, + /// Verification, adjudication, or blinded review. + High, +} + +impl ReasoningEffort { + /// Return the stable wire name for this effort. + #[must_use] + pub const fn wire_name(self) -> &'static str { + match self { + Self::Minimal => "minimal", + Self::Low => "low", + Self::Medium => "medium", + Self::High => "high", + } + } +} + +/// Bounded interpretation task the router may schedule. +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub enum InterpretationTaskKind { + /// Semantic span extraction or classification. + SpanClassification, + /// Concept merge or alignment review. + ConceptAlignment, + /// Blinded K/model-selection review after statistical gates. + BlindedModelReview, + /// Evidence-grounded narrative synthesis. + NarrativeSynthesis, + /// Adversarial unsupported-claim verification. + AdversarialVerification, + /// Routine schema conversion or formatting. + SchemaConversion, +} + +impl InterpretationTaskKind { + /// Return the stable wire name for this task kind. + #[must_use] + pub const fn wire_name(self) -> &'static str { + match self { + Self::SpanClassification => "span_classification", + Self::ConceptAlignment => "concept_alignment", + Self::BlindedModelReview => "blinded_model_review", + Self::NarrativeSynthesis => "narrative_synthesis", + Self::AdversarialVerification => "adversarial_verification", + Self::SchemaConversion => "schema_conversion", + } + } + + /// Default reasoning effort for a worker assigned this task. + #[must_use] + pub const fn default_effort(self) -> ReasoningEffort { + match self { + Self::SchemaConversion => ReasoningEffort::Minimal, + Self::SpanClassification => ReasoningEffort::Low, + Self::ConceptAlignment | Self::NarrativeSynthesis => ReasoningEffort::Medium, + Self::BlindedModelReview | Self::AdversarialVerification => ReasoningEffort::High, + } + } +} + +/// Orchestration role assigned under the selected mode. +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub enum OrchestrationRole { + /// Lightweight coordinator or thinker. + Thinker, + /// Task worker that proposes an interpretation. + Worker, + /// Independent evidence-only verifier. + Verifier, + /// Committee adjudicator. + Adjudicator, + /// Adaptive conductor under an explicit budget. + Conductor, +} + +impl OrchestrationRole { + /// Return the stable wire name for this role. + #[must_use] + pub const fn wire_name(self) -> &'static str { + match self { + Self::Thinker => "thinker", + Self::Worker => "worker", + Self::Verifier => "verifier", + Self::Adjudicator => "adjudicator", + Self::Conductor => "conductor", + } + } +} + +/// One role plus its recorded reasoning effort. +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub struct RoleAssignment { + role: OrchestrationRole, + effort: ReasoningEffort, +} + +impl RoleAssignment { + /// Assigned orchestration role. + #[must_use] + pub const fn role(self) -> OrchestrationRole { + self.role + } + + /// Recorded reasoning effort for this role. + #[must_use] + pub const fn effort(self) -> ReasoningEffort { + self.effort + } +} + +/// Document attempt to override TEPP orchestration authority. +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub enum DocumentControlAttempt { + /// No document-supplied override. + None, + /// Document tried to set orchestration policy. + Policy, + /// Document tried to set the access list. + AccessList, + /// Document tried to supply provider credentials. + Credentials, +} + +/// Inputs to the governed orchestration router. +/// +/// Scores are CPU `f64` unit intervals. Documents cannot set policy, access +/// lists, or credentials; those attempts fail closed. +#[derive(Clone, Debug)] +pub struct OrchestrationRequest { + /// Must equal [`ORCHESTRATION_POLICY_VERSION`]. + pub policy_version: String, + /// Interpretation task being scheduled. + pub task_kind: InterpretationTaskKind, + /// Task risk in `[0, 1]`. + pub risk_score: f64, + /// Ambiguity in `[0, 1]`. + pub ambiguity_score: f64, + /// Evidence sufficiency in `[0, 1]`. + pub evidence_sufficiency: f64, + /// Explicit test-time token budget. + pub compute_budget_tokens: u64, + /// Document-supplied override attempt, if any. + pub document_control: DocumentControlAttempt, + /// Whether deterministic scientific gates already passed. + pub scientific_gate_passed: bool, + /// TEPP-owned access profile identifiers. + pub access_list: Vec, +} + +/// Recorded orchestration plan. Construct only via [`route_orchestration`]. +#[derive(Clone, Debug, Eq, PartialEq)] +pub struct OrchestrationPlan { + mode: OrchestrationMode, + task_kind: InterpretationTaskKind, + stages: u8, + recursion_depth: u8, + decomposition_code: &'static str, + access_list: Vec, + roles: Vec, + token_budget: u64, + fallback_mode: OrchestrationMode, + policy_version: String, + proposal_only: bool, + scientific_authority_code: &'static str, +} + +impl OrchestrationPlan { + /// Selected orchestration mode. + #[must_use] + pub const fn mode(&self) -> OrchestrationMode { + self.mode + } + + /// Interpretation task whose context this plan represents. + #[must_use] + pub const fn task_kind(&self) -> InterpretationTaskKind { + self.task_kind + } + + /// Recorded workflow stage count. + #[must_use] + pub const fn stage_count(&self) -> u8 { + self.stages + } + + /// Recorded recursion depth. + #[must_use] + pub const fn recursion_depth(&self) -> u8 { + self.recursion_depth + } + + /// Decomposition code used for ablation. + #[must_use] + pub const fn decomposition_code(&self) -> &'static str { + self.decomposition_code + } + + /// TEPP-owned access list copied onto the plan. + #[must_use] + pub fn access_list(&self) -> &[String] { + &self.access_list + } + + /// Role assignments with per-role reasoning effort. + #[must_use] + pub fn roles(&self) -> &[RoleAssignment] { + &self.roles + } + + /// Allocated test-time budget in tokens. + #[must_use] + pub const fn token_budget(&self) -> u64 { + self.token_budget + } + + /// Bounded fallback mode if this plan cannot complete. + #[must_use] + pub const fn fallback_mode(&self) -> OrchestrationMode { + self.fallback_mode + } + + /// Policy version bound into the plan. + #[must_use] + pub fn policy_version(&self) -> &str { + &self.policy_version + } + + /// LLM output remains a proposal, never scientific authority. + #[must_use] + pub const fn proposal_only(&self) -> bool { + self.proposal_only + } + + /// Stable code naming the authoritative scientific gate family. + #[must_use] + pub const fn scientific_authority_code(&self) -> &'static str { + self.scientific_authority_code + } +} + +impl fmt::Display for OrchestrationPlan { + fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { + write!( + formatter, + "orchestration mode={} stages={} recursion={} proposal={}", + self.mode.wire_name(), + self.stages, + self.recursion_depth, + self.proposal_only + ) + } +} + +/// Comparable-budget ablation record. Direct is the required baseline. +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub struct BudgetAblationRecord { + baseline_mode: OrchestrationMode, + compared_mode: OrchestrationMode, + baseline_budget: u64, + compared_budget: u64, + comparable: bool, +} + +impl BudgetAblationRecord { + /// Required direct baseline mode. + #[must_use] + pub const fn baseline_mode(self) -> OrchestrationMode { + self.baseline_mode + } + + /// Mode compared against the direct baseline. + #[must_use] + pub const fn compared_mode(self) -> OrchestrationMode { + self.compared_mode + } + + /// Baseline token budget. + #[must_use] + pub const fn baseline_budget(self) -> u64 { + self.baseline_budget + } + + /// Compared token budget. + #[must_use] + pub const fn compared_budget(self) -> u64 { + self.compared_budget + } + + /// Whether the two budgets are within a 10 percent relative band. + #[must_use] + pub const fn comparable(self) -> bool { + self.comparable + } +} + +/// Credential-free binding TEPP may hand to contextual-orchestrator. +#[derive(Clone, Debug, Eq, PartialEq)] +pub struct ContextualOrchestratorBinding { + contract_version: u16, + mode: OrchestrationMode, + policy_version: String, + evidence_manifest_hash: String, + access_list: Vec, + roles: Vec, + token_budget: u64, + includes_credentials: bool, +} + +impl ContextualOrchestratorBinding { + /// Binding contract version. + #[must_use] + pub const fn contract_version(&self) -> u16 { + self.contract_version + } + + /// Mode the orchestrator may execute. + #[must_use] + pub const fn mode(&self) -> OrchestrationMode { + self.mode + } + + /// Policy version the orchestrator must not replace. + #[must_use] + pub fn policy_version(&self) -> &str { + &self.policy_version + } + + /// Evidence manifest digest; not raw source. + #[must_use] + pub fn evidence_manifest_hash(&self) -> &str { + &self.evidence_manifest_hash + } + + /// TEPP-owned access list. + #[must_use] + pub fn access_list(&self) -> &[String] { + &self.access_list + } + + /// Role assignments copied from the plan. + #[must_use] + pub fn roles(&self) -> &[RoleAssignment] { + &self.roles + } + + /// Allocated budget copied from the plan. + #[must_use] + pub const fn token_budget(&self) -> u64 { + self.token_budget + } + + /// Credentials are never included on this binding. + #[must_use] + pub const fn includes_credentials(&self) -> bool { + self.includes_credentials + } +} + +/// Route an interpretation task onto a versioned orchestration plan. +/// +/// The selected plan is a proposal. Deterministic statistical gates remain +/// authoritative. Documents cannot change policy, access lists, or credentials. +/// +/// # Errors +/// +/// Returns [`ApiError::UnsupportedContractVersion`] when the policy version is +/// not [`ORCHESTRATION_POLICY_VERSION`]. Returns +/// [`ApiError::AuthorizationDenied`] when a document supplied policy, access, +/// or credentials. Returns [`ApiError::InvalidWirePayload`] for non-unit +/// scores or empty access-list tokens. +pub fn route_orchestration(request: &OrchestrationRequest) -> Result { + validate_request(request)?; + let preferred = preferred_mode(request); + let minimum_mode = minimum_acceptable_mode(request.task_kind); + let mode = fit_mode(preferred, request.compute_budget_tokens, minimum_mode); + Ok(build_plan(request, mode)) +} + +/// Record a comparable-budget ablation against a required direct baseline. +/// +/// # Errors +/// +/// Returns [`ApiError::InvalidWirePayload`] when the baseline is not +/// [`OrchestrationMode::Direct`], or when the two plans disagree on task kind +/// or access list. Plans cannot disagree on policy version because the only +/// constructor, [`route_orchestration`], accepts the current version exactly. +pub fn record_budget_ablation( + baseline: &OrchestrationPlan, + compared: &OrchestrationPlan, +) -> Result { + if baseline.mode != OrchestrationMode::Direct + || baseline.task_kind != compared.task_kind + || baseline.access_list != compared.access_list + { + return Err(ApiError::InvalidWirePayload); + } + Ok(BudgetAblationRecord { + baseline_mode: baseline.mode, + compared_mode: compared.mode, + baseline_budget: baseline.token_budget, + compared_budget: compared.token_budget, + comparable: budgets_are_comparable(baseline.token_budget, compared.token_budget), + }) +} + +/// Bind a non-abstaining plan for contextual-orchestrator execution. +/// +/// The binding never includes credentials or raw source. TEPP retains +/// scientific authority; the orchestrator may only execute the recorded mode. +/// +/// # Errors +/// +/// Returns [`ApiError::AuthorizationDenied`] for [`OrchestrationMode::Abstain`]. +/// Returns [`ApiError::InvalidWirePayload`] unless the evidence manifest is a +/// canonical lowercase `sha256:` digest rather than raw source text. +pub fn bind_contextual_orchestrator( + plan: &OrchestrationPlan, + evidence_manifest_hash: &str, +) -> Result { + require_sha256_digest(evidence_manifest_hash)?; + if plan.mode == OrchestrationMode::Abstain { + return Err(ApiError::AuthorizationDenied); + } + Ok(ContextualOrchestratorBinding { + contract_version: ORCHESTRATION_CONTRACT_VERSION, + mode: plan.mode, + policy_version: plan.policy_version.clone(), + evidence_manifest_hash: evidence_manifest_hash.to_owned(), + access_list: plan.access_list.clone(), + roles: plan.roles.clone(), + token_budget: plan.token_budget, + includes_credentials: false, + }) +} + +fn require_sha256_digest(value: &str) -> Result<(), ApiError> { + let Some(digest) = value.strip_prefix("sha256:") else { + return Err(ApiError::InvalidWirePayload); + }; + if digest.len() == 64 + && digest + .bytes() + .all(|byte| byte.is_ascii_digit() || (b'a'..=b'f').contains(&byte)) + { + Ok(()) + } else { + Err(ApiError::InvalidWirePayload) + } +} + +fn validate_request(request: &OrchestrationRequest) -> Result<(), ApiError> { + if request.policy_version != ORCHESTRATION_POLICY_VERSION { + return Err(ApiError::UnsupportedContractVersion); + } + if request.document_control != DocumentControlAttempt::None { + return Err(ApiError::AuthorizationDenied); + } + if request.compute_budget_tokens > MAX_ORCHESTRATION_TOKEN_BUDGET + || request.access_list.len() > MAX_ORCHESTRATION_ACCESS_ENTRIES + { + return Err(ApiError::LimitExceeded); + } + require_unit_score(request.risk_score)?; + require_unit_score(request.ambiguity_score)?; + require_unit_score(request.evidence_sufficiency)?; + for (index, token) in request.access_list.iter().enumerate() { + require_nonempty(token)?; + if token.len() > MAX_ORCHESTRATION_ACCESS_TOKEN_BYTES { + return Err(ApiError::LimitExceeded); + } + if request.access_list[..index].contains(token) { + return Err(ApiError::InvalidWirePayload); + } + } + Ok(()) +} + +fn require_unit_score(value: f64) -> Result<(), ApiError> { + if value.is_finite() && (0.0..=1.0).contains(&value) { + Ok(()) + } else { + Err(ApiError::InvalidWirePayload) + } +} + +fn preferred_mode(request: &OrchestrationRequest) -> OrchestrationMode { + if request.evidence_sufficiency < EVIDENCE_FLOOR { + return OrchestrationMode::Abstain; + } + if request.task_kind == InterpretationTaskKind::BlindedModelReview + && !request.scientific_gate_passed + { + return OrchestrationMode::Abstain; + } + match request.task_kind { + InterpretationTaskKind::SchemaConversion => OrchestrationMode::Direct, + InterpretationTaskKind::BlindedModelReview => OrchestrationMode::Committee, + InterpretationTaskKind::AdversarialVerification => OrchestrationMode::Verify, + InterpretationTaskKind::SpanClassification => { + if request.risk_score < LOW_COMPLEXITY && request.ambiguity_score < LOW_COMPLEXITY { + OrchestrationMode::Direct + } else { + OrchestrationMode::Verify + } + } + InterpretationTaskKind::ConceptAlignment => { + if request.ambiguity_score >= HIGH_COMPLEXITY { + OrchestrationMode::Committee + } else { + OrchestrationMode::Verify + } + } + InterpretationTaskKind::NarrativeSynthesis => { + if request.risk_score >= HIGH_COMPLEXITY || request.ambiguity_score >= HIGH_COMPLEXITY { + OrchestrationMode::Conductor + } else { + OrchestrationMode::Verify + } + } + } +} + +const fn minimum_acceptable_mode(task_kind: InterpretationTaskKind) -> OrchestrationMode { + match task_kind { + InterpretationTaskKind::BlindedModelReview => OrchestrationMode::Committee, + InterpretationTaskKind::SpanClassification + | InterpretationTaskKind::ConceptAlignment + | InterpretationTaskKind::NarrativeSynthesis + | InterpretationTaskKind::AdversarialVerification + | InterpretationTaskKind::SchemaConversion => OrchestrationMode::Direct, + } +} + +fn fit_mode( + preferred: OrchestrationMode, + budget: u64, + minimum_mode: OrchestrationMode, +) -> OrchestrationMode { + let mut mode = preferred; + loop { + if budget >= mode.minimum_token_budget() { + return mode; + } + if mode == minimum_mode { + return OrchestrationMode::Abstain; + } + mode = mode.fallback_mode(); + } +} + +fn build_plan(request: &OrchestrationRequest, mode: OrchestrationMode) -> OrchestrationPlan { + let token_budget = if mode == OrchestrationMode::Abstain { + 0 + } else { + request.compute_budget_tokens + }; + OrchestrationPlan { + mode, + task_kind: request.task_kind, + stages: mode.stage_count(), + recursion_depth: mode.recursion_depth(), + decomposition_code: mode.decomposition_code(), + access_list: request.access_list.clone(), + roles: assign_roles(mode, request.task_kind), + token_budget, + fallback_mode: mode.fallback_mode(), + policy_version: request.policy_version.clone(), + proposal_only: true, + scientific_authority_code: "deterministic_statistical_gates", + } +} + +fn assign_roles(mode: OrchestrationMode, task: InterpretationTaskKind) -> Vec { + let worker = RoleAssignment { + role: OrchestrationRole::Worker, + effort: task.default_effort(), + }; + match mode { + OrchestrationMode::Direct => vec![worker], + OrchestrationMode::Verify => vec![ + worker, + RoleAssignment { + role: OrchestrationRole::Verifier, + effort: ReasoningEffort::High, + }, + ], + OrchestrationMode::Committee => vec![ + worker, + worker, + RoleAssignment { + role: OrchestrationRole::Adjudicator, + effort: ReasoningEffort::High, + }, + ], + OrchestrationMode::Conductor => vec![ + RoleAssignment { + role: OrchestrationRole::Conductor, + effort: ReasoningEffort::High, + }, + RoleAssignment { + role: OrchestrationRole::Thinker, + effort: task.default_effort(), + }, + worker, + RoleAssignment { + role: OrchestrationRole::Verifier, + effort: ReasoningEffort::High, + }, + ], + OrchestrationMode::Abstain => Vec::new(), + } +} + +fn budgets_are_comparable(left: u64, right: u64) -> bool { + // A saturated product is zero exactly when either budget is zero. Expressing + // the guard this way keeps the helper total without an unreachable arm in + // the public Direct-baseline call path. + left.saturating_mul(right) > 0 + && left + .abs_diff(right) + .saturating_mul(COMPARABLE_BUDGET_NUMERATOR) + <= left.max(right) +} + +#[cfg(test)] +mod tests { + use super::{ + DocumentControlAttempt, InterpretationTaskKind, ORCHESTRATION_CONTRACT_VERSION, + ORCHESTRATION_POLICY_VERSION, OrchestrationMode, OrchestrationRequest, OrchestrationRole, + ReasoningEffort, bind_contextual_orchestrator, budgets_are_comparable, + record_budget_ablation, route_orchestration, + }; + use crate::ApiError; + + fn request( + task_kind: InterpretationTaskKind, + risk: f64, + ambiguity: f64, + evidence: f64, + budget: u64, + ) -> OrchestrationRequest { + OrchestrationRequest { + policy_version: ORCHESTRATION_POLICY_VERSION.into(), + task_kind, + risk_score: risk, + ambiguity_score: ambiguity, + evidence_sufficiency: evidence, + compute_budget_tokens: budget, + document_control: DocumentControlAttempt::None, + scientific_gate_passed: true, + access_list: Vec::new(), + } + } + + #[test] + fn thresholds_and_fallbacks_cover_remaining_arms() { + let boundary = route_orchestration(&request( + InterpretationTaskKind::SpanClassification, + 0.35, + 0.10, + 0.35, + 8_000, + )) + .expect("boundary"); + assert_eq!(boundary.mode(), OrchestrationMode::Verify); + assert_eq!(boundary.fallback_mode(), OrchestrationMode::Direct); + assert_eq!( + OrchestrationMode::Committee.fallback_mode(), + OrchestrationMode::Verify + ); + assert_eq!( + OrchestrationMode::Abstain.fallback_mode(), + OrchestrationMode::Abstain + ); + assert_eq!(OrchestrationMode::Committee.recursion_depth(), 0); + assert_eq!(OrchestrationMode::Conductor.stage_count(), 4); + assert_eq!(OrchestrationMode::Direct.minimum_token_budget(), 4_000); + assert_eq!(OrchestrationMode::Verify.minimum_token_budget(), 8_000); + assert_eq!(OrchestrationMode::Committee.minimum_token_budget(), 16_000); + assert_eq!(OrchestrationMode::Conductor.minimum_token_budget(), 24_000); + assert_eq!(OrchestrationMode::Abstain.minimum_token_budget(), 0); + + let narrative_boundary = route_orchestration(&request( + InterpretationTaskKind::NarrativeSynthesis, + 0.50, + 0.10, + 0.90, + 24_000, + )) + .expect("narrative boundary"); + assert_eq!(narrative_boundary.mode(), OrchestrationMode::Conductor); + assert_eq!(narrative_boundary.stage_count(), 4); + + let committee_budget = route_orchestration(&request( + InterpretationTaskKind::NarrativeSynthesis, + 0.70, + 0.70, + 0.90, + 16_000, + )) + .expect("committee fit"); + assert_eq!(committee_budget.mode(), OrchestrationMode::Committee); + assert_eq!(committee_budget.fallback_mode(), OrchestrationMode::Verify); + assert_eq!(committee_budget.roles().len(), 3); + + let underfunded_blinded_review = route_orchestration(&request( + InterpretationTaskKind::BlindedModelReview, + 0.40, + 0.40, + 0.80, + 8_000, + )) + .expect("underfunded blinded review"); + assert_eq!( + underfunded_blinded_review.mode(), + OrchestrationMode::Abstain + ); + } + + #[test] + fn ablation_getters_and_rejection_paths_are_observable() { + assert!(!budgets_are_comparable(8_000, 32_000)); + assert!(budgets_are_comparable(16_000, 16_000)); + assert!(!budgets_are_comparable(0, 16_000)); + assert!(!budgets_are_comparable(16_000, 0)); + + let direct = route_orchestration(&request( + InterpretationTaskKind::SpanClassification, + 0.1, + 0.1, + 1.0, + 8_000, + )) + .expect("direct"); + let verify = route_orchestration(&request( + InterpretationTaskKind::SpanClassification, + 0.8, + 0.1, + 1.0, + 32_000, + )) + .expect("verify"); + let record = record_budget_ablation(&direct, &verify).expect("wide band"); + assert!(!record.comparable()); + assert_eq!(record.baseline_mode(), OrchestrationMode::Direct); + assert_eq!(record.compared_mode(), OrchestrationMode::Verify); + assert_eq!(record.baseline_budget(), 8_000); + assert_eq!(record.compared_budget(), 32_000); + assert!( + record_budget_ablation(&direct, &direct) + .expect("same-budget direct comparison") + .comparable() + ); + + let committee = route_orchestration(&request( + InterpretationTaskKind::BlindedModelReview, + 0.8, + 0.8, + 1.0, + 32_000, + )) + .expect("committee"); + assert_eq!( + record_budget_ablation(&committee, &direct), + Err(ApiError::InvalidWirePayload), + ); + assert_eq!( + record_budget_ablation(&direct, &committee), + Err(ApiError::InvalidWirePayload), + ); + let mut different_access = request( + InterpretationTaskKind::SpanClassification, + 0.8, + 0.1, + 1.0, + 32_000, + ); + different_access.access_list.push("evidence_spans".into()); + let different_access = route_orchestration(&different_access).expect("different access"); + assert_eq!( + record_budget_ablation(&direct, &different_access), + Err(ApiError::InvalidWirePayload), + ); + } + + #[test] + fn binding_getters_and_rejection_paths_are_observable() { + let direct = route_orchestration(&request( + InterpretationTaskKind::SpanClassification, + 0.1, + 0.1, + 1.0, + 8_000, + )) + .expect("direct"); + let verify = route_orchestration(&request( + InterpretationTaskKind::SpanClassification, + 0.8, + 0.1, + 1.0, + 32_000, + )) + .expect("verify"); + + let binding = bind_contextual_orchestrator( + &verify, + "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + ) + .expect("bind"); + bind_contextual_orchestrator( + &verify, + "sha256:0000000000000000000000000000000000000000000000000000000000000000", + ) + .expect("digit-only canonical digest"); + assert_eq!(binding.contract_version(), ORCHESTRATION_CONTRACT_VERSION); + assert_eq!(binding.roles().len(), 2); + assert_eq!(binding.token_budget(), 32_000); + assert!(binding.access_list().is_empty()); + assert!(!binding.includes_credentials()); + assert_eq!(binding.mode(), OrchestrationMode::Verify); + assert_eq!(binding.policy_version(), ORCHESTRATION_POLICY_VERSION); + assert_eq!( + binding.evidence_manifest_hash(), + "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" + ); + assert_eq!( + bind_contextual_orchestrator(&verify, "raw evidence"), + Err(ApiError::InvalidWirePayload), + ); + assert_eq!( + bind_contextual_orchestrator(&verify, "sha256:a"), + Err(ApiError::InvalidWirePayload), + ); + assert_eq!( + bind_contextual_orchestrator( + &verify, + "sha256:gggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggg", + ), + Err(ApiError::InvalidWirePayload), + ); + let abstain = route_orchestration(&request( + InterpretationTaskKind::SpanClassification, + 0.1, + 0.1, + 0.1, + 8_000, + )) + .expect("abstain"); + assert_eq!( + bind_contextual_orchestrator( + &abstain, + "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + ), + Err(ApiError::AuthorizationDenied), + ); + + let conductor = route_orchestration(&request( + InterpretationTaskKind::NarrativeSynthesis, + 0.8, + 0.8, + 1.0, + 32_000, + )) + .expect("conductor"); + let thinker = conductor + .roles() + .iter() + .find(|assignment| assignment.role() == OrchestrationRole::Thinker) + .expect("routed thinker"); + assert_eq!(thinker.role().wire_name(), "thinker"); + assert_eq!(thinker.effort(), ReasoningEffort::Medium); + assert_eq!(thinker.effort().wire_name(), "medium"); + assert_eq!( + direct.to_string(), + "orchestration mode=direct stages=1 recursion=0 proposal=true" + ); + } + + #[test] + fn preferred_modes_cover_concept_and_narrative_boundaries() { + let cases = [ + ( + InterpretationTaskKind::SpanClassification, + 0.10, + 0.40, + OrchestrationMode::Verify, + ), + ( + InterpretationTaskKind::ConceptAlignment, + 0.1, + 0.50, + OrchestrationMode::Committee, + ), + ( + InterpretationTaskKind::ConceptAlignment, + 0.1, + 0.49, + OrchestrationMode::Verify, + ), + ( + InterpretationTaskKind::NarrativeSynthesis, + 0.1, + 0.50, + OrchestrationMode::Conductor, + ), + ( + InterpretationTaskKind::NarrativeSynthesis, + 0.1, + 0.49, + OrchestrationMode::Verify, + ), + ]; + + for (task_kind, risk_score, ambiguity_score, expected_mode) in cases { + let plan = route_orchestration(&request( + task_kind, + risk_score, + ambiguity_score, + 1.0, + 32_000, + )) + .expect("boundary route"); + assert_eq!(plan.mode(), expected_mode); + } + } + + #[test] + fn empty_policy_and_negative_infinity_fail_closed() { + let mut empty = request( + InterpretationTaskKind::SpanClassification, + 0.1, + 0.1, + 0.9, + 8_000, + ); + empty.policy_version.clear(); + assert_eq!( + route_orchestration(&empty), + Err(ApiError::UnsupportedContractVersion) + ); + assert_eq!( + route_orchestration(&request( + InterpretationTaskKind::SpanClassification, + f64::NEG_INFINITY, + 0.1, + 0.9, + 8_000, + )), + Err(ApiError::InvalidWirePayload) + ); + } + + #[test] + fn finite_out_of_range_score_fails_closed() { + assert_eq!( + route_orchestration(&request( + InterpretationTaskKind::SpanClassification, + 1.01, + 0.1, + 0.9, + 8_000, + )), + Err(ApiError::InvalidWirePayload) + ); + } +} diff --git a/crates/tepp_api/tests/orchestration_resource_bounds_contract.rs b/crates/tepp_api/tests/orchestration_resource_bounds_contract.rs new file mode 100644 index 00000000..19efac2a --- /dev/null +++ b/crates/tepp_api/tests/orchestration_resource_bounds_contract.rs @@ -0,0 +1,68 @@ +//! Orchestration routing must bound billable compute and access-list resources. + +use tepp_api::{ + ApiError, DocumentControlAttempt, InterpretationTaskKind, MAX_ORCHESTRATION_ACCESS_ENTRIES, + MAX_ORCHESTRATION_ACCESS_TOKEN_BYTES, MAX_ORCHESTRATION_TOKEN_BUDGET, + ORCHESTRATION_POLICY_VERSION, OrchestrationMode, OrchestrationRequest, route_orchestration, +}; + +fn request() -> OrchestrationRequest { + OrchestrationRequest { + policy_version: ORCHESTRATION_POLICY_VERSION.into(), + task_kind: InterpretationTaskKind::SpanClassification, + risk_score: 0.1, + ambiguity_score: 0.1, + evidence_sufficiency: 1.0, + compute_budget_tokens: 4_000, + document_control: DocumentControlAttempt::None, + scientific_gate_passed: true, + access_list: vec!["evidence.read".into()], + } +} + +#[test] +fn token_budget_is_bounded_before_any_plan_is_created() { + let mut oversized = request(); + oversized.compute_budget_tokens = MAX_ORCHESTRATION_TOKEN_BUDGET + 1; + assert_eq!( + route_orchestration(&oversized), + Err(ApiError::LimitExceeded) + ); + + let mut boundary = request(); + boundary.compute_budget_tokens = MAX_ORCHESTRATION_TOKEN_BUDGET; + assert!(route_orchestration(&boundary).is_ok()); +} + +#[test] +fn access_list_cardinality_and_token_bytes_are_bounded() { + let mut too_many = request(); + too_many.access_list = (0..=MAX_ORCHESTRATION_ACCESS_ENTRIES) + .map(|index| format!("scope.{index}")) + .collect(); + assert_eq!(route_orchestration(&too_many), Err(ApiError::LimitExceeded)); + + let mut too_long = request(); + too_long.access_list = vec!["x".repeat(MAX_ORCHESTRATION_ACCESS_TOKEN_BYTES + 1)]; + assert_eq!(route_orchestration(&too_long), Err(ApiError::LimitExceeded)); +} + +#[test] +fn duplicate_access_tokens_fail_closed_instead_of_amplifying_authority() { + let mut duplicate = request(); + duplicate.access_list = vec!["evidence.read".into(), "evidence.read".into()]; + assert_eq!( + route_orchestration(&duplicate), + Err(ApiError::InvalidWirePayload) + ); +} + +#[test] +fn verify_mode_falls_back_to_direct_at_the_direct_budget_boundary() { + let mut budgeted = request(); + budgeted.risk_score = 0.35; + budgeted.compute_budget_tokens = OrchestrationMode::Direct.minimum_token_budget(); + + let plan = route_orchestration(&budgeted).expect("bounded request"); + assert_eq!(plan.mode(), OrchestrationMode::Direct); +} diff --git a/crates/tepp_api/tests/orchestration_router_contract.rs b/crates/tepp_api/tests/orchestration_router_contract.rs new file mode 100644 index 00000000..4950aad6 --- /dev/null +++ b/crates/tepp_api/tests/orchestration_router_contract.rs @@ -0,0 +1,463 @@ +//! Adaptive orchestration routing refuses document-controlled policy. + +use tepp_api::{ + ApiError, DocumentControlAttempt, InterpretationTaskKind, ORCHESTRATION_CONTRACT_VERSION, + ORCHESTRATION_POLICY_VERSION, OrchestrationMode, OrchestrationRequest, OrchestrationRole, + ReasoningEffort, bind_contextual_orchestrator, record_budget_ablation, route_orchestration, +}; + +fn request( + task_kind: InterpretationTaskKind, + risk: f64, + ambiguity: f64, + evidence: f64, + budget: u64, +) -> OrchestrationRequest { + OrchestrationRequest { + policy_version: ORCHESTRATION_POLICY_VERSION.into(), + task_kind, + risk_score: risk, + ambiguity_score: ambiguity, + evidence_sufficiency: evidence, + compute_budget_tokens: budget, + document_control: DocumentControlAttempt::None, + scientific_gate_passed: true, + access_list: vec!["evidence_spans".into()], + } +} + +#[test] +fn low_risk_span_classification_routes_direct() { + let plan = route_orchestration(&request( + InterpretationTaskKind::SpanClassification, + 0.10, + 0.10, + 0.90, + 8_000, + )) + .expect("direct"); + assert_eq!(plan.mode(), OrchestrationMode::Direct); + assert_eq!(plan.mode().wire_name(), "direct"); + assert_eq!(plan.stage_count(), 1); + assert_eq!(plan.recursion_depth(), 0); + assert_eq!(plan.decomposition_code(), "single_call"); + assert_eq!(plan.fallback_mode(), OrchestrationMode::Abstain); + assert!(plan.proposal_only()); + assert_eq!( + plan.scientific_authority_code(), + "deterministic_statistical_gates" + ); + assert_eq!(plan.policy_version(), ORCHESTRATION_POLICY_VERSION); + assert_eq!(plan.access_list(), ["evidence_spans"]); + assert_eq!(plan.roles().len(), 1); + assert_eq!(plan.roles()[0].role(), OrchestrationRole::Worker); + assert_eq!(plan.roles()[0].effort(), ReasoningEffort::Low); + assert_eq!(plan.token_budget(), 8_000); + assert!(!plan.to_string().contains("token")); +} + +#[test] +fn schema_conversion_is_direct_with_minimal_effort() { + let plan = route_orchestration(&request( + InterpretationTaskKind::SchemaConversion, + 0.90, + 0.90, + 0.80, + 4_000, + )) + .expect("schema"); + assert_eq!(plan.mode(), OrchestrationMode::Direct); + assert_eq!(plan.roles()[0].effort(), ReasoningEffort::Minimal); + assert_eq!( + InterpretationTaskKind::SchemaConversion.default_effort(), + ReasoningEffort::Minimal + ); +} + +#[test] +fn material_risk_or_adversarial_work_routes_verify() { + let high_risk = route_orchestration(&request( + InterpretationTaskKind::SpanClassification, + 0.70, + 0.20, + 0.80, + 12_000, + )) + .expect("high risk"); + assert_eq!(high_risk.mode(), OrchestrationMode::Verify); + assert_eq!(high_risk.decomposition_code(), "producer_then_verifier"); + assert_eq!(high_risk.stage_count(), 2); + assert_eq!(high_risk.roles().len(), 2); + assert_eq!(high_risk.roles()[1].role(), OrchestrationRole::Verifier); + assert_eq!(high_risk.roles()[1].effort(), ReasoningEffort::High); + assert_eq!(high_risk.fallback_mode(), OrchestrationMode::Direct); + + let adversarial = route_orchestration(&request( + InterpretationTaskKind::AdversarialVerification, + 0.20, + 0.20, + 0.80, + 12_000, + )) + .expect("adversarial"); + assert_eq!(adversarial.mode(), OrchestrationMode::Verify); + assert_eq!( + InterpretationTaskKind::AdversarialVerification.default_effort(), + ReasoningEffort::High + ); +} + +#[test] +fn concept_alignment_and_low_narrative_use_verify_or_committee() { + let aligned = route_orchestration(&request( + InterpretationTaskKind::ConceptAlignment, + 0.20, + 0.20, + 0.80, + 16_000, + )) + .expect("concept low"); + assert_eq!(aligned.mode(), OrchestrationMode::Verify); + assert_eq!( + InterpretationTaskKind::ConceptAlignment.default_effort(), + ReasoningEffort::Medium + ); + + let ambiguous = route_orchestration(&request( + InterpretationTaskKind::ConceptAlignment, + 0.20, + 0.70, + 0.80, + 16_000, + )) + .expect("concept high"); + assert_eq!(ambiguous.mode(), OrchestrationMode::Committee); + assert_eq!( + ambiguous.decomposition_code(), + "blinded_parallel_then_adjudicate" + ); + assert_eq!(ambiguous.stage_count(), 3); + assert!( + ambiguous + .roles() + .iter() + .any(|role| role.role() == OrchestrationRole::Adjudicator) + ); + + let narrative_low = route_orchestration(&request( + InterpretationTaskKind::NarrativeSynthesis, + 0.20, + 0.20, + 0.80, + 16_000, + )) + .expect("narrative low"); + assert_eq!(narrative_low.mode(), OrchestrationMode::Verify); +} + +#[test] +fn blinded_review_requires_scientific_gate_and_uses_committee() { + let passed = route_orchestration(&request( + InterpretationTaskKind::BlindedModelReview, + 0.40, + 0.40, + 0.80, + 20_000, + )) + .expect("committee"); + assert_eq!(passed.mode(), OrchestrationMode::Committee); + assert_eq!( + InterpretationTaskKind::BlindedModelReview.default_effort(), + ReasoningEffort::High + ); + + let mut rejected = request( + InterpretationTaskKind::BlindedModelReview, + 0.40, + 0.40, + 0.80, + 20_000, + ); + rejected.scientific_gate_passed = false; + let abstain = route_orchestration(&rejected).expect("llm cannot rescue"); + assert_eq!(abstain.mode(), OrchestrationMode::Abstain); + assert_eq!(abstain.decomposition_code(), "no_forced_answer"); + assert!(abstain.roles().is_empty()); + assert_eq!(abstain.token_budget(), 0); +} + +#[test] +fn complex_synthesis_routes_conductor_when_budget_allows() { + let plan = route_orchestration(&request( + InterpretationTaskKind::NarrativeSynthesis, + 0.60, + 0.70, + 0.85, + 32_000, + )) + .expect("conductor"); + assert_eq!(plan.mode(), OrchestrationMode::Conductor); + assert_eq!(plan.mode().wire_name(), "conductor"); + assert_eq!(plan.decomposition_code(), "adaptive_roles_under_budget"); + assert_eq!(plan.recursion_depth(), 2); + assert!( + plan.roles() + .iter() + .any(|role| role.role() == OrchestrationRole::Conductor) + ); + assert_eq!(plan.fallback_mode(), OrchestrationMode::Committee); +} + +#[test] +fn insufficient_evidence_and_tiny_budget_abstain() { + let evidence = route_orchestration(&request( + InterpretationTaskKind::NarrativeSynthesis, + 0.80, + 0.80, + 0.20, + 32_000, + )) + .expect("evidence"); + assert_eq!(evidence.mode(), OrchestrationMode::Abstain); + assert_eq!(evidence.mode().wire_name(), "abstain"); + + let budget = route_orchestration(&request( + InterpretationTaskKind::SpanClassification, + 0.10, + 0.10, + 0.90, + 100, + )) + .expect("tiny budget"); + assert_eq!(budget.mode(), OrchestrationMode::Abstain); +} + +#[test] +fn budget_steps_down_without_changing_scientific_authority() { + let stepped = route_orchestration(&request( + InterpretationTaskKind::NarrativeSynthesis, + 0.70, + 0.70, + 0.90, + 10_000, + )) + .expect("step down"); + assert_eq!(stepped.mode(), OrchestrationMode::Verify); + assert_eq!( + stepped.scientific_authority_code(), + "deterministic_statistical_gates" + ); + assert!(stepped.proposal_only()); +} + +#[test] +fn document_controlled_policy_access_or_credentials_are_denied() { + for attempt in [ + DocumentControlAttempt::Policy, + DocumentControlAttempt::AccessList, + DocumentControlAttempt::Credentials, + ] { + let mut hostile = request( + InterpretationTaskKind::SpanClassification, + 0.10, + 0.10, + 0.90, + 8_000, + ); + hostile.document_control = attempt; + assert_eq!( + route_orchestration(&hostile), + Err(ApiError::AuthorizationDenied) + ); + } +} + +#[test] +fn invalid_scores_policy_version_and_access_tokens_fail_closed() { + let mut version = request( + InterpretationTaskKind::SpanClassification, + 0.10, + 0.10, + 0.90, + 8_000, + ); + version.policy_version = "tepp.orchestration.v0".into(); + assert_eq!( + route_orchestration(&version), + Err(ApiError::UnsupportedContractVersion) + ); + + for (risk, ambiguity, evidence) in [ + (f64::NAN, 0.1, 0.9), + (0.1, f64::INFINITY, 0.9), + (0.1, 0.1, -0.01), + (1.01, 0.1, 0.9), + ] { + assert_eq!( + route_orchestration(&request( + InterpretationTaskKind::SpanClassification, + risk, + ambiguity, + evidence, + 8_000, + )), + Err(ApiError::InvalidWirePayload) + ); + } + + let mut empty_token = request( + InterpretationTaskKind::SpanClassification, + 0.10, + 0.10, + 0.90, + 8_000, + ); + empty_token.access_list = vec![" ".into()]; + assert_eq!( + route_orchestration(&empty_token), + Err(ApiError::InvalidWirePayload) + ); +} + +#[test] +fn comparable_budget_ablation_requires_direct_baseline() { + let direct = route_orchestration(&request( + InterpretationTaskKind::SpanClassification, + 0.10, + 0.10, + 0.90, + 16_000, + )) + .expect("direct"); + let verify = route_orchestration(&request( + InterpretationTaskKind::SpanClassification, + 0.80, + 0.20, + 0.80, + 16_000, + )) + .expect("verify"); + let record = record_budget_ablation(&direct, &verify).expect("ablation"); + assert_eq!(record.baseline_mode(), OrchestrationMode::Direct); + assert_eq!(record.compared_mode(), OrchestrationMode::Verify); + assert_eq!(record.baseline_budget(), 16_000); + assert_eq!(record.compared_budget(), 16_000); + assert!(record.comparable()); + + let conductor = route_orchestration(&request( + InterpretationTaskKind::NarrativeSynthesis, + 0.70, + 0.70, + 0.90, + 32_000, + )) + .expect("conductor"); + assert_eq!( + record_budget_ablation(&conductor, &direct), + Err(ApiError::InvalidWirePayload) + ); + + let abstain = route_orchestration(&request( + InterpretationTaskKind::SpanClassification, + 0.10, + 0.10, + 0.10, + 32_000, + )) + .expect("abstain"); + let not_comparable = record_budget_ablation(&direct, &abstain).expect("report difference"); + assert!(!not_comparable.comparable()); +} + +#[test] +fn contextual_orchestrator_binding_never_carries_credentials() { + let plan = route_orchestration(&request( + InterpretationTaskKind::ConceptAlignment, + 0.20, + 0.20, + 0.80, + 16_000, + )) + .expect("verify"); + let binding = bind_contextual_orchestrator( + &plan, + "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + ) + .expect("bind"); + assert_eq!(binding.contract_version(), ORCHESTRATION_CONTRACT_VERSION); + assert_eq!(binding.mode(), OrchestrationMode::Verify); + assert_eq!( + binding.evidence_manifest_hash(), + "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" + ); + assert!(!binding.includes_credentials()); + assert_eq!(binding.access_list(), plan.access_list()); + assert_eq!(binding.policy_version(), ORCHESTRATION_POLICY_VERSION); + + let abstain = route_orchestration(&request( + InterpretationTaskKind::NarrativeSynthesis, + 0.80, + 0.80, + 0.10, + 8_000, + )) + .expect("abstain"); + assert_eq!( + bind_contextual_orchestrator( + &abstain, + "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" + ), + Err(ApiError::AuthorizationDenied) + ); + assert_eq!( + bind_contextual_orchestrator(&plan, ""), + Err(ApiError::InvalidWirePayload) + ); +} + +#[test] +fn wire_names_cover_every_public_variant() { + assert_eq!(OrchestrationMode::Verify.wire_name(), "verify"); + assert_eq!(OrchestrationMode::Committee.wire_name(), "committee"); + assert_eq!(ReasoningEffort::Low.wire_name(), "low"); + assert_eq!(ReasoningEffort::Medium.wire_name(), "medium"); + assert_eq!(ReasoningEffort::High.wire_name(), "high"); + assert_eq!(ReasoningEffort::Minimal.wire_name(), "minimal"); + assert_eq!( + InterpretationTaskKind::SpanClassification.wire_name(), + "span_classification" + ); + assert_eq!( + InterpretationTaskKind::ConceptAlignment.wire_name(), + "concept_alignment" + ); + assert_eq!( + InterpretationTaskKind::BlindedModelReview.wire_name(), + "blinded_model_review" + ); + assert_eq!( + InterpretationTaskKind::NarrativeSynthesis.wire_name(), + "narrative_synthesis" + ); + assert_eq!( + InterpretationTaskKind::AdversarialVerification.wire_name(), + "adversarial_verification" + ); + assert_eq!( + InterpretationTaskKind::SchemaConversion.wire_name(), + "schema_conversion" + ); + assert_eq!(OrchestrationRole::Thinker.wire_name(), "thinker"); + assert_eq!(OrchestrationRole::Worker.wire_name(), "worker"); + assert_eq!(OrchestrationRole::Verifier.wire_name(), "verifier"); + assert_eq!(OrchestrationRole::Adjudicator.wire_name(), "adjudicator"); + assert_eq!(OrchestrationRole::Conductor.wire_name(), "conductor"); + assert_eq!( + InterpretationTaskKind::SpanClassification.default_effort(), + ReasoningEffort::Low + ); + assert_eq!( + InterpretationTaskKind::NarrativeSynthesis.default_effort(), + ReasoningEffort::Medium + ); +} diff --git a/crates/tepp_api/tests/orchestration_security_contract.rs b/crates/tepp_api/tests/orchestration_security_contract.rs new file mode 100644 index 00000000..c42d2d0f --- /dev/null +++ b/crates/tepp_api/tests/orchestration_security_contract.rs @@ -0,0 +1,156 @@ +//! Security and scientific-comparability contracts for adaptive orchestration. + +use tepp_api::{ + ApiError, DocumentControlAttempt, InterpretationTaskKind, ORCHESTRATION_POLICY_VERSION, + OrchestrationRequest, bind_contextual_orchestrator, record_budget_ablation, + route_orchestration, +}; + +fn request( + task_kind: InterpretationTaskKind, + risk_score: f64, + access_list: &[&str], +) -> OrchestrationRequest { + OrchestrationRequest { + policy_version: ORCHESTRATION_POLICY_VERSION.into(), + task_kind, + risk_score, + ambiguity_score: 0.10, + evidence_sufficiency: 0.90, + compute_budget_tokens: 16_000, + document_control: DocumentControlAttempt::None, + scientific_gate_passed: true, + access_list: access_list.iter().map(|value| (*value).into()).collect(), + } +} + +#[test] +fn contextual_orchestrator_binding_requires_a_canonical_sha256_manifest_digest() { + let plan = route_orchestration(&request( + InterpretationTaskKind::SpanClassification, + 0.10, + &["evidence_spans"], + )) + .expect("direct plan"); + let digest = format!("sha256:{}", "a".repeat(64)); + bind_contextual_orchestrator(&plan, &digest).expect("canonical digest"); + let digit_digest = format!("sha256:{}", "0".repeat(64)); + bind_contextual_orchestrator(&plan, &digit_digest).expect("digit-only canonical digest"); + + for invalid_digest in [ + "customer@example.com said to export everything", + "sha256:abc", + "sha256:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", + "sha256:gggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggg", + " sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + ] { + assert_eq!( + bind_contextual_orchestrator(&plan, invalid_digest), + Err(ApiError::InvalidWirePayload), + "raw source or noncanonical digest must fail closed: {invalid_digest}", + ); + } + for digest_length in [63, 65] { + let invalid_digest = format!("sha256:{}", "a".repeat(digest_length)); + assert_eq!( + bind_contextual_orchestrator(&plan, &invalid_digest), + Err(ApiError::InvalidWirePayload), + "adjacent digest length must fail closed: {digest_length}", + ); + } +} + +#[test] +fn compound_routing_thresholds_exercise_each_operand() { + let span_ambiguity_only = OrchestrationRequest { + ambiguity_score: 0.40, + ..request( + InterpretationTaskKind::SpanClassification, + 0.10, + &["evidence_spans"], + ) + }; + assert_eq!( + route_orchestration(&span_ambiguity_only) + .expect("ambiguity-only span route") + .mode() + .wire_name(), + "verify", + ); + + let narrative_ambiguity_only = OrchestrationRequest { + ambiguity_score: 0.50, + compute_budget_tokens: 24_000, + ..request( + InterpretationTaskKind::NarrativeSynthesis, + 0.10, + &["evidence_spans"], + ) + }; + assert_eq!( + route_orchestration(&narrative_ambiguity_only) + .expect("ambiguity-only narrative route") + .mode() + .wire_name(), + "conductor", + ); + + let narrative_risk_only = OrchestrationRequest { + compute_budget_tokens: 24_000, + ..request( + InterpretationTaskKind::NarrativeSynthesis, + 0.50, + &["evidence_spans"], + ) + }; + assert_eq!( + route_orchestration(&narrative_risk_only) + .expect("risk-only narrative route") + .mode() + .wire_name(), + "conductor", + ); +} + +#[test] +fn budget_ablation_requires_the_same_task_and_access_context() { + let baseline = route_orchestration(&request( + InterpretationTaskKind::SpanClassification, + 0.10, + &["evidence_spans"], + )) + .expect("direct baseline"); + assert_eq!( + baseline.task_kind(), + InterpretationTaskKind::SpanClassification + ); + let comparable = route_orchestration(&request( + InterpretationTaskKind::SpanClassification, + 0.80, + &["evidence_spans"], + )) + .expect("verify comparison"); + record_budget_ablation(&baseline, &comparable).expect("same-context ablation"); + + let different_task = route_orchestration(&request( + InterpretationTaskKind::AdversarialVerification, + 0.10, + &["evidence_spans"], + )) + .expect("different task"); + assert_eq!( + record_budget_ablation(&baseline, &different_task), + Err(ApiError::InvalidWirePayload), + ); + + let different_access = route_orchestration(&request( + InterpretationTaskKind::SpanClassification, + 0.80, + &["identity_mappings"], + )) + .expect("different access"); + assert_eq!( + record_budget_ablation(&baseline, &different_access), + Err(ApiError::InvalidWirePayload), + ); +} diff --git a/docs/API_CONTRACT.md b/docs/API_CONTRACT.md index 09b7998d..95a1b64f 100644 --- a/docs/API_CONTRACT.md +++ b/docs/API_CONTRACT.md @@ -1,7 +1,7 @@ # TEPP API and Modular Integration Contract **Status:** Accepted target contract; exact endpoints are introduced only with executable services. -**Last reviewed:** 2026-08-10 +**Last reviewed:** 2026-08-13 ## 1. Authority boundary @@ -18,7 +18,7 @@ Current protected main exposes Rust library/domain contracts, not a production H | interval relation/reasoner API | `temporal_core` | event/relation validation | active-PR #6 | | event/relation/membership API | future TEPP crates/services | naruon, analytics, UI | accepted-target | | semantic/topic measurement API | future TEPP measurement service | naruon, batch jobs, visual analytics | accepted-target | -| LLM interpretation provider port | TEPP interpretation gateway | contextual-orchestrator | accepted-target | +| LLM interpretation provider port | `tepp_api` orchestration router + future HTTP gateway | contextual-orchestrator | partial | | model/artifact/export API | `tepp_api` export envelopes + future HTTP service | standalone UI/CWL consumers | partial | | analysis-run request/accepted contracts | `tepp_api` v1 wire DTOs | naruon, orchestrator, UI | active-PR | @@ -118,7 +118,7 @@ Before any naruon, contextual-orchestrator, or NVIDIA NIM submission, callers mu ### contextual-orchestrator -TEPP may call a provider-neutral interpretation/orchestration port for semantic unitization, blinded model review, and evidence-bounded interpretation. The orchestrator does not own TEPP's statistical truth, source evidence, model registry, merge/release authority, or scientific acceptance. Detailed port boundary and credential separation are recorded in [`docs/connectors/contextual-orchestrator-interpretation-port.md`](connectors/contextual-orchestrator-interpretation-port.md). +TEPP may call a provider-neutral interpretation/orchestration port for semantic unitization, blinded model review, and evidence-bounded interpretation. Callers first obtain a plan from `tepp_api::route_orchestration` and may bind it with `tepp_api::bind_contextual_orchestrator` using an evidence-manifest digest. The orchestrator does not own TEPP's statistical truth, source evidence, model registry, merge/release authority, or scientific acceptance. Detailed port boundary and credential separation are recorded in [`docs/connectors/contextual-orchestrator-interpretation-port.md`](connectors/contextual-orchestrator-interpretation-port.md). ### organization `.github` diff --git a/docs/DOCUMENTATION_ASSESSMENT.md b/docs/DOCUMENTATION_ASSESSMENT.md index 2a14a651..ccbfb967 100644 --- a/docs/DOCUMENTATION_ASSESSMENT.md +++ b/docs/DOCUMENTATION_ASSESSMENT.md @@ -36,7 +36,7 @@ Decision status and implementation maturity are separate. ADR `Accepted` means t | Compliance/assurance readiness | PRESENT-CURRENT | `docs/COMPLIANCE_READINESS.md`; CSAP/SOC 2/ISO/NIST mappings without certification claims | | Test / scientific validation | PRESENT-CURRENT | `docs/TEST_STRATEGY.md`; true-parameter recovery, uncertainty, leakage, invariance, CPU/GPU parity and replacement-lineage evidence rules | | Operability / recovery / release | PRESENT-CURRENT | `docs/OPERABILITY.md`; ADR 0014 separates release authority from green CI | -| LLM orchestration / test-time compute | PRESENT-CURRENT | `docs/LLM_ORCHESTRATION.md` + ADR 0010; Fugu/Conductor/TRINITY motivate tested allocation, not authority | +| LLM orchestration / test-time compute | PRESENT-CURRENT | `docs/LLM_ORCHESTRATION.md` + ADR 0010 + `tepp_api` router doctoring; Fugu/Conductor/TRINITY motivate tested allocation, not authority | | Autonomous development/review/merge authority | PRESENT-CURRENT as design | ADR 0015 separates model proposal, deterministic verification, publication, independent review, and merge/release authority; implementation remains accepted-target | | Standards / APA 7 doctoring | PRESENT-CURRENT | research register covers psychometrics/topic/time/event/Unicode/security/AI-governance/orchestration foundations | | Traceability | PRESENT-CURRENT | `docs/TRACEABILITY.md` maps requirements, owning ADRs, canonical replacement lineage, and maturity | diff --git a/docs/LLM_ORCHESTRATION.md b/docs/LLM_ORCHESTRATION.md index 42af070c..98102aa5 100644 --- a/docs/LLM_ORCHESTRATION.md +++ b/docs/LLM_ORCHESTRATION.md @@ -1,7 +1,7 @@ # TEPP LLM Orchestration and Test-Time Compute Contract -**Status:** Accepted target orchestration baseline; production implementation is not yet shipped. -**Last reviewed:** 2026-08-10 +**Status:** Partial — `tepp_api::route_orchestration` is the governed selector; live provider execution is not yet shipped. +**Last reviewed:** 2026-08-13 ## 1. Purpose @@ -29,7 +29,7 @@ These results motivate experiments; they do not prove that deeper orchestration | conductor | complex evidence synthesis or multi-stage semantic reasoning | adaptive roles/topology under explicit budget | | abstain | provider/evidence/validation insufficient | no forced answer | -A router chooses the cheapest mode expected to satisfy the quality/risk profile, but latency is not the primary objective. Quality, evidence support, calibration, disagreement, controllability, and reproducibility dominate. +`tepp_api::route_orchestration` is the governed selector. It chooses the cheapest mode expected to satisfy the quality/risk profile, but latency is not the primary objective. Quality, evidence support, calibration, disagreement, controllability, and reproducibility dominate. The returned plan is a proposal: `scientific_authority_code` remains `deterministic_statistical_gates`. ## 4. Explicit experimental variables diff --git a/docs/TRACEABILITY.md b/docs/TRACEABILITY.md index dfcdd9e8..f6739641 100644 --- a/docs/TRACEABILITY.md +++ b/docs/TRACEABILITY.md @@ -31,16 +31,16 @@ The full APA 7th standards/literature register remains `docs/research/standards- | posterior ESEM / longitudinal invariance / DSEM | ADR 0005 | future `psychometric_core` | accepted-target | | CPU bounded multithreading + GPU/VRAM streaming/parity | ADR 0001/0006 | future `compute_backend` | accepted-target | | TDT detection/tracking vs CHRONOS schema/prediction/temporal consistency | ADR 0016; PRD/research | future `event_intelligence` | accepted-target | -| evidence-bounded LLM interpretation | ADR 0010/0012; PRD | future `interpretation_gateway` | accepted-target | -| adaptive direct/verify/committee/conductor test-time compute | ADR 0010; `docs/LLM_ORCHESTRATION.md` | future contextual-orchestrator integration + ablation evidence | accepted-target | -| purpose-bound PII handling without blanket masking | ADR 0009; `docs/PRIVACY_DATA_GOVERNANCE.md` | `tepp_api` export authorization plus provider-payload minimization / elevated re-identification on the active PR; persistence retention/deletion remaining | partial | -| tenant/purpose/role/lifetime access and identity separation | ADR 0009; Threat Model | `tepp_api` time-bounded `PurposeGrant` + cross-tenant denial on the active PR; persistent `access_grant` storage remaining | partial | +| evidence-bounded LLM interpretation | ADR 0010/0012; PRD | `tepp_api` router plus future `interpretation_gateway` | partial | +| adaptive direct/verify/committee/conductor test-time compute | ADR 0010; `docs/LLM_ORCHESTRATION.md` | `tepp_api::route_orchestration` + ablation record on the active PR; live contextual-orchestrator execution remaining | partial | +| purpose-bound PII handling without blanket masking | ADR 0009; `docs/PRIVACY_DATA_GOVERNANCE.md` | `tepp_api` export authorization plus provider-payload minimization / elevated re-identification implemented-main; persistence retention/deletion remaining | partial | +| tenant/purpose/role/lifetime access and identity separation | ADR 0009; Threat Model | `tepp_api` time-bounded `PurposeGrant` + cross-tenant denial implemented-main; persistent `access_grant` storage remaining | partial | | standalone + modular CWL MSA / no cross-service DB coupling | ADR 0011; `docs/API_CONTRACT.md` | current standalone crates; future service ports | partial | | naruon modular artifact consumer boundary | ADR 0011/0012; API contract | `docs/connectors/naruon-artifact-consumer.md` + PR #22 versioned consumer contract on protected main; `tepp_api` HTTP interchange (active PR); live HTTP service remaining | partial | | contextual-orchestrator interpretation port boundary | ADR 0010/0011; LLM orchestration | `docs/connectors/contextual-orchestrator-interpretation-port.md`; live port remaining | partial | | Actions registry identities bound to protected-main tree (orphan disable) | Operability; GitHub Actions REST | `scripts/actions_workflow_fleet.py` + issue #20 tests/doctoring; live disable remains operator-authorized | active-PR | | autonomous model proposal separated from verification/publication/review/merge | ADR 0015 | future safe OpenCode/NVIDIA autonomous-development workflow | accepted-target | -| contextual-orchestrator execution boundary | ADR 0010/0011 | provider-neutral orchestration port; TEPP retains scientific authority | accepted-target | +| contextual-orchestrator execution boundary | ADR 0010/0011 | credential-free `bind_contextual_orchestrator` on the active PR; live HTTP remaining | partial | | foundation validation / release-readiness ledger | ADR 0014; Test Strategy | PR #24 `docs/validation/temporal-event-foundation.md` on protected main | implemented-main | | scientific claim promotion separated from design/implementation/release | ADR 0014; ADR policy | documentation/CI/domain validation/release evidence | partial | | CSAP/SOC 2/ISO/NIST assurance readiness | `docs/COMPLIANCE_READINESS.md`; research register | repository controls + future deployment evidence | accepted-target / deployment-owned | diff --git a/docs/adr/0010-adaptive-llm-orchestration.md b/docs/adr/0010-adaptive-llm-orchestration.md index 20ae9ae0..d04bd6bc 100644 --- a/docs/adr/0010-adaptive-llm-orchestration.md +++ b/docs/adr/0010-adaptive-llm-orchestration.md @@ -1,7 +1,7 @@ # ADR 0010 — Adaptive LLM orchestration and test-time compute **Decision status:** Accepted -**Implementation maturity:** accepted-target +**Implementation maturity:** partial — `tepp_api` governed router, comparable-budget ablation record, and credential-free contextual-orchestrator binding are implemented on the active PR and are not implemented-main until exact-head checks, review, and protected-main integration complete; live NIM execution, learned conductor calibration, and production ablation evidence remain accepted-target **Date:** 2026-08-10 **Supersedes:** The LLM orchestration-selection/ablation clauses previously co-located in ADR 0006. ADR 0006 remains authoritative for GPU/VRAM and model-credential separation; ADR 0015 governs autonomous repository-write/review/merge authority. diff --git a/docs/adr/README.md b/docs/adr/README.md index f16c2345..258eb7f3 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -14,8 +14,8 @@ Read [`ADR_POLICY.md`](ADR_POLICY.md) first. **Decision status and implementatio | [0006](0006-vram-gpu-nvidia-orchestration.md) | VRAM-adaptive GPU compute and model-credential boundary | Accepted | accepted-target | LLM orchestration policy superseded by ADR 0010; autonomous development authority governed by ADR 0015. | | [0007](0007-rust-workspace-quality-gates.md) | Explicit Rust workspace, pinned toolchains, and exact quality gates | Accepted | implemented-main | ADR 0014 governs scientific/product claim promotion beyond repository-quality tooling. | | [0008](0008-immutable-evidence-identities-digests-and-spans.md) | Immutable evidence identities, `SHA-256` digests, exact spans, and strict wire reconstruction | Accepted | implemented-main | ADR 0013 governs future persistence/reproducibility/split authority. | -| [0009](0009-purpose-bound-pii-governance.md) | Purpose-bound PII governance without blanket masking | Accepted | partial | Persistence retention/deletion/legal-hold (`0007`) implemented-main; provider-payload minimization and elevated re-identification are on the active PR; deployment evidence remains accepted-target. | -| [0010](0010-adaptive-llm-orchestration.md) | Adaptive LLM orchestration and test-time compute | Accepted | accepted-target | Owns direct/verify/committee/conductor selection, budget, role/topology, and ablation policy. | +| [0009](0009-purpose-bound-pii-governance.md) | Purpose-bound PII governance without blanket masking | Accepted | partial | Persistence retention/deletion/legal-hold (`0007`) and provider-payload minimization implemented-main; deployment evidence remains accepted-target. | +| [0010](0010-adaptive-llm-orchestration.md) | Adaptive LLM orchestration and test-time compute | Accepted | partial | `tepp_api` router/ablation/orchestrator binding on the active PR; live NIM execution and production ablation evidence remain accepted-target. | | [0011](0011-standalone-modular-msa-boundary.md) | Standalone operation and modular CWL MSA boundary | Accepted | partial | Owns cross-service persistence/credential/API authority; no direct cross-service application-table coupling. | | [0012](0012-temporal-relational-shared-latent-topic-measurement.md) | Temporal Relational Shared-Latent Topic Measurement (TRSL-TM) | Accepted | accepted-target | Owns topic backend compatibility, global topic identity, method effects, K/model-selection prerequisites, and compositional topic coordinates. | | [0013](0013-bitemporal-persistence-reproducibility-and-split-authority.md) | Bitemporal persistence, reproducibility manifests, and relation-aware split authority | Accepted | partial | Owns PostgreSQL adapter semantics, immutable run/split manifests, leakage-safe partitions, and recovery identity; optional `live-sqlx` `PgPool`, live PG CI, tenant RLS, and `0006` membership implemented-main; `0007` retention/deletion/legal-hold on the active PR; remaining physical ERD/backup accepted-target. | diff --git a/docs/connectors/contextual-orchestrator-interpretation-port.md b/docs/connectors/contextual-orchestrator-interpretation-port.md index 62889fc2..377c91c3 100644 --- a/docs/connectors/contextual-orchestrator-interpretation-port.md +++ b/docs/connectors/contextual-orchestrator-interpretation-port.md @@ -1,7 +1,7 @@ # contextual-orchestrator interpretation port for TEPP -**Status:** Accepted-target modular integration contract -**Last reviewed:** 2026-08-12 +**Status:** Partial modular integration contract — `tepp_api::bind_contextual_orchestrator` is the credential-free binding; live HTTP remains accepted-target. +**Last reviewed:** 2026-08-13 ## Boundary @@ -16,12 +16,12 @@ LLM/provider settings are execution policy only. Deterministic scientific gates ## Allowed orchestration modes -TEPP may allocate test-time computation between: +`tepp_api::route_orchestration` selects `direct`, `verify`, `committee`, `conductor`, or `abstain` from TEPP-owned risk, ambiguity, evidence-sufficiency, and token-budget scores. `tepp_api::bind_contextual_orchestrator` may be called only for a non-abstaining plan and never includes credentials or raw source. TEPP may allocate test-time computation between: 1. direct model routing with bounded reasoning effort; 2. deeper multi-agent workflows with recorded workflow depth, decomposition, access lists, recursion, role-specific reasoning effort, verification/adjudication, and comparable-budget ablations. -These allocations are guided by Fugu, Conductor, and TRINITY research cited in `docs/research/standards-and-literature.md` and `docs/LLM_ORCHESTRATION.md`. +These allocations are guided by Fugu, Conductor, and TRINITY research cited in `docs/research/standards-and-literature.md` and `docs/LLM_ORCHESTRATION.md`. Documents cannot change policy, access lists, or credentials. ## Credential separation diff --git a/docs/research/adaptive-orchestration-router.md b/docs/research/adaptive-orchestration-router.md new file mode 100644 index 00000000..58c4a54e --- /dev/null +++ b/docs/research/adaptive-orchestration-router.md @@ -0,0 +1,47 @@ +# Adaptive orchestration router and comparable-budget ablation + +## Scope + +This note documents the `tepp_api` governed router that implements the first executable slice of ADR 0010 without a database migration and without live model I/O: + +1. `route_orchestration` selects versioned modes `direct`, `verify`, `committee`, `conductor`, or `abstain` from CPU `f64` unit-interval risk, ambiguity, and evidence-sufficiency scores plus an explicit token budget; +2. the plan records workflow stage count, recursion depth, decomposition, TEPP-owned access lists, and role-specific reasoning effort; +3. documents cannot change policy, access lists, or credentials (`DocumentControlAttempt`); +4. blinded model review that failed a deterministic scientific gate abstains — LLM preference cannot rescue a statistically rejected candidate; +5. `record_budget_ablation` requires a `direct` baseline and the same task kind, policy version, and access list before it reports whether the compared budget is within a 10 percent relative band; +6. `bind_contextual_orchestrator` emits a credential-free execution binding, accepts only a canonical lowercase `sha256:` evidence-manifest digest, and refuses abstention. + +Live NVIDIA NIM HTTP, learned conductor calibration, and production-quality claims remain accepted-target. + +## Authoritative sources + +Xu, J., Sun, Q., Schwendeman, P., Nielsen, S., Cetin, E., & Tang, Y. (2026). *TRINITY: An evolved LLM coordinator* [Preprint]. arXiv. https://arxiv.org/abs/2512.04695 + +Nielsen, S., Cetin, E., Schwendeman, P., Sun, Q., Xu, J., & Tang, Y. (2026). *Learning to orchestrate agents in natural language with the Conductor* [Preprint]. arXiv. https://arxiv.org/abs/2512.04388 + +Tang, Y., Cetin, E., Xu, J., Sun, Q., Nielsen, S., Richard, V., Goda, H., Tymchenko, I., Nguyen, N., Lee, H., Ashiga, M., Kotyan, S., Kuroki, S., & Clanuwat, T. (2026). *Sakana Fugu technical report* [Preprint]. arXiv. https://arxiv.org/abs/2606.21228 + +## Application + +TRINITY motivates lightweight model/role delegation rather than a fixed deep graph (Xu et al., 2026). Conductor motivates recording topology, instructions, and recursive test-time scaling as explicit variables (Nielsen et al., 2026). Fugu frames production orchestration as a query-adaptive scaffold behind a model-compatible boundary (Tang et al., 2026). TEPP therefore treats the router as a deterministic policy object: deeper modes must justify themselves against a direct baseline at a comparable budget, and LLM output remains an untrusted proposal under deterministic statistical gates. These citations are experimental motivation, not authority to replace TEPP estimands. + +## Verification + +- low-risk span classification and schema conversion route `direct`; +- material risk, adversarial verification, and low-ambiguity concept/narrative work route `verify`; +- high-ambiguity concept alignment routes `committee`; gated blinded review routes `committee` only with its full minimum budget and otherwise abstains; +- high-complexity narrative synthesis routes `conductor` when the budget allows and steps down otherwise; +- insufficient evidence, failed scientific gates, and sub-minimum budgets abstain; +- document-controlled policy, access, or credentials are denied; +- non-unit scores and unknown policy versions fail closed; +- ablation rejects a non-direct baseline or a comparison that changes task kind, policy version, or access list, and reports incomparable zero or wide-band budgets; +- orchestrator bindings omit credentials, reject raw source and malformed evidence-manifest values, and refuse abstention. + +## Resource-authority boundary + +A request may allocate at most 1,000,000 tokens, 64 TEPP-owned access +capabilities, and 128 UTF-8 bytes per capability token. Duplicate +capabilities fail closed instead of amplifying or obscuring authority. +These are admission limits, not a promise that the full budget will be +consumed; downstream providers remain subject to stricter policy and cost +controls. diff --git a/docs/research/standards-and-literature.md b/docs/research/standards-and-literature.md index 6e0438fe..bfda7a79 100644 --- a/docs/research/standards-and-literature.md +++ b/docs/research/standards-and-literature.md @@ -160,4 +160,4 @@ Nielsen, S., Cetin, E., Schwendeman, P., Sun, Q., Xu, J., & Tang, Y. (2026). Lea Tang, Y., Cetin, E., Xu, J., Sun, Q., Nielsen, S., Richard, V., Goda, H., Tymchenko, I., Nguyen, N., Lee, H., Ashiga, M., Kotyan, S., Kuroki, S., & Clanuwat, T. (2026). *Sakana Fugu technical report* [Preprint]. arXiv. https://arxiv.org/abs/2606.21228 -TRINITY motivates lightweight learned model/role delegation over multiple turns; Conductor motivates query-adaptive natural-language workflow/topology/instruction generation and recursive test-time scaling; Fugu demonstrates a production-oriented family of query-adaptive agentic scaffolds building on these research lines (Xu et al., 2026; Nielsen et al., 2026; Tang et al., 2026). TEPP therefore treats direct routing, verification, fixed multi-agent workflows, adaptive orchestration, stage count, decomposition, recursion, access lists, role-specific reasoning effort, and total test-time budget as explicit experimental variables. Deeper/more-agent orchestration is never assumed better by default. Comparable-budget ablation, evidence support, calibration, disagreement, safety, cost, and failure behavior are required before a production claim. +TRINITY motivates lightweight learned model/role delegation over multiple turns; Conductor motivates query-adaptive natural-language workflow/topology/instruction generation and recursive test-time scaling; Fugu demonstrates a production-oriented family of query-adaptive agentic scaffolds building on these research lines (Xu et al., 2026; Nielsen et al., 2026; Tang et al., 2026). TEPP therefore treats direct routing, verification, fixed multi-agent workflows, adaptive orchestration, stage count, decomposition, recursion, access lists, role-specific reasoning effort, and total test-time budget as explicit experimental variables. `tepp_api::route_orchestration` is the deterministic selector for those variables; live provider execution and production-quality claims remain later work. Deeper/more-agent orchestration is never assumed better by default. Comparable-budget ablation, evidence support, calibration, disagreement, safety, cost, and failure behavior are required before a production claim. See `docs/research/adaptive-orchestration-router.md`. diff --git a/docs/validation/temporal-event-foundation.md b/docs/validation/temporal-event-foundation.md index 7c50db23..aae1a06e 100644 --- a/docs/validation/temporal-event-foundation.md +++ b/docs/validation/temporal-event-foundation.md @@ -23,7 +23,8 @@ This report tracks exact-head scientific and engineering evidence required befor | Truth corpora / manifests | `tepp_simulation` | implemented-main | — | deterministic generator tests | Task 10 / PR #18 | | Recovery metrics | `validation_core` | implemented-main | — | RMSE/bias/coverage/MC gates | Task 11 / PR #19 | | Versioned API/export contracts | `tepp_api` | implemented-main | naruon HTTP interchange | unknown-field/version/limit + naruon HTTPS interchange tests | Task 12 / PR #21; live HTTP service remaining | -| Purpose-bound provider payloads | `tepp_api` | active-PR | provider-payload minimization | expired/not-yet-valid/inverted/cross-tenant/impossible-calendar grant, mapping refusal, audited elevated re-id replay | ADR 0009; `docs/research/provider-payload-minimization.md` | +| Purpose-bound provider payloads | `tepp_api` | implemented-main | provider-payload minimization | expired/not-yet-valid/inverted/cross-tenant/impossible-calendar grant, mapping refusal, audited elevated re-id replay | ADR 0009; `docs/research/provider-payload-minimization.md` | +| Adaptive orchestration router | `tepp_api` | accepted-target | active PR | mode selection, document-control denial, ablation, credential-free bind | ADR 0010; `docs/research/adaptive-orchestration-router.md` | | CWL modular connectors | `docs/connectors/*` | implemented-main | — | contract docs + examples | PR #22; live HTTP ports remaining | | Release SBOM/provenance generator | `scripts/release_evidence.py` | partial | — | generate+validate in CI | Task 13 partial / PR #28 |