Agent/verified distillation study 05

CHANGELOG.md

@@ -1,5 +1,18 @@
 # Changelog
 
+## 0.4.0a0 — unreleased
+
+- Complete bounded deterministic skeptic discovery for verifier gaps, disagreements,
+  abstentions, missing counterfactuals, learned-provider disagreement, and verifier
+  health holes. Candidates remain proposal-only and carry visible evidence lineage.
+- Add source-preserving distillation groups, lineage-bound principles and strategies,
+  explicit negative memory, frozen transfer predictions/evaluations, study-arm contracts,
+  deterministic ablations, retrieval, and calibration metrics.
+- Add fail-closed development/transfer visibility access and a CPU transition-frequency
+  learned-provider artifact with deterministic serialization, reload, OOD abstention, and
+  diagnostic-only observations.
+- Add the no-network `mnel distill-reference` study and distillation record schema.
+
 ## 0.3.0a0 — unreleased
 
 - Add bounded binary views and an identity-keyed shared store for transition, pair,

README.md

@@ -37,15 +37,18 @@ experience, negative memory, causal attribution, transfer-gated principles, reus
 strategies, and append-only candidate lineage rather than relying exclusively on
 conventional neural-weight training.
 
-> **Current status:** functional `0.3.0a0` iteration. The repository now includes a
+> **Current status:** functional `0.4.0a0` study iteration. The repository now includes a
 > backend-neutral accelerator placement policy, optional Torch/Accelerate adapter,
 > process-local persistent Rust host, reusable identity-bound snapshots, bounded and
 > normalized diagnostic results, failure quarantine, an executable Rust HMM baseline,
 > deterministic runtime measurements, and bounded investigator context/workspace
 > contracts, an executable local-harness/worktree path, and a validated Rust v1 dynamic
 > provider loader, and an executable bounded Forge-oriented diagnostic lifecycle with
 > compact snapshot views, a verifier registry, reference probes, witnesses, mutations,
-> comparison, health, coverage, and omitted-question candidates. It still does not provide process isolation,
+> comparison, health, coverage, and deterministic skeptic candidates. It also includes
+> source-preserving distillation groups, lineage-bound strategies, negative-memory
+> retrieval, equal-budget study controls, retrieval/calibration metrics, and a tiny
+> reloadable CPU transition-frequency diagnostic provider. It still does not provide process isolation,
 > unattended model execution, distributed scheduling, protected final custody, formal
 > MNCS/MNCDS conformance, or automatic RAVEL promotion.
 
@@ -115,6 +118,14 @@ copy their authority or silently create substitute implementations.
   quarantine state, coverage records, learned-provider observation normalization, and
   proposal-only omitted-question candidates;
 - deterministic reference workflow, JSON schemas, mutation-oriented tests, and CI.
+- bounded skeptic-driven omitted-question discovery with evidence lineage, duplicate
+  suppression, coverage/health/disagreement gaps, and proposal-only authority;
+- source-preserving distillation groups, provisional principles/strategies, transfer
+  prediction freezing, negative-memory-aware retrieval, deterministic study arms and
+  ablations, and retrieval/calibration metrics;
+- `mnel distill-reference`, a no-network held-out study that writes an evidence ledger
+  and exercises controls, hidden-transfer access, strategy transfer, and a tiny
+  reloadable transition-frequency learned provider. Its observations remain diagnostic.
 
 ## Install
 
@@ -188,6 +199,18 @@ candidate:
 mnel forge-reference --workspace build/forge-reference
 ```
 
+Run the deterministic 0.4 distillation study:
+
+```bash
+mnel distill-reference --workspace build/distill-reference
+mnel ledger verify build/distill-reference/distill-evidence.jsonl
+```
+
+The study preserves development records, prevents development code from reading hidden
+or future-final records, freezes the transfer prediction before held-out evaluation,
+and reports measurements rather than a success verdict. Its reference feature grouping
+is an explicit bounded baseline, not semantic understanding.
+
 Verify and summarize the resulting ledger:
 
 ```bash

docs/ARCHITECTURE.md

@@ -114,6 +114,30 @@ and inconclusive experience. Records are content-identified and chained in order
 Learned observations retain declaration, weight, feature, query, snapshot, calibration,
 and limitation identities before they are eligible for later study.
 
+### Verified distillation and study plane
+
+The 0.4 data plane makes reuse explicit without replacing evidence:
+
+```text
+development-visible episodes
+  -> attribution-linked source-preserving group
+  -> provisional principle/strategy with counterexamples
+  -> negative-memory-aware retrieval
+  -> frozen transfer prediction
+  -> hidden transfer evidence
+  -> retained/quarantined/rejected strategy lineage
+```
+
+`mnel.distillation` exposes explicit development and transfer access views. Development
+study code cannot retrieve `TRANSFER_HIDDEN` or `FUTURE_FINAL` records, and a strategy
+cannot be repaired from its own hidden-transfer result. Groups, principles, strategies,
+and negative memory retain source identities; heuristic grouping is labeled as a
+reference feature baseline rather than semantic truth. Retrieval returns record classes
+and match reasons separately, while calibration and transfer status remain measurements.
+Study arms and ablations carry equal-budget declarations and deterministic identities.
+The `distill-reference` command is a synthetic, no-network study surface, not an
+evaluator or a substitute for Forge, Fabric, MNCS, MNCDS, or RAVEL.
+
 ### Distillation plane
 
 VED proposes compact principles and strategies while preserving source lineage,

docs/INTEGRATIONS.md

@@ -37,6 +37,13 @@ The external `mncs-forge-mcp` checkout is optional. The MNEL adapter contract is
 identity-bound and provider-neutral; no developer-local Forge path is a runtime
 dependency, and no hidden network or model service is invoked by the reference study.
 
+The 0.4 distillation study consumes the same identified diagnostic plane but remains an
+MNEL-side research harness. Its groups, strategies, retrieval results, learned-provider
+observations, and transfer records are append-only measurements. It does not turn a
+retrieval hit, calibrated score, witness, or synthetic transfer result into Forge/MNCS
+authority. A future external Forge adapter must supply its own identified evaluator and
+keep hidden-transfer access outside development study code.
+
 ## MNCS Fabric
 
 Fabric distributes identified experiment bundles, captures node capabilities, and

docs/LEARNED_PROVIDER_RUNTIME.md

@@ -163,6 +163,15 @@ the same identified payload can be reused by a deterministic micro-verifier and
 learned provider without reparsing ad hoc JSON. Composite snapshots reference component
 identities rather than duplicating their payloads.
 
+The 0.4 reference study also demonstrates the control-plane side of a small learned
+provider in `mnel.reference_provider`: a transition-frequency artifact is trained only
+from a development access view, binds its dataset/record/feature/training/calibration
+identities, serializes with an artifact digest, reloads deterministically, and emits a
+bounded observation with abstention/OOD flags. This is an `external-experimental`
+diagnostic provider. It is not a verifier and is not exported through ABI v1; native
+export remains a later task once the artifact contract has a measured compatible runtime
+boundary.
+
 ## Native-language exceptions
 
 A non-Rust provider may enter `native-trusted` only when its manifest includes:

docs/LEARNING_MODEL.md

@@ -67,3 +67,19 @@ A credible MNEL study should include:
 - hidden transfer evaluation.
 
 A better final candidate alone does not prove the learning process caused the gain.
+
+## Executable 0.4 foundation
+
+The current deterministic study path preserves raw development episodes and builds
+identity-bound feature groups, attribution-linked provisional strategies, and explicit
+negative memory. Retrieval keeps episodic, causal, semantic, procedural, and negative
+records as separate classes; a negative-memory conflict can demote a strategy without
+deleting its positive lineage. Transfer predictions are frozen before hidden evaluation,
+and hidden results cannot repair the same candidate.
+
+`mnel distill-reference` exercises A0, A4, and A5 declarations plus aggregate-only,
+shuffled-attribution, random, success-memory, negative-memory, fixed-policy,
+equal-budget, and hidden-transfer controls. It computes retrieval and calibration
+measurements and trains a tiny transition-frequency diagnostic provider that serializes
+and reloads deterministically. These are measurements from a bounded synthetic fixture,
+not evidence of general learning improvement or evaluator authority.

docs/RESEARCH_PROTOCOL.md

@@ -35,3 +35,19 @@ development environments, selection material, and hidden transfer environment.
 
 The study must preserve every failed candidate and must not use transfer or final
 observations for same-candidate repair.
+
+## Machine-readable study contracts
+
+MNEL now represents each arm as an identity-bound `mnel-study-arm/0.4` record with
+allowed/forbidden information, retrieval and memory modes, attribution/strategy
+availability, recursion allowance, and an equal operation/wall-time/candidate budget.
+Control transformations are separate `mnel-ablation-spec/0.4` records. Shuffled
+attribution uses a recorded seed and preserves the original attribution records; memory
+ablations change only the eligible view. The reference study can therefore reproduce
+the same control identities without deleting source evidence or reading hidden transfer
+outcomes during development.
+
+The current 0.4 reference study is deliberately synthetic and deterministic. Its
+transition-frequency provider is a diagnostic observation source with explicit artifact,
+training dataset, feature extractor, calibration, and reload identities. It is not a
+verifier, evaluator, or promotion mechanism.

docs/ROADMAP.md

@@ -45,7 +45,9 @@
 - **Implemented:** bounded counterfactual and registered mutation probe support;
 - **Implemented:** independent-probe comparison preserving agreement, disagreement, and incomplete coverage;
 - **Implemented:** verifier health, quarantine, and coverage records;
-- **Started:** deterministic skeptic-driven omitted-question candidate discovery;
+- **Implemented:** deterministic skeptic-driven omitted-question candidate discovery with
+  bounded, deduplicated, lineage-checked candidates for coverage holes, disagreements,
+  abstentions, missing mutation counterparts, and verifier health gaps;
 - **Implemented:** identity-bound transition, tabular, pair, trace, graph, and composite diagnostic snapshots with
   immutable compact binary payloads, producer/source/dependency/extractor identities, and
   deterministic content identities suitable for deterministic probes and learned
@@ -55,14 +57,23 @@
 
 ## 0.4 — verified distillation and learned-provider studies
 
-- semantic clustering without source deletion;
-- attribution and transfer workflows;
-- success- and negative-memory ablations;
-- shuffled-attribution and aggregate-only controls;
-- strategy retrieval and calibration metrics;
-- train and calibrate the initial heterogeneous learned-provider portfolio;
+- **Implemented:** source-preserving deterministic reference grouping with explicit
+  extractor/method identities and limitations;
+- **Implemented:** attribution-linked provisional principles and strategies, negative
+  memory, frozen transfer predictions, held-out transfer evidence, and same-candidate
+  hidden-repair rejection;
+- **Implemented:** success- and negative-memory, shuffled-attribution, aggregate-only,
+  random, fixed-policy, equal-budget, and hidden-transfer control specifications;
+- **Implemented:** explicit class-preserving retrieval with negative-memory demotion and
+  precision/recall/hit-rate/reuse/diversity metrics plus bounded calibration metrics;
+- **Implemented:** deterministic `mnel distill-reference` study and a tiny reloadable CPU
+  transition-frequency learned provider with OOD abstention and diagnostic-only output;
+- **Started:** heterogeneous learned-provider portfolio training and calibration;
 - export Python-trained providers into the versioned native runtime boundary;
-- compare every learned provider against deterministic and classical baselines;
+- **Started:** compare learned providers against deterministic and classical baselines;
+  the reference transition study records deterministic and heuristic controls; the
+  existing Rust HMM baseline is not input-compatible with this compact fixture and is
+  retained as a separate native runtime baseline;
 - random, heuristic, single-provider, and diversity-routed controls;
 - correlated-error, disagreement, abstention, and out-of-distribution studies;
 - useful confirmed probes per operation, latency, memory, energy, and cold-start metrics;

docs/THREAT_MODEL.md

@@ -71,6 +71,16 @@ callbacks and in-place authoritative snapshot mutation are not accepted. Learned
 provider observations and verifier witnesses remain distinct diagnostic records, and
 neither can authorize conformance or promotion.
 
+The 0.4 study layer adds an explicit visibility gate: development access rejects hidden
+transfer and future-final records, clustering/training/retrieval operate on an eligible
+view, and transfer prediction is frozen before a separate transfer-evaluator view can
+read the held-out result. Same-candidate repair from that result is rejected. Shuffled
+attribution and memory ablations create new study identities while preserving their
+source records. Deterministic feature grouping is bounded and explicitly labeled as a
+heuristic limitation, so it cannot masquerade as semantic understanding. Distillation,
+retrieval, calibration, and learned-provider observations remain proposal or diagnostic
+records; none includes evaluator verdict authority.
+
 ### Apparent independence
 
 Multiple local machines run the same operator-controlled stack. This is replication,