006 local ml tier

.github/workflows/ci.yml

@@ -4,6 +4,10 @@ on:
   push:
     branches: [main]
   pull_request:
+  workflow_dispatch:
+
+permissions:
+  contents: read
 
 env:
   CARGO_TERM_COLOR: always
@@ -21,40 +25,54 @@ jobs:
       - run: cargo fmt --all --check
 
   clippy:
-    name: lint
+    name: lint (${{ matrix.configuration }})
     runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - configuration: workspace
+            args: --workspace
+          - configuration: offline-cli
+            args: -p please-cli --no-default-features
+          - configuration: candle
+            args: -p please-ml --features candle
+          - configuration: ml-cli
+            args: -p please-cli --features ml-candle
+          - configuration: offline-ml-cli
+            args: -p please-cli --no-default-features --features ml-candle
     steps:
       - uses: actions/checkout@v4
       - uses: dtolnay/rust-toolchain@stable
         with:
           components: clippy
       - uses: Swatinem/rust-cache@v2
-      - run: cargo clippy --workspace --all-targets -- -D warnings
-      # Feature 004 made `plz` a two-configuration build. A lint job that only ever sees the default one
-      # never reads a line of the judgement tier's CLI wiring, so `-D warnings` would be enforced on half
-      # the code that ships.
-      - run: cargo clippy --workspace --all-targets --features please-cli/judge -- -D warnings
+      - run: cargo clippy ${{ matrix.args }} --all-targets --locked -- -D warnings
 
   test:
-    name: test
+    name: test (${{ matrix.configuration }})
     runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - configuration: workspace
+            args: --workspace
+          - configuration: offline-cli
+            args: -p please-cli --no-default-features
+          - configuration: candle
+            args: -p please-ml --features candle
+          - configuration: ml-cli
+            args: -p please-cli --features ml-candle
+          - configuration: offline-ml-cli
+            args: -p please-cli --no-default-features --features ml-candle
     steps:
       - uses: actions/checkout@v4
       - uses: dtolnay/rust-toolchain@stable
       - uses: Swatinem/rust-cache@v2
-      # `--no-fail-fast`, and it is not a style preference. Cargo stops after the first test *target*
-      # that fails, so while `please-core --test fixtures` is red — the false-positive rate is under
-      # SC-003's threshold with 17 of the required 200 benign cases — every target sorting after it is
-      # never executed: preparation, ruleset_load, sanitize, scaling, scan, score, seams,
-      # no_self_steering. Eight binaries, silently, including the FR-020 security properties.
-      #
-      # The failing suite still fails the job. The difference is that the run now says what else is
-      # broken instead of stopping at the first thing.
-      - run: cargo test --workspace --no-fail-fast
-      # Same reasoning as clippy above: the tier's own tests — the adversarial property test, the
-      # fail-closed suite, credential resolution — are only compiled when the feature is on. The
-      # offline ones are the majority and they are where the security properties live.
-      - run: cargo test --workspace --features please-cli/judge --no-fail-fast
+      # The workspace default already includes judge. Offline CLI and Candle are distinct builds.
+      # Real-weight tests are explicitly ignored here; ml-inference.yml verifies and runs them.
+      - run: cargo test ${{ matrix.args }} --locked --no-fail-fast
 
   # Principle V's embeddability claim is only worth what proves it. This job IS that proof: the core
   # must build for a target with no filesystem, no network, no threads, and no monotonic clock. It is
@@ -91,6 +109,8 @@ jobs:
       - run: cargo fmt --manifest-path crates/eval/Cargo.toml --check
       - run: cargo clippy --manifest-path crates/eval/Cargo.toml --all-targets -- -D warnings
       - run: cargo test --manifest-path crates/eval/Cargo.toml
+      - run: cargo test --manifest-path crates/eval/Cargo.toml --features shipping-judge,shipping-ml --test product --test boundary
+      - run: python3 -B crates/eval/scripts/test_prepare_dataset_holdout.py
       # `corpus/generated.jsonl` is a committed derived artifact, which is only safe if regenerating it
       # is byte-identical. This is that assertion: a diff means an input changed without the corpus being
       # regenerated, or the generator stopped being deterministic. Either way a human looks.
@@ -101,7 +121,7 @@ jobs:
       #
       # `--release`, because `repo_prose` scans every document in the tree and a debug build of the regex
       # engine makes that a minute rather than a second.
-      - run: cargo run --release --manifest-path crates/eval/Cargo.toml -- run --offline
+      - run: cargo run --release --manifest-path crates/eval/Cargo.toml -- run --offline --mode mechanism
       - run: cargo run --release --manifest-path crates/eval/Cargo.toml -- gate --offline
 
   # FR-106. The built-in fast path skips compiled resource validation at default limits, on the grounds
@@ -139,14 +159,9 @@ jobs:
       - uses: Swatinem/rust-cache@v2
       - run: ./ci/check-dependencies.sh
       - run: ./ci/check-core-isolation.sh
-      # `ci/check-cli-dependencies.sh` was here, asserting the default `plz` graph carried no HTTP or TLS
-      # crate. The judgement tier is now on by default, so that assertion is false by intent and the check
-      # is gone with it. The two above still cover `please-core`, which is where the no-network guarantee
-      # actually lives — and the wasm32 job proves it independently.
-      #
-      # A `--no-default-features` build of `plz` still carries no HTTP or TLS crate. Nothing checks that
-      # any more; it is now a property of the Cargo manifest rather than a gate.
-      - run: cargo build -p please-cli --no-default-features
+      - run: bash ci/check-cli-dependencies.sh
+      - run: bash ci/check-ml-isolation.sh
+      - run: cargo build -p please-cli --no-default-features --locked
 
   # SC-404 / FR-413. Its own job because it runs the whole suite with canary credentials in the
   # environment, which is not something to fold into a job people read as "lint".

.github/workflows/ml-inference.yml

@@ -0,0 +1,51 @@
+name: ML inference
+
+on:
+  workflow_dispatch:
+  push:
+    tags: ['v*']
+
+permissions:
+  contents: read
+
+jobs:
+  real-weights:
+    name: verified public models and real Candle inference
+    runs-on: ubuntu-latest
+    timeout-minutes: 45
+    env:
+      PLEASE_EVAL_CACHE: ${{ github.workspace }}/.cache/ci-models
+      HF_HUB_DISABLE_IMPLICIT_TOKEN: '1'
+    steps:
+      - uses: actions/checkout@v4
+      - uses: dtolnay/rust-toolchain@stable
+      - uses: actions/setup-python@v5
+        with:
+          python-version: '3.12'
+      - uses: Swatinem/rust-cache@v2
+      - uses: actions/cache@v4
+        id: models
+        with:
+          path: .cache/ci-models/models
+          key: public-inference-models-${{ hashFiles('crates/eval/corpus/models.toml') }}
+      - name: Install pinned download client
+        if: steps.models.outputs.cache-hit != 'true'
+        run: python -m pip install 'huggingface_hub==1.27.0'
+      - name: Fetch the two ungated model bundles
+        if: steps.models.outputs.cache-hit != 'true'
+        run: >-
+          cargo run --manifest-path crates/eval/Cargo.toml --locked --
+          model fetch protectai-deberta-v3-small all-minilm-l6-v2
+      - name: Cache Rust dependencies before the offline gate
+        run: |
+          cargo fetch --locked
+          cargo fetch --manifest-path crates/eval/Cargo.toml --locked
+      - name: Verify all asset hashes and execute inference offline
+        shell: bash
+        run: bash ci/check-ml-inference.sh 2>&1 | tee ml-inference.log
+      - uses: actions/upload-artifact@v4
+        if: always()
+        with:
+          name: ml-inference
+          path: ml-inference.log
+          if-no-files-found: warn

.github/workflows/release-quality.yml

@@ -0,0 +1,29 @@
+name: Release quality
+
+on:
+  workflow_dispatch:
+  push:
+    tags: ['v*']
+
+permissions:
+  contents: read
+
+jobs:
+  fixture-quality:
+    name: fixture release criteria (SC-002 and SC-003)
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: dtolnay/rust-toolchain@stable
+      - uses: Swatinem/rust-cache@v2
+      # These targets intentionally fail until detection and corpus-size requirements are met.
+      - name: Check absolute quality targets
+        shell: bash
+        run: |
+          cargo test -p please-core --test fixtures --locked -- --ignored --nocapture 2>&1 | tee fixture-quality.log
+      - uses: actions/upload-artifact@v4
+        if: always()
+        with:
+          name: fixture-quality
+          path: fixture-quality.log
+          if-no-files-found: error

CONTEXT.md

@@ -0,0 +1,70 @@
+# PLEASE
+
+PLEASE detects prompt-injection attempts in text reaching an AI agent and evaluates detection against named corpus slices. This glossary records domain terms used in the architecture discussions.
+
+## Language
+
+**Evaluation run**:
+A named evaluation with a fixed selection of intended slices, saved results, and the pipeline configuration that produced them. The intended slices are declared before scanning; a different selection requires a new run name.
+_Avoid_: Experiment (also used for separate model-feasibility work)
+
+**Slice**:
+A named collection of evaluation rows defined by a corpus query or a local corpus reader, with its own reporting and gate eligibility.
+_Avoid_: Dataset (a slice may cover only part of a dataset)
+
+**Incomplete evaluation run**:
+An evaluation run whose intended results are not all available and valid, including a run interrupted before completion or containing corrupt saved results.
+_Avoid_: Clean run (absence of usable results is not successful evaluation)
+
+**Partial report**:
+A report showing the available valid results of an incomplete evaluation run and explicitly identifying its incompleteness.
+_Avoid_: Successful evaluation (producing a report does not mean the gate passed)
+
+**Unverified evaluation run**:
+An evaluation run whose saved records cannot establish completeness, including older runs without a declared selection of intended slices. It cannot pass the gate and requires a rerun.
+_Avoid_: Complete legacy run (existing result files do not prove that all intended results were saved)
+
+**Gate**:
+The evaluation check that applies baseline and criterion rules and returns a pass or failure; completeness must be established before a run can pass. Incomplete and unverified evaluation runs fail it.
+_Avoid_: Report (a report presents results without establishing a pass)
+
+**Judge response envelope**:
+The provider's response containing completion metadata and content blocks, including the requested classification tool call.
+_Avoid_: Judge report (the bound report is constructed after response validation)
+
+**Judge response acceptance**:
+The checks establishing that a judge response envelope is complete and unambiguous before its tool input is interpreted against a captured request.
+_Avoid_: Judgement (acceptance checks do not determine what a finding means)
+
+**Rule-set acquisition**:
+Reading caller-supplied rule files in order, parsing each with its file attribution, and asking core
+preparation to build the built-in base plus those additions and disabled rule IDs. Acquisition returns
+one complete engine or an attributed error; it never continues with only the files that loaded.
+_Avoid_: Rule preparation (core's separate, filesystem-free validation and construction step)
+
+**Frame eligibility**:
+Whether a raw rule match satisfies its declared anchor in the bytes being searched. A frame-anchored
+match is eligible only when its start is a semantic-unit boundary according to the shared structure
+predicate. Ineligible matches enter neither findings nor suppressions.
+_Avoid_: Quoting suppression (a separate decision applied to an already eligible finding)
+
+**Frame-aware matching**:
+Matching that enforces frame eligibility within the searched buffer before returning rule matches.
+Direct and decoded buffers use their own coordinates; decoded evidence is attributed to its original
+encoded region only after matching.
+_Avoid_: Context review (the optional judge's interpretation is a separate operation)
+
+## Agreed behavior
+
+Agreed during the architecture discussion on 2026-09-11 and implemented by the evaluation run module in `crates/eval/src/run.rs`:
+
+- An incomplete evaluation run may produce a clearly marked partial report.
+- Its gate must fail even when every available slice meets its baseline.
+- A run declares its intended slices before scanning; that selection remains fixed.
+- Adding slices requires a new run name.
+- A run must prove completeness before its gate can pass; inability to prove it is a failure requiring a rerun.
+- Older saved runs without evidence of their intended selection are unverified and require a rerun; their existence does not grant a compatibility exception to the gate.
+
+The current implementation restarts interrupted evaluations under a fresh run name; automatic resume is not implemented. The module saves fixed slice definitions, publishes results atomically, and verifies row counts and checksums before the gate can pass.
+
+Completeness here concerns the integrity of saved evaluation results for the supplied input rows; input-corpus verification and detector coverage gaps within saved rows remain separate concerns.