feat(security): real ONNX inference via openai/privacy-filter

[graydawnc]

PR 3/7 — real ONNX inference via openai/privacy-filter + Settings runtime info

Stub-to-real handoff. Replaces the empty-array pf:analyze handler with a working transformers.js + ONNX Runtime Web pipeline running OpenAI's Privacy Filter model, pins the manifest to the matching HuggingFace commit, and surfaces the live runtime (WebGPU adapter label or "WASM (CPU)") in the Settings card.

Model selection — why openai/privacy-filter Q4

The space was canvassed before pinning. Six candidates considered:

Model	License	Params	ONNX/transformers.js	Code-aware	Verdict
openai/privacy-filter	Apache 2.0 ✓	1.5 B / 50 M active MoE	Pre-quantized Q4 published, transformers.js native	❌ NL only	✅ chosen
bigcode/starpii	"no redistribution"	110 M	–	✅ best	❌ license blocks shipping in OSS Electron app
iiiorg/piiranha-v1	CC-BY-NC-ND	280 M	–	❌	❌ NC-ND
Isotonic/distilbert_ai4privacy	CC-BY-NC	66 M	✅	❌	❌ NC
gretelai/gretel-gliner-bi-base	Apache 2.0 ✓	GLiNER base	–	❌	⚠️ no ONNX at time of pin
knowledgator/gliner-pii-base	Apache 2.0 ✓	197 MB UINT8 ONNX	✅	❌	⚠️ F1 lower (0.81 vs PF's claimed 0.96)

Decision drivers:

• License: Spool ships MIT/Apache OSS in an Electron bundle. Anything CC-BY-NC* or "no redistribution" was out.
• transformers.js compatibility: needed an ONNX variant with pre-baked tokenizer JSON. Skipped models that would need a custom build pipeline.
• PII class coverage: OpenAI PF emits 8 classes including secret — the category Spool's regex can miss (novel SaaS token formats). Other candidates skewed toward general PII.
• Pre-quantized: avoid hosting a quantization step. PF ships Q4 directly.

Quant tier choice (Q4 vs FP16 vs full)

Tier	Size	Quality	Latency	Pick
FP32	~3 GB	best	slowest	❌ download/inference cost
FP16	~1.5 GB	~99% of FP32	medium	⚠️ still large
Q4	945 MB	~94 % of FP32 on PII	fastest	✅ chosen

Q4 is the only tier that fits a "users will actually wait for this" download budget. OpenAI's eval shows class-level precision degradation is ≤2 pp for the categories we map (email/phone/DOB/secret).

Honest limitation: OOD on code content

Documented for future readers — PF's training set is natural-language PII. Spool sessions mix code blocks, tool output, npm package syntax. The published precision numbers apply to "PII only" content; OOD precision crashes (model card line_breaks 0.45, phonetic 0.27). PRs 6 & 7 follow up: PR 6 restricts class-mapping to the high-precision categories; PR 7 pulls the toggle into an Experimental section and ships regex-only as the security feature's default.

Inference pipeline

flowchart LR
  subgraph Hidden["Hidden BrowserWindow"]
    direction TB
    L["pipeline load<br/>transformers.js"]
    T["tokenize text"]
    R["ONNX inference<br/>BIOES tokens"]
    V["Viterbi span decode"]
    M["character offsets<br/>+ class + score"]
  end

  Req["pf-analyze-req<br/>text"] --> T
  T --> R
  R --> V
  V --> M
  M --> Res["pf-analyze-res<br/>matches array"]

  L -.once on boot.-> R

Loading

OpenAI Privacy Filter is a 1.5 B-parameter / 50 M-active MoE token classifier with 8 PII classes (private_person, private_email, private_phone, private_url, private_address, private_date, account_number, secret). Output is BIOES-tagged tokens with viterbi-decoded spans + a calibrated confidence score.

Custom protocol for model loading

The 917 MB ONNX weights live outside the Vite-served renderer and exceed protobuf's 2 GB limit (onnx_data side-file). Loading from disk requires a custom scheme that transformers.js will resolve:

flowchart LR
  TJS["transformers.js<br/>pretrained loader"] -->|fetch pf-model://onnx/model_q4.onnx_data| PROTO["pf-model:// handler"]
  PROTO --> FS["file stream from userData/models/"]
  FS -->|Response w/ streamed body| TJS

Loading

The protocol is privileged (bytesAvailable: true, secure: true, corsEnabled: true) so transformers.js's fetch() resolves it like a normal URL.

What's in this PR

Pinned model (model-manifest.ts)

• hfRepo: 'openai/privacy-filter', hfCommit: 7ffa9a0 (Apache 2.0)
• 6 files / 945 MB total — config.json, tokenizer.json, tokenizer_config.json, viterbi_calibration.json, onnx/model_q4.onnx, onnx/model_q4.onnx_data
• Real SHA-256 hashes for each file (was placeholder 0x0... in PR 1)

Inference renderer (pf-inference.ts)

• Adds @huggingface/transformers as dep
• Pipeline boot on first ready handshake
• BIOES → span decoding + viterbi calibration applied
• Confidence floor at 0.85 (suppresses model's noisy tail)

Custom protocol (pf-model-protocol.ts)

• registerPfModelScheme() at module top-level (before app.ready)
• registerPfModelProtocol() handler reads from pfModelDir(), streams large files via createReadStream (917 MB onnx_data would OOM if loaded into memory)

Settings runtime info chip

• New PF_GET_RUNTIME_INFO IPC channel + matching preload + renderer api passthrough
• Returns null when ModelHost isn't running so the card hides the badge without an extra "not running" branch
• PfDownloadCard: in installed phase, polls runtime info every 2 s and renders "WebGPU · Apple M2 Pro" / "WASM (CPU)" under the model footer
• Polling stops when pfEnabled is off

Class-mapping (initial)

• Adds mapPfMatches(raw, ctx) — maps PF's private_email / private_phone / etc. to Spool's SensitiveKind
• Returns null for kinds we don't trust (rule details refined in PR 6)

Test plan

• App suite: 271/271
• Manual verification on real model (procedure in commit message)
• No automated e2e for real ONNX — would require shipping the 945 MB bundle

Builds on PR 2 (engine wiring). Followed by PR 4 (in-page callout + activation flow).

🤖 Generated with Claude Code

[graydawnc]

• feat(security): hide PF UI entry points; move to opt-in Experimental section #298
• feat(security): restrict PF to high-precision PII categories #297
• feat(security): forward PF inference window console to main + dev devtools #296
• feat(security): in-page Privacy Filter discovery callout #295
• feat(security): real ONNX inference via openai/privacy-filter #294 👈 (View in Graphite)
• feat(security): wire ModelHost lifecycle + pf-analyze IPC bridge #293
• feat(security): hidden inference window skeleton for Privacy Filter ML #292
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.