轉錄幻覺過濾 Phase 2：confidence-gated (full) 模式

[kiki830621]

Problem / Motivation

Phase A (#98) shipped a backend-agnostic denylist hallucination filter (--hallucination-filter <off|denylist>). Denylist catches known-boilerplate strings but can't catch novel silent-segment hallucinations. Whisper emits per-segment signals — no_speech_prob and compression_ratio — that flag hallucinated segments generically (high no-speech probability, abnormal compression ratio from repetition), independent of the exact text.

Type

feature

Expected

A full mode (--hallucination-filter full) that, on top of the denylist, drops cues by confidence signals:

• no_speech_prob > threshold (silent-segment hallucination)
• compression_ratio > threshold (repetition hallucination)
• optionally avg_logprob < threshold

Blocked-by / Prerequisite

The segment model currently carries only confidence (WhisperKit's avgLogprob). no_speech_prob / compression_ratio are computed by WhisperKit but dropped at the engine boundary. full mode requires:

1. Plumbing these fields through RawSegment → TranscriptSegment.
2. Each engine populating them, with a per-backend fallback — whisper.cpp / Parakeet may not provide them, in which case full degrades to denylist for that backend.

Scope note

HallucinationFilterMode already exists (off | denylist); this adds a full case. The MVP deliberately shipped <off|denylist> only (no stub full) to avoid a misleading contract.

Source: deferred from #98 (Phase 2), filed at /idd-close time.

[kiki830621]

Diagnosis

Type

feature

Root Cause / Analysis

Requirements breakdown (the issue already carries the design; #98 built the seams):

1. Signal plumbing — RawTranscription.RawSegment and TranscriptSegment gain optional noSpeechProb: Double? and compressionRatio: Double? (nil-default → every existing engine/test compiles unchanged). WhisperKit's TranscriptionSegment already computes both (noSpeechProb, compressionRatio — confirmed in the 0.18 checkout during bug: ExternalEngineTests hung-adapter watchdog flakes on CI — 1-hour hang then wrong error message #91); WhisperKitEngine.transcribeRaw maps them through. Every other constructor site (engine seam normalization, diarization relabeling, the filter's own re-indexing) forwards the fields.
2. full mode — HallucinationFilterMode gains .full: everything denylist does, plus drop a cue when its confidence signals cross Whisper's canonical thresholds — the joint silence rule (noSpeechProb > 0.6 && avgLogprob < -1.0, mirroring openai-whisper's skip condition) or the repetition rule (compressionRatio > 2.4). Constants named and documented, not magic numbers.
3. Per-backend degradation for free — a segment with nil signals can never trip a threshold, so on whisper.cpp / Parakeet (which don't populate them) full behaves exactly like denylist with no per-backend switch. This IS the fallback the issue asks for; documented rather than coded as branching.
4. CLI already parses the mode by rawValue over CaseIterable — adding the case extends --hallucination-filter <off|denylist|full> automatically (verify help text mentions it).

Output-format note: the new fields stay filter-internal in v1 — SRT/VTT/txt/json emission is unchanged (no schema churn for downstream parsers).

Impact

• Sources/BestASRKit/Engines/Engine.swift (RawSegment + seam mapping), Models/DataModels.swift (TranscriptSegment), Engines/WhisperKitEngine.swift (populate), Output/HallucinationFilter.swift (.full + thresholds), CLI help text, tests.
• Shares WhisperKitEngine + CLI surface with 轉錄幻覺過濾 Phase 3：decode-param knobs (WhisperKit) #101 → same-file group; implemented on one cluster branch with 轉錄幻覺過濾 Phase 3：decode-param knobs (WhisperKit) #101, one PR refs both.

Strategy

• Plumb noSpeechProb/compressionRatio through RawSegment → TranscriptSegment (nil-default, all constructor sites)
• WhisperKitEngine populates both from the WhisperKit segment
• .full mode in HallucinationFilter with named canonical thresholds (joint silence rule + repetition rule); nil signals never trip (degrades to denylist)
• Tests: threshold trips/edge (at-threshold no-drop), nil-signal degradation, denylist behavior preserved under full, reindex keeps new fields
• CLI/README doc-sync for <off|denylist|full>

Conflict Class

C_shared_module_coord — shares WhisperKitEngine.swift + TranscribeOptions/CLI surface with #101; the two are sequenced adjacently on one branch (batch rule: same-file issues stay one group).

Complexity

Simple — the issue body already settles the design; mechanical plumbing + one filter case + tests, following #98's established choke-point pattern.

Risks

• Threshold semantics: applying noSpeechProb alone (without the avgLogprob conjunction) over-drops quiet-but-real speech — mitigated by using Whisper's canonical joint rule.
• Constructor-site sweep must be complete (filter reindex + diarization relabel + engine seam) or fields silently drop mid-pipeline — covered by a plumbing test asserting fields survive the filter pass.

Residue

(none)

[kiki830621]

✅ Verify — #100 confidence-gated full filter mode (opus adversarial pass, cluster with #101)

Verdict: SHIP — zero findings (one self-refuted nit recorded). Build green; 352 tests / 75 suites; the new suites repeated ×3 with no flake.

Attacked and confirmed clean

• Threshold fidelity: joint silence rule (noSpeechProb > 0.6 && avgLogprob < -1.0) and repetition rule (compressionRatio > 2.4) match openai-whisper semantics; strict-inequality boundaries consistent across code, tests, and README (compressionRatio == 2.4 is kept, as asserted).
• Plumbing completeness: all three TranscriptSegment construction sites (engine seam, withSpeaker, filter reindex) forward the new signals; diarization relabels via withSpeaker (forwards) and the filter runs after diarization at the choke point — signals survive end-to-end. Other backends leave the fields nil, which can never trip a threshold → full degrades to denylist per backend with no branching, exactly per spec.
• No schema churn: JSON uses its own SegmentJSON (new fields not serialized); SRT/VTT/txt untouched — the output-format promise holds.
• Mode contracts: confidence gating is hard-gated on mode == .full; denylist ignoring signals is test-covered.

Nit (self-refuted, recorded): avgLogprob == -1.0 exactly is kept where openai-whisper's <= would skip — a measure-zero boundary on a continuous float, no practical effect.

Stopping at verified — cluster PR (with #101) follows; merge + manual close are yours.

Refs #100

[kiki830621]

Closing Summary

Problem

The #98 denylist filter only catches known boilerplate strings; novel silent-segment and repetition hallucinations sail through. Whisper computes per-segment signals (no_speech_prob, compression_ratio) that flag these generically, but bestASR dropped them at the engine boundary — full mode had been deliberately deferred from the #98 MVP rather than shipped as a stub (#100).

Root Cause

Feature gap by design: the segment model carried only confidence (avgLogprob); the confidence signals WhisperKit already computed had no path through RawSegment → TranscriptSegment → the filter choke point.

Solution

• RawSegment and TranscriptSegment gained optional noSpeechProb / compressionRatio (nil-default — every existing engine and constructor site compiles unchanged); WhisperKit populates them, and all three construction sites (engine seam, withSpeaker, filter reindex) forward them, so the signals survive diarization and reach the filter.
• HallucinationFilterMode.full: everything denylist does, plus the canonical openai-whisper rules as named constants — the joint silence rule (noSpeechProb > 0.6 AND avgLogprob < -1.0; a conjunction on purpose, either signal alone also fires on quiet-but-real speech) and the repetition rule (compressionRatio > 2.4).
• Per-backend degradation for free: nil signals can never trip a threshold, so on whisper.cpp / Parakeet full behaves exactly like denylist with zero branching — the fallback the issue asked for, structural rather than coded.
• Output formats unchanged: the signals stay filter-internal (JSON's SegmentJSON doesn't serialize them) — no schema churn for downstream parsers.

Verification

• Opus adversarial verify (cluster with 轉錄幻覺過濾 Phase 3：decode-param knobs (WhisperKit) #101): SHIP, zero findings — threshold fidelity vs openai-whisper semantics, plumbing-completeness sweep across every constructor site, diarization-ordering survival, schema-unchanged claim, and mode-contract isolation all attacked and held. Report: 轉錄幻覺過濾 Phase 2：confidence-gated (full) 模式 #100 (comment)
• 6 new gate-driven tests (conjunction semantics, boundary keep at == 2.4, nil degradation, denylist-under-full, mode isolation, reindex signal preservation); full suite 352/75 green; PR Hallucination filtering Phases 2+3 — confidence-gated full mode + WhisperKit decode knobs #102 CI green.

Changes

• afe0e66 feat: confidence-gated 'full' hallucination-filter mode
• PR Hallucination filtering Phases 2+3 — confidence-gated full mode + WhisperKit decode knobs #102 (squash-merged, cluster with 轉錄幻覺過濾 Phase 3：decode-param knobs (WhisperKit) #101)

Related

• 轉錄後過濾 Whisper 常見幻覺（no-speech 段的罐頭字幕 / 重複殘影） #98 (Phase A denylist), 轉錄幻覺過濾 Phase 3：decode-param knobs (WhisperKit) #101 (Phase 3 decode-side knobs — complementary suppression at the source). The MCP transcribe tool reaches full automatically via its hallucination_filter string parameter.