Skip to content

HFI: the three text run-fns drop refusals — ~5 LOC per site, and the tool-calling path already gets it free #879

Description

@sroussey

Defect

@workglow/huggingface-inference's three text run-fns read only delta.content from each chat chunk, so a delta.refusal — the OpenAI-shaped safety field the provider's own SDK surfaces — is discarded. The task completes with empty or truncated text and no refusal output, which is exactly the case StreamRefusal and REFUSAL_OUTPUT_KEY were added for (packages/task-graph/src/task/StreamTypes.ts:84-95), including CacheCoordinator.ts:145-158's rule that a refusal must not be memoised as the answer.

Verified at 9a80450:

  • providers/huggingface-inference/src/ai/common/HFI_TextGeneration.ts:91const delta = chunk.choices?.[0]?.delta?.content ?? "";
  • providers/huggingface-inference/src/ai/common/HFI_TextRewriter.ts:54 — same line
  • providers/huggingface-inference/src/ai/common/HFI_TextSummary.ts:54 — same line

grep -ric efusal providers/huggingface-inference/src0 in those three files. The provider's tool-calling path already handles it, for free, because it routes through the shared helper: HFI_ToolCalling.tsaccumulateOpenAIChatStream.

Cross-provider state at HEAD (grep -ric efusal providers/*/src): gemini 35, anthropic 22, xai 14, openrouter 14, deepseek 14, openai 7; ollama, HFI, HFT, node-llama-cpp, llamacpp-server, chrome-ai, cactus, tf-mediapipe, stable-diffusion-server, mlx all 0. That is 6 full + 1 partial of 15, unchanged for six review cycles.

Proposed fix

HFI is the cheapest remaining move on that matrix because it is OpenAI-shaped and the helper already exists. Either:

  • route the three text run-fns through accumulateOpenAIChatStream the way HFI_ToolCalling.ts does, or
  • add the ~5 lines per site that xAI and OpenRouter carry:
const refusal = chunk.choices?.[0]?.delta?.refusal;
if (refusal) emit({ type: "refusal", textDelta: refusal });

Then extend packages/test/src/test/ai-provider/CrossProviderRefusals.test.ts to cover HFI, so the count is asserted rather than grepped.

Why it matters

Takes refusal from 6 to 7 of 15, and it is the last provider in the set where the surrounding machinery is already present — every other zero in the list is a local provider with no refusal concept at the wire level. A dropped refusal is not a visible failure: the task completes normally with a short or empty answer, and without the refusal output CacheCoordinator will happily memoise it, so a transient safety-classifier flake replays as the answer on every subsequent run.

Carried as a review recommendation for five cycles (2026-07-20 through 2026-08-31) without being tracked. Filing it so it stops being a recommendation. Snapshot: workglow-dev/prdanalysis/grades/2026-08-31/libs-providers.md.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions