OpenAI Chat Completions ignores the refusal field

[dajiaohuang]

Summary

OpenAI Chat Completions responses that report a refusal in message.refusal can be treated as malformed output instead of a provider-declared refusal.

Reproduction

A Chat Completions response with finish_reason set to stop, message.content set to null, and message.refusal set to a refusal string passes through the shared result handler. The handler checks finish_reason, then returns message.content or an empty string; it does not inspect message.refusal. A minimal in-memory response reproduced an empty ProviderResult text without raising TypeSafeError.

Expected behavior

Raise TypeSafeError that identifies the refusal and preserve the response trace, without spending a malformed-output corrective retry.

Actual behavior

The refusal is reduced to empty text and enters malformed JSON validation. With corrective retries enabled, the adapter can retry a response the provider has already identified as a refusal; without retries, the caller receives a validation error rather than a refusal error.

Related coverage

Issue #43 handles Anthropic and OpenAI Responses refusals. Its description leaves Chat Completions aside because #38 covers finish_reason. Issue #38 rejects non-success finish reasons, but this response has finish_reason=stop and the refusal is in message.refusal, so the cases are distinct.

A focused regression should cover sync and async Chat Completions responses with message.refusal, including the no-corrective-retry behavior.

[aoright]

Confirmed. When OpenAI Chat Completions returns a refusal, the payload structure typically sets finish_reason="stop" with message.content=None and message.refusal="<explanation>".

Because _result() in src/system_one_adapter/providers/openai.py currently inspects only finish_reason and then evaluates choice.message.content or "", the provider response passes through as an empty string "". This triggers malformed JSON parsing errors and consumes corrective retry attempts against a response that the model provider explicitly rejected.

Proposed Implementation

In src/system_one_adapter/providers/openai.py::_result:

1. Check not response.choices up front to record the attempt and fail fast with TypeSafeError.
2. Record choice.finish_reason.
3. Check getattr(choice.message, "refusal", None). If present, immediately raise TypeSafeError(f"OpenAI chat completion was a refusal: {refusal}").

def _result(response: Any) -> ProviderResult:
    if not response.choices:
        record_response(response, finish_reason=None)
        raise TypeSafeError("OpenAI chat completion returned no choices.")
    choice = response.choices[0]
    record_response(response, finish_reason=choice.finish_reason)
    finish_reason = choice.finish_reason
    if finish_reason not in ("stop", None):
        raise TypeSafeError(f"OpenAI chat completion did not complete: {finish_reason}.")
    refusal = getattr(choice.message, "refusal", None)
    if refusal:
        raise TypeSafeError(f"OpenAI chat completion was a refusal: {refusal}")
    return ProviderResult(
        text=choice.message.content or "",
        input_tokens=getattr(response.usage, "prompt_tokens", None),
        output_tokens=getattr(response.usage, "completion_tokens", None),
    )

Verification and Tests

Added unit tests in tests/test_provider_nonanswers.py parametrized across sync (OpenAIProvider) and async (AsyncOpenAIProvider) in both structured and prompted modes:

• test_chat_completion_refusal: Validates that TypeSafeError is raised with the refusal string, retry_reasons == [], len(llm_attempts) == 1, and the debug trace preserves message.refusal.
• test_chat_completion_empty_choices: Validates handling of empty choices without throwing an unhandled IndexError.

All 432 unit tests pass and pyrefly check reports 0 errors.

Branch and commit for review:
aoright@27d71f1
(Branch: fix/openai-chat-refusal)

[dajiaohuang]

Thanks for confirming the refusal payload and sharing your fix branch. I do not see a PR for it yet. Do you intend to open one from that branch, or would you prefer a separate submission? I want to avoid creating competing PRs for the same behavior.

[aoright]

Thanks for following up! Direct pull request creation on typesafe-ai/system-one-adapter-python is restricted by repository policy to collaborators (CreatePullRequest permission required), which prevents external forks from opening pull requests directly.

The branch fix/openai-chat-refusal on my fork (commit 27d71f1) is fully ready, tested (432/432 passed, pyrefly reports 0 errors), and addresses both refusal interception (#46) and empty choices handling (#47) with regression test coverage in tests/test_provider_nonanswers.py.

Compare: main...aoright:system-one-adapter-python:fix/openai-chat-refusal

Maintainers can pull directly from this branch. If collaborator permissions are granted to contributors, I can open the pull request directly from this branch.