Skip to content

fix(backend): realign BYOK security tests with enrolled-key contract - #12302

Open
aryanorastar wants to merge 1 commit into
BasedHardware:mainfrom
aryanorastar:codex/12289-byok-security-contract
Open

fix(backend): realign BYOK security tests with enrolled-key contract#12302
aryanorastar wants to merge 1 commit into
BasedHardware:mainfrom
aryanorastar:codex/12289-byok-security-contract

Conversation

@aryanorastar

@aryanorastar aryanorastar commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

What changed and why

Fixes #12289.

The BYOK security suite still mocked the removed aggregate subscription.get_byok_keys binding and asserted permissive partial-enrollment behavior after production moved to per-provider keys plus strict enrolled-provider validation. This test-only repair targets the dependencies production now uses and aligns the assertions with the current fail-closed contract; no backend behavior changes.

Product invariants affected

none

How it was verified

Reproduced on current main (932e2f5f06):

cd backend && .venv/bin/python -m pytest tests/unit/test_byok_security.py -q
8 failed, 96 passed

Verified this branch with the identical command:

104 passed

The repository preflight also passes all 25 selected checks when run with this PR body:

scripts/pr-preflight --pr-body-file /tmp/omi-12289-pr-body.md

The broader pre-push hook reaches Pyright and reports two pre-existing unused imports in backend/utils/stt/streaming.py:37 (SafeSonioxSocket and process_audio_soniox). That file is byte-for-byte outside this branch's diff, so the branch was pushed with only that inherited hook failure bypassed.

Tests

backend/tests/unit/test_byok_security.py now covers the current contract directly:

  • quota bypass requires a validated BYOK identity, enrolled provider fingerprint, and provider-specific key;
  • OpenRouter, OpenAI, Anthropic, and Gemini keys qualify for LLM BYOK, while Deepgram alone does not;
  • a single-provider enrollment accepts that provider's header;
  • multi-provider enrollment rejects missing headers over HTTP and WebSocket;
  • an empty stored fingerprint rejects the request key rather than allowing an unverified or paid-key fallback.

Failure class (fixes)

Failure-Class: none

Review in cubic

Update stale mocks to target the current per-provider BYOK helpers and
assert the fail-closed enrollment behavior implemented by the middleware.

Reproduces BasedHardware#12289's BYOK test failures on main and restores the focused
security suite from 8 failures / 96 passes to 104 passes.

Failure-Class: none
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@Git-on-my-level

Copy link
Copy Markdown
Collaborator

Thanks @aryanorastar - verified end to end; this cleanly repairs the BYOK security suite.

What I ran (isolated worktrees, nothing shared with your branch):

Notes on backend/tests/unit/test_byok_security.py (only changed file):

  • The old @patch('utils.subscription.get_byok_keys', ...) decorators targeted a module binding that no longer exists; utils/subscription.py now imports the per-provider helpers (get_byok_key, get_byok_uid, get_cached_byok_state, has_validated_byok_keys), and the rewrite patches exactly those seams - so the tests now intercept the production path instead of a dead attribute.
  • test_partial_headers_when_byok_active_raise_403 and the WebSocket case now pin the fail-closed behavior utils/byok.py actually ships (403 / error BYOK key header missing for enrolled provider: anthropic); the old versions asserted permissive behavior and were failing against current production.
  • test_empty_fingerprint_entry_rejects_request_key asserting the full 403 fingerprint-mismatch detail is a strictly stronger check than the old key-dropping assertion.
  • test_single_provider_enrollment_accepts_its_only_header matches the enrolled-provider loop (only the provider you enrolled is required).
  • The quota tests now assert get_chat_quota_snapshot is never reached on the BYOK bypass - sharper than before.

Heads-up on the red Backend unit suite check: it is unrelated to this PR. The job exits at the pyright step on two unused-import errors in backend/utils/stt/streaming.py:37 (SafeSonioxSocket, process_audio_soniox) - that failure exists on main itself and the pytest step never runs. Worth a small separate fix so this suite can report green.

No production behavior changes here; BYOK validation semantics remain exactly what main ships. Leaving for human maintainer review before merge.

Automated maintainer-assist review on behalf of the Omi maintainers.


by AI on behalf of David — if you need David’s attention urgently, please @Git-on-my-level and escalate with need human response.

@Git-on-my-level Git-on-my-level added positive-signal Good PR — positive signal, not a formal approval backend Backend Task (python) labels Aug 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend Backend Task (python) positive-signal Good PR — positive signal, not a formal approval

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test_byok_security fails 8 tests on main: removed 'unused' import was the mock.patch target

2 participants