Skip to content

feat(llm_local): add GET /v1/models for client preflight - #86

Merged
felipepenha merged 2 commits into
GenAI-Security-Project:mainfrom
wehnsdaefflae:feat/llm-local-models-endpoint
Sep 20, 2026
Merged

felipepenha merged 2 commits into
GenAI-Security-Project:mainfrom
wehnsdaefflae:feat/llm-local-models-endpoint

Conversation

@wehnsdaefflae

Copy link
Copy Markdown

Summary

Closes #82. Adds GET /v1/models to the llm_local mock.

#84 by @MRX-72 does the same thing. This one differs in two places, both open to discussion.

Changes

  • app/mocks/openai.py: new GET /v1/models, returning a typed ModelList built from the sandbox's own config (OLLAMA_MODEL, default gpt-oss:20b). It does not call Ollama.
    • Reason: clients call this route to check whether the sandbox is alive. If the route needs the model backend, a sandbox that is up but still pulling a model comes back as a 500. That is the same false negative as the 404, one layer down.
    • Trade-off, noted in the docstring: listing from config can advertise a model that the first completion then fails on. POST /v1/chat/completions still returns backend failures as a 500.
  • app/mocks/openai.py: verify_api_key now declares the header optional, so a request without Authorization gets 401 Missing Authorization header, expected: Bearer sk-mock-key instead of FastAPI's 422. That is the cause of llm_local: the mock requires Authorization: Bearer sk-mock-key and the README does not say so #83. The key is already public: make up prints it, both clients hardcode it, four READMEs list it.
  • app/mocks/README.md: the endpoint table and mock credential that section 4 of that file asks every mock to document.
  • tests/test_mock_routes.py: new. llm_local had no tests; this follows the layout mcp_local/tests/ uses.

How to test

cd sandboxes/llm_local
PYTHONPATH=. uv run pytest tests/ -q

Passes with Ollama switched off:

8 passed

Eight cases: response shape, the configured model id, that the route answers with no backend, the four auth paths, and module import.

curl http://localhost:8000/v1/models -H "Authorization: Bearer sk-mock-key"
# {"object":"list","data":[{"id":"gpt-oss:20b","object":"model","created":0,"owned_by":"ollama"}]}

curl -i http://localhost:8000/v1/models
# HTTP/1.1 401
# {"detail":"Missing Authorization header, expected: Bearer sk-mock-key"}

Notes

Clients call this route to find out whether the sandbox is up before they
send anything, so it answers from the sandbox's own configuration instead
of asking Ollama. A sandbox that is running but still pulling a model
would otherwise come back as a 500, which is the same false negative as
the 404 it replaces.

verify_api_key now declares the Authorization header optional, so a
request sending none gets a 401 naming the expected credential rather
than FastAPI's 422 about a missing required header.

llm_local had no tests. This adds eight, covering the response shape, the
configured model id, the four auth paths and the no-backend case. They
pass with Ollama switched off.

Closes GenAI-Security-Project#82.
Comment thread sandboxes/llm_local/app/mocks/openai.py Outdated
Felipe's review on GenAI-Security-Project#86: autonomous red-teaming tools such as agent0 read a named
credential in an error body as secret leakage and report it. The 401 still says what
a client needs, which is that the header is missing.

The test that asserted the old string now asserts the new one and that the response
carries no sk-mock-key anywhere.
wehnsdaefflae pushed a commit to wehnsdaefflae/GenAI-Red-Team-Lab that referenced this pull request Sep 20, 2026
Felipe's review on GenAI-Security-Project#87: the response documented here is the one GenAI-Security-Project#86 changes, so the
table now reads `401 Missing Authorization header` with no credential in it.

Also drops an em-dash from the Authentication paragraph.
@felipepenha
felipepenha merged commit ec48935 into GenAI-Security-Project:main Sep 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

llm_local: no GET /v1/models, so OpenAI-compatible preflight checks report a healthy sandbox as down

2 participants