Skip to content

feat(kg): support local/OpenAI-compatible LLM endpoints - #816

Open
bferanmi806-sketch wants to merge 1 commit into
HelpCode-ai:mainfrom
bferanmi806-sketch:kg-llm-custom-endpoint
Open

bferanmi806-sketch wants to merge 1 commit into
HelpCode-ai:mainfrom
bferanmi806-sketch:kg-llm-custom-endpoint

Conversation

@bferanmi806-sketch

Copy link
Copy Markdown

Fixes #598. @keysersoft — implemented per your approved contract.

Contract compliance:

  • Uses KG_LLM_BASE_URL (not AI_BASE_URL); optional KG_LLM_API_KEY (not AI_*); keeps using KG_LLM_MODEL.
  • Base URL set → OpenAI-compatible request goes there; empty key valid (Ollama/LM Studio need none).
  • Authorization: Bearer … only when a key exists (custom path); hosted paths byte-identical.
  • No response_format on the custom endpoint — JSON instruction + existing parsing instead.
  • Unusable local JSON → warn log with model + endpoint, return empty result so the enrichment/skill pass is skipped, not crashed.
  • OpenAI/OpenRouter/Anthropic behaviour preserved when no base URL is set; batch stays Anthropic-only.

Tests: new llm-client.spec.ts (9/9 pass) — config resolution (custom routing, empty-key validity, model passthrough, slash trim, hosted unchanged) and request construction (no Authorization with empty key, Authorization with key, no response_format on custom, present on OpenAI, hosted parse errors still throw, custom parse failure resolves {json:{}}). eslint clean on touched files; llm-client.ts has zero tsc errors. (Pre-existing: repo-wide tsc and 5 prisma-dependent suites fail on clean tree too — generated Prisma client absent in this env.)

Docs: .env.example + docs/knowledge-graph.md updated; tested model named (qwen2.5).

Real local run (honest harness note: the Ollama binary download is blocked in this sandbox, so this ran Qwen2.5-0.5B-Instruct via llama.cpp behind a minimal /v1/chat/completions front — the identical request/response contract Ollama serves):

  • resolved: {provider:custom, model:qwen2.5, apiKey:'', baseUrl:http://localhost:11435/v1}
  • server saw: response_format=null auth=absent
  • parsed: {relationships:[{from:e0, to:e1, kind:same_identity, confidence:0.9, reason:e0 is a person and e1 is a customer}]}

Happy to sign the CLA when the bot asks.

)

When KG_LLM_BASE_URL is set, the KG enrichment/skill LLM path talks to
that endpoint (Ollama, LM Studio, ...) using KG_LLM_MODEL. KG_LLM_API_KEY
is optional and only sent as Authorization when present; response_format
is never sent on this path — the JSON instruction plus shared parsing
apply, and unusable replies are logged (model + endpoint) and skipped
as an empty result instead of crashing the flow. Hosted OpenAI /
OpenRouter / Anthropic behaviour and Anthropic-only batch mode are
unchanged.
@github-actions

github-actions Bot commented Oct 1, 2026

Copy link
Copy Markdown


Thank you for your submission, we really appreciate it. Like many open-source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution. You can sign the CLA by just posting a Pull Request Comment same as the below format.


I have read the CLA Document and I hereby sign the CLA


Odunayo Balogun seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.

@github-actions

github-actions Bot commented Oct 1, 2026

Copy link
Copy Markdown

👋 Welcome, @bferanmi806-sketch, and thanks for opening your first PR on AnythingMCP!

A few quick pointers:

  • Make sure CI is green before requesting review (Backend, Frontend, Playwright, CodeQL, Trivy).
  • If this is a new adapter, the parametrised catalog.spec.ts test will validate it automatically.
  • Sign off your commits if you can — it's not blocking, just nice to have.

Someone from the core team will look at this within ~48h. If you don't hear back, please ping us in Discussions / Q&A.

⭐ While you wait — if you find AnythingMCP useful, a star helps others discover it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

OpenAI-compatible endpoint for the AI features

1 participant