feat(kg): support local/OpenAI-compatible LLM endpoints - #816
bferanmi806-sketch wants to merge 1 commit into
Conversation
) When KG_LLM_BASE_URL is set, the KG enrichment/skill LLM path talks to that endpoint (Ollama, LM Studio, ...) using KG_LLM_MODEL. KG_LLM_API_KEY is optional and only sent as Authorization when present; response_format is never sent on this path — the JSON instruction plus shared parsing apply, and unusable replies are logged (model + endpoint) and skipped as an empty result instead of crashing the flow. Hosted OpenAI / OpenRouter / Anthropic behaviour and Anthropic-only batch mode are unchanged.
|
I have read the CLA Document and I hereby sign the CLA Odunayo Balogun seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. |
|
👋 Welcome, @bferanmi806-sketch, and thanks for opening your first PR on AnythingMCP! A few quick pointers:
Someone from the core team will look at this within ~48h. If you don't hear back, please ping us in Discussions / Q&A. ⭐ While you wait — if you find AnythingMCP useful, a star helps others discover it. |
Fixes #598. @keysersoft — implemented per your approved contract.
Contract compliance:
KG_LLM_BASE_URL(notAI_BASE_URL); optionalKG_LLM_API_KEY(notAI_*); keeps usingKG_LLM_MODEL.Authorization: Bearer …only when a key exists (custom path); hosted paths byte-identical.response_formaton the custom endpoint — JSON instruction + existing parsing instead.Tests: new
llm-client.spec.ts(9/9 pass) — config resolution (custom routing, empty-key validity, model passthrough, slash trim, hosted unchanged) and request construction (noAuthorizationwith empty key,Authorizationwith key, noresponse_formaton custom, present on OpenAI, hosted parse errors still throw, custom parse failure resolves{json:{}}).eslintclean on touched files;llm-client.tshas zerotscerrors. (Pre-existing: repo-widetscand 5 prisma-dependent suites fail on clean tree too — generated Prisma client absent in this env.)Docs:
.env.example+docs/knowledge-graph.mdupdated; tested model named (qwen2.5).Real local run (honest harness note: the Ollama binary download is blocked in this sandbox, so this ran Qwen2.5-0.5B-Instruct via llama.cpp behind a minimal
/v1/chat/completionsfront — the identical request/response contract Ollama serves):{provider:custom, model:qwen2.5, apiKey:'', baseUrl:http://localhost:11435/v1}response_format=null auth=absent{relationships:[{from:e0, to:e1, kind:same_identity, confidence:0.9, reason:e0 is a person and e1 is a customer}]}Happy to sign the CLA when the bot asks.