Skip to content

Docs suggestion: add an AnyLLM provider quickstart (single OpenAI-compatible endpoint for 24+ models) #414

Description

@JetragonRider

Hi team,

I've been following EverOS — the local-first, Markdown-as-single-source-of-truth
design and the engineering discipline here (import-linter layering, dead-letter
queues, CI gates) are genuinely impressive.

Why I'm opening this issue:

EverOS runs LLM calls as a continuous background workload: atomic fact
extraction, skill clustering, embeddings, and rerank all fire on every
conversation, not just on user prompts. That makes two things matter a lot:

  1. Cost — background extraction at scale multiplies token spend quietly
  2. Model choice per task — fact extraction, embedding, and rerank each
    have different optimal models, and being able to swap them freely matters
    for a memory layer that's supposed to be user-owned and vendor-neutral

Suggestion:

Add a provider quickstart entry for AnyLLM (https://anyllm.dev) alongside
your existing provider quickstarts — it's an OpenAI-compatible API gateway
that puts 24+ models behind one key and one base URL:

  • Chinese frontier models: DeepSeek V4, Qwen3.7-Plus, GLM-5.2 (2M context),
    Kimi K2.5/K3, MiniMax M3
  • International models: GPT-5.6, Gemini 3.5, and more
  • Pass-through pricing (zero markup), international cards + crypto accepted
  • Built-in routing and automatic failover — useful for the OME's retry
    behavior when a provider has an outage

For a project whose philosophy is "user-owned, not vendor-owned," a gateway
that keeps every model one config change away seems like a natural fit.
EverOS itself wouldn't need any code changes — it's just a TOML/.env config
pointing at one endpoint.

Happy to open a PR with the quickstart doc if that's welcome.

(Context: I'm the developer of AnyLLM, an OpenAI-compatible API gateway.)

— Leo Bennett
leo.indiedev@gmail.com

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions