Supporting Embedding models

[cecilphillip]

On my machine, foundry model ls only returns a list of chat completion models. Are there any plans to add embedding models as well?

[natke]

Thanks @cecilphillip! Yes embedding models are definitely on our roadmap

[congiuluc]

@natke any news about embedding models support/availability?

[alxxdev]

Hi @cecilphillip,

Embeddings are actually already supported in Foundry Local via the OpenAI-compatible /v1/embeddings endpoint. The catalog includes embedding models:

$ foundry model list --type embedding -v
+---------------------+-----------+--------+--------+-------+--------+----------+------------------------------------+------------------------------------+
| Model Name          | Type      | Size   | Device | Tools | Cached | License  | Display Name                       | Model ID                           |
+---------------------+-----------+--------+--------+-------+--------+----------+------------------------------------+------------------------------------+
| qwen3-embedding-0.6b| Embedding | 495 MB | CPU    | ○     | ●      | apache-2.0| qwen3-embedding-0.6b-generic-cpu  | qwen3-embedding-0.6b-generic-cpu:1 |
| qwen3-embedding-8b  | Embedding | 5.6 GB | CPU    | ○     | ○      | apache-2.0| qwen3-embedding-8b-generic-cpu    | qwen3-embedding-8b-generic-cpu:1   |
+---------------------+-----------+--------+--------+-------+--------+----------+------------------------------------+------------------------------------+

To use them:

foundry model download qwen3-embedding-0.6b
foundry model load qwen3-embedding-0.6b

Then call the API:

curl http://127.0.0.1:<port>/v1/embeddings \
  -H "Content-Type: application/json" \
  -d '{"model":"qwen3-embedding-0.6b","input":"hello world"}'

Or from Python with the OpenAI SDK:

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:<port>/v1", api_key="none")
resp = client.embeddings.create(model="qwen3-embedding-0.6b", input="hello world")
print(resp.data[0].embedding)

No CLI command for embeddings yet (only HTTP API), but the functionality is there. Hope this helps!