Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions core/http/endpoints/mcp/localai_assistant_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,10 @@
return "stub-backend-job", nil
}

func (stubClient) InstallLibraryBackend(_ context.Context, _ localaitools.InstallLibraryBackendRequest) (*localaitools.InstallLibraryBackendResult, error) {

Check failure on line 86 in core/http/endpoints/mcp/localai_assistant_test.go

View workflow job for this annotation

GitHub Actions / golangci-lint

method stubClient.InstallLibraryBackend already declared at core/http/endpoints/mcp/localai_assistant_test.go:78:19 (typecheck)
return &localaitools.InstallLibraryBackendResult{Status: "installed", Name: "stub-backend", SourceID: "stub-source"}, nil
}

func (stubClient) UpgradeBackend(_ context.Context, _ string) (string, error) {
return "stub-upgrade-job", nil
}
Expand Down
24 changes: 24 additions & 0 deletions docs/content/features/model-gallery.md
Original file line number Diff line number Diff line change
Expand Up @@ -1112,3 +1112,27 @@ is still running is reported as queued until the installer picks it up:
A job ID is queryable from the moment `/models/apply` returns it, so a `404`/`500`
from this endpoint means the ID is genuinely unknown rather than merely waiting
its turn.

## LFM2.5 Turbo Brilliance

The gallery includes DavidAU's LFM2.5-24B-A2B Turbo Brilliance build for
text chat and reasoning through llama.cpp. The IQ4_XS entry links to a
Q8_0 MAX variant. Both use the embedded Turbo Brilliance chat template.

Install with automatic variant selection:

```bash
local-ai models install lfm2.5-24b-a2b-turbo-brilliance-iq4-xs
```

Select Q8_0 MAX explicitly:

```bash
local-ai models install lfm2.5-24b-a2b-turbo-brilliance-iq4-xs --variant lfm2.5-24b-a2b-turbo-brilliance-q8
```

Both entries default to a 32,768-token context, temperature 0.1, top-k 50,
and repetition penalty 1.05. The weights support up to 131,072 tokens;
increase `context_size` in the installed configuration if needed.
See the [publisher's model card](https://huggingface.co/DavidAU/LFM2.5-24B-A2B-Turbo-Brilliance-50X-NEO-MAX-Imatrix-GGUF)
for the template's prompt controls and Liquid's LFM license.
84 changes: 84 additions & 0 deletions gallery/index.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,90 @@
# written by scripts/models/cmd/modelcatalog and stays only so releases before
# the cutover keep working. CI fails when it differs from the generated copy:
# go run ./scripts/models/cmd/modelcatalog --library gallery/models/index.yaml --out gallery/index.yaml
- url: github:mudler/LocalAI/gallery/virtual.yaml@master
name: lfm2.5-24b-a2b-turbo-brilliance-iq4-xs
description: |
DavidAU's Turbo Brilliance build of LFM2.5-24B-A2B, a text-only reasoning
model with 24B total parameters and 2.45B active parameters. This entry
uses the IQ4_XS GGUF quantization and the embedded Turbo Brilliance
chat template for multilingual chat and tool use.
license: other
urls:
- https://huggingface.co/LiquidAI/LFM2.5-24B-A2B
- https://huggingface.co/DavidAU/LFM2.5-24B-A2B-Turbo-Brilliance-50X-NEO-MAX-Imatrix-GGUF
icon: https://cdn-uploads.huggingface.co/production/uploads/61b8e2ba285851687028d395/2b08LKpev0DNEk6DlnWkY.png
tags:
- llm
- gguf
- reasoning
- cpu
- gpu
files:
- filename: llama-cpp/models/lfm2.5-24b-a2b-turbo-brilliance/LFM2.5-24B-A2B-TBrill-NEO-IQ4_XS.gguf
sha256: a1562db04b263e5406ac8412e07be6ea92941ea653653e9f678ab13f430e59b1
uri: https://huggingface.co/DavidAU/LFM2.5-24B-A2B-Turbo-Brilliance-50X-NEO-MAX-Imatrix-GGUF/resolve/69107daec71f075c94d7d84929b4a0222d1b8446/LFM2.5-24B-A2B-TBrill-NEO-IQ4_XS.gguf
overrides:
backend: llama-cpp
context_size: 32768
function:
automatic_tool_parsing_fallback: true
grammar:
disable: true
known_usecases:
- chat
- completion
options:
- use_jinja:true
parameters:
model: llama-cpp/models/lfm2.5-24b-a2b-turbo-brilliance/LFM2.5-24B-A2B-TBrill-NEO-IQ4_XS.gguf
repeat_penalty: 1.05
temperature: 0.1
top_k: 50
template:
use_tokenizer_template: true
variants:
- model: lfm2.5-24b-a2b-turbo-brilliance-q8
- url: github:mudler/LocalAI/gallery/virtual.yaml@master
name: lfm2.5-24b-a2b-turbo-brilliance-q8
description: |
DavidAU's Turbo Brilliance build of LFM2.5-24B-A2B, a text-only reasoning
model with 24B total parameters and 2.45B active parameters. This entry
uses the Q8_0 MAX GGUF quantization and the embedded Turbo Brilliance
chat template for multilingual chat and tool use.
license: other
urls:
- https://huggingface.co/LiquidAI/LFM2.5-24B-A2B
- https://huggingface.co/DavidAU/LFM2.5-24B-A2B-Turbo-Brilliance-50X-NEO-MAX-Imatrix-GGUF
icon: https://cdn-uploads.huggingface.co/production/uploads/61b8e2ba285851687028d395/2b08LKpev0DNEk6DlnWkY.png
tags:
- llm
- gguf
- reasoning
- cpu
- gpu
files:
- filename: llama-cpp/models/lfm2.5-24b-a2b-turbo-brilliance/LFM2.5-24B-A2B-TBrill-MAX-NEO-Q8_0.gguf
sha256: 17de10d567a0678dc6050445ecef3007ee1bf50df8d94a73d558d4133ad8aa4e
uri: https://huggingface.co/DavidAU/LFM2.5-24B-A2B-Turbo-Brilliance-50X-NEO-MAX-Imatrix-GGUF/resolve/69107daec71f075c94d7d84929b4a0222d1b8446/LFM2.5-24B-A2B-TBrill-MAX-NEO-Q8_0.gguf
overrides:
backend: llama-cpp
context_size: 32768
function:
automatic_tool_parsing_fallback: true
grammar:
disable: true
known_usecases:
- chat
- completion
options:
- use_jinja:true
parameters:
model: llama-cpp/models/lfm2.5-24b-a2b-turbo-brilliance/LFM2.5-24B-A2B-TBrill-MAX-NEO-Q8_0.gguf
repeat_penalty: 1.05
temperature: 0.1
top_k: 50
template:
use_tokenizer_template: true
- url: github:mudler/LocalAI/gallery/virtual.yaml@master
name: adavar-3b-q4
description: |
Expand Down
100 changes: 100 additions & 0 deletions gallery/models/index.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,106 @@
# changes and never edit it by hand. CI fails when it is stale. A build from
# master reads it as the default model source; a tagged build reads the gallery
# until the release cutover.
- kind: model
id: lfm2.5-24b-a2b-turbo-brilliance-iq4-xs
name: lfm2.5-24b-a2b-turbo-brilliance-iq4-xs
version: 0.0.0-gallery.285918856e35
description: |
DavidAU's Turbo Brilliance build of LFM2.5-24B-A2B, a text-only reasoning
model with 24B total parameters and 2.45B active parameters. This entry
uses the IQ4_XS GGUF quantization and the embedded Turbo Brilliance
chat template for multilingual chat and tool use.
license: other
urls:
- https://huggingface.co/LiquidAI/LFM2.5-24B-A2B
- https://huggingface.co/DavidAU/LFM2.5-24B-A2B-Turbo-Brilliance-50X-NEO-MAX-Imatrix-GGUF
tags:
- llm
- gguf
- reasoning
- cpu
- gpu
icon: https://cdn-uploads.huggingface.co/production/uploads/61b8e2ba285851687028d395/2b08LKpev0DNEk6DlnWkY.png
channel: gallery
policy:
requireSignature: false
schemaVersion: 1
model:
url: github:mudler/LocalAI/gallery/virtual.yaml@master
files:
- filename: llama-cpp/models/lfm2.5-24b-a2b-turbo-brilliance/LFM2.5-24B-A2B-TBrill-NEO-IQ4_XS.gguf
sha256: a1562db04b263e5406ac8412e07be6ea92941ea653653e9f678ab13f430e59b1
uri: https://huggingface.co/DavidAU/LFM2.5-24B-A2B-Turbo-Brilliance-50X-NEO-MAX-Imatrix-GGUF/resolve/69107daec71f075c94d7d84929b4a0222d1b8446/LFM2.5-24B-A2B-TBrill-NEO-IQ4_XS.gguf
overrides:
backend: llama-cpp
context_size: 32768
function:
automatic_tool_parsing_fallback: true
grammar:
disable: true
known_usecases:
- chat
- completion
options:
- use_jinja:true
parameters:
model: llama-cpp/models/lfm2.5-24b-a2b-turbo-brilliance/LFM2.5-24B-A2B-TBrill-NEO-IQ4_XS.gguf
repeat_penalty: 1.05
temperature: 0.1
top_k: 50
template:
use_tokenizer_template: true
variants:
- model: lfm2.5-24b-a2b-turbo-brilliance-q8
- kind: model
id: lfm2.5-24b-a2b-turbo-brilliance-q8
name: lfm2.5-24b-a2b-turbo-brilliance-q8
version: 0.0.0-gallery.401e9a9ccabf
description: |
DavidAU's Turbo Brilliance build of LFM2.5-24B-A2B, a text-only reasoning
model with 24B total parameters and 2.45B active parameters. This entry
uses the Q8_0 MAX GGUF quantization and the embedded Turbo Brilliance
chat template for multilingual chat and tool use.
license: other
urls:
- https://huggingface.co/LiquidAI/LFM2.5-24B-A2B
- https://huggingface.co/DavidAU/LFM2.5-24B-A2B-Turbo-Brilliance-50X-NEO-MAX-Imatrix-GGUF
tags:
- llm
- gguf
- reasoning
- cpu
- gpu
icon: https://cdn-uploads.huggingface.co/production/uploads/61b8e2ba285851687028d395/2b08LKpev0DNEk6DlnWkY.png
channel: gallery
policy:
requireSignature: false
schemaVersion: 1
model:
url: github:mudler/LocalAI/gallery/virtual.yaml@master
files:
- filename: llama-cpp/models/lfm2.5-24b-a2b-turbo-brilliance/LFM2.5-24B-A2B-TBrill-MAX-NEO-Q8_0.gguf
sha256: 17de10d567a0678dc6050445ecef3007ee1bf50df8d94a73d558d4133ad8aa4e
uri: https://huggingface.co/DavidAU/LFM2.5-24B-A2B-Turbo-Brilliance-50X-NEO-MAX-Imatrix-GGUF/resolve/69107daec71f075c94d7d84929b4a0222d1b8446/LFM2.5-24B-A2B-TBrill-MAX-NEO-Q8_0.gguf
overrides:
backend: llama-cpp
context_size: 32768
function:
automatic_tool_parsing_fallback: true
grammar:
disable: true
known_usecases:
- chat
- completion
options:
- use_jinja:true
parameters:
model: llama-cpp/models/lfm2.5-24b-a2b-turbo-brilliance/LFM2.5-24B-A2B-TBrill-MAX-NEO-Q8_0.gguf
repeat_penalty: 1.05
temperature: 0.1
top_k: 50
template:
use_tokenizer_template: true
- kind: model
id: adavar-3b-q4
name: adavar-3b-q4
Expand Down
Loading