Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions docs/content/features/model-gallery.md
Original file line number Diff line number Diff line change
Expand Up @@ -622,6 +622,29 @@ where:
- `bert-embeddings` is the model name in the gallery
(read its [config here](https://github.com/mudler/LocalAI/blob/master/gallery/index.yaml)).

### Defiance 27B

[Defiance](https://huggingface.co/ttindall/Qwen3.8-27B-Defiance-Uncensored-MTP-GGUF)
combines HauhauCS's Qwen3.8-27B weights with a customized Turbo Brilliance chat
template. The gallery offers IQ4_XS and Q6_K_P GGUF builds for llama.cpp.
Both use the embedded template and a default context of 32,768 tokens.

Install with automatic variant selection:

```bash
local-ai models install qwen3.8-27b-defiance-iq4
```

To select the Q6_K_P build explicitly:

```bash
local-ai models install qwen3.8-27b-defiance-iq4 --variant qwen3.8-27b-defiance-q6
```

These entries support text input and ordinary decoding. They do not install a
vision projector or enable MTP. They use the template embedded in the GGUF,
which predates the publisher's separate template updates.

### Model variants

Some gallery entries offer several builds of the same model: different
Expand Down
66 changes: 66 additions & 0 deletions gallery/index.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,72 @@
# written by scripts/models/cmd/modelcatalog and stays only so releases before
# the cutover keep working. CI fails when it differs from the generated copy:
# go run ./scripts/models/cmd/modelcatalog --library gallery/models/index.yaml --out gallery/index.yaml
- url: github:mudler/LocalAI/gallery/qwen3.yaml@master
name: qwen3.8-27b-defiance-iq4
description: |
Defiance combines HauhauCS's uncensored Qwen3.8-27B weights with a
customized Turbo Brilliance chat template for reasoning and roleplay.
This IQ4_XS GGUF build uses llama.cpp and the embedded template,
with a 32,768-token context. Text input only; MTP is not enabled.
license: apache-2.0
urls:
- https://huggingface.co/ttindall/Qwen3.8-27B-Defiance-Uncensored-MTP-GGUF
- https://huggingface.co/HauhauCS/Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTP-GGUF
- https://huggingface.co/DavidAU/Qwen3.8-27B-Turbo-Brilliance-50X-Reasoning-Instruct-NEO-MAX-MTP-GGUF
tags:
- llm
- gguf
- cpu
- gpu
- reasoning
- text-generation
files:
- filename: Qwen3.8-27B-Defiance-Uncensored-IQ4_XS.gguf
sha256: d48cdb6e627b4e74b862938daf5ac2bfb7b82c5f02627a02a3b4927e7bb35f2f
uri: https://huggingface.co/ttindall/Qwen3.8-27B-Defiance-Uncensored-MTP-GGUF/resolve/1e664931451831589ce16389a55e0d5605dd4bc5/Qwen3.8-27B-Defiance-Uncensored-IQ4_XS.gguf
overrides:
context_size: 32768
parameters:
min_p: 0.05
model: Qwen3.8-27B-Defiance-Uncensored-IQ4_XS.gguf
repeat_penalty: 1
temperature: 0.7
top_k: 64
top_p: 0.95
variants:
- model: qwen3.8-27b-defiance-q6
- url: github:mudler/LocalAI/gallery/qwen3.yaml@master
name: qwen3.8-27b-defiance-q6
description: |
Defiance combines HauhauCS's uncensored Qwen3.8-27B weights with a
customized Turbo Brilliance chat template for reasoning and roleplay.
This Q6_K_P GGUF build uses llama.cpp and the embedded template,
with a 32,768-token context. Text input only; MTP is not enabled.
license: apache-2.0
urls:
- https://huggingface.co/ttindall/Qwen3.8-27B-Defiance-Uncensored-MTP-GGUF
- https://huggingface.co/HauhauCS/Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTP-GGUF
- https://huggingface.co/DavidAU/Qwen3.8-27B-Turbo-Brilliance-50X-Reasoning-Instruct-NEO-MAX-MTP-GGUF
tags:
- llm
- gguf
- cpu
- gpu
- reasoning
- text-generation
files:
- filename: Qwen3.8-27B-Defiance-Uncensored-Q6_K_P.gguf
sha256: 0ce1abf7e488a5efbd4408af763f18e21bdc220c93106f658d95607dc97868a3
uri: https://huggingface.co/ttindall/Qwen3.8-27B-Defiance-Uncensored-MTP-GGUF/resolve/1e664931451831589ce16389a55e0d5605dd4bc5/Qwen3.8-27B-Defiance-Uncensored-Q6_K_P.gguf
overrides:
context_size: 32768
parameters:
min_p: 0.05
model: Qwen3.8-27B-Defiance-Uncensored-Q6_K_P.gguf
repeat_penalty: 1
temperature: 0.7
top_k: 64
top_p: 0.95
- url: github:mudler/LocalAI/gallery/virtual.yaml@master
name: adavar-3b-q4
description: |
Expand Down
82 changes: 82 additions & 0 deletions gallery/models/index.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,88 @@
# changes and never edit it by hand. CI fails when it is stale. A build from
# master reads it as the default model source; a tagged build reads the gallery
# until the release cutover.
- kind: model
id: qwen3.8-27b-defiance-iq4
name: qwen3.8-27b-defiance-iq4
version: 0.0.0-gallery.9833dd29edab
description: |
Defiance combines HauhauCS's uncensored Qwen3.8-27B weights with a
customized Turbo Brilliance chat template for reasoning and roleplay.
This IQ4_XS GGUF build uses llama.cpp and the embedded template,
with a 32,768-token context. Text input only; MTP is not enabled.
license: apache-2.0
urls:
- https://huggingface.co/ttindall/Qwen3.8-27B-Defiance-Uncensored-MTP-GGUF
- https://huggingface.co/HauhauCS/Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTP-GGUF
- https://huggingface.co/DavidAU/Qwen3.8-27B-Turbo-Brilliance-50X-Reasoning-Instruct-NEO-MAX-MTP-GGUF
tags:
- llm
- gguf
- cpu
- gpu
- reasoning
- text-generation
channel: gallery
policy:
requireSignature: false
schemaVersion: 1
model:
url: github:mudler/LocalAI/gallery/qwen3.yaml@master
files:
- filename: Qwen3.8-27B-Defiance-Uncensored-IQ4_XS.gguf
sha256: d48cdb6e627b4e74b862938daf5ac2bfb7b82c5f02627a02a3b4927e7bb35f2f
uri: https://huggingface.co/ttindall/Qwen3.8-27B-Defiance-Uncensored-MTP-GGUF/resolve/1e664931451831589ce16389a55e0d5605dd4bc5/Qwen3.8-27B-Defiance-Uncensored-IQ4_XS.gguf
overrides:
context_size: 32768
parameters:
min_p: 0.05
model: Qwen3.8-27B-Defiance-Uncensored-IQ4_XS.gguf
repeat_penalty: 1
temperature: 0.7
top_k: 64
top_p: 0.95
variants:
- model: qwen3.8-27b-defiance-q6
- kind: model
id: qwen3.8-27b-defiance-q6
name: qwen3.8-27b-defiance-q6
version: 0.0.0-gallery.34421ee644a5
description: |
Defiance combines HauhauCS's uncensored Qwen3.8-27B weights with a
customized Turbo Brilliance chat template for reasoning and roleplay.
This Q6_K_P GGUF build uses llama.cpp and the embedded template,
with a 32,768-token context. Text input only; MTP is not enabled.
license: apache-2.0
urls:
- https://huggingface.co/ttindall/Qwen3.8-27B-Defiance-Uncensored-MTP-GGUF
- https://huggingface.co/HauhauCS/Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTP-GGUF
- https://huggingface.co/DavidAU/Qwen3.8-27B-Turbo-Brilliance-50X-Reasoning-Instruct-NEO-MAX-MTP-GGUF
tags:
- llm
- gguf
- cpu
- gpu
- reasoning
- text-generation
channel: gallery
policy:
requireSignature: false
schemaVersion: 1
model:
url: github:mudler/LocalAI/gallery/qwen3.yaml@master
files:
- filename: Qwen3.8-27B-Defiance-Uncensored-Q6_K_P.gguf
sha256: 0ce1abf7e488a5efbd4408af763f18e21bdc220c93106f658d95607dc97868a3
uri: https://huggingface.co/ttindall/Qwen3.8-27B-Defiance-Uncensored-MTP-GGUF/resolve/1e664931451831589ce16389a55e0d5605dd4bc5/Qwen3.8-27B-Defiance-Uncensored-Q6_K_P.gguf
overrides:
context_size: 32768
parameters:
min_p: 0.05
model: Qwen3.8-27B-Defiance-Uncensored-Q6_K_P.gguf
repeat_penalty: 1
temperature: 0.7
top_k: 64
top_p: 0.95
- kind: model
id: adavar-3b-q4
name: adavar-3b-q4
Expand Down
Loading