Skip to content

Commit 64210d9

Browse files
bobbyjohnstxclaude
andcommitted
feat(skill): add vLLM provider health check to tc-doctor
Detects and fixes: output limit too small for thinking models, auto-discovery overriding user config, reasoning capability not detected, context window sizing. Provides concrete tinycode.jsonc fix with unique provider ID to avoid auto-discovery collision. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 5ee23e0 commit 64210d9

1 file changed

Lines changed: 45 additions & 1 deletion

File tree

  • packages/tinycode/src/skill/defaults/tc-doctor

‎packages/tinycode/src/skill/defaults/tc-doctor/SKILL.md‎

Lines changed: 45 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -142,7 +142,51 @@ else
142142
fi
143143
```
144144

145-
### 8. Disk space
145+
### 8. vLLM / Local LLM Provider Health
146+
Check vLLM provider configuration for common issues — auto-detected models often have wrong limits or missing capabilities.
147+
148+
Use the tinycode API (not shell) to inspect the current provider state. Read the config at `~/.config/tinycode/tinycode.jsonc` (user overrides) and compare with what the provider list reports.
149+
150+
**Check for these issues and fix them:**
151+
152+
#### a. Output limit too small for thinking models
153+
If a model name contains "qwen" (any case) and reasoning is false, or if the output limit is less than 2000 tokens, the model likely needs a config override. Qwen3 models use `<think>` blocks that consume output budget — 1638 tokens (the default 80/20 split) is too small.
154+
155+
**Fix:** Write or update `~/.config/tinycode/tinycode.jsonc` with a custom provider entry using a unique provider ID (not "vllm" — that collides with auto-discovery). Set output limit to 4000 and reasoning to true:
156+
157+
```json
158+
{
159+
"provider": {
160+
"vllm-qwen3": {
161+
"npm": "@ai-sdk/openai-compatible",
162+
"api": "http://<vllm-host>:<port>/v1",
163+
"models": {
164+
"<model-id>": {
165+
"reasoning": true,
166+
"limit": {
167+
"context": 4000,
168+
"output": 4000
169+
}
170+
}
171+
}
172+
}
173+
},
174+
"model": "vllm-qwen3/<model-id>"
175+
}
176+
```
177+
178+
#### b. Auto-discovery overriding user config
179+
If a provider ID in `tinycode.jsonc` matches the auto-discovered provider ID (e.g., both use "vllm"), auto-discovery overwrites the user's limits and capabilities. The fix is to use a unique provider ID like "vllm-qwen3" or "vllm-custom" in the config.
180+
181+
#### c. Context window too large for available memory
182+
If the auto-detected context limit is much larger than the model can actually serve (causes OOM or timeouts), set an explicit limit. For Qwen3-30B on a single GPU with 8K max_model_len, use context=4000 output=4000.
183+
184+
#### d. Reasoning capability not detected
185+
vLLM may not advertise the "thinking" capability for models that support `<think>` blocks. If the model name contains "qwen3", "deepseek", or other known reasoning models, set `reasoning: true` in the config override.
186+
187+
**After fixing:** Tell the user to restart tinycode or refresh the browser. The model will appear with correct limits under the custom provider ID.
188+
189+
### 9. Disk space
146190
```bash
147191
df -h / /home/tinycode/.local/share/tinycode 2>/dev/null | tail -n +2 | while read fs size used avail pct mount; do
148192
pct_num=${pct%\%}

0 commit comments

Comments
 (0)