You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
feat(skill): add vLLM provider health check to tc-doctor
Detects and fixes: output limit too small for thinking models, auto-discovery
overriding user config, reasoning capability not detected, context window
sizing. Provides concrete tinycode.jsonc fix with unique provider ID to
avoid auto-discovery collision.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copy file name to clipboardExpand all lines: packages/tinycode/src/skill/defaults/tc-doctor/SKILL.md
+45-1Lines changed: 45 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -142,7 +142,51 @@ else
142
142
fi
143
143
```
144
144
145
-
### 8. Disk space
145
+
### 8. vLLM / Local LLM Provider Health
146
+
Check vLLM provider configuration for common issues — auto-detected models often have wrong limits or missing capabilities.
147
+
148
+
Use the tinycode API (not shell) to inspect the current provider state. Read the config at `~/.config/tinycode/tinycode.jsonc` (user overrides) and compare with what the provider list reports.
149
+
150
+
**Check for these issues and fix them:**
151
+
152
+
#### a. Output limit too small for thinking models
153
+
If a model name contains "qwen" (any case) and reasoning is false, or if the output limit is less than 2000 tokens, the model likely needs a config override. Qwen3 models use `<think>` blocks that consume output budget — 1638 tokens (the default 80/20 split) is too small.
154
+
155
+
**Fix:** Write or update `~/.config/tinycode/tinycode.jsonc` with a custom provider entry using a unique provider ID (not "vllm" — that collides with auto-discovery). Set output limit to 4000 and reasoning to true:
156
+
157
+
```json
158
+
{
159
+
"provider": {
160
+
"vllm-qwen3": {
161
+
"npm": "@ai-sdk/openai-compatible",
162
+
"api": "http://<vllm-host>:<port>/v1",
163
+
"models": {
164
+
"<model-id>": {
165
+
"reasoning": true,
166
+
"limit": {
167
+
"context": 4000,
168
+
"output": 4000
169
+
}
170
+
}
171
+
}
172
+
}
173
+
},
174
+
"model": "vllm-qwen3/<model-id>"
175
+
}
176
+
```
177
+
178
+
#### b. Auto-discovery overriding user config
179
+
If a provider ID in `tinycode.jsonc` matches the auto-discovered provider ID (e.g., both use "vllm"), auto-discovery overwrites the user's limits and capabilities. The fix is to use a unique provider ID like "vllm-qwen3" or "vllm-custom" in the config.
180
+
181
+
#### c. Context window too large for available memory
182
+
If the auto-detected context limit is much larger than the model can actually serve (causes OOM or timeouts), set an explicit limit. For Qwen3-30B on a single GPU with 8K max_model_len, use context=4000 output=4000.
183
+
184
+
#### d. Reasoning capability not detected
185
+
vLLM may not advertise the "thinking" capability for models that support `<think>` blocks. If the model name contains "qwen3", "deepseek", or other known reasoning models, set `reasoning: true` in the config override.
186
+
187
+
**After fixing:** Tell the user to restart tinycode or refresh the browser. The model will appear with correct limits under the custom provider ID.
0 commit comments