You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
ROADMAP ultraworkers#124: --model accepts any string with zero validation; typos silently pass through; empty string accepted; status JSON has no model provenance
Dogfooded 2026-04-18 on main HEAD bb76ec9 from /tmp/cdAA2.
--model flag has zero validation:
claw --model sonet status → model:'sonet' (typo passthrough)
claw --model '' status → model:'' (empty accepted)
claw --model garbage status → model:'garbage' (any string)
Valid aliases do resolve:
sonnet → claude-sonnet-4-6
opus → claude-opus-4-6
Config aliases also resolve via resolve_model_alias_with_config
But unresolved strings pass through silently. Typo 'sonet'
becomes literal model ID sent to API → fails late with
'model not found' after full context assembly.
Compare:
--reasoning-effort: validates low|medium|high. Has guard.
--permission-mode: validates against known set. Has guard.
--model: no guard. Any string.
--base-commit: no guard (ultraworkers#122). Same pattern.
status JSON:
{model: 'sonet'} — shows resolved name only.
No model_source (flag/config/default).
No model_raw (pre-resolution input).
No model_valid (known to any provider).
Claw can't distinguish typo from exact model from alias.
Trace:
main.rs:470-480 --model parsing:
model = value.clone(); index += 2;
No validation. Raw string stored.
main.rs:1032-1046 resolve_model_alias_with_config:
resolves known aliases. Unknown strings pass through.
main.rs:~4951 status JSON builder:
reports resolved model. No source/raw/valid fields.
Fix shape (~65 lines):
- Reject empty string at parse time
- Warn on unresolved aliases with fuzzy-match suggestion
- Add model_source, model_raw to status JSON
- Add model-validity check to doctor
- Regression per failure mode
Joins ultraworkers#105 (4-surface model disagreement) — model pair:
ultraworkers#105 status ignores config model, doctor mislabels
ultraworkers#124 --model flag unvalidated, no provenance in JSON
Joins ultraworkers#122 (--base-commit zero validation) — unvalidated-flag
pair: same parser pattern, no guards.
Joins Silent-flag/documented-but-unenforced as 17th.
Joins Truth-audit — status model field has no provenance.
Joins Parallel-entry-point asymmetry as 10th.
Filed in response to Clawhip pinpoint nudge 1495000973914144819
in #clawcode-building-in-public.
**Blocker.** None. Localized in `rust/crates/tools/src/lib.rs:370` + status/doctor JSON plumbing.
4407
4407
4408
4408
**Source.** Jobdori dogfood 2026-04-18 against `/tmp/cdZZ` on main HEAD `2bf2a11` in response to Clawhip pinpoint nudge at `1494993419536306176`. Joins **Silent-flag / documented-but-unenforced** (#96–#101, #104, #108, #111, #115, #116, #117, #118, #119, #121, #122) as 16th member — `--allowedTools` has undocumented whitespace-separator behavior, undocumented normalization asymmetry, and silent duplicate-acceptance. Joins **Permission-audit / tool-allow-list** (#94, #97, #101, #106, #115, #120) as 7th — asymmetric normalization means claw allow-lists don't round-trip cleanly between canonical representations. Joins **Truth-audit / diagnostic-integrity** — status/doctor JSON hides what the allowed-tools set actually is. Joins **Parallel-entry-point asymmetry** (#91, #101, #104, #105, #108, #114, #117, #122) as 9th — `--allowedTools` vs `.claw.json permissions.allow` are two entry points that likely disagree on normalization (worth separate sweep). Natural bundle: **#97 + #123** — `--allowedTools` trust-gap pair: empty silently blocks (#97) + asymmetric normalization + invisible runtime state (#123). Also **Flagship permission-audit sweep 8-way (grown)**: #50 + #87 + #91 + #94 + #97 + #101 + #115 + **#123**. Also **Permission-audit 7-way (grown)**: #94 + #97 + #101 + #106 + #115 + #120 + **#123**. Session tally: ROADMAP #123.
4409
+
4410
+
124. **`--model` accepts any string with zero validation — typos like `sonet` silently pass through to the API where they fail late with an opaque error; empty string `""` is silently accepted as a model name; `status` JSON shows the resolved model but not the user's raw input, so post-hoc debugging of "why did my model flag not work?" requires re-reading the process argv** — dogfooded 2026-04-18 on main HEAD `bb76ec9` from `/tmp/cdAA2`.
4411
+
4412
+
**Concrete repro.**
4413
+
```
4414
+
# Typo alias silently passed through:
4415
+
$ claw --model sonet --output-format json status | jq .model
4416
+
"sonet"
4417
+
# No warning that "sonet" is not a known alias or model.
4418
+
# At prompt time this would fail with "model not found" from the API.
4419
+
4420
+
# Empty string accepted:
4421
+
$ claw --model '' --output-format json status | jq .model
4422
+
""
4423
+
# Empty model string silently accepted.
4424
+
4425
+
# Garbage string:
4426
+
$ claw --model 'totally-not-a-real-model-xyz123' --output-format json status | jq .model
4427
+
"totally-not-a-real-model-xyz123"
4428
+
# No validation. Any string accepted.
4429
+
4430
+
# Valid aliases do resolve:
4431
+
$ claw --model sonnet --output-format json status | jq .model
4432
+
"claude-sonnet-4-6"
4433
+
$ claw --model opus --output-format json status | jq .model
Resolves aliases at CliAction construction time. If the string matches a known alias (`sonnet` → `claude-sonnet-4-6`), it resolves. If not, the raw string passes through unchanged.
4460
+
- `claw status` JSON builder at `main.rs:~4951` reports the resolved `model` field. No `model_source` (flag/config/default), no `model_raw` (pre-resolution input), no `model_valid` (whether known to any provider).
4461
+
- At Prompt execution time (with real credentials), the model string is sent to the API. An unknown model fails with `"model not found"` or equivalent provider error. The failure is late (after system prompt assembly, context building, etc.) and carries the model ID in an API error message — not in a pre-flight check.
4462
+
4463
+
**Why this is specifically a clawability gap.**
4464
+
1. *Typo → late failure.* `claw --model sonet -p "do work"` assembles the full context, sends to API, gets rejected. Billable token overhead if the provider charges for failed requests (some do). At minimum, wasted local compute for prompt assembly.
4465
+
2. *No pre-flight check.* `claw --model unknown-model status` succeeds with exit 0. A claw preflighting with `status` cannot detect that the model is bogus until it actually makes an API call.
4466
+
3. *Empty string accepted.* `--model ""` is a runtime bomb: the model string is empty, and the API request will fail with a confusing "model is required" or similar empty-field error.
4467
+
4. *`status` JSON doesn't show model provenance.* A claw reading `{model: "sonet"}` can't tell if the user typed `sonet` (typo), if it's a config alias that resolved to `sonet`, or if it's the default. No `model_source: "flag"|"config"|"default"` field.
4468
+
5. *Joins #105 (4-surface model disagreement).* #105 said `status` ignores `.claw.json` model, doctor mislabels aliases. **#124** adds: `--model` flag input isn't validated or provenance-tracked, so the model field in status is unverifiable from outside.
4469
+
6. *Joins #122 (`--base-commit` zero validation)* — same parser pattern: flag takes any string, stores raw, no validation. `--model` and `--base-commit` are sibling unvalidated flags.
4470
+
7. *Compare `--reasoning-effort`* at `main.rs:498-510` — validates `"low"|"medium"|"high"`. Has a guard. `--model` has none.
4471
+
8. *Compare `--permission-mode`* — validates against known set. Has a guard. `--model` has none.
4472
+
4473
+
**Fix shape — validate at parse time or preflight, surface provenance.**
4474
+
1. *Reject obviously-bad values at parse time.* Empty string: error immediately. Starts with `-`: probably swallowed flag (per #122 pattern). ~5 lines.
4475
+
2. *Warn on unresolved aliases.* If `resolve_model_alias_with_config(input) == input` (no resolution happened) AND input doesn't look like a full model ID (no `/` for provider-prefixed, no `claude-` prefix, no `openai/` prefix), emit a structured warning: `"model '{input}' is not a known alias; it will be sent as-is to the provider. Did you mean 'sonnet'?"`. Use fuzzy match against known aliases. ~25 lines.
4476
+
3. *Add `model_source` and `model_raw` to status JSON.* `model_source: "flag"|"config"|"default"`, `model_raw: "<what the user typed>"`, `model_resolved: "<after alias resolution>"`. A claw can verify provenance. ~15 lines.
4477
+
4. *Add model-validity check to `doctor`.* Doctor already has an `auth` check. Add a `model` check: given the resolved model string, check if it matches known Anthropic/OpenAI model patterns. Emit `warn` if not. ~20 lines.
4478
+
5. *Regression tests.* (a) `--model ""` → parse error. (b) `--model sonet` → structured warning with "Did you mean 'sonnet'?". (c) `--model sonnet` → resolves silently. (d) Status JSON has `model_source: "flag"` + `model_raw: "sonnet"` + `model: "claude-sonnet-4-6"`. (e) Doctor model check warns on unknown model.
4479
+
4480
+
**Acceptance.** `claw --model sonet status` emits a structured warning about the unresolved alias and suggests correction. `claw --model '' status` fails at parse time. Status JSON includes `model_source` and `model_raw`. Doctor includes a model-validity check.
4481
+
4482
+
**Blocker.** None. Localized across parse + status JSON + doctor check.
4483
+
4484
+
**Source.** Jobdori dogfood 2026-04-18 against `/tmp/cdAA2` on main HEAD `bb76ec9` in response to Clawhip pinpoint nudge at `1495000973914144819`. Joins **Silent-flag / documented-but-unenforced** (#96–#101, #104, #108, #111, #115, #116, #117, #118, #119, #121, #122, #123) as 17th — `--model` silently accepts garbage with no validation. Joins **Truth-audit / diagnostic-integrity** — status JSON model field has no provenance. Joins **Parallel-entry-point asymmetry** (#91, #101, #104, #105, #108, #114, #117, #122, #123) as 10th — `--model` flag, `.claw.json model`, and the default model are three sources that disagree (#105 adjacent). Natural bundle: **#105 + #124** — model-resolution pair: 4-surface disagreement (#105) + no validation + no provenance (#124). Also **#122 + #124** — unvalidated-flag pair: `--base-commit` accepts anything (#122) + `--model` accepts anything (#124). Same parser pattern. Session tally: ROADMAP #124.
0 commit comments