Ornith-1.5: validate end-to-end support + plan Qwen3.5 vision

[pekkah]

Full plan: docs/ornith-15-plan.md (branch claude/ornith-1-5-variants-acu88x). Follow-up to #411.

Background

Ornith-1.5 (ornith-ai, MIT, released 2026-08-18) is not a new architecture, and not even a new architecture variant — it is a second-generation self-improvement RL post-train of the same Qwen3.5 bases as Ornith-1.0.

ornith-ai/Ornith-1.5-9B/config.json and deepreinforce-ai/Ornith-1.0-9B/config.json agree on every architectural field: Qwen3_5ForConditionalGeneration, 32 layers, hidden 4096, 16 heads / 4 KV heads, head_dim 256, intermediate_size 12288, full_attention_interval 4, linear_conv_kernel_dim 4, linear_num_key_heads 16 / linear_num_value_heads 32, partial_rotary_factor 0.25, rope_theta 1e7, mrope_section [11,11,10], mtp_num_hidden_layers 1, vocab 248320, ctx 262144. Only transformers_version and the weights differ.

Ornith-1.5-35B-A3B (40 layers, hidden 2048, 16/2 heads, 256 experts / 8 active, moe_intermediate_size 512, shared_expert_intermediate_size 512) matches the hyperparameters Ornith10ArchitectureTests.Ornith35BMoe_RoutesToHybridSsmMoEPath already pins, exactly.

So all three text variants should run on the paths #411 validated, with no code changes. The genuinely new thing is that ornith-ai publishes first-party GGUF and mmproj vision projectors for every size, making the Qwen3.5 vision tower (deferred as out-of-scope in #411) reachable in GGUF form for the first time.

Variant matrix

Variant	HF arch	GGUF arch (expected)	SharpInference path	Q4_K_M	Verdict
Ornith-1.5-9B	qwen3_5	qwen35	HybridGdnForwardPass / CudaHybridGdnForwardPass / VulkanHybridGdnForwardPass — 24 GDN + 8 full-attn of 32	5.63 GB	Supported today. Primary target; -g -1 fits 8 GB VRAM (measured for 1.0-9B, same shape)
Ornith-1.5-35B-A3B	qwen3_5_moe	qwen35moe	same hybrid GDN path + MoE FFN, incl. --cpu-moe	21.7 GB	Supported today, unvalidated on real weights (inherits #411's open 35B item)
Ornith-1.5-397B	qwen3_5_moe	qwen35moe	same, only via SharpInference.Pipeline VRAM→RAM→NVMe tiering	240 GB (single file)	Supported in principle; not a practical target
MTP community quants (protoLabsAI/Ornith-1.5-9B-MTP-GGUF, mudler/…-APEX-MTP-GGUF, SC117/…-MTP-APEX-GGUF)	—	+ nextn_predict_layers=1	MtpDecoder (--mtp)	Should work unmodified; unvalidated
Vision (mmproj-Ornith-1.5-*-BF16.gguf, ~900 MB)	qwen3_5_vision	clip	—	—	Not supported — see Phase 4

Chat template is the Qwen3.6 XML wire format (<tool_call><function=…><parameter=…>) with enable_thinking and a pre-opened <think> — already handled by QwenToolCallAdapter (registered for both qwen35 and qwen35moe) and our ChatTemplate / --no-thinking plumbing. Recommended sampling (temp 0.6–1.0, top_p 0.95, top_k 20, min_p 0, presence_penalty 1.5) maps 1:1 onto existing Sampler knobs.

Not verified

The research session's egress policy blocks huggingface.co for direct download (CONNECT … 403), so the real GGUF metadata was not inspected. The arch strings above are inferred from the identical 1.0 configs plus #411's confirmation that the real 1.0-9B GGUF reports general.architecture = qwen35 with GDN tensors present. Phase 1 starts by confirming this.

Phase 0 — Plumbing (½ day, no weights needed)

• scripts/download-model.ps1: add ornith15-9b (Q4_K_M, 5.63 GB) and ornith15-35b (Q4_K_M, 21.7 GB) presets pointing at ornith-ai/*-GGUF — extend the ValidateSet, set SizeGB so the free-disk guard fires
• Rename Ornith10ArchitectureTests → OrnithArchitectureTests and fix the 9B fixture: it currently uses synthetic 48 layers / 32 heads / key_length 128, whereas the real 9B is 32 layers / 16 heads / 4 KV / key_length 256 / rope.dimension_count 64. Add a 1.5-9B case asserting 24 GDN + 8 attention layers
• CLAUDE.md + .claude/skills/run-models/SKILL.md: fold 1.5 into the Ornith section (same path as 1.0, new org/URLs, vision still unsupported)

Phase 1 — Validate 9B on real weights (1 day, needs network + GPU)

• list-metadata / list-tensors on Ornith-1.5-9B-Q4_K_M.gguf — confirm general.architecture, nextn_predict_layers, GDN tensor presence, and whether any mrope/imrope keys appear (record the dump in the plan doc)
• Greedy cross-check vs llama.cpp at --temp 0 (parity-check level 2), then perplexity (level 4)
• Tool-call round trip through QwenToolCallAdapter; server smoke test with enable_thinking:false

Phase 2 — Validate 35B-A3B (1 day, needs a big-RAM box)

• Same as Phase 1 on the MoE path, plus --cpu-moe expert offload (also closes the 35B item left open in Ornith-1.0: validate end-to-end support (Qwen3.5/Gemma4 agentic-coding finetunes) #411)

Phase 3 — MTP quants (1–2 days)

• Run protoLabsAI/Ornith-1.5-9B-MTP-GGUF with --mtp; verify acceptance rate and GdnStateCache rollback. Zero code expected — the head loads when NumMtpLayers > 0 and blk.{NumLayers}.nextn.* exist. If an APEX repack lays the MTP block out differently, document rather than special-case

Phase 4 — Qwen3.5 vision (3–4 weeks; split into its own issue, schedule against #126)

The only real engineering in the family. CPU-first:

• 4a mmproj loader — generalize VisionModel from the hardcoded gemma4uv projector to a projector-type dispatch; read the qwen3_5 ViT config. BF16 tensors already covered by Dequantize.DequantBF16
• 4b preprocessing — reuse ImagePreprocessor.CalcSizePreservedRatio with align = patch_size × spatial_merge = 32 (already the llama.cpp Qwen smart-resize), swap in Qwen normalization constants, duplicate frames for temporal_patch_size = 2
• 4c ViT forward — 27 layers, hidden 1152, 16 heads, intermediate 4304, gelu_pytorch_tanh, full (non-causal) attention, learned position table of 2304 (48×48) interpolated to the actual grid. deepstack_visual_indexes is empty, so no Qwen3-VL deepstack merge — a plain ViT
• 4d merger — 2×2 spatial merge → MLP → out_hidden_size (4096 for 9B, 2048 for 35B) → soft tokens for <|image_pad|> slots
• 4e interleaved M-RoPE — per-token (t,h,w) triples, sections [11,11,10] over the 64 rotated dims. Touches position bookkeeping in the KV cache and rope kernels in all four forward passes (CPU, Vulkan, CUDA, hybrid GDN siblings). Gate on image tokens being present so the text path stays bit-identical (with equal t/h/w positions M-RoPE degenerates to the NEOX RoPE we already run — which is why text-only works today)
• 4f prompt plumbing — <|vision_start|><|image_pad|><|vision_end|> expansion, CLI --image/--mmproj wiring, server image content parts

Done when soft-token embeddings match llama.cpp mtmd on a fixed image within tolerance and a caption smoke test reads correctly.

Phase 5 — Deferred

• YaRN RoPE scaling (factor: 4.0 over 262144 → ~1M window). Native 262K works today
• 397B (240 GB Q4_K_M) through Pipeline tiering — a scale experiment, not a supported configuration

Risks

#	Risk	Mitigation
R1	GGUF arch string isn't qwen35/qwen35moe (converter may tag VL-wrapped checkpoints differently)	Phase 1 step 1 is list-metadata; everything else is contingent on it
R2	Text GGUF carries M-RoPE metadata we silently ignore	Harmless while t/h/w positions are equal, but confirm in Phase 1 and assert in a test
R3	Phase 4e's rope change regresses text-only numerics across 4 forward passes	Gate on image presence; parity-check level 2 before/after on a text-only prompt
R4	MTP community repacks (APEX) deviate from the nextn.* layout	Document and skip rather than special-case
R5	No local llama.cpp reference for the qwen3_5 vision projector	Confirm the tools/mtmd implementation exists upstream before starting Phase 4