Feat(model support): ideogram4 support

[Pfannkuchensack]

Summary

Adds first-class Ideogram 4 (text-to-image) support to InvokeAI — a new open-weight 9.3B single-stream DiT with a Qwen3-VL-8B text encoder and flow-matching sampler.

The defining trait of this model is that it is trained on a structured JSON prompt that describes the scene as a list of regions, each with a bounding box ([y_min, x_min, y_max, x_max], normalized 0–1000, origin top-left) and a text description. Plain text works but is markedly lower quality.

The headline feature here is that this JSON is auto-assembled on the frontend from the existing Canvas Regional Guidance layers: the global prompt becomes the overall description, and each enabled region contributes one element (its drawn rect → bbox, its prompt → description). Users can also paste raw JSON to drive the model directly.

Why this is purely frontend string assembly: Ideogram 4 does not use spatial attention masks for regions (unlike FLUX/SDXL/Z-Image regional guidance) — the region boxes are encoded as text inside the single JSON string fed to Qwen3-VL. So the backend only ever sees one prompt string; no mask-conditioning code is touched.

How

Backend (invokeai/backend/ideogram4/, vendored from the Apache-2.0 reference, copyright headers retained):

• DiT (modeling_ideogram4.py), FLUX2-style KL VAE (autoencoder.py + latent_norm.py), logit-normal flow-match scheduler + presets (scheduler.py, sampler_configs.py), nf4/fp8 quantized loading, and InvokeAI-side denoise.py / text_encoding.py / sampling_utils.py.
• Dual-branch asymmetric CFG: positive runs the conditional transformer over [text]+[image] tokens, negative runs the unconditional transformer over image-only tokens with zeroed LLM features (v = gw·pos + (1−gw)·neg). ⇒ no negative prompt. A transformer_pair.py wrapper keeps both transformers co-resident through the loop so the cache doesn't swap them every step (nf4 ≈ 10 GB resident during denoise; fits 24 GB).
• Model-manager registration: new BaseModelType.Ideogram4, a Qwen3-VL text-encoder type, config detector + diffusers config, and a loader mirroring Z-Image.
• Four invocations: ideogram4_model_loader, ideogram4_text_encoder, ideogram4_denoise, ideogram4_latents_to_image; new Ideogram4ConditioningInfo + field/output; ideogram4_txt2img generation mode.

Frontend:

• buildIdeogram4Prompt.ts — Regions→JSON assembly (raw-JSON passthrough; stable key order; bbox clamped/rounded to 0–1000) with unit tests.
• buildIdeogram4Graph.ts — text2img-only graph builder + enqueue wiring. Uses a decoy string node for the assembled JSON so the linear-UI batch injector doesn't clobber it, while plain-text prompts still flow through the real prompt node (so dynamic prompts / batching keep working).
• Params + UI: a Sampler Preset combobox (Quality 48 / Default 20 / Turbo 12) as the primary control, plus Advanced overrides that actually apply to this model — Steps, Guidance Scale, Schedule Shift (mu) and a Color Palette picker. The irrelevant Advanced controls (VAE, CLIP Skip, CFG Rescale, Seamless, Color Compensation) are hidden for Ideogram 4.
• Metadata recall for all of the above.

Dependencies: bumps transformers to >=5.5,<5.6 (Qwen3-VL landed in 4.57; the encoder needs it) and compel to >=2.4.0,<3, with the necessary adaptations to the FLUX / Z-Image loaders, the safety checker, the HF metadata fetcher and model_util.

Out of scope (v1): img2img / inpaint / outpaint, ControlNet / IP-Adapter / LoRA, and the optional local "Magic Prompt" plain-text→JSON expander (parked — see Merge Plan).

Related Issues / Discussions

• Overlaps the transformers 5.x bump tracked in PR feat - Migrate to Transformers 5.5.4 #9248 — this branch carries the same bump (>=5.5,<5.6) plus the cross-model adaptations it requires.

QA Instructions

Requires the gated weights (ideogram-ai/ideogram-4-nf4 — nf4 is the 24 GB path, CUDA/bitsandbytes only) plus the Qwen3-VL encoder + VAE sub-dependencies.

1. Install & select an Ideogram 4 model; open the Canvas/Generate tab. Confirm the model shows under its own group, dimensions default to 1024×1024 (multiples of 16), and the Generation settings show the Sampler Preset control instead of Scheduler/CFG.
2. Regions → JSON: type an overall description, add 1–2 Regional Guidance layers each with a prompt + a drawn box, and Invoke. In the result's metadata, confirm the assembled JSON has the correct key order and elements[*].bbox (0–1000, [y_min, x_min, y_max, x_max]) matching where you drew the boxes, and that element placement in the image roughly matches.
3. Raw-JSON passthrough: paste a hand-written JSON object into the prompt box → it is sent unchanged (no region wrapping).
4. Plain text: with no regions and a plain prompt, confirm normal text2img still works and that dynamic prompts / batching still expand (decoy-node behavior).
5. Sampler presets: Quality / Default / Turbo produce the expected step counts.
6. Advanced overrides: Steps / Guidance Scale / mu show the active preset's value as the "auto" default and can be overridden + reset; Color Palette swatches inject style_description.color_palette (auto-build mode only — ignored for raw JSON).
7. Metadata recall: load an Ideogram 4 image and recall — preset, overrides and palette restore (and are guarded to only recall onto an Ideogram 4 model).

Frontend gates (from invokeai/frontend/web/): pnpm lint and pnpm test:no-watch (includes buildIdeogram4Prompt.test.ts). tsc clean.

Merge Plan

• Large PR + dependency bump. This raises transformers to >=5.5,<5.6 and compel to >=2.4.0,<3, which touches many models (FLUX, Z-Image, safety checker, HF metadata fetch). Coordinate with / sequence after PR feat - Migrate to Transformers 5.5.4 #9248 (the transformers 5.x bump) to avoid a double-bump conflict, and time it to not collide with a pending release. Broad regression QA across existing model types is warranted, not just Ideogram 4.
• Follow-ups (not in this PR): GGUF loader for the custom DiT (more VRAM headroom), starter_models.py entries, and the optional local Magic Prompt node (blocked on the upstream system-prompts PR feat: add System Prompts library for Expand Prompt button #9152, whose migration_32 collides with main and must be renumbered to 33 first).

Checklist

• The PR has a short but descriptive title, suitable for a changelog
• Tests added / updated (if applicable) — frontend unit tests for the Regions→JSON assembly
• ❗Changes to a redux slice have a corresponding migration — new params fields use zod defaults; confirm if a migration is needed
• Documentation added / updated (if applicable)
• Updated What's New copy (if doing a release after this PR)