Skip to content

MLXChatExample: register Gemma 4 (E2B / E4B) VLMs#473

Open
john-rocky wants to merge 1 commit into
ml-explore:mainfrom
john-rocky:feat/gemma4-chatexample
Open

MLXChatExample: register Gemma 4 (E2B / E4B) VLMs#473
john-rocky wants to merge 1 commit into
ml-explore:mainfrom
john-rocky:feat/gemma4-chatexample

Conversation

@john-rocky
Copy link
Copy Markdown

@john-rocky john-rocky commented May 1, 2026

Summary

Registers the two smaller Gemma 4 VLM checkpoints already published in `VLMRegistry` (`gemma4_E2B_it_4bit`, `gemma4_E4B_it_4bit`) in MLXChatExample's `availableModels`, so they appear alongside the existing qwen / smol VLMs in the model picker on macOS and iOS. The 31B and 26BA4B variants are intentionally left out — too large to be practical on supported iOS devices.

Depends on (or coexists with) ml-explore/mlx-swift-lm#256 for video support; image and text input work today against any Gemma 4 mlx-swift-lm release that ships the model registration (already merged via #180 / #185).

Verification

  • ✅ `xcodebuild MLXChatExample` builds for macOS arm64, generic iOS Simulator, and generic iOS Device.
  • On-device iOS verified — iPhone 17 Pro, `gemma4:E2B` selected from the picker, 8 s video → 42.53 tok/s decode, transcript matched the video content. (Run against a build that includes Add Gemma 4 video tower support mlx-swift-lm#256.)

Test plan

  • macOS / iOS Simulator / iOS device all build.
  • On-device run on iPhone 17 Pro produces a coherent transcript for an 8 s video clip at 42.53 tok/s.

Adds the two smaller `gemma4` checkpoints already published in
`VLMRegistry` (`gemma4_E2B_it_4bit`, `gemma4_E4B_it_4bit`) to the
chat app's `availableModels`, so they appear alongside the existing
qwen / smol VLMs in the model picker on macOS and iOS. The 31B and
26BA4B variants are intentionally left out — they are too large to
be practical on the supported iOS devices.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant