Repository navigation
Granite NC + full fine-tune Qwen3-0.6B recipe, MLX eval, compare cleanup - #171
Merged
Merged
Conversation
…eanup Recipe: back to the Granite Speech 5.0 turboctc-nc encoder, whose CTC-trained output is close to token-shaped, with Qwen3-0.6B fully fine-tuned instead of LoRA. Decoder settings are the old stage_1 recipe's measured ones: decoder lr 2e-5 (chosen from attention-weight drift), embeddings trainable, decoder weight decay 0.01, float32 storage so 2e-5 updates are not rounded away under bf16 (compute stays bf16). Batch 256 x 1 (a measured ~27% of the card at 64), 2 epochs, prefetch 2, eval cap 1000 per dataset. hub_model_id is named for the recipe. The tiny preset pins its LoRA setup explicitly. Serving: `ta serve` loads float32-stored checkpoints in bf16 on CUDA/MPS. flash_attention_2 rejects float32 inputs with no autocast to cast them, and decoding read twice the bytes. DTYPE_CONFIG_FIELDS moves to scripts/inference so serve and eval share one list. MLX: `ta eval --mlx` transcribes a Granite Speech 5.0 + fully fine-tuned Qwen3 checkpoint on Apple silicon (scripts/mlx_inference.py). mlx-audio's Granite encoder port, an MLX twin of MLPAudioProjector, mlx-lm's Qwen3 with the checkpoint's decoder weights, and the PyTorch pipeline's own chunking, lead-in, prompt, 12-gram guard, silence gate and text cleanup. Each run takes the Hub repo's latest revision and converts it once per revision into ~/.cache/tiny-audio/mlx. Per stage in float32 it matches PyTorch to <=3e-5. Optional macOS-only `mlx` dependency group; typings/ stubs keep strict pyright passing on Linux, and conftest skips the MLX tests where MLX is absent. Compare: drops the entity table and the corpus WER bootstrap table. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Tests job failed the coverage gate's 50% new-file floor: MLX was a darwin-only optional group, so CI skipped tests/test_mlx_inference.py and scripts/mlx_inference.py measured 0%. MLX publishes Linux CPU wheels (mlx + mlx-cpu, manylinux_2_35 x86-64), so the group now installs `mlx[cpu]` on Linux and the test job runs `poetry install --with dev,mlx`. Every MLX op the tests use was checked in a linux/amd64 container (features, RMSNorm/ Linear/GELU, the logits mask, Qwen3 generation from embeddings, bf16 safetensors). mlx-audio's sounddevice is never imported on our path. Pods are unaffected: they install `poetry export --only main`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follows #170 (merged). Moves the default recipe from Qwen3-ASR + SmolLM3 to Granite NC + a fully fine-tuned Qwen3-0.6B: the Qwen3-ASR + SmolLM3 run trailed the Granite stack on every dataset (+1.68 corpus WER, CI [+1.22, +2.15], half-trained).
Default recipe
ibm-granite/granite-speech-5.0-470m-turboctc-nc, frozen (CC-BY-NC-SA).Qwen/Qwen3-0.6B, fully fine-tuned incl. embeddings, at the old stage_1 recipe's measured settings: decoder lr 2e-5, decoder weight decay 0.01, float32 storage so 2e-5 updates are not rounded away under bf16 (compute stays bf16 via autocast).hub_model_id: mazesmazes/tiny-audio-granite-qwen3-0.6b.tinylaptop preset pins its LoRA setup explicitly.Serving fix
ta serveloads float32-stored checkpoints in bf16 on CUDA/MPS; flash_attention_2 rejects float32 inputs without autocast, and decoding read twice the bytes.DTYPE_CONFIG_FIELDSnow lives inscripts/inference.py, shared by serve and eval.MLX evaluation (
ta eval --mlx)scripts/mlx_inference.py): mlx-audio's Granite encoder, an MLX twin ofMLPAudioProjector, mlx-lm's Qwen3, plus the PyTorch pipeline's own chunking, lead-in, prompt, 12-gram guard, silence gate and text cleanup.~/.cache/tiny-audio/mlx. Results are saved as<model>-mlx.mlxdependency group (lock adds packages only);typings/stubs keep strict pyright passing on Linux (with a.gitignoreexception for theirmodels/dirs); the MLX tests are skipped where MLX is absent.ta analysis compareTest plan
ta dev check(29 gates),ta dev test(suite + coverage floors), precommit hook.ta eval --mlxend to end on a checkpoint with the full-fine-tune layout.ta eval --mlxvs the PyTorch eval on the same clips once the training run's checkpoint is pushed.production.yaml.Known follow-ups (from #170's code review, still open)
asr_modeling.pyimports the Qwen3-ASR module unconditionally; make it optional like the Granite import.audio_model_idinstead of YAML.chunk_max_seconds.pack_sequencesin the config schema.🤖 Generated with Claude Code