Skip to content

Space on ZeroGPU; publish granite-qwen-500-3 as mazesmazes/tiny-audio - #173

Merged
alexkroman merged 1 commit into
mainfrom
space-zerogpu-granite-qwen-500-3
Oct 23, 2026
Merged

alexkroman merged 1 commit into
mainfrom
space-zerogpu-granite-qwen-500-3

Conversation

@alexkroman

Copy link
Copy Markdown
Owner

Summary

Space: ZeroGPU instead of RunPod

  • The Space now runs the model in-process on ZeroGPU. demo/app.py already had the @spaces.GPU path, so this PR adds the model's dependencies back to demo/requirements.txt and puts suggested_hardware: zero-a10g plus preload_from_hub (every model the pipeline loads) in the Space README.
  • ta deploy without --endpoint-url now deletes any leftover ENDPOINT_URL variable (otherwise the app keeps calling the server) and requests zero-a10g. --endpoint-url still works as before: CPU hardware plus a ta serve server.
  • Adds librosa to the Space and Hub requirements. Nemotron's streaming feature extractor imports it, so diarization failed without it.
  • Adds merge_lora, which folds LoRA adapters into the decoder weights at load (20.6 → 36.7 tok/s on a Mac). It does nothing for the new fully fine-tuned model; it's there for LoRA models set via MODEL_ID.

Model card: granite-qwen-500-3 is now mazesmazes/tiny-audio

  • Hub main (already published, commit 26ce2edf) is a copy of tiny-audio-granite-qwen-500-3@989df838: the Granite Speech 5.0 470M NC encoder plus a fully fine-tuned Qwen3-0.6B, at step 16,159. config.json has model_dtype set to bfloat16 to match how evals run it. The previous model is tagged granite-qwen-frozen-4.
  • The license is now CC BY-NC-SA 4.0, inherited from the -nc encoder. The card points to granite-qwen-frozen-4 (MIT) for commercial use.
  • There's no benchmark table yet, because this model hasn't had an n=1000 eval. It needs filling in after one.
  • The course docs now describe deploying the Space on ZeroGPU.

After merge

poetry run ta deploy --repo-id mazesmazes/tiny-audio

Test plan

  • ta dev check and ta dev test pass (1592 passed); the precommit gate passed
  • A fresh pipeline("mazesmazes/tiny-audio", trust_remote_code=True) load from the Hub runs in bf16 and transcribes the LibriSpeech and earnings-call demo examples correctly
  • Deploy the Space and try transcription, timestamps and diarization on ZeroGPU

🤖 Generated with Claude Code

Space
- Runs the model in-process on ZeroGPU instead of calling the RunPod
  `ta serve` pod. app.py already had the @spaces.GPU path; requirements
  get the model's dependencies back, and the README front matter gets
  suggested_hardware: zero-a10g and preload_from_hub for every model the
  pipeline loads.
- `ta deploy` without --endpoint-url now clears a leftover ENDPOINT_URL
  variable (it would keep the app calling the server) and requests
  zero-a10g hardware. --endpoint-url still moves the Space to CPU and a
  `ta serve` server.
- librosa added to the Space and Hub requirements: Nemotron's streaming
  feature extractor imports it, so diarization failed without it.
- merge_lora folds a LoRA decoder's adapters into its weights at load
  (20.6 -> 36.7 tok/s on a Mac, see merge_lora_adapters). A no-op for
  the new fully fine-tuned model; kept for LoRA models via MODEL_ID.

Model card
- mazesmazes/tiny-audio main is now tiny-audio-granite-qwen-500-3@989df838
  (Granite Speech 5.0 470M NC encoder + fully fine-tuned Qwen3-0.6B,
  step 16,159), with config.json's model_dtype set to bfloat16 to match
  how evals run it. Previous model tagged granite-qwen-frozen-4.
- License is CC BY-NC-SA 4.0, inherited from the -nc encoder; the card
  points to granite-qwen-frozen-4 (MIT) for commercial use.
- No benchmark table yet: there is no n=1000 eval of this model.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@alexkroman
alexkroman merged commit 60ce5fc into main Oct 23, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants