Add Apple Silicon (Metal) support - #2
Merged
Merged
Conversation
Dyno was NVIDIA-only. The bench/tune/report pipeline is already backend-agnostic (llama-bench runs on Metal, -ngl/-fa/-ctk/-ctv/-t all valid), so the only hard lock-in was GPU/VRAM detection. - detect._apple_silicon_gpu(): on Darwin+arm64, report the chip name and unified memory (≈ total RAM) as the GPU memory pool; wired into detect_hardware() as a fallback when no NVIDIA GPU is found (NVIDIA result never clobbered) - tune._detect_vram_mib(): fall back to Apple unified memory after NVIDIA paths - detect table labels the row "Unified memory" for Apple Silicon - New tests/test_detect.py covers the Apple path and the NVIDIA regression guard (detect.py previously had no tests) - CI: add macos-latest (arm64) to the matrix — validates the macOS code path - Docs: README + pyproject now list Apple Silicon; removed from out-of-scope Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Adds first-class Apple Silicon (Metal) support by extending hardware detection and VRAM/memory heuristics to treat Apple unified memory as the effective GPU memory pool, plus tests and CI coverage to validate the new path.
Changes:
- Add Apple Silicon GPU + unified memory detection and use it as a fallback in
detect_hardware(). - Extend the tuner’s VRAM detection to fall back to Apple unified memory when NVIDIA detection isn’t available.
- Add Apple-focused detection tests and expand CI to run on
macos-latest.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tests/test_detect.py | Adds unit tests covering Apple Silicon detection and NVIDIA-preference regression. |
| src/llama_dyno/tune.py | Extends VRAM detection to fall back to Apple unified memory. |
| src/llama_dyno/detect.py | Implements Apple Silicon detection and wires it into the hardware fingerprint fallback. |
| src/llama_dyno/cli.py | Updates dyno detect display label to show “Unified memory” on Apple Silicon. |
| README.md | Updates docs to include Apple Silicon prerequisites and support status. |
| pyproject.toml | Updates package description/keywords to include Apple Silicon/Metal. |
| .github/workflows/ci.yml | Adds macOS to CI matrix. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+270
to
+271
| # No NVIDIA GPU found — fall back to Apple Silicon (unified memory) if present. | ||
| if gpu_name == "Unknown" or vram == 0: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stage 1 of the usefulness roadmap (Apple Silicon → Ollama → AMD → LM Studio). Makes Dyno work on Macs, not just NVIDIA.
Why this is small
The bench/tune/report pipeline is already backend-agnostic —
llama-benchruns on Metal builds and-ngl/-fa/-ctk/-ctv/-tare all valid there. CPU/RAM detection already handled Darwin. The only hard NVIDIA lock-in was GPU/VRAM detection.Changes
detect._apple_silicon_gpu()— on Darwin + arm64, reports the chip (sysctl machdep.cpu.brand_string, e.g. "Apple M2 Max") and unified memory (≈ total RAM) as the GPU memory pool. Wired intodetect_hardware()as a fallback only when no NVIDIA GPU is found — a real NVIDIA result is never clobbered.tune._detect_vram_mib()— falls back to Apple unified memory after the NVIDIA paths. Search logic unchanged (on Metal, OOM is rare so the sweep naturally settles onngl=99).dyno detectlabels the memory row "Unified memory" on Apple Silicon.tests/test_detect.py(detect.py had none): Apple path on arm64,Noneon Linux/Intel-mac, thedetect_hardwarefallback, and a regression guard proving NVIDIA is preferred.macos-latest(arm64) to the matrix; validates the macOS code path on a real runner.Tests
39 passing (was 34). Metal bench itself is verified manually on a Mac (no Metal GPU in CI).
🤖 Generated with Claude Code