Skip to content

Fix MPS crashes: float64 swap scores and guidance map resize - #69

Merged
hvoss-tech merged 2 commits into
hvoss-tech:mainfrom
IsraelShok:fix/mps-float64-swap-constraint
Nov 18, 2026
Merged

hvoss-tech merged 2 commits into
hvoss-tech:mainfrom
IsraelShok:fix/mps-float64-swap-constraint

Conversation

@IsraelShok

@IsraelShok IsraelShok commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #68.
Fixes #70.

Two MPS-only crashes on Apple Silicon. Both come from ops MPS doesn't support, and both are fixed by doing a small one-off step on the CPU.

1. float64 cast with "Swaps: At most" (#68)

Problem

With Swaps: At most enabled on mps, the optimization crashes with Cannot convert a MPS Tensor to float64 dtype. score.detach().to("cpu", torch.float64) attempts the float64 cast on the MPS device, which has no float64.

Change

In src/autoforge/Helper/ConstraintHelper.py, copy to the host first, then cast:

- score.detach().to("cpu", torch.float64)
+ score.detach().cpu().to(torch.float64)

Applied in both call sites (_viterbi_swaps and project_assignment). No behavior change on CPU/CUDA.

2. "Adaptive pool MPS" in height guidance pixel targets (#70)

Problem

Once the Depth Anything model is available, the WebUI adds default depth height guidance ({"strength": 1.0, "depth": True}). On mps, every such run crashes at the first discrete check:

RuntimeError: Adaptive pool MPS: input sizes must be divisible by output sizes. Non-divisible input sizes are not implemented on MPS device yet.

HeightGuidance.maps() shrinks the full-resolution target and weight maps to the solving size with F.interpolate(mode="area"), which is adaptive_avg_pool2d. MPS supports it only when the input size is an exact multiple of the output size (pytorch/pytorch#96056), and these sizes usually aren't. Offline without the model cached, the default guidance is dropped silently, which hides the crash.

Change

In src/autoforge/Helper/HeightGuidance.py (maps()), when downsizing with area on an MPS tensor, interpolate on the CPU and move the result back to the device. This follows the workaround already in Optimizer.py (around line 812). maps() caches each shape, so the CPU step runs once per size, not every iteration. No behavior change on CPU/CUDA, and upsampling (bilinear) is unchanged.

Verification

  • _viterbi_swaps on a random mps score tensor runs and returns the expected shape (previously raised the float64 TypeError).
  • Other float64 call sites (DeterministicOps.segment_gather, _cluster._segment_sums) are already guarded by DeviceUtils.has_float64.
  • New test test_maps_downsize_to_non_divisible_shape_on_mps (skipped without MPS) shrinks a 1000×750 map to 333×250 on mps. It failed with the Adaptive pool MPS error before the change. It now passes, and the result matches the CPU result within 1e-5.
  • tests/test_height_guidance.py: 25 passed, 4 skipped on Apple Silicon (torch 2.12.0).
  • No other mode="area" / adaptive_avg_pool uses remain in src/autoforge outside the already-guarded Optimizer.py path.

🤖 Generated with Claude Code

IsraelShok and others added 2 commits November 18, 2026 12:52
On MPS, .to("cpu", torch.float64) casts on the device first, and MPS has
no float64, so setting "Swaps: At most" crashed with "Cannot convert a
MPS Tensor to float64 dtype". Move to the CPU first, then cast.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MPS implements "area" interpolation (adaptive_avg_pool2d) only when the
input size is an exact multiple of the output size. HeightGuidance.maps
shrinks the depth target to the solving size, which usually doesn't divide
evenly, so every MPS run with depth guidance crashed at the first discrete
check. Do that resize on the CPU and move the result back; maps() caches
each shape, so it runs once per size.

Fixes hvoss-tech#70.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@IsraelShok IsraelShok changed the title Fix MPS float64 crash with the swap-limit constraint Fix MPS crashes: float64 swap scores and guidance map resize Nov 18, 2026
@hvoss-tech

Copy link
Copy Markdown
Owner

Thank you very much for the PR :)

@hvoss-tech
hvoss-tech merged commit 421a5da into hvoss-tech:main Nov 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants