Resize a symbolic input to a fixed output size with the iOS17 resize op

[LeSingh1]

Problem

F.interpolate(x, size=(H, W), mode="bilinear") — resize to a fixed output size — aborts conversion as soon as the input's spatial dims are flexible:

import torch, torch.nn as nn, torch.nn.functional as F, coremltools as ct

class M(nn.Module):
    def forward(self, x):
        return F.interpolate(x, size=(4, 4), mode="bilinear", align_corners=False)

x = torch.rand(1, 2, 8, 8)
ct.convert(
    torch.jit.trace(M().eval(), x),
    inputs=[ct.TensorType(shape=ct.Shape([1, 2, 8, ct.RangeDim(1, 64)]))],
    convert_to="mlprogram",
    minimum_deployment_target=ct.target.iOS17,
)

File ".../sympy/core/expr.py", line 375, in __float__
    raise TypeError("Cannot convert expression to float")
TypeError: Cannot convert expression to float

mb.upsample_bilinear resizes by a constant scale factor, so the converter derives one from the requested output size via _get_scales_from_output_size, i.e. output_size / input_size. When input_size is a sympy symbol that division has no constant answer and the floor inside raises, from the middle of the converter with nothing pointing at what the model did.

Resizing a variably-sized input down to a fixed size is the normal shape of image preprocessing inside a model, and it is a long-standing complaint — the same _get_scales_from_output_size failure is what #970 hit, and #1916 is the nearest-neighbor sibling of it.

Fix

The iOS 17 resize op takes the target size directly rather than a scale factor, precisely for "a tensor needs to be resized to a dynamic shape" (its own docstring). So when

• the input's height or width is symbolic, and
• no scale factor was supplied, and
• the output size is a compile-time constant,

lower to mb.resize instead. Static input shapes are untouched and still go through mb.upsample_bilinear, as do all the scale-factor forms.

Sampling mode: I checked mb.resize against torch for every mode on this shape — UNALIGN_CORNERS is bit-identical to torch's align_corners=False and ALIGN_CORNERS to align_corners=True, so the mapping is exact rather than approximate.

Below iOS 17 there is no op that can express this, so it now raises a NotImplementedError naming the deployment target to raise, instead of the sympy TypeError.

Scope

Deliberately bilinear-only. I probed mb.resize with interpolation_mode="NEAREST_NEIGHBOR" for upsample_nearest2d and it only accepts DEFAULT sampling, whose rounding disagrees with torch except when the scale is an exact integer, so mapping it would trade a crash for silently different numbers. upsample_nearest2d (the failure in #1916) keeps its current behavior.

Test

TestUpsample::test_upsample_bilinear2d_with_output_size_dynamic resizes a (1, 3, 9, 22) input with both spatial dims flexible to (4, 4) (down) and (16, 20) (up, and non-square) with align_corners both ways, on the TorchScript and torch.export frontends, and compares against torch at two different input sizes.

Without the fix all 8 mlprogram parametrizations fail with the sympy TypeError; with it they pass. neuralnetwork is skipped since resize is an iOS 17 mlprogram op.

Verification

pytest coremltools/converters/mil/frontend/torch/test/test_torch_ops.py -k TestUpsample

passes on macOS / Apple silicon, torch 2.12, with the pre-existing xfails unchanged.