Fix MinkowskiInterpolation zero/under-weighted results at sparse boundaries (renormalize over existing neighbors) - #16
Merged
Conversation
…arse boundaries MinkowskiInterpolation computes multilinear weights over the 2^D lattice corners surrounding each query point and silently drops corners that are not occupied in the sparse tensor. The surviving weights were never renormalized, so any query with a partially missing neighborhood was systematically under-weighted -- down to exactly zero right next to valid data (NVIDIA#477, NVIDIA#363, NVIDIA#383). Fix in MinkowskiInterpolationFunction.forward (Python side, covers the CPU and GPU backends and every dtype with one implementation): scatter-add the returned weights per output row, and for rows whose corner count is below 2^D divide both the output features and the weights by the surviving weight sum. The renormalized weights are what get saved for backward and returned via return_weights=True, so the gradient and the reported weights stay exactly consistent with the produced features. Backward compatibility: - exact-on-voxel and full-neighborhood queries are bit-identical to the previous behavior (their rows are scaled by exactly 1.0, and the fast path returns the backend tensors untouched when no partial row exists) - queries whose neighborhood has no occupied corner still return zeros - TensorField.splat consumes raw weights via the coordinate-manager API and is intentionally unaffected (splatting is a scatter, not an average) - 16-bit inputs accumulate the weight sum in fp32 tests/python/interpolation_renorm.py adds CPU+GPU regression coverage: exact-voxel bit-equality, full-neighborhood bit-equality vs the raw backend output, boundary queries vs a renormalized fp64 reference, weight sums of one, fp64 gradcheck over partial neighborhoods, and fp16/bf16 autocast tolerance checks (GPU tests conftest-gated as usual).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the long-standing boundary bug in
MinkowskiInterpolation/SparseTensor.features_at_coordinatesreported upstream as NVIDIA/MinkowskiEngine#477 (duplicates #363, #383).Root cause
The backend (
interpolation_map_weightinsrc/coordinate_map_cpu.hpp/src/coordinate_map_gpu.cu) computes multilinear weightsprod_d (1 - |q_d - c_d| / stride_d)over the2^Dlattice corners surrounding each query point and silently drops corners that are not occupied in the sparse tensor (CPU: hash miss skipped; GPU:thrust::remove_ifon the unused key). The surviving weights were never renormalized, so any query with a partially missing neighborhood was under-weighted by the factorsum(existing weights) < 1— down to exactly zero for queries adjacent to valid data.Before / after
3D lattice: fully occupied 2x2x2 cube at
(0..1)^3with features 1..8, plus an isolated voxel at(5,5,5)with feature 9. Identical results on CPU and CUDA:(0,0,0)exact voxel(5,5,5)exact voxel(0.25,0.5,0.75)interior(5.25,5.25,5.25)next to isolated voxel(0.5,0.5,1.75)just outside cube face(20.5,20.5,20.5)empty regionFix
Python-side, in
MinkowskiInterpolationFunction.forward: scatter-add the backend-returnedweightsper output row; for rows whose found-corner count is below2^D(and weight sum > 0), divide both the output features and the weights by the surviving weight sum. The renormalized weights are saved for backward and returned viareturn_weights=True, so gradients and reported weights stay exactly consistent with the produced features — the existing backward kernels are untouched.Why Python-side rather than in the kernels:
coo_spmmpath, and all dtypes;InterpolationBackwardCPU/GPU);interpolation_map_weightin both backends plus the spmm accumulation.Backward compatibility
1.0, and when a batch has no partial row at all the backend tensors are returned untouched (fast path).features_at_coordinates).TensorField.splat()consumes raw weights through the coordinate-manager API and is intentionally unaffected — splatting is a scatter, not an average, so un-renormalized weights are correct there.Validation
tests/python/interpolation_renorm.py(GPU cases conftest-gated like the rest of the suite): exact-voxel bit-equality, full-neighborhood bit-equality vs the raw backend output, boundary queries vs a renormalized fp64 reference, per-query weight sums of 1, fp64torch.autograd.gradcheckover partial neighborhoods (CPU+CUDA), fp16/bf16 autocast tolerance variants. 9/9 pass locally on an RTX 5090 (CUDA 12.8, torch 2.9.1).tests/python/interpolation.pygreen except the pre-existing conftest-skipped entry (test_gpu's 10M-iteration leak loop validated via a trimmed 200-iteration run incl. gradcheck).convolution.py(minus TestPCD),pool.py,half_precision.py— 37 passed, 3 pre-existing skips.🤖 Generated with Claude Code