COMP: Build all ITK Python wrapping as abi3 (stable ABI)

[hjmjohnson]

Makes ITKVtkGlue's Python wrapping stable-ABI clean, which removes the last blocker to building all ITK Python wrapping as abi3. With that gone, ITK_USE_PYTHON_LIMITED_API had only one reachable value, so it and the surrounding machinery are deleted (CMake/ITKSetPython3Vars.cmake: 33 added, 100 removed).

Closes #6711. Also lowers the cost of #4656 (weekly PyPI wheels): abi3-only means one wheel per platform instead of one per Python version.

Verified on macOS 26 / arm64 only. No Linux or Windows verification, and no Slicer test result — see "Environments verified" for exactly what was and was not exercised.

The ITKVtkGlue fix, and a defect in the approach #6711 proposed

VtkGlue.i exchanged pointers with VTK through vtkPythonUtil, whose header chain reaches PyVTKObject.h and touches PyTypeObject members that Py_LIMITED_API hides. The typemaps now use Limited API calls only, parsing VTK's __this__ encoding and reconstructing through Addr=0x....

VTK::WrappingPythonCore is dropped from the wrapping link interface. This is a correctness requirement rather than tidying: that library is built against one libpython, so an extension linking it cannot be version-agnostic.

Defect in the sketch from #6711. The issue proposed PyObject_CallFunction(cls, "s", addr), i.e. vtkImageData("Addr=0x…"). On VTK 9.6.2 this fails, but only for vtkImageData and vtkPolyData, and only in the pointer-to-object direction. The cause is not ABI-related: vtkmodules/util/data_model.py registers @vtkImageData.override with a Python subclass whose __init__(self, **kwargs) is keyword-only, so the positional address string never reaches VTK's C-level reconstruction. vtkImageExport, vtkImageImport, vtkObject and vtkPoints have no override and were unaffected.

The fix is to call __new__ explicitly: it bypasses the override's __init__ while still returning the enhanced ImageData type, with pointer identity preserved.

Neither __this__ nor Addr=0x... is documented VTK API, so PythonVtkGlueABI3EncodingTest exists to fail loudly and specifically if VTK changes either one.

Why removing the option is safe rather than merely convenient

ITK_WRAP_PYTHON_MINIMUM_VERSION and the abi3 floor are both 3.11, and the option auto-enabled whenever the interpreter met the abi3 floor. Since wrapping already hard-errors below 3.11, that condition was true for every supported interpreter. The non-SABI branch was reachable only by setting the option OFF explicitly, and the only documented reason to do so was ITKVtkGlue's FATAL_ERROR.

Empirically: removing the option changed 524 build targets, not 8237. The python3_add_library() arguments were byte-identical before and after, so only relinking occurred.

FindPython3 supplies Development.Module implicitly alongside Development.SABIModule — the configure log reports it as found even though it is no longer requested. That is why dropping the explicit request is safe.

Environments verified (and what was not)

Platform: macOS 26 / arm64, Apple clang via conda-forge toolchain.
VTK: github.com/slicer/VTK at 6181bb1223bbc499a340a1644f5356e7e152c318 — the SHA Slicer's SuperBuild/External_VTK.cmake pins for its 9.6 series — built with Qt6 6.9.1, full rendering, and Python wrapping.

Build and test

Check	Result
ctest -R Python	175/175 passed
PythonVtkGlueABI3EncodingTest	passed
PythonVtkGlueRoundTripTest	passed
Rebuild after all 9 commits	421/421, 0 FAILED:
pre-commit run --all-files	exit 0

Artifact audit

Property	Value
Wrapped modules	98
Non-abi3 modules	0
_ITKVtkGluePython.abi3.so undefined Py* symbols	71, all Limited API
PyVTK* / vtkPythonUtil symbols	0
libvtkWrappingPythonCore linked	0
libpython linked	none

Multi-interpreter. Extensions were built against pixi Python 3.13.9; the others are Homebrew builds, a different distribution. VTK's Python wrappers were built separately for cp311/cp312/cp313/cp314 against the same VTK C++ libraries, so each row pairs the same ITK .abi3.so with a different version-specific VTK.

Interpreter	abi3 module loads	Filter result	VtkGlue round-trip
3.11.15	yes	7.0	OK
3.12.13	yes	7.0	OK
3.13.9 (build interpreter)	yes	7.0	OK
3.14.6	yes	7.0	OK

The 3.11 and 3.14 rows are the substantive ones: that .abi3.so was compiled against cp313 VTK headers yet drove full round-trips against cp311 and cp314 VTK modules. Under the previous design, which linked VTK::WrappingPythonCore, those pairings were structurally impossible.

Configure matrix

Scenario	Expected	Result
Wrapping ON + VtkGlue ON	configures	exit 0
Wrapping OFF	configures, no SABI required	exit 0
Wrapping ON, BUILD_TESTING=OFF	NumPy not required	exit 0, NumPy absent
Python 3.10	rejected with the floor message	exit 1, correct message

Downstream

• Standalone ITK with ITK_WRAP_PYTHON=OFF, Module_ITKVtkGlue=ON: 2730/2730 targets, 0 errors.
• Slicer: its ITK external project built successfully against this branch, together with ~20 other external projects (VTK, CTK, DCMTK, Python, OpenSSL). No Slicer tests ran. The Slicer build is blocked in GDCM by a conda/system library shadowing problem in the local test environment: with -isystem $CONDA_PREFIX/include present, conda's iconv.h (#define iconv_open libiconv_open) is paired with the macOS SDK's libiconv.tbd, giving undefined _libiconv_open; removing that flag instead breaks Slicer's symbol-prefixed zlib with undefined _crc32. This is an environment issue, not a code one, and is independent of this branch: the diff is wrapping-only and Slicer's ITK is ITK_WRAP_PYTHON=OFF, which makes this branch a no-op for it.

Not verified: Linux, Windows, any non-arm64 target, and any completed Slicer test suite.

Three things that look simplifiable under abi3 but are not

Recorded so they are not "fixed" later:

• The exact-version pinning in ITKSetPython3Vars.cmake governs which headers and import library the compile uses. abi3 relaxes which interpreters the result runs on — a different guarantee.
• WITH_SOABI is what produces the .abi3.so suffix under USE_SABI.
• unset(Python3_FIND_ABI) is an unrelated FindPython component-discovery workaround.

Also note the per-Python-version wheel matrix that SABI-only would collapse lives in ITKPythonPackage, not this repository. This branch enables that follow-up; it does not perform it.

That follow-up is what makes #4656 (weekly builds published to PyPI) materially cheaper. ITK supports Python 3.11-3.14 over roughly five platform targets (linux x86_64/aarch64, macOS x86_64/arm64, Windows), so a release today is on the order of 20 wheels; abi3-only reduces that to about 5 — one per platform, valid for every supported interpreter. For a weekly cadence that is a 4x cut in build jobs, upload volume, and failure surface, which is the difference between a schedule that is sustainable and one that is not.

This does not resolve #4656 — the scheduled workflow and the PyPI publishing still have to be built in ITKPythonPackage. It only removes the combinatorial part of the cost.

[dzenanz]

Looks good on a glance. Matt and Brad should review this.

[hjmjohnson]

Force-push a8b614a044 → 30bd48d4d0b: content-only fix for the ITK.macOS.Python failure (no rebase mixed in — the branch was already current with main).

libITKCommon failed to link with undefined _Py_BuildValue / _Py_Dealloc. Fixed by folding into the commit that introduced it.

Root cause

Modules/Core/Common/src/CMakeLists.txt has two uses of _itk_python_target: the INCLUDE_DIRECTORIES source property, and target_link_libraries(ITKCommon PRIVATE ${_itk_python_target}) about 100 lines further down. The commit that replaced the variable with Python3::SABIModule updated only the first, leaving the second to expand to an empty string. ITKCommon then stopped linking Python3::SABIModule and lost its INTERFACE_LINK_OPTIONS, i.e. -undefined dynamic_lookup.

It failed on exactly one configuration because three things have to line up: an undefined CMake variable expands to empty without error, ELF shared objects tolerate undefined symbols, and static archives never resolve them at all. Only a shared macOS build links libITKCommon.dylib strictly enough to notice — hence green Linux Python (177 tests) and green CDash while macOS Python failed at 764/5404 with 0 tests run.

Verified by configuring main and this branch side by side with BUILD_SHARED_LIBS=ON: main emits 98 dynamic_lookup link options, the broken branch 97, and the fixed branch 98 again, with the missing one being libITKCommon. libITKCommon-6.0.1.dylib now links, and nm -u shows _Py_BuildValue and _Py_Dealloc present as permitted undefined symbols.

[hjmjohnson]

Two force-pushes just landed, split by concern: 33b3a662354 → c046803a5c7 is a plain rebase on main (nothing else), and c046803a5c7 → d542b02554f is content only — the fix for Greptile's P1 unvalidated-pointer finding. The second compare link shows just the two VtkGlue files.

[blowekamp]

What are the implications of this for building against the Python Free Thread ABI. Currently Free Threads are not stable and there is expected to be a separate stable ABI for 3.15:
https://peps.python.org/pep-0803/

[dzenanz]

I read this, and the requirements for abi3t are abi3 + no GIL locking. So this PR is a step in the right direction, as best as I can tell.

[hjmjohnson]

I do think this is a mandatory step toward abi3t, but I am not an expert on those details. The primary motivator was to remove complexity in the cmake options, and remove unnecessary dependance on non-abi3 code.