python: migrate to nanobind and publish abi3 wheel

[anuraaga]

Hi @rsc - I was interested in an abi3 wheel so new Python versions can be supported from day 1. pybind doesn't support abi3 so this migrates to nanobind. I understand a core infra change like that can be questionable, and if this can't be accepted, it is what it is. I hope it may be feasible given nanobind is written by pybind's author as its modern replacement.

This migrates to nanobind and wires in a PY_LIMITED_API env var to control building an abi3 wheel in bazel and setuptools. The migration to nanobind is largely mechanical renaming and inlining throw std::runtime_error that was previously a pybind macro, and wrapping a string explicitly as bytes since the default was changed. The non-mechanical change is a small wrapper to extract PyBuffer which nanobind doesn't provide.

I have verified the CI workflow on my fork

https://github.com/anuraaga/re2/actions/runs/29892421139
https://github.com/anuraaga/re2/actions/runs/29892414442

(#649 fixes the issue that macOS 13 builds don't start up at all)

I have done a very barebones perf check with relatively small inputs sampled from the cc benchmark file that stress the bindings more than with larger inputs

case	pybind11	nanobind	nanobind+abi3	nanobind vs pybind	abi3 vs nanobind
short_search (26 B)	2106	372	415	5.7× faster	+11%
captures_search (3 groups)	2660	447	560	6.0× faster	+25%
escape / QuoteMeta	962	126	131	7.6× faster	+4%
compile (RE2 construct)	5256	3206	3132	1.6× faster	−2% (noise)
findall_digits (768 hits)	2,583,568	456,551	462,377	5.7× faster	+1%
easy0_64k (scan 64 KiB)	2011	284	274	7.1× faster	−4% (noise)
hard_64k (scan 64 KiB)	2013	282	274	7.2× faster	−3% (noise)

nanobind is built from scratch to be faster than pybind. I would have been happy with relatively same but it is actually a lot faster, more than I was expecting. So regardless of abi3 compatibility, this seems like a nice improvement.

The above also compares abi3 to nanobind - it is about what you'd expect, the more iteration (i.e. capture groups), the more overhead vs string operations that generally perform similarly between them. But, even with a 25% measured in one case, that is after a huge speedup. I intentionally made sure not to drop any wheels here, but my recommendation is to take advantage of the perf we got to accept the smaller hit, and only publish 3 wheels, 3.10 and 3.11 (nanobind only supports abi3 from 3.12) and abi3. This means you won't have to sign-in once a year to release a new wheel :-) If this sounds good, I can update this PR with that change, or send another PR for it. For reference, protobuf already only publishes a abi3 wheel - they are able to do it from 3.10 since they manually bind without pybind or nanobind.

I also noticed that macOS-specific wheels are published. I get the feeling there is some legacy reason for this, and most modern projects would just set the min SDK version to 13 and publish a single wheel. That is independent from this PR but if making a change to wheel coverage, this would be a good time to simplify that too.

For context, the main motivation for this is I have started maintaining a Python project with a hard dependency on re2 (for syntax reasons). I'd be very happy if that project could work on day-1 when a new Python version like 3.15 is released. For reference, issues like #647 would go away too.

[anuraaga]

FWIU, this is a Bazel 9 strictness issue where 32-bit windows stops getting an implicit test toolchain setup. We don't need the test targets built anyways so I changed it to just the exception.

It is unrelated to the motivation for this PR, I kept in a separate commit c5b2c79 but let me know if I should send a separate PR isntead

[anuraaga]

I was surprised this wasn't here yet. I added as a separate commit 862b674 but happy to remove or extract, I would very easily accidentally commit something massive without it

[anuraaga]

Note I was unable to build from main on my macOS without passing -std=c++17, so while nanobind requires it, I think at least bazel+macOS also does