Proposal: optional native parser backend with pure-Python fallback

[lowmiaq-gmail]

python-dotenv optional native parser backend

Problem

python-dotenv parsing is on the configuration path of real consumers such as
pydantic-settings. We measured complete consumer calls, not parser-only
throughput, on the same macOS arm64 / CPython 3.14 environment.

Proposal

• Keep the current pure-Python parser and pip install python-dotenv default.
• Add an optional native extra whose backend-only dependency is
  fast-dotenv-rs-backend.
• Keep the Rust package separate from the dotenv namespace and console
  script; the adapter only converts lossless parser records into the existing
  Binding and Original objects.
• If the extra/backend is absent, the existing pure-Python path remains in
  use. If an explicitly selected backend violates its versioned contract or
  raises, surface that failure rather than silently masking it.

Runtime evidence

• Upstream base: a00cb2eed0704cd6d2071b2004c37e95ccc86ee5.
• Backend implementation: fast-dotenv-rs commits
  7cbdca0 and 95ea5c1 in the Master integration clone; shared Rust core
  and backend-only wheel are ready for review.
• Upstream adapter patch: python-dotenv commits
  0c1d17b9878ae2b390616fbd8b64f605f9fd9d67 and
  f5856435e229485040e0e11530a191b0f51007df.
• Exact Binding differential: 1,088 records, including 500 generated valid
  and 500 generated malformed records; native OFF, native ON, and backend
  direct paths each retained 0 mismatches.
• Upstream suite: 264 passed, 1 skipped in both OFF and ON runs, using a
  portable fixture for the host-specific printenv --version test.
• Native call proof: fast_dotenv_rs_backend.parse_bindings was observed on
  the same parse_stream path; backend and official package coexist without
  a top-level dotenv collision.
• Real consumer benchmark: dotenv_values, load_dotenv,
  pydantic-settings service/worker settings, repeated loads, and cold
  startup across small/medium/large fixtures. Semantic hashes matched;
  30 p50 metrics classified as 21 win / 8 neutral / 1 loss under a 2%
  neutral band. The only loss was small cold CLI startup (-1.6675 ms);
  representative warm savings were 0.1266 ms for small dotenv_values,
  0.7477 ms for medium, and 4.9392 ms for large.

Maintainer questions

1. Is an optional backend extra acceptable, with backend wheels maintained
   outside the default python-dotenv release path?
2. Should activation use the native extra, another name, or a different
   boundary?
3. Which platform, PyPy/free-threaded, supply-chain, and rollback guarantees
   are required before merge?

This is a concrete proposal with a merge-ready local patch. The backend is not
yet published to PyPI, and the current wheel evidence is macOS arm64 only;
those are intentionally open release gates, not hidden claims.

[lowmiaq-gmail]

Update with external CI evidence (2026-08-27): the backend-only branch now has a completed GitHub Actions run with 10/10 jobs passing across Linux x86_64/arm64, macOS arm64/Intel, and Windows x86_64, on CPython 3.10 and 3.14. Each job built an abi3 backend wheel, installed it alongside python-dotenv, verified namespace isolation and the backend contract, and ran the 3 backend contract tests.

Branch: https://github.com/lowmiaq-gmail/fast-dotenv-rs/tree/codex/backend-only-native-ci-20260827
CI run: https://github.com/lowmiaq-gmail/fast-dotenv-rs/actions/runs/33076711589

The backend is still intentionally not published to PyPI; the proposal continues to preserve the default pure-Python path and is awaiting maintainer direction.