Fix NormalizeText.remove_accents — use NFKD (#45)

[matthewhorridge]

Summary

Issue #45 framed this as a docs/configurability question, but on investigation the existing behaviour was a silent bug: remove_accents was using NFKC (a composing normalization form), which left pre-composed accented characters like 'é' (U+00E9) as a single code point. The subsequent unicodedata.combining() filter only catches separate combining marks, so the accent was never removed — \"café\" came back as \"café\".

Quick repro:

>>> NormalizeText(Normalization.ACCENT).transform(\"café\")  # before
'café'
>>> NormalizeText(Normalization.ACCENT).transform(\"café\")  # after
'cafe'

The fix

Switch the normalization form to NFKD (compatibility decomposition):

• NFD or NFKD both split 'é' into 'e' + U+0301 (combining acute), which the existing filter then drops correctly.
• NFKD over NFD because the compatibility decomposition also folds presentational variants useful for harmonization: the ligature 'fi' → 'fi', superscript digits → plain digits, etc. Documented in the docstring so callers who need to preserve those can normalize input to NFC beforehand.

Tests

Five new tests in test_primitives_serialization.py:

• Common Latin accents (café, résumé, naïve, piñata, über, Ångström) on pre-composed input
• Same logic on pre-decomposed input (verifies NFKD is idempotent)
• Unaccented text is unchanged
• Compatibility folding (file → file, x² → x2) — pinning the NFKD choice
• Serialization roundtrip with the remove_accents value

Total: 167/167 tests pass (was 162).

Test plan

• pytest green
• Manual repro of the original bug fixed (café → cafe)

🤖 Generated with Claude Code