Pascal/Camel/Snake/Kebab/Macro delete combining marks and non-ASCII digits: "हिन्दी भाषा" → "ह_न_द_भ_ष", NFD "éclair" → "EClair"

[matt-edmondson]

What's wrong

ReplaceNonAlphaNumericWithSpace (CaseConverter/CaseConverter.cs:43) keeps a code point only if it passes char.IsLetter or is an ASCII digit '0'..'9'. Two kinds of word characters fail that test and get replaced with a space:

1. Combining marks (UnicodeCategory.NonSpacingMark, SpacingCombiningMark, EnclosingMark). These carry Indic vowel signs and viramas, Thai and Arabic vowel marks, and the accents in NFD-decomposed Latin text. IsWordBoundary (around :139-144) has a related problem: a mark that follows a letter counts as "a non-letter", so a word boundary is placed before it.
2. Non-ASCII decimal digits (Arabic-Indic ١٢٣, Devanagari १२३, and so on). The word splitter already uses the Unicode-aware char.IsDigit, so these fail only in this one helper.

ToTitleCase does not go through this path, so it keeps both kinds of character. The sibling APIs therefore disagree on the same input.

Repro

Input	Call	Actual	Expected
"हिन्दी भाषा"	ToSnakeCase()	"ह_न_द_भ_ष"	"हिन्दी_भाषा"
"हिन्दी भाषा"	ToPascalCase()	"हनदभष"	"हिन्दीभाषा"
"éclair" (NFD: e + U+0301)	ToPascalCase()	"EClair"	"Éclair" (or "Éclair")
"éclair" (NFD)	ToSnakeCase()	"e_clair"	"éclair"
"café au lait" (NFD)	ToPascalCase()	"CafeAuLait"	"CaféAuLait"
"x١٢٣y"	ToPascalCase() / ToSnakeCase()	"XY" / "x_y"	"X١٢٣Y" / "x_١٢٣y"
"x١٢٣y"	ToTitleCase()	"X ١٢٣Y" (keeps the digits)	—

Why it matters

These calls corrupt text without any error. Whole scripts come out unreadable (Devanagari, Bengali, Thai, Tamil, vowelled Arabic). NFD input also breaks, and NFD is common: macOS filesystem names arrive decomposed, as does text from some editors and normalisers. This is the same class of bug as the astral-letter fix (#70/#71), which already established that letters from every script are kept, just for a different set of Unicode categories.

Suggested fix

• In ReplaceNonAlphaNumericWithSpace, also keep code points whose category is NonSpacingMark, SpacingCombiningMark or EnclosingMark. Replace the ASCII digit check with char.IsDigit(input, i).
• In IsWordBoundary, never break before a combining mark, and treat a mark as continuing the letter before it. For example, when deciding "previous is a letter", skip back over marks.
• In the first-character upper/lower mapping, keep any marks that follow the base letter attached to it.

Acceptance: the table above produces the expected column, and there are regression tests for Devanagari, NFD Latin and Arabic-Indic digits across Pascal, Camel, Snake, Kebab and Macro.

[matt-edmondson]

Triage

• Category: Bug
• Priority: Medium. The output is silently corrupted for whole scripts (Devanagari, Thai, vowelled Arabic, etc.) and for NFD Latin input such as macOS filenames. There is no crash, and Latin NFC input, the common case, is unaffected.
• Area / suggested owner: ReplaceNonAlphaNumericWithSpace and IsWordBoundary in CaseConverter.cs.
• Related: A letter after a digit starts a new word in Title/Pascal/Camel but not in Snake/Kebab: "1st place" → "1St Place", "abc123def" → "abc_123def" #78 (inconsistent word-boundary handling around digits across the same Pascal/Camel/Snake/Kebab family). Both touch IsWordBoundary, so coordinate the fixes to avoid conflicts. This follows on from the earlier astral-letter fix (Letters outside the BMP are silently deleted by every case conversion #70/Preserve letters outside the BMP in every case conversion #71).
• Duplicates / in progress: None found. No open PR covers this.
• Next step: Keep Mn/Mc/Me code points and Unicode digits (char.IsDigit(input, i)), never break before a combining mark, and add Devanagari, NFD Latin and Arabic-Indic digit regression tests across all five casings.

---

Generated by Claude Code