What's wrong
Commit 65c8554 ("Fold case invariantly in regex matching") added RegexOptions.CultureInvariant, but only when the caller passes TextFilterCaseSensitivity.CaseInsensitive (TextFilter/TextFilter.cs, ~line 462):
RegexOptions regexOptions = caseSensitivity is TextFilterCaseSensitivity.CaseInsensitive
? RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant
: RegexOptions.Compiled;
Users can also turn ignore-case on inside the pattern with (?i), which is ordinary regex syntax. On that path the regex is compiled with RegexOptions.Compiled alone, so it folds case using the thread's CurrentCulture at construction time. The compiled regex is then cached under a key made only of pattern and case sensitivity (for example "s:(?i)img"). This is the cache-poisoning problem the comment above that line describes, which the fix left open on this path.
Repro
TextFilter.IsMatch("IMG_1234.JPG", "(?i)img", TextFilterType.Regex, TextFilterMatchOptions.ByWholeString);
// default CaseSensitive
Each order below ran in a fresh process:
== tr-TR first
tr-TR: False
en-US: False <- poisoned by the cache
== en-US first
en-US: True
tr-TR: True
The same query with pattern img and the explicit CaseInsensitive option returns True under tr-TR.
The result is that the same filter, in the same app, gives different answers depending on which thread's culture compiled it first. That is a nondeterministic filter bug which is hard to diagnose.
Suggested fix
Always include CultureInvariant. It only has an effect when ignore-case is on, whether set by the option or inline, so case-sensitive matching is unchanged:
RegexOptions regexOptions = RegexOptions.Compiled | RegexOptions.CultureInvariant
| (caseSensitivity is TextFilterCaseSensitivity.CaseInsensitive ? RegexOptions.IgnoreCase : RegexOptions.None);
I checked this with raw Regex under tr-TR:
(?i)img with Compiled alone against IMG: False
(?i)img with Compiled | CultureInvariant: True
- Case-sensitive
img with CultureInvariant against IMG: False, unchanged
Acceptance: add a test modelled on ARegexCompiledUnderOneCultureAnswersTheSameUnderAnother that:
- uses the pattern
(?i)img with the default CaseSensitive option
- compiles the pattern under tr-TR first
- asserts the match returns
true under both tr-TR and en-US
What's wrong
Commit 65c8554 ("Fold case invariantly in regex matching") added
RegexOptions.CultureInvariant, but only when the caller passesTextFilterCaseSensitivity.CaseInsensitive(TextFilter/TextFilter.cs, ~line 462):Users can also turn ignore-case on inside the pattern with
(?i), which is ordinary regex syntax. On that path the regex is compiled withRegexOptions.Compiledalone, so it folds case using the thread'sCurrentCultureat construction time. The compiled regex is then cached under a key made only of pattern and case sensitivity (for example"s:(?i)img"). This is the cache-poisoning problem the comment above that line describes, which the fix left open on this path.Repro
Each order below ran in a fresh process:
The same query with pattern
imgand the explicitCaseInsensitiveoption returnsTrueunder tr-TR.The result is that the same filter, in the same app, gives different answers depending on which thread's culture compiled it first. That is a nondeterministic filter bug which is hard to diagnose.
Suggested fix
Always include
CultureInvariant. It only has an effect when ignore-case is on, whether set by the option or inline, so case-sensitive matching is unchanged:I checked this with raw
Regexunder tr-TR:(?i)imgwithCompiledalone againstIMG: False(?i)imgwithCompiled | CultureInvariant: TrueimgwithCultureInvariantagainstIMG: False, unchangedAcceptance: add a test modelled on
ARegexCompiledUnderOneCultureAnswersTheSameUnderAnotherthat:(?i)imgwith the defaultCaseSensitiveoptiontrueunder both tr-TR and en-US