Skip to content

Regex with inline (?i) folds case by the current culture, and the first caller's culture is cached for everyone: "(?i)img" stops matching "IMG_1234.JPG" after a tr-TR caller #120

Description

@matt-edmondson

What's wrong

Commit 65c8554 ("Fold case invariantly in regex matching") added RegexOptions.CultureInvariant, but only when the caller passes TextFilterCaseSensitivity.CaseInsensitive (TextFilter/TextFilter.cs, ~line 462):

RegexOptions regexOptions = caseSensitivity is TextFilterCaseSensitivity.CaseInsensitive
    ? RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant
    : RegexOptions.Compiled;

Users can also turn ignore-case on inside the pattern with (?i), which is ordinary regex syntax. On that path the regex is compiled with RegexOptions.Compiled alone, so it folds case using the thread's CurrentCulture at construction time. The compiled regex is then cached under a key made only of pattern and case sensitivity (for example "s:(?i)img"). This is the cache-poisoning problem the comment above that line describes, which the fix left open on this path.

Repro

TextFilter.IsMatch("IMG_1234.JPG", "(?i)img", TextFilterType.Regex, TextFilterMatchOptions.ByWholeString);
// default CaseSensitive

Each order below ran in a fresh process:

== tr-TR first
tr-TR: False
en-US: False        <- poisoned by the cache
== en-US first
en-US: True
tr-TR: True

The same query with pattern img and the explicit CaseInsensitive option returns True under tr-TR.

The result is that the same filter, in the same app, gives different answers depending on which thread's culture compiled it first. That is a nondeterministic filter bug which is hard to diagnose.

Suggested fix

Always include CultureInvariant. It only has an effect when ignore-case is on, whether set by the option or inline, so case-sensitive matching is unchanged:

RegexOptions regexOptions = RegexOptions.Compiled | RegexOptions.CultureInvariant
    | (caseSensitivity is TextFilterCaseSensitivity.CaseInsensitive ? RegexOptions.IgnoreCase : RegexOptions.None);

I checked this with raw Regex under tr-TR:

  • (?i)img with Compiled alone against IMG: False
  • (?i)img with Compiled | CultureInvariant: True
  • Case-sensitive img with CultureInvariant against IMG: False, unchanged

Acceptance: add a test modelled on ARegexCompiledUnderOneCultureAnswersTheSameUnderAnother that:

  • uses the pattern (?i)img with the default CaseSensitive option
  • compiles the pattern under tr-TR first
  • asserts the match returns true under both tr-TR and en-US

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions