Skip to content

grep: use ASCII word boundaries in the C locale#75

Open
ColumbusLabs wants to merge 1 commit into
uutils:mainfrom
ColumbusLabs:codex-ascii-word-boundaries
Open

grep: use ASCII word boundaries in the C locale#75
ColumbusLabs wants to merge 1 commit into
uutils:mainfrom
ColumbusLabs:codex-ascii-word-boundaries

Conversation

@ColumbusLabs

Copy link
Copy Markdown

Fixes #64.

Summary

  • compile Oniguruma patterns using its ASCII encoding
  • search input using the same ASCII encoding
  • classify -w boundaries with ASCII word-character rules
  • add regression coverage for non-ASCII bytes on both sides of matches

This makes bytes at or above 0x80 non-word characters under the repository's current C-locale behavior, matching GNU grep for both -w and regex \b.

Testing

  • cargo fmt --all -- --check
  • cargo check --all-targets --workspace
  • cargo clippy --all-targets --workspace -p uu_grep -- -D warnings
  • cargo test --all-targets --workspace

@lhecker

lhecker commented Jul 21, 2026

Copy link
Copy Markdown
Collaborator

Does Codex know why this PR is wrong or would you like me to explain it? The linked issue does hint at it.

@ColumbusLabs

Copy link
Copy Markdown
Author

You are right: this patch applies ASCII encoding globally, while the approach discussed on #64 is conditional on C-locale behavior. Since locale selection is not implemented here yet, that would regress the existing UTF-8 path. Before I revise it, would you prefer locale-aware encoding selection as the prerequisite, or a narrower -w boundary fix with regex \b deferred?

@lhecker

lhecker commented Jul 22, 2026

Copy link
Copy Markdown
Collaborator

If I remember correctly, -w is Unicode-aware under UTF-8 locales, just like \b. If correct, the only correct PR would add proper locale support (i.e. your former option).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

-w / \b treat non-ASCII bytes as word characters, unlike GNU in the C locale

2 participants