Skip to content

core: introduce internal core::pattern::{Split,SplitN} types - #161754

Draft
pacak wants to merge 3 commits into
rust-lang:mainfrom
pacak:push-qvvokrsvvqww
Draft

core: introduce internal core::pattern::{Split,SplitN} types#161754
pacak wants to merge 3 commits into
rust-lang:mainfrom
pacak:push-qvvokrsvvqww

Conversation

@pacak

@pacak pacak commented Aug 25, 2026 •
edited by rustbot
Loading

Copy link
Copy Markdown
Contributor

Introduce core::pattern::Split and core::pattern::SplitN internal types
which can be used to implement iterators splitting haystack into parts.
Convert str’s Split-family of iterators to use them. In the future,
more haystacks will use those internal types.

Co-authored-by: Peter Jaszkowiak p.jaszkow@gmail.com

@pacak: Fixed some typos, added a few #[inline]. Since there's no
H::Cursor - I had to add ctx: PhantomData<H>.

This PR is part of a stack containing 15 PRs:

  1. main
  2. generalize-pattern #161606
  3. "core: introduce internal core::pattern::{Split,SplitN} types" (this PR) (Compare)
  4. push-wlxvvvxzrsly #161755 (Compare)
  5. core: add try_next_code_point{,_reverse} internal functions #161756 (Compare)
  6. core: add internal core::str_bytes module handling string-like slices #161757 (Compare)
  7. coretests: Add a few tests for backward multibyte predicate #161758 (Compare)
  8. core: add concept of Flavour to core::str_bytes #161759 (Compare)
  9. core: add memchr-based single-ascii byte search to str_bytes #161760 (Compare)
  10. core/std: inline small functions #161761 (Compare)
  11. core::str_bytes: use unsafe for indexing #161762 (Compare)
  12. core: skip emitting rejects if result doesn't cary it #161608 (Compare)
  13. std: add pattern matching to OsStr #161763 (Compare)
  14. core: add core::pattern::Predicate wrapper type #161764 (Compare)
  15. core, std: support various [char] matchers for Pattern<&OsStr> #161765 (Compare)
  16. std: add OsStr pattern benchmarks #161610 (Compare)

@rustbot rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Aug 25, 2026
This was referenced Aug 25, 2026
Draft
Draft
Draft
Open
Draft
Draft
Draft
Draft
Draft
Draft
Draft
Merged
Draft
Draft
Draft
@pacak

pacak commented Aug 25, 2026

Copy link
Copy Markdown
Contributor Author

r? @nia-e

@pacak
pacak marked this pull request as ready for review August 25, 2026 22:27
@rustbot

rustbot commented Aug 25, 2026

Copy link
Copy Markdown
Collaborator

stdarch is developed in its own repository. If possible, consider making this change to rust-lang/stdarch instead.

cc @Amanieu, @folkertdev, @sayantn

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Aug 25, 2026
@rustbot

rustbot commented Aug 25, 2026

Copy link
Copy Markdown
Collaborator

⚠️ Warning ⚠️

  • The following commits have merge commits (commits with multiple parents) in your changes. We have a no merge policy so these commits will need to be removed for this pull request to be merged.

    You can start a rebase with the following commands:

    $ # rebase
$ git pull --rebase https://github.com/rust-lang/rust.git main
$ git push --force-with-lease

@rustbot rustbot added the has-merge-commits PR has merge commits, merge with caution. label Aug 25, 2026
@pacak
pacak marked this pull request as draft August 26, 2026 07:29
@rustbot rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 26, 2026
@pacak
pacak force-pushed the push-qvvokrsvvqww branch from c7b6cd5 to c1d7ee7 Compare August 26, 2026 07:35
@rust-bors

This comment has been minimized.

Sign in to view
@pacak
pacak force-pushed the push-qvvokrsvvqww branch from c1d7ee7 to b056496 Compare August 26, 2026 19:56
Zalathar added a commit to Zalathar/rust that referenced this pull request Sep 17, 2026
@Zalathar
Rollup merge of rust-lang#161596 - pacak:more-pat-tests, r=nia-e
20f2600 
coretests: Add more pattern tests.

Right now things are undertested and underspecified.

Some of the library code would get in a loop if searcher starts returning empty rejects.

And there's no tests for backwards multi byte char matchers. Pull request I'm reviving had a problem implementing that, so making sure it's tested before the actual code lands.

Right now it is possible to break both tests (and user code) without breaking anything else in the test suite I think.

This commit is extracted from rust-lang#160971 with slight modifications.

<!-- start jj-vine stack -->
This PR is part of a stack containing 16 PRs:

1. `main`
2. **"coretests: Add more pattern tests." (this PR)**
3. rust-lang#161606 ([Compare](https://github.com/pacak/rust/compare/more-pat-tests..generalize-pattern))
4. rust-lang#161754 ([Compare](https://github.com/pacak/rust/compare/generalize-pattern..push-qvvokrsvvqww))
5. rust-lang#161755 ([Compare](https://github.com/pacak/rust/compare/push-qvvokrsvvqww..push-wlxvvvxzrsly))
6. rust-lang#161756 ([Compare](https://github.com/pacak/rust/compare/push-wlxvvvxzrsly..push-lkvnuzsvuqzt))
7. rust-lang#161757 ([Compare](https://github.com/pacak/rust/compare/push-lkvnuzsvuqzt..push-wuxtmysppkst))
8. rust-lang#161758 ([Compare](https://github.com/pacak/rust/compare/push-wuxtmysppkst..push-tnkxmkmykopl))
9. rust-lang#161759 ([Compare](https://github.com/pacak/rust/compare/push-tnkxmkmykopl..push-umzmopsxmwry))
10. rust-lang#161760 ([Compare](https://github.com/pacak/rust/compare/push-umzmopsxmwry..push-ptqpwloxpyru))
11. rust-lang#161761 ([Compare](https://github.com/pacak/rust/compare/push-ptqpwloxpyru..push-rksmzopsvzlk))
12. rust-lang#161762 ([Compare](https://github.com/pacak/rust/compare/push-rksmzopsvzlk..push-mkxpywoqusmn))
13. rust-lang#161608 ([Compare](https://github.com/pacak/rust/compare/push-mkxpywoqusmn..flavor-pattern))
14. rust-lang#161763 ([Compare](https://github.com/pacak/rust/compare/flavor-pattern..push-xoslomrztmly))
15. rust-lang#161764 ([Compare](https://github.com/pacak/rust/compare/push-xoslomrztmly..push-pqpumynltyml))
16. rust-lang#161765 ([Compare](https://github.com/pacak/rust/compare/push-pqpumynltyml..push-svrvnxkpuqul))
17. rust-lang#161610 ([Compare](https://github.com/pacak/rust/compare/push-svrvnxkpuqul..osstr-pattern-tip))
<!-- end jj-vine stack -->
Zalathar added a commit to Zalathar/rust that referenced this pull request Sep 17, 2026
@Zalathar
Rollup merge of rust-lang#161596 - pacak:more-pat-tests, r=nia-e
48a74e2 
coretests: Add more pattern tests.

Right now things are undertested and underspecified.

Some of the library code would get in a loop if searcher starts returning empty rejects.

And there's no tests for backwards multi byte char matchers. Pull request I'm reviving had a problem implementing that, so making sure it's tested before the actual code lands.

Right now it is possible to break both tests (and user code) without breaking anything else in the test suite I think.

This commit is extracted from rust-lang#160971 with slight modifications.

<!-- start jj-vine stack -->
This PR is part of a stack containing 16 PRs:

1. `main`
2. **"coretests: Add more pattern tests." (this PR)**
3. rust-lang#161606 ([Compare](https://github.com/pacak/rust/compare/more-pat-tests..generalize-pattern))
4. rust-lang#161754 ([Compare](https://github.com/pacak/rust/compare/generalize-pattern..push-qvvokrsvvqww))
5. rust-lang#161755 ([Compare](https://github.com/pacak/rust/compare/push-qvvokrsvvqww..push-wlxvvvxzrsly))
6. rust-lang#161756 ([Compare](https://github.com/pacak/rust/compare/push-wlxvvvxzrsly..push-lkvnuzsvuqzt))
7. rust-lang#161757 ([Compare](https://github.com/pacak/rust/compare/push-lkvnuzsvuqzt..push-wuxtmysppkst))
8. rust-lang#161758 ([Compare](https://github.com/pacak/rust/compare/push-wuxtmysppkst..push-tnkxmkmykopl))
9. rust-lang#161759 ([Compare](https://github.com/pacak/rust/compare/push-tnkxmkmykopl..push-umzmopsxmwry))
10. rust-lang#161760 ([Compare](https://github.com/pacak/rust/compare/push-umzmopsxmwry..push-ptqpwloxpyru))
11. rust-lang#161761 ([Compare](https://github.com/pacak/rust/compare/push-ptqpwloxpyru..push-rksmzopsvzlk))
12. rust-lang#161762 ([Compare](https://github.com/pacak/rust/compare/push-rksmzopsvzlk..push-mkxpywoqusmn))
13. rust-lang#161608 ([Compare](https://github.com/pacak/rust/compare/push-mkxpywoqusmn..flavor-pattern))
14. rust-lang#161763 ([Compare](https://github.com/pacak/rust/compare/flavor-pattern..push-xoslomrztmly))
15. rust-lang#161764 ([Compare](https://github.com/pacak/rust/compare/push-xoslomrztmly..push-pqpumynltyml))
16. rust-lang#161765 ([Compare](https://github.com/pacak/rust/compare/push-pqpumynltyml..push-svrvnxkpuqul))
17. rust-lang#161610 ([Compare](https://github.com/pacak/rust/compare/push-svrvnxkpuqul..osstr-pattern-tip))
<!-- end jj-vine stack -->
Zalathar added a commit to Zalathar/rust that referenced this pull request Sep 17, 2026
@Zalathar
Rollup merge of rust-lang#161596 - pacak:more-pat-tests, r=nia-e
fa0c4bc 
coretests: Add more pattern tests.

Right now things are undertested and underspecified.

Some of the library code would get in a loop if searcher starts returning empty rejects.

And there's no tests for backwards multi byte char matchers. Pull request I'm reviving had a problem implementing that, so making sure it's tested before the actual code lands.

Right now it is possible to break both tests (and user code) without breaking anything else in the test suite I think.

This commit is extracted from rust-lang#160971 with slight modifications.

<!-- start jj-vine stack -->
This PR is part of a stack containing 16 PRs:

1. `main`
2. **"coretests: Add more pattern tests." (this PR)**
3. rust-lang#161606 ([Compare](https://github.com/pacak/rust/compare/more-pat-tests..generalize-pattern))
4. rust-lang#161754 ([Compare](https://github.com/pacak/rust/compare/generalize-pattern..push-qvvokrsvvqww))
5. rust-lang#161755 ([Compare](https://github.com/pacak/rust/compare/push-qvvokrsvvqww..push-wlxvvvxzrsly))
6. rust-lang#161756 ([Compare](https://github.com/pacak/rust/compare/push-wlxvvvxzrsly..push-lkvnuzsvuqzt))
7. rust-lang#161757 ([Compare](https://github.com/pacak/rust/compare/push-lkvnuzsvuqzt..push-wuxtmysppkst))
8. rust-lang#161758 ([Compare](https://github.com/pacak/rust/compare/push-wuxtmysppkst..push-tnkxmkmykopl))
9. rust-lang#161759 ([Compare](https://github.com/pacak/rust/compare/push-tnkxmkmykopl..push-umzmopsxmwry))
10. rust-lang#161760 ([Compare](https://github.com/pacak/rust/compare/push-umzmopsxmwry..push-ptqpwloxpyru))
11. rust-lang#161761 ([Compare](https://github.com/pacak/rust/compare/push-ptqpwloxpyru..push-rksmzopsvzlk))
12. rust-lang#161762 ([Compare](https://github.com/pacak/rust/compare/push-rksmzopsvzlk..push-mkxpywoqusmn))
13. rust-lang#161608 ([Compare](https://github.com/pacak/rust/compare/push-mkxpywoqusmn..flavor-pattern))
14. rust-lang#161763 ([Compare](https://github.com/pacak/rust/compare/flavor-pattern..push-xoslomrztmly))
15. rust-lang#161764 ([Compare](https://github.com/pacak/rust/compare/push-xoslomrztmly..push-pqpumynltyml))
16. rust-lang#161765 ([Compare](https://github.com/pacak/rust/compare/push-pqpumynltyml..push-svrvnxkpuqul))
17. rust-lang#161610 ([Compare](https://github.com/pacak/rust/compare/push-svrvnxkpuqul..osstr-pattern-tip))
<!-- end jj-vine stack -->
@pacak
pacak force-pushed the push-qvvokrsvvqww branch 2 times, most recently from a514b07 to 2593e0c Compare September 17, 2026 11:14
JonathanBrouwer added a commit to JonathanBrouwer/rust that referenced this pull request Sep 17, 2026
@JonathanBrouwer
Rollup merge of rust-lang#161596 - pacak:more-pat-tests, r=nia-e
8b0c9ba 
coretests: Add more pattern tests.

Right now things are undertested and underspecified.

Some of the library code would get in a loop if searcher starts returning empty rejects.

And there's no tests for backwards multi byte char matchers. Pull request I'm reviving had a problem implementing that, so making sure it's tested before the actual code lands.

Right now it is possible to break both tests (and user code) without breaking anything else in the test suite I think.

This commit is extracted from rust-lang#160971 with slight modifications.

<!-- start jj-vine stack -->
This PR is part of a stack containing 16 PRs:

1. `main`
2. **"coretests: Add more pattern tests." (this PR)**
3. rust-lang#161606 ([Compare](https://github.com/pacak/rust/compare/more-pat-tests..generalize-pattern))
4. rust-lang#161754 ([Compare](https://github.com/pacak/rust/compare/generalize-pattern..push-qvvokrsvvqww))
5. rust-lang#161755 ([Compare](https://github.com/pacak/rust/compare/push-qvvokrsvvqww..push-wlxvvvxzrsly))
6. rust-lang#161756 ([Compare](https://github.com/pacak/rust/compare/push-wlxvvvxzrsly..push-lkvnuzsvuqzt))
7. rust-lang#161757 ([Compare](https://github.com/pacak/rust/compare/push-lkvnuzsvuqzt..push-wuxtmysppkst))
8. rust-lang#161758 ([Compare](https://github.com/pacak/rust/compare/push-wuxtmysppkst..push-tnkxmkmykopl))
9. rust-lang#161759 ([Compare](https://github.com/pacak/rust/compare/push-tnkxmkmykopl..push-umzmopsxmwry))
10. rust-lang#161760 ([Compare](https://github.com/pacak/rust/compare/push-umzmopsxmwry..push-ptqpwloxpyru))
11. rust-lang#161761 ([Compare](https://github.com/pacak/rust/compare/push-ptqpwloxpyru..push-rksmzopsvzlk))
12. rust-lang#161762 ([Compare](https://github.com/pacak/rust/compare/push-rksmzopsvzlk..push-mkxpywoqusmn))
13. rust-lang#161608 ([Compare](https://github.com/pacak/rust/compare/push-mkxpywoqusmn..flavor-pattern))
14. rust-lang#161763 ([Compare](https://github.com/pacak/rust/compare/flavor-pattern..push-xoslomrztmly))
15. rust-lang#161764 ([Compare](https://github.com/pacak/rust/compare/push-xoslomrztmly..push-pqpumynltyml))
16. rust-lang#161765 ([Compare](https://github.com/pacak/rust/compare/push-pqpumynltyml..push-svrvnxkpuqul))
17. rust-lang#161610 ([Compare](https://github.com/pacak/rust/compare/push-svrvnxkpuqul..osstr-pattern-tip))
<!-- end jj-vine stack -->
JonathanBrouwer added a commit to JonathanBrouwer/rust that referenced this pull request Sep 17, 2026
@JonathanBrouwer
Rollup merge of rust-lang#161596 - pacak:more-pat-tests, r=nia-e
92b9d6c 
coretests: Add more pattern tests.

Right now things are undertested and underspecified.

Some of the library code would get in a loop if searcher starts returning empty rejects.

And there's no tests for backwards multi byte char matchers. Pull request I'm reviving had a problem implementing that, so making sure it's tested before the actual code lands.

Right now it is possible to break both tests (and user code) without breaking anything else in the test suite I think.

This commit is extracted from rust-lang#160971 with slight modifications.

<!-- start jj-vine stack -->
This PR is part of a stack containing 16 PRs:

1. `main`
2. **"coretests: Add more pattern tests." (this PR)**
3. rust-lang#161606 ([Compare](https://github.com/pacak/rust/compare/more-pat-tests..generalize-pattern))
4. rust-lang#161754 ([Compare](https://github.com/pacak/rust/compare/generalize-pattern..push-qvvokrsvvqww))
5. rust-lang#161755 ([Compare](https://github.com/pacak/rust/compare/push-qvvokrsvvqww..push-wlxvvvxzrsly))
6. rust-lang#161756 ([Compare](https://github.com/pacak/rust/compare/push-wlxvvvxzrsly..push-lkvnuzsvuqzt))
7. rust-lang#161757 ([Compare](https://github.com/pacak/rust/compare/push-lkvnuzsvuqzt..push-wuxtmysppkst))
8. rust-lang#161758 ([Compare](https://github.com/pacak/rust/compare/push-wuxtmysppkst..push-tnkxmkmykopl))
9. rust-lang#161759 ([Compare](https://github.com/pacak/rust/compare/push-tnkxmkmykopl..push-umzmopsxmwry))
10. rust-lang#161760 ([Compare](https://github.com/pacak/rust/compare/push-umzmopsxmwry..push-ptqpwloxpyru))
11. rust-lang#161761 ([Compare](https://github.com/pacak/rust/compare/push-ptqpwloxpyru..push-rksmzopsvzlk))
12. rust-lang#161762 ([Compare](https://github.com/pacak/rust/compare/push-rksmzopsvzlk..push-mkxpywoqusmn))
13. rust-lang#161608 ([Compare](https://github.com/pacak/rust/compare/push-mkxpywoqusmn..flavor-pattern))
14. rust-lang#161763 ([Compare](https://github.com/pacak/rust/compare/flavor-pattern..push-xoslomrztmly))
15. rust-lang#161764 ([Compare](https://github.com/pacak/rust/compare/push-xoslomrztmly..push-pqpumynltyml))
16. rust-lang#161765 ([Compare](https://github.com/pacak/rust/compare/push-pqpumynltyml..push-svrvnxkpuqul))
17. rust-lang#161610 ([Compare](https://github.com/pacak/rust/compare/push-svrvnxkpuqul..osstr-pattern-tip))
<!-- end jj-vine stack -->
rust-bors Bot pushed a commit that referenced this pull request Sep 17, 2026
@bors
Unrolled build for #161596 in rollup 162907
e06b230 
Rollup merge of #161596 - pacak:more-pat-tests, r=nia-e

coretests: Add more pattern tests.

Right now things are undertested and underspecified.

Some of the library code would get in a loop if searcher starts returning empty rejects.

And there's no tests for backwards multi byte char matchers. Pull request I'm reviving had a problem implementing that, so making sure it's tested before the actual code lands.

Right now it is possible to break both tests (and user code) without breaking anything else in the test suite I think.

This commit is extracted from #160971 with slight modifications.

<!-- start jj-vine stack -->
This PR is part of a stack containing 16 PRs:

1. `main`
2. **"coretests: Add more pattern tests." (this PR)**
3. #161606 ([Compare](https://github.com/pacak/rust/compare/more-pat-tests..generalize-pattern))
4. #161754 ([Compare](https://github.com/pacak/rust/compare/generalize-pattern..push-qvvokrsvvqww))
5. #161755 ([Compare](https://github.com/pacak/rust/compare/push-qvvokrsvvqww..push-wlxvvvxzrsly))
6. #161756 ([Compare](https://github.com/pacak/rust/compare/push-wlxvvvxzrsly..push-lkvnuzsvuqzt))
7. #161757 ([Compare](https://github.com/pacak/rust/compare/push-lkvnuzsvuqzt..push-wuxtmysppkst))
8. #161758 ([Compare](https://github.com/pacak/rust/compare/push-wuxtmysppkst..push-tnkxmkmykopl))
9. #161759 ([Compare](https://github.com/pacak/rust/compare/push-tnkxmkmykopl..push-umzmopsxmwry))
10. #161760 ([Compare](https://github.com/pacak/rust/compare/push-umzmopsxmwry..push-ptqpwloxpyru))
11. #161761 ([Compare](https://github.com/pacak/rust/compare/push-ptqpwloxpyru..push-rksmzopsvzlk))
12. #161762 ([Compare](https://github.com/pacak/rust/compare/push-rksmzopsvzlk..push-mkxpywoqusmn))
13. #161608 ([Compare](https://github.com/pacak/rust/compare/push-mkxpywoqusmn..flavor-pattern))
14. #161763 ([Compare](https://github.com/pacak/rust/compare/flavor-pattern..push-xoslomrztmly))
15. #161764 ([Compare](https://github.com/pacak/rust/compare/push-xoslomrztmly..push-pqpumynltyml))
16. #161765 ([Compare](https://github.com/pacak/rust/compare/push-pqpumynltyml..push-svrvnxkpuqul))
17. #161610 ([Compare](https://github.com/pacak/rust/compare/push-svrvnxkpuqul..osstr-pattern-tip))
<!-- end jj-vine stack -->
@rust-bors

This comment has been minimized.

Sign in to view
@pacak
core: convert Pattern<'a> into Pattern<H: Haystack>
06edb7c 
Add a Haystack trait describing something that can be searched in and
make core::str::Pattern (and related types) generic on that trait.
This will allow Pattern to be used for types other than str (most
notably OsStr).

This somewhat follows the Pattern API 2.0 design.  While that design is
apparently abandoned (?), it is somewhat helpful when going for patterns
on OsStr, so I’m going with it unless someone tells me otherwise. ;)

For now leave Pattern, Haystack et al in core::str::pattern.  Since
they are no longer str-specific, I’ll move them to core::pattern in
future commit.  This one leaves them in place to make the diff
smaller.

@pacak:

Original code had pattern parametrized by a reference (`Pattern<&str>`),
I changed it to be `Pattern<str>`.

Original code had indices in Haystack abstracted as an associated type
Cursor. Replaced with usize - Cursor adds noise with not much value.

Changed wording in 2-3 places - for example Searcher is generic over a
few types so it makes more sense to talk about split points in general
with utf8 split points as an example for `&str`.
@pacak
core: move Pattern et al to core::pattern module
5ad3f42 
Pattern is no longer str-specific, so move it from core::str::pattern
module to a new core::pattern module.  This introduces no changes in
behaviour or implementation.  Just moves stuff around and adjusts
documentation.
@pacak
core: introduce internal core::pattern::{Split,SplitN} types
fe6d2d5 
Introduce core::pattern::Split and core::pattern::SplitN internal types
which can be used to implement iterators splitting haystack into parts.
Convert str’s Split-family of iterators to use them.  In the future,
more haystacks will use those internal types.

Co-authored-by: Peter Jaszkowiak <p.jaszkow@gmail.com>

@pacak: Fixed some typos, added a few `#[inline]`. Since there's no
`H::Cursor` - I had to add `ctx: PhantomData<H>`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@nia-e nia-e

Labels

has-merge-commits PR has merge commits, merge with caution. S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-libs Relevant to the library team, which will review and decide on the PR/issue.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@pacak @rustbot @nia-e