UnicodeSet documentation by eggrobin · Pull Request #4171 · unicode-org/icu

eggrobin · 2026-09-08T18:45:07Z

TODO: Please describe your changes here.

TODO: Please read the following on ICU Contributing, and then delete these instructions.

Thank you for your pull request!

For general info on contributing: https://github.com/unicode-org/icu/blob/main/CONTRIBUTING.md
Associating PRs with Jira issues
- We require each pull request to be associated with a Jira issue.
- Reuse existing issues for minor changes:
  - ICU 79 docs minor fixes: ICU-23248 — User Guide & API docs typos etc., and version updates (e.g., dependabot for User Guide)
  - ICU 79 code warnings/version updates: ICU-23247 — Fix compiler warnings. Update versions of code-related dependencies (e.g., dependabot).
Contributors license agreement (CLA):
- You will be automatically asked to sign the CLA before the PR is accepted.
- To sign the CLA: https://cla-assistant.io/unicode-org/icu
- For terms of use and license, see https://www.unicode.org/copyright.html

TODO: Fill out the checklist below.

Checklist

Required: Issue filed: ICU-NNNNN
Required: The PR title must be prefixed with a JIRA Issue number. Example: "ICU-NNNNN Fix xyz"
Required: Each commit message must be prefixed with a JIRA Issue number. Example: "ICU-NNNNN Fix xyz"
Issue accepted (done by Technical Committee after discussion)
Tests included, if applicable
API docs and/or User Guide docs changed or added, if applicable
Approver: Feel free to merge on my behalf

markusicu · 2026-09-09T20:28:13Z

-character strings. The contents of that object can be specified either by
-patterns or by building them programmatically.
+A UnicodeSet is an object that represents a finite set of Unicode code point
+sequences.  The contents of that object can be specified either by


Suggested change

sequences. The contents of that object can be specified either by

sequences, optimized for single code points. The contents of that object can be specified either by

markusicu · 2026-09-09T20:28:46Z

-character strings. The contents of that object can be specified either by
-patterns or by building them programmatically.
+A UnicodeSet is an object that represents a finite set of Unicode code point
+sequences.  The contents of that object can be specified either by


Please avoid double spaces between sentences.

markusicu · 2026-09-09T20:29:03Z

-patterns or by building them programmatically.
+A UnicodeSet is an object that represents a finite set of Unicode code point
+sequences.  The contents of that object can be specified either by
+patterns using the UnicodeSet syntax defined in 


Suggested change

patterns using the UnicodeSet syntax defined in

pattern strings using the UnicodeSet syntax defined in

markusicu · 2026-09-09T20:30:06Z

+ * <em>character classes</em> used in regular expressions. A UnicodeSet specifies a finite set of
+ * Unicode code points sequences.


Suggested change

* <em>character classes</em> used in regular expressions. A UnicodeSet specifies a finite set of

* Unicode code points sequences.

* <em>character classes</em> used in regular expressions. A UnicodeSet specifies a finite set of

* Unicode code point sequences, optimized for single code points.

markusicu · 2026-09-09T20:33:39Z

+ * toPattern()</code> API from the <code>
 * java.text.Format</code>-derived classes. Unlike the methods that add characters, add categories,
 * and control the logic of the set, the method <code>applyPattern()</code> sets all attributes of a
 * <code>UnicodeSet</code> at once, based on a string pattern.


Suggested change

* <code>UnicodeSet</code> at once, based on a string pattern.

* <code>UnicodeSet</code> at once, based on a pattern string.

markusicu · 2026-09-09T21:01:54Z

+* `\u` [four-hexadecimal-digits](https://www.unicode.org/reports/tr61/#four-hexadecimal-digits)
+  `\u` [four-hexadecimal-digits](https://www.unicode.org/reports/tr61/#four-hexadecimal-digits)
+  where the first constituent  [four-hexadecimal-digits](https://www.unicode.org/reports/tr61/#four-hexadecimal-digits)
+  represent a high surrogate and the second constituent
+  [four-hexadecimal-digits](https://www.unicode.org/reports/tr61/#four-hexadecimal-digits)


optional:
when you refer to the same grammar production multiple times closely together, i would put a link only on the first one; the rest can just be plain text
(more readable .md file)

markusicu · 2026-09-09T21:07:36Z

+* `$` is added as a [set-operator](https://www.unicode.org/reports/tr61/#set-operator), and the
+  following alternatives are added to [Content](https://www.unicode.org/reports/tr61/#Content)</a>:
+  > | `$`  
+  > | [ElementList](https://www.unicode.org/reports/tr61/#ElementList) `$`  
+  > | [UnescapedHyphenMinus](https://www.unicode.org/reports/tr61/#UnescapedHyphenMinus) [ElementList](https://www.unicode.org/reports/tr61/#ElementList) `$`
+  The following alternative is added to [ElementList](https://www.unicode.org/reports/tr61/#ElementList):
+  > | `$` [Elements](https://www.unicode.org/reports/tr61/#Elements)
+  The following alternative is added to [Union](https://www.unicode.org/reports/tr61/#Union):
+  > | `$` [UnicodeSet](https://www.unicode.org/reports/tr61/#UnicodeSet)
+  When the [set-operator](https://www.unicode.org/reports/tr61/#set-operator) `$` occurs
+  as an immediate constituent of a [Content](https://www.unicode.org/reports/tr61/#Content)</a>,
+  it represents the noncharacter code point U+FFFF.
+  When it occurs anywhere else, it represents the character U+0024 $ DOLLAR SIGN.


please precede the legalese with some prose like "a literal $ at the end of a set represents the noncharacter code point U+FFFF"

... and maybe even add something like "this is used in Transliterator rule syntax"
with a link to
https://unicode-org.github.io/icu/userguide/transforms/general/rules.html#%C3%A6ther

Yes, I was thinking about that as I was updating the comments in the parser accordingly (the legalese is going to get worse!).

markusicu · 2026-09-09T21:09:38Z

+  element is introduced:
+  > variable ⩴ $ reference
+  where the function `SymbolTable::parseReference` defines the syntactic category
+  reference. The expansion of a variable is defined by `SymbolTable::lookup`; it


I think the word reference wants to be quoted or emphasized or something, to suggest that it's the term introduced in the lexical element above

markusicu · 2026-09-09T21:10:15Z

-> :point_right: **Note**:*ICU Regular Expression set expressions have a different (but similar) syntax,
-and a different set of recognized backslash escapes. \[Sets\] in ICU Regular
-Expressions follow the conventions from Perl and Java regular expressions rather
-than the pattern syntax from ICU UnicodeSet.*


I think we should keep this note somewhere on this page

markusicu · 2026-09-09T21:12:09Z

+ * the support for variables interacts with the API: If a {@link SymbolTable} is passed to the
+ * constructor of {@code UnicodeSet}, a new lexical element is introduced:
 *
 * <blockquote>
 *
- * <table>
- *     <tr style="vertical-align: top">
- *       <td style="white-space: nowrap; vertical-align: top;text-align:right;"><code>pattern :=&nbsp; </code></td>
- *       <td style="vertical-align: top;"><code>('[' '^'? item* ']') |
- *       property</code></td>
- *     </tr>
- *     <tr style="vertical-align: top">
- *       <td style="white-space: nowrap; vertical-align: top;text-align:right;"><code>item :=&nbsp; </code></td>
- *       <td style="vertical-align: top;"><code>char | (char '-' char) | pattern-expr<br>
- *       </code></td>
- *     </tr>
- *     <tr style="vertical-align: top">
- *       <td style="white-space: nowrap; vertical-align: top;text-align:right;"><code>pattern-expr :=&nbsp; </code></td>
- *       <td style="vertical-align: top;"><code>pattern | pattern-expr pattern |
- *       pattern-expr op pattern<br>
- *       </code></td>
- *     </tr>
- *     <tr style="vertical-align: top">
- *       <td style="white-space: nowrap; vertical-align: top;text-align:right;"><code>op :=&nbsp; </code></td>
- *       <td style="vertical-align: top;"><code>'&amp;' | '-'<br>
- *       </code></td>
- *     </tr>
- *     <tr style="vertical-align: top">
- *       <td style="white-space: nowrap; vertical-align: top;text-align:right;"><code>special :=&nbsp; </code></td>
- *       <td style="vertical-align: top;"><code>'[' | ']' | '-'<br>
- *       </code></td>
- *     </tr>
- *     <tr style="vertical-align: top">
- *       <td style="white-space: nowrap; vertical-align: top;text-align:right;"><code>char :=&nbsp; </code></td>
- *       <td style="vertical-align: top;"><em>any character that is not</em><code> special<br>
- *       | ('\\' </code><em>any character</em><code>)<br>
- *       | ('&#92;u' hex hex hex hex)<br>
- *       </code></td>
- *     </tr>
- *     <tr style="vertical-align: top">
- *       <td style="white-space: nowrap; vertical-align: top;text-align:right;"><code>hex :=&nbsp; </code></td>
- *       <td style="vertical-align: top;"><code>'0' | '1' | '2' | '3' | '4' | '5' | '6' | '7' | '8' | '9' |<br>
- *       &nbsp;&nbsp;&nbsp;&nbsp;'A' | 'B' | 'C' | 'D' | 'E' | 'F' | 'a' | 'b' | 'c' | 'd' | 'e' | 'f'</code></td>
- *     </tr>
- *     <tr>
- *       <td style="white-space: nowrap; vertical-align: top;text-align:right;"><code>property :=&nbsp; </code></td>
- *       <td style="vertical-align: top;"><em>a Unicode property set pattern</em></td>
- *     </tr>
- *   </table>
+ * variable ⩴ <code>$</code> reference


optional: maybe reduce the duplication between this+following and the equivalent User Guide text?

eggrobin added 5 commits September 8, 2026 18:35

Wörter

7784909

Some feedback from Markus

35a7b68

\\

c954a9c

wooords

ab9a851

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

2719952

markusicu self-assigned this Sep 8, 2026

eggrobin added 2 commits September 9, 2026 14:06

words

ff8b3c3

meow

3afc6df

markusicu reviewed Sep 9, 2026

View reviewed changes

Navigation Menu

Uh oh!

Uh oh!

UnicodeSet documentation - #4171

UnicodeSet documentation#4171
eggrobin wants to merge 7 commits into
unicode-org:mainfrom
eggrobin:unicodeset-docs

eggrobin commented Sep 8, 2026

Uh oh!

markusicu Sep 9, 2026

Uh oh!

markusicu Sep 9, 2026

Uh oh!

markusicu Sep 9, 2026

Uh oh!

markusicu Sep 9, 2026

Uh oh!

markusicu Sep 9, 2026

Uh oh!

markusicu Sep 9, 2026

Uh oh!

markusicu Sep 9, 2026

Uh oh!

eggrobin Sep 10, 2026

Uh oh!

markusicu Sep 9, 2026

Uh oh!

markusicu Sep 9, 2026

Uh oh!

markusicu Sep 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	sequences. The contents of that object can be specified either by
	sequences, optimized for single code points. The contents of that object can be specified either by

	patterns using the UnicodeSet syntax defined in
	pattern strings using the UnicodeSet syntax defined in

		* <em>character classes</em> used in regular expressions. A UnicodeSet specifies a finite set of
		* Unicode code points sequences.

	* <code>UnicodeSet</code> at once, based on a string pattern.
	* <code>UnicodeSet</code> at once, based on a pattern string.

Uh oh!

Uh oh!

Conversation

eggrobin commented Sep 8, 2026

Checklist

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants