Scope leading-particle behaviour to the default name order, and fix the ambiguity detail (#355)

AGENTS.md

@@ -176,7 +176,7 @@ The 2.0 rewrite lands as underscore-private modules alongside the v1 code. These
 - **Method organization**, fixed section order in every class: fields + `__post_init__` validation → alternative constructors → dunders (construction/equality → protocol → operators) → properties → public methods by concern (access → editing → comparison → rendering delegates) → private helpers last, except a helper serving exactly one section may sit at that section's head. Sanctioned deviation, facade layer only: `HumanName` and the shim `Constants` organize by v1 concern groups (`# -- render defaults --`, `# -- config / parsing --`, `# -- fields --`, ..., dunders and pickle last) — the classes mirror v1's own surface and die in 3.0; the canonical order still binds every core type.
 - **Validation is eager and fail-loud**: every `raise` states the offending value, the expected form, and the fix. Exception taxonomy: wrong type — including wrong element type inside a collection, bare `str` where an iterable of strings is expected, or a `Mapping` where a plain iterable is expected — raises `TypeError`; well-typed but unacceptable values raise `ValueError`; failed enum lookups stay `ValueError` for any input (stdlib `EnumType` precedent). **When the message hands the reader code to paste, that code has to survive a type checker** — nameparser ships `py.typed`. #337's segmenterless warning offered `Policy(segment_scripts=())`, an `arg-type` error, because these fields are annotated with what they STORE rather than everything the constructor accepts. Prefer the `frozenset()` / `()` spellings in messages and docstrings, and pin the offered spelling in a test — the warning tests matched on `ja_segmenter` and never checked the actionable half of the message. **A warning emitted in `Parser.__post_init__` needs `parser_for` to re-emit it from its own frame** (the `catch_warnings(record=True)` block at its return): `__post_init__`'s `stacklevel` is sized for direct `Parser(...)` construction, and through `parser_for`'s extra frame the default one-line rendering attributes the warning to the library's own `return Parser(...)` — the exact call the message tells the user to change becomes invisible. No single stacklevel serves both entry points; a new construction warning gets the re-emission for free, but a new CONSTRUCTION SITE for `Parser` inside this package needs its own re-emission or its callers get library-attributed warnings (#337 review).
 - **Guard, hint, and emit for the WHOLE family, and parametrize the test over it**: a check added to one member of a set belongs on all of it, and the test must sweep the family, not one example. This session shipped `_reject_str_and_mapping` on `Policy` but not `PolicyPatch`, the bytes decode hint on three of five config entry points, and a regex-sync roster missing four of its copies — each a separate follow-up bug that a `{class} × {field} × {bad-value}` parametrization would have caught and a per-example test hid. When you find you're guarding member N, grep for the other members first.
-- **Ambiguities are emitted at the DECISION site**: an `Ambiguity` records a fork the parse had to call, not a token that sits in an ambiguous vocabulary. Emit where the branch is taken — the trailing-suffix peel in `_assign`, the delimiter escape's follow-up in `classify` — never by scanning for a `vocab:*-ambiguous` tag. The same tagged token is a genuine fork in one position and unremarkable in another (`do` mid-name in "Joao da Silva do Amaral de Souza" chooses nothing). **A branch that runs but changes nothing is not a decision either** -- the prefix chain's `merge(k, j)` executes even when `j == k + 1`, folding a piece into itself, and keying on "the code got here" reported a fork for all 39 ambiguous particles on "Dr. Van Jr.", where the particle stayed the GIVEN name and `_assign` reported the same token again. Check that the branch actually claimed something (`j > k + 1`) before recording. Structure also structure often settles the question before it arises, which is why `PARTICLE_OR_GIVEN` is deliberately not emitted on the `FAMILY_COMMA` path and `SUFFIX_OR_NAME` is not emitted for "Ma, Jack". The decision site also has the token index and the detail text in hand, which the tag scan would have to reconstruct. **If a fork's two branches are taken in DIFFERENT stages, every one of them needs the emitter** -- `PARTICLE_OR_GIVEN` is decided in `_assign` when the ambiguous particle stays a lone leading piece and in `_group` when a title shifts it off index 0 and the prefix chain claims it, so both report; for two years only the first did. The stage-ownership map in `tests/v2/pipeline/test_state.py` must list `ambiguities` for each such stage, and it passes vacuously until a case row exercises the path, so add the row too. Report BOTH directions of a two-way fork — "John Smith MA" (read as a suffix) and "Jack MA" (read as the family name) are equally guesses. Every kind needs a trigger in `tests/v2/test_contracts.py::_AMBIGUITY_TRIGGERS` (an explicit `None`, strict-xfail, while reserved), and case-table rows pin expected kinds exactly, so a new emitter shows up in both immediately. **Pin the decision, not the vocabulary**: the only titled-particle test used an UNAMBIGUOUS particle, so it walked the right code path and proved nothing about the branch under test -- two criticals passed 1539 tests. A row contrasting the two readings ("John Smith V" against "John Smith B") is what makes an emitter's absence meaningful.
+- **Ambiguities are emitted at the DECISION site**: an `Ambiguity` records a fork the parse had to call, not a token that sits in an ambiguous vocabulary. Emit where the branch is taken — the trailing-suffix peel in `_assign`, the delimiter escape's follow-up in `classify` — never by scanning for a `vocab:*-ambiguous` tag. The same tagged token is a genuine fork in one position and unremarkable in another (`do` mid-name in "Joao da Silva do Amaral de Souza" chooses nothing). **A branch that runs but changes nothing is not a decision either** -- the prefix chain's `merge(k, j)` executes even when `j == k + 1`, folding a piece into itself, and keying on "the code got here" reported a fork for all 39 ambiguous particles on "Dr. Van Jr.", where the particle stayed a lone leading name piece — the GIVEN name under the default order, the family name under `FAMILY_FIRST` — and `_assign` reported the same token again. Check that the branch actually claimed something (`j > k + 1`) before recording. Structure also structure often settles the question before it arises, which is why `PARTICLE_OR_GIVEN` is deliberately not emitted on the `FAMILY_COMMA` path and `SUFFIX_OR_NAME` is not emitted for "Ma, Jack". The decision site also has the token index and the detail text in hand, which the tag scan would have to reconstruct. **If a fork's two branches are taken in DIFFERENT stages, every one of them needs the emitter** -- `PARTICLE_OR_GIVEN` is decided in `_assign` when the ambiguous particle stays a lone leading piece and in `_group` when a title shifts it off index 0 and the prefix chain claims it, so both report; for two years only the first did. The stage-ownership map in `tests/v2/pipeline/test_state.py` must list `ambiguities` for each such stage, and it passes vacuously until a case row exercises the path, so add the row too. Report BOTH directions of a two-way fork — "John Smith MA" (read as a suffix) and "Jack MA" (read as the family name) are equally guesses. Every kind needs a trigger in `tests/v2/test_contracts.py::_AMBIGUITY_TRIGGERS` (an explicit `None`, strict-xfail, while reserved), and case-table rows pin expected kinds exactly, so a new emitter shows up in both immediately. **Pin the decision, not the vocabulary**: the only titled-particle test used an UNAMBIGUOUS particle, so it walked the right code path and proved nothing about the branch under test -- two criticals passed 1539 tests. A row contrasting the two readings ("John Smith V" against "John Smith B") is what makes an emitter's absence meaningful.
 - **A kind is worth adding only if a reader would hesitate too**: the test is not "does the code take a branch" but whether a person reading that input would genuinely be unsure. "Smith, John V" reads as a middle initial to anyone -- the comma settles it -- so reporting it would be noise that teaches callers to ignore the field, which costs more than the missing report. Reachability of the second branch is necessary, not sufficient. Prefer leaving a fork silent and documenting the omission (see the comma paths in concepts.rst) over emitting on input nobody finds ambiguous.
 - **Parser owns config-dependent conveniences**: `Parser.matches`/`Parser.capitalized`/`Parser.revise` exist because the `ParsedName` equivalents fall back to DEFAULT config for str/omitted arguments (documented loudly in both docstrings). `revise` harvests tokens from a full sub-parse of each replacement value (tags kept minus `FOLDED_TAG`, roles forced, ambiguities discarded); the merge tail is shared with `replace()` via `ParsedName._with_field_tokens`. `Parser.capitalized` delegates through `name.capitalized(self.lexicon)` specifically so `_parser` never imports `_render` — keep it that way.
 - **Per-word vocabulary fields warn on multi-word entries** (`_normset`/`_normpairs` via `_warn_dead_entry`, UserWarning, never a raise — see the given_name_titles Gotcha for why raising is wrong). `given_name_titles` is the one multi-word-matched field and is exempt; `_edit` passes `warn=False` (add() warns once via the new instance's `__post_init__`; remove() stores nothing). The default vocabulary and every locale pack must stay warning-free (`test_default_lexicon_builds_warning_free`, `test_pack_vocabulary_entries_are_single_words`).

docs/concepts.rst

@@ -159,11 +159,14 @@ Some calls are irreducibly ambiguous — both readings are legitimate,
 and no amount of rule-tuning resolves them without breaking some other
 name. Those surface as entries on ``ParsedName.ambiguities`` instead
 of being silently guessed away. The canonical example: a leading "Van"
-reads as a given name — the right call for the actor Van Johnson, the
-wrong one for a bare "Van Buren", and nothing in the two-word shape
-distinguishes them — so the parse records a ``particle-or-given``
-ambiguity alongside its answer. You can inspect ``ambiguities`` to decide, case
-by case, whether your data needs a second look.
+reads as a name of its own rather than as the start of a surname —
+the right call for the actor Van Johnson, the wrong one for a bare
+"Van Buren", and nothing in the two-word shape distinguishes them — so
+the parse records a ``particle-or-given`` ambiguity alongside its
+answer. (Which name that piece then becomes is a separate question,
+and ``name_order``'s: the given name under the default.) You can
+inspect ``ambiguities`` to decide, case by case, whether your data
+needs a second look.
 
 An ambiguity records a *decision*, not a word. The same token in a
 different position may present no fork at all: ``do`` is in the

docs/customize.rst

@@ -166,10 +166,14 @@ a suffix only when written with periods:
     'M.A.'
 
 ``particles_ambiguous`` is the same idea for surname particles. A
-particle listed there may also be a given name, so a name that starts
-with one keeps its given name; a particle *not* listed there is never a
-given name, so a name starting with it has no given name at all — the
-whole thing is the surname:
+particle listed there may also be a given name, which is what makes a
+leading one a decision to take; a particle *not* listed there never
+is, so there is nothing to decide. Under the default name order that
+shows up as whether the name has a given name at all: one that starts
+with a listed particle keeps it, while one starting with an unlisted
+particle has no given name — the whole thing is the surname. (Which
+field each piece lands in is ``name_order``'s question, covered
+below.)
 
 .. doctest::
 
@@ -181,7 +185,9 @@ whole thing is the surname:
     'de Mesnil'
 
 If your data never uses ``Van`` as a given name, take it out of the
-ambiguous set and leading ``van`` becomes part of the surname:
+ambiguous set: a leading ``van`` is then no decision at all, so no
+ambiguity is recorded, and under the default order it becomes part of
+the surname:
 
 .. doctest::
 

docs/release_log.rst

@@ -20,6 +20,10 @@ Release Log
 
     - Change every vocabulary set in ``nameparser.config`` to a ``frozenset``: ``TITLES``, ``GIVEN_NAME_TITLES``, ``SUFFIX_WORDS``, ``SUFFIX_ACRONYMS``, ``SUFFIX_ACRONYMS_AMBIGUOUS``, ``GLUED_HONORIFICS``, ``PARTICLES``, ``NON_GIVEN_NAME_PARTICLES``, ``BOUND_GIVEN_NAMES``, ``CONJUNCTIONS`` and ``MAIDEN_MARKERS`` (``KOREAN_SURNAMES`` already was one). Editing one in place -- ``TITLES.add("dean")``, the old way of changing a global default -- now raises ``AttributeError: 'frozenset' object has no attribute 'add'`` at the line that writes it. It was never a reliable way to change a default: whether an edit reached a given parse depended on which config objects had already been built, so one program could hold two disagreeing defaults with nothing to say so. To change the defaults for ``HumanName``, build a private ``Constants`` and pass it (``c = Constants(); c.titles.add("dean"); HumanName(name, constants=c)``); mutating the shared ``CONSTANTS`` still works, but warns and goes away in 3.0. For the 2.0 API, build a lexicon and pass it to a parser (``Parser(lexicon=Lexicon.default().add(titles={"dean"}))``). Neither is affected by this change. ``CAPITALIZATION_EXCEPTIONS`` is a mapping, not a set, and is unchanged. See :doc:`migrate` and :doc:`customize` (#293)
 
+    **Behavior Changes**
+
+    - Change the ``detail`` text of a ``PARTICLE_OR_GIVEN`` ambiguity to name the role the leading particle was actually given. It said "read as a given name" under every ``name_order``, which is false under ``Policy(name_order=FAMILY_FIRST)`` -- there ``"Van Johnson"`` reads as family ``Van``, given ``Johnson``, and the report described the reading not taken. It now ends "read as a family name" in that case, reading the role off the assigned token the way ``SUFFIX_OR_NAME`` already did -- that kind names both parts (``read as a family name rather than a post-nominal``), while this one names only the part it took. The ``kind`` is unchanged and stays ``PARTICLE_OR_GIVEN``: the fork really is particle-or-given, and only the human-readable text moved. Default-order output is identical (#355)
+
     **Deprecations**
 
     - Rename the four vocabularies whose 1.x names described the fields they feed in v1's words, so the data layer matches the ``Lexicon``:

docs/usage.rst

@@ -116,9 +116,12 @@ names together as easily as two surnames:
     'de la Vega y Rodriguez'
 
 Position matters in exactly one place: the start of a name. A particle
-there has no surname to attach to yet, so it either becomes the given
-name or turns the whole name into a surname, depending on whether it is
-one that can double as a given name:
+there has no surname to attach to yet, so what decides the reading is
+whether it is one that can double as a given name. Where the pieces
+then land is ``name_order``'s question — see :doc:`customize` — and
+the destinations below are the default given-first order's: the
+particle either becomes the given name or turns the whole name into a
+surname:
 
 .. doctest::
 

nameparser/_lexicon.py

@@ -332,10 +332,21 @@ class Lexicon:
     #: ("van", "de", "bin", ...). Full default list:
     #: :data:`~nameparser.config.particles.PARTICLES`.
     particles: frozenset[str] = frozenset()
-    #: Subset of particles that can also BE a given name: a leading
-    #: one reads as given and records a particle-or-given ambiguity
-    #: ("Van Johnson", but also "Van Buren"). No constant of its own
-    #: -- the default derives
+    #: Subset of particles that can also BE a given name ("Van
+    #: Johnson", but also "Van Buren"). Membership decides nothing
+    #: about chaining: the prefix chain skips index 0 unconditionally
+    #: and never consults this set, so it leaves a leading particle a
+    #: piece of its own whether listed or not -- "de Mesnil" groups
+    #: into two pieces exactly as "van Gogh" does. What membership
+    #: decides is what becomes of that piece afterwards. Under EITHER
+    #: ``name_order`` a member records a particle-or-given ambiguity
+    #: and a non-member records none; under the default given-first
+    #: order a non-member is additionally folded back into the family
+    #: name once roles exist, so the whole name is the surname ("de
+    #: Mesnil" -- a bare "de", with nothing to fold into, is left
+    #: alone). Which field each piece lands in is ``name_order``'s
+    #: question, not this set's.
+    #: No constant of its own -- the default derives
     #: as particles minus
     #: :data:`~nameparser.config.particles.NON_GIVEN_NAME_PARTICLES`
     #: (which marks the opposite, never-given subset).

nameparser/_pipeline/_assign.py

@@ -274,10 +274,18 @@ def _assign_main(seg_idx: int, state: ParseState,
         head = pieces[name_pieces[0]]
         if (len(head) == 1 and len(name_pieces) > 1
                 and "vocab:particle-ambiguous" in tokens[head[0]].tags):
+            # the loops above gave the head piece its role from
+            # `order`, which is _effective_order's answer and not
+            # necessarily name_order's -- a script_orders entry
+            # overrides it. So read the role off the token rather than
+            # assume given, or re-derive it here; same reason as
+            # SUFFIX_OR_NAME just above.
+            token = tokens[head[0]]
+            assert token.role is not None
             ambiguities.append(PendingAmbiguity(
                 AmbiguityKind.PARTICLE_OR_GIVEN,
-                f"leading {tokens[head[0]].text!r} may be a family-name "
-                f"particle; read as a given name",
+                f"leading {token.text!r} may be a family-name "
+                f"particle; read as a {token.role.value} name",
                 tuple(head)))
 
 

nameparser/_pipeline/_group.py

@@ -211,7 +211,8 @@ def merge(lo: int, hi: int, add: Set[str] = frozenset(),
                 j += 1
             # The other half of PARTICLE_OR_GIVEN. _assign reports the
             # fork when an ambiguous particle stays a lone leading piece
-            # ("Van Johnson" -> given); the chain here takes the
+            # ("Van Johnson" -> given under the default order, family
+            # under FAMILY_FIRST); the chain here takes the
             # opposite branch whenever a title shifts it off index 0
             # ("Dr. Van Johnson" -> family "Van Johnson"). A fork whose
             # two sides are decided in different stages needs an emitter

nameparser/_types.py

@@ -356,10 +356,20 @@ class AmbiguityKind(StrEnum):
     #: Smith B"). Which name part was declined depends on position and
     #: ``name_order``, so ``detail`` names it rather than the kind.
     SUFFIX_OR_NAME = "suffix-or-name"
-    #: A leading ambiguous particle was read as a given name -- the
-    #: right call for "Van Johnson" (the actor's given name), the
-    #: wrong one for a bare "Van Buren" (the presidential surname);
-    #: the two-word shape cannot distinguish them.
+    #: An ambiguous particle at the head of a name is either a
+    #: particle or a name in its own right -- "Van Johnson" is the
+    #: actor's given name, a bare "Van Buren" the presidential
+    #: surname, and the two-word shape cannot distinguish them. Two
+    #: shapes report this kind, decided in different stages, and
+    #: ``detail`` is what tells them apart. A particle left standing
+    #: alone chained nothing and was assigned a role, which ``detail``
+    #: names ("read as a given name") -- that role is whatever
+    #: assignment gave it, so it follows ``name_order`` and any
+    #: ``script_orders`` entry, which is why the kind cannot name it.
+    #: A particle a title shifted off the front ("Dr. Van Johnson")
+    #: was instead claimed by the prefix chain, and ``detail`` says
+    #: that and names no field at all: grouping runs before roles
+    #: exist, so that text is the same under every order.
     PARTICLE_OR_GIVEN = "particle-or-given"
     #: A nickname/maiden delimiter opened without closing (or closed
     #: without opening); the text was kept as literal name content, so