Split test_regex_sync.py at the seam it already had

AGENTS.md

@@ -87,7 +87,7 @@ uv run sphinx-build -b html docs dist/docs
 #    tools/differential/expected_since_<that version>.toml. Until both are done a
 #    bare compare.py measures against two minors back while reporting the
 #    previous one, and _allowlist_for hard-errors on the missing file.
-#    tests/v2/test_regex_sync.py sweeps every expected_since_*.toml (#333), so
+#    tests/v2/test_ledger_guards.py sweeps every expected_since_*.toml (#333), so
 #    the new ledger's hand copies of _SCRIPT_RANGES and of the honorific and
 #    Latin vocabularies are checked from the day the file lands. All three
 #    rosters find copies by their SYNTAX -- a span class or an alternation --
@@ -188,7 +188,7 @@ The 2.0 rewrite lands as underscore-private modules alongside the v1 code. These
 - **The segmenter contract**: the optional `Parser(segmenter=...)` hook is parse-totality's ONE exception (locales spec section 4). Everything inside that exception is a bug in USER CODE, never a fact about the name, so it is surfaced rather than absorbed: the segmenter's own exceptions propagate, and the two protocol violations the stage can detect for itself — an answer of the wrong type, and one cutting at or past the end of the token it was handed — raise `TypeError`/`ValueError` from `_script_segment` for the same reason. The line to hold when adding a check there: a protocol violation by the segmenter's AUTHOR raises, while an adapter's defense against its own third-party library (`locales/ja.py`'s repertoire, length, reconstruction and score guards) declines with `None`, because what those catch is a fact about the content.
 - **Pickling**: v2 types must round-trip (`Parser` is picklable by construction, and it holds a `Lexicon`; the one qualifier is that a `Parser` pickles iff its segmenter does — see the segmenter bullet above). Every frozen type assigns `_guarded_getstate`/`_guarded_setstate` (`_types.py`) in its class body (`@dataclass(slots=True)` would override inherited pickle methods) — unpickling fails at the LOAD site on field-layout skew, and values are deliberately NOT re-validated (pickle is not a security boundary; canonical state comes from a validated instance). `Lexicon` keeps its own copy of the guard (layering) plus the `mappingproxy` slot rebuild; a new unpicklable slot type needs the same treatment plus a round-trip test.
 - **One sanctioned global**: the (future) cached default `Parser`. Lazily cached FROZEN singletons (`Lexicon.default()`'s `functools.cache`, the future default parser) are constants, not state; any second piece of module-level MUTABLE state requires amending the conventions doc, on purpose, in review. Sanctioned exceptions, facade layer only: `_config_shim.CONSTANTS` (the v1 shared singleton, mutable by design) and `_facade._WARNED_SUBCLASSES` (the once-per-subclass hook-warning dedup set) — both deleted with the layer in 3.0.
-- **Tests**: all v2 tests live in the `tests/v2/` package (its `conftest.py` overrides the v1 dual-run fixture — v2 code never reads shared `CONSTANTS`), one test module per source module plus the cross-cutting ones (`test_reprs.py`, `test_layering.py`, `test_contracts.py`, `test_properties.py`, `test_benchmark.py`, the `cases.py`/`test_cases.py` table, and `test_regex_sync.py`, which pins every hand-copied pattern or codepoint table against its source wherever the copy lives — including copies outside the package), names stating behavior. Never assert `Lexicon.default()` contents; the narrow sourcing spot-checks in `test_default_sources_v1_vocabulary` that pin the v1→v2 migration contract (e.g. the flipped `particles_ambiguous` model) are the sanctioned exception.
+- **Tests**: all v2 tests live in the `tests/v2/` package (its `conftest.py` overrides the v1 dual-run fixture — v2 code never reads shared `CONSTANTS`), one test module per source module plus the cross-cutting ones (`test_reprs.py`, `test_layering.py`, `test_contracts.py`, `test_properties.py`, `test_benchmark.py`, the `cases.py`/`test_cases.py` table, `test_regex_sync.py`, which pins the pipeline's hand-copied patterns against their sources, and `test_ledger_guards.py`, which pins the differential harness's hand copies and bounds what each ledger rule may claim — sharing `_differential_fixtures.py` with `test_differential.py` for the handles on `tools/differential/`), names stating behavior. Never assert `Lexicon.default()` contents; the narrow sourcing spot-checks in `test_default_sources_v1_vocabulary` that pin the v1→v2 migration contract (e.g. the flipped `particles_ambiguous` model) are the sanctioned exception.
 
 ## Extension Patterns
 

tests/v2/_differential_fixtures.py

@@ -0,0 +1,99 @@
+"""Shared handles on the differential harness's own data.
+
+Imported by tests/v2/test_ledger_guards.py and tests/v2/test_differential.py,
+which both need to reach `tools/differential/` -- the ledgers, the
+corpora, and compare.py itself. Neither is importable the ordinary way:
+`tools/` is outside `testpaths` and is not a package, so compare.py is
+loaded by path, and the ledgers and corpora are data files a Python
+module could not import if it wanted to.
+
+Not a conftest: these are constants and plain helpers, not fixtures in
+pytest's sense, and two modules want them by name. Not a test module
+either -- the guards that keep this data honest live next to the tests
+that rely on it, in test_ledger_guards.py.
+"""
+import importlib.util
+import json
+import re
+import tomllib
+from pathlib import Path
+from types import ModuleType
+
+from nameparser import _policy
+
+_TOOLS = Path(__file__).parents[2] / "tools" / "differential"
+
+#: Every baseline's ledger, swept rather than named. #332 added a second
+#: file whose four hand copies went unchecked because the pins in
+#: test_ledger_guards.py named the 1.4 one by filename, and the count grows by one per
+#: release -- see AGENTS.md's release step 8.
+_LEDGERS = sorted(_TOOLS.glob("expected_since_*.toml"))
+
+#: Every name the harness classifies, deduplicated. The ledgers exist
+#: to explain diffs on THESE strings and no others, so "what does this
+#: rule claim?" is answerable here without parsing anything -- a plain
+#: regex search, no baseline wheel, no network.
+#:
+#: This is what test_ledger_guards.py's corpus checks read, and why
+#: they hold where four rounds of syntactic ones did not. Depth-0 pipes, nesting
+#: levels and probe strings are all proxies for the question that
+#: actually matters; a rule cannot widen its corpus reach and still
+#: answer this one the same way, however it is spelled.
+_CORPUS_NAMES = sorted({
+    json.loads(line)
+    for path in sorted(_TOOLS.glob("corpus*.jsonl"))
+    for line in path.read_text(encoding="utf-8").splitlines() if line.strip()})
+
+
+def _claimed(name_regex: str) -> list[str]:
+    """Corpus names a rule's regex matches."""
+    return [name for name in _CORPUS_NAMES if re.search(name_regex, name)]
+
+
+def _unclassified_names() -> list[str]:
+    """Corpus names carrying no codepoint _SCRIPT_RANGES classifies."""
+    has_classified = _policy._script_matcher(*_policy._SCRIPT_RANGES)
+    return [name for name in _CORPUS_NAMES if not has_classified(name)]
+
+
+#: Built once. The expression this replaced sat inside a
+#: comprehension's condition, so it rebuilt the script matcher AND
+#: rescanned all 751 names per candidate name rather than per rule --
+#: measured around 400x a frozenset lookup, machine-dependent. The
+#: rescan was the cost; the rebuild alone is minor.
+_UNCLASSIFIED_NAMES = frozenset(_unclassified_names())
+
+
+def _rules(ledger: Path) -> list[dict]:
+    """The [[change]] table of one ledger."""
+    # .get, matching compare.py. The open cycle's ledger is created at
+    # release with no `change` key at all -- an empty [[change]] array
+    # cannot be appended to in TOML -- so an absent key IS the empty
+    # ledger here, not a malformed file. What stops that leniency from
+    # hiding a typo'd table header lives in tests/v2/test_differential.py:
+    # every other ledger must be non-empty, and the open one may define
+    # nothing but `change`.
+    return tomllib.loads(
+        ledger.read_text(encoding="utf-8")).get("change", [])
+
+
+def load_tool(stem: str) -> ModuleType:
+    """A module from tools/differential/, loaded by path.
+
+    `tools/` is outside testpaths, and adding it would run
+    --doctest-modules over the corpus builders, so these are imported
+    this way rather than made importable. Two callers need it and
+    wrote the same six lines each: test_differential.py for compare.py,
+    test_ledger_guards.py for build_cjk_corpus.py.
+
+    Neither has import-time side effects -- compare.py's main() is
+    behind a __name__ guard, and build_cjk_corpus.py only defines
+    functions -- so importing them to read a constant or call one
+    function is safe.
+    """
+    spec = importlib.util.spec_from_file_location(
+        f"differential_{stem}", _TOOLS / f"{stem}.py")
+    assert spec is not None and spec.loader is not None, stem
+    module = importlib.util.module_from_spec(spec)
+    spec.loader.exec_module(module)
+    return module

tests/v2/test_differential.py

@@ -2,33 +2,21 @@
 
 `tools/` is outside `testpaths`, and adding it would run
 `--doctest-modules` over the corpus builders, so `compare.py` is
-imported by path here -- the same way `test_regex_sync.py` already
-imports `build_cjk_corpus`.
+imported by path -- through `_differential_fixtures.load_compare`,
+shared with `test_ledger_guards.py` so the loader exists once.
 
 Only pure logic is covered: nothing here spawns `uv` or the network.
 What is tested is what produces FALSE CONFIDENCE when it silently
 misbehaves -- which surfaces get compared, which ledger gets consulted,
 and above all whether a version tell is believed.
 """
-import importlib.util
 from pathlib import Path
-from types import ModuleType
 
 import pytest
 
-_TOOLS = Path(__file__).parents[2] / "tools" / "differential"
+from ._differential_fixtures import _TOOLS, load_tool
 
-
-def _load_compare() -> ModuleType:
-    spec = importlib.util.spec_from_file_location(
-        "differential_compare", _TOOLS / "compare.py")
-    assert spec is not None and spec.loader is not None
-    module = importlib.util.module_from_spec(spec)
-    spec.loader.exec_module(module)
-    return module
-
-
-compare = _load_compare()
+compare = load_tool("compare")
 
 
 def test_parse_version_pads_a_short_release_to_three_parts() -> None: