[MaxGhenis]

This makes the PolicyEngine→Axiom parity rule (CONTRIBUTING, "Mirror policy changes in Axiom") checked and assisted rather than manual, for every country model.

What's in it

1. A reusable PR check (.github/workflows/axiom-parity.yml, package axiom-parity/). Country repos call it on pull_request.

• It applies only when a PR changes <package>/parameters/, variables/ or reforms/. Markdown under those paths doesn't count.
• The check resolves each claim on the axiom: line against GitHub:
  • queued: the issue exists in a TheAxiomFoundation/rulespec-* repo, carries pe-parity, stands open (or closed as completed), and passes a dispatch-ready lint over its body and comments. The lint needs six things: module path, corpus citation (not a corpus release file), verbatim law, required outputs, a pasteable review_finding, and companion tests.
  • encoded: a rulespec PR, open or merged. It warns when the PR carries no encoder manifest, a JSON file under .axiom/encoding-manifests/.
  • encoded-correct: every module named exists on rulespec main, and so does any #rule cited. For each module, a cited companion case or file exists and outputs a rule of that module.
  • n/a: an allowed category (below), as the PR's only claim. For metadata and docs-tests, no changed policy file may change a parameter value or formula. The check reads every file, compares parameter YAML with descriptive keys removed, and compares Python by AST with docstrings and class-level labels removed.
  • needed: never passes.
• A PR can make several claims (;, |, several lines, or a bullet list after a colon), and every claim must pass.
• A line claims a status only in the canonical shape: right after the reference or citation it applies to, or opening its clause, with nothing before it that denies or conditions it. "rulespec-us#1416 is not [yet] queued" and "If rulespec-us#9 encoded, rulespec-us#1416 queued" claim nothing, whatever their punctuation.
• It verifies the claims a PR makes. It can't tell that every changed provision has a claim; the finder comment lists every referenced citation so a reviewer can compare.
• mode: warn reports in the job summary and annotations without failing. mode: enforce fails the job.

2. A parity finder (axiom-parity suggest). It runs in the same workflow and keeps one comment on the PR while the check fails.

• It reads the reference URLs on the changed parameters and variables and maps them to corpus citation paths. The mapping covers:
  • legislation.gov.uk and gov.uk;
  • the US Code via Cornell, uscode.house.gov and govinfo;
  • the CFR via eCFR, Cornell and govinfo;
  • CA, MD and OH codes and 14 more state code sites, IRS revenue procedures, plus a section-number match for about 50 other state hosts.
• It looks the citations up in a shallow, sparse clone of the rulespec repo and in the open pe-parity issues.
• It suggests encoded-correct candidates together with the companion cases that output each module.
• When a citation's source document already has an open issue, it suggests extending that issue. When nothing covers a document, it drafts one issue for it. A draft's TODOs fail the lint until a person fills them in.

3. A drain design (axiom-parity/docs/drain.md), with two tools:

• corpus-pin lists the citations the encoder could read in the corpus release a rulespec clone pins: rows with an operative body. It lists rows marked repealed apart, because the encoder refuses them. It reads axiom-corpus with git, using the encoder owner's procedure.
• drain-plan extracts each issue's target citations and review_finding (every finding block, body first, then comments), files each target under its source document, and prices a wave from measured run costs.
  • A batch is one document, so no run mixes jurisdictions, which the encoder refuses.
  • The plan treats a module a finding names as requested work unless the words around it mark an import, a deferral, a precedent or a description. When the issue gives no citation for requested work, the issue is held.
  • It applies the encoder owner's lists as data: blocked modules (declared, named in a finding, or owning the target citation on rulespec main), held citations and its dispatch-preparation verdicts, source patterns, and assembled pipelines, where the change an issue asks for is a hand edit that a signed run does not make.

CONTRIBUTING now lists the n/a categories and the issue headings, and names the check.

Allowed n/a categories (please check this list)

The rule names infra, data, ui, microsim-only and emulator-mapping. Three more cover policy-path PRs that change no provision of law:

• docs-tests: documentation and tests;
• metadata: labels, references and units;
• not-law: contributed reforms, proposals, and announcements not yet legislated.

The replay below shows these three accounted for most n/a PRs. The check rejects a reason saying "Axiom doesn't encode this yet" unless the line gives an explicit category, because queued covers that case. Three US PRs used that reason: policyengine-us#9667, #9656 and #9588.

Evidence

Replay over every PR merged since 2026-09-26 (scripts/replay_history.py, diffing each merge commit locally):

Repo	Merged	Change policy files	Would pass	Main failures
policyengine-uk	98	81	49	no line 13, line with no claim 13, needed 3, issue not dispatch-ready 2, n/a category 1
policyengine-us	203	173	81	no line 44, issue not dispatch-ready 20, line with no claim 17, n/a category 4

• I read each failure class and fixed every parser false positive the replay surfaced:
  • lines inside code fences;
  • a bullet list continuing a line;
  • dict(title=, href=) references;
  • 0000-01-01 parameter dates;
  • references moved under a dated value's metadata;
  • prose lines that start with "Axiom:".
• The grammar reads no claim in prose it doesn't recognise, so 12 UK and 7 US pull requests that passed under the looser parser now fail with "no claim found". The message tells the author to start the declaration with its reference or status, and shows the accepted form, axiom: rulespec-uk#370 queued (SI 2014/1230 reg 6A(4)).
• Rulespec lookups use main today, so a few old claims pass only because a module landed later.

Finder on the three open UK PRs the earlier replay session checked by hand (#2240–#2242). It reproduces those findings:

• Rulespec encodes FA 2004 s.190: uk:statutes/ukpga/2004/12/190, with cases earnings_above_basic_amount and earnings_below_basic_amount.
• ss.188, 189, 227, 227ZA, 228A and 233 have no module. The finder drafts one issue for FA 2004.
• FA 2003 Sch 4ZA has no module, so it drafts an SDLT issue.
• rulespec-uk#367 queues UC Act 2025 s.1 and passes the dispatch-ready lint.

The check passes on #2240's merge ref: it verifies s.190 against its module and both cases, and rulespec-uk#444 passes the dispatch-ready lint.

Backlog (axiom-parity backlog):

• rulespec-uk: 79 of 83 open pe-parity issues are dispatch-ready.
• rulespec-us: 84 of 175.
  • 85 US issues lack a pasteable review_finding; the 71 filed on 2026-10-09 include 60 of them.
  • 21 name no module.
• By the plan's own checks (corpus pin, the encoder owner's blocked modules and citations, manual paths paused), 16 source documents pass: 17 citations, 16 signed runs, completing 11 issues. Without the citation holds and the manual-path pause it is 28 documents and 31 citations, completing 17 issues.
• Passing the plan doesn't clear a document for dispatch. The encoder owner prepared all 16 documents for dispatch on 2026-10-10:
  • the owner dispatched 4 that day (Colorado 39-22-2003 and three New Mexico sections), and they wait on the signing approval;
  • 5 wait on an encoder fix (axiom-encode#1819);
  • 3 rely on a source that isn't in the pinned corpus;
  • the encoder can't run on the 4 UK documents: rulespec-uk has no working signed route. The signed workflow reads .axiom/workflow-toolchain.toml, which rulespec-uk lacks, and rulespec-uk's layout test allows no encoding manifest. The encoder owner has queued that route question for Max.
• drain-plan reads those verdicts back as holds. With them and the UK hold applied, it lists no document that is ready and not already dispatched.
• Some lint-ready issues are held for the work they ask for: 10 ask for a change to an assembled pipeline, and 13 ask for work on a module and give no citation for it (2 fall in both groups).
• Two earlier versions of this description overcounted ready documents: 25 (31 issues, 45 provisions), then 16 completing 15 issues. Review rounds 3 and 4 caught both.

Encoder costs (axiom-parity/docs/drain.md), from the 453 targeted-signed-reencode runs between 2026-09-01 and 10-10:

• The 121 runs since 2026-09-25 (GPT-6 Luna, escalating to Sol) cost $7.08 in model spend; 11 succeeded. Mean $0.058 a run, p90 $0.150, most $0.496.
• Four citations from the 453 runs have a manifest on rulespec-us main. GPT-6 Sol encoded one of them on 2026-10-01.

Invariants (property-tested with Hypothesis)

• The parser never raises, is deterministic, and finds no lines in text without an axiom: prefix.
• Rendered claims, legal ids and rulespec refs round-trip through the parser.
• Lint: for every subset of sections removed, the missing elements equal that subset; found and missing partition the six elements; a draft the finder writes is never dispatch-ready.
• Behaviour: descriptive edits never count as a change; value edits and changed formula locals always do; a file compared with itself never changes.
• Check: a policy PR passes if and only if it has at least one claim and every claim is good; adding a good claim never breaks a passing PR; the check is deterministic.
• Differential: warn and enforce block on the same findings, and warn shows no errors.
• Differential: anything the drain can paste as a review_finding, the lint counts.
• Differential: the URL mapper never contradicts 427 URL-to-citation pairs that an independent pass checked against the corpus.
• Drain: a batch is ready if and only if every issue is lint-ready and every target is in the pin.
• Drain: every batch holds one source document and one jurisdiction; each target sits in exactly one batch; a batch needs its citations over 16, rounded up, runs.
• Parser: an aside never cancels a negation ("is not (…) queued", "is not, …, queued" claim nothing).
• Lint: an input is never an expected outcome, however many the issue lists, and a named input is never a required output, whatever its type, entity or period.
• Parser: emphasis never turns a denial into a claim.
• Parser: a condition or denial before a reference, in a continuation lead or in a bullet never makes a claim, whatever the order of reference and status (tests/test_round7_claims.py).
• Drain: a module named in a root-first pairing, a brace group or a removal stays requested work in either rulespec layout, so the owner's holds reach it. The brace group can sit in the jurisdiction, and the removal can coordinate several objects or carry asides. An import or deferral clause after a removal keeps its own reading.
• Parser: a line that puts a legal citation ahead of its reference and status makes no claim, and its problem shows the accepted form (tests/test_round8_diagnostics.py).
• Lint: a name that its declaring sentence designates an input never becomes a required output, whatever whitespace or Markdown sits inside its name, type, entity and period (tests/test_round9_outputs.py).
• Drain: a removal governs every module after it until a new predicate starts a clause, however the finding joins the modules, and a module a removal takes something from counts as requested work whatever the removed object (tests/test_round9_removals.py).
• Drain: the bracket scan matches brackets in one pass, so 16,000 unmatched openers finish in under a second (tests/test_round9_scan.py).
• Check: only a JSON file under .axiom/encoding-manifests/ counts as an encoder manifest (tests/test_round9_manifests.py).
• Neighbourhood grids (tests/test_neighbourhoods.py): the tests try each free-text rule over every combination of the markers, wrappers and wordings near the ones it covers. Examples are 14 negations × 12 asides × 49 emphasis pairs for a denied status, and 8 list markers × 6 separators for a module-and-citation line.

.github/workflows/axiom-parity-tool.yml runs all 394 tests. actionlint is clean.

Review

Independent GPT-6.1 Sol reviews, each with reproductions:

• Round 1 (505822f) requested changes: 4 P1 and 9 P2 findings.
• Round 2 (b94b1bc) requested changes: 2 P1 and 10 P2 findings, plus 5 false positives in a 30-PR sample.
• Round 3 (9f04a68) requested changes: 6 P1 findings, 8 P2 and 1 P3. It reproduced the replay, lint, pin and cost figures, and found the five earlier false positives fixed.
• Round 4 (f858521) requested changes: 5 P1 findings and 6 P2. It reproduced every figure and found no misjudged PR in a fresh 20-PR sample.
• Round 5 (d787a40) requested changes: 8 P1 findings and 1 P2. It reproduced every figure and found no misjudged PR in a fresh 20-PR sample.
• Round 6 (6269fb0) requested changes: 8 P1 findings and 2 P2, most of them again in how the parser read status words.
• Round 7 (d09c652) requested changes: 5 P1 code findings, one stale figure in this description, 1 P2 and 1 P3. It re-ran all ten round 6 reproductions and found them fixed.
• The parser doesn't guess at negations. A status counts only in the canonical shape: right after its reference or citation (only separators, closing marks and a few words like "is" between), or opening its clause, with no denying or conditioning word before it. A property test checks that for any words between a reference and a status, the line makes a claim exactly when the grammar allows every one.
• 2aaad0d7f fixes every code finding from the seven rounds, each with a regression test.
• Round 8 (2aaad0d) requested changes: 3 P1 findings and 1 P2. It re-ran the round 7 reproductions and found them fixed, reproduced the replay, lint and drain figures, and judged the 19 newly failing declarations safe-side rejections whose message should show the accepted form.
• 38a4381 fixes all four, each with a regression test.
• Round 9 (38a4381) requested changes: 3 P1 findings, 1 P2 and 1 P3. It found the round 8 reproductions fixed, reproduced all 301 replay verdicts, and rebuilt the drain plan byte for byte.
• 686c90b fixes all five. Each class now has its invariant in the code and a Hypothesis property over the inputs around it, and each fault's property fails on 38a4381.
• Round 10 reviews 686c90b.

Rollout

1. Merge this.
2. Merge the caller PRs, which use mode: warn: Run the Axiom parity check on pull requests (warn-only) policyengine-uk#2249, Run the Axiom parity check on pull requests (warn-only) policyengine-us#10048, Run the Axiom parity check on pull requests (warn-only) policyengine-canada#558, Run the Axiom parity check on pull requests (warn-only) policyengine-il#12 and Run the Axiom parity check on pull requests (warn-only) policyengine-ng#28. They also update or add the PR templates' axiom line. The uk and us callers point at this branch, so their runs exercised the workflow end to end; both ran green.
3. After about two weeks of warn-only runs, switch them to mode: enforce.
4. Then make "Axiom parity" a required check in branch protection. That is a settings change for a repo admin.

The rule doesn't list policyengine-nz. Its hub can opt in with the same five-line caller.

axiom: n/a: infra: CI tooling for the parity rule; no country-model policy changes

🤖 Generated with Claude Code