feat(language): two files that mean the same model write one text

[FBumann]

Prompt: "Provide a spec dump that removes ambiguity from ordering and other things that let the file differ but mean the same" (#487) — then, setting the scope: "Lets build both rungs as stacked PRs" and "Test the dump thoroughly" … "Rebase 530 onto main." … "This conflicts" … "And do we show how to use this in a how-to? Diff changes etc?" … "add tests tracking the sorting behaviour!"

Note

The following content was generated by AI.

spec.to_yaml(canonical=True), and python -m mathspec canonical model.yaml from a shell. Two files that mean the same model write one text, so a diff shows what they mean differently rather than how they were spelled.

constraints:
  k:
    dims: [h, u]
    expression: |-
      -cap
      + cost * p
      + p
      <= 10

A new how-to page, Compare two models, shows how to diff two files in this form: once, through git diff with a textconv driver, and from Python. #718 is stacked on this PR and adds --check and --write for CI.

The rules, whole

What is sorted

• The sections come in the order of the fields on Spec (version, description, dimensions, relations, parameters, variables, constraints, objective, expressions, macros, piecewise, sos, assumptions), whatever order the file wrote them in. The keys of a declaration also come in one order.
• Declarations sort by name, within every section.
• The terms of a sum sort; each keeps the sign it was written under.
• The factors of a product sort.
• A call's keyword arguments sort by keyword.
• Sorting is by the text an operand prints as, which is a total order over trees: a printed tree parses back to the tree it came from (chore(parser): every expression node prints itself as the file writes it #529's fence), so two operands that print alike are one tree.
• A sign is the tie-breaker, never part of the key, so a - b and b - a order their terms alike and differ only in which is negative.

What is not sorted, because moving it changes the model

• Subtraction, division and exponentiation keep their operands where they are.
• A call's positional arguments keep their places.
• The regions of a cases: block keep the file's order.
• A piecewise block's links keep the file's order.

What is folded

• A sign written twice cancels: - -x is x, and + -x is - x, wherever it stands, including at the top of a tree.
• A sum under a plus is spliced into the sum around it, on either side: a + (b + c) is a + b + c.
• A product is spliced on either side: (a * b) * c and a * (b * c) are one product.
• A sum under a minus is not spliced. a - (b - c) keeps its group, because splicing would flip every sign inside — a rewrite of what the file wrote rather than an ordering of it.
• Each term and factor is flattened after it is normalised as well as before, because normalising reveals what the written tree hid: a unary plus over a product is one factor until the plus is folded away.

What is never done

• No constant is folded into another. 2 * 3 stays 2 * 3: a coefficient that changed is what a reviewer is looking for.
• No macro is expanded. A file that calls one and a file that writes it out still differ, which the thread already called a fair difference.
• No predicate in the where grammar is touched — a where: string, and an assumption's holds:. It is a second grammar, and printing it is in neither PR of this stack.
• No dims list is reordered.

How it is laid out

• An expression of one term is one line, operator and all.
• A sum of two or more terms breaks one term to a line, each line under its own sign, and a comparison's operator opens the last line. A term that changes is then one line of a diff.
• Every operand that is itself an operator is bracketed, so the text never relies on the reader knowing what binds tighter.
• A multi-line expression is written as a YAML block scalar, so the newlines are real rather than escaped.

What the form guarantees, and what it does not

• Dumping the form again gives the same text. It is a fixed point, not a rewriting.
• The form loads to the same model: every declaration is there, under its own name, on its own frame, with its sense and its domain.
• It does not load to an equal Spec. A reprinted expression is a different string, so the default to_yaml() — the round trip reading.md documents — is untouched and the form is an argument away.
• The form holds no YAML comments.
• Sorting variables: changes the order a piecewise: expansion meets them in, so a constraint the expansion emits can carry its dims in another order. The frame is the same set; a consumer that lays arrays out in dims order lays them out differently.

The merge onto 0.1.0, the sorting tests and the how-to page

Merge. main renamed the package from math_spec to mathspec (#702), so canonical.py conflicted with the moved package. origin/main was merged in, not rebased, and there was no force-push. The module, its tests and the CLI verb now use the new name. The PR adds its line under ## Upcoming version, which the changelog check on main now requires.

Sorting tests in tests/test_canonical.py:

• test_a_file_written_backwards_writes_the_same_text: every model in the tree, with its sections, the declarations in each section and the keys in each declaration all written in reverse, writes the same text.
• test_the_sections_come_in_one_order: every model's sections follow a pinned SECTIONS list, so a reordered field on Spec fails a test instead of changing every diff without notice.
• test_every_section_is_sorted_by_name: every section of every model.
• test_an_order_the_form_keeps_is_a_difference_in_the_text: dims, the regions of a cases: block, the links of a piecewise block and the predicates of a where are each reversed alone, and each changes the text.

mutation	result
declarations keep the file's order	53 tests fail, among them 26 written_backwards and 25 sorted_by_name
every dims list is sorted	a-declarations-dims fails

Docs. New docs/howto/compare.md, in the nav under How-to guides. reading.md names the section order and links the how-to page. The textconv recipe was run in a scratch repository: swapping two factors shows no diff, and changing a coefficient shows one line.

The rebase onto 0.0.0-alpha.120

The branch was rebased onto main as one commit carrying the same change: its ten commits held two merges of main and would not replay one by one. The dump itself is unchanged; two things around it are.

what moved on main	what this branch does now
an expansion no longer derives parameters from a curve, so to_yaml() no longer refuses one (#638)	the refusal this branch read before canonical, and _derived_parameters, are gone with it; main's test that an expansion declares exactly the file's parameters replaces the two here that tested the refusal
to_program refuses a piecewise: block left as written (#626)	the sweep that loads every model twice expands the curves first and keeps the sets, so all nine sections are still compared

The earlier merge onto alpha.111 had already taken main's node printers, the explicit-any rule, the expressions rename and the move of the page to docs/reference/reading.md; that is all in the one commit now.

Testing it thoroughly, and the three bugs that found

The models in the tree carry a few hundred expression shapes between them, and all of them were already idempotent. Generated trees were not:

bug	what it printed	why it is wrong
a sum under a plus left whole	3 - (-(2 + d)) → 2 / + d / + 3	the layout flattens what the form kept, so the text reads back as three terms sorted differently
a product group left whole	(-(5 / 5)) * ((d - 2) * c) → (((-2) + d) * c) * (-(5 / 5))	the same, one operator over
a sign at the top of a tree	- -(b + d) → b + d	the tree still held both signs, so the text and the tree disagreed

None changes a value — every one of them is text that reads back as a different tree, which is what makes a normal form not normal. Two earlier bugs of the same kind (a leading negative term, a dropped bracket) were found the same way.

Four properties now hold, as tests:

• A fixed point on any tree at all — 2000 generated trees, every node kind the grammar has, comparisons and calls included: print, re-parse, normalise, compare.
• The value is unchanged — 500 trees evaluated before and after the rewrite. Sorting, folding and splicing are rewrites, and a rewrite that changed a value would be a wrong answer rather than an untidy file.
• Every spelling of one sum writes one text — 150 sums, every permutation of their terms with signs attached, rather than the one permutation a test author would have picked.
• The form declares the same model — every model in the repository, loaded twice: the same names in all nine sections, the same frame, sense and domain on each.

Plus the unit cases for each rule above, the idempotence sweep over all 30 models, and two CLI tests.

Verification

pixi cannot be installed in this container (the proxy refuses pixi.sh), so the gates ran in a Python 3.12 virtualenv with the versions pixi.toml pins. On head 81993c9:

pytest tests -n auto                    1846 passed, 21 skipped
ruff check . / ruff format --check .    clean
pyrefly check (1.2.0)                   0 errors (9 suppressed)
reuse lint                              compliant
typos                                   clean (the changed files)
prettier --check (compare.md, reading.md)  clean
python -m zensical build --strict       No issues found

Not run: compile-tex, zizmor, taplo and the rest of pixi run lint's hooks.

Mutation table, taken before the first merge of main. canonical.py carries no change since but its annotations and the package rename, so each guard is the same line:

guard deleted	result
a sum under a plus is not spliced	caught — 2 cases of test_a_sum_flattens_into_the_terms_its_text_shows
a product on the right is not spliced	caught — a-product-on-the-right-is-spliced
a group under a minus is spliced too	caught — 3 tests, the fixed point among them
nothing is re-spliced after it is normalised	caught — test_the_form_is_a_fixed_point_on_any_tree_at_all
a sign at the top of a tree is not folded	caught — the same
_signed folds no sign	caught — [pypsa_linearized_uc] and and-a-sign-written-twice-is-folded-once
a term's line is not bracketed	caught — a-bracketed-group-stays-one-term
the first line is signed by hand instead of through _sum	caught — three pypsa models
keyword arguments keep the file's order	caught — a-calls-kwargs-are-sorted
declarations keep the file's order	caught — the headline test and the sort test

The first two survived until a test was written for them: the re-splice covers the same shapes, so the suite stayed green without them. They are guards a purpose-built test now reaches, rather than guards nothing needs.

Prose measured: docs/reference/reading.md n 91, median 13, 10 over 25 (main: n 71, median 13, 8 over 25). docs/howto/compare.md n 25, median 10, 1 over 25.

Coverage moved: test_the_verbs_are_check_and_the_formats_and_nothing_else is now ..._the_two_readers_and_the_formats_... and pins {check, canonical} | FORMATS. The canonical verb's own CLI tests sit in tests/test_canonical.py, with the form rather than with the renderer.

Not done: the canonical verb takes no --expand, which main added to the typeset verbs while this branch was open.

The dump was written in session 01WZLCgQU3TMWMHQU9HkQrum; the merge onto alpha.111 in 01LCUHhoVyReQaBh2pj8CuGd; the rebase in 015h57WkBDnpxrknuJ5zZy9F; the merge onto 0.1.0, the sorting tests and the how-to page in 0159UqsefN2Xzi5XJ4ppoU4y.

🤖 Generated with Claude Code

https://claude.ai/code/session_0159UqsefN2Xzi5XJ4ppoU4y

[read-the-docs-community]

Documentation build overview

📚 math-spec | 🛠️ Build #34754220 | 📁 Comparing 6b715a4 against latest (5311baf)

  🔍 Preview build  

66 files changed · + 12 added · ± 31 modified · - 23 deleted

+ Added

• first-model/index.html
• howto/curve-by-hand/index.html
• howto/declare-a-column/index.html
• howto/pin-a-variable/index.html
• howto/see-an-expansion/index.html
• reference/api/index.html
• reference/glossary/index.html
• reference/program/index.html
• reference/reading/index.html
• reference/language/assumptions/index.html
• and 2 more...

± Modified

• index.html
• CHANGELOG/index.html
• contributing/index.html
• examples/index.html
• about/limits/index.html
• about/what-counts-as-language/index.html
• about/what-counts-as-public-api/index.html
• examples/commitment/index.html
• examples/dispatch/index.html
• examples/operators/index.html
• and 21 more...

- Deleted

• reference/language/reading/index.html
• reference/language/reported/index.html
• reference/math_spec/advice/index.html
• reference/math_spec/boundedness/index.html
• reference/math_spec/degree/index.html
• reference/math_spec/dimensions/index.html
• reference/math_spec/errors/index.html
• reference/math_spec/exclusivity/index.html
• reference/math_spec/expansion/index.html
• reference/math_spec/lowering/index.html
• and 13 more...

[read-the-docs-community]

Documentation build overview

📚 mathspec | 🛠️ Build #34757737 | 📁 Comparing 81993c9 against latest (ca2e04b)

  🔍 Preview build  

4 files changed

+ howto/compare/index.html
± changelog/index.html
± reference/api/index.html
± reference/reading/index.html

[FBumann]

Prompt: "I think we should close all superseeded ones that are from me (FBumann), and name what superseeded them "Superseeded by #xyz""

Note

The following content was generated by AI.

Superseded by #731.

---

Generated by Claude Code