feat(language): a base and its patches compose into one model, so shared math travels as files rather than as a script

[FBumann]

Prompt: "Then let's discuss how we need to change merge to handle this as well, Or if we need another method / Or do we need flags inside the file really?" — and, after the discussion, "Yes, build it!"

Note

The following content was generated by AI.

What this changes

model = ms.override('base.yaml', {'pathway': 'pathway.yaml', 'project': 'project.yaml'})

A framework ships math; a project extends it. #12 is where this is argued, and #244 took the other side of it — this is the side it did not take.

A patch says only what it changes, because declarations are laid over field by field:

constraints:
  flow_out_max: { foreach: [node, tech, carrier, snapshot, investstep] }

A declaration set to null is removed:

constraints:
  no_longer_relevant: null
variables:
  cap: null # a decision becomes a number the next block declares
parameters:
  cap: { dims: [generator] }

examples/composed/operate.yaml is that patch in four lines, laid over the four-fragment library from #244: capacity stops being a decision and every constraint expression and the objective are identical either side of it. It is executed on the file.md page rather than described there.

The three decisions, and why

Two verbs, not a mode on merge. Composition and override obey opposite laws — a name two peers declare is a collision, a name a patch declares is the point — and erroring on a shared name cannot also mean winning with it. A mode= flag would make one call mean two things. The collision error is a real safety property for component libraries (two templates both declaring cost by accident), and a layered model must not have it.

	merge	override
Shape	N peers	base + patches
A shared name	an error	the mechanism
Order	irrelevant, associative	the instruction

A marker in the file is genuinely needed — but only for removal. This is the part I think is under-appreciated in #12: an ordered list of files gets you replacement but not deletion. A declaration a patch does not mention is left alone, so "absent" means unchanged and a deletion has no other spelling. That is the irreducible core of the issue, and @brynpickering is right that it has to travel with the file rather than sit in a script.

null rather than active:, and on @FBumann's own objection. His complaint is that twenty deactivated components make a file harder to read than not having them. null is one line carrying no lingering declaration to skip past, where {active: false} is one line and gives active a meaning in ordinary model files — where it buys nothing and costs exactly the readability he named. @brynpickering offered null himself in the thread; this is that suggestion, with the ambiguity he worried about resolved below.

The ambiguity null had, and how it is resolved

He noted that a parameter-to-variable swap gets murky:

parameters:
  becomes_a_var: null
variables:
  becomes_a_var: ...

The problem underneath is real and sharper than stated: where: null is already a legal value — a variable with no mask — so if null meant "remove" everywhere, one spelling would mean two things.

The marker is positional. constraints: {ramp: null} removes the constraint; variables: {p: {where: null}} sets that variable's mask to none. Removal is a declaration-level marker and reaches no deeper. That is rule 4's instinct — position decides which kinds of name are legal — applied one level out, and it makes the swap above unambiguous rather than merely conventional.

Why nothing in the schema changes

A patch is not a model, for the same reason #244 established a fragment is not: it is read before validation. So null never appears in a file anyone loads as a model, no block gains a key, and a validated Spec cannot contain one.

That is also the answer to #13, which was closed design: declined on "I'm not sure if we should widen the language this way". Its motivating case — capacity a variable in expansion, a parameter in dispatch — is now sayable at file level, and the language is not widened: variables: and parameters: stay separate blocks and the schema is untouched. Three routes now reach that outcome and none is redundant: #249's fix is programmatic and mid-workflow, a patch is file-level and shareable, and the declined component_type: was neither.

The risk, and the answer to it

override is dangerous in a way merge is not: it is designed to collide, so a patch that quietly replaces a constraint the base relied on yields a valid model meaning something else, with no error anywhere.

The mitigation needs nothing new. #244 proved a composed model round-trips through to_yaml, so the artifact to review is the output and a patch's real effect is a diff of base against result. That is asserted here too. I would document that as the workflow rather than build provenance reporting, until someone hits a case where the diff is not enough.

The one thing it does refuse: removing a declaration the base does not have, named with the near miss. A removal is a claim about what is there, so a stale one is a patch that no longer describes what it lands on — a base that moved on, or a section confused for another.

What lpspec removes or changes

Nothing. override returns a mapping and every verb already takes one, so lps.solve(ms.override(base, patches), sources) works against an unchanged engine — same as merge.

Verified

Toolchain at the versions pixi.toml pins: ruff format --check / ruff check (0.16.1) clean, pyrefly (1.2.0) 0 errors, reuse compliant, typos clean, prettier --check over **/*.{md,yml,yaml} clean, pytest -q 926 passed, 0 skipped (915 on the base — 11 new), mkdocs build --strict builds, python -m tools.schema and python -m tests.typesetting.golden regenerate to no diff.

Not run: zizmor, taplo, and the tectonic half of compile-tex. mkdocs build --strict needed docs.python.org/objects.inv dropped from mkdocs.yml, which this environment's proxy refuses; that affects cross-links into CPython's docs and nothing in this diff.

Mutation table

Mutation	Result
null removes at any depth, not just a declaration	1 failed
a declaration replaces wholesale instead of field by field	2 failed
a stale removal is ignored	1 failed
the base is mutated in place	4 failed
patches applied in reverse, so the first wins	1 failed
restored	all green

The first needed a second attempt: over is not None inside a isinstance(over, dict) branch is always true, so my first mutation changed nothing and passed the whole suite — a catch that did not happen. Re-expressed as popping the key on a nested None.

Deliberately not done

No active:. #12 stays open; this implements the removal it needs by the spelling that costs the model schema nothing. If deactivate-in-place is wanted for its own sake — keeping a switched-off declaration so it can be switched back — that is a separate argument this does not make.

No provenance for which patch won which key. The composed model is to_yaml-able and the diff is the answer; a report would be a second way to learn the same thing.

Order is not made explicit in the files. A patch carries its removals, but which file is the base and what order the rest apply in is still the caller's — and I think rightly, since a patch that named its own position could not be reused against a different base.

Changed in review. _lay_over returned None and wrote into its argument, which made override's "the base is never mutated" guarantee a property of the caller rather than of the function; it now takes a base and returns the laid result. And the stale-removal message called a sos collision a so and a piecewise one a piecewis, which #244 now fixes for both verbs.

Stack: ninth, on #249 → #248 → #247 → #246 → #245 → #244 → #243 → #242 → #168.

---

Generated by Claude Code

[read-the-docs-community]

Documentation build overview

📚 math-spec | 🛠️ Build #34317134 | 📁 Comparing ed85bb7 against latest (a9237ec)

  🔍 Preview build  

25 files changed · + 6 added · ± 19 modified

+ Added

• about/what-counts-as-a-verb/index.html
• examples/commitment/index.html
• examples/composed/index.html
• reference/math_spec/exclusivity/index.html
• reference/math_spec/merge/index.html
• reference/math_spec/transforms/index.html

± Modified

• examples/index.html
• reference/notation/index.html
• reference/language/declarations/index.html
• reference/language/dimensions/index.html
• reference/language/expressions/index.html
• reference/language/file/index.html
• reference/language/reading/index.html
• reference/math_spec/dimensions/index.html
• reference/math_spec/expression_parser/index.html
• reference/math_spec/lowering/index.html
• and 9 more...

[FBumann]

Note

The following content was generated by AI.

Closed at the author's request, in a sweep of the open PRs. Closed rather than retargeted, because the whole stack under it is closed.

• The base is claude/fix-a-decision, which is feat(language): a decided variable becomes a supplied number, so a subproblem is a call rather than a second file #249, closed alongside this. Under that is feat(language): several files compose into one model, so a component library is templates rather than generated YAML #244, and under that claude/parameter-coverage, which is feat(language): a parameter says whether its table must cover every coordinate, so a row lost in preparation is not read as a mask #243 — closed unmerged on 2 September.
• Draft since 30 August. 61 files, 4663 additions, conflicted against main, and most of that diff is examples and documentation pages that feat(language): a relation between dimensions, walked in the direction each call names #437 and docs: the README shows the math a model prints, in all three formats #449 have both moved since.

Bringing this up to date means resolving three dead stack levels and then a rename across every example it adds. Rebuilding override against current main is less work and starts from the language as it now is.

#12 stays open, and the three decisions in this description are the reason to keep it open: two verbs rather than a mode on merge, a marker in the file needed only for removal, and null as that marker with the ambiguity resolved positionally. Those survive this PR and should be re-argued there, not lost with the branch.

---

Generated by Claude Code