feat(language): several files compose into one model, so a component library is templates rather than generated YAML

[FBumann]

Prompt: "Please do a few stacked PRs implementing this / All stacked onto the cases pr / And to the description, add what lpspec removes or changes" — and later, "fold it into examples/ as a gallery page"

Note

The following content was generated by AI.

What this changes

ms.merge({...}) takes fragments and hands back one mapping to validate, resolve and lower exactly once.

model = ms.merge({'generator': 'generator.yaml', 'demand': 'demand.yaml'}, description='a fleet')
spec = ms.to_spec(model)  # one flat namespace, checked here and nowhere else

One decision makes it cheap: a fragment is not a Spec. It is merged before it is validated, so a template may name what a sibling declares — a shared bus, the flow every component writes into — without being loadable alone. No free-name concept, no import list, no new resolution rule: validation still runs once, over one flat namespace, exactly as for a file somebody typed.

The split that gives merging its meaning:

• dimensions and lookups are the coordinate space, shared on purpose. Declared twice and agreeing, one declaration; disagreeing, the disagreement is the error.
• Everything else is the math, which a template owns, so a name two fragments both declare is a collision naming both.
• Objectives are summed, each term parenthesised. A fragment running the other way is refused rather than negated: a composed model has one sense and nothing in the files says which.

⚠ This takes a side in an open design question

Read this before the rest. The third bullet above is not settled, and an earlier version of this description presented it as though it were.

#12 is open and contested, and #30 is named in it. @brynpickering there:

you can't delete YAML entries when you have multiple files … as I mention in #30, I want the YAMLs to hold the information about how to handle possible clashes when combined with other YAMLs in the same project

That is a different composition model from this one, and the two are not compatible:

	this PR	the model #12/#30 describes
Fragments are	disjoint peers	a base, plus overrides
A name in two fragments	is an error	is the mechanism
Order	irrelevant — merging is associative	significant, later wins
A framework's base math, extended by a project	cannot be expressed	is the motivating case

Associativity is a property of the model chosen here, not a neutral win. A layered-override merge needs order to matter, so the test pinning associativity would be wrong under that design rather than merely unnecessary.

So: this is a complete implementation of disjoint composition, which is what ceiling.md's component-library section describes (topology is data, templates bounded by type). #250 later builds the other half as a second verb, override, on the argument that the two obey opposite laws and neither is a mode of the other.

The collision error names the rewrite it usually is

A collision refuses, and the question is what the author should do about it. Two of the same kind of thing — a battery and a pumped hydro — are the same math with different numbers, so they are two rows of a dimension, not two fragments, and the message says that first:

fragments 'battery' and 'hydro' both declare the variable 'soc'. Two of the same kind of
thing are two rows of a dimension rather than two fragments: merge the template once, and
let the data carry both. Different math under one spelling is a rename — call one of them
something else.

The earlier message only offered the rename, which is the right advice for genuinely different math and the wrong advice for the case a component library hits most. The #29 promise moved out of the message and into the module docstring, where a maintainer reads it and a file's author does not need it.

The same pass fixed a message defect in both merge and override: the singular noun was section[:-1], so a sos collision was reported as a so and a piecewise one as a piecewis. There is a parametrized case asserting the English now.

The worked example

examples/composed/ is four files and a gallery page. base.yaml declares the coupling surface — one flow per port, one balance per bus — and the other three name flow without declaring it, so each is a load error alone.

The balance does not grow when a component type is added. Adding storage.yaml as a fourth fragment:

three components -> balance: sum(flow, by=port_bus) == 0
four  components -> balance: sum(flow, by=port_bus) == 0
balance identical            : True
objective, four  : (sum(gen_p * gen_cost)) + (sum(st_soc) * st_holding)

That is what the port convention buys, and it needs no language this repository does not already have — each component pins its own port's flow with at(flow, by=…) rather than owning a flow variable the composer must then sum by hand.

Why

#30, under the reading above. ceiling.md already argues the shape — topology is data, compose-then-build producing one Spec before a single lower pass, structure bounded by component types while cardinality lives in data. What was missing was the function.

Names are not rewritten, the deliberate boundary. Qualified names are #29 and need an unparser this repo does not have. Until they land, a library keeps templates apart by naming them apart — which is what examples/pypsa.yaml already does with Generator_p_nom and Link_ramp_limit_up.

What lpspec removes or changes

Nothing. Not one line. Every verb already takes str | Path | dict | Spec and merge returns a mapping, so this works today against an unchanged engine:

lps.solve(ms.merge({'generator': gen, 'demand': demand}), sources)

Because names are not rewritten the composed declarations keep the fragments' spellings, so the sources mapping is just the union and no manifest, binder or diagnostic moves. That is the layering's strongest evidence: the piece enabling component libraries costs the engine nothing, being a Spec → Spec function on the far side of the waist.

Verified

Toolchain reconstructed at the versions pixi.toml pins: ruff format --check / ruff check (0.16.1, pinned) clean; pyrefly (1.2.0, pinned) 0 errors; pytest -q 854 passed, 0 skipped. reuse, typos, prettier --check over **/*.{md,yml,yaml}, render_tex (27 models) and mkdocs build --strict were run on the top of the stack (#250), which contains these commits.

Not run: zizmor, taplo (neither's files changed), and the tectonic half of compile-tex.

Two CI failures this PR earned, and what they say

Both were in gates I had claimed I could not run. Recording them because the claim was the problem, not the code.

Prettier reformatted the new YAML. {dtype: int} → { dtype: int }. My local check globbed **/*.md; .lefthook.yaml globs *.{md,yml,yaml}, and these were the first YAML files the stack added.

render_tex typesets every file under examples/ — and a composition fragment is a load error alone, which is the entire point of the directory. tools/render_tex.py already had NOT_MODELS for exactly this category (the symbol tables), so the fragments join it. It still renders 27 models, and the composition itself is covered twice over: the gallery page renders it, and test_a_composed_model_prints_as_math typesets it in all three formats.

That second one is the useful lesson. compile-tex was on my "not run" list, and it caught a real defect the other seven gates could not.

Mutation table

Every guard deleted in turn on a clean tree, restored with git checkout --:

Mutation	Result
an owned name claimed twice not refused (last wins)	1 failed
a shared declaration's disagreement not compared	1 failed
objective senses not compared	1 failed
language versions not compared	1 failed
objective terms joined unparenthesised	1 failed
restored	all green

The last is the quiet one: a + b where b is sum(x) * k reassociates, so an unparenthesised join composes a different objective and nothing else in the suite would notice.

A composed model is still one a reviewer can open

Hard rule 5 wants the model to be the file you review and diff, and "a construct that cannot be printed is not in the language." Both are asserted: to_yaml on a merged model writes a file that loads back to the same Spec, and all three formats render it.

Writing that round-trip test found the bug in the third commit: merge read a str fragment as YAML text where to_spec reads one as a path. merge now takes exactly what every other verb takes.

Deliberately not done — and one correction

No qualified names (#29); needs an unparser, and merging is useful without it.

No provenance returned. With no renaming every declaration keeps its fragment's name, so the origin is recoverable by looking, and collisions name both fragments when refused. It becomes essential the day #29 renames things.

No active:. #12 is open; nothing here implements or precludes it, though the collision rule above is the part that would have to give — and #250 is the verb that gives it, without touching this one.

Correction. An earlier version of this description claimed exclusivity.overlapping() from #168, pointed at where masks on a shared variable, would catch "two components claiming the same port". Wrong, and I withdraw it: two constraints bearing on one variable is ordinary linear programming, and because each declaration is owned by exactly one fragment a cross-fragment mask overlap is not expressible.

The residual gap is real, and building the example above gave it a name: port_bus is a lookup, a partial lookup is legal, and sum(by=) places an unmapped port's terms nowhere — so a mis-wired system solves and is quietly wrong. That is #245, stacked on this.

Stack: third of nine, on #243 → #242 → #168.

[read-the-docs-community]

Documentation build overview

📚 math-spec | 🛠️ Build #34319929 | 📁 Comparing 86ea399 against latest (a9237ec)

  🔍 Preview build  

22 files changed · + 4 added · ± 18 modified

+ Added

• examples/commitment/index.html
• examples/composed/index.html
• reference/math_spec/exclusivity/index.html
• reference/math_spec/merge/index.html

± Modified

• examples/index.html
• reference/notation/index.html
• reference/language/declarations/index.html
• reference/language/dimensions/index.html
• reference/language/expressions/index.html
• reference/language/file/index.html
• reference/math_spec/dimensions/index.html
• reference/math_spec/expression_parser/index.html
• reference/math_spec/lowering/index.html
• reference/math_spec/model/index.html
• and 8 more...

[FBumann]

Note

The following content was generated by AI.

Closed at the author's request, in a sweep of the open PRs. Closed rather than retargeted, because the base branch never landed.

The facts behind that:

• The base is claude/parameter-coverage, which is feat(language): a parameter says whether its table must cover every coordinate, so a row lost in preparation is not read as a mask #243. feat(language): a parameter says whether its table must cover every coordinate, so a row lost in preparation is not read as a mask #243 was closed unmerged on 2 September. So this branch is stacked on work that is not in main and is not going to be.
• The surviving half of feat(language): a parameter says whether its table must cover every coordinate, so a row lost in preparation is not read as a mask #243 is feat(language): a parameter and a lookup each say whether their data must be complete, so a row lost in preparation is not read as a mask #245, which combines both halves and targets main directly.
• Draft since 29 August, 53 files, conflicted against main. feat(language): a relation between dimensions, walked in the direction each call names #437 replaced lookups: with relations: and docs: the README shows the math a model prints, in all three formats #449 renamed the gallery variables, which this branch's examples and docs pages carry throughout.
• The description says this PR takes one contested side of [DISCUSS] Allow (de-)activation of any component #12, and that feat(language): a base and its patches compose into one model, so shared math travels as files rather than as a script #250 builds the other side as a second verb.

The design question is the valuable part here, and it lives on #12 and #30, where it is still open. The implementation is a fragment merger written against a language that has since renamed the block it reads.

Not lost: the collision-error wording, the port-convention example, and the argument that a fragment is not a Spec are all in this thread. A fresh merge against current main can start from them.

---

Generated by Claude Code