examples/pypsa.yaml splits into topic files that merge back to the same model

[FBumann]

Prompt: "please explore how we can decompose the big pypsa yaml file into many small, topic related yaml files. And how we can test if everything works. I imagine that the merging of those yaml files must ultimatively result in the same spec. Start from #620. We should probably merge main into it. Then use the feature from 690. For testing, we might need 718"

Prompt: "Open an issue with your proposal and your findings about it."

Prompt: "I think we can be sure that we need to add parameters and expressions to given... Lets just do it as a stacked PR onto 690. Then retry. Iterate until the split is clean, reasonable and its easy to add new features via a fragment."

Note

The following content was generated by AI.

Two language changes make the split work: given: reads parameters and expressions (#728), and fragments add terms to one additive expression (#729). With both, #723 cuts pypsa.yaml into 24 fragments that merge back to its rows.

Plan

1. Land feat(language): a file says what it reads, and fragments and patches compose into one model #571/feat(language): merge and override load every model they compose and return one #690, then feat(language): a fragment reads a parameter or a named expression another file declares, under given: #728 and feat(language): fragments add terms to a named expression that its reader marks as a sum, and merge sums them #729 on top, and feat(language): two files that mean the same model write one text #530/feat(cli): canonical --check fails a model file that is not in the canonical form, and --write rewrites it #718.
2. Land docs(pypsa): a single spec covers every pypsa model class and component #620.
3. The migration PR from main: the fragments of chore: examples/pypsa/ cuts pypsa.yaml into 24 topic fragments that merge back to its rows #723 become the source under examples/pypsa/, examples/pypsa.yaml goes, and the gallery page, examples/symbols/pypsa.yaml, tests/test_pypsa_collapses.py and the reference coverage check read merge(examples/pypsa/*.yaml). tests/test_pypsa_split.py of chore: examples/pypsa/ cuts pypsa.yaml into 24 topic fragments that merge back to its rows #723 moves with it; its row check compares against the one file once, in the migration PR, and is dropped after.

Layout

A component is a family of files, and every system topic is one file:

• Components: generator, link, process, each with _commitment, _ramping and _maintenance; storage_unit, store, line, transformer, load.
• Hubs, each one row over an additive sum: network (Bus_injection == 0) and power_flow (Cycle_angle_sum == 0).
• System topics: cost (operating cost and CVaR), carrier, global_constraints, security.
• core: the shared weights.

A new component is a new family. It adds its shares to the hubs, and no other file changes.

Findings, and what was tried first

• With given: as in feat(language): merge and override load every model they compose and return one #690 (variables and constraints only), 491 of 652 declarations must share one file.
• With parameters under given:, the largest forced group is 18; with expressions too, 1.
• With given: alone, the 14-file split merges back exactly. But each hub file lists every component, so a new component edits four other files.
• With additive shares, every fragment except core can be left out, and the rest merges into a whole model (checked in tests/test_pypsa_split.py).
• Copying declarations between fragments instead of reading them under given: also merges back, at 1100 copied lines. Rejected.
• The canonical form of feat(language): two files that mean the same model write one text #530 is what makes the check exact: merge joins objectives and shares in fragment-name order, with brackets.
• canonical_dict did not sort names under given: (feat(language): two files that mean the same model write one text #530 × feat(language): a file says what it reads, and fragments and patches compose into one model #571). Fixed on chore: examples/pypsa/ cuts pypsa.yaml into 24 topic fragments that merge back to its rows #723 with a regression test.