feat(language): a quantity may state its value by region

[FBumann]

Prompt: "I want to update the repo to the latest changes in mathspec. Look at the 3 open draft PRs. Do PRs adopting the fixes and features. Stacked. Pin the hash in them."

and, on finding the first reading of #36 too narrow: "Look at #36, allowing the usage in constraints"

and: "if it helps also change our examples, use the new cases"

Note

The following content was generated by AI.

Adopts energy-models/math-spec#36 — expressions: gains a cases: key: one quantity whose value varies by region, with a foreach: naming the frame the regions partition. Pinned at 35d62ac.

No longer stacked: #1230 merged as 3e018da, so this now targets main directly and carries one commit. It earlier absorbed the pin waypoint that was #1231, whose content #36 already contains.

Still a commit pin, and still a draft: #36 is an open draft, and math-spec#33 (the partition check it stands on) is too. Only #31 has merged, released as v0.0.0-alpha.9 — which is what main pins today.

The corpus model that wanted it

pypsa_mixed_cycling exists because PyPSA's cyclic_state_of_charge is a column, so one network runs both storage regimes at once. Its page already said the two regimes are "one rule and a different predecessor" — and then wrote that rule three times under complementary masks, because there was no way to say it once. Now there is:

expressions:
  opening_level:
    foreach: [snapshot, storage]
    cases:
      wrapped: { when: "cyclic", expression: "shift(soc, over=snapshot, offset=1, edge='wrap')" }
      seeded:  { when: "not cyclic and position(snapshot) == 0", expression: soc_initial }
      carried: { when: "not cyclic and position(snapshot) > 0", expression: "shift(soc, over=snapshot, offset=1)" }

constraints:
  energy_balance:
    foreach: [snapshot, storage]
    expression: soc == opening_level + p_store - p_dispatch

Three constraints become one. The old shape also needed a fourth idea to work: the carried block's bare shift vacates each seeded unit's first snapshot, so that row was dropped and a third block wrote it back, with diagnostics().omissions reporting the gap. Under cases: there is no dropped row — the arm that vacates the first position is the arm that does not claim it. Objective still 4800.0 against PyPSA's own recording.

The generated page prints the definition once, and it composes with #44's upright/italic convention now on main — \mathit{soc} is the solver's, \mathrm{cyclic} is given, and opening_level is italic because its arms reach a variable (math-spec 68ed8a8):

$$\mathit{opening\_level}_{t,s} = \begin{cases} \mathit{soc}_{t \ominus 1,s} & \text{if } \mathrm{cyclic}_{s} \\ \mathrm{soc}^{\mathrm{initial}}_{s} & \text{if } \neg \mathrm{cyclic}_{s} \wedge \mathrm{pos}(t) = 0 \\ \mathit{soc}_{t - 1,s} & \text{if } \neg \mathrm{cyclic}_{s} \wedge \mathrm{pos}(t) > 0 \end{cases}$$

What converting the model found

An arm's absence was reaching outside its arm. Absence propagates into a comparison and deletes the row, and the engine collects that across every term of a row at once — so the carried arm's vacated first snapshot deleted the seeded row that defines that very coordinate. The recurrence went unanchored with nothing to say so, and the model solved to 3200.0 instead of 4800.0.

_arm_presence confines it: the admitted set is the value exists or this arm does not claim the coordinate. It is the one place a presence widens past the edge it names, and only for an arm that has one. Found by converting a real model — every unit test written before it used arms that vacate nothing.

The partition is the whole mechanism

The arms are proved disjoint and exhaustive at load, with no data bound. Both lanes lean on that, each its own way:

• Relational. A CompiledExpression is already a sum of fragments, so restricting each arm's fragments to the coordinates its when claims makes that sum a selection. The terminal sum(coeff) adds one contribution per coordinate because only one arm ever reaches it.
• Eager. Each arm is zeroed outside its own region and added, via .where(mask, 0) — linopy fills vars with -1 there, so the term is genuinely dropped rather than kept at coefficient zero, and the scalar reaches const only. Zero rather than nan: the arms that do not apply are not absent data, and a nan would spread through the addition and delete the coordinate.

A constant keeps a row outside its arm and a term does not. Constants are left-joined onto the rows and a missing one is a gap: the coverage check cannot tell an arm that does not apply from a parameter row nobody supplied, so the arm says so with an explicit zero. Terms are inner-joined and summed, where an absent row already contributes nothing — a zero there would be an explicit zero coefficient handed to the solver per non-applying arm.

Reading one back

A cased expression declares no body, so result.expression(name) and the eager expression() both read it through its own reference — body_of() hands back the name, expansion builds the arms. Neither lane assembles them itself.

Verified

Full suite on Python 3.12, HiGHS, on the merged main: 2978 passed, 323 skipped, 1 xfailed. ruff check, ruff format --check and pyrefly check over src/ clean.

Not checked: the gurobi and xpress sinks — neither wheel is installed here, so their 10 tests fail identically before and after and are the entire delta from a green run. The typst golden tests skip for want of the binary. Nothing was measured — the extra joins an arm costs are unquantified, _arm_presence materialises a coordinate product where the old presence named a one-column edge, and this PR makes no performance claim either way.

Mutation table

Five guards, each deleted or altered in turn against a committed tree, __pycache__ dropped either side, restored with git checkout --, tree asserted clean afterwards. Taken before the last upstream re-pin and the rebase onto merged main; the guards and their line numbers are untouched by both, but the table was not re-run.

tools.mutate's two, against the full suite:

mutation	result
the arm-widening loop	caught
the term/const split	caught

The other three are a changed expression rather than a deletion, which the tool does not express, so they were taken by hand with the same three precautions, against the relevant test files:

mutation	result
the arm mask cut — join(claimed, how='semi') dropped, so every arm applies everywhere	caught, 4 failed
the eager lane's .where(mask, 0) dropped, so arms are summed unmasked	caught, 4 failed
the arm-presence confinement — p.presences passed through unchanged	caught, 2 failed: the focused test and pypsa_mixed_cycling

Tree clean after the run.

Deliberately not done

• Only one corpus model was converted. pypsa_mixed_cycling is the one whose page already argued for the shape; the other ports are uniform and would gain nothing. Upstream's own examples/commitment.yaml was not ported in.
• No check_partition call from this package. Validation runs it upstream; a second caller here would be a second place the same claim is made.
• plan.Cases carries the declared foreach, not the union of the arms — an arm narrower than the frame broadcasts, which is what a parameter with fewer dims already does.
• The eager lane adds arms with + rather than linopy.merge. merge takes only linopy expressions, and an arm is routinely a plain DataArray — a constant case, or a whole constant case set — where it raises IndexError / AttributeError. Using it would mean a type-dispatch wrapper for what its own docs call "a bit faster than summing", in the lane that is the differential oracle. Not measured, so no speed claim; the lane's existing idiom is already reduce(operator.add, …).

Stays a draft until math-spec#36 lands in a release — the pin goes back to a tag then, as #1230's did.

[read-the-docs-community]

Documentation build overview

📚 lpspec | 🛠️ Build #34196985 | 📁 Comparing 0088fa2 against latest (e292dc8)

  🔍 Preview build  

49 files changed · ± 49 modified

± Modified

• index.html
• interactive/index.html
• about/changelog/index.html
• examples/dispatch/index.html
• examples/facility_location/index.html
• examples/genx_piecewise_fuel/index.html
• examples/monthly_budget/index.html
• examples/multi_period/index.html
• examples/osemosys_utopia/index.html
• examples/piecewise/index.html
• and 39 more...

[codspeed]

Merging this PR will not alter performance

✅ 16 untouched benchmarks
⏩ 42 skipped benchmarks¹

---

_{Comparing feat/expression-cases (0088fa2) with main (3e018da)²}

Footnotes

1. 42 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

2. No successful run was found on main (5379048) during the generation of this report, so 3e018da was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[FBumann]

Note

The following content was generated by AI.

Reviewed at high effort and fixed. Four defects, all of them lane divergences — a file one lane built and the other refused, or built differently — which is the class that makes the differential tests stop being an oracle. Each was reproduced on both lanes before and after.

#	what	before	after
1	cased expression as a divisor or under **	check accepted, eager solved (280.0), streaming died on a bare AssertionError: a divisor that adds is refused at load	LaneError naming the rewrite
2	cased expression under sum(over=)	LaneError vs eager 210.0	both 210.0
3	a parameter only one arm reads, supplied only there	streaming 130.0 vs eager DataError	both 130.0
4	read-back when every arm is narrower than foreach	came back keyed by generator alone, 2 rows	[snapshot, generator], 6 rows

2 and 4 were one root cause. _restricted widened an arm only as far as its own mask reached, on the reasoning that the remaining dims broadcast at the row join. True for a constraint row, and wrong everywhere the dim has to be there — sum(over=) finds no slots, and a read drops the axis. A cased expression's dims are its declared foreach (that is what dims_of answers upstream), so the fragments now carry that whatever any one arm names.

1 is now refused rather than crashed. Each arm is its own frame and a quotient inverts one, so folding the arms together would be a join per arm to rebuild what the partition already guarantees. It is a LaneError and not a LanguageError on purpose: the file is inside the language and the eager lane evaluates it, so this is this lane's shortfall — the same shape as the existing sum over a constant part (#1137). The check asks the node, not the fragment count, which is what keeps an operand that adds on its original plan-boundary assertion.

3 was the eager lane being stricter: check_constant_side_covers narrowed by the constraint's mask and knew nothing of arm masks, so a parameter one arm reads had to be dense over the whole frame. It now narrows per arm, and takes the union where two arms read the same parameter. A gap inside an arm is still refused on both lanes — test_a_gap_inside_an_arm_is_still_refused_on_both_lanes pins that, because narrowing a question must not stop it being asked.

Five regression tests added, four of them differential. Full suite 2983 passed, 323 skipped, 1 xfailed — the usual 10 gurobi/xpress-licence failures are the whole delta. ruff, ruff format, pyrefly clean.

Mutation table, re-taken

The guards changed, so this replaces the earlier table for the three that moved. Committed tree, __pycache__ dropped either side, restored with git checkout --, tree asserted clean after; run against test_expression_cases.py, test_ports.py and test_corpus_parity.py.

mutation	result
the declared-frame widening — back to widening only as far as the mask reaches	caught, 2 failed
the cased-divisor refusal — removed, so the bare assertion fires again	caught, 1 failed
the arm-presence confinement — p.presences passed through unchanged	caught, 3 failed

Tree clean after the run.

The fifth finding, not fixed

The review also flagged that .github/workflows/canary.yml runs the suite against math-spec main, which has neither cases: nor the partition check — so the nightly canary will be red on tests/test_expression_cases.py and the pypsa_mixed_cycling port for as long as this PR pins an unmerged branch. That is a true and expected consequence of the pin, not a defect in the code, and it clears itself when math-spec#36 merges. Recording it here rather than papering over it with a skip.

---

Generated by Claude Code

[FBumann]

Prompt: close everything we dont need anymore in general […] Close stale stuff.

Note

The following content was generated by AI.

Not rebasable. This branch edits src/lpspec/lowering.py and src/lpspec/relational/plan.py; both are gone from the tree — the lowering pass and Program are math-spec's, reached through to_program. The language half has to be written upstream and the lane half against a Program this repository no longer produces, so there is no diff here to carry forward.

The design is on #711, which stays open. feat/expression-cases is left in place at 0088fa2 as the reference implementation.

The branch is not deleted.