Skip to content

docs(pypsa): component data spans a scenario wherever pypsa reads it per scenario - #688

Merged
FabianHofmann merged 2 commits into
pypsa-completefrom
docs/pypsa-scenario-data
Sep 24, 2026
Merged

docs(pypsa): component data spans a scenario wherever pypsa reads it per scenario#688
FabianHofmann merged 2 commits into
pypsa-completefrom
docs/pypsa-scenario-data

Conversation

@FabianHofmann

@FabianHofmann FabianHofmann commented Sep 24, 2026

Copy link
Copy Markdown
Contributor

Prompt: "1", choosing the session's option "Do both steps of the recommendation": correct the rung 14 claim on #620, then do the wide edit as its own PR stacked on pypsa-complete.

Note

The following content was generated by AI.

In examples/pypsa.yaml, a parameter now spans scenario exactly when PyPSA 1.3.0 reads it per scenario. That covers 160 of 237 parameters, and the capital cost is weighted by scenario, as in PyPSA. Rungs 41 and 42 bind on that data. Stacked on #620.

Method, gate output, alternatives

PyPSA behaviour at the v1.3.0 tag.

  • PyPSA reads every component attribute through c.da, which unstacks the scenario level (components/array.py:332-395).
  • The bounds on built capacity become one row per scenario (constraints.py:885-895).
  • The capital cost is weighted by the scenario weights (optimize.py:405-412, :448-454).
  • PyPSA refuses a difference across scenarios in the shape attributes (consistency.py:1174-1195).

The change.

  • Parameters: 160 of 237 gain a leading scenario.
  • Relations: the three *_maintenance_cover relations gain a scenario key.
  • Expressions: 22 of 49 gain the axis.
  • Constraints: 21 of 226 gain it, which are the *_ext_*_nom_lower/upper and *_nom_set rows.
  • Objective: each capital-cost term is multiplied by scenario_weight.
  • Collapse test: SCENARIO_PARAMETERS in tests/test_pypsa_collapses.py is replaced by two checks that follow the rule.
    • A standard parameter may change only by a leading scenario.
    • A parameter PyPSA holds equal across scenarios carries none.
    • Adding scenario to Generator_capital_weight makes the second check fail.

Deliberately scenario-free.

  • Refused by PyPSA if they differ: *_extendable, *_committable, *_nom_mod, GlobalConstraint_type, and the values derived from build year, lifetime and active.
  • Reduced over the scenarios by PyPSA: *_maintainable, *_p_min_pu_nonneg, Carrier_max_growth and Carrier_max_relative_growth (components.py:1016-1019, constraints.py:397-401, global_constraints.py:226-230).
  • Read from the first scenario only: the cycle weights (networks.py:1354-1361).
  • Mishandled or refused by PyPSA with scenarios: the delays, the transformer phase shift, the BODF (outage factors) and *_tech_capacity_weight.
  • Network-wide: snapshot weightings, period weights, scenario_weight, CVaR_* and transmission_losses.

The line and link volume weights do gain scenario, so rung 40's per-scenario carrier_attribute and investment_period are now done.

Rungs. Each rung's objective with per-scenario values, and with the same values in both scenarios:

Rung Differs by scenario Per scenario Uniform
41 marginal_cost 20/80 and link efficiency 0.9/0.6 17964.0 15660.0
42 capital_cost 20/60 and p_nom_max 100/30 5050.0 1943.0

Rung 14's per-scenario p_max_pu changes from out to done.

New out rows. Each is a PyPSA 1.3.0 misbehaviour and was run:

  • A per-scenario link delay applies in every scenario (constraints.py:1269).
  • A transformer in a cycle crashes (constraints.py:1654).
  • Any committable component crashes with scenarios (constraints.py:1872). I reproduced this myself: KeyError "not all values found in index 'scenario'".
  • *_p_nom_set crashes with scenarios (constraints.py:1708).

Gates.

  • pixi run lint: green.
  • pixi run test: 1740 passed. I ran it again myself.
  • pixi run docs-build: clean.
  • pixi run compile-tex: 27 documents, after clearing build/tex.
  • uv run --script examples/references/pypsa/reference.py --check: every rung solves to its record. I ran it again myself.

Worse. In the typeset dimension list, the scenario entry now names the three long *_maintenance_cover relations.

Not done. I added no language construct. A flag that adds scenario to every parameter would need a second list of exceptions. I preferred the explicit edit.

@read-the-docs-community

read-the-docs-community Bot commented Sep 24, 2026

Copy link
Copy Markdown

Documentation build overview

📚 math-spec | 🛠️ Build #34740821 | 📁 Comparing e20eb56 against latest (9c7c0a5)

  🔍 Preview build  

30 files changed · ± 24 modified · - 6 deleted

± Modified

- Deleted

@FabianHofmann
FabianHofmann merged commit e52b90d into pypsa-complete Sep 24, 2026
6 checks passed
@FabianHofmann FabianHofmann mentioned this pull request Sep 24, 2026
Closed
26 of 37 tasks
@FBumann FBumann added the docs Documentation pages, guides, reference and README label Sep 24, 2026 — with Claude
FBumann added a commit that referenced this pull request Sep 28, 2026
@FBumann @FabianHofmann @claude
docs(pypsa): the pypsa spec is also 24 topic files that merge back to…
22cdebd 
… it, each component adding its share of a sum by name (#736)

* docs(pypsa): a single file covers the standard, stochastic and multi-period classes

Fold pypsa_stochastic and pypsa_multi_period into examples/pypsa.yaml; retire
the sibling pages, symbols and example files. Add the rung 14 and rung 15
sections to the gallery page, and guard the plain-run collapse to the standard
model with a frozen shape fixture.

* docs(pypsa): the spec covers the process and transformer components

Add the Process component (a generalized multi-port converter, superset of
Link) and the full-depth Transformer (a passive branch like Line, with tap
ratio and a fixed phase shift in the KVL cycle) to examples/pypsa.yaml, with
symbols, the regenerated gallery, rung 17 and rung 18 reference networks and
oracles, and the collapse guard extended to the new standard names.

* docs(pypsa): the spec covers quadratic marginal cost

* docs(pypsa): the spec covers transmission losses

* docs(pypsa): the spec covers the secant loss mode

* docs(pypsa): the secant loss mode carries a solved reference

rung_19 records the secant-mode triangle solved through the pinned pypsa,
objective 10840.93, the same 150 rows as the tangent rung. The two loss-cut
blocks now stand for both their tangent and secant PyPSA names, matched by a
gallery helper that reads every backticked name before the dash.

* docs(pypsa): the spec covers the optimised transformer phase shift

A phase-shifting transformer's angle becomes a per-snapshot decision where
phase_shift_min < phase_shift_max, bounded by them and entering the KVL cycle
sum in place of the fixed constant; a plain run keeps the shift fixed and
collapses byte-for-byte. rung_20 records the phase-shifter triangle solved
through the pinned pypsa, objective 16455.0.

* docs(pypsa): the carrier growth limit binds every extendable component

* docs(pypsa): the spec covers transformer transmission losses

A transformer carries its own loss under transmission_losses, as a line does:
the loss counted against its rating, its cap, its tangent or secant cuts and half
of it at either bus. A plain run collapses byte-for-byte. rung_22 and rung_23
record the transformer triangle in both modes, objectives 10643.48 and 10822.00.

* docs(pypsa): a committable unit serving its brought-in down time stays off

Generator_com_status_must_stay_down mirrors the must-stay-up block over a
data-prep mask, status fixed to zero; PyPSA names the row
Generator-com-status-min_down_time_must_stay_up. rung_24 records a cheap unit
held off for two snapshots, objective 9007.5.

* docs(pypsa): a committable link carries the generator's unit commitment

Link gains the status, start-up and shut-down variables, the committable,
big-M and modular p bounds, transitions, up and down times, both must-stay
rules, the committed ramp rows, n_mod and its costs, mirroring Generator.
rung_25 records five committable links, objective 14013.0.

* docs(pypsa): a committable process carries the generator's unit commitment

Process gains the same unit commitment blocks as Link, over its internal
power. rung_26 restates rung 25's links as processes drawing a quarter more
than they deliver, objective 15956.125.

* docs(pypsa): a ramp row follows pypsa for a modular committed build and a start-up ramp alone

A committable extendable modular unit takes the ordinary ramp rows against
one module, not the big-M rows. A start-up or shut-down ramp alone builds
the row, and a missing limit reads as the full build. rung_27 (45469.5) and
rung_28 (83283.0) record both for Generator, Link and Process.

* docs(pypsa): storage cycles or reopens per investment period, and a ramp restarts at a period start

* docs(pypsa): a security-constrained run limits every branch flow after any one listed outage

* docs(pypsa): an mga flag caps the system cost at a budget and minimises weighted builds

* docs(pypsa): mga leaves the standard spec until it can land as a patch file

Reverts 5109686. The commit is kept on docs/pypsa-mga and waits for #571.

* docs(pypsa): a storage built in a later period opens at its first active snapshot

* docs(pypsa): a unit may be scheduled off for maintenance

* docs(pypsa): the spec states what maintenance and late-opening storage assume of the data

* docs(pypsa): a growth limit counts no transformer, as pypsa does

* docs(pypsa): a global constraint may count one investment period, weighted by its years

* docs(pypsa): a process, a storage unit and a store may carry a quadratic marginal cost

* docs(pypsa): a store's power and a storage unit's dispatch and charging may each be pinned to a schedule

* docs(pypsa): a link's and a process's delay applies within each investment period

* docs(pypsa): a negative relative growth adds nothing to a carrier's growth limit, as pypsa clips it at zero

* docs(pypsa): a tech capacity expansion limit on a network with several scenarios is refused, as pypsa does

* docs(pypsa): the carrier growth rung says a transformer counts in no carrier

* docs(pypsa): a global constraint takes its own constant and sense in each scenario

* docs(pypsa): the stochastic rung says which component data spans a scenario

* docs(pypsa): component data spans a scenario wherever pypsa reads it per scenario (#688)

* docs(pypsa): component data spans a scenario wherever pypsa reads it per scenario

* docs(pypsa): two rungs show operating and first-stage data that differ by scenario

* docs(pypsa): a component's sign turns its term in the bus balance around

* docs(pypsa): the linearized commitment file keeps a unit down, ramps at the full build where a limit is missing, and schedules maintenance

* docs(pypsa): a ramp limit may change over time and lift at a snapshot

* docs(pypsa): a unit that came in running ramps from its p_init into the first snapshot

* docs(pypsa): the relaxed commitment file ramps every generator per snapshot and from its p_init, tightens at the full build and signs its balance

* docs(pypsa): a start and a stop cost what they cost, with no snapshot or period weight

* docs(pypsa): a growth limit binds only under multi_investment_periods

* docs(pypsa): a load that is not active draws nothing from its bus

* docs(pypsa): a security-constrained run dissipates no transmission loss

* docs: the changelog lists the pypsa spec that covers every model class and component

* docs(pypsa): an efficiency, a rate or a phase shift may change from snapshot to snapshot

* docs: the changelog lists efficiencies per snapshot

* docs(pypsa): a growth limit counts an asset only in the first period it stands in

A rung may now record a PyPSA bug: its intended objective from an oracle,
and what PyPSA 1.3.0 gives instead. Rung 51 records PyPSA/PyPSA#1938.

* docs(pypsa): a transmission cost or volume limit holds in every scenario

Rungs 52 and 53 record PyPSA/PyPSA#1939, where PyPSA 1.3.0 drops both rows.

* docs(pypsa): each scenario delays a link's and a process's flow by its own delay

Rung 54 records PyPSA/PyPSA#1941, where PyPSA 1.3.0 delivers the flow twice.

* docs(pypsa): transformer cycles, security-constrained runs, fixed builds and committable units hold in every scenario

Rungs 55 to 58 record PyPSA/PyPSA#1942 and PyPSA/PyPSA#1913, where PyPSA 1.3.0 raises.

* docs(pypsa): the first-snapshot ramp refusals link the open PyPSA question

* feat(language): two files that state the same spec write one text, and canonical --check fails a file that is not in it

Spec.to_yaml(canonical=True) writes the normal form: sections in one
order, declarations sorted by name, every expression printed from its
parsed tree with the terms of a sum and the factors of a product sorted,
one term per line. python -m mathspec canonical writes it; --check exits
1 for a file not in the form and --write rewrites it.

Squashes #530 and #718 onto main after #721, in its words: a file states
a spec. Also normalises a named expression written on one line, which the
form passed through as written.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Sr4SCw5A7GUF5ARhdCaKR1

* Add the changelog link for #731

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Sr4SCw5A7GUF5ARhdCaKR1

* feat(language): a spec is composed from files that each state part of it, and patched with files that each change part of it

merge composes fragments as peers: each loads on its own, owns what it
declares, and reads what a sibling declares under given:, which now takes
parameters, variables, named expressions and row families. A given
expression's dims bound what it reads. A given: expressions: entry marked
additive: true says the name is a sum other files add terms to; each
contributor declares an ordinary named expression, merge sums them and
keeps the marked entry. override lays patches over a base, field by
field, with null as the removal. Both return a loaded Spec.

Squashes #571, #690, #728 and #729 onto #731, in #721's words. Adds the
tutorial 'A spec in several files', with a test that runs every step, and
sorts the names under given: in the canonical form.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Sr4SCw5A7GUF5ARhdCaKR1

* Add the changelog link for #732

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Sr4SCw5A7GUF5ARhdCaKR1

* refactor(language): a term of a sum is an ordinary named expression, and only `merge` holds it to the rules of the sum

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H1F2gprjeiGPnvDaWjZwBB

* Add the changelog link for #734

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H1F2gprjeiGPnvDaWjZwBB

* The reader's description wins, and the term checks stay at load

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H1F2gprjeiGPnvDaWjZwBB

* feat(language): a file adds a term to an expression another file defines, under `given:`

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H1F2gprjeiGPnvDaWjZwBB

* Walk the loaded fragments by value

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H1F2gprjeiGPnvDaWjZwBB

* A term names an expression of its file, and merge adds it by name

`term:` on a `given: expressions:` entry now names a named expression the
same file declares, and no longer takes an expression written inline. The
term is then free: it takes `cases:`, a description and every rule of a
named expression, and it counts as read by the math.

`merge` adds the terms by name, without brackets around a name, and keeps
each term as a named expression of the composed spec. A refusal for terms
that land on no name names only a near miss.

The typeset term line is `name = ⋯ + term`. The tutorial, the reference,
the how-to, the golden model and the notation page follow. The #734 line
leaves the changelog; #732 carries the design.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Sr4SCw5A7GUF5ARhdCaKR1

* Name the term in a term collision, and suggest only a name a term can land on

Two fragments that declare one term name now hear that terms share one
namespace, not the dimension-rows advice. Terms that land on no name are
refused with a near miss among the definitions and readings only, since a
term name is no place for a term to land; `did_you_mean` takes
`listing=False` for the case where only a near miss helps. The how-to says
to name each term after its component.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Sr4SCw5A7GUF5ARhdCaKR1

* Cut pypsa.yaml into topic fragments whose components add named terms

`examples/pypsa/` holds the 24 fragments `tools/pypsa_split.py` writes from
#620's `examples/pypsa.yaml`. Each component's share of a sum (the bus
balance, the cycle sum, the operating cost, the carrier additions and the
five global-constraint totals) is a named expression of its own, such as
`Generator_injection`, and the `term:` of its `given:` entry names it. The
reader of each sum describes it: `network` and `power_flow` for the two
rows, `core` for the rest.

`check` merges the fragments to the canonical form of the one file with its
sums written as named terms, and compares that file with `pypsa.yaml` row by
row, every term substituted back.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Sr4SCw5A7GUF5ARhdCaKR1

* Add the changelog line for #736

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Sr4SCw5A7GUF5ARhdCaKR1

* fix(language): a merged spec's descriptions do not depend on the order the fragments are passed in, and a reader's fills one its owner left out (#739)

* Choose a merged description by fragment name, and let a reader's fill a gap

A shared dimension or relation, a reading several fragments share, and a
sum built from terms took the description of whichever fragment was passed
first. Each now takes the first description in the fragments' name order, as
the objective already did, so the order of the arguments reaches no field.
The order of the declarations still follows the order passed in, which is
presentation.

A folded given declaration dropped its reader's description even where the
declaration it folds into had none; it now fills that gap, and yields to the
owner's own.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Sr4SCw5A7GUF5ARhdCaKR1

* Add the changelog line for #739

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Sr4SCw5A7GUF5ARhdCaKR1

---------

Co-authored-by: Claude <noreply@anthropic.com>

* refactor(language): a term prints as the definition it is, and one rule folds every reading (#738)

* refactor(language): a term prints as the definition it is, and merge takes the fragments in name order once

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H1F2gprjeiGPnvDaWjZwBB

* Add the changelog link for #738

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H1F2gprjeiGPnvDaWjZwBB

* Retitle the changelog line: the name-order rule is #739's

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H1F2gprjeiGPnvDaWjZwBB

---------

Co-authored-by: Claude <noreply@anthropic.com>

* fix(typeset): a substituted term prints its leading minus as a subtraction, and the PyPSA split has a page per fragment

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H1F2gprjeiGPnvDaWjZwBB

* Expect the substituted sum over both dimensions, and link the changelog line to #740

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H1F2gprjeiGPnvDaWjZwBB

* feat(language): a named expression may declare the frame it is read over

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H1F2gprjeiGPnvDaWjZwBB

* Add the changelog link for #741

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H1F2gprjeiGPnvDaWjZwBB

* feat(language): a sum other files add terms to is declared as an expression with a frame and no body

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H1F2gprjeiGPnvDaWjZwBB

* Add the changelog link for #742

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H1F2gprjeiGPnvDaWjZwBB

* The owner of each PyPSA sum declares it with a frame and no body

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H1F2gprjeiGPnvDaWjZwBB

* The golden model substitutes a signed sum into a plus

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H1F2gprjeiGPnvDaWjZwBB

* docs: a file restates a shared dimension as its dtype alone, and merge carries the one description written for it (#743)

* An empty sum prints as a definition with an ellipsis for its body

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H1F2gprjeiGPnvDaWjZwBB

* The owner pages of the PyPSA split print their sums with an ellipsis

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H1F2gprjeiGPnvDaWjZwBB

* Keep main's patch case in test_composition, which #740 carried from an earlier #742

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CQxVP5uX2V4rpvJNPhbyR2

---------

Co-authored-by: Fabian <fab.hof@gmx.de>
Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@FBumann FBumann Awaiting requested review from FBumann FBumann is a code owner
@brynpickering brynpickering Awaiting requested review from brynpickering brynpickering is a code owner

Assignees

No one assigned

Labels

docs Documentation pages, guides, reference and README

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@FabianHofmann @FBumann