diff --git a/documentation/changelog.rst b/documentation/changelog.rst index 86e3a06d59..f8c902861f 100644 --- a/documentation/changelog.rst +++ b/documentation/changelog.rst @@ -36,6 +36,7 @@ New features * In the UI, the flex-context editor supports editing commitments (name, baseline and deviation prices, each accepting a fixed value or a sensor), also within each commodity context; the commitment's commodity follows the commodity tab being edited, and new commitments start with zero-valued baseline and prices [see `PR #2287 `_] * A commitment in the flex-context now requires a ``baseline`` and at least one deviation price (``up-price`` and/or ``down-price``), as already documented; commitment costs are reported in the scheduling results under the user-given name (with a ``(custom)`` suffix in the rare case the name collides with a scheduler-internal commitment name) [see `PR #2287 `_] * Support commodity-converting devices, such as a CHP, e-boiler or heat pump, by describing each of the device's commodity ports as a flex-model entry sharing one ``coupling`` group with fixed flow ratios (``coupling-coefficient``) [see `PR #2218 `_] +* Commodities without energy prices in the flex-context (e.g. a heat or steam network without a grid connection) are scheduled as internal nodes whose devices must balance each other at every time step [see `PR #2289 `_] * Commodity contexts that omit grid-connection fields (prices and site capacities) now get smart defaults instead of failing or silently leaving the grid unconstrained — for instance, a bare ``{"commodity": "gas"}`` is treated as having no grid connection; see :ref:`commodity_context_defaults` for the full rules [see `PR #2272 `_] * CLI support for adding/editing account attributes [see `PR #2242 `_] * Improve chart axis domain for event values not around zero, with a per-sub-chart ``y-axis`` option in ``sensors_to_show`` (default ``zero``, which pads the axis out to include zero) that can be set to ``data`` to fit a sub-chart's y-axis to the values shown, to an explicit ``[min, max]`` domain that the axis will cover at least (expanding to fit data beyond it), or to a strict ``{"min": min, "max": max}`` domain that the axis will never exceed (clamping data beyond it, with a warning when that happens), editable from the graph editor [see `PR #2244 `_] diff --git a/documentation/features/scheduling.rst b/documentation/features/scheduling.rst index 9db8fb7732..a653211690 100644 --- a/documentation/features/scheduling.rst +++ b/documentation/features/scheduling.rst @@ -41,6 +41,11 @@ The ``flex-context`` is independent of the type of flexible device that is optim With the flexibility context, we aim to describe the system in which the flexible assets operate, such as its physical and contractual limitations. For multi-commodity scheduling problems, the flex-context can be defined separately per commodity (e.g. electricity and gas). See :ref:`tut_multi_commodity` for a hands-on example. +A *non-electricity* commodity that defines no energy prices and no capacity (grid-connection) fields in the flex-context (e.g. a heat or steam network without a grid connection) is treated as an internal node: +its devices must balance each other at every time step, so everything produced into the node is consumed from it within the same time step. +Electricity is the exception: it is always assumed to be grid-connected, so electricity without a price raises an error rather than becoming an internal node. +Devices that convert between commodities (such as a CHP unit, gas boiler or electric heater) are described in the flex-model, one entry per commodity port, tied together by a ``coupling`` group. See :ref:`tut_converters` for a worked example flex-model. + Fields can have fixed values, but some fields can also point to sensors, so they will always represent the dynamics of the asset's environment (as long as that sensor has current data). The full list of flex-context fields follows below. For more details on the possible formats for field values, see :ref:`variable_quantities`. @@ -361,8 +366,15 @@ The sensor-referenced form is convenient when you pass the whole flex-model in o Here, the battery and PV installation may each individually schedule up to 2 kW, but their combined power flowing through the shared inverter is hard-limited to 2.5 kW. +.. _inflexible_devices_in_flex_model: + +Inflexible devices in the flex-model +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + Inflexible (measured) devices can be modelled in the flex-model too — for example, an unschedulable base load. To do so, model the inflexible device as its own asset and give its flex-model entry a single ``inflexible-consumption`` or ``inflexible-production`` reference to the sensor recording its power (the field name sets the sign convention, and source filters may be added). Such an entry carries no schedulable-device fields; it simply declares a fixed device whose power is accounted for. Like any device entry, it may set a ``commodity`` (defaulting to electricity), and its fixed power is then netted into that commodity's grid connection. +There are two places to declare an inflexible device, and the choice is about *where* it belongs rather than *what* it does. Listing its sensor in the flex-context's ``inflexible-consumption``/``inflexible-production`` fields describes plain site base load, which is a property of the connection. Giving it its own flex-model entry describes a device that sits somewhere specific in the asset tree — under a particular inverter, feeder or commodity — which is a property of the device. Both net the same fixed power into the grid connection. + The ``group`` field is optional on such an entry. Without it, the device is simply accounted for under the grid connection (just like listing its sensor in the flex-context's ``inflexible-consumption``/``inflexible-production`` fields, only declared on the asset instead). With it, the device *also* joins that group through the ordinary ``group`` field, exactly like a flexible member (the group's own flex-model entry, defining its capacities, must still be present), so that its fixed load or supply additionally counts towards the group's intermediate power constraint — for example, a base load sitting behind the same inverter or feeder as a battery. diff --git a/documentation/tut/multi-commodity.rst b/documentation/tut/multi-commodity.rst index 3861a0c7cf..69d85857af 100644 --- a/documentation/tut/multi-commodity.rst +++ b/documentation/tut/multi-commodity.rst @@ -257,6 +257,47 @@ The commitment-cost result keeps these as separate entries — ``electricity net .. note:: This same pattern extends to more devices and more commodities. Add further entries to the ``flex-model`` list (each with its ``commodity``) and a matching entry in the ``flex-context`` ``commodities`` list. As long as all commodities share one currency, FlexMeasures optimises them together and reports each commodity's cost on its own. + +.. _tut_converters: + +Converters between commodities +============================== + +A **converter** turns one commodity into another — a CHP unit (gas → electricity + steam), a gas boiler (gas → heat) or an electric heater (electricity → heat). +In the ``flex-model`` a converter is not a single device but **one entry per commodity port**, tied together by a shared ``coupling`` name. +The ``coupling-coefficient`` on each port fixes the conversion ratio relative to the shared coupling variable, so the ports always move together. + +For example, a CHP that turns gas into steam and electricity: + +.. code-block:: json + + [ + {"sensor": 6, "commodity": "gas", "coupling": "chp", "coupling-coefficient": 1.0, "consumption-capacity": "20 kW"}, + {"sensor": 7, "commodity": "steam", "coupling": "chp", "coupling-coefficient": 0.5, "production-capacity": "10 kW"}, + {"sensor": 8, "commodity": "electricity", "coupling": "chp", "coupling-coefficient": 0.3, "production-capacity": "6 kW"} + ] + +Here each kW of gas input produces 0.5 kW of steam and 0.3 kW of electricity. +Each port gives exactly one directional capacity, and that is what marks its direction: +the gas port only consumes, and the steam and electricity ports only produce. +The opposite direction defaults to zero, so it does not need to be written out. + +Note that the capacities agree with the coefficients: +20 kW of gas is the most the unit can burn, which is what caps steam at 10 kW and electricity at 6 kW. + +**Internal nodes.** A *non-electricity* commodity that lists no energy price and no capacity (grid-connection) field in the ``flex-context`` (e.g. a steam or heat network with no grid connection) is treated as an **internal node**: its devices must balance each other at every time step, so everything converters produce into the node must be consumed from it within the same step. +You can still create a flex-context entry for such a commodity to define ``inflexible-consumption`` and/or ``inflexible-production`` (its fixed demand and supply) without FlexMeasures assuming a grid connection is available. +Alternatively, omit it from the ``flex-context`` entirely (fixed demand and supply can also be set up with flex-models of their own, see :ref:`inflexible_devices_in_flex_model`). +Electricity is the exception: it is always assumed to be grid-connected, so missing electricity prices raise an error rather than turning electricity into an internal node. + +Coupled converters and internal nodes compose, which is what lets these two features reach beyond the single unit shown above. +Chain a few of them and you can describe an industrial site: converters feeding internal nodes, those nodes feeding further converters, and only the priced commodities meeting the grid. +Nothing extra is needed to get there — no new fields, just more entries of the kinds already shown. +We do not walk through such a site here, and yours will not look like anyone else's, but the pieces are the ones on this page. + +**Pricing what flows through a node.** An internal node's flows sum to zero by construction, so a commitment scoped to the node itself would bind nothing: there is no net flow to price. +To put a cost on throughput — pipe wear, say, or a conversion levy — scope the commitment to the devices *producing* into the node, whose summed flow is what actually passes through it. + We hope this demonstration helped to illustrate multi-commodity scheduling. To revisit scheduling several devices that share a single commodity and stock, head back to :ref:`tut_multi_feed_storage`. Next, in :ref:`tut_toy_schedule_process`, we'll turn to something different: the optimal timing of processes with fixed energy work and duration. diff --git a/flexmeasures/data/models/planning/highspy_optimization.py b/flexmeasures/data/models/planning/highspy_optimization.py index 7312f77ca7..01b60199f1 100644 --- a/flexmeasures/data/models/planning/highspy_optimization.py +++ b/flexmeasures/data/models/planning/highspy_optimization.py @@ -224,6 +224,7 @@ def device_scheduler_highspy( # noqa C901 ems_constraint_groups: list[list[int]] | None = None, device_power_bands: list[list[tuple[float, float]] | None] | None = None, coupling_groups: dict[str, list[tuple[int, float]]] | None = None, + balance_groups: dict[str, list[int]] | None = None, ) -> tuple[list[pd.Series], float, HighspySolverResults, HighspyModel]: """Direct HiGHS implementation of ``device_scheduler``. @@ -255,6 +256,7 @@ def device_scheduler_highspy( # noqa C901 ems_constraint_groups=ems_constraint_groups, device_power_bands=device_power_bands, coupling_groups=coupling_groups, + balance_groups=balance_groups, ) # Local aliases, so that the model below reads as it did before the (solver-agnostic) @@ -272,6 +274,7 @@ def device_scheduler_highspy( # noqa C901 Md, Mc = problem.Md, problem.Mc band_lookup = problem.band_lookup coupling_device_specs = problem.coupling_device_specs + balance_group_specs = problem.balance_group_specs _initial_stock_of = problem.initial_stock_of # --------------------------------------------------------------- @@ -649,6 +652,18 @@ def _add_commitment_rows(c, quantity, jj, lb, ub, devices, is_stock) -> None: np.tile([1.0, -float(coeff)], (T, 1)), ) + # node_balance_constraints: an internal commodity node stores nothing, + # so everything produced into it is consumed from it within the same time step, + # i.e. sum_d(ems_power[d, j]) == 0. + # Groups with no devices were dropped during preparation, so they add no row here, + # matching the Pyomo path's Constraint.Skip for that case. + for devices_in_node in balance_group_specs: + js = np.arange(T) + idx = np.column_stack([col_ems + int(d) * T + js for d in devices_in_node]) + rows.add_uniform_rows( + np.zeros(T), np.zeros(T), idx, np.ones_like(idx, dtype=float) + ) + # --------------------------------------------------------------- # Build and solve the HiGHS model # --------------------------------------------------------------- diff --git a/flexmeasures/data/models/planning/linear_optimization.py b/flexmeasures/data/models/planning/linear_optimization.py index f174b5ca3a..24b7e7ec2e 100644 --- a/flexmeasures/data/models/planning/linear_optimization.py +++ b/flexmeasures/data/models/planning/linear_optimization.py @@ -134,6 +134,7 @@ def device_scheduler( # noqa C901 initial_stock: float | list[float] = 0, stock_groups: dict[int, list[int]] | None = None, coupling_groups: dict[str, list[tuple[int, float]]] | None = None, + balance_groups: dict[str, list[int]] | None = None, ems_constraint_groups: list[list[int]] | None = None, device_power_bands: list[list[tuple[float, float]] | None] | None = None, ) -> tuple[list[pd.Series], float, SolverResults, ConcreteModel]: @@ -188,6 +189,16 @@ def device_scheduler( # noqa C901 binary variables (one per device per band per time step). Use None (per device or for the whole argument) for devices without band restrictions. + :param balance_groups: Flow-balance constraints for internal commodity nodes (e.g. a heat or steam network without a grid connection). + Each entry maps a node name to a list of device indices whose commodity-side flows must balance at every time step: + ``sum_d(ems_power[d, j]) == 0``. + In other words, everything produced into the node is consumed from it within the same time step; + the node itself stores nothing. + Derivative efficiencies and stock deltas describe each device's own stock-side conversion, + and do not enter this commodity-side balance. + To add storage to a node, include a storage device in the group: + its flow absorbs the imbalance, and its stock is bounded by its own device constraints. + Potentially deprecated arguments: commitment_quantities: amounts of flow specified in commitments (both previously ordered and newly requested) - e.g. in MW or boxes/h @@ -237,6 +248,7 @@ def device_scheduler( # noqa C901 ems_constraint_groups=ems_constraint_groups, device_power_bands=device_power_bands, coupling_groups=coupling_groups, + balance_groups=balance_groups, ) # Local aliases, @@ -255,6 +267,7 @@ def device_scheduler( # noqa C901 band_lookup = problem.band_lookup _initial_stock_of = problem.initial_stock_of coupling_device_specs = problem.coupling_device_specs + balance_group_specs = problem.balance_group_specs # Add indices for devices (d), datetimes (j) and commitments (c) model.d = RangeSet(0, len(device_constraints) - 1, doc="Set of devices") @@ -675,6 +688,27 @@ def flow_coupling_rule(m, c, j): model.coupling_device_range, model.j, rule=flow_coupling_rule ) + if balance_group_specs: + model.balance_group_range = RangeSet(0, len(balance_group_specs) - 1) + + def node_balance_rule(m, b, j): + """Balance the power flows of an internal commodity node at every time step. + + Everything produced into the node must be consumed from it within the same time step. + The balance sums the devices' commodity-side flows (ems_power); + derivative efficiencies and stock deltas describe each device's own stock-side conversion (e.g. of a shared buffer), + and do not enter the commodity balance. + """ + return ( + 0, + sum(m.ems_power[d, j] for d in balance_group_specs[b]), + 0, + ) + + model.node_balance_constraints = Constraint( + model.balance_group_range, model.j, rule=node_balance_rule + ) + # Power bands (S2 operation modes): a banded device must operate within # exactly one of its declared signed power ranges at every time step. # Which band it runs in (per time step) is a free binary decision variable; diff --git a/flexmeasures/data/models/planning/scheduling_problem.py b/flexmeasures/data/models/planning/scheduling_problem.py index 840a5b98a1..f54a82d130 100644 --- a/flexmeasures/data/models/planning/scheduling_problem.py +++ b/flexmeasures/data/models/planning/scheduling_problem.py @@ -222,6 +222,9 @@ class SchedulingProblem: #: (group index, device index, coefficient) triples for hard flow-coupling constraints coupling_device_specs: list[tuple[int, int, float]] + #: device lists of the balance groups (internal commodity nodes), empty groups dropped + balance_group_specs: list[list[int]] + initial_stock: float | list[float] #: The commitments as passed in, before the sub-commitment split. @@ -272,6 +275,7 @@ def prepare_scheduling_problem( # noqa C901 ems_constraint_groups: list[list[int]] | None = None, device_power_bands: list[list[tuple[float, float]] | None] | None = None, coupling_groups: dict[str, list[tuple[int, float]]] | None = None, + balance_groups: dict[str, list[int]] | None = None, ) -> SchedulingProblem: """Normalise and validate ``device_scheduler``'s arguments into a SchedulingProblem. @@ -369,6 +373,13 @@ def prepare_scheduling_problem( # noqa C901 for d_idx, coeff in members: coupling_device_specs.append((g_idx, d_idx, coeff)) + # Collect the device lists of the balance groups (internal commodity nodes). + balance_group_specs: list[list[int]] = [] + if balance_groups: + balance_group_specs = [ + list(devices) for devices in balance_groups.values() if devices + ] + # Move commitments from old structure to new if commitments is None: commitments = [] @@ -546,6 +557,7 @@ def prepare_scheduling_problem( # noqa C901 Mc=Mc, band_lookup=band_lookup, coupling_device_specs=coupling_device_specs, + balance_group_specs=balance_group_specs, initial_stock=initial_stock, original_commitments=original_commitments, ) diff --git a/flexmeasures/data/models/planning/storage.py b/flexmeasures/data/models/planning/storage.py index bf68ed9a99..c3ecc45bd2 100644 --- a/flexmeasures/data/models/planning/storage.py +++ b/flexmeasures/data/models/planning/storage.py @@ -179,6 +179,10 @@ def _prepare(self, skip_validation: bool = False) -> tuple: # noqa: C901 # with signed coefficients per canonical device index. self.coupling_groups = inventory.coupling_groups + # Balance groups for internal commodity nodes (commodities without energy prices, i.e. without a grid connection) + # are derived further below, once the devices of each commodity are enumerated. + self.balance_groups: dict[str, list[int]] = {} + # Group entries (intermediate power constraints on groups of devices, e.g. a # sub-EMS) come classified from the inventory, together with the resolved # (leaf) group membership. Accessing `group_to_devices` also detects cyclic @@ -439,10 +443,34 @@ def device_list_series( if production_price is None: production_price = consumption_price - if consumption_price is None: - raise ValueError( - f"Missing consumption price for commodity '{commodity}'." + # A context is an internal node only when the user gave no grid-connection signal at all -- + # neither prices nor capacity fields. + # See CommodityFlexContextSchema.fill_grid_connection_defaults, which flags this with is_internal_node. + # A commodity given a capacity but no price still carries smart-defaulted zero prices, + # yet it declares a grid connection, so it must not be treated as an internal node. + is_internal_node = consumption_price is None or ( + commodity_context.get("is_internal_node", False) + and consumption_price_sensor is None + and production_price_sensor is None + ) + + if is_internal_node: + if commodity == "electricity": + # Electricity is assumed to be grid-connected, + # so a missing price is treated as a configuration error rather than as an internal node. + raise ValueError( + f"Missing consumption price for commodity '{commodity}'." + ) + # A non-electricity commodity without energy prices is treated as an internal node, + # e.g. a heat or steam network without a grid connection: + # its devices must balance each other at every time step, + # and it needs no commitments or EMS-level capacity constraints. + current_app.logger.debug( + f"Commodity '{commodity}' has no energy prices; treating it as an " + f"internal node whose devices (indices {devices}) balance each other." ) + self.balance_groups[commodity] = list(devices) + continue # Energy prices for this commodity. up_deviation_prices = get_continuous_series_sensor_or_quantity( @@ -3330,6 +3358,7 @@ def compute(self, skip_validation: bool = False) -> SchedulerOutputType: initial_stock=initial_stock, stock_groups=self.stock_groups, coupling_groups=self.coupling_groups if self.coupling_groups else None, + balance_groups=getattr(self, "balance_groups", None) or None, device_power_bands=[ dc.attrs.get("operation_modes") for dc in device_constraints ], diff --git a/flexmeasures/data/models/planning/tests/test_commitments.py b/flexmeasures/data/models/planning/tests/test_commitments.py index e9d94de23e..be406ead94 100644 --- a/flexmeasures/data/models/planning/tests/test_commitments.py +++ b/flexmeasures/data/models/planning/tests/test_commitments.py @@ -1852,9 +1852,15 @@ def _flow_df(**kwargs) -> pd.DataFrame: def _run_factory_scenario( gas_price: float, elec_price: float, + use_balance_groups: bool = False, ) -> tuple: """Run the simplified factory scenario and return the 7 device schedules. + With ``use_balance_groups=False``, + the heat and steam nodes are balanced via shared stock groups whose first ("reference") device carries min=max=0 stock bounds. + With ``use_balance_groups=True``, + the same nodes are expressed directly as ``balance_groups``, needing neither stock groups nor reference-device bounds. + Devices ~~~~~~~ d=0 e-heater electricity → heat coupling (ems_power ≥ 0, i.e. consumes electricity) @@ -1905,11 +1911,14 @@ def _df(**kwargs) -> pd.DataFrame: defaults.update(kwargs) return pd.DataFrame(defaults, index=index) + # With balance groups, no reference device needs min=max=0 stock bounds. + node_bounds = {} if use_balance_groups else {"min": 0.0, "max": 0.0} + device_constraints = [ # d=0 e-heater: heat-node reference device. The min=max=0 forces the heat # node to balance at every step (zero-capacity flow node), making # the per-step dispatch deterministic despite flat prices. - _df(min=0.0, max=0.0, **{"derivative max": HEATER_POWER_MAX}), + _df(**node_bounds, **{"derivative max": HEATER_POWER_MAX}), # d=1 gas boiler: up to 100 kW gas → 100 kW heat (efficiency 1 for clean maths in test) _df(**{"derivative max": BOILER_GAS_MAX, "commodity": "gas"}), # d=2 steamer: can only produce steam (negative ems_power). @@ -1927,8 +1936,7 @@ def _df(**kwargs) -> pd.DataFrame: # d=4 CHP heat output: positive ems_power adds heat to the steam node. # The min=max=0 forces the steam node to balance at every step. _df( - min=0.0, - max=0.0, + **node_bounds, **{ "derivative min": -CHP_GAS_MAX * ETA_HEAT, "derivative max": 0.0, @@ -1952,11 +1960,18 @@ def _df(**kwargs) -> pd.DataFrame: index=index, ) - # stock group: all heat-buffer devices share the same stock - # (key 0 is an arbitrary group id, not a device index) - heat_group_id = 0 - steam_group_id = 1 - stock_groups = {heat_group_id: [0, 1, 2], steam_group_id: [2, 4, 6]} + # Node membership: the steamer (d=2) converts heat to steam, + # so it belongs to both nodes (its single flow drains heat and feeds steam). + heat_node = [0, 1, 2] + steam_node = [2, 4, 6] + if use_balance_groups: + stock_groups = None + balance_groups = {"heat": heat_node, "steam": steam_node} + else: + # stock group: all heat-buffer devices share the same stock. + # Keys 0 and 1 are arbitrary group ids, not device indices. + stock_groups = {0: heat_node, 1: steam_node} + balance_groups = None # CHP coupling: coefficients are signed efficiency fractions. # coeff_heat = -η_heat = -0.5 → P_heat = -0.5 * alpha = -0.5 * P_gas @@ -1998,6 +2013,7 @@ def _df(**kwargs) -> pd.DataFrame: commitments=commitments, stock_groups=stock_groups, coupling_groups=coupling_groups, + balance_groups=balance_groups, ) assert results.solver.termination_condition == "optimal", ( @@ -2007,12 +2023,14 @@ def _df(**kwargs) -> pd.DataFrame: return tuple(schedules) -def test_factory_chp_dispatch(): +@pytest.mark.parametrize("use_balance_groups", [False, True]) +def test_factory_chp_dispatch(use_balance_groups): """Factory: CHP + gas boiler + e-heater competing to meet a fixed steam demand. - The shared heat buffer (modelled via ``stock_groups``) is drained at a - constant rate of 15 kW by the steam demand device. Two price scenarios - verify that the optimizer correctly chooses the cheapest heat source. + The heat and steam nodes are balanced either via shared stock groups with a min=max=0 reference device (``use_balance_groups=False``), + or via explicit ``balance_groups`` (``use_balance_groups=True``) — both must yield the same dispatch. + The steam node is drained at a constant rate of 15 kW by the steam demand device. + Two price scenarios verify that the optimizer correctly chooses the cheapest heat source. Scenario A — gas cheaper than electricity ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -2061,7 +2079,9 @@ def test_factory_chp_dispatch(): # Scenario A: gas cheaper — CHP at max, gas boiler fills the rest # # ------------------------------------------------------------------ # (e_heater, gas_boiler, steamer, chp_gas, chp_heat, chp_power, demand) = ( - _run_factory_scenario(gas_price=20.0, elec_price=50.0) + _run_factory_scenario( + gas_price=20.0, elec_price=50.0, use_balance_groups=use_balance_groups + ) ) expected_chp_gas = pd.Series(20.0, index=e_heater.index) @@ -2126,7 +2146,9 @@ def test_factory_chp_dispatch(): # Scenario B: electricity cheaper — e-heater meets all demand # # ------------------------------------------------------------------ # (e_heater, gas_boiler, steamer, chp_gas, chp_heat, chp_power, demand) = ( - _run_factory_scenario(gas_price=100.0, elec_price=10.0) + _run_factory_scenario( + gas_price=100.0, elec_price=10.0, use_balance_groups=use_balance_groups + ) ) expected_eheater_b = pd.Series(15.0, index=e_heater.index) @@ -2174,7 +2196,9 @@ def test_factory_chp_dispatch(): # Scenario C: gas slightly cheaper — gas boiler at max, e-heater fills the rest # # --------------------------------------------------------------------------------- # (e_heater, gas_boiler, steamer, chp_gas, chp_heat, chp_power, demand) = ( - _run_factory_scenario(gas_price=50.0, elec_price=55.0) + _run_factory_scenario( + gas_price=50.0, elec_price=55.0, use_balance_groups=use_balance_groups + ) ) expected_chp_gas = pd.Series(0.0, index=e_heater.index) diff --git a/flexmeasures/data/models/planning/tests/test_highspy_equivalence.py b/flexmeasures/data/models/planning/tests/test_highspy_equivalence.py index 90407d91ad..42d0df885e 100644 --- a/flexmeasures/data/models/planning/tests/test_highspy_equivalence.py +++ b/flexmeasures/data/models/planning/tests/test_highspy_equivalence.py @@ -351,6 +351,31 @@ def scenario_chp_coupling_groups(): ) +def scenario_internal_commodity_balance(): + """A heat node with a producer and a fixed consumer, and no grid connection. + + The node stores nothing, so the two devices' flows must cancel at every time step. + The consumer's draw is pinned by "derivative equals", + so the balance is what forces the producer to match it: + drop the constraint and the producer has no reason to run at all. + """ + index = make_index() + prices = make_prices(index) + consumer = make_flow_device(-1, 0) + consumer["derivative equals"] = -0.4 + return dict( + device_constraints=[ + make_flow_device(0, 1), # heat producer, free within its band + consumer, # heat consumer, fixed draw + make_battery_constraints(), + ], + ems_constraints=initialize_df(COLUMNS, START, END, RESOLUTION), + commitments=[make_energy_commitment(index, prices, devices=2)], + initial_stock=[0.0, 0.0, 0.5], + balance_groups={"heat": [0, 1]}, + ) + + @pytest.mark.parametrize( "make_scenario", [ @@ -361,6 +386,7 @@ def scenario_chp_coupling_groups(): scenario_ems_level_flow_commitment, scenario_ems_level_commodity_commitment, scenario_chp_coupling_groups, + scenario_internal_commodity_balance, ], ids=lambda f: f.__name__.replace("scenario_", ""), ) diff --git a/flexmeasures/data/models/planning/tests/test_storage.py b/flexmeasures/data/models/planning/tests/test_storage.py index 1589a47280..4bf74a73a1 100644 --- a/flexmeasures/data/models/planning/tests/test_storage.py +++ b/flexmeasures/data/models/planning/tests/test_storage.py @@ -15,6 +15,7 @@ ) from flexmeasures.data.models.planning.storage import StorageScheduler from flexmeasures.data.models.planning.utils import initialize_index +from flexmeasures.data.models.data_sources import DataSource from flexmeasures.data.models.time_series import Sensor, TimedBelief from flexmeasures.data.models.planning.tests.utils import ( check_constraints, @@ -2161,6 +2162,175 @@ def test_storage_scheduler_chp_coupling(app, db): ) +def test_factory_chp_dispatch_through_storage_scheduler(app, db): + """The full factory scenario, scheduled end-to-end through ``StorageScheduler.compute()``. + + That is a CHP, a gas boiler and an e-heater, together meeting a fixed steam demand. + + Unlike the engine-level ``test_factory_chp_dispatch``, which passes balance groups to ``device_scheduler`` directly, + this test only supplies a flex-model and a flex-context. + Each converter is described as one device per commodity port, tied together by a coupling group. + The heat and steam commodities have no energy prices in the flex-context, + so the scheduler derives internal-node balance groups for them. + + Topology (flex-model device indices):: + + electricity (grid) --0--> [e-heater] --1--> heat + gas (grid) --2--> [boiler] --3--> heat + heat --4--> [steamer] --5--> steam + gas (grid) --6--> [CHP] --7--> steam + --8--> electricity (grid) + steam --9--> fixed 15 kW demand (inflexible sensor) + + Prices: gas 20 EUR/MWh, electricity 50 EUR/MWh. + Marginal cost per kW of steam: CHP (20·20 − 50·6) / 10 = 10, boiler-via-steamer 20, e-heater-via-steamer 50. + So the CHP runs at maximum (20 kW gas → 10 kW steam + 6 kW power), + and the boiler covers the remaining 5 kW of steam via the steamer; + the e-heater stays off. + """ + factory_type = get_or_create_model(GenericAssetType, name="factory") + factory = GenericAsset( + name="Factory (end-to-end CHP dispatch)", generic_asset_type=factory_type + ) + db.session.add(factory) + db.session.flush() + + start = pd.Timestamp("2026-01-01T00:00:00+01:00") + end = pd.Timestamp("2026-01-01T04:00:00+01:00") + resolution = timedelta(hours=1) + + def make_sensor(name: str) -> Sensor: + sensor = Sensor( + name=name, generic_asset=factory, unit="MW", event_resolution=resolution + ) + db.session.add(sensor) + return sensor + + eheater_elec_in = make_sensor("e-heater electricity input") + eheater_heat_out = make_sensor("e-heater heat output") + boiler_gas_in = make_sensor("boiler gas input") + boiler_heat_out = make_sensor("boiler heat output") + steamer_heat_in = make_sensor("steamer heat input") + steamer_steam_out = make_sensor("steamer steam output") + chp_gas_in = make_sensor("CHP gas input") + chp_steam_out = make_sensor("CHP steam output") + chp_power_out = make_sensor("CHP power output") + steam_demand = make_sensor("steam demand") + db.session.flush() + + # A constant 15 kW steam demand, recorded as beliefs. + # By default, power sensors store consumption as negative values. + # (get_power_values flips the sign to the scheduler's consumption-positive convention.) + index = initialize_index(start, end, resolution) + source = get_or_create_model(DataSource, name="test source", type="forecaster") + db.session.add_all( + TimedBelief( + sensor=steam_demand, + source=source, + event_start=dt, + belief_time=start, + event_value=-15e-3, # 15 kW in MW + ) + for dt in index + ) + db.session.commit() + + def input_port(sensor: Sensor, commodity: str, coupling: str, max_power: str): + return { + "sensor": sensor.id, + "commodity": commodity, + "coupling": coupling, + "coupling-coefficient": 1.0, + "power-capacity": max_power, + "production-capacity": "0 kW", + } + + def output_port( + sensor: Sensor, commodity: str, coupling: str, coefficient: float = 1.0 + ): + return { + "sensor": sensor.id, + "commodity": commodity, + "coupling": coupling, + "coupling-coefficient": coefficient, + "power-capacity": "1 MW", + "consumption-capacity": "0 kW", + } + + flex_model = [ + input_port(eheater_elec_in, "electricity", "eheater", "100 kW"), + output_port(eheater_heat_out, "heat", "eheater"), + input_port(boiler_gas_in, "gas", "boiler", "10 kW"), + output_port(boiler_heat_out, "heat", "boiler"), + input_port(steamer_heat_in, "heat", "steamer", "1 MW"), + output_port(steamer_steam_out, "steam", "steamer"), + input_port(chp_gas_in, "gas", "chp", "20 kW"), + output_port(chp_steam_out, "steam", "chp", coefficient=0.5), + output_port(chp_power_out, "electricity", "chp", coefficient=0.3), + ] + + flex_context = [ + { + "commodity": "electricity", + "consumption-price": "50 EUR/MWh", + "production-price": "50 EUR/MWh", + }, + { + "commodity": "gas", + "consumption-price": "20 EUR/MWh", + "production-price": "20 EUR/MWh", + }, + { + # No prices: steam is an internal node. Its fixed demand is inflexible. + "commodity": "steam", + "inflexible-device-sensors": [steam_demand.id], + }, + # The heat commodity has no context at all: also an internal node. + ] + + scheduler = StorageScheduler( + asset_or_sensor=factory, + start=start, + end=end, + resolution=resolution, + belief_time=start, + flex_model=flex_model, + flex_context=flex_context, + return_multiple=True, + ) + + results = scheduler.compute(skip_validation=True) + + # The scheduler derived one balance group per priceless commodity: + # heat: e-heater out (1), boiler out (3), steamer in (4) + # steam: steamer out (5), CHP out (7), inflexible demand (9) + assert scheduler.balance_groups == {"heat": [1, 3, 4], "steam": [5, 7, 9]} + + schedules = { + r["sensor"]: r["data"] for r in results if r.get("name") == "storage_schedule" + } + + expected_mw = { + eheater_elec_in: 0.0, + eheater_heat_out: 0.0, + boiler_gas_in: 0.005, + boiler_heat_out: -0.005, + steamer_heat_in: 0.005, + steamer_steam_out: -0.005, + chp_gas_in: 0.020, + chp_steam_out: -0.010, + chp_power_out: -0.006, + } + for sensor, expected_value in expected_mw.items(): + np.testing.assert_allclose( + schedules[sensor], + expected_value, + rtol=1e-4, + atol=1e-9, + err_msg=f"Unexpected schedule for {sensor.name}", + ) + + def test_off_tick_soc_relaxation_covers_all_devices_of_a_shared_stock( add_battery_assets, db ): diff --git a/flexmeasures/data/schemas/scheduling/__init__.py b/flexmeasures/data/schemas/scheduling/__init__.py index 60c63ebad4..7ed14ae0dc 100644 --- a/flexmeasures/data/schemas/scheduling/__init__.py +++ b/flexmeasures/data/schemas/scheduling/__init__.py @@ -688,6 +688,14 @@ def fill_grid_connection_defaults(self, data: dict, original_data: dict, **kwarg or has_power_capacity ) + # Record durably whether this commodity carries no user-given grid-connection signal at all -- + # neither prices nor capacity fields. + # Only then may the scheduler treat it as an internal node (devices balancing each other). + # A capacity field (cases 4-6) declares a grid connection, + # so a commodity with a capacity but no price is NOT an internal node, + # even though its prices get smart-defaulted to zero here. + data["is_internal_node"] = not any_given + currency = data.get("shared_currency_unit") or "EUR" zero_price = ur.Quantity(f"0 {currency}/MWh") zero_capacity = ur.Quantity("0 MW") diff --git a/flexmeasures/data/schemas/scheduling/storage.py b/flexmeasures/data/schemas/scheduling/storage.py index 87ddc83ca0..e62fa82a57 100644 --- a/flexmeasures/data/schemas/scheduling/storage.py +++ b/flexmeasures/data/schemas/scheduling/storage.py @@ -34,8 +34,6 @@ is_energy_unit, ) -ALLOWED_COMMODITIES = {"electricity", "gas"} - #: User-facing (hyphenated) flex-model keys that may co-exist with an inflexible-device #: declaration: its own identity/grouping/commodity and the always-defaulted activation @@ -59,6 +57,18 @@ ) +def _validate_commodity_name(commodity: str): + """Reject blank/whitespace-only commodity names. + + Any non-blank label is allowed: an internal commodity node can carry a name like "steam" or "heat", + so the set of commodities is open rather than an enumeration. + """ + if not isinstance(commodity, str) or not commodity.strip(): + raise ValidationError( + "commodity must be a non-empty string.", field_name="commodity" + ) + + def _validate_coupling_name(coupling: str | None): """Reject blank/whitespace-only coupling names. @@ -865,7 +875,6 @@ class DBStorageFlexModelSchema(Schema): required=False, data_key="commodity", load_default="electricity", - validate=OneOf(["electricity", "gas"]), metadata=dict(description="Commodity label for this device/asset."), ) @@ -904,6 +913,10 @@ def validate_group(self, group: dict, **kwargs): def validate_inflexible_device(self, data: dict, original_data: dict, **kwargs): validate_inflexible_flex_model_entry(data, original_data) + @validates("commodity") + def validate_commodity(self, commodity: str, **kwargs): + _validate_commodity_name(commodity) + @validates("coupling") def validate_coupling(self, coupling: str | None, **kwargs): _validate_coupling_name(coupling) diff --git a/flexmeasures/data/schemas/tests/test_scheduling.py b/flexmeasures/data/schemas/tests/test_scheduling.py index 6192aaefc8..15dffa17b7 100644 --- a/flexmeasures/data/schemas/tests/test_scheduling.py +++ b/flexmeasures/data/schemas/tests/test_scheduling.py @@ -1338,6 +1338,37 @@ def test_commodity_flex_context_smart_defaults(context_input, expected): ] +@pytest.mark.parametrize( + ["context_input", "expected_is_internal_node"], + [ + # No grid-connection signal at all -> internal node. + ({"commodity": "gas"}, True), + # A price declares a grid connection -> not an internal node. + ({"commodity": "gas", "consumption-price": "10 EUR/MWh"}, False), + # A capacity field also declares a grid connection, even without any price, + # since its prices get smart-defaulted to zero. + # This must NOT be flagged as an internal node -- + # otherwise the scheduler would skip EMS constraints, + # and force a per-step balance for a genuinely grid-connected commodity. + ({"commodity": "gas", "site-consumption-capacity": "5 MW"}, False), + ({"commodity": "gas", "site-production-capacity": "5 MW"}, False), + ({"commodity": "gas", "site-power-capacity": "5 MW"}, False), + ], +) +def test_commodity_flex_context_internal_node_flag( + context_input, expected_is_internal_node +): + """A commodity is an internal node only when the user gave neither prices nor any capacity/grid-connection field. + + See CommodityFlexContextSchema.fill_grid_connection_defaults. + """ + from flexmeasures.data.schemas.scheduling import CommodityFlexContextSchema + + loaded = CommodityFlexContextSchema().load(context_input) + + assert loaded.get("is_internal_node", False) == expected_is_internal_node + + @pytest.mark.parametrize( ["flex_context_listing", "fails"], [ @@ -1892,3 +1923,75 @@ def test_db_flex_context_schema_inflexible_devices( ] } ) + + +def test_db_flex_model_accepts_an_internal_commodity(app): + """A db-stored flex-model accepts a commodity outside electricity and gas. + + Internal commodity nodes carry labels like "steam" or "heat", + so the set of commodities is open rather than an enumeration. + Without this, a converter feeding an internal node could be scheduled but not stored. + """ + loaded = DBStorageFlexModelSchema().load( + { + "commodity": "steam", + "coupling": "chp", + "coupling-coefficient": 0.5, + "production-capacity": "10 kW", + } + ) + assert loaded["commodity"] == "steam" + + +@pytest.mark.parametrize("blank", ["", " ", "\t"]) +def test_blank_commodity_is_rejected(app, blank): + """An open commodity set still excludes blank names, on both schemas.""" + with pytest.raises(ValidationError): + DBStorageFlexModelSchema().load({"commodity": blank}) + with pytest.raises(ValidationError): + StorageFlexModelSchema( + start=datetime(2026, 6, 1, tzinfo=pytz.utc), sensor=None + ).load({"commodity": blank, "power-capacity": "20 kW"}) + + +def test_tutorial_chp_example_validates(app): + """The CHP example in the multi-commodity tutorial validates as written. + + It is the example a reader copies, so it should load through the schema that stores it, + and each port's coupling direction should resolve from its single directional capacity. + """ + from flexmeasures.data.models.planning.devices import ( + _resolve_coupling_coefficient, + ) + + ports = [ + ( + { + "commodity": "gas", + "coupling-coefficient": 1.0, + "consumption-capacity": "20 kW", + }, + 1.0, + ), + ( + { + "commodity": "steam", + "coupling-coefficient": 0.5, + "production-capacity": "10 kW", + }, + -0.5, + ), + ( + { + "commodity": "electricity", + "coupling-coefficient": 0.3, + "production-capacity": "6 kW", + }, + -0.3, + ), + ] + for entry, expected_coefficient in ports: + loaded = DBStorageFlexModelSchema().load({**entry, "coupling": "chp"}) + assert _resolve_coupling_coefficient(loaded) == pytest.approx( + expected_coefficient + )