Balance internal commodity nodes (heat/steam networks) via first-class balance groups

documentation/changelog.rst

@@ -36,6 +36,7 @@ New features
 * In the UI, the flex-context editor supports editing commitments (name, baseline and deviation prices, each accepting a fixed value or a sensor), also within each commodity context; the commitment's commodity follows the commodity tab being edited, and new commitments start with zero-valued baseline and prices [see `PR #2287 <https://www.github.com/FlexMeasures/flexmeasures/pull/2287>`_]
 * A commitment in the flex-context now requires a ``baseline`` and at least one deviation price (``up-price`` and/or ``down-price``), as already documented; commitment costs are reported in the scheduling results under the user-given name (with a ``(custom)`` suffix in the rare case the name collides with a scheduler-internal commitment name) [see `PR #2287 <https://www.github.com/FlexMeasures/flexmeasures/pull/2287>`_]
 * Support commodity-converting devices, such as a CHP, e-boiler or heat pump, by describing each of the device's commodity ports as a flex-model entry sharing one ``coupling`` group with fixed flow ratios (``coupling-coefficient``) [see `PR #2218 <https://www.github.com/FlexMeasures/flexmeasures/pull/2218>`_]
+* Commodities without energy prices in the flex-context (e.g. a heat or steam network without a grid connection) are scheduled as internal nodes whose devices must balance each other at every time step [see `PR #2289 <https://www.github.com/FlexMeasures/flexmeasures/pull/2289>`_]
 * Commodity contexts that omit grid-connection fields (prices and site capacities) now get smart defaults instead of failing or silently leaving the grid unconstrained — for instance, a bare ``{"commodity": "gas"}`` is treated as having no grid connection; see :ref:`commodity_context_defaults` for the full rules [see `PR #2272 <https://www.github.com/FlexMeasures/flexmeasures/pull/2272>`_]
 * CLI support for adding/editing account attributes [see `PR #2242 <https://www.github.com/FlexMeasures/flexmeasures/pull/2242>`_]
 * Improve chart axis domain for event values not around zero, with a per-sub-chart ``y-axis`` option in ``sensors_to_show`` (default ``zero``, which pads the axis out to include zero) that can be set to ``data`` to fit a sub-chart's y-axis to the values shown, to an explicit ``[min, max]`` domain that the axis will cover at least (expanding to fit data beyond it), or to a strict ``{"min": min, "max": max}`` domain that the axis will never exceed (clamping data beyond it, with a warning when that happens), editable from the graph editor [see `PR #2244 <https://www.github.com/FlexMeasures/flexmeasures/pull/2244>`_]

documentation/features/scheduling.rst

@@ -41,6 +41,11 @@ The ``flex-context`` is independent of the type of flexible device that is optim
 With the flexibility context, we aim to describe the system in which the flexible assets operate, such as its physical and contractual limitations.
 For multi-commodity scheduling problems, the flex-context can be defined separately per commodity (e.g. electricity and gas). See :ref:`tut_multi_commodity` for a hands-on example.
 
+A *non-electricity* commodity that defines no energy prices and no capacity (grid-connection) fields in the flex-context (e.g. a heat or steam network without a grid connection) is treated as an internal node:
+its devices must balance each other at every time step, so everything produced into the node is consumed from it within the same time step.
+Electricity is the exception: it is always assumed to be grid-connected, so electricity without a price raises an error rather than becoming an internal node.
+Devices that convert between commodities (such as a CHP unit, gas boiler or electric heater) are described in the flex-model, one entry per commodity port, tied together by a ``coupling`` group. See :ref:`tut_converters` for a worked example flex-model.
+
 Fields can have fixed values, but some fields can also point to sensors, so they will always represent the dynamics of the asset's environment (as long as that sensor has current data).
 The full list of flex-context fields follows below.
 For more details on the possible formats for field values, see :ref:`variable_quantities`.
@@ -361,8 +366,15 @@ The sensor-referenced form is convenient when you pass the whole flex-model in o
 
 Here, the battery and PV installation may each individually schedule up to 2 kW, but their combined power flowing through the shared inverter is hard-limited to 2.5 kW.
 
+.. _inflexible_devices_in_flex_model:
+
+Inflexible devices in the flex-model
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
 Inflexible (measured) devices can be modelled in the flex-model too — for example, an unschedulable base load. To do so, model the inflexible device as its own asset and give its flex-model entry a single ``inflexible-consumption`` or ``inflexible-production`` reference to the sensor recording its power (the field name sets the sign convention, and source filters may be added). Such an entry carries no schedulable-device fields; it simply declares a fixed device whose power is accounted for. Like any device entry, it may set a ``commodity`` (defaulting to electricity), and its fixed power is then netted into that commodity's grid connection.
 
+There are two places to declare an inflexible device, and the choice is about *where* it belongs rather than *what* it does. Listing its sensor in the flex-context's ``inflexible-consumption``/``inflexible-production`` fields describes plain site base load, which is a property of the connection. Giving it its own flex-model entry describes a device that sits somewhere specific in the asset tree — under a particular inverter, feeder or commodity — which is a property of the device. Both net the same fixed power into the grid connection.
+
 The ``group`` field is optional on such an entry. Without it, the device is simply accounted for under the grid connection (just like listing its sensor in the flex-context's ``inflexible-consumption``/``inflexible-production`` fields, only declared on the asset instead). With it, the device *also* joins that group through the ordinary ``group`` field, exactly like a flexible member (the group's own flex-model entry, defining its capacities, must still be present), so that its fixed load or supply additionally counts towards the group's intermediate power constraint — for example, a base load sitting behind the same inverter or feeder as a battery.
 
 

documentation/tut/multi-commodity.rst

@@ -257,6 +257,47 @@ The commitment-cost result keeps these as separate entries — ``electricity net
 
 .. note:: This same pattern extends to more devices and more commodities. Add further entries to the ``flex-model`` list (each with its ``commodity``) and a matching entry in the ``flex-context`` ``commodities`` list. As long as all commodities share one currency, FlexMeasures optimises them together and reports each commodity's cost on its own.
 
+
+.. _tut_converters:
+
+Converters between commodities
+==============================
+
+A **converter** turns one commodity into another — a CHP unit (gas → electricity + steam), a gas boiler (gas → heat) or an electric heater (electricity → heat).
+In the ``flex-model`` a converter is not a single device but **one entry per commodity port**, tied together by a shared ``coupling`` name.
+The ``coupling-coefficient`` on each port fixes the conversion ratio relative to the shared coupling variable, so the ports always move together.
+
+For example, a CHP that turns gas into steam and electricity:
+
+.. code-block:: json
+
+    [
+      {"sensor": 6, "commodity": "gas",         "coupling": "chp", "coupling-coefficient": 1.0, "consumption-capacity": "20 kW"},
+      {"sensor": 7, "commodity": "steam",       "coupling": "chp", "coupling-coefficient": 0.5, "production-capacity": "10 kW"},
+      {"sensor": 8, "commodity": "electricity", "coupling": "chp", "coupling-coefficient": 0.3, "production-capacity": "6 kW"}
+    ]
+
+Here each kW of gas input produces 0.5 kW of steam and 0.3 kW of electricity.
+Each port gives exactly one directional capacity, and that is what marks its direction:
+the gas port only consumes, and the steam and electricity ports only produce.
+The opposite direction defaults to zero, so it does not need to be written out.
+
+Note that the capacities agree with the coefficients:
+20 kW of gas is the most the unit can burn, which is what caps steam at 10 kW and electricity at 6 kW.
+
+**Internal nodes.** A *non-electricity* commodity that lists no energy price and no capacity (grid-connection) field in the ``flex-context`` (e.g. a steam or heat network with no grid connection) is treated as an **internal node**: its devices must balance each other at every time step, so everything converters produce into the node must be consumed from it within the same step.
+You can still create a flex-context entry for such a commodity to define ``inflexible-consumption`` and/or ``inflexible-production`` (its fixed demand and supply) without FlexMeasures assuming a grid connection is available.
+Alternatively, omit it from the ``flex-context`` entirely (fixed demand and supply can also be set up with flex-models of their own, see :ref:`inflexible_devices_in_flex_model`).
+Electricity is the exception: it is always assumed to be grid-connected, so missing electricity prices raise an error rather than turning electricity into an internal node.
+
+Coupled converters and internal nodes compose, which is what lets these two features reach beyond the single unit shown above.
+Chain a few of them and you can describe an industrial site: converters feeding internal nodes, those nodes feeding further converters, and only the priced commodities meeting the grid.
+Nothing extra is needed to get there — no new fields, just more entries of the kinds already shown.
+We do not walk through such a site here, and yours will not look like anyone else's, but the pieces are the ones on this page.
+
+**Pricing what flows through a node.** An internal node's flows sum to zero by construction, so a commitment scoped to the node itself would bind nothing: there is no net flow to price.
+To put a cost on throughput — pipe wear, say, or a conversion levy — scope the commitment to the devices *producing* into the node, whose summed flow is what actually passes through it.
+
 We hope this demonstration helped to illustrate multi-commodity scheduling.
 To revisit scheduling several devices that share a single commodity and stock, head back to :ref:`tut_multi_feed_storage`.
 Next, in :ref:`tut_toy_schedule_process`, we'll turn to something different: the optimal timing of processes with fixed energy work and duration.

flexmeasures/data/models/planning/highspy_optimization.py

@@ -224,6 +224,7 @@ def device_scheduler_highspy(  # noqa C901
     ems_constraint_groups: list[list[int]] | None = None,
     device_power_bands: list[list[tuple[float, float]] | None] | None = None,
     coupling_groups: dict[str, list[tuple[int, float]]] | None = None,
+    balance_groups: dict[str, list[int]] | None = None,
 ) -> tuple[list[pd.Series], float, HighspySolverResults, HighspyModel]:
     """Direct HiGHS implementation of ``device_scheduler``.
 
@@ -255,6 +256,7 @@ def device_scheduler_highspy(  # noqa C901
         ems_constraint_groups=ems_constraint_groups,
         device_power_bands=device_power_bands,
         coupling_groups=coupling_groups,
+        balance_groups=balance_groups,
     )
 
     # Local aliases, so that the model below reads as it did before the (solver-agnostic)
@@ -272,6 +274,7 @@ def device_scheduler_highspy(  # noqa C901
     Md, Mc = problem.Md, problem.Mc
     band_lookup = problem.band_lookup
     coupling_device_specs = problem.coupling_device_specs
+    balance_group_specs = problem.balance_group_specs
     _initial_stock_of = problem.initial_stock_of
 
     # ---------------------------------------------------------------
@@ -649,6 +652,18 @@ def _add_commitment_rows(c, quantity, jj, lb, ub, devices, is_stock) -> None:
                 np.tile([1.0, -float(coeff)], (T, 1)),
             )
 
+    # node_balance_constraints: an internal commodity node stores nothing,
+    # so everything produced into it is consumed from it within the same time step,
+    # i.e. sum_d(ems_power[d, j]) == 0.
+    # Groups with no devices were dropped during preparation, so they add no row here,
+    # matching the Pyomo path's Constraint.Skip for that case.
+    for devices_in_node in balance_group_specs:
+        js = np.arange(T)
+        idx = np.column_stack([col_ems + int(d) * T + js for d in devices_in_node])
+        rows.add_uniform_rows(
+            np.zeros(T), np.zeros(T), idx, np.ones_like(idx, dtype=float)
+        )
+
     # ---------------------------------------------------------------
     # Build and solve the HiGHS model
     # ---------------------------------------------------------------

flexmeasures/data/models/planning/linear_optimization.py

@@ -134,6 +134,7 @@ def device_scheduler(  # noqa C901
     initial_stock: float | list[float] = 0,
     stock_groups: dict[int, list[int]] | None = None,
     coupling_groups: dict[str, list[tuple[int, float]]] | None = None,
+    balance_groups: dict[str, list[int]] | None = None,
     ems_constraint_groups: list[list[int]] | None = None,
     device_power_bands: list[list[tuple[float, float]] | None] | None = None,
 ) -> tuple[list[pd.Series], float, SolverResults, ConcreteModel]:
@@ -188,6 +189,16 @@ def device_scheduler(  # noqa C901
                                 binary variables (one per device per band per time step). Use None (per device
                                 or for the whole argument) for devices without band restrictions.
 
+    :param balance_groups:      Flow-balance constraints for internal commodity nodes (e.g. a heat or steam network without a grid connection).
+                                Each entry maps a node name to a list of device indices whose commodity-side flows must balance at every time step:
+                                ``sum_d(ems_power[d, j]) == 0``.
+                                In other words, everything produced into the node is consumed from it within the same time step;
+                                the node itself stores nothing.
+                                Derivative efficiencies and stock deltas describe each device's own stock-side conversion,
+                                and do not enter this commodity-side balance.
+                                To add storage to a node, include a storage device in the group:
+                                its flow absorbs the imbalance, and its stock is bounded by its own device constraints.
+
     Potentially deprecated arguments:
         commitment_quantities: amounts of flow specified in commitments (both previously ordered and newly requested)
             - e.g. in MW or boxes/h
@@ -237,6 +248,7 @@ def device_scheduler(  # noqa C901
         ems_constraint_groups=ems_constraint_groups,
         device_power_bands=device_power_bands,
         coupling_groups=coupling_groups,
+        balance_groups=balance_groups,
     )
 
     # Local aliases,
@@ -255,6 +267,7 @@ def device_scheduler(  # noqa C901
     band_lookup = problem.band_lookup
     _initial_stock_of = problem.initial_stock_of
     coupling_device_specs = problem.coupling_device_specs
+    balance_group_specs = problem.balance_group_specs
 
     # Add indices for devices (d), datetimes (j) and commitments (c)
     model.d = RangeSet(0, len(device_constraints) - 1, doc="Set of devices")
@@ -675,6 +688,27 @@ def flow_coupling_rule(m, c, j):
             model.coupling_device_range, model.j, rule=flow_coupling_rule
         )
 
+    if balance_group_specs:
+        model.balance_group_range = RangeSet(0, len(balance_group_specs) - 1)
+
+        def node_balance_rule(m, b, j):
+            """Balance the power flows of an internal commodity node at every time step.
+
+            Everything produced into the node must be consumed from it within the same time step.
+            The balance sums the devices' commodity-side flows (ems_power);
+            derivative efficiencies and stock deltas describe each device's own stock-side conversion (e.g. of a shared buffer),
+            and do not enter the commodity balance.
+            """
+            return (
+                0,
+                sum(m.ems_power[d, j] for d in balance_group_specs[b]),
+                0,
+            )
+
+        model.node_balance_constraints = Constraint(
+            model.balance_group_range, model.j, rule=node_balance_rule
+        )
+
     # Power bands (S2 operation modes): a banded device must operate within
     # exactly one of its declared signed power ranges at every time step.
     # Which band it runs in (per time step) is a free binary decision variable;

flexmeasures/data/models/planning/scheduling_problem.py

@@ -222,6 +222,9 @@ class SchedulingProblem:
     #: (group index, device index, coefficient) triples for hard flow-coupling constraints
     coupling_device_specs: list[tuple[int, int, float]]
 
+    #: device lists of the balance groups (internal commodity nodes), empty groups dropped
+    balance_group_specs: list[list[int]]
+
     initial_stock: float | list[float]
 
     #: The commitments as passed in, before the sub-commitment split.
@@ -272,6 +275,7 @@ def prepare_scheduling_problem(  # noqa C901
     ems_constraint_groups: list[list[int]] | None = None,
     device_power_bands: list[list[tuple[float, float]] | None] | None = None,
     coupling_groups: dict[str, list[tuple[int, float]]] | None = None,
+    balance_groups: dict[str, list[int]] | None = None,
 ) -> SchedulingProblem:
     """Normalise and validate ``device_scheduler``'s arguments into a SchedulingProblem.
 
@@ -369,6 +373,13 @@ def prepare_scheduling_problem(  # noqa C901
             for d_idx, coeff in members:
                 coupling_device_specs.append((g_idx, d_idx, coeff))
 
+    # Collect the device lists of the balance groups (internal commodity nodes).
+    balance_group_specs: list[list[int]] = []
+    if balance_groups:
+        balance_group_specs = [
+            list(devices) for devices in balance_groups.values() if devices
+        ]
+
     # Move commitments from old structure to new
     if commitments is None:
         commitments = []
@@ -546,6 +557,7 @@ def prepare_scheduling_problem(  # noqa C901
         Mc=Mc,
         band_lookup=band_lookup,
         coupling_device_specs=coupling_device_specs,
+        balance_group_specs=balance_group_specs,
         initial_stock=initial_stock,
         original_commitments=original_commitments,
     )