Support intermediate power constraints on groups of devices (flex-model group field)

[Flix6x]

Description

Adds support for intermediate power constraints: power limits on the aggregate flow of a group of devices, sitting between per-device power capacities and site-level (EMS) constraints. Closes #2092 (a hybrid inverter whose battery + PV children could jointly exceed the inverter's limit) and implements the flex-model-centric design from the discussion #1418 comment. Includes #2277.

Design

Device entries in a multi-device storage flex-model reference a group; the group's own flex-model entry constrains the group's aggregate power:

[
  {"sensor": 1, "power-capacity": "2 kW", "group": {"sensor": 5}},
  {"sensor": 2, "production-capacity": "2 kW", "consumption-capacity": "0 kW", "group": {"sensor": 5}},
  {"sensor": 5, "power-capacity": "2.5 kW"}
]

• power-capacity on the group entry → hard constraint (both directions).
• consumption-capacity / production-capacity → soft constraints, enforced via breach commitments with default breach prices (10000 currency/kW; no user-set group prices — consistent with the smart-defaults policy in Smarter commodity-context defaults and Commitment commodity field #2272).
• Groups can be referenced by sensor ({"sensor": <id>}, shown above) or by asset ({"asset": <id>}). The asset variant is what DB-stored flex-models naturally produce, making the feature fully definable via an asset tree: store partial flex-models on the assets and trigger the parent with an empty flex-model.
• Saving the group's schedule: sensor-referenced groups save the aggregate to the group power sensor; asset-referenced groups (which have no power sensor) save it via their consumption/production output sensors, following the standard conventions (consumption only → full profile consumption-positive; production only → full profile production-positive on save; both → clip-split into positive parts on each sensor). Either kind may define output sensors.
• Nested groups are supported (cycles rejected); group members must share a commodity; groups require multi-device flex-models.

Implementation

Built entirely on the existing device-group solver machinery — hard bounds via ems_constraints/ems_constraint_groups (multi-commodity work, #1946) and soft bounds via grouped FlowCommitments (#1934). No optimizer changes.

• Schema: new group field (sensor or asset reference, exactly one) on StorageFlexModelSchema and DBStorageFlexModelSchema; sensor variant validated to reference a power sensor.
• MetaStorageScheduler._prepare: group entries split from device entries (validations: dangling refs, device-only fields on group entries, mixed commodities, cycles, single-sensor mode) and per-group hard/soft constraints built mirroring the site-level capacity/breach patterns.
• UI: group added to the flex-model editor; the editor recommends the parent asset for a child's group field (one-click suggestion) and hints in both the parent's and children's editors when the parent's flex-model defines a power-capacity.
• Docs: new "Intermediate power constraints" section in the scheduling feature docs, plus a new toy-example tutorial driven entirely by DB-stored flex-models.

Bug fixes along the way

• Latent device-index misalignment in _prepare: per-device lists were built from the full flex-model list (including stock-only entries) instead of the filtered device models.
• AssetTriggerSchema.check_flex_model_sensors raised KeyError on asset-only flex-model entries.
• The freeze_server_now test fixture leaked its server_now monkeypatch into all subsequently run tests in the same process, causing order-dependent duplicate-key failures when scheduling jobs saved beliefs with identical frozen belief times.

Tests

• Solver-level (test_group_constraints.py, 18 tests): the [FlexMeasures/Scheduler] Investigate Missing Parent-Child Physical Constraint Propagation #2092 inverter case with a control run proving the constraint binds, soft directional bounds + breach commitment costs, output-sensor saving in all three modes, nested/mixed-kind groups, cycle detection, validation errors, and a pure-DB-tree end-to-end test (all flex-models stored as asset attributes, scheduling triggered with an empty flex-model).
• API-level: end-to-end trigger → job → beliefs on the group sensor equal the sum of member schedules; 422 for a group sensor outside the asset tree.
• UI tests for the editor hints; schema tests for both reference variants.
• Full serial regression (planning + schemas + UI + API schedule suites): 537 passed, 3 xfailed. Pre-commit and docs build clean.

🤖 Generated with Claude Code

https://claude.ai/code/session_01TAad46Ayg86DHpL6nY54sX

[read-the-docs-community]

Documentation build overview

📚 flexmeasures | 🛠️ Build #33649483 | 📁 Comparing 1ede35e against latest (1954f1f)

  🔍 Preview build  

51 files changed · + 6 added · ± 45 modified

+ Added

• _autodoc/GROUP.html
• _autosummary/flexmeasures.data.models.planning.devices.html
• _autosummary/flexmeasures.data.models.planning.soc_projection.html
• _autosummary/flexmeasures.data.schemas.scheduling.utils.html
• host/multi-tenancy.html
• tut/toy-example-group-constraints.html

± Modified

• changelog.html
• configuration.html
• genindex.html
• getting-started.html
• http-routingtable.html
• index.html
• py-modindex.html
• _autodoc/SOC_MAXIMA.html
• _autodoc/SOC_MINIMA.html
• _autodoc/SOC_TARGETS.html
• and 35 more...

[Flix6x]

Brought in line with origin/main, reconciling the group-field intermediate power constraints with the DeviceInventory/DeviceRegistry work (#2321/#2325): combined the devices.py imports (group_key_label + _resolve_stock_key) and kept both sides’ new DeviceInventory tests. All green locally — 18 test_device_inventory + 18 test_group_constraints + the factory/shared-stock scheduler tests.

Re the OpenAPI spec: regenerated it, so the version is back to 1.0.0 (it was a stale 0.33.2 from the branch) — that addresses the openapi-specs.json:10 comment.

[Copilot]

Pull request overview

This PR adds support for intermediate power constraints by introducing a group field in storage flex-model entries, enabling hard/soft aggregate power limits over groups of devices (e.g., hybrid inverter bottlenecks) without changing the optimizer, and wiring the feature through schema validation, scheduling logic, UI hints/editor behavior, tests, and documentation.

Changes:

• Add group (sensor- or asset-referenced) to storage flex-model schemas and OpenAPI/metadata, including validation logic.
• Extend the StorageScheduler pipeline to classify group entries, enforce group hard/soft constraints, and save group aggregate schedules (including via output sensors for asset-referenced groups).
• Add UI hints + extensive test coverage and documentation/tutorial updates for DB-driven asset-tree configurations.

Reviewed changes

Copilot reviewed 20 out of 20 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
flexmeasures/ui/views/assets/views.py	Passes parent/child + power-capacity presence flags to the asset properties template for UI hints
flexmeasures/ui/tests/test_asset_crud.py	Adds UI test coverage for the new group-field hint behavior
flexmeasures/ui/templates/assets/asset_properties.html	Adds hint banners and editor-side “use parent as group” behavior in the flex-model editor
flexmeasures/ui/static/openapi-specs.json	Updates generated OpenAPI specs to document the new group field/schema
flexmeasures/data/schemas/tests/test_scheduling.py	Adds schema tests for group sensor/asset variants and power-sensor validation
flexmeasures/data/schemas/scheduling/storage.py	Introduces GroupReferenceSchema and validates group references (power-sensor check, exactly-one-of sensor/asset)
flexmeasures/data/schemas/scheduling/metadata.py	Adds GROUP metadata used for docs/OpenAPI/UI schema rendering
flexmeasures/data/schemas/scheduling/init.py	Adds UI schema-spec entry for group and fixes sensor-checking for asset-only flex-model entries
flexmeasures/data/models/planning/tests/test_group_constraints.py	New planning-level test module covering hard/soft group constraints, nesting, output saving, and DB-tree E2E
flexmeasures/data/models/planning/tests/test_device_inventory.py	Adds tests for classifying group entries, resolving leaf membership, and cycle detection
flexmeasures/data/models/planning/storage.py	Implements group constraint enforcement and group schedule output handling in the scheduler
flexmeasures/data/models/planning/devices.py	Extends DeviceInventory to classify group entries and resolve transitive group membership
flexmeasures/conftest.py	Fixes freeze_server_now fixture leakage by restoring patched imports and server_now functions
flexmeasures/api/v3_0/tests/test_asset_schedules_fresh_db.py	Adds API-level fresh-db tests for group constraint enforcement and tree membership validation
documentation/tut/toy-example-group-constraints.rst	New tutorial showing a fully DB-driven asset-tree setup using group constraints
documentation/index.rst	Adds the new tutorial to the documentation index
documentation/features/scheduling.rst	Documents group field semantics and intermediate power constraints
documentation/concepts/commitments.rst	Updates commitments docs to reflect current group-related capability and roadmap
documentation/changelog.rst	Adds changelog entries for intermediate group constraints and asset-referenced groups
documentation/api/change_log.rst	Adds API changelog entries describing the new group field behavior

[Flix6x]

Addressed the three review findings (commit 7686b39):

1. XSS (asset_properties.html) — the parent asset name is user-controlled and was interpolated into innerHTML; now HTML-escaped (via a textContent DOM helper) before insertion in the group hint and button label.
2. Error path (storage.py) — the "group must reference a power sensor" ValidationError is now attached to group instead of sensor.
3. GroupReferenceSchema base — now inherits from SharedSensorReferenceSchema instead of SensorReferenceSchema, so it accepts only sensor/asset and no longer carries the source-* belief-query filters or the "variable quantity" description. Regenerated openapi-specs.json to match.

All green locally: 130 tests across test_group_constraints + schema test_scheduling, plus flake8/black/mypy and the OpenAPI-generation hook.

[saerts-gp]

Thanks for building this. #2092 describes our situation exactly: a hybrid inverter whose battery and PV children can
jointly exceed the inverter's rating.

We solve it downstream today, on 0.32.0, with a StorageScheduler subclass that walks the
asset tree and constrains the summed power of an intermediate asset's flexible descendants.
It works, but it reaches into solver internals, so we would much rather retire it in favour
of group. We've put that migration on our roadmap for the autumn.

Before we plan it, we'd like to check that we've understood the feature correctly. Below is
our reading — we'd be grateful if you could correct anything that's wrong, since we'd rather
find out now than during the migration.

Our understanding

1. Group membership is explicit, not inferred from the asset tree: a device joins a group
   only by carrying a group field, and the entry that is the group is the one whose own
   sensor or asset matches what the members reference. Being a child of an asset in the
   tree does not by itself make a device a member of that asset's group.
2. There are two ways to reference a group. {"sensor": <id>} saves the group's aggregate
   power to that power sensor. {"asset": <id>} describes a group with no power sensor of
   its own, and the aggregate is saved via that entry's consumption / production output
   sensors instead.
3. On the group's own entry, power-capacity is a hard limit in both directions, while
   consumption-capacity / production-capacity are soft, enforced through breach
   commitments at a fixed default breach price that the user does not set.
4. Deeper hierarchies are expressed by a group entry itself carrying a group field, and
   membership resolves transitively down to the leaf devices, with cycles rejected.
5. Groups constrain devices described in the flex-model. Inflexible devices coming from the
   flex-context cannot be group members, so an intermediate limit does not account for
   measured load sitting behind the same node.
6. Groups need a multi-device flex-model, members must share a commodity, a group entry must
   define at least one of the three capacities, and it must not carry device-level fields
   such as state-of-charge or efficiencies.

How we would map our sites onto it

Our tree is site → point of common coupling → distribution board → inverter → {battery, PV,
PV}, where the PCC, the board and the inverter each have their own rating. Our reading of
#2277 is that we express this by storing, on each asset's own flex-model, a
group: {"asset": <parent asset id>} — battery and PV pointing at the inverter, the inverter
pointing at the board, the board pointing at the PCC — and then trigger scheduling on the
site with an empty flex-model, since all of it already lives on the asset tree. Is that the
intended way to express a multi-level hierarchy, or would you model it differently?

One question that isn't about the code

Do you have a sense of which release will carry this, and roughly when? We can see it's on
main but not in v0.33.0, v0.33.1 or v1.0.0.dev0. Our reading is that a backport to the
0.32/0.33 line isn't realistic, since the feature builds on the DeviceInventory work
(#2321/#2325) and ems_constraint_groups (#1946) — is that right? It mainly decides whether
we do one upgrade or two, and we're happy to wait for whatever you consider the stable path.

Once we've confirmed the above we'll have a couple of follow-ups about our specific asset
data, but they only make sense if we've got the fundamentals right. Happy to test the feature
against real site data once it's in a release we can install — we have historical schedules
from exactly the #2092 configuration that violate the inverter rating, which should make a
reasonable before/after check.

[Flix6x]

• I believe 1-6 are all correct. I'll double check and follow up with another comment.
• 5 does feel like an oversight, so this may warrant an issue. Without it, there's no way to model an inflexible device within a group. Which would seem like a quite common case.
• 1 is work in progress: inferring group structure from asset trees. We encountered some complications, and I consider it unlikely to ship in v1.
• Your proposed site model appears to use groups correctly. If you encounter issues with it, please post the flex-model and flex-context (from the scheduling job) in an issue, preferably with constant values rather than sensor references, so it can be more easily reproduced. Or a self-contained script, or even in the form of a pytest, which would make it easiest to bring into the repo to make sure your case won't see any regression.
• The intermediate power constraints are meant for v1.0.0 (we keep track of this using the PR's milestone).