Skip to content

Add an advisory agent hook for mid-phrase line breaks - #2386

Merged
Flix6x merged 1 commit into
mainfrom
chore/agent-reflow-hook
Aug 3, 2026
Merged

Add an advisory agent hook for mid-phrase line breaks#2386
Flix6x merged 1 commit into
mainfrom
chore/agent-reflow-hook

Conversation

@Flix6x

@Flix6x Flix6x commented Aug 3, 2026

Copy link
Copy Markdown
Member

Description

The docstring convention — break lines only after punctuation — has been missed in three PRs in a row (#2364, #2218, #2385), each time caught by a reviewer rather than the author. #2384 documented the rule harder, and the very next PR broke it. Documentation is evidently not the missing piece.

  • .claude/hooks/reflow_check.py — the check
  • .claude/hooks/reflow-check.sh + wiring in .claude/settings.json, as a PreToolUse hook on git commit

Two deliberate choices

It is an agent hook, not a pre-commit hook. The misses have been an agent's, and contributors shouldn't pay a false-positive tax for that. It's also the only form that works: the check is a heuristic, and a heuristic gate that a human hits on embedded OpenAPI YAML gets disabled. An agent can read "line 2028 may break mid-phrase", look at the line, and judge — the same way worktree-guard.sh explains itself rather than just failing.

It is advisory and never blocks. Exits 0 always, and inspects only the lines being added. A legacy file is therefore not a wall — the repo has ~4350 candidate hits across 72% of files, and reflowing unrelated prose is not something a commit should be doing.

On the false-positive rate

Docstrings here carry OpenAPI YAML, shell commands and bullet lists — none of it prose, none of it ending in punctuation. Suppressing those cuts the noise where it matters:

file before suppression after
api/v3_0/sensors.py 494 59
api/v3_0/assets.py 385 15
data/models/planning/storage.py 199 173

storage.py stays high because it is genuinely prose-heavy — that's the honest number rather than a tuned one. Since the hook only looks at added lines, the standing count matters little; what matters is that a new mid-phrase break gets surfaced.

How to test

Proven capable of failing (per #2384):

# staged docstring broken mid-phrase -> reported
Possible mid-phrase line breaks in docstrings or comments:
  probe_demo.py:4: This sentence breaks in the middle of a

# correctly reflowed -> silent

It caught a real one while I was writing #2385: a comment of mine ending "…so running each test twice in one".

Further improvements

  • If it proves accurate enough in practice, a blocking variant scoped to flexmeasures/data/models/planning/** could follow. Better to run it advisory first.

Related items

The docstring convention -- break lines only after punctuation -- has been
missed in three PRs in a row (#2364, #2218, #2385), each time caught by a
reviewer rather than by the author. #2384 documented the rule harder, and the
very next PR broke it, so documentation is evidently not the missing piece.

Adds .claude/hooks/reflow_check.py and wires it as a PreToolUse hook on
`git commit`, next to the existing pre-commit and worktree-guard hooks.

Two deliberate choices:

It is an agent hook, not a pre-commit hook. The misses have been an agent's,
and contributors should not pay a false-positive tax for that. It is also the
only form that works: the check is a heuristic, and a heuristic gate that a
human hits on embedded OpenAPI YAML gets disabled, whereas an agent can read
"line 2028 may break mid-phrase", look, and judge.

It is advisory and never blocks. It exits 0 always, and only inspects lines
being added, so a legacy file is not a wall -- the repo has ~4350 candidate
hits, and reflowing unrelated prose is not a thing a commit should do.

Suppressing non-prose (bullets, RST directives, doctests, embedded YAML and
JSON, shell continuations) cuts the noise substantially where docstrings carry
API specs: sensors.py 494 hits to 59, assets.py 385 to 15. Files that are
genuinely prose-heavy stay high (storage.py 199 to 173), which is the honest
answer rather than a tuned one.

Self-tested both directions: a docstring broken mid-phrase is reported, a
correctly reflowed one is silent.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X2jLjsQzwztDc7Nxz1ozmQ
Signed-off-by: F.N. Claessen <claessen@seita.nl>
@read-the-docs-community

Copy link
Copy Markdown

Documentation build overview

📚 flexmeasures | 🛠️ Build #33897286 | 📁 Comparing 961e245 against latest (b33ec32)

  🔍 Preview build  

1 file changed
± api/v3_0.html

@Flix6x Flix6x added the AI Infra label Aug 3, 2026
@Flix6x Flix6x self-assigned this Aug 3, 2026
@Flix6x Flix6x added this to the 1.0.0 milestone Aug 3, 2026
@Flix6x
Flix6x merged commit 7480666 into main Aug 3, 2026
13 checks passed
@Flix6x
Flix6x deleted the chore/agent-reflow-hook branch August 3, 2026 23:04
Flix6x added a commit that referenced this pull request Aug 4, 2026
…-node log

Addresses part of the review:

- Removes .git-exclude, which held a lone "conftest.py" and reached this branch
  through the 2026-08-02 merge from feat/chp. It is on no other branch and has
  no business in the tree.
- Applies both tutorial suggestions verbatim, and replaces the dangling "This
  is how a whole factory is scheduled" with what it referred to (internal nodes
  and coupled converters together).
- Points the new cross-reference at flex_models_and_schedulers. My first
  attempt invented a label that does not exist, which would have failed the
  docs build.
- Drops the internal-node message from info to debug. Treating an unpriced
  commodity as an internal node is the normal path, not an event worth a line
  per schedule.

Still outstanding from the same review, and deliberately not rushed here:
reflowing this PR's docstrings and comments (the checker from #2386 reports 52
mid-phrase breaks in lines this PR adds), and re-checking the tests against the
mutation-test policy from #2384.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X2jLjsQzwztDc7Nxz1ozmQ
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Flix6x added a commit that referenced this pull request Aug 4, 2026
Breaks every line after punctuation, never mid-phrase, per the convention in
.github/instructions/docstrings.instructions.md.

Done by hand across all 48 prose sites the #2386 checker reported in lines this
PR adds, in linear_optimization.py, storage.py, highspy_optimization.py,
schemas/scheduling/__init__.py and the three test modules.

An automated re-wrap was tried first and reverted. It broke two files outright,
collapsing a closing docstring quote onto the following code line, and where it
did parse it left lines dangling on "In other words," and "To add storage to a
node," -- which satisfies "ends in punctuation" while splitting the clause the
rule exists to protect. The check generalises; the fix does not.

Four hits remain and are deliberate: the ASCII topology diagram in
test_factory_chp_dispatch_through_storage_scheduler, which sits in a literal
block and is not prose.

488 passed, 3 xfailed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X2jLjsQzwztDc7Nxz1ozmQ
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Flix6x added a commit that referenced this pull request Aug 4, 2026
…s balance groups (#2289)

* feat: add gas-price field to the Flex-context schema

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* apply black

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* feat: add a test case for two flexible devices with commodity

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* use expected datatypes

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* feat: split commitments per commodity

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* feat: split commitments per commodity

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* Revert "use expected datatypes"

This reverts commit b22c6d7.

* feat: add a test case for different commodities

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: do not produce gas

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: add stock-id field in Storage and DB flex model schemas

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* feat: build stock groups

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* feat: get stock groups

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* feat: add a test case for multi feed stock

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* feat: create a flow commitment for prefering to charge sooner devices

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* add soc constraints for boiler

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* add some assert statments

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* update and add new assertions with clear explanation

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* update the docstring

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* feat: add support for shared storage

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* remove the breakpoint

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* feat: update the test case for two devices with shared stock

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* feat: add assertions with clear reasons

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* refactor: move tiny-price-slope decleration out of the for loop

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* Revert "refactor: move tiny-price-slope decleration out of the for loop"

This reverts commit 2becd02.

* refactor: move tiny-price-slope decleration out of the for loop

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: add data_key attr

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* add missing commodity description and it's field in ui flexmodel schema

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: add missing gas-price field in UI Flexcontext schema

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* Add support for multi-device charging of shared storage

Introduce stock_groups mapping to link multiple devices to a shared SOC.

Aggregate stock delta across devices sharing the same battery.

Update stock change calculation to use combined device flows.

Add device-to-group and group-to-devices lookup for efficient shared stock computation.

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: wrong timezone; the test relied on the preference to charge sooner and discharge later, rather than on the EPEX price transition, as the inline test documentation advertised

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: move preference to charge sooner and discharge later into a StockCommitment to prefer being full

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: test case no longer relies on arbitrage opportunity coming from artificial price slope

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: check for optimal schedule

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: prefer a full storage earlier over later

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: update commitment name and inline comments

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: touch up test explanation

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: update test case given preference for a full battery

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* delete: clean up comment

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: model the preference to curtail later within the same StockCommitment, using a tiny price slope to prefer a fuller SoC sooner rather than later, by lowering penalties later

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: reduce tiny price slope

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: delete duplicate changelog entry

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: fix broken link

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* Revert "fix: reduce tiny price slope"

This reverts commit bf16e63.

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: soc unit conversion

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: adapt test to check for 1 hour of free energy at 15-min scheduling resolution

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* style: black

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: check curtailment preference per distinct device

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: set tight tolerance for HiGHS solver

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* refactor: merge if-blocks

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: update test_two_flexible_assets_with_commodity

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: check curtailment preference per distinct device

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: set tight tolerance for HiGHS solver

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* refactor: merge if-blocks

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: use iloc

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: diminish tiny price slope by number of planning steps

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* refactor: always diminish tiny price slope by number of planning steps, such that its relative weight does not grow with the number of steps

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* chore: increment StorageScheduler version

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: sum all devices soc contribution, and use individual device efficiencies

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* update test case for multi feed stock

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* expect to charge the battery early to see the effect of fully discharge

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: update the assert statements according to the scheduler results

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: use approximation to compare battery and heat pump costs

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* update the assert statements with prefered to charge battery sooner than later

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* dev: first step in resolving merge conflicts

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* chore: code annotation

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: not all flex-models have sensors

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: update the expected ev and battery costs

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: add device id to get costs for the given device

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: static method has no self

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* delete: remove inapplicable fields for stock model

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: fix interpretation of test results

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: move initialization of ems_constraints

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: resolve merge conflicts on _build_soc_schedule, copied from Ahmad

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: remove redundant code block

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* dev: use "state-of-charge" key instead of "sensor" key for stock models

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: skip StockCommitment for device models that outsource their stock model to a separately modeled device

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: old flex models that describe a device that serves both as a feeder and stock are both categorized as device models and stock models

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: model stock devices using the state-of-charge field instead of the sensor field

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: identify asset to merge with db flex-model

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: validation

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: flex-model setup in test

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: create stock group

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* use soc-sensor in case of missing power sensor and also correct stock groups

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: create stock model for a model which has itself stock

* update the assert statements

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* remove stock-id field

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: correct the stock groups

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* refactor: remove unneccessary test function

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: shared soc-gain, soc-usage, soc-minima and soc-maxima

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: shared StockCommitment for preferring a full SoC

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* dev: todo

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* dev: add "test" test case

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: rewrite test to not rely on multi-commodity

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: rewrite test to prove that only when both HPs share a single commitment does the optimizer treat their stocks as a combined resource

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: do not coerce device_group into a time series

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: changelog entry

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: pretty_print is not specifically for FlowCommitments

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: add (back) inline dev notes

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: strengthen asserts

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: add check for exact electricity costs expected

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: sum over electricity costs; and move to StockCommitment to model preference for full soc sooner

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: replace vague assert with explicit asserts

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: lose confusing comment

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: backwards compatibility in case no device is specified;

The two changes together are the backwards-compatible default that was missing: when no device is specified (device=None), to_frame() must emit NaN in the device_group column (not crash), so the optimizer routes the commitment through ems_flow_commitment_equalities exactly as it did before device_group was introduced.

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: when device is present but device_group is absent, fall back to the pre-existing behaviour — each device is its own group

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* dev: update expectations of how costs are shared between EV and battery

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix commodity-level commitments by grouping devices and aligning device series with scheduler index

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix commodity-level commitments by grouping devices and aligning device series with scheduler index

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* update the test cases for net commodity consumption and production

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* dev: Support commodity-specific prices and site capacities in storage scheduler

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* dev: Add commodity-specific flex-context schema

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* dev: Add dynamic commodity prices and split flex-context settings to capacity scheduling test

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* feat: create a shared schema for flex-context and commodity-context

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* update the test case to have inflexible-devices-sensors for each commodity-flex-context

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* refactor: loop over flex-context fields and choose all fields except 'gas-price' for electricity as commodity

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: add inflexible-device-sensors to the gas commodity model

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: use net energy costs instead of individual device costs

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: comment out the buggy lines

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: remove self

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* feat: coupling groups for CHP

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: test factory model

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: merge conflicts

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* refactor: make variables for gas boiler and e-heater capacities

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: clarify e-heater efficiency assumption

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: add scenario with merit order: gas boiler ≪ e-heater ≪ CHP

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: support flex-model coupling constraint in StorageScheduler

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: clarify calculation of coupling coefficients

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: invert interpretation of coefficients to better match thermal and electrical efficiencies

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: support multiple inputs to coupling point

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: stop collapsing the heat buffer and steam node in the factory test

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* tests/planning: align storage CHP coupling test with current coefficient validation

Context:\n- test_storage_scheduler_chp_coupling failed because positive coefficients summed to 1.5 while current scheduler validation requires 1.0\n\nChange:\n- adjusted the storage CHP test coefficients and expectations to satisfy current validation semantics\n- kept the test focused on verifying coupled gas/heat/power behavior

* planning/coupling: infer internal sign from directional capacities

Context:\n- Coupling coefficients in flex-models were user-facing signed values, which was error-prone and not user-friendly\n\nChange:\n- treat flex-model coupling-coefficient as a positive magnitude\n- infer internal sign from capacities (consumption-capacity=0 -> output/negative, production-capacity=0 -> input/positive)\n- remove strict positive-sum validation in scheduler coupling-group construction\n- update storage CHP coupling test and schema/openapi documentation to reflect positive-only coefficient input

* tests/planning: clarify signed internal CHP coefficients in storage docstring

Context:\n- The storage CHP test docstring should distinguish user-facing positive flex-model coefficients from the signed internal coefficients\n\nChange:\n- documented that the flex-model uses positive magnitudes\n- explicitly stated the intended internal coefficients: 1.0, -0.5, -0.3

* fix: update the test cases according device level costs

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* docs/scheduling: add COMMODITY and GAS_PRICE metadata field documentation
Context:
- Test test_all_metadata_fields_are_documented was failing because these fields were not documented
- Part of multi-commodity feature development
Change:
- Added ``commodity`` field to the storage flex-model table
- Added ``gas-price`` field to the flex-context table

* fix: add device-model in groups if it's missing

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: restore SOC constraints and state-of-charge handling broken by multi-feed-stock refactor

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: fall back to deprecated price fields

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: typo

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: store commitment costs on job meta

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* refactor: clarify which job is which

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: update test expectation: the battery could save more?

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* dev: add todo

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: price window should match scheduling window

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: comment out unreasoned check

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: update test expectation; apparently the battery could save more?

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* dev: add todo

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* chore: flake8

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* dev: exclude commodities field from flex-context schema referencing a dedicated issue

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: only save commitment costs on job if we have a job to save it on

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: inflexible devices are electricity devices by default

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* delete: no more need for backwards-compatibility of the temporary gas-price field (only used during development)

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* chore: black

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* chore: black

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: optional dict key

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: keep ems-constraints and fix the test cases (#2233)

* fix: keep ems-constraints and fix the test cases

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* Update flexmeasures/data/models/planning/storage.py

Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com>

* fix: update the comment and raise value error if ems_constraints_group is not passed

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

---------

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com>
Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>

* fix: keep ems-constraints and fix the test cases (#2233)

* fix: keep ems-constraints and fix the test cases

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* Update flexmeasures/data/models/planning/storage.py

Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com>

* fix: update the comment and raise value error if ems_constraints_group is not passed

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

---------

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com>
Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: only raise in case of multiple EMS constraint DataFrames

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* chore: the wait for marshmallow-code/apispec#999 is over

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: allow any commodity, with electricity and gas serving as examples

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* delete: remove unreleased flex-context field for gas price

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: add all relaxation fields to the list of fields to ignore when moving old flex-context fields into the electricity commodity context

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* delete: gas_price is no longer a field (remove reference to unreleased field)

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* delete: just treat the whole old flex-context as the electricity flex-context

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* chore: update openapi-specs.json

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: list the commodity field first rather than last

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: commodity is a field in both flex-model and flex-context

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: flex-model commodity can also be more than just electricity and gas

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: remove mention of gas-price field

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: adjust scheduling section for multi-commodity

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: adjust field descriptions for multi-commodity

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: add type annotation

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: keep track of inflexible device sensors per commodity, too

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* refactor: place all group args at the end

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* dev: add todos for checking prices

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: test should exclude COMMODITY_FLEX_CONTEXT and COMMODITY_FLEX_MODEL, which are named commodity in scheduling.rst

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: require an unambiguous flow direction for coupled devices

The sign of a coupling coefficient is inferred from directional capacities,
and a device with both directions open (or blocked) was silently treated as
an input. Reject such flex-models with a validation error instead.

Also promote the coupling field descriptions to MetaData constants and
document both fields in the storage flex-model table.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016MLCUiSdXDqDBmg8GbYp1B

* feat: balance internal commodity nodes with first-class balance groups

Adds a balance_groups argument to device_scheduler: each group lists the
devices of an internal commodity node (e.g. a heat or steam network without
a grid connection) whose stock-side flows must sum to zero at every time
step. This replaces the reference-device min=max=0 stock-group workaround
used by the factory scenario, which is now tested in both modes.

The StorageScheduler derives balance groups from the flex-config: a
non-electricity commodity without energy prices becomes an internal node
(previously this raised 'Missing consumption price'). Together with
coupling groups (one flex-model entry per converter port), this makes the
factory scenario (CHP + gas boiler + e-heater meeting a fixed steam demand)
schedulable end-to-end through StorageScheduler.compute().

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016MLCUiSdXDqDBmg8GbYp1B

* docs: point balance-groups changelog entry at PR #2279

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016MLCUiSdXDqDBmg8GbYp1B

* fix: balance internal commodity nodes on power flows, not stock-side terms

A device can sit in both a commodity balance group (via its commodity) and
a shared-stock group (via its state-of-charge sensor), e.g. a steamer that
discharges a heat buffer to produce steam. Its derivative efficiencies and
stock delta (e.g. the buffer's soc-usage losses assigned to it) describe
the stock-side conversion and must not leak into the commodity balance:
what crosses the node is the device's power flow (ems_power).

Found while running a realistic factory scenario, where the heat buffer's
soc-usage drain was distorting the steam balance.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016MLCUiSdXDqDBmg8GbYp1B

* docs: point the balance-groups changelog entry at PR #2289

PR #2279 was closed in favour of #2289, which carries the feature now.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015qxM7UZ5wHTz3ftz1Mf9yy
Signed-off-by: F.N. Claessen <felix@seita.nl>

* fix: preserve overlapping stock-group membership for commodity converters

Merging main's group-indexed stock recursion (#2282/#2325) rebuilt
group_to_devices from a single-valued device_to_group (last assignment
wins), which cannot represent a converter device that belongs to more
than one stock group (e.g. a steamer bridging a heat node and a steam
node). That silently dropped such devices from all but their last group,
orphaning coupled outputs (the CHP dispatched to zero in the cheap-gas
merit-order scenario).

Keep main's namespaced group keys (the stock-id/device-index collision
fix) but build group_to_devices directly from the declared stock groups,
preserving full overlapping membership; device_to_group still records a
single primary group for per-device stock bounds.

Restores test_factory_chp_dispatch (all three merit-order scenarios) on
top of the new recursive stock formulation.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016MLCUiSdXDqDBmg8GbYp1B

* fix: keep internal-node detection working alongside smart commodity defaults

After merging main's #2272 smart commodity-context defaults, a price-free
commodity context (e.g. a bare {"commodity": "steam"}) gets a
smart-defaulted zero consumption-price, which defeated #2289's
priceless-commodity internal-node detection: the steam balance group
vanished and the CHP dispatched incorrectly through the StorageScheduler.

Record durably on each commodity context whether any price field was
user-given (prices_are_defaulted) and treat a context whose prices were
all defaulted (and which uses no price sensors) as an internal node.
Ports the combo-branch fix upstream now that #2272 has landed on main.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016MLCUiSdXDqDBmg8GbYp1B

* style: fix flake8 F811 and black formatting in test_storage.py

Remove a duplicate `GenericAsset` import (F811, a merge artifact) and add
the two blank lines black expects before
test_off_tick_soc_relaxation_covers_all_devices_of_a_shared_stock. Fixes the
failing pre-commit Check on this PR.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016MLCUiSdXDqDBmg8GbYp1B
Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: add a converter + internal-node example flex-model

Address review: the converter/coupling paragraph in scheduling.rst now
cross-references a worked example. Adds a "Converters between commodities"
section to the multi-commodity tutorial showing a CHP described as one
entry per commodity port tied by a coupling group, plus how an unpriced
commodity becomes an internal balance node.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016MLCUiSdXDqDBmg8GbYp1B
Signed-off-by: F.N. Claessen <claessen@seita.nl>

* style: drop duplicate GenericAsset import in test_storage.py (F811)

The main-merge re-introduced a duplicate `GenericAsset` import (already
imported alongside GenericAssetType); flake8 F811. Remove it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016MLCUiSdXDqDBmg8GbYp1B
Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix(schema): support coupling in db-stored flex-models and reject blank coupling names

Address two review findings on the CHP coupling work:

1. `DBStorageFlexModelSchema` (used to validate persisted flex-models, e.g.
   in patch_asset) did not declare `coupling`/`coupling-coefficient`, so
   storing a db flex-model containing `coupling` failed with an unknown-field
   ValidationError. Add both fields to that schema, mirroring how they are
   declared on `StorageFlexModelSchema`.

2. A provided `coupling` name could be an empty/whitespace-only string, which
   would become a coupling-group key and silently couple unrelated devices
   under an empty group. Reject blank/whitespace-only names via a shared
   `_validate_coupling_name` helper wired into a `@validates("coupling")` on
   both schemas.

Add tests: a db-stored flex-model with `coupling` validates and round-trips,
and blank `coupling` values (on both schemas) raise a ValidationError.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016MLCUiSdXDqDBmg8GbYp1B
Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix(scheduling): treat capacity-only commodity as grid-connected, not internal node

Address four review findings:

1. A commodity that declares a grid connection via capacity fields only
   (site-power/consumption/production-capacity) but no explicit price was
   wrongly flagged as having defaulted prices, so the scheduler misclassified
   it as an internal node (EMS constraints skipped, per-step balance forced).
   The internal-node determination now considers capacity fields too: a
   commodity is an internal node only when the user gave neither prices nor any
   capacity/grid-connection signal. Renamed the durable flag from
   prices_are_defaulted to is_internal_node accordingly, and added a test.
2. Fixed the balance_groups docstring in linear_optimization.py to describe the
   actual commodity-side sum(ems_power)==0 balance (not a stock-side flow one).
3. + 4. Docs (scheduling.rst, multi-commodity.rst): clarified that electricity
   is always assumed grid-connected, so missing electricity prices raise an
   error rather than turning electricity into an internal node.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016MLCUiSdXDqDBmg8GbYp1B
Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix(ui): add coupling fields to UI_FLEX_MODEL_SCHEMA

DBStorageFlexModelSchema gained coupling/coupling-coefficient, but the UI
flex-model schema did not, breaking test_ui_flexmodel_schema (which enforces
parity between the two). Add the matching UI entries.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016MLCUiSdXDqDBmg8GbYp1B
Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat(scheduling): smart-default coupled device flow direction

Infer a coupled (commodity-converting) device's flow direction from which
directional capacity is given, defaulting the unspecified opposite direction
to zero, mirroring how a missing directional site capacity defaults to zero.
An input port now needs only a consumption-capacity, an output port only a
production-capacity; setting the opposite direction to a fixed 0 remains valid
for back-compat. Only genuinely ambiguous cases (both directions flow, or
neither) are rejected.

Also move the #2218 coupling changelog entry onto this branch.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016MLCUiSdXDqDBmg8GbYp1B
Signed-off-by: F.N. Claessen <claessen@seita.nl>

* style: reflow coupling docstrings to break only after punctuation

CLAUDE.md asks that docstrings and comments break lines only after punctuation,
never mid-phrase, so review comments and text search stay stable. Copilot
flagged ten places in this PR where the coupling work did not follow it.

Reflowed the direction-inference docstrings in devices.py and storage.py, the
"->" bullet list, and the four coupling schema tests, whose docstrings also
repeated the test name in their first line where a sentence belongs.

Text only. 446 passed, 3 xfailed across the planning and scheduling-schema
suites.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X2jLjsQzwztDc7Nxz1ozmQ
Signed-off-by: F.N. Claessen <claessen@seita.nl>

* review: use timezone-aware datetimes in the coupling schema tests

The four StorageFlexModelSchema(start=datetime(2026, 6, 1)) calls built naive
datetimes, against the repo's timezone-awareness convention, and one of them
predates this PR. The file already imports pytz.

Also reflows the _validate_coupling_name docstring, missed in the previous pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X2jLjsQzwztDc7Nxz1ozmQ
Signed-off-by: F.N. Claessen <claessen@seita.nl>

* Accept arbitrary commodities in a stored flex-model

Internal commodity nodes are the point of this PR, and their labels are open --
"steam", "heat", whatever the site calls the node. But DBStorageFlexModelSchema
pinned commodity to OneOf(["electricity", "gas"]), while the API-facing schema
only required a non-empty string. So a converter feeding an internal node could
be scheduled and not stored: triggering worked, persisting the same flex-model
on an asset was rejected.

Found by validating the tutorial's own CHP example against the schema, which
failed on its steam port.

Drops the enumeration and applies the same non-empty check both schemas now
share, so a blank commodity is still rejected. Also removes ALLOWED_COMMODITIES,
which nothing referenced and which duplicated the restriction being lifted.

Simplifies the tutorial example while there: one directional capacity per port
rather than a power-capacity plus an explicit zero, now that the direction is
inferred from whichever capacity is given (#2218). Its magnitudes were also
misleading -- 20 kW of gas caps steam at 10 kW and electricity at 6 kW, where
both were written as 1 MW.

Tests: the tutorial example validates and each port's coupling direction
resolves; an internal commodity is accepted; a blank one is rejected on both
schemas. Checked that the first two fail when the OneOf is put back.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X2jLjsQzwztDc7Nxz1ozmQ
Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: use inflexible-consumption in the internal-node guidance

The multi-commodity tutorial still pointed readers at
inflexible-device-sensors, which #2358 deprecated in favour of
inflexible-consumption / inflexible-production. An internal node's fixed demand
consumes, so inflexible-consumption is the field.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X2jLjsQzwztDc7Nxz1ozmQ
Signed-off-by: F.N. Claessen <claessen@seita.nl>

* review: drop a stray file, clarify the tutorial, quieten the internal-node log

Addresses part of the review:

- Removes .git-exclude, which held a lone "conftest.py" and reached this branch
  through the 2026-08-02 merge from feat/chp. It is on no other branch and has
  no business in the tree.
- Applies both tutorial suggestions verbatim, and replaces the dangling "This
  is how a whole factory is scheduled" with what it referred to (internal nodes
  and coupled converters together).
- Points the new cross-reference at flex_models_and_schedulers. My first
  attempt invented a label that does not exist, which would have failed the
  docs build.
- Drops the internal-node message from info to debug. Treating an unpriced
  commodity as an internal node is the normal path, not an event worth a line
  per schedule.

Still outstanding from the same review, and deliberately not rushed here:
reflowing this PR's docstrings and comments (the checker from #2386 reports 52
mid-phrase breaks in lines this PR adds), and re-checking the tests against the
mutation-test policy from #2384.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X2jLjsQzwztDc7Nxz1ozmQ
Signed-off-by: F.N. Claessen <claessen@seita.nl>

* review: reflow this PR's docstrings and comments

Breaks every line after punctuation, never mid-phrase, per the convention in
.github/instructions/docstrings.instructions.md.

Done by hand across all 48 prose sites the #2386 checker reported in lines this
PR adds, in linear_optimization.py, storage.py, highspy_optimization.py,
schemas/scheduling/__init__.py and the three test modules.

An automated re-wrap was tried first and reverted. It broke two files outright,
collapsing a closing docstring quote onto the following code line, and where it
did parse it left lines dangling on "In other words," and "To add storage to a
node," -- which satisfies "ends in punctuation" while splitting the clause the
rule exists to protect. The check generalises; the fix does not.

Four hits remain and are deliberate: the ASCII topology diagram in
test_factory_chp_dispatch_through_storage_scheduler, which sits in a literal
block and is not prose.

488 passed, 3 xfailed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X2jLjsQzwztDc7Nxz1ozmQ
Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: give inflexible devices in the flex-model their own section

The guidance was a pair of loose paragraphs inside the group-constraints
discussion, with no heading and no label, so nothing could link to it -- the
multi-commodity tutorial had to point at the whole flex-models chapter instead.

Promotes it to its own subsection with a label, and points the tutorial's
cross-reference at it.

Adds the part that was missing rather than merely unlabelled: when to use a
flex-context entry and when to use a flex-model entry. Site base load is a
property of the connection; a device sitting under a particular inverter,
feeder or commodity is a property of the device. Both net the same fixed power
into the grid connection, so the choice is about where it belongs, which was
nowhere stated.

Checked that every cross-reference in the two touched files resolves to a
label that exists.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X2jLjsQzwztDc7Nxz1ozmQ
Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: hint at larger sites rather than implying we walked through one

The closing paragraph read as a summary of a whole-factory example, but no such
example is worked through on the page, and no two industrial sites look alike.
It now says what it is actually there to say: coupled converters and internal
nodes compose, so chaining them describes a site, and getting there needs no
new fields -- only more entries of the kinds already shown.

Also adds the throughput note. An internal node's flows sum to zero by
construction, so a commitment scoped to the node binds nothing; to price pipe
wear or a conversion levy, scope it to the devices producing into the node,
whose summed flow is what passes through it. That was verified on the PR: a
commitment over a node's own devices gives an identical schedule and cost at
wear prices of 99 and 0.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X2jLjsQzwztDc7Nxz1ozmQ
Signed-off-by: F.N. Claessen <claessen@seita.nl>

---------

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Co-authored-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Co-authored-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant