Separate validation from parsing in model and study modules

AGENTS.md

@@ -72,7 +72,7 @@ The codebase is split into three packages along a solver-dependency boundary:
 **`gems_craft/model/`** — Immutable model templates.
 - `Model`: defines component behavior (parameters, variables, constraints, ports)
 - `Library`: a collection of models, loaded from YAML
-- `Taxonomy` (`taxonomy.py`): categories naming the items a model must expose. Models opt in via `taxonomy-category`; `check_library_against_taxonomy` enforces conformance, called from `parse_yaml_library` for libraries declaring a `taxonomy`. The caller supplies the `Taxonomy`; `load_study` reads it from the optional `input/taxonomy.yml`.
+- `Taxonomy` (`taxonomy.py`): categories naming the items a model must expose. Models opt in via `taxonomy-category`. `taxonomy.py` holds the data and `load_taxonomy` only; conformance lives in `validation.py` (`check_library_against_taxonomy`, and `validate_libraries_against_taxonomy` for libraries declaring a `taxonomy`), which `load_study` calls after parsing — `parse_yaml_library` stays a pure reader. `load_study` reads the `Taxonomy` from the optional `input/taxonomy.yml`.
 - `PortTypeSchema` (`parsing.py`) also declares the hybrid-only `area-connection` (`AreaConnectionSchema`) and `thermal-capacity-connection` (`PortThermalCapacitySchema`) fields; `resolve_library.py`'s `_convert_port_type` parses and discards them for every library, hybrid or not.
 
 **`gems_craft/expression/`** — Mathematical expression language and AST (structural/static analysis only — no numeric evaluation).

docs/CHANGELOG.md

@@ -5,11 +5,19 @@ All notable changes to GemsPy are documented here.
 ## [Unreleased]
 
 ### Changed
+- **Sequential mode carry-over length can now be controlled by the user through the parameter  `carry-over-length` (default: `block-overlap`). This also fixes an incorrect stitching for `block-overlap >= 2`, where the previous hardcoded behaviour pinned block
+  *N+1*'s first timestep to block *N*'s **last** timestep — a different absolute timestep. 
 - **linopy upgraded to `>=0.9.0`** - the minimum supported Python version rises
   to **3.11** accordingly (linopy 0.9 requires Python >= 3.11).
-- **Breaking** - parsing a library that declares a `taxonomy` raises `ValueError`
-  if no taxonomy is supplied, or if its id differs from the declared one.
+- **Breaking** - loading a study whose library declares a `taxonomy` raises
+  `ValueError` if no taxonomy is supplied, or if its id differs from the declared
+  one.
 - **Breaking** - `TaxonomyData` renamed to `TaxonomySchema`; no alias kept.
+- **Breaking** - `check_library_against_taxonomy` moved from
+  `gems_craft.model.taxonomy` to the new `gems_craft.model.validation`, and
+  `consistency_check` from `gems_craft.study.resolve_components` to the new
+  `gems_craft.study.validation`. Reading and validating are now separate modules,
+  as in `optim_config/`. No behavior change; import paths only.
 
 ### Added
 - **Integer strategy and thermal heuristics** - components can now set
@@ -20,10 +28,18 @@ All notable changes to GemsPy are documented here.
   compute tighter variable bounds from the first solve. Each model declares
   what a heuristic reads/writes via `models[].heuristics` in
   `optim-config.yml`.
-
-- **Taxonomy conformance checked at parse time** - `parse_yaml_library` takes an
-  optional `taxonomy` and checks every library declaring a `taxonomy` field.
-  `load_study` reads it from the optional `input/taxonomy.yml`.
+- **`lower_bound(variable_name)`** and **`upper_bound(variable_name)`** operators in the
+  expression language, usable in `extra-outputs` and port-field-definitions. Both take a bare
+  variable identifier and return its *current* lower/upper bound post-solve — in particular
+  reflecting mutations made by thermal heuristics (see "Integer strategy and thermal
+  heuristics" above), which previously had no way to be surfaced in results. Validated at
+  model-build time; using them inside constraints, binding-constraints, objective
+  contributions, or variable bounds raises a `ValueError`.
+
+- **Taxonomy conformance checked when a study is loaded** - `load_study` reads
+  the optional `input/taxonomy.yml` and calls
+  `validate_libraries_against_taxonomy` on every library declaring a `taxonomy`
+  field. `parse_yaml_library` is unchanged and performs no validation.
 
 ### Fixed
 - **Standard library parsing now accepts hybrid port-type fields** -

docs/agents/testing.md

@@ -23,7 +23,7 @@ nature (they build a study with `gems_craft` and solve it with `gems_runner` in
 | Unit — gems_craft / system | `tests/unittests/gems_craft/system/` | Model, network, and port object behaviour |
 | Unit — gems_craft / system parsing | `tests/unittests/gems_craft/system_parsing/` | System YAML parsing |
 | Unit — gems_craft_hybrid | `tests/unittests/gems_craft_hybrid/` | Hybrid GEMS/Antares Simulator schema parsing |
-| Unit — gems_runner / expression | `tests/unittests/gems_runner/expression/` | Solver-output expression evaluation (`dual()`, `reduced_cost()`, `variable()`) |
+| Unit — gems_runner / expression | `tests/unittests/gems_runner/expression/` | Solver-output expression evaluation (`dual()`, `reduced_cost()`, `lower_bound()`, `upper_bound()`, `variable()`) |
 | Unit — gems_runner / simulation | `tests/unittests/gems_runner/simulation/` | Full problem build + solve on small networks |
 | End-to-end — functional | `tests/e2e/functional/` | Cross-cutting tests: library/system combinations, stochastic, investment, scenario builder |
 | End-to-end — models | `tests/e2e/models/` | Model-level tests (andromede-v1 models, operator tests, proof-of-concept models) |

docs/user-guide/optim-config.md

@@ -41,7 +41,8 @@ solver-options:
 resolution:
   mode: sequential-subproblems   # see section below
   block-length: 168               # one week (in timesteps)
-  block-overlap: 0
+  block-overlap: 24               # consecutive blocks share one day
+  carry-over-length: 24           # optional; omitted → defaults to block-overlap
 
 # Per-model configuration (optional)
 models:
@@ -220,7 +221,8 @@ optimisation subproblems.
 |---|---|---|---|
 | `mode` | str | `"frontal"` | Resolution strategy (see below) |
 | `block-length` | int | — | Timesteps per window; required for windowed modes |
-| `block-overlap` | int | `0` | Extra overlap timesteps between consecutive blocks |
+| `block-overlap` | int | `0` | Sequential mode only (rejected in other modes): shared timesteps between consecutive blocks; must satisfy `0 <= block-overlap < block-length` |
+| `carry-over-length` | int | `block-overlap` | Sequential mode only (rejected in other modes): how many of the shared timesteps are pinned to the previous block's values; must satisfy `0 <= carry-over-length <= block-overlap` |
 
 ### `frontal` (default)
 
@@ -236,18 +238,92 @@ Produces globally optimal results.
 
 ### `sequential-subproblems`
 
-The horizon is split into non-overlapping (or slightly overlapping) windows of
-`block-length` timesteps.  Blocks are solved **one after the other**; the state
-of inter-block dynamics (e.g. storage level) is carried over from one block to
-the next.
+The horizon is split into windows of `block-length` timesteps, each starting
+`block-length - block-overlap` timesteps after the previous one.  Blocks are
+solved **one after the other**; the state of inter-block dynamics (e.g. storage
+level) is carried over from one block to the next by pinning the leading
+timesteps of each block to the values the previous block already computed.
 
 ~~~ yaml
 resolution:
   mode: sequential-subproblems
-  block-length: 168   # one week
-  block-overlap: 0
+  block-length: 168       # one week
+  block-overlap: 24       # one day shared between consecutive blocks
+  carry-over-length: 24   # optional; omitted → defaults to block-overlap (full pin)
+                          # 0 is legal and explicit: overlap solved twice, no stitching
 ~~~
 
+Three parameters shape the stitching between consecutive blocks.  Illustrative
+example with `block-length: 10`, `block-overlap: 4`, `carry-over-length: 3`
+(a partial pin, so all three parameters are visible at once):
+
+~~~ text
+abs t     0   1   2   3   4   5   6   7   8   9   10  11  12  13  14  15
+Block N   0   1   2   3   4   5   6   7   8   9
+          └──────────────────────────────────────┘
+                     block-length = 10
+
+Block N+1                         0   1   2   3   4   5   6   7   8   9
+                                  └──────────────────────────────────────┘
+                                             block-length = 10
+
+                                  |------------|  overlap = 4  (t=6..9: solved by BOTH blocks)
+                                  |========|  carry-over = 3  (t=6..8: PINNED to block N's value)
+                                              ^  t=9: still shared, but free in N+1 (re-optimized)
+~~~
+
+Reading it:
+
+- **`block-length`** — width of each block's own window (10 for both here).
+- **`block-overlap`** — how far block *N+1*'s start reaches back into block
+  *N*'s window (4 → t=6..9 exist in both solves).  The overlap gives block
+  *N+1* real historical values for lag-dependent constraints (e.g. a storage
+  balance using `soc[t-1]`, or min up/down durations spanning several hours).
+- **`carry-over-length`** — how many of those *shared* leading timesteps of
+  block *N+1* get hard-pinned (`var[t] == value from block N`) to block *N*'s
+  already-solved values, counted from the earliest shared timestep (t=6), not
+  from t=9.  Here `carry-over-length: 3 < overlap: 4`, so t=6,7,8 are frozen
+  but t=9 is left free — an MPC-style partial pin where the optimizer may
+  revise the tail of the overlap with more lookback context.
+
+Defaults and special values:
+
+- **Omitted** `carry-over-length` resolves to `block-overlap`: the whole
+  overlap zone is pinned.  This is the right default when the overlap exists
+  to provide history for lag-dependent constraints without re-litigating
+  decisions the previous block already made.
+- **Explicit `carry-over-length: 0`** is legal and distinct from omitting the
+  field: blocks overlap for lag-constraint history, but no timestep is pinned
+  — block *N+1* re-solves the whole overlap window independently.
+- Validation requires `0 <= carry-over-length <= block-overlap` (and
+  `0 <= block-overlap < block-length`), with no special case at
+  `block-overlap: 0`.
+
+Overlapping timesteps appear once per block in the simulation table, tagged
+with the `block` column — nothing is lost or silently merged.  Downstream
+tooling decides which block's version of a shared timestep is authoritative;
+`carry-over-length` only controls how much two consecutive blocks may
+*disagree* on that shared window.
+
+**What the carry-over pins.**  The mechanism is plain *variable fixing*: for
+block *N+1*, every time-dependent variable whose block-relative timestep falls
+in `[0, carry-over-length[` is fixed to the value block *N* computed for the
+**same absolute timestep**.  Two consequences are worth spelling out:
+
+- It is **not** an initial-condition mechanism.  Block *N+1*'s problem is not
+  given the value of the timestep *preceding* its window, so a `t-1` time-shift
+  operator at the block's first timestep still resolves against that block's own
+  border condition (cyclic by default) rather than reaching into block *N*.
+- It applies to **all** time-dependent variables of all models, not only
+  state-like ones such as a storage level.  Finer, per-model granularity can be
+  added later if a use case needs it.
+
+**Time-independent** variables (`structure.time = False`, e.g. an investment
+capacity) are never carried over — nothing links their values across blocks, so
+each block sizes them independently.  Sequential mode is therefore not suited to
+investment problems; use `frontal` or `benders-decomposition` for those.
+
+
 
 ### `parallel-subproblems`
 
@@ -417,7 +493,7 @@ Two heuristics are built in, each expecting a fixed set of
 
 | Heuristic | `inputs` elements | `outputs` elements |
 |---|---|---|
-| `fast` | `generation_power`, `cluster_max_generation`, `min_power_per_unit`, `max_power_per_unit`, `min_up_duration`, `min_down_duration` | `minimum_generation_power` |
+| `fast` | `generation_power`, `cluster_max_generation`, `min_power_per_unit`, `max_power_per_unit`, `min_up_duration`, `min_down_duration` | `minimum_generation_power`, `maximum_generation_power` |
 | `accurate` | `num_units_on_opt`, `num_units_max`, `min_up_duration`, `min_down_duration` | `minimum_num_units_on` |
 
 !!! note

src/gems_craft/expression/copy.py

@@ -23,6 +23,7 @@
     ExpressionNode,
     FloorNode,
     LiteralNode,
+    LowerBoundNode,
     MaxNode,
     MinNode,
     ParameterNode,
@@ -34,6 +35,7 @@
     TimeEvalNode,
     TimeShiftNode,
     TimeSumNode,
+    UpperBoundNode,
     VariableNode,
 )
 from .visitor import ExpressionVisitorOperations, visit
@@ -111,6 +113,12 @@ def dual(self, node: DualNode) -> ExpressionNode:
     def reduced_cost(self, node: ReducedCostNode) -> ExpressionNode:
         return ReducedCostNode(node.variable_id)
 
+    def lower_bound(self, node: LowerBoundNode) -> ExpressionNode:
+        return LowerBoundNode(node.variable_id)
+
+    def upper_bound(self, node: UpperBoundNode) -> ExpressionNode:
+        return UpperBoundNode(node.variable_id)
+
 
 def copy_expression(expression: ExpressionNode) -> ExpressionNode:
     return visit(expression, CopyVisitor())

src/gems_craft/expression/degree.py

@@ -20,6 +20,7 @@
     CeilNode,
     DualNode,
     FloorNode,
+    LowerBoundNode,
     MaxNode,
     MinNode,
     PortFieldAggregatorNode,
@@ -29,6 +30,7 @@
     TimeEvalNode,
     TimeShiftNode,
     TimeSumNode,
+    UpperBoundNode,
 )
 
 from .expression import (
@@ -132,6 +134,12 @@ def dual(self, node: DualNode) -> int | float:
     def reduced_cost(self, node: ReducedCostNode) -> int | float:
         return math.inf
 
+    def lower_bound(self, node: LowerBoundNode) -> int | float:
+        return math.inf
+
+    def upper_bound(self, node: UpperBoundNode) -> int | float:
+        return math.inf
+
 
 def compute_degree(expression: ExpressionNode) -> int | float:
     return visit(expression, ExpressionDegreeVisitor())

src/gems_craft/expression/equality.py

@@ -32,6 +32,7 @@
     CeilNode,
     DualNode,
     FloorNode,
+    LowerBoundNode,
     MaxNode,
     MinNode,
     PortFieldAggregatorNode,
@@ -42,6 +43,7 @@
     TimeEvalNode,
     TimeShiftNode,
     TimeSumNode,
+    UpperBoundNode,
 )
 
 
@@ -115,6 +117,10 @@ def visit(self, left: ExpressionNode, right: ExpressionNode) -> bool:
             return self.dual(left, right)
         if isinstance(left, ReducedCostNode) and isinstance(right, ReducedCostNode):
             return self.reduced_cost(left, right)
+        if isinstance(left, LowerBoundNode) and isinstance(right, LowerBoundNode):
+            return self.lower_bound(left, right)
+        if isinstance(left, UpperBoundNode) and isinstance(right, UpperBoundNode):
+            return self.upper_bound(left, right)
         raise NotImplementedError(f"Equality not implemented for {left.__class__}")
 
     def literal(self, left: LiteralNode, right: LiteralNode) -> bool:
@@ -217,6 +223,12 @@ def dual(self, left: DualNode, right: DualNode) -> bool:
     def reduced_cost(self, left: ReducedCostNode, right: ReducedCostNode) -> bool:
         return left.variable_id == right.variable_id
 
+    def lower_bound(self, left: LowerBoundNode, right: LowerBoundNode) -> bool:
+        return left.variable_id == right.variable_id
+
+    def upper_bound(self, left: UpperBoundNode, right: UpperBoundNode) -> bool:
+        return left.variable_id == right.variable_id
+
 
 def expressions_equal(
     left: ExpressionNode, right: ExpressionNode, abs_tol: float = 0, rel_tol: float = 0

src/gems_craft/expression/expression.py

@@ -357,6 +357,16 @@ class ReducedCostNode(ExpressionNode):
     variable_id: str
 
 
+@dataclass(frozen=True, eq=False)
+class LowerBoundNode(ExpressionNode):
+    variable_id: str
+
+
+@dataclass(frozen=True, eq=False)
+class UpperBoundNode(ExpressionNode):
+    variable_id: str
+
+
 def sum_expressions(expressions: Sequence[ExpressionNode]) -> ExpressionNode:
     if len(expressions) == 0:
         return LiteralNode(0)

src/gems_craft/expression/indexing.py

@@ -27,6 +27,7 @@
     ExpressionNode,
     FloorNode,
     LiteralNode,
+    LowerBoundNode,
     MaxNode,
     MinNode,
     MultiplicationNode,
@@ -40,6 +41,7 @@
     TimeEvalNode,
     TimeShiftNode,
     TimeSumNode,
+    UpperBoundNode,
     VariableNode,
 )
 from .visitor import ExpressionVisitor, T, visit
@@ -156,6 +158,12 @@ def dual(self, node: DualNode) -> IndexingStructure:
     def reduced_cost(self, node: ReducedCostNode) -> IndexingStructure:
         return self.context.get_variable_structure(node.variable_id)
 
+    def lower_bound(self, node: LowerBoundNode) -> IndexingStructure:
+        return self.context.get_variable_structure(node.variable_id)
+
+    def upper_bound(self, node: UpperBoundNode) -> IndexingStructure:
+        return self.context.get_variable_structure(node.variable_id)
+
 
 def compute_indexation(
     expression: ExpressionNode, provider: IndexingStructureProvider

src/gems_craft/expression/parsing/parse_expression.py

@@ -21,9 +21,11 @@
     Comparator,
     ComparisonNode,
     DualNode,
+    LowerBoundNode,
     PortFieldAggregatorNode,
     PortFieldNode,
     ReducedCostNode,
+    UpperBoundNode,
     maximum,
     minimum,
 )
@@ -194,6 +196,22 @@ def _visit_reduced_cost(self, arg_exprs: list) -> ExpressionNode:
             raise ValueError(f"'{vid}' is not a variable of the model.")
         return ReducedCostNode(vid)
 
+    def _visit_lower_bound(self, arg_exprs: list) -> ExpressionNode:
+        if len(arg_exprs) != 1:
+            raise ValueError("lower_bound() requires exactly 1 argument.")
+        vid = arg_exprs[0].getText()  # type: ignore
+        if vid not in self.identifiers.variables:
+            raise ValueError(f"'{vid}' is not a variable of the model.")
+        return LowerBoundNode(vid)
+
+    def _visit_upper_bound(self, arg_exprs: list) -> ExpressionNode:
+        if len(arg_exprs) != 1:
+            raise ValueError("upper_bound() requires exactly 1 argument.")
+        vid = arg_exprs[0].getText()  # type: ignore
+        if vid not in self.identifiers.variables:
+            raise ValueError(f"'{vid}' is not a variable of the model.")
+        return UpperBoundNode(vid)
+
     # Visit a parse tree produced by ExprParser#function.
     def visitFunction(self, ctx: ExprParser.FunctionContext) -> ExpressionNode:
         function_name: str = ctx.IDENTIFIER().getText()  # type: ignore
@@ -204,6 +222,10 @@ def visitFunction(self, ctx: ExprParser.FunctionContext) -> ExpressionNode:
             return self._visit_dual(arg_exprs)
         if function_name == "reduced_cost":
             return self._visit_reduced_cost(arg_exprs)
+        if function_name == "lower_bound":
+            return self._visit_lower_bound(arg_exprs)
+        if function_name == "upper_bound":
+            return self._visit_upper_bound(arg_exprs)
 
         args: list[ExpressionNode] = (
             [expr.accept(self) for expr in arg_exprs]  # type: ignore