feat(language): a lookup may be conditioned on a second dimension, so a generator's zone can change by period

docs/reference/language/dimensions.md

@@ -85,11 +85,12 @@ lookups:
   period_of: { over: snapshot, into: period }
 ```
 
-| Field         |                                                                      |                |
-| ------------- | -------------------------------------------------------------------- | -------------- |
-| `over`        | required — the dimension whose members carry the map                 |                |
-| `into`        | required — the dimension its values are labels of, other than `over` |                |
-| `description` | free text, never parsed                                              | default `null` |
+| Field         |                                                                                                                 |                |
+| ------------- | --------------------------------------------------------------------------------------------------------------- | -------------- |
+| `over`        | required — the dimension whose members carry the map                                                            |                |
+| `into`        | required — the dimension its values are labels of, other than `over`                                            |                |
+| `per`         | the dimensions the map is conditioned on, besides those two ([below](#maps-that-vary-along-a-second-dimension)) | default `[]`   |
+| `description` | free text, never parsed                                                                                         | default `null` |
 
 The target must be a declared dimension, and it must differ from `over`. The
 values are checked against it when the data binds, which is the check that makes
@@ -114,6 +115,46 @@ Every lookup name joins the flat namespace, so a lookup may not shadow a
 dimension, and that includes its own target. The map from `generator` onto `bus`
 is called `gen_bus`, never a second `bus`.
 
+### Maps that vary along a second dimension
+
+`over:` alone gives every generator one zone for the whole model. A generator
+whose bidding zone changes by period needs a second key, and `per:` names it:
+
+```yaml
+dimensions:
+  generator: { dtype: str }
+  zone: { dtype: str }
+  period: { dtype: int }
+lookups:
+  zone_of: { over: generator, into: zone, per: [period] }
+parameters:
+  demand: { dims: [zone, period] }
+variables:
+  p: { foreach: [generator, period] }
+constraints:
+  zone_balance:
+    foreach: [zone, period]
+    expression: sum(p, by=zone_of) >= demand
+```
+
+`over:` is consumed, `into:` is produced, and each `per` dimension is joined on
+and kept. So `sum(p, by=zone_of)` takes `p[generator, period]` to
+`[zone, period]`, and `at(price, by=zone_of)` reads `price[zone, period]` back
+at `[generator, period]`, which is the price of the zone this generator sat in
+that period. Nothing changes at the call site. Every operator that takes a
+lookup takes a conditioned one: `shift(by=)` and `position(by=)` group within
+each `per` coordinate, and a `where` naming the lookup is read at them.
+
+Four rules follow, and the loader decides each of them before any data binds:
+
+- **Each `per` dimension is declared, and is neither `over` nor `into`.** A
+  dimension named twice is refused as well.
+- **The operand carries every `per` dimension.** The map varies along them, so
+  there is no reading it at a coordinate that lacks them.
+- **A `by=` list shares its `per` as it shares its `over`.** One grouping is one
+  join.
+- **Two lookups compared in a `where` share it.** Otherwise no row carries both.
+
 ### How the map is supplied
 
 The map is a source key like any other, under the lookup's own name. It carries
@@ -132,6 +173,11 @@ on no bus. A null in the value column is refused, because a missing row already
 says the same thing. A key that matches no label of `over` is an error rather
 than a new member.
 
+A conditioned map carries one further key column per `per` dimension, named
+after it, and is single-valued per `(over, *per)`. A generator in two zones in
+one period is refused, where a `0`/`1` membership parameter says it legally and
+silently.
+
 Values are never inferred from the parameters that use the target. If they were,
 a mistyped label would extend the label set instead of being rejected.
 

docs/reference/language/expressions.md

@@ -172,8 +172,8 @@ QUOTED     ::= "'" chars "'" | '"' chars '"'
 | `name` (bare)                    | dimension                        | A load error. It would be true everywhere. Compare it against something instead                                                                                                                                                                                                                     |
 | `name OP value`                  | parameter                        | Element-wise, and a null compares false. The right-hand side is a literal, or a bare name read as a string label                                                                                                                                                                                    |
 | `name OP value`                  | dimension                        | A filter on the frame's own coordinate column                                                                                                                                                                                                                                                       |
-| `name OP value`                  | lookup                           | A filter on the lookup's value, so the `over` dimension has to be in the frame. A null compares false                                                                                                                                                                                               |
-| `name OP name`                   | two lookups                      | Legal only where both lookups are over the same dimension and into the same dimension. `from != to` excludes a self-loop                                                                                                                                                                            |
+| `name OP value`                  | lookup                           | A filter on the lookup's value, so the `over` dimension has to be in the frame, and so does every dimension it is [`per`](dimensions.md#maps-that-vary-along-a-second-dimension). A null compares false                                                                                             |
+| `name OP name`                   | two lookups                      | Legal only where both lookups are over the same dimension, into the same dimension, and conditioned on the same ones. `from != to` excludes a self-loop                                                                                                                                             |
 | `position(name) OP i`            | dimension                        | Where the row sits along the dimension's own order. `0` is first, and a negative number counts from the end                                                                                                                                                                                         |
 | `position(name, by=lookup) OP i` | a dimension and a lookup over it | The same, counted within each group the lookup makes                                                                                                                                                                                                                                                |
 | `AND` `OR` `NOT`                 | —                                | Case-insensitive. `NOT` binds tighter than `AND`, and `AND` tighter than `OR`                                                                                                                                                                                                                       |

docs/reference/language/operators.md

@@ -17,6 +17,7 @@ model can never depend on what a caller registered. A composition of them goes i
 | `sum(array, over=dim)`                             | `dim` collapses. `array` must carry `dim`                                                                                                         |
 | `sum(array, by=lookup)`                            | The dimension that the lookup is over collapses onto the dimension it maps into                                                                   |
 | `sum(array, by=[lookup, …])`                       | The same, onto every dimension that the lookups map into. All the lookups must be over the same dimension                                         |
+| `sum(array, by=conditioned)`                       | A lookup that declares `per:` is joined on those dimensions too. The array carries them, and the result keeps them                                |
 | `at(array, by=lookup)`                             | The dimension that the lookup maps into is replaced by the dimension it is over                                                                   |
 | `shift(array, over=dim, offset=n)`                 | The value `n` positions earlier along `dim`. The vacated edge is **absent**                                                                       |
 | `shift(array, over=dim, offset=n, edge='wrap')`    | The value `n` positions earlier, counted cyclically, so nothing is vacated                                                                        |
@@ -77,6 +78,10 @@ outflow, with no adjacency matrix and no join written by hand.
 Give **at most one** of `over=` and `by=`. A lookup carries its own dimensions,
 so `by=` leaves `over=` nothing to add.
 
+A lookup [conditioned on a second dimension](dimensions.md#maps-that-vary-along-a-second-dimension)
+is joined on that dimension as well. The operand carries it, the sum keeps it,
+and each group is one coordinate of it.
+
 The lookup's values are the group labels, checked against the target dimension
 when the data binds. A group with no members contributes nothing, and a member
 whose lookup value is null belongs to no group. An empty group is a value rather
@@ -96,6 +101,10 @@ once by every line that touches the bus, is `at(decision, by=line_bus)`.
 A fine label whose lookup value is null reads nothing, and its row is absent.
 That matches the null group in `sum(by=)`.
 
+Through a lookup [conditioned on a second dimension](dimensions.md#maps-that-vary-along-a-second-dimension)
+`at` reads the coarse value at the row's own coordinate of that dimension. That
+is the price of the zone this generator sat in that period.
+
 ## `sum_back`
 
 `sum_back(x, over=d, within=n)` is the sum of the last `n` positions along `d`,

docs/reference/notation.md

@@ -49,6 +49,7 @@ lookups:
   zone_of: { over: bus, into: zone }
   area_of: { over: bus, into: zone } # a second map into the same set, to compare against
   season_of: { over: snapshot, into: season }
+  gen_zone: { over: generator, into: zone, per: [snapshot] } # a map conditioned on a second dimension, which it is read at and keeps
 
 parameters:
   p_max: { dims: [generator] }
@@ -70,7 +71,7 @@ parameters:
 | Symbol | Meaning |
 |---|---|
 | $`\mathcal{T}`$ | index $`t`$ — `snapshot` (`int` coordinates) with $`\mathrm{season\_of}: \mathcal{T} \to \mathcal{S}`$ |
-| $`\mathcal{G}`$ | index $`g`$ — `generator` with $`\mathrm{gen\_bus}: \mathcal{G} \to \mathcal{B},\ \mathrm{gen\_tech}: \mathcal{G} \to \mathcal{E}`$ |
+| $`\mathcal{G}`$ | index $`g`$ — `generator` with $`\mathrm{gen\_bus}: \mathcal{G} \to \mathcal{B},\ \mathrm{gen\_tech}: \mathcal{G} \to \mathcal{E},\ \mathrm{gen\_zone}: \mathcal{G} \times \mathcal{T} \to \mathcal{Z}`$ |
 | $`\mathcal{B}`$ | index $`b`$ — `bus` with $`\mathrm{zone\_of}: \mathcal{B} \to \mathcal{Z},\ \mathrm{area\_of}: \mathcal{B} \to \mathcal{Z}`$ |
 | $`\mathcal{Z}`$ | index $`z`$ — `zone` |
 | $`\mathcal{S}`$ | index $`s`$ — `season` |
@@ -402,6 +403,35 @@ pulled_back_twice:
 \mathit{units}_{g} \le \mathrm{tech\_cap}_{\mathrm{gen\_bus}(g),\mathrm{gen\_tech}(g)} \qquad \forall\, g \in \mathcal{G}
 ```
 
+#### `zonal`
+
+a grouping through a conditioned map: the condition reads the dim the map is per, and the row keeps it
+
+```yaml
+zonal:
+  foreach: [snapshot, zone]
+  expression: sum(p, by=gen_zone) <= zone_cap
+```
+
+```math
+\sum_{g \in \mathcal{G} \,:\, \mathrm{gen\_zone}(g,\ t) = z} p_{t,g} \le \mathrm{zone\_cap}_{z} \qquad \forall\, t \in \mathcal{T},\ z \in \mathcal{Z}
+```
+
+#### `zonal_pullback`
+
+its adjoint, reading the slot the row's own snapshot puts the generator in
+
+```yaml
+zonal_pullback:
+  foreach: [snapshot, generator]
+  where: "gen_zone == 'north' AND position(generator, by=gen_zone) == 0"
+  expression: p <= at(spill * zone_cap, by=gen_zone)
+```
+
+```math
+p_{t,g} \le \mathit{spill}_{t} \cdot \mathrm{zone\_cap}_{\mathrm{gen\_zone}(g,\ t)} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} \,:\, \mathrm{gen\_zone}(g,\ t) = \text{'}\mathrm{north}\text{'} \wedge \mathrm{pos}_{\mathrm{gen\_zone}(g,\ t)}(g) = 0
+```
+
 #### `arithmetic`
 
 division, both unary signs, a sign beside a sign, floats with and without an exponent, bracketing

schema/math-spec.schema.json

@@ -218,7 +218,7 @@
     },
     "LookupBlock": {
       "additionalProperties": false,
-      "description": "A named single-valued map out of one dimension ``into:`` another.\n\nIts values are labels of ``into``, which is what ``sum(by=)`` and\n``at(by=)`` land terms on::\n\n    lookups:\n      bus_of: {over: generator, into: bus}\n\nThe map itself is data, and arrives at bind time under the lookup's name.",
+      "description": "A named single-valued map out of one dimension ``into:`` another.\n\nIts values are labels of ``into``, which is what ``sum(by=)`` and\n``at(by=)`` land terms on. ``per:`` conditions the map on further\ndimensions, which every operator joins on and passes through::\n\n    lookups:\n      bus_of: {over: generator, into: bus}\n      zone_of: {over: generator, into: zone, per: [period]}\n\nThe map itself is data, and arrives at bind time under the lookup's name,\nsingle-valued per ``(over, *per)``.",
       "properties": {
         "description": {
           "anyOf": [
@@ -239,6 +239,14 @@
         "over": {
           "title": "Over",
           "type": "string"
+        },
+        "per": {
+          "default": [],
+          "items": {
+            "type": "string"
+          },
+          "title": "Per",
+          "type": "array"
         }
       },
       "required": [

src/math_spec/_expression_parser.py

@@ -120,11 +120,14 @@ class LookupNode:
     ``dimension`` is the one every lookup is over — what ``sum`` consumes and
     ``at`` produces — and ``into`` the targets, one per name in the order
     written; ``sum(x, by=[gen_bus, gen_tech])`` is one grouping, not two.
+    ``per`` is the dims every one of them is conditioned on, which the
+    operand must carry and the operator passes through.
     """
 
     names: tuple[str, ...]
     dimension: str
     into: tuple[str, ...]
+    per: tuple[str, ...] = ()
 
     @property
     def shown(self) -> str:

src/math_spec/dimensions.py

@@ -161,6 +161,7 @@ def _sum_dims(node: FunctionCallNode, inner: frozenset[str], schema: Spec, conte
                 'drop the sum, or fix the dim',
             )
         )
+    _check_conditioned(f'sum(by={by.shown})', by, inner, context)
     collides = sorted(set(by.into) & (inner - {by.dimension}))
     if collides:
         raise DimensionError(
@@ -185,6 +186,7 @@ def _at_dims(node: FunctionCallNode, inner: frozenset[str], schema: Spec, contex
             f'{sorted(inner)}). A pullback needs the coarse dims to read *from* — '
             f'sum is the direction that produces them.'
         )
+    _check_conditioned(f'at(by={by.shown})', by, inner, context)
     if by.dimension in inner - set(by.into):
         raise DimensionError(
             f'{context}: at(by={by.shown}) places terms onto '
@@ -228,9 +230,20 @@ def _translation_dims(node: FunctionCallNode, inner: frozenset[str], schema: Spe
                 f"'{over.name}' carries it, so no coordinate has a neighbour inside a group — "
                 f"partition by a lookup over '{over.name}'."
             )
+        _check_conditioned(f'{node.name}(over={over.name}, by={partition.shown})', partition, inner, context)
     return inner
 
 
+def _check_conditioned(call: str, by: LookupNode, inner: frozenset[str], context: str) -> None:
+    """A lookup conditioned ``per`` some dims is read at them, so the operand carries every one."""
+    if missing := sorted(set(by.per) - inner):
+        raise DimensionError(
+            f'{context}: {call} reads a lookup conditioned per {missing}, which the expression '
+            f'does not carry (dims {sorted(inner)}). The map varies along those dims, so the operand '
+            f'has to be read at them — index it by them, or declare the lookup without them.'
+        )
+
+
 #: The dim rule of each built-in, by name.
 _CALL_RULES: dict[str, Callable[[FunctionCallNode, frozenset[str], Spec, str], frozenset[str]]] = {
     'sum': _sum_dims,

src/math_spec/lowering.py

@@ -143,7 +143,9 @@ def lower_program(expanded: _ExpandedSpec) -> program.Program:
     dimensions = {
         dname: program.DimensionDeclaration(
             tuple(
-                program.LookupDeclaration(lname, lk.into) for lname, lk in expanded.lookups.items() if lk.over == dname
+                program.LookupDeclaration(lname, lk.into, tuple(lk.per))
+                for lname, lk in expanded.lookups.items()
+                if lk.over == dname
             ),
             ddef.dtype,
         )
@@ -272,7 +274,9 @@ def sum(self, node: FunctionCallNode) -> program.ExpressionNode:
             assert isinstance(over_node, DimensionNode), 'resolution refuses an over= that is not a dimension'
             return program.Sum(operand, (over_node.name,))
         assert isinstance(by_node, LookupNode), 'resolution refuses a by= that is not a lookup'
-        return program.GroupSum(operand, over=by_node.dimension, coordinate=by_node.names, into=by_node.into)
+        return program.GroupSum(
+            operand, over=by_node.dimension, coordinate=by_node.names, into=by_node.into, per=by_node.per
+        )
 
     def at(self, node: FunctionCallNode) -> program.ExpressionNode:
         """``at(x, by=lookup)`` — the adjoint of :meth:`sum`'s ``by=`` form."""
@@ -283,6 +287,7 @@ def at(self, node: FunctionCallNode) -> program.ExpressionNode:
             over=by_node.dimension,
             coordinate=by_node.names,
             into=by_node.into,
+            per=by_node.per,
         )
 
     def sum_back(self, node: FunctionCallNode) -> program.ExpressionNode: