feat(language): a lookup may map into its own dimension, so a representative snapshot is sayable

docs/reference/language/dimensions.md

@@ -85,14 +85,15 @@ lookups:
   period_of: { over: snapshot, into: period }
 ```
 
-| Field         |                                                                                                                                 |                |
-| ------------- | ------------------------------------------------------------------------------------------------------------------------------- | -------------- |
-| `over`        | required — the map's key dimensions: one, or a list in the order the table carries them ([below](#keyed-by-several-dimensions)) |                |
-| `into`        | required — the dimension its values are labels of, which is not a key                                                           |                |
-| `description` | free text, never parsed                                                                                                         | default `null` |
-
-The target must be a declared dimension, and it must be none of the keys. The
-values are checked against it when the data binds, which is the check that makes
+| Field         |                                                                                                                                        |                |
+| ------------- | -------------------------------------------------------------------------------------------------------------------------------------- | -------------- |
+| `over`        | required — the map's key dimensions: one, or a list in the order the table carries them ([below](#keyed-by-several-dimensions))        |                |
+| `into`        | required — the dimension its values are labels of; one of the keys, where the map is [into its own dimension](#into-its-own-dimension) |                |
+| `description` | free text, never parsed                                                                                                                | default `null` |
+
+The target must be a declared dimension. It may be one of the keys, where the
+map goes [into its own dimension](#into-its-own-dimension). The values are
+checked against the target when the data binds, which is the check that makes
 `sum(by=)` safe.
 
 That check is also why a label set the model only ever _selects_ on is declared
@@ -167,16 +168,63 @@ Six rules follow, and the loader decides each of them before any data binds:
 - **Each key is a declared dimension, named once.** The target is not one of
   them.
 
+### Into its own dimension
+
+A map may land in the dimension it is keyed by. The representative snapshot is
+the case: every snapshot names the one that stands for it, which is how a
+clustered year runs on a few typical days.
+
+```yaml
+dimensions:
+  snapshot: { dtype: int }
+lookups:
+  rep_of: { over: snapshot, into: snapshot }
+variables:
+  p: { foreach: [snapshot] }
+constraints:
+  representative:
+    foreach: [snapshot]
+    expression: p == at(p, by=rep_of) # every snapshot takes its representative's value
+  weighted:
+    foreach: [snapshot]
+    expression: sum(p, by=rep_of) <= 100 # the snapshots a representative stands for, summed onto it
+```
+
+No rule changes. The walked key is consumed and the target is produced, and
+here they are the same dimension, so `sum(by=)` and `at(by=)` both leave the
+frame as it was. A snapshot that no other snapshot names is an empty group, and
+contributes nothing. `shift(by=rep_of)` walks inside each representative's
+group, and `position(snapshot, by=rep_of)` counts within it. The table carries
+`snapshot` and `rep_of`, under the naming rule every lookup follows.
+
+A self-map is directional, because a lookup is a function: one value per key,
+and the declaration says which way the arrow points. `rep_of` sends every
+snapshot to its representative and never the other way. The two verbs are the
+two walks of that one arrow, as they are for every lookup. `at` reads along it,
+so each snapshot takes its representative's value. `sum` reads against it, so
+each representative collects the snapshots that point at it. The inverse of a
+many-to-one map is one-to-many, which is reachable as a grouping and never as a
+function. For a bijection, a successor map `next_of`, the two walks are the two
+directions outright. Two steps along the arrow are two nested calls,
+`at(at(x, by=rep_of), by=rep_of)`, because the frame is unchanged at each. What
+has no direction is not a lookup: an undirected neighbour relation between buses
+is a parameter over `[bus, bus]`, as every
+[many-to-many relation](#dimension-lookup-or-parameter) is.
+
+Selecting the representatives themselves, the rows where the map is the
+identity, is not a comparison the language has: a lookup is never compared to a
+dimension. Declare a `bool` parameter for them.
+
 ### How the map is supplied
 
 The map is a source key like any other, under the lookup's own name. It carries
-two columns, each named after the dimension it holds: the `over` dimension, and
-the target:
+one column per key, each named after its dimension, and the value column, named
+after the lookup:
 
 ```python
 sources = {
     'generator': ['g1', 'g2', 'g3'],
-    'gen_bus': pl.DataFrame({'generator': ['g1', 'g2'], 'bus': ['north', 'south']}),
+    'gen_bus': pl.DataFrame({'generator': ['g1', 'g2'], 'gen_bus': ['north', 'south']}),
 }
 ```
 
@@ -185,8 +233,7 @@ on no bus. A null in the value column is refused, because a missing row already
 says the same thing. A key that matches no label of `over` is an error rather
 than a new member.
 
-A map with several keys carries one column per key, named after its dimension,
-and is single-valued per key tuple. A generator in two zones in one period is
+A map with several keys is single-valued per key tuple. A generator in two zones in one period is
 refused, where a `0`/`1` membership parameter says it legally and silently.
 
 Values are never inferred from the parameters that use the target. If they were,

docs/reference/notation.md

@@ -50,6 +50,7 @@ lookups:
   area_of: { over: bus, into: zone } # a second map into the same set, to compare against
   season_of: { over: snapshot, into: season }
   gen_zone: { over: [generator, snapshot], into: zone } # a map keyed by two dimensions: a call walks one and joins on the other
+  rep_of: { over: snapshot, into: snapshot } # a map into its own dimension: the representative snapshot
 
 parameters:
   p_max: { dims: [generator] }
@@ -70,7 +71,7 @@ parameters:
 
 | Symbol | Meaning |
 |---|---|
-| $`\mathcal{T}`$ | index $`t`$ — `snapshot` (`int` coordinates) with $`\mathrm{season\_of}: \mathcal{T} \to \mathcal{S},\ \mathrm{gen\_zone}: \mathcal{G} \times \mathcal{T} \to \mathcal{Z}`$ |
+| $`\mathcal{T}`$ | index $`t`$ — `snapshot` (`int` coordinates) with $`\mathrm{season\_of}: \mathcal{T} \to \mathcal{S},\ \mathrm{gen\_zone}: \mathcal{G} \times \mathcal{T} \to \mathcal{Z},\ \mathrm{rep\_of}: \mathcal{T} \to \mathcal{T}`$ |
 | $`\mathcal{G}`$ | index $`g`$ — `generator` with $`\mathrm{gen\_bus}: \mathcal{G} \to \mathcal{B},\ \mathrm{gen\_tech}: \mathcal{G} \to \mathcal{E},\ \mathrm{gen\_zone}: \mathcal{G} \times \mathcal{T} \to \mathcal{Z}`$ |
 | $`\mathcal{B}`$ | index $`b`$ — `bus` with $`\mathrm{zone\_of}: \mathcal{B} \to \mathcal{Z},\ \mathrm{area\_of}: \mathcal{B} \to \mathcal{Z}`$ |
 | $`\mathcal{Z}`$ | index $`z`$ — `zone` |
@@ -375,6 +376,20 @@ pullback:
 \mathit{spill}_{t} \le \mathrm{zone\_cap}_{\mathrm{zone\_of}(b)} \qquad \forall\, t \in \mathcal{T},\ b \in \mathcal{B}
 ```
 
+#### `representative`
+
+a map into its own dimension, walked both ways: the frame is unchanged and the index is primed
+
+```yaml
+representative:
+  foreach: [snapshot]
+  expression: sum(spill, by=rep_of) <= at(spill, by=rep_of)
+```
+
+```math
+\sum_{t' \in \mathcal{T} \,:\, \mathrm{rep\_of}(t') = t} \mathit{spill}_{t'} \le \mathit{spill}_{\mathrm{rep\_of}(t)} \qquad \forall\, t \in \mathcal{T}
+```
+
 #### `grouped_twice`
 
 one grouping through two maps: the domain carries both conditions

schema/math-spec.schema.json

@@ -218,7 +218,7 @@
     },
     "LookupBlock": {
       "additionalProperties": false,
-      "description": "A named single-valued map from one or more key dimensions ``into:`` another.\n\nIts values are labels of ``into``, which is what ``sum(by=)`` and\n``at(by=)`` land terms on. ``over:`` is one dimension or a list \u2014 the\nmap's key columns, in the order the table carries them::\n\n    lookups:\n      bus_of: {over: generator, into: bus}\n      zone_of: {over: [generator, period], into: zone}\n\nThe map itself is data, and arrives at bind time under the lookup's name,\nsingle-valued per key tuple.",
+      "description": "A named single-valued map from one or more key dimensions ``into:`` a dimension.\n\nIts values are labels of ``into``, which is what ``sum(by=)`` and\n``at(by=)`` land terms on. ``over:`` is one dimension or a list \u2014 the\nmap's key columns, in the order the table carries them \u2014 and ``into``\nmay be one of them, which is how a representative snapshot is declared::\n\n    lookups:\n      bus_of: {over: generator, into: bus}\n      zone_of: {over: [generator, period], into: zone}\n      rep_of: {over: snapshot, into: snapshot}\n\nThe map itself is data, and arrives at bind time under the lookup's name,\nsingle-valued per key tuple, its value column named after the lookup.",
       "properties": {
         "description": {
           "anyOf": [

src/math_spec/model.py

@@ -154,18 +154,20 @@ def _also_written_as(
 
 
 class LookupBlock(_StrictBlock):
-    """A named single-valued map from one or more key dimensions ``into:`` another.
+    """A named single-valued map from one or more key dimensions ``into:`` a dimension.
 
     Its values are labels of ``into``, which is what ``sum(by=)`` and
     ``at(by=)`` land terms on. ``over:`` is one dimension or a list — the
-    map's key columns, in the order the table carries them::
+    map's key columns, in the order the table carries them — and ``into``
+    may be one of them, which is how a representative snapshot is declared::
 
         lookups:
           bus_of: {over: generator, into: bus}
           zone_of: {over: [generator, period], into: zone}
+          rep_of: {over: snapshot, into: snapshot}
 
     The map itself is data, and arrives at bind time under the lookup's name,
-    single-valued per key tuple.
+    single-valued per key tuple, its value column named after the lookup.
     """
 
     _label: ClassVar[str] = 'a lookup declaration'
@@ -815,7 +817,7 @@ def _frame_dimensions(self) -> Iterator[str]:
             )
 
     def _lookup_targets(self) -> Iterator[str]:
-        """A lookup is keyed by declared dimensions, each once, and maps into another declared one."""
+        """A lookup is keyed by declared dimensions, each once, and maps into a declared one — its own included."""
         for lname, lk in self.lookups.items():
             if not lk.keys:
                 yield f"Lookup '{lname}' has no key dimension: 'over:' names the dimension(s) the map is keyed by."
@@ -831,8 +833,6 @@ def _lookup_targets(self) -> Iterator[str]:
                     f"Declare it under 'dimensions:' — the target is what the "
                     f'lookup values are checked against.'
                 )
-            elif lk.into in lk.keys:
-                yield (f"Lookup '{lname}' maps '{lk.into}' into itself. A lookup maps into a different dimension.")
 
     def _bound_names(self) -> Iterator[str]:
         """A named bound is a numeric parameter."""

tests/test_dimensions.py

@@ -36,6 +36,7 @@
         'gen_bus': {'over': 'generator', 'into': 'bus'},
         'snap_bus': {'over': 'snapshot', 'into': 'bus'},
         'gen_zone': {'over': ['generator', 'snapshot'], 'into': 'zone'},
+        'rep_of': {'over': 'snapshot', 'into': 'snapshot'},
     },
     'parameters': {
         'p_max': {'dims': ['generator']},
@@ -125,6 +126,13 @@ def namespace() -> Namespace:
             id='a-partition-along-one-key-joined-on-the-other',
         ),
         pytest.param('sum(p, by=gen_bus.generator)', {'snapshot', 'bus'}, id='the-dot-is-legal-on-a-one-key-lookup'),
+        pytest.param('sum(p, by=rep_of)', {'snapshot', 'generator'}, id='a-map-into-its-own-dimension-keeps-the-frame'),
+        pytest.param('at(p, by=rep_of)', {'snapshot', 'generator'}, id='and-so-does-its-pullback'),
+        pytest.param(
+            "shift(p, over=snapshot, offset=1, edge='wrap', by=rep_of)",
+            {'snapshot', 'generator'},
+            id='a-partition-into-its-own-dimension',
+        ),
     ],
 )
 def test_dim_inference(expr, expected):
@@ -375,6 +383,8 @@ def test_a_zero_step_vacates_nothing_and_needs_no_edge(self):
         pytest.param('snap_bus == "b1"', {'snapshot'}, id='a-lookup-through-the-dim-it-maps-out-of'),
         pytest.param('gen_zone == "z1"', {'generator', 'snapshot'}, id='a-two-key-lookup-through-both-keys'),
         pytest.param('gen_zone', {'generator', 'snapshot'}, id='a-bare-two-key-lookup-the-same'),
+        pytest.param('rep_of == 3', {'snapshot'}, id='a-map-into-its-own-dimension-through-its-key'),
+        pytest.param('position(snapshot, by=rep_of) == 0', {'snapshot'}, id='a-position-within-a-representative'),
         pytest.param(
             'position(generator, by=gen_zone.generator) == 0',
             {'generator', 'snapshot'},

tests/test_validation.py

@@ -570,15 +570,9 @@ class TestRulesDecidedWithoutData:
                 {'lookups.lk.over': 'z'}, ("references undeclared dimension 'z'",), id='lookup-over-undeclared'
             ),
             pytest.param({'lookups.lk.into': 'z'}, ("targets undeclared dimension 'z'",), id='lookup-into-undeclared'),
-            pytest.param({'lookups.lk.into': 'g'}, ("maps 'g' into itself",), id='lookup-into-itself'),
             pytest.param(
                 {'lookups.lk.over': ['g', 'z']}, ("references undeclared dimension 'z'",), id='lookup-key-undeclared'
             ),
-            pytest.param(
-                {'dimensions.z': {}, 'lookups.lk.over': ['g', 'h']},
-                ("maps 'h' into itself",),
-                id='lookup-into-one-of-its-keys',
-            ),
             pytest.param(
                 {'lookups.lk.over': ['g', 'g']}, ("names 'g' twice under 'over:'",), id='lookup-keyed-by-a-dim-twice'
             ),

tests/typesetting/golden/latex.out

@@ -9,7 +9,7 @@
 
 \paragraph{Sets}
 \begin{description}
-\item[{$\mathcal{T}$}] index $t$ --- \texttt{snapshot} (\texttt{int} coordinates) with $\mathrm{season\_of}: \mathcal{T} \to \mathcal{S},\ \mathrm{gen\_zone}: \mathcal{G} \times \mathcal{T} \to \mathcal{Z}$
+\item[{$\mathcal{T}$}] index $t$ --- \texttt{snapshot} (\texttt{int} coordinates) with $\mathrm{season\_of}: \mathcal{T} \to \mathcal{S},\ \mathrm{gen\_zone}: \mathcal{G} \times \mathcal{T} \to \mathcal{Z},\ \mathrm{rep\_of}: \mathcal{T} \to \mathcal{T}$
 \item[{$\mathcal{G}$}] index $g$ --- \texttt{generator} with $\mathrm{gen\_bus}: \mathcal{G} \to \mathcal{B},\ \mathrm{gen\_tech}: \mathcal{G} \to \mathcal{E},\ \mathrm{gen\_zone}: \mathcal{G} \times \mathcal{T} \to \mathcal{Z}$
 \item[{$\mathcal{B}$}] index $b$ --- \texttt{bus} with $\mathrm{zone\_of}: \mathcal{B} \to \mathcal{Z},\ \mathrm{area\_of}: \mathcal{B} \to \mathcal{Z}$
 \item[{$\mathcal{Z}$}] index $z$ --- \texttt{zone}
@@ -92,6 +92,7 @@
 \text{history} && \sum_{t' \in \mathcal{T} \,:\, 0 \le t \ominus t' < \mathrm{min\_up}} \mathit{on}_{t',g} & \le \mathit{units}_{g} && \forall\, t \in \mathcal{T},\ g \in \mathcal{G} \\
 \text{seasonal\_window} && \sum_{t' \in \mathcal{T} \,:\, 0 \le t -^{\mathrm{season\_of}(t)} t' < 3} \mathit{on}_{t',g} & \le \mathit{units}_{g} && \forall\, t \in \mathcal{T},\ g \in \mathcal{G} \\
 \text{pullback} && \mathit{spill}_{t} & \le \mathrm{zone\_cap}_{\mathrm{zone\_of}(b)} && \forall\, t \in \mathcal{T},\ b \in \mathcal{B} \\
+\text{representative} && \sum_{t' \in \mathcal{T} \,:\, \mathrm{rep\_of}(t') = t} \mathit{spill}_{t'} & \le \mathit{spill}_{\mathrm{rep\_of}(t)} && \forall\, t \in \mathcal{T} \\
 \text{grouped\_twice} && \sum_{g \in \mathcal{G} \,:\, \mathrm{gen\_bus}(g) = b \wedge \mathrm{gen\_tech}(g) = e} p_{t,g} & \le \mathrm{tech\_cap}_{b,e} && \forall\, t \in \mathcal{T},\ b \in \mathcal{B},\ e \in \mathcal{E} \\
 \text{pulled\_back\_twice} && \mathit{units}_{g} & \le \mathrm{tech\_cap}_{\mathrm{gen\_bus}(g),\mathrm{gen\_tech}(g)} && \forall\, g \in \mathcal{G} \\
 \text{zonal} && \sum_{g \in \mathcal{G} \,:\, \mathrm{gen\_zone}(g,\ t) = z} p_{t,g} & \le \mathrm{zone\_cap}_{z} && \forall\, t \in \mathcal{T},\ z \in \mathcal{Z} \\

tests/typesetting/golden/markdown.out

@@ -6,7 +6,7 @@ every character a notation escapes, set as text: link\_to, 100% & \#1 costs \$5
 
 | Symbol | Meaning |
 |---|---|
-| $`\mathcal{T}`$ | index $`t`$ — `snapshot` (`int` coordinates) with $`\mathrm{season\_of}: \mathcal{T} \to \mathcal{S},\ \mathrm{gen\_zone}: \mathcal{G} \times \mathcal{T} \to \mathcal{Z}`$ |
+| $`\mathcal{T}`$ | index $`t`$ — `snapshot` (`int` coordinates) with $`\mathrm{season\_of}: \mathcal{T} \to \mathcal{S},\ \mathrm{gen\_zone}: \mathcal{G} \times \mathcal{T} \to \mathcal{Z},\ \mathrm{rep\_of}: \mathcal{T} \to \mathcal{T}`$ |
 | $`\mathcal{G}`$ | index $`g`$ — `generator` with $`\mathrm{gen\_bus}: \mathcal{G} \to \mathcal{B},\ \mathrm{gen\_tech}: \mathcal{G} \to \mathcal{E},\ \mathrm{gen\_zone}: \mathcal{G} \times \mathcal{T} \to \mathcal{Z}`$ |
 | $`\mathcal{B}`$ | index $`b`$ — `bus` with $`\mathrm{zone\_of}: \mathcal{B} \to \mathcal{Z},\ \mathrm{area\_of}: \mathcal{B} \to \mathcal{Z}`$ |
 | $`\mathcal{Z}`$ | index $`z`$ — `zone` |
@@ -172,6 +172,12 @@ p_{t,g} \le p_{t \boxminus_{0}^{\mathrm{season\_of}(t)} 1,g} \qquad \forall\, t
 \mathit{spill}_{t} \le \mathrm{zone\_cap}_{\mathrm{zone\_of}(b)} \qquad \forall\, t \in \mathcal{T},\ b \in \mathcal{B}
 ```
 
+**`representative`**
+
+```math
+\sum_{t' \in \mathcal{T} \,:\, \mathrm{rep\_of}(t') = t} \mathit{spill}_{t'} \le \mathit{spill}_{\mathrm{rep\_of}(t)} \qquad \forall\, t \in \mathcal{T}
+```
+
 **`grouped_twice`**
 
 ```math

tests/typesetting/golden/model.yaml

@@ -27,6 +27,7 @@ lookups:
   area_of: { over: bus, into: zone } # a second map into the same set, to compare against
   season_of: { over: snapshot, into: season }
   gen_zone: { over: [generator, snapshot], into: zone } # a map keyed by two dimensions: a call walks one and joins on the other
+  rep_of: { over: snapshot, into: snapshot } # a map into its own dimension: the representative snapshot
 
 parameters:
   p_max: { dims: [generator] }
@@ -147,6 +148,9 @@ constraints:
   pullback: # at(), which re-indexes through a lookup instead of an offset
     foreach: [snapshot, bus]
     expression: spill <= at(zone_cap, by=zone_of)
+  representative: # a map into its own dimension, walked both ways: the frame is unchanged and the index is primed
+    foreach: [snapshot]
+    expression: sum(spill, by=rep_of) <= at(spill, by=rep_of)
   grouped_twice: # one grouping through two maps: the domain carries both conditions
     foreach: [snapshot, bus, technology]
     expression: sum(p, by=[gen_bus, gen_tech]) <= tech_cap