diff --git a/docs/reference/language/dimensions.md b/docs/reference/language/dimensions.md index f338ff2b..c04432ff 100644 --- a/docs/reference/language/dimensions.md +++ b/docs/reference/language/dimensions.md @@ -85,14 +85,15 @@ lookups: period_of: { over: snapshot, into: period } ``` -| Field | | | -| ------------- | ------------------------------------------------------------------------------------------------------------------------------- | -------------- | -| `over` | required — the map's key dimensions: one, or a list in the order the table carries them ([below](#keyed-by-several-dimensions)) | | -| `into` | required — the dimension its values are labels of, which is not a key | | -| `description` | free text, never parsed | default `null` | - -The target must be a declared dimension, and it must be none of the keys. The -values are checked against it when the data binds, which is the check that makes +| Field | | | +| ------------- | -------------------------------------------------------------------------------------------------------------------------------------- | -------------- | +| `over` | required — the map's key dimensions: one, or a list in the order the table carries them ([below](#keyed-by-several-dimensions)) | | +| `into` | required — the dimension its values are labels of; one of the keys, where the map is [into its own dimension](#into-its-own-dimension) | | +| `description` | free text, never parsed | default `null` | + +The target must be a declared dimension. It may be one of the keys, where the +map goes [into its own dimension](#into-its-own-dimension). The values are +checked against the target when the data binds, which is the check that makes `sum(by=)` safe. That check is also why a label set the model only ever _selects_ on is declared @@ -167,16 +168,63 @@ Six rules follow, and the loader decides each of them before any data binds: - **Each key is a declared dimension, named once.** The target is not one of them. +### Into its own dimension + +A map may land in the dimension it is keyed by. The representative snapshot is +the case: every snapshot names the one that stands for it, which is how a +clustered year runs on a few typical days. + +```yaml +dimensions: + snapshot: { dtype: int } +lookups: + rep_of: { over: snapshot, into: snapshot } +variables: + p: { foreach: [snapshot] } +constraints: + representative: + foreach: [snapshot] + expression: p == at(p, by=rep_of) # every snapshot takes its representative's value + weighted: + foreach: [snapshot] + expression: sum(p, by=rep_of) <= 100 # the snapshots a representative stands for, summed onto it +``` + +No rule changes. The walked key is consumed and the target is produced, and +here they are the same dimension, so `sum(by=)` and `at(by=)` both leave the +frame as it was. A snapshot that no other snapshot names is an empty group, and +contributes nothing. `shift(by=rep_of)` walks inside each representative's +group, and `position(snapshot, by=rep_of)` counts within it. The table carries +`snapshot` and `rep_of`, under the naming rule every lookup follows. + +A self-map is directional, because a lookup is a function: one value per key, +and the declaration says which way the arrow points. `rep_of` sends every +snapshot to its representative and never the other way. The two verbs are the +two walks of that one arrow, as they are for every lookup. `at` reads along it, +so each snapshot takes its representative's value. `sum` reads against it, so +each representative collects the snapshots that point at it. The inverse of a +many-to-one map is one-to-many, which is reachable as a grouping and never as a +function. For a bijection, a successor map `next_of`, the two walks are the two +directions outright. Two steps along the arrow are two nested calls, +`at(at(x, by=rep_of), by=rep_of)`, because the frame is unchanged at each. What +has no direction is not a lookup: an undirected neighbour relation between buses +is a parameter over `[bus, bus]`, as every +[many-to-many relation](#dimension-lookup-or-parameter) is. + +Selecting the representatives themselves, the rows where the map is the +identity, is not a comparison the language has: a lookup is never compared to a +dimension. Declare a `bool` parameter for them. + ### How the map is supplied The map is a source key like any other, under the lookup's own name. It carries -two columns, each named after the dimension it holds: the `over` dimension, and -the target: +one column per key, each named after its dimension, and the value column, named +after the lookup: ```python sources = { 'generator': ['g1', 'g2', 'g3'], - 'gen_bus': pl.DataFrame({'generator': ['g1', 'g2'], 'bus': ['north', 'south']}), + 'gen_bus': pl.DataFrame({'generator': ['g1', 'g2'], 'gen_bus': ['north', 'south']}), } ``` @@ -185,8 +233,7 @@ on no bus. A null in the value column is refused, because a missing row already says the same thing. A key that matches no label of `over` is an error rather than a new member. -A map with several keys carries one column per key, named after its dimension, -and is single-valued per key tuple. A generator in two zones in one period is +A map with several keys is single-valued per key tuple. A generator in two zones in one period is refused, where a `0`/`1` membership parameter says it legally and silently. Values are never inferred from the parameters that use the target. If they were, diff --git a/docs/reference/notation.md b/docs/reference/notation.md index 5f49ef46..43a4eb7a 100644 --- a/docs/reference/notation.md +++ b/docs/reference/notation.md @@ -50,6 +50,7 @@ lookups: area_of: { over: bus, into: zone } # a second map into the same set, to compare against season_of: { over: snapshot, into: season } gen_zone: { over: [generator, snapshot], into: zone } # a map keyed by two dimensions: a call walks one and joins on the other + rep_of: { over: snapshot, into: snapshot } # a map into its own dimension: the representative snapshot parameters: p_max: { dims: [generator] } @@ -70,7 +71,7 @@ parameters: | Symbol | Meaning | |---|---| -| $`\mathcal{T}`$ | index $`t`$ — `snapshot` (`int` coordinates) with $`\mathrm{season\_of}: \mathcal{T} \to \mathcal{S},\ \mathrm{gen\_zone}: \mathcal{G} \times \mathcal{T} \to \mathcal{Z}`$ | +| $`\mathcal{T}`$ | index $`t`$ — `snapshot` (`int` coordinates) with $`\mathrm{season\_of}: \mathcal{T} \to \mathcal{S},\ \mathrm{gen\_zone}: \mathcal{G} \times \mathcal{T} \to \mathcal{Z},\ \mathrm{rep\_of}: \mathcal{T} \to \mathcal{T}`$ | | $`\mathcal{G}`$ | index $`g`$ — `generator` with $`\mathrm{gen\_bus}: \mathcal{G} \to \mathcal{B},\ \mathrm{gen\_tech}: \mathcal{G} \to \mathcal{E},\ \mathrm{gen\_zone}: \mathcal{G} \times \mathcal{T} \to \mathcal{Z}`$ | | $`\mathcal{B}`$ | index $`b`$ — `bus` with $`\mathrm{zone\_of}: \mathcal{B} \to \mathcal{Z},\ \mathrm{area\_of}: \mathcal{B} \to \mathcal{Z}`$ | | $`\mathcal{Z}`$ | index $`z`$ — `zone` | @@ -375,6 +376,20 @@ pullback: \mathit{spill}_{t} \le \mathrm{zone\_cap}_{\mathrm{zone\_of}(b)} \qquad \forall\, t \in \mathcal{T},\ b \in \mathcal{B} ``` +#### `representative` + +a map into its own dimension, walked both ways: the frame is unchanged and the index is primed + +```yaml +representative: + foreach: [snapshot] + expression: sum(spill, by=rep_of) <= at(spill, by=rep_of) +``` + +```math +\sum_{t' \in \mathcal{T} \,:\, \mathrm{rep\_of}(t') = t} \mathit{spill}_{t'} \le \mathit{spill}_{\mathrm{rep\_of}(t)} \qquad \forall\, t \in \mathcal{T} +``` + #### `grouped_twice` one grouping through two maps: the domain carries both conditions diff --git a/schema/math-spec.schema.json b/schema/math-spec.schema.json index d37e744a..320af7e8 100644 --- a/schema/math-spec.schema.json +++ b/schema/math-spec.schema.json @@ -218,7 +218,7 @@ }, "LookupBlock": { "additionalProperties": false, - "description": "A named single-valued map from one or more key dimensions ``into:`` another.\n\nIts values are labels of ``into``, which is what ``sum(by=)`` and\n``at(by=)`` land terms on. ``over:`` is one dimension or a list \u2014 the\nmap's key columns, in the order the table carries them::\n\n lookups:\n bus_of: {over: generator, into: bus}\n zone_of: {over: [generator, period], into: zone}\n\nThe map itself is data, and arrives at bind time under the lookup's name,\nsingle-valued per key tuple.", + "description": "A named single-valued map from one or more key dimensions ``into:`` a dimension.\n\nIts values are labels of ``into``, which is what ``sum(by=)`` and\n``at(by=)`` land terms on. ``over:`` is one dimension or a list \u2014 the\nmap's key columns, in the order the table carries them \u2014 and ``into``\nmay be one of them, which is how a representative snapshot is declared::\n\n lookups:\n bus_of: {over: generator, into: bus}\n zone_of: {over: [generator, period], into: zone}\n rep_of: {over: snapshot, into: snapshot}\n\nThe map itself is data, and arrives at bind time under the lookup's name,\nsingle-valued per key tuple, its value column named after the lookup.", "properties": { "description": { "anyOf": [ diff --git a/src/math_spec/model.py b/src/math_spec/model.py index dfbd9fbc..17cdb1eb 100644 --- a/src/math_spec/model.py +++ b/src/math_spec/model.py @@ -154,18 +154,20 @@ def _also_written_as( class LookupBlock(_StrictBlock): - """A named single-valued map from one or more key dimensions ``into:`` another. + """A named single-valued map from one or more key dimensions ``into:`` a dimension. Its values are labels of ``into``, which is what ``sum(by=)`` and ``at(by=)`` land terms on. ``over:`` is one dimension or a list — the - map's key columns, in the order the table carries them:: + map's key columns, in the order the table carries them — and ``into`` + may be one of them, which is how a representative snapshot is declared:: lookups: bus_of: {over: generator, into: bus} zone_of: {over: [generator, period], into: zone} + rep_of: {over: snapshot, into: snapshot} The map itself is data, and arrives at bind time under the lookup's name, - single-valued per key tuple. + single-valued per key tuple, its value column named after the lookup. """ _label: ClassVar[str] = 'a lookup declaration' @@ -815,7 +817,7 @@ def _frame_dimensions(self) -> Iterator[str]: ) def _lookup_targets(self) -> Iterator[str]: - """A lookup is keyed by declared dimensions, each once, and maps into another declared one.""" + """A lookup is keyed by declared dimensions, each once, and maps into a declared one — its own included.""" for lname, lk in self.lookups.items(): if not lk.keys: yield f"Lookup '{lname}' has no key dimension: 'over:' names the dimension(s) the map is keyed by." @@ -831,8 +833,6 @@ def _lookup_targets(self) -> Iterator[str]: f"Declare it under 'dimensions:' — the target is what the " f'lookup values are checked against.' ) - elif lk.into in lk.keys: - yield (f"Lookup '{lname}' maps '{lk.into}' into itself. A lookup maps into a different dimension.") def _bound_names(self) -> Iterator[str]: """A named bound is a numeric parameter.""" diff --git a/tests/test_dimensions.py b/tests/test_dimensions.py index b8156c22..00c2ba4a 100644 --- a/tests/test_dimensions.py +++ b/tests/test_dimensions.py @@ -36,6 +36,7 @@ 'gen_bus': {'over': 'generator', 'into': 'bus'}, 'snap_bus': {'over': 'snapshot', 'into': 'bus'}, 'gen_zone': {'over': ['generator', 'snapshot'], 'into': 'zone'}, + 'rep_of': {'over': 'snapshot', 'into': 'snapshot'}, }, 'parameters': { 'p_max': {'dims': ['generator']}, @@ -125,6 +126,13 @@ def namespace() -> Namespace: id='a-partition-along-one-key-joined-on-the-other', ), pytest.param('sum(p, by=gen_bus.generator)', {'snapshot', 'bus'}, id='the-dot-is-legal-on-a-one-key-lookup'), + pytest.param('sum(p, by=rep_of)', {'snapshot', 'generator'}, id='a-map-into-its-own-dimension-keeps-the-frame'), + pytest.param('at(p, by=rep_of)', {'snapshot', 'generator'}, id='and-so-does-its-pullback'), + pytest.param( + "shift(p, over=snapshot, offset=1, edge='wrap', by=rep_of)", + {'snapshot', 'generator'}, + id='a-partition-into-its-own-dimension', + ), ], ) def test_dim_inference(expr, expected): @@ -375,6 +383,8 @@ def test_a_zero_step_vacates_nothing_and_needs_no_edge(self): pytest.param('snap_bus == "b1"', {'snapshot'}, id='a-lookup-through-the-dim-it-maps-out-of'), pytest.param('gen_zone == "z1"', {'generator', 'snapshot'}, id='a-two-key-lookup-through-both-keys'), pytest.param('gen_zone', {'generator', 'snapshot'}, id='a-bare-two-key-lookup-the-same'), + pytest.param('rep_of == 3', {'snapshot'}, id='a-map-into-its-own-dimension-through-its-key'), + pytest.param('position(snapshot, by=rep_of) == 0', {'snapshot'}, id='a-position-within-a-representative'), pytest.param( 'position(generator, by=gen_zone.generator) == 0', {'generator', 'snapshot'}, diff --git a/tests/test_validation.py b/tests/test_validation.py index 4775d098..08022813 100644 --- a/tests/test_validation.py +++ b/tests/test_validation.py @@ -570,15 +570,9 @@ class TestRulesDecidedWithoutData: {'lookups.lk.over': 'z'}, ("references undeclared dimension 'z'",), id='lookup-over-undeclared' ), pytest.param({'lookups.lk.into': 'z'}, ("targets undeclared dimension 'z'",), id='lookup-into-undeclared'), - pytest.param({'lookups.lk.into': 'g'}, ("maps 'g' into itself",), id='lookup-into-itself'), pytest.param( {'lookups.lk.over': ['g', 'z']}, ("references undeclared dimension 'z'",), id='lookup-key-undeclared' ), - pytest.param( - {'dimensions.z': {}, 'lookups.lk.over': ['g', 'h']}, - ("maps 'h' into itself",), - id='lookup-into-one-of-its-keys', - ), pytest.param( {'lookups.lk.over': ['g', 'g']}, ("names 'g' twice under 'over:'",), id='lookup-keyed-by-a-dim-twice' ), diff --git a/tests/typesetting/golden/latex.out b/tests/typesetting/golden/latex.out index d862d811..4c63e843 100644 --- a/tests/typesetting/golden/latex.out +++ b/tests/typesetting/golden/latex.out @@ -9,7 +9,7 @@ \paragraph{Sets} \begin{description} -\item[{$\mathcal{T}$}] index $t$ --- \texttt{snapshot} (\texttt{int} coordinates) with $\mathrm{season\_of}: \mathcal{T} \to \mathcal{S},\ \mathrm{gen\_zone}: \mathcal{G} \times \mathcal{T} \to \mathcal{Z}$ +\item[{$\mathcal{T}$}] index $t$ --- \texttt{snapshot} (\texttt{int} coordinates) with $\mathrm{season\_of}: \mathcal{T} \to \mathcal{S},\ \mathrm{gen\_zone}: \mathcal{G} \times \mathcal{T} \to \mathcal{Z},\ \mathrm{rep\_of}: \mathcal{T} \to \mathcal{T}$ \item[{$\mathcal{G}$}] index $g$ --- \texttt{generator} with $\mathrm{gen\_bus}: \mathcal{G} \to \mathcal{B},\ \mathrm{gen\_tech}: \mathcal{G} \to \mathcal{E},\ \mathrm{gen\_zone}: \mathcal{G} \times \mathcal{T} \to \mathcal{Z}$ \item[{$\mathcal{B}$}] index $b$ --- \texttt{bus} with $\mathrm{zone\_of}: \mathcal{B} \to \mathcal{Z},\ \mathrm{area\_of}: \mathcal{B} \to \mathcal{Z}$ \item[{$\mathcal{Z}$}] index $z$ --- \texttt{zone} @@ -92,6 +92,7 @@ \text{history} && \sum_{t' \in \mathcal{T} \,:\, 0 \le t \ominus t' < \mathrm{min\_up}} \mathit{on}_{t',g} & \le \mathit{units}_{g} && \forall\, t \in \mathcal{T},\ g \in \mathcal{G} \\ \text{seasonal\_window} && \sum_{t' \in \mathcal{T} \,:\, 0 \le t -^{\mathrm{season\_of}(t)} t' < 3} \mathit{on}_{t',g} & \le \mathit{units}_{g} && \forall\, t \in \mathcal{T},\ g \in \mathcal{G} \\ \text{pullback} && \mathit{spill}_{t} & \le \mathrm{zone\_cap}_{\mathrm{zone\_of}(b)} && \forall\, t \in \mathcal{T},\ b \in \mathcal{B} \\ +\text{representative} && \sum_{t' \in \mathcal{T} \,:\, \mathrm{rep\_of}(t') = t} \mathit{spill}_{t'} & \le \mathit{spill}_{\mathrm{rep\_of}(t)} && \forall\, t \in \mathcal{T} \\ \text{grouped\_twice} && \sum_{g \in \mathcal{G} \,:\, \mathrm{gen\_bus}(g) = b \wedge \mathrm{gen\_tech}(g) = e} p_{t,g} & \le \mathrm{tech\_cap}_{b,e} && \forall\, t \in \mathcal{T},\ b \in \mathcal{B},\ e \in \mathcal{E} \\ \text{pulled\_back\_twice} && \mathit{units}_{g} & \le \mathrm{tech\_cap}_{\mathrm{gen\_bus}(g),\mathrm{gen\_tech}(g)} && \forall\, g \in \mathcal{G} \\ \text{zonal} && \sum_{g \in \mathcal{G} \,:\, \mathrm{gen\_zone}(g,\ t) = z} p_{t,g} & \le \mathrm{zone\_cap}_{z} && \forall\, t \in \mathcal{T},\ z \in \mathcal{Z} \\ diff --git a/tests/typesetting/golden/markdown.out b/tests/typesetting/golden/markdown.out index 5706fe32..4cf78f71 100644 --- a/tests/typesetting/golden/markdown.out +++ b/tests/typesetting/golden/markdown.out @@ -6,7 +6,7 @@ every character a notation escapes, set as text: link\_to, 100% & \#1 costs \$5 | Symbol | Meaning | |---|---| -| $`\mathcal{T}`$ | index $`t`$ — `snapshot` (`int` coordinates) with $`\mathrm{season\_of}: \mathcal{T} \to \mathcal{S},\ \mathrm{gen\_zone}: \mathcal{G} \times \mathcal{T} \to \mathcal{Z}`$ | +| $`\mathcal{T}`$ | index $`t`$ — `snapshot` (`int` coordinates) with $`\mathrm{season\_of}: \mathcal{T} \to \mathcal{S},\ \mathrm{gen\_zone}: \mathcal{G} \times \mathcal{T} \to \mathcal{Z},\ \mathrm{rep\_of}: \mathcal{T} \to \mathcal{T}`$ | | $`\mathcal{G}`$ | index $`g`$ — `generator` with $`\mathrm{gen\_bus}: \mathcal{G} \to \mathcal{B},\ \mathrm{gen\_tech}: \mathcal{G} \to \mathcal{E},\ \mathrm{gen\_zone}: \mathcal{G} \times \mathcal{T} \to \mathcal{Z}`$ | | $`\mathcal{B}`$ | index $`b`$ — `bus` with $`\mathrm{zone\_of}: \mathcal{B} \to \mathcal{Z},\ \mathrm{area\_of}: \mathcal{B} \to \mathcal{Z}`$ | | $`\mathcal{Z}`$ | index $`z`$ — `zone` | @@ -172,6 +172,12 @@ p_{t,g} \le p_{t \boxminus_{0}^{\mathrm{season\_of}(t)} 1,g} \qquad \forall\, t \mathit{spill}_{t} \le \mathrm{zone\_cap}_{\mathrm{zone\_of}(b)} \qquad \forall\, t \in \mathcal{T},\ b \in \mathcal{B} ``` +**`representative`** + +```math +\sum_{t' \in \mathcal{T} \,:\, \mathrm{rep\_of}(t') = t} \mathit{spill}_{t'} \le \mathit{spill}_{\mathrm{rep\_of}(t)} \qquad \forall\, t \in \mathcal{T} +``` + **`grouped_twice`** ```math diff --git a/tests/typesetting/golden/model.yaml b/tests/typesetting/golden/model.yaml index 4a49b312..bcc1d4ba 100644 --- a/tests/typesetting/golden/model.yaml +++ b/tests/typesetting/golden/model.yaml @@ -27,6 +27,7 @@ lookups: area_of: { over: bus, into: zone } # a second map into the same set, to compare against season_of: { over: snapshot, into: season } gen_zone: { over: [generator, snapshot], into: zone } # a map keyed by two dimensions: a call walks one and joins on the other + rep_of: { over: snapshot, into: snapshot } # a map into its own dimension: the representative snapshot parameters: p_max: { dims: [generator] } @@ -147,6 +148,9 @@ constraints: pullback: # at(), which re-indexes through a lookup instead of an offset foreach: [snapshot, bus] expression: spill <= at(zone_cap, by=zone_of) + representative: # a map into its own dimension, walked both ways: the frame is unchanged and the index is primed + foreach: [snapshot] + expression: sum(spill, by=rep_of) <= at(spill, by=rep_of) grouped_twice: # one grouping through two maps: the domain carries both conditions foreach: [snapshot, bus, technology] expression: sum(p, by=[gen_bus, gen_tech]) <= tech_cap diff --git a/tests/typesetting/golden/typst.out b/tests/typesetting/golden/typst.out index 68285be7..5fb828be 100644 --- a/tests/typesetting/golden/typst.out +++ b/tests/typesetting/golden/typst.out @@ -4,7 +4,7 @@ every character a notation escapes, set as text: link\_to, 100% & \#1 costs \$5 {net} \~ ^ \\ \*star\* \@ref \, and `a_name` in backticks set in code == Sets -/ $cal(T)$: index $t$ --- `snapshot` (`int` coordinates) with $upright("season_of"): cal(T) arrow.r cal(S), upright("gen_zone"): cal(G) times cal(T) arrow.r cal(Z)$ +/ $cal(T)$: index $t$ --- `snapshot` (`int` coordinates) with $upright("season_of"): cal(T) arrow.r cal(S), upright("gen_zone"): cal(G) times cal(T) arrow.r cal(Z), upright("rep_of"): cal(T) arrow.r cal(T)$ / $cal(G)$: index $g$ --- `generator` with $upright("gen_bus"): cal(G) arrow.r cal(B), upright("gen_tech"): cal(G) arrow.r cal(E), upright("gen_zone"): cal(G) times cal(T) arrow.r cal(Z)$ / $cal(B)$: index $b$ --- `bus` with $upright("zone_of"): cal(B) arrow.r cal(Z), upright("area_of"): cal(B) arrow.r cal(Z)$ / $cal(Z)$: index $z$ --- `zone` @@ -79,6 +79,7 @@ $ upright("budgeted") & italic("spend")_(t) & <= upright("budget") & forall t in upright("history") & sum_(t' in cal(T) colon 0 <= t minus.o t' < upright("min_up")) italic("on")_(t',g) & <= italic("units")_(g) & forall t in cal(T), g in cal(G) \ upright("seasonal_window") & sum_(t' in cal(T) colon 0 <= t -^(upright("season_of")(t)) t' < 3) italic("on")_(t',g) & <= italic("units")_(g) & forall t in cal(T), g in cal(G) \ upright("pullback") & italic("spill")_(t) & <= upright("zone_cap")_(upright("zone_of")(b)) & forall t in cal(T), b in cal(B) \ + upright("representative") & sum_(t' in cal(T) colon upright("rep_of")(t') = t) italic("spill")_(t') & <= italic("spill")_(upright("rep_of")(t)) & forall t in cal(T) \ upright("grouped_twice") & sum_(g in cal(G) colon upright("gen_bus")(g) = b and upright("gen_tech")(g) = e) p_(t,g) & <= upright("tech_cap")_(b,e) & forall t in cal(T), b in cal(B), e in cal(E) \ upright("pulled_back_twice") & italic("units")_(g) & <= upright("tech_cap")_(upright("gen_bus")(g),upright("gen_tech")(g)) & forall g in cal(G) \ upright("zonal") & sum_(g in cal(G) colon upright("gen_zone")(g, t) = z) p_(t,g) & <= upright("zone_cap")_(z) & forall t in cal(T), z in cal(Z) \