diff --git a/docs/reference/language/dimensions.md b/docs/reference/language/dimensions.md index 7b4373b7..f338ff2b 100644 --- a/docs/reference/language/dimensions.md +++ b/docs/reference/language/dimensions.md @@ -85,13 +85,13 @@ lookups: period_of: { over: snapshot, into: period } ``` -| Field | | | -| ------------- | -------------------------------------------------------------------- | -------------- | -| `over` | required — the dimension whose members carry the map | | -| `into` | required — the dimension its values are labels of, other than `over` | | -| `description` | free text, never parsed | default `null` | +| Field | | | +| ------------- | ------------------------------------------------------------------------------------------------------------------------------- | -------------- | +| `over` | required — the map's key dimensions: one, or a list in the order the table carries them ([below](#keyed-by-several-dimensions)) | | +| `into` | required — the dimension its values are labels of, which is not a key | | +| `description` | free text, never parsed | default `null` | -The target must be a declared dimension, and it must differ from `over`. The +The target must be a declared dimension, and it must be none of the keys. The values are checked against it when the data binds, which is the check that makes `sum(by=)` safe. @@ -107,13 +107,66 @@ error. Several lookups may group at once. `sum(x, by=[gen_bus, gen_tech])` groups through both maps in one reduction and lands on `bus` and `technology`. Every -lookup in the list must be `over:` the same dimension, and each must target a +lookup in the list must walk the same dimension, and each must target a different one. A member that either map leaves out belongs to no group. Every lookup name joins the flat namespace, so a lookup may not shadow a dimension, and that includes its own target. The map from `generator` onto `bus` is called `gen_bus`, never a second `bus`. +### Keyed by several dimensions + +A map keyed by one dimension gives every generator one zone for the whole +model. A generator whose bidding zone changes by period needs a second key, and +`over:` takes a list of them: + +```yaml +dimensions: + generator: { dtype: str } + zone: { dtype: str } + period: { dtype: int } +lookups: + zone_of: { over: [generator, period], into: zone } +parameters: + demand: { dims: [zone, period] } +variables: + p: { foreach: [generator, period] } +constraints: + zone_balance: + foreach: [zone, period] + expression: sum(p, by=zone_of.generator) >= demand +``` + +A call walks one key and joins on the rest. The dot says which: +`by=zone_of.generator` consumes `generator`, produces `zone`, and joins on +`period`. So `sum(p, by=zone_of.generator)` takes `p[generator, period]` to +`[zone, period]`, and `at(price, by=zone_of.generator)` reads +`price[zone, period]` back at `[generator, period]`, which is the price of the +zone this generator sat in that period. The same table walked along its other +key is a different sum: `sum(p, by=zone_of.period)` takes `p` to +`[generator, zone]`, each generator's output over the periods it spent in each +zone. + +Six rules follow, and the loader decides each of them before any data binds: + +- **The dot names a key.** Write it wherever the lookup has more than one key. + Without it the call is refused, because the operator cannot know which key it + consumes. With one key the dot is redundant and legal, so `by=gen_bus` and + `by=gen_bus.generator` are the same call. +- **The operand carries every key but the one walked.** The map is read at + those keys, so there is no reading it at a coordinate that lacks them. +- **The walked key is the walked dimension.** `shift(x, over=d, by=l.k)`, + `sum_back(x, over=d, by=l.k)` and `position(d, by=l.k)` need `k` to be `d`. + Each groups the rows of `d` within one coordinate of the other keys. +- **A `by=` list walks one dimension.** `by=[a.k, b.k]` is one grouping, so + every lookup in it names the same key dimension. Each joins on its own other + keys. +- **A `where` reads every key.** `zone_of == 'north'`, a bare `zone_of` and + `zone_of != area_of` are filters on the key table, so the frame carries all of + a lookup's keys, and two lookups compared carry the same keys. +- **Each key is a declared dimension, named once.** The target is not one of + them. + ### How the map is supplied The map is a source key like any other, under the lookup's own name. It carries @@ -132,6 +185,10 @@ on no bus. A null in the value column is refused, because a missing row already says the same thing. A key that matches no label of `over` is an error rather than a new member. +A map with several keys carries one column per key, named after its dimension, +and is single-valued per key tuple. A generator in two zones in one period is +refused, where a `0`/`1` membership parameter says it legally and silently. + Values are never inferred from the parameters that use the target. If they were, a mistyped label would extend the label set instead of being rejected. @@ -144,14 +201,23 @@ but a column named after the lookup is refused rather than read. Every column of data is one of the three. What decides which is what the math does with the column, not what the column holds: -| The column… | is declared as | because | -| -------------------------------------------------------------------------------------- | ------------------------------------- | --------------------------------------------------------------------------------------------- | -| is an axis: something is indexed by it, or an aggregation lands terms on it | a `dimension` | its members are the coordinate set every table over it is reindexed onto | -| has one value per member of a dimension and points at another — a generator's bus | a `lookup` into that dimension | it is a map that `sum(by=)` and `at(by=)` walk, and its values are checked against the target | -| is a label set the model only selects on or counts within — a period, a season, a zone | a `dimension`, and a `lookup` into it | the membership check is worth one line and one member list | -| scales terms — a coefficient, a bound, an offset | a `parameter` (`float` or `int`) | arithmetic is over numbers ([dtype](declarations.md#parameters)) | -| is a per-row attribute the math only selects on — a fuel, a constraint's sense | a `str` parameter | it names rows rather than scaling them, and no set is declared to check its values against | -| is a mask | a `bool` parameter | a bare name in a `where` is its own answer | +| The column… | is declared as | because | +| ----------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | +| is an axis: something is indexed by it, or an aggregation lands terms on it | a `dimension` | its members are the coordinate set every table over it is reindexed onto | +| has one value per member of a dimension, or per tuple of several, and points at another — a generator's bus, a generator's zone by period | a `lookup` into that dimension | it is a map that `sum(by=)` and `at(by=)` walk, and its values are checked against the target | +| relates members of two dimensions many-to-many — a link's several buses with their efficiencies, a cycle's lines | a `parameter` over both | `bool` where it only selects, numeric where it weights. The aggregation is `sum(w * x, over=a)`, and a pair the table lacks is absent | +| is a label set the model only selects on or counts within — a period, a season, a zone | a `dimension`, and a `lookup` into it | the membership check is worth one line and one member list | +| scales terms — a coefficient, a bound, an offset | a `parameter` (`float` or `int`) | arithmetic is over numbers ([dtype](declarations.md#parameters)) | +| is a per-row attribute the math only selects on — a fuel, a constraint's sense | a `str` parameter | it names rows rather than scaling them, and no set is declared to check its values against | +| is a mask | a `bool` parameter | a bare name in a `where` is its own answer | + +**A many-to-many relation is a parameter, weighted or not.** Pairs alone are a +`bool` parameter, written `connection: {dims: [entity, bus], dtype: bool}` and +read with `where: connection`. Pairs with a weight are a numeric one, and the +aggregation needs no lookup: `sum(efficiency * p, over=entity)` lands on `bus`, +because `efficiency[entity, bus]` has a row exactly where the pair exists. A +lookup is the single-valued case, where the language checks a claim a parameter +cannot make. Two rules follow from the table. If `b` has one value per `a`, then `b` is a **lookup** over `a`, and not a dimension: a `foreach` product over two diff --git a/docs/reference/language/expressions.md b/docs/reference/language/expressions.md index 03769fce..b840d021 100644 --- a/docs/reference/language/expressions.md +++ b/docs/reference/language/expressions.md @@ -91,7 +91,7 @@ fixed at load: | --------------------------------------- | ------------------------------------------------------------------------------------------------------------ | | expression (`p * cost`) | a variable, or a parameter whose values are numbers ([dtype](declarations.md#parameters)) | | dimension argument (`over=`) | a dimension | -| lookup argument (`by=` on `sum` / `at`) | a lookup, and never a dimension | +| lookup argument (`by=` on `sum` / `at`) | a lookup, dotted with the key it walks where it has several, and never a dimension | | `where` string | a parameter, variable, dimension or lookup ([where strings](#where-strings)) | | `bounds.lower` / `bounds.upper` | a parameter name, or a number | | the `edge` key of `shift` | `'wrap'` in quotes, or a bare number. Never a dimension | @@ -172,8 +172,8 @@ QUOTED ::= "'" chars "'" | '"' chars '"' | `name` (bare) | dimension | A load error. It would be true everywhere. Compare it against something instead | | `name OP value` | parameter | Element-wise, and a null compares false. The right-hand side is a literal, or a bare name read as a string label | | `name OP value` | dimension | A filter on the frame's own coordinate column | -| `name OP value` | lookup | A filter on the lookup's value, so the `over` dimension has to be in the frame. A null compares false | -| `name OP name` | two lookups | Legal only where both lookups are over the same dimension and into the same dimension. `from != to` excludes a self-loop | +| `name OP value` | lookup | A filter on the lookup's value, so [every key](dimensions.md#keyed-by-several-dimensions) has to be in the frame. A null compares false | +| `name OP name` | two lookups | Legal only where both lookups have the same keys and map into the same dimension. `from != to` excludes a self-loop | | `position(name) OP i` | dimension | Where the row sits along the dimension's own order. `0` is first, and a negative number counts from the end | | `position(name, by=lookup) OP i` | a dimension and a lookup over it | The same, counted within each group the lookup makes | | `AND` `OR` `NOT` | — | Case-insensitive. `NOT` binds tighter than `AND`, and `AND` tighter than `OR` | diff --git a/docs/reference/language/operators.md b/docs/reference/language/operators.md index 7f3954cf..3c807905 100644 --- a/docs/reference/language/operators.md +++ b/docs/reference/language/operators.md @@ -17,6 +17,7 @@ model can never depend on what a caller registered. A composition of them goes i | `sum(array, over=dim)` | `dim` collapses. `array` must carry `dim` | | `sum(array, by=lookup)` | The dimension that the lookup is over collapses onto the dimension it maps into | | `sum(array, by=[lookup, …])` | The same, onto every dimension that the lookups map into. All the lookups must be over the same dimension | +| `sum(array, by=lookup.key)` | A lookup keyed by several dimensions is walked along the named key. The others are joined on, so the array carries them and the result keeps them | | `at(array, by=lookup)` | The dimension that the lookup maps into is replaced by the dimension it is over | | `shift(array, over=dim, offset=n)` | The value `n` positions earlier along `dim`. The vacated edge is **absent** | | `shift(array, over=dim, offset=n, edge='wrap')` | The value `n` positions earlier, counted cyclically, so nothing is vacated | @@ -77,6 +78,10 @@ outflow, with no adjacency matrix and no join written by hand. Give **at most one** of `over=` and `by=`. A lookup carries its own dimensions, so `by=` leaves `over=` nothing to add. +A lookup [keyed by several dimensions](dimensions.md#keyed-by-several-dimensions) +is walked along the key the dot names and joined on the others. The operand +carries them, the sum keeps them, and each group is one coordinate of them. + The lookup's values are the group labels, checked against the target dimension when the data binds. A group with no members contributes nothing, and a member whose lookup value is null belongs to no group. An empty group is a value rather @@ -96,6 +101,10 @@ once by every line that touches the bus, is `at(decision, by=line_bus)`. A fine label whose lookup value is null reads nothing, and its row is absent. That matches the null group in `sum(by=)`. +Through a lookup [keyed by several dimensions](dimensions.md#keyed-by-several-dimensions) +`at` reads the coarse value at the row's own coordinate of the keys not walked. +That is the price of the zone this generator sat in that period. + ## `sum_back` `sum_back(x, over=d, within=n)` is the sum of the last `n` positions along `d`, diff --git a/docs/reference/notation.md b/docs/reference/notation.md index b9cb558f..5f49ef46 100644 --- a/docs/reference/notation.md +++ b/docs/reference/notation.md @@ -49,6 +49,7 @@ lookups: zone_of: { over: bus, into: zone } area_of: { over: bus, into: zone } # a second map into the same set, to compare against season_of: { over: snapshot, into: season } + gen_zone: { over: [generator, snapshot], into: zone } # a map keyed by two dimensions: a call walks one and joins on the other parameters: p_max: { dims: [generator] } @@ -69,8 +70,8 @@ parameters: | Symbol | Meaning | |---|---| -| $`\mathcal{T}`$ | index $`t`$ — `snapshot` (`int` coordinates) with $`\mathrm{season\_of}: \mathcal{T} \to \mathcal{S}`$ | -| $`\mathcal{G}`$ | index $`g`$ — `generator` with $`\mathrm{gen\_bus}: \mathcal{G} \to \mathcal{B},\ \mathrm{gen\_tech}: \mathcal{G} \to \mathcal{E}`$ | +| $`\mathcal{T}`$ | index $`t`$ — `snapshot` (`int` coordinates) with $`\mathrm{season\_of}: \mathcal{T} \to \mathcal{S},\ \mathrm{gen\_zone}: \mathcal{G} \times \mathcal{T} \to \mathcal{Z}`$ | +| $`\mathcal{G}`$ | index $`g`$ — `generator` with $`\mathrm{gen\_bus}: \mathcal{G} \to \mathcal{B},\ \mathrm{gen\_tech}: \mathcal{G} \to \mathcal{E},\ \mathrm{gen\_zone}: \mathcal{G} \times \mathcal{T} \to \mathcal{Z}`$ | | $`\mathcal{B}`$ | index $`b`$ — `bus` with $`\mathrm{zone\_of}: \mathcal{B} \to \mathcal{Z},\ \mathrm{area\_of}: \mathcal{B} \to \mathcal{Z}`$ | | $`\mathcal{Z}`$ | index $`z`$ — `zone` | | $`\mathcal{S}`$ | index $`s`$ — `season` | @@ -402,6 +403,49 @@ pulled_back_twice: \mathit{units}_{g} \le \mathrm{tech\_cap}_{\mathrm{gen\_bus}(g),\mathrm{gen\_tech}(g)} \qquad \forall\, g \in \mathcal{G} ``` +#### `zonal` + +a grouping through a two-key map, walked along one key: the condition reads the other, and the row keeps it + +```yaml +zonal: + foreach: [snapshot, zone] + expression: sum(p, by=gen_zone.generator) <= zone_cap +``` + +```math +\sum_{g \in \mathcal{G} \,:\, \mathrm{gen\_zone}(g,\ t) = z} p_{t,g} \le \mathrm{zone\_cap}_{z} \qquad \forall\, t \in \mathcal{T},\ z \in \mathcal{Z} +``` + +#### `zonal_history` + +the same table walked along its other key + +```yaml +zonal_history: + foreach: [generator, zone] + expression: sum(p, by=gen_zone.snapshot) <= zone_cap +``` + +```math +\sum_{t \in \mathcal{T} \,:\, \mathrm{gen\_zone}(g,\ t) = z} p_{t,g} \le \mathrm{zone\_cap}_{z} \qquad \forall\, g \in \mathcal{G},\ z \in \mathcal{Z} +``` + +#### `zonal_pullback` + +its adjoint, reading the slot the row's own snapshot puts the generator in + +```yaml +zonal_pullback: + foreach: [snapshot, generator] + where: "gen_zone == 'north' AND position(generator, by=gen_zone.generator) == 0" + expression: p <= at(spill * zone_cap, by=gen_zone.generator) +``` + +```math +p_{t,g} \le \mathit{spill}_{t} \cdot \mathrm{zone\_cap}_{\mathrm{gen\_zone}(g,\ t)} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} \,:\, \mathrm{gen\_zone}(g,\ t) = \text{'}\mathrm{north}\text{'} \wedge \mathrm{pos}_{\mathrm{gen\_zone}(g,\ t)}(g) = 0 +``` + #### `arithmetic` division, both unary signs, a sign beside a sign, floats with and without an exponent, bracketing diff --git a/schema/math-spec.schema.json b/schema/math-spec.schema.json index 9d5dc8c3..d37e744a 100644 --- a/schema/math-spec.schema.json +++ b/schema/math-spec.schema.json @@ -218,7 +218,7 @@ }, "LookupBlock": { "additionalProperties": false, - "description": "A named single-valued map out of one dimension ``into:`` another.\n\nIts values are labels of ``into``, which is what ``sum(by=)`` and\n``at(by=)`` land terms on::\n\n lookups:\n bus_of: {over: generator, into: bus}\n\nThe map itself is data, and arrives at bind time under the lookup's name.", + "description": "A named single-valued map from one or more key dimensions ``into:`` another.\n\nIts values are labels of ``into``, which is what ``sum(by=)`` and\n``at(by=)`` land terms on. ``over:`` is one dimension or a list \u2014 the\nmap's key columns, in the order the table carries them::\n\n lookups:\n bus_of: {over: generator, into: bus}\n zone_of: {over: [generator, period], into: zone}\n\nThe map itself is data, and arrives at bind time under the lookup's name,\nsingle-valued per key tuple.", "properties": { "description": { "anyOf": [ @@ -237,8 +237,18 @@ "type": "string" }, "over": { - "title": "Over", - "type": "string" + "anyOf": [ + { + "type": "string" + }, + { + "items": { + "type": "string" + }, + "type": "array" + } + ], + "title": "Over" } }, "required": [ diff --git a/src/math_spec/_expression_parser.py b/src/math_spec/_expression_parser.py index 642a9f6a..f94ff0ac 100644 --- a/src/math_spec/_expression_parser.py +++ b/src/math_spec/_expression_parser.py @@ -55,7 +55,12 @@ class NumberNode: @dataclass(frozen=True) class NameNode: - """A bare name whose kind only the schema knows; resolution rewrites every one into a typed node.""" + """A bare name whose kind only the schema knows; resolution rewrites every one into a typed node. + + In a kwarg value it may be dotted — ``by=zone_of.generator`` — which names + a lookup and the key column the operator walks; the grammar admits the + dot nowhere else. + """ name: str @@ -117,19 +122,26 @@ def shown(self) -> str: class LookupNode: """A resolved reference to one or more declared lookups, legal only in a kwarg value. - ``dimension`` is the one every lookup is over — what ``sum`` consumes and - ``at`` produces — and ``into`` the targets, one per name in the order - written; ``sum(x, by=[gen_bus, gen_tech])`` is one grouping, not two. + ``dimension`` is the key column every lookup is walked along — what + ``sum`` consumes and ``at`` produces — and ``into`` the targets, one per + name in the order written; ``sum(x, by=[gen_bus, gen_tech])`` is one + grouping, not two. ``keys`` is each lookup's key columns in declared + order; the ones other than ``dimension`` are joined on, so the operand + must carry them and the operator passes them through. """ names: tuple[str, ...] dimension: str into: tuple[str, ...] + keys: tuple[tuple[str, ...], ...] = () @property def shown(self) -> str: - """The kwarg value as the author wrote it, for an error message.""" - return shown(self.names) + """The kwarg value as the author wrote it, for an error message: dotted where the lookup has several keys.""" + keys = self.keys or tuple((self.dimension,) for _ in self.names) + return shown( + tuple(n if len(k) == 1 else f'{n}.{self.dimension}' for n, k in zip(self.names, keys, strict=True)) + ) @dataclass(frozen=True) @@ -358,12 +370,16 @@ def _build_grammar() -> pp.ParserElement: number = inf_literal | pp.Regex(rf'{REAL}|\d+').set_parse_action(lambda t: NumberNode(float(t[0]))) name = pp.Regex(NAME) + dotted = pp.Regex(rf'{NAME}\.{NAME}') quoted = (pp.QuotedString("'") | pp.QuotedString('"')).set_parse_action(lambda t: KeywordNode(str(t[0]))) - name_list = (pp.Suppress('[') + pp.DelimitedList(name) + pp.Suppress(']')).set_parse_action( + name_list = (pp.Suppress('[') + pp.DelimitedList(dotted | name) + pp.Suppress(']')).set_parse_action( lambda t: NameListNode(tuple(str(x) for x in t)) ) - kwarg = (name + pp.Suppress('=') + (quoted | name_list | arith)).set_parse_action(lambda t: (t[0], t[1])) + dotted_node = dotted.copy().set_parse_action(lambda t: NameNode(str(t[0]))) + kwarg = (name + pp.Suppress('=') + (quoted | name_list | dotted_node | arith)).set_parse_action( + lambda t: (t[0], t[1]) + ) pos_arg = arith arg_list = pp.Optional(pp.DelimitedList(kwarg | pos_arg)) func_call = (name + pp.Suppress('(') + arg_list + pp.Suppress(')')).set_parse_action(_make_func_call) diff --git a/src/math_spec/_where_parser.py b/src/math_spec/_where_parser.py index ea873445..da610888 100644 --- a/src/math_spec/_where_parser.py +++ b/src/math_spec/_where_parser.py @@ -112,7 +112,7 @@ def _build_where_grammar() -> pp.ParserElement: lambda t: _Quoted(t[0]) ) - grouped_by = pp.Suppress(',') + pp.Suppress(pp.Keyword('by')) + pp.Suppress('=') + name + grouped_by = pp.Suppress(',') + pp.Suppress(pp.Keyword('by')) + pp.Suppress('=') + pp.Regex(rf'{NAME}(\.{NAME})?') comparator = pp.one_of(list(get_args(PredicateOperator))) position_call = ( diff --git a/src/math_spec/advice.py b/src/math_spec/advice.py index 9e90b5a6..92edc7aa 100644 --- a/src/math_spec/advice.py +++ b/src/math_spec/advice.py @@ -51,7 +51,7 @@ def _never_an_axis(program: Program) -> list[Advice]: for declaration in (*program.parameters.values(), *program.variables.values(), *program.constraints.values()): reached.update(declaration.dims) reached |= _produced_axes(program) - reached |= {lk.target for _, lk in program.lookups} + reached |= {lk.target for lk in program.lookups.values()} return [ Advice( diff --git a/src/math_spec/dimensions.py b/src/math_spec/dimensions.py index 1939032c..3c044343 100644 --- a/src/math_spec/dimensions.py +++ b/src/math_spec/dimensions.py @@ -161,6 +161,7 @@ def _sum_dims(node: FunctionCallNode, inner: frozenset[str], schema: Spec, conte 'drop the sum, or fix the dim', ) ) + _check_joined(f'sum(by={by.shown})', by, inner, context) collides = sorted(set(by.into) & (inner - {by.dimension})) if collides: raise DimensionError( @@ -185,6 +186,7 @@ def _at_dims(node: FunctionCallNode, inner: frozenset[str], schema: Spec, contex f'{sorted(inner)}). A pullback needs the coarse dims to read *from* — ' f'sum is the direction that produces them.' ) + _check_joined(f'at(by={by.shown})', by, inner, context) if by.dimension in inner - set(by.into): raise DimensionError( f'{context}: at(by={by.shown}) places terms onto ' @@ -224,13 +226,25 @@ def _translation_dims(node: FunctionCallNode, inner: frozenset[str], schema: Spe if partition.dimension != over.name: raise DimensionError( f'{context}: {node.name}(over={over.name}, by={partition.shown}) walks ' - f"'{over.name}' but groups by a lookup over '{partition.dimension}'. No row of " + f"'{over.name}' but groups along '{partition.dimension}'. No row of " f"'{over.name}' carries it, so no coordinate has a neighbour inside a group — " - f"partition by a lookup over '{over.name}'." + f"partition by a lookup keyed by '{over.name}', walked along it." ) + _check_joined(f'{node.name}(over={over.name}, by={partition.shown})', partition, inner, context) return inner +def _check_joined(call: str, by: LookupNode, inner: frozenset[str], context: str) -> None: + """A lookup's keys other than the one walked are joined on, so the operand carries every one.""" + joined = {k for keys in by.keys for k in keys if k != by.dimension} + if missing := sorted(joined - inner): + raise DimensionError( + f'{context}: {call} joins on {missing}, which the expression does not carry ' + f'(dims {sorted(inner)}). A lookup keyed by several dimensions is walked along one and ' + f'read at the others — index the operand by them, or walk a different key.' + ) + + #: The dim rule of each built-in, by name. _CALL_RULES: dict[str, Callable[[FunctionCallNode, frozenset[str], Spec, str], frozenset[str]]] = { 'sum': _sum_dims, diff --git a/src/math_spec/lowering.py b/src/math_spec/lowering.py index 4be53d60..687e5fe3 100644 --- a/src/math_spec/lowering.py +++ b/src/math_spec/lowering.py @@ -143,7 +143,9 @@ def lower_program(expanded: _ExpandedSpec) -> program.Program: dimensions = { dname: program.DimensionDeclaration( tuple( - program.LookupDeclaration(lname, lk.into) for lname, lk in expanded.lookups.items() if lk.over == dname + program.LookupDeclaration(lname, lk.into, lk.keys) + for lname, lk in expanded.lookups.items() + if dname in lk.keys ), ddef.dtype, ) @@ -272,7 +274,9 @@ def sum(self, node: FunctionCallNode) -> program.ExpressionNode: assert isinstance(over_node, DimensionNode), 'resolution refuses an over= that is not a dimension' return program.Sum(operand, (over_node.name,)) assert isinstance(by_node, LookupNode), 'resolution refuses a by= that is not a lookup' - return program.GroupSum(operand, over=by_node.dimension, coordinate=by_node.names, into=by_node.into) + return program.GroupSum( + operand, over=by_node.dimension, coordinate=by_node.names, into=by_node.into, keys=by_node.keys + ) def at(self, node: FunctionCallNode) -> program.ExpressionNode: """``at(x, by=lookup)`` — the adjoint of :meth:`sum`'s ``by=`` form.""" @@ -283,6 +287,7 @@ def at(self, node: FunctionCallNode) -> program.ExpressionNode: over=by_node.dimension, coordinate=by_node.names, into=by_node.into, + keys=by_node.keys, ) def sum_back(self, node: FunctionCallNode) -> program.ExpressionNode: diff --git a/src/math_spec/model.py b/src/math_spec/model.py index d5246b97..dfbd9fbc 100644 --- a/src/math_spec/model.py +++ b/src/math_spec/model.py @@ -154,23 +154,31 @@ def _also_written_as( class LookupBlock(_StrictBlock): - """A named single-valued map out of one dimension ``into:`` another. + """A named single-valued map from one or more key dimensions ``into:`` another. Its values are labels of ``into``, which is what ``sum(by=)`` and - ``at(by=)`` land terms on:: + ``at(by=)`` land terms on. ``over:`` is one dimension or a list — the + map's key columns, in the order the table carries them:: lookups: bus_of: {over: generator, into: bus} + zone_of: {over: [generator, period], into: zone} - The map itself is data, and arrives at bind time under the lookup's name. + The map itself is data, and arrives at bind time under the lookup's name, + single-valued per key tuple. """ _label: ClassVar[str] = 'a lookup declaration' - over: str + over: str | list[str] into: str description: str | None = None + @property + def keys(self) -> tuple[str, ...]: + """The key columns in declared order, however ``over:`` was written.""" + return (self.over,) if isinstance(self.over, str) else tuple(self.over) + class DimensionBlock(_StrictBlock): """A declared dimension, and the dtype its coordinates must be. @@ -675,9 +683,9 @@ class Spec(_StrictBlock): piecewise: dict[str, PiecewiseBlock] = {} sos: dict[str, SosBlock] = {} - def lookups_of(self, dimension: str) -> dict[str, str]: - """The lookups over *dimension*: name -> the dim they map into.""" - return {n: lk.into for n, lk in self.lookups.items() if lk.over == dimension} + def lookups_of(self, dimension: str) -> dict[str, LookupBlock]: + """The lookups keyed by *dimension*, by name.""" + return {n: lk for n, lk in self.lookups.items() if dimension in lk.keys} @classmethod @override @@ -807,19 +815,24 @@ def _frame_dimensions(self) -> Iterator[str]: ) def _lookup_targets(self) -> Iterator[str]: - """A lookup is over a declared dimension and maps into a different declared one.""" + """A lookup is keyed by declared dimensions, each once, and maps into another declared one.""" for lname, lk in self.lookups.items(): - if lk.over not in self.dimensions: - yield (undeclared_dimension('Lookup', lname, lk.over)) - if lk.into is not None: - if lk.into not in self.dimensions: - yield ( - f"Lookup '{lname}' targets undeclared dimension '{lk.into}'. " - f"Declare it under 'dimensions:' — the target is what the " - f'lookup values are checked against.' - ) - elif lk.into == lk.over: - yield (f"Lookup '{lname}' maps '{lk.over}' into itself. A lookup maps into a different dimension.") + if not lk.keys: + yield f"Lookup '{lname}' has no key dimension: 'over:' names the dimension(s) the map is keyed by." + yield from (undeclared_dimension('Lookup', lname, d) for d in lk.keys if d not in self.dimensions) + yield from ( + f"Lookup '{lname}' names '{d}' twice under 'over:'. A map has each key dimension once." + for d, count in Counter(lk.keys).items() + if count > 1 + ) + if lk.into not in self.dimensions: + yield ( + f"Lookup '{lname}' targets undeclared dimension '{lk.into}'. " + f"Declare it under 'dimensions:' — the target is what the " + f'lookup values are checked against.' + ) + elif lk.into in lk.keys: + yield (f"Lookup '{lname}' maps '{lk.into}' into itself. A lookup maps into a different dimension.") def _bound_names(self) -> Iterator[str]: """A named bound is a numeric parameter.""" diff --git a/src/math_spec/program.py b/src/math_spec/program.py index 1ade8d87..a6557f1a 100644 --- a/src/math_spec/program.py +++ b/src/math_spec/program.py @@ -262,19 +262,22 @@ class Sum(Expression): @dataclass(frozen=True) class GroupSum(Expression): - """Sum ``operand`` through coordinates declared on dim ``over``. - - ``coordinate`` names lookups carried by dim ``over`` whose values are - labels of the matching dim in ``into``; the result replaces ``over`` with - all of them. The two tuples are the same length and their order pairs - them: several coordinates are one grouping into a product of targets, - consumed in a single join. + """Sum ``operand`` through coordinates keyed by dim ``over``. + + ``coordinate`` names lookups whose values are labels of the matching dim + in ``into``; the result replaces ``over`` with all of them. The tuples + are the same length and their order pairs them: several coordinates are + one grouping into a product of targets, consumed in a single join. + ``keys`` is each coordinate's key columns in declared order — ``over`` + among them — and the join keys on all of them: the operand carries every + key but ``over``, and they survive in the result. """ operand: ExpressionNode over: str coordinate: tuple[str, ...] into: tuple[str, ...] + keys: tuple[tuple[str, ...], ...] = () @dataclass(frozen=True) @@ -283,13 +286,16 @@ class At(Expression): Same mapping table, walked the other way: ``GroupSum`` consumes ``over`` and produces ``into``, this consumes ``into`` and produces ``over``. The - join fans out, many ``over`` labels sharing one ``into`` tuple. + join fans out, many ``over`` labels sharing one ``into`` tuple — at each + coordinate of the other ``keys``, which the operand carries and the + result keeps. """ operand: ExpressionNode over: str coordinate: tuple[str, ...] into: tuple[str, ...] + keys: tuple[tuple[str, ...], ...] = () @dataclass(frozen=True) @@ -438,21 +444,24 @@ def children(expression: ExpressionNode) -> tuple[ExpressionNode, ...]: class LookupDeclaration(NamedTuple): - """One declared lookup over a dimension. + """One declared lookup: a map from its ``keys`` into ``target``. Its values are labels of ``target``, checked for containment once the dim tables exist — which keeps a mistyped label from silently dropping its terms in the join that places them — and it is what ``sum(by=)`` lands - terms on. + terms on. ``keys`` is the key dimensions in the order the table carries + them; the map is single-valued per key tuple. An operator walks one key + and joins on the rest. """ name: str target: str + keys: tuple[str, ...] @dataclass(frozen=True) class DimensionDeclaration: - """A dimension and the lookups its labels carry.""" + """A dimension and the lookups keyed by it — by it alone, or by it among others.""" lookups: tuple[LookupDeclaration, ...] = () #: What the labels are, as the file declares them. A dimension is read from @@ -463,7 +472,7 @@ class DimensionDeclaration: @property def targets(self) -> dict[str, str]: - """Each map over the dimension, to the dimension its values are labels of. + """Each map keyed by the dimension, to the dimension its values are labels of. The question every consumer of a ``by=`` asks, and asked here so it has one answer: an operator grouping through a lookup names the target as @@ -894,9 +903,9 @@ def dimension(self, name: str) -> DimensionDeclaration: return _declared(self.dimensions, name, 'dimension') @property - def lookups(self) -> tuple[tuple[str, LookupDeclaration], ...]: - """Every lookup in the program, with the dimension it is over.""" - return tuple((dimension, lk) for dimension, d in self.dimensions.items() for lk in d.lookups) + def lookups(self) -> dict[str, LookupDeclaration]: + """Every lookup in the program by name, each once — a lookup keyed by two dimensions sits under both.""" + return {lk.name: lk for d in self.dimensions.values() for lk in d.lookups} def parameter(self, name: str) -> ParameterDeclaration: return _declared(self.parameters, name, 'parameter') @@ -1057,44 +1066,46 @@ class DimensionPositionNode: """Compare where a row sits along a dimension against a position — ``position(snapshot) == 0``. Both sides are integers, negative counting from the end. With ``by`` the - position is counted within each group the lookup makes. + position is counted within each group the lookup makes, at each coordinate + of the lookup's other ``keys`` — ``name`` is one of them. """ name: str op: PredicateOperator position: int by: str | None = None + keys: tuple[str, ...] = () @dataclass(frozen=True) class LookupComparisonNode: """Compare a lookup's values against a literal — ``period_of == 2030``. - ``over`` is the dimension the lookup maps out of. + ``keys`` is the lookup's key dimensions; the leaf is read at all of them. """ name: str - over: str + keys: tuple[str, ...] op: PredicateOperator value: float | str | datetime.date @dataclass(frozen=True) class LookupPairComparisonNode: - """Compare two lookups over one dimension — ``from != to``, row by row on that dimension's table.""" + """Compare two lookups with the same ``keys`` — ``from != to``, row by row on that key table.""" name: str other: str - over: str + keys: tuple[str, ...] op: PredicateOperator @dataclass(frozen=True) class LookupDefinedNode: - """True where the named lookup has a value — a null says the label belongs to no group.""" + """True where the named lookup has a value — a null says the key belongs to no group.""" name: str - over: str + keys: tuple[str, ...] @dataclass(frozen=True) @@ -1191,19 +1202,21 @@ def _atom_dims(atom: TypedPredicateNode) -> frozenset[str]: A parameter or variable leaf carries its own dims off the declaration; a comparison on a dimension is read through that dimension, and a lookup - through the dimension it maps out of — the dim it leaves, not the one it - lands in. Separate from the union because the load-time frame check - reports per leaf. Closed by ``assert_never``: a predicate node added - without a reading is a type error here, at the one place that has to grow - a branch, rather than a wrong dim set at the first model to use it. + through its key dimensions — the dims it leaves, not the one it lands in. + Separate from the union because the load-time frame check reports per + leaf. Closed by ``assert_never``: a predicate node added without a reading + is a type error here, at the one place that has to grow a branch, rather + than a wrong dim set at the first model to use it. """ match atom: case ParameterComparisonNode() | ParameterDefinedNode() | VariableDefinedNode(): return frozenset(atom.dims) - case DimensionComparisonNode() | DimensionPositionNode(): + case DimensionComparisonNode(): return frozenset({atom.name}) + case DimensionPositionNode(): + return frozenset({atom.name, *atom.keys}) case LookupComparisonNode() | LookupPairComparisonNode() | LookupDefinedNode(): - return frozenset({atom.over}) + return frozenset(atom.keys) case _: assert_never(atom) diff --git a/src/math_spec/resolution.py b/src/math_spec/resolution.py index 7dca8cee..eaee8b76 100644 --- a/src/math_spec/resolution.py +++ b/src/math_spec/resolution.py @@ -103,7 +103,7 @@ def __init__( variables: Iterable[str], parameters: Iterable[str], dimensions: Iterable[str], - lookups: Mapping[str, tuple[str, str]], + lookups: Mapping[str, tuple[tuple[str, ...], str]], dtypes: Mapping[str, DeclaredDtype], leaf_dims: Mapping[str, tuple[str, ...]], constraints: Iterable[str], @@ -118,8 +118,8 @@ def __init__( #: name -> declared dtype, for dimensions, parameters and lookups alike; #: what a where comparison checks its literal against. self.dtypes: dict[str, DeclaredDtype] = dict(dtypes) - #: lookup name -> ``(over, into)``. - self.lookups: dict[str, tuple[str, str]] = dict(lookups) + #: lookup name -> ``(keys, into)``. + self.lookups: dict[str, tuple[tuple[str, ...], str]] = dict(lookups) #: parameter or variable name -> the dims it is read through — #: parameters by their ``dims``, variables by their frame. Stamped onto #: each leaf a where names, the way a lookup leaf carries ``over``. @@ -135,7 +135,7 @@ def of(cls, schema: Spec) -> Namespace: schema.variables, schema.parameters, schema.dimensions, - {n: (lk.over, lk.into) for n, lk in schema.lookups.items()}, + {n: (lk.keys, lk.into) for n, lk in schema.lookups.items()}, { **{p: pd.dtype for p, pd in schema.parameters.items()}, **{d: dd.dtype for d, dd in schema.dimensions.items()}, @@ -160,8 +160,8 @@ def kind(self, name: str) -> DeclarationKind | None: return 'lookup' return None - def over_of(self, lookup: str) -> str: - """The dimension *lookup* maps out of.""" + def keys_of(self, lookup: str) -> tuple[str, ...]: + """The key dimensions of *lookup*, in declared order.""" return self.lookups[lookup][0] def into_of(self, lookup: str) -> str: @@ -559,32 +559,36 @@ def _dual(self, node: FunctionCallNode) -> ArithmeticNode: return DualNode(value.name) def _lookup_ref(self, value: ArithmeticNode, operator: str, key: str) -> ArithmeticNode: - """An operator kwarg whose *value* must name lookups. + """An operator kwarg whose *value* must name lookups, each with the key column it walks. A lookup carries its own dimensions, so nothing else in the call is consulted: the names alone decide both the dim the operator consumes and - the ones it produces. A bracketed list is one grouping through several - maps at once rather than a composition of groupings, so its members must - share the dim they are over and must not target the same dim twice. + the ones it produces. ``name.column`` says which key column is walked, + and is required exactly where the lookup has more than one. A + bracketed list is one grouping through several maps at once rather + than a composition of groupings, so its members must walk the same + dimension and must not target the same dim twice. """ - names = names_in(value) - if not names: + written = names_in(value) + if not written: self.errors.append(f'{self.context}: {operator}({key}=...) must name a lookup.') return value ns = self.ns - named = [self._not_a_lookup(name, operator, key) for name in names] - if any(problem is not None for problem in named): - self.errors.extend(problem for problem in named if problem is not None) + refs = [self._walked(spelling, operator, key) for spelling in written] + if any(isinstance(ref, str) for ref in refs): + self.errors.extend(ref for ref in refs if isinstance(ref, str)) return value + walked = [ref for ref in refs if not isinstance(ref, str)] + names = tuple(name for name, _ in walked) - over = {ns.over_of(name) for name in names} + over = {dim for _, dim in walked} if len(over) > 1: self.errors.append( - f'{self.context}: {operator}({key}={shown(names)}) groups through lookups over ' - f'different dimensions ({", ".join(f"{n} over {ns.over_of(n)}" for n in names)}). ' - f'One grouping consumes one dimension, so every lookup in the list must be ' - f'over the same one — group through them in turn instead, one call each.' + f'{self.context}: {operator}({key}={shown(written)}) groups through lookups along ' + f'different dimensions ({", ".join(f"{n} along {d}" for n, d in walked)}). ' + f'One grouping consumes one dimension, so every lookup in the list must walk ' + f'the same one — group through them in turn instead, one call each.' ) return value @@ -592,13 +596,42 @@ def _lookup_ref(self, value: ArithmeticNode, operator: str, key: str) -> Arithme repeated = sorted({t for t in targets if targets.count(t) > 1}) if repeated: self.errors.append( - f'{self.context}: {operator}({key}={shown(names)}) targets {repeated} more than once. ' + f'{self.context}: {operator}({key}={shown(written)}) targets {repeated} more than once. ' f'Each lookup in the list produces its own dimension, so two that land on the ' f'same one would need it twice — drop one, or group into a dimension of its own.' ) return value - return LookupNode(names, dimension=next(iter(over)), into=targets) + return LookupNode( + names, dimension=next(iter(over)), into=targets, keys=tuple(ns.keys_of(name) for name in names) + ) + + def _walked(self, spelling: str, operator: str, key: str) -> tuple[str, str] | str: + """``lookup`` or ``lookup.column`` as the lookup and the key dimension it walks, or the refusal. + + The dot is required exactly where the lookup has several key columns: + with one there is nothing to choose, with several the file has to say, + because the operator consumes one and joins on the rest. + """ + ns, context = self.ns, self.context + name, _, column = spelling.partition('.') + if (problem := self._not_a_lookup(name, operator, key)) is not None: + return problem + keys = ns.keys_of(name) + if not column: + if len(keys) == 1: + return name, keys[0] + return ( + f"{context}: {operator}({key}={name}): '{name}' is keyed by {list(keys)}, and the call has to " + f'say which key {operator} walks — the others are joined on and kept. Write ' + f'{" or ".join(f"{key}={name}.{k}" for k in keys)}.' + ) + if column not in keys: + return ( + f"{context}: {operator}({key}={spelling}): '{column}' is not a key of '{name}', which is keyed by " + f'{list(keys)}. The dot names the key column the operator walks.' + ) + return name, column def _not_a_lookup(self, name: str, operator: str, key: str) -> str | None: """Why *name* is not a lookup; ``None`` where it is one.""" @@ -660,7 +693,7 @@ def _where_name(self, node: UnresolvedNameNode) -> WhereNode | UnresolvedWhereNo f'Remove it, or compare it: where: "{node.name} > 0".' ) case 'lookup': - return LookupDefinedNode(node.name, ns.over_of(node.name)) + return LookupDefinedNode(node.name, ns.keys_of(node.name)) case 'variable': if node.name == self.self_variable: self.errors.append( @@ -673,7 +706,7 @@ def _where_name(self, node: UnresolvedNameNode) -> WhereNode | UnresolvedWhereNo return node def _position(self, node: UnresolvedPositionNode) -> DimensionPositionNode | UnresolvedPositionNode: - """``position(dim[, by=lookup]) i``: the name a dimension, ``by=`` a lookup over it.""" + """``position(dim[, by=lookup]) i``: the name a dimension, ``by=`` a lookup keyed by it.""" ns, context = self.ns, self.context if node.dimension not in ns.dimensions: self.errors.append( @@ -685,14 +718,27 @@ def _position(self, node: UnresolvedPositionNode) -> DimensionPositionNode | Unr if node.by is None: return DimensionPositionNode(node.dimension, node.op, node.position, node.by) call = f'position({node.dimension}, by={node.by})' - if ns.kind(node.by) != 'lookup': + name, _, column = node.by.partition('.') + if ns.kind(name) != 'lookup': + self.errors.append( + f"{context}: '{call}' groups by '{name}', which is {_declared_as(ns, name)}. " + f'``by=`` takes a lookup keyed by that dimension. ' + f'{did_you_mean(name, ns.lookups, label="Lookups")}' + ) + return node + keys = ns.keys_of(name) + if column and column not in keys: + self.errors.append( + f"{context}: '{call}': '{column}' is not a key of '{name}', which is keyed by {list(keys)}." + ) + return node + if not column and len(keys) > 1: self.errors.append( - f"{context}: '{call}' groups by '{node.by}', which is {_declared_as(ns, node.by)}. " - f'``by=`` takes a lookup over that dimension. ' - f'{did_you_mean(node.by, ns.lookups, label="Lookups")}' + f"{context}: '{call}': '{name}' is keyed by {list(keys)}, and the call has to say which key " + f'position counts along — by={name}.{node.dimension}.' ) return node - over = ns.over_of(node.by) + over = column or keys[0] if over != node.dimension: self.errors.append( f"{context}: '{call}' counts positions along '{node.dimension}' but groups by a " @@ -700,7 +746,7 @@ def _position(self, node: UnresolvedPositionNode) -> DimensionPositionNode | Unr f"position within a group to name — group by a lookup over '{node.dimension}'." ) return node - return DimensionPositionNode(node.dimension, node.op, node.position, node.by) + return DimensionPositionNode(node.dimension, node.op, node.position, name, keys) def _comparison(self, node: UnresolvedComparisonNode) -> WhereNode | UnresolvedWhereNode: """``name literal``, or the one structural form ``lookup lookup``.""" @@ -711,7 +757,7 @@ def _comparison(self, node: UnresolvedComparisonNode) -> WhereNode | UnresolvedW if (refusal := _lookup_pair_error(context, node, value, ns)) is not None: self.errors.append(refusal) return node - return LookupPairComparisonNode(node.name, value, ns.over_of(node.name), node.op) + return LookupPairComparisonNode(node.name, value, ns.keys_of(node.name), node.op) self.errors.append(_declared_rhs_error(context, node, value, rhs_kind)) return node @@ -732,7 +778,7 @@ def _comparison(self, node: UnresolvedComparisonNode) -> WhereNode | UnresolvedW case 'dimension': return DimensionComparisonNode(node.name, node.op, value) case 'lookup': - return LookupComparisonNode(node.name, ns.over_of(node.name), node.op, value) + return LookupComparisonNode(node.name, ns.keys_of(node.name), node.op, value) case 'variable': self.errors.append( f"{context}: where references variable '{node.name}'. A where " @@ -883,18 +929,18 @@ def _declared_rhs_error(context: str, node: UnresolvedComparisonNode, value: str def _lookup_pair_error(context: str, node: UnresolvedComparisonNode, other: str, ns: Namespace) -> str | None: """Why two lookups may not be compared, or ``None`` where they may. - They must map out of the same dimension, or no row carries both; and into - the same one, or no value of one is ever a value of the other. Both wrong - answers are silent, and a build's data library decides which one. + They must have the same keys, or no row carries both; and map into the + same dimension, or no value of one is ever a value of the other. Both + wrong answers are silent, and a build's data library decides which one. """ comparison = f"'{node.name} {node.op} {other}'" - left_over, right_over = ns.over_of(node.name), ns.over_of(other) - if left_over != right_over: + left_keys, right_keys = ns.keys_of(node.name), ns.keys_of(other) + if set(left_keys) != set(right_keys): return ( - f'{context}: {comparison} compares lookups over different dimensions ' - f"('{left_over}' and '{right_over}') — there is no row carrying both, so the " - f'comparison has nothing to test. Two lookups may be compared only where they ' - f'map out of the same dimension.' + f'{context}: {comparison} compares lookups keyed by different dimensions ' + f"('{node.name}' by {list(left_keys)}, '{other}' by {list(right_keys)}) — there is no row " + f'carrying both, so the comparison has nothing to test. Two lookups may be compared only ' + f'where they have the same keys.' ) left, right = ns.into_of(node.name), ns.into_of(other) if left != right: diff --git a/src/math_spec/typesetting/walk.py b/src/math_spec/typesetting/walk.py index 2df0d4a2..3bcbaf23 100644 --- a/src/math_spec/typesetting/walk.py +++ b/src/math_spec/typesetting/walk.py @@ -273,9 +273,20 @@ def _translation(self, step: _Step) -> str: self.noticed.grouped = True return self.format.superscript(operator, step.within) - def _lookup(self, name: str, index: str) -> str: - """A coordinate map applied to an index: ``bus(g)``.""" - return self.format.apply(self.format.upright(name), index) + def _lookup(self, name: str, keys: Iterable[str], walked: str, index: str, ctx: _Context | None = None) -> str: + """A coordinate map applied to its keys in declared order: ``bus(g)``, or ``zone(g, p)``. + + The key *walked* reads *index* — a reduction's dummy, or the row's own + subscript — and every other key reads under *ctx*, as the row being + written reads it: pulled back or translated where the row is. + """ + + def at(dim: str) -> str: + if dim == walked: + return index + return ctx.subscript(dim) if ctx is not None else self.symbols.index[dim] + + return self.format.apply(self.format.upright(name), self.format.joined([at(d) for d in keys], '')) def _context(self, frame: Iterable[str] = ()) -> _Context: return _Context(self, bound=tuple(frame)) @@ -412,16 +423,16 @@ def _call(self, node: FunctionCallNode, ctx: _Context) -> tuple[str, int]: if node.name == 'at': by = node.kwargs['by'] assert isinstance(by, LookupNode) - for name, into in zip(by.names, by.into, strict=True): - ctx = ctx.pulled_back(into, self._lookup(name, ctx.subscript(by.dimension))) + for name, into, keys in zip(by.names, by.into, by.keys, strict=True): + ctx = ctx.pulled_back(into, self._lookup(name, keys, by.dimension, ctx.subscript(by.dimension), ctx)) return self._arithmetic(node.args[0], ctx) if (by := node.kwargs.get('by')) is not None: assert isinstance(by, LookupNode) dummy, inner = ctx.reducing(by.dimension) conditions = [ - f'{self._lookup(name, dummy)} {self._op("equal")} {ctx.subscript(into)}' - for name, into in zip(by.names, by.into, strict=True) + f'{self._lookup(name, keys, by.dimension, dummy, ctx)} {self._op("equal")} {ctx.subscript(into)}' + for name, into, keys in zip(by.names, by.into, by.keys, strict=True) ] domain = ( f'{self._membership(by.dimension, dummy)} {self._op("such_that")} ' @@ -450,7 +461,7 @@ def _group(self, by: ArithmeticNode | None, dim: str) -> str: if by is None: return '' assert isinstance(by, LookupNode) - return self._lookup(by.names[0], self.symbols.index[dim]) + return self._lookup(by.names[0], by.keys[0], dim, self.symbols.index[dim]) def _width(self, node: ArithmeticNode) -> str: """``sum_back``'s ``within=``: a number, or a parameter's own symbol. @@ -532,23 +543,22 @@ def _where(self, node: WhereNode, ctx: _Context) -> tuple[str, int]: ) if isinstance(node, DimensionPositionNode): - grouping = None if node.by is None else self._lookup(node.by, ctx.subscript(node.name)) + grouping = None if node.by is None else self._lookup(node.by, node.keys, '', '', ctx) place = self._position(ctx.subscript(node.name), grouping) ordinal = self._ordinal(node.name, node.position, grouping) return f'{place} {self._op(_PREDICATES[node.op])} {ordinal}', comparison if isinstance(node, LookupComparisonNode): - applied = self._lookup(node.name, ctx.subscript(node.over)) + applied = self._lookup(node.name, node.keys, '', '', ctx) return f'{applied} {self._op(_PREDICATES[node.op])} {self._literal(node.value)}', comparison if isinstance(node, LookupPairComparisonNode): - index = ctx.subscript(node.over) - left = self._lookup(node.name, index) - right = self._lookup(node.other, index) + left = self._lookup(node.name, node.keys, '', '', ctx) + right = self._lookup(node.other, node.keys, '', '', ctx) return f'{left} {self._op(_PREDICATES[node.op])} {right}', comparison if isinstance(node, LookupDefinedNode): - applied = self._lookup(node.name, ctx.subscript(node.over)) + applied = self._lookup(node.name, node.keys, '', '', ctx) return f'{applied} {self.format.prose(" is defined")}', comparison if isinstance(node, NotNode): @@ -824,6 +834,10 @@ def _over(self, dims: list[str]) -> str: product = self.format.joined([self.symbols.set[d] for d in dims], self._op('times')) return f' over {self.format.math(product)}' + def _domain(self, keys: Iterable[str]) -> str: + """A map's domain in the legend: the product of its key sets, in declared order.""" + return self.format.joined([self.symbols.set[d] for d in keys], self._op('times')) + def _coords(self, dim: str, noticed: Noticed) -> str: """The dimension's carried structure: each lookup as the map it is (``bus_of: G ↦ B``). @@ -838,8 +852,8 @@ def _coords(self, dim: str, noticed: Noticed) -> str: if targeted: maps = self.format.joined( [ - f'{self.format.upright(c)}: {self.symbols.set[dim]} {self._op("maps_to")} {self.symbols.set[target]}' - for c, target in targeted.items() + f'{self.format.upright(c)}: {self._domain(lk.keys)} {self._op("maps_to")} {self.symbols.set[lk.into]}' + for c, lk in targeted.items() ], '', ) @@ -885,7 +899,7 @@ def translation_notes(self, noticed: Noticed) -> list[str]: f'at that boundary is built and carries {self.format.math("v")} rather than being dropped.' ) if noticed.grouped: - applied = self._lookup('lookup', 't') + applied = self._lookup('lookup', ('t',), 't', 't') counted = self.format.math(f't {self.format.superscript(self._op("cyclic_minus"), applied)} k') note = ( f'{counted} denotes a translation counted inside the group a lookup puts {self.format.math("t")} ' @@ -914,7 +928,7 @@ def position_notes(self, noticed: Noticed) -> list[str]: f'labels and {place} against positions.' ) if 'grouped' in noticed.positions: - applied = self._lookup('lookup', 't') + applied = self._lookup('lookup', ('t',), 't', 't') grouped = self.format.math(self.format.apply(self.format.subscript(self._op('position'), [applied]), 't')) group = self.format.math(self.format.subscript(self.format.script('T'), [applied])) notes.append( diff --git a/tests/test_dimensions.py b/tests/test_dimensions.py index c96fb1b7..b8156c22 100644 --- a/tests/test_dimensions.py +++ b/tests/test_dimensions.py @@ -30,15 +30,19 @@ 'snapshot': {'dtype': 'int'}, 'generator': {'dtype': 'str'}, 'bus': {'dtype': 'str'}, + 'zone': {'dtype': 'str'}, }, 'lookups': { 'gen_bus': {'over': 'generator', 'into': 'bus'}, 'snap_bus': {'over': 'snapshot', 'into': 'bus'}, + 'gen_zone': {'over': ['generator', 'snapshot'], 'into': 'zone'}, }, 'parameters': { 'p_max': {'dims': ['generator']}, 'cost': {'dims': ['generator']}, 'load': {'dims': ['snapshot', 'bus']}, + 'zone_cap': {'dims': ['zone']}, + 'zone_load': {'dims': ['snapshot', 'zone']}, 'spinup': {'dims': ['generator'], 'dtype': 'int'}, 'horizon': {'dims': ['snapshot'], 'dtype': 'int'}, 'bus_lead': {'dims': ['bus'], 'dtype': 'int'}, @@ -100,6 +104,27 @@ def namespace() -> Namespace: id='a-by-makes-a-width-over-another-dim-readable-one-window-per-group', ), pytest.param('p + 1', {'snapshot', 'generator'}, id='a-scalar-broadcasts'), + pytest.param( + 'sum(p, by=gen_zone.generator)', + {'snapshot', 'zone'}, + id='a-two-key-lookup-consumes-the-key-it-walks-and-keeps-the-other', + ), + pytest.param( + 'sum(p, by=gen_zone.snapshot)', + {'generator', 'zone'}, + id='the-same-table-walked-along-its-other-key', + ), + pytest.param( + 'at(zone_load, by=gen_zone.generator)', + {'snapshot', 'generator'}, + id='its-pullback-keeps-the-joined-key-too', + ), + pytest.param( + "shift(p, over=generator, offset=1, edge='wrap', by=gen_zone.generator)", + {'snapshot', 'generator'}, + id='a-partition-along-one-key-joined-on-the-other', + ), + pytest.param('sum(p, by=gen_bus.generator)', {'snapshot', 'bus'}, id='the-dot-is-legal-on-a-one-key-lookup'), ], ) def test_dim_inference(expr, expected): @@ -187,6 +212,26 @@ def test_a_bare_name_reaches_the_variable_a_dual_the_same_named_constraint(): r"varies over \['bus'\], which that coordinate does not carry", id='a-named-offset-is-read-where-the-expression-has-a-coordinate', ), + pytest.param( + 'sum(cost, by=gen_zone.generator)', + r"sum\(by=gen_zone.generator\) joins on \['snapshot'\], which the expression does not carry", + id='a-grouped-sum-needs-the-keys-it-joins-on', + ), + pytest.param( + 'at(zone_cap, by=gen_zone.generator)', + r"at\(by=gen_zone.generator\) joins on \['snapshot'\]", + id='a-pullback-needs-the-keys-it-joins-on', + ), + pytest.param( + "shift(cost, over=generator, offset=1, edge='wrap', by=gen_zone.generator)", + r"by=gen_zone.generator\) joins on \['snapshot'\]", + id='a-partition-needs-the-keys-it-joins-on', + ), + pytest.param( + "shift(p, over=generator, offset=1, edge='wrap', by=gen_zone.snapshot)", + r"walks 'generator' but groups along 'snapshot'", + id='a-partition-walks-the-key-the-shift-walks', + ), ], ) def test_an_ill_dimensioned_expression_is_rejected(expr, match): @@ -328,6 +373,13 @@ def test_a_zero_step_vacates_nothing_and_needs_no_edge(self): pytest.param('snapshot == 0', {'snapshot'}, id='a-dimension-through-itself'), pytest.param('position(snapshot) == 0', {'snapshot'}, id='a-position-through-the-axis-it-counts'), pytest.param('snap_bus == "b1"', {'snapshot'}, id='a-lookup-through-the-dim-it-maps-out-of'), + pytest.param('gen_zone == "z1"', {'generator', 'snapshot'}, id='a-two-key-lookup-through-both-keys'), + pytest.param('gen_zone', {'generator', 'snapshot'}, id='a-bare-two-key-lookup-the-same'), + pytest.param( + 'position(generator, by=gen_zone.generator) == 0', + {'generator', 'snapshot'}, + id='a-position-within-a-group-of-a-two-key-lookup-reads-both-keys', + ), pytest.param('p_max > 0 AND snapshot == 0', {'generator', 'snapshot'}, id='a-conjunction-reads-both-sides'), pytest.param('NOT p_max > 0', {'generator'}, id='a-negation-reads-what-it-negates'), pytest.param('False', set(), id='a-literal-reads-nothing'), diff --git a/tests/test_lowering.py b/tests/test_lowering.py index 3bab6500..b9e1c597 100644 --- a/tests/test_lowering.py +++ b/tests/test_lowering.py @@ -404,22 +404,22 @@ def test_a_power_lowers_to_a_node_of_its_own(dispatch_schema): pytest.param('sum(q, over=h)', Sum(Variable('q'), ('h',)), id='an-over-consumes-the-dim-it-names'), pytest.param( 'sum(p, by=lk)', - GroupSum(Variable('p'), over='g', coordinate=('lk',), into=('h',)), + GroupSum(Variable('p'), over='g', coordinate=('lk',), into=('h',), keys=(('g',),)), id='a-grouped-sum-names-the-dim-it-consumes-and-the-one-it-lands-on', ), pytest.param( 'sum(p, by=[lk])', - GroupSum(Variable('p'), over='g', coordinate=('lk',), into=('h',)), + GroupSum(Variable('p'), over='g', coordinate=('lk',), into=('h',), keys=(('g',),)), id='a-one-element-list-is-the-plain-form', ), pytest.param( 'sum(p, by=[lk, lk2])', - GroupSum(Variable('p'), over='g', coordinate=('lk', 'lk2'), into=('h', 'z')), + GroupSum(Variable('p'), over='g', coordinate=('lk', 'lk2'), into=('h', 'z'), keys=(('g',), ('g',))), id='two-coordinates-are-one-grouping-with-paired-tuples', ), pytest.param( 'at(r, by=lk)', - At(Variable('r'), over='g', coordinate=('lk',), into=('h',)), + At(Variable('r'), over='g', coordinate=('lk',), into=('h',), keys=(('g',),)), id='a-pullback-walks-the-same-table-back', ), pytest.param( @@ -465,6 +465,52 @@ def test_a_construct_lowers_to_its_node(shapes_schema, expression, expected): assert lowered == expected, 'the whole frozen node, so no field is asserted by omission' +def test_a_two_key_lookup_lowers_with_its_keys_and_the_one_it_walks(): + """Every node reading the lookup carries its keys in declared order, so a consumer joins on the right columns.""" + program = to_program( + { + 'dimensions': {'snapshot': {'dtype': 'int'}, 'generator': {}, 'zone': {}}, + 'lookups': {'zone_of': {'over': ['generator', 'snapshot'], 'into': 'zone'}}, + 'parameters': {'price': {'dims': ['snapshot', 'zone']}}, + 'variables': { + 'p': {'foreach': ['snapshot', 'generator'], 'where': "zone_of == 'A' AND zone_of"}, + 'first': { + 'foreach': ['snapshot', 'generator'], + 'where': 'position(generator, by=zone_of.generator) == 0', + }, + }, + 'constraints': { + 'zonal': {'foreach': ['snapshot', 'zone'], 'expression': 'sum(p, by=zone_of.generator) <= 1'}, + 'priced': {'foreach': ['snapshot', 'generator'], 'expression': 'p <= at(price, by=zone_of.generator)'}, + 'history': {'foreach': ['generator', 'zone'], 'expression': 'sum(p, by=zone_of.snapshot) <= 1'}, + }, + } + ) + + declared = LookupDeclaration('zone_of', 'zone', ('generator', 'snapshot')) + assert program.dimension('generator').lookups == (declared,), 'the lookup sits under its first key' + assert program.dimension('snapshot').lookups == (declared,), 'and under its second' + assert program.lookups == {'zone_of': declared}, 'and once in the program' + assert program.constraints['zonal'].lhs == GroupSum( + Variable('p'), over='generator', coordinate=('zone_of',), into=('zone',), keys=(('generator', 'snapshot'),) + ), 'a grouped sum names the key it walks and every key it joins on' + assert program.constraints['history'].lhs == GroupSum( + Variable('p'), over='snapshot', coordinate=('zone_of',), into=('zone',), keys=(('generator', 'snapshot'),) + ), 'the same table walked along its other key' + assert program.constraints['priced'].rhs == At( + Parameter('price'), over='generator', coordinate=('zone_of',), into=('zone',), keys=(('generator', 'snapshot'),) + ), 'and so does its adjoint' + p_where = program.variable('p').where + assert p_where is not None + assert [(type(a).__name__, a.keys) for a in p_where.atoms] == [ + ('LookupComparisonNode', ('generator', 'snapshot')), + ('LookupDefinedNode', ('generator', 'snapshot')), + ], 'every where leaf reading the lookup carries its keys' + first_where = program.variable('first').where + assert first_where is not None + assert first_where.dims == {'generator', 'snapshot'}, 'a position within a group is read at every key' + + def test_a_binary_variable_lowers_to_a_binary_domain(): program = to_program(schema_of(DISPATCH_YAML, **{'variables.p.domain': 'binary', 'variables.p.bounds': {}})) assert program.variable('p').domain == 'binary' @@ -543,8 +589,8 @@ def test_a_lookup_names_the_dimension_its_values_label(): constraints={}, objective=None, dimensions={ - 'snapshot': DimensionDeclaration((LookupDeclaration('season_of', 'season'),)), - 'generator': DimensionDeclaration((LookupDeclaration('at_bus', 'bus'),)), + 'snapshot': DimensionDeclaration((LookupDeclaration('season_of', 'season', ('snapshot',)),)), + 'generator': DimensionDeclaration((LookupDeclaration('at_bus', 'bus', ('generator',)),)), }, ) @@ -552,10 +598,7 @@ def test_a_lookup_names_the_dimension_its_values_label(): 'one dimension names its own maps and no other dimension' ) assert program.dimension('generator').targets == {'at_bus': 'bus'}, 'and the same for the second' - assert [(d, lk.name) for d, lk in program.lookups] == [ - ('snapshot', 'season_of'), - ('generator', 'at_bus'), - ], 'every map with the dimension it is over, in declaration order' + assert list(program.lookups) == ['season_of', 'at_bus'], 'every map once, by name, in declaration order' def test_an_unknown_dimension_is_a_near_miss_rather_than_an_empty_declaration(): diff --git a/tests/test_validation.py b/tests/test_validation.py index 0e33184a..4775d098 100644 --- a/tests/test_validation.py +++ b/tests/test_validation.py @@ -571,6 +571,48 @@ class TestRulesDecidedWithoutData: ), pytest.param({'lookups.lk.into': 'z'}, ("targets undeclared dimension 'z'",), id='lookup-into-undeclared'), pytest.param({'lookups.lk.into': 'g'}, ("maps 'g' into itself",), id='lookup-into-itself'), + pytest.param( + {'lookups.lk.over': ['g', 'z']}, ("references undeclared dimension 'z'",), id='lookup-key-undeclared' + ), + pytest.param( + {'dimensions.z': {}, 'lookups.lk.over': ['g', 'h']}, + ("maps 'h' into itself",), + id='lookup-into-one-of-its-keys', + ), + pytest.param( + {'lookups.lk.over': ['g', 'g']}, ("names 'g' twice under 'over:'",), id='lookup-keyed-by-a-dim-twice' + ), + pytest.param({'lookups.lk.over': []}, ('has no key dimension',), id='lookup-with-no-key'), + pytest.param( + { + 'dimensions.z': {}, + 'lookups.lk.over': ['g', 'z'], + 'variables.q.foreach': ['g', 'h', 'z'], + 'objective': {'expression': 'sum(sum(q, by=lk))'}, + }, + ("'lk' is keyed by ['g', 'z'], and the call has to say which key sum walks", 'by=lk.g or by=lk.z'), + id='by-a-two-key-lookup-without-the-dot', + ), + pytest.param( + {'objective': {'expression': 'sum(sum(p, by=lk.h))'}}, + ("'h' is not a key of 'lk', which is keyed by ['g']",), + id='by-a-column-that-is-not-a-key', + ), + pytest.param( + {'variables.q.where': 'position(g, by=lk.h) == 0'}, + ("'h' is not a key of 'lk'",), + id='position-by-a-column-that-is-not-a-key', + ), + pytest.param( + { + 'dimensions.z': {}, + 'lookups.lk.over': ['g', 'z'], + 'variables.q.foreach': ['g', 'h', 'z'], + 'variables.q.where': 'position(g, by=lk) == 0', + }, + ('the call has to say which key position counts along', 'by=lk.g'), + id='position-by-a-two-key-lookup-without-the-dot', + ), pytest.param( {'lookups.g': {'over': 'h', 'into': 'g'}}, ("Lookup 'g' collides with the dimension",), @@ -701,7 +743,7 @@ class TestRulesDecidedWithoutData: ), pytest.param( {'lookups.hk': {'over': 'h', 'into': 'g'}, 'objective': {'expression': 'sum(sum(q, by=[lk, hk]))'}}, - ('groups through lookups over different dimensions',), + ('groups through lookups along different dimensions',), id='by-lookups-over-different-dimensions', ), pytest.param( @@ -709,6 +751,24 @@ class TestRulesDecidedWithoutData: ("targets ['h'] more than once",), id='by-the-same-target-twice', ), + pytest.param( + { + 'dimensions.z': {}, + 'lookups.lz': {'over': ['g', 'h'], 'into': 'z'}, + 'objective': {'expression': 'sum(sum(q, by=[lk, lz.h]))'}, + }, + ('groups through lookups along different dimensions',), + id='by-lookups-walking-different-keys', + ), + pytest.param( + { + 'dimensions.z': {}, + 'lookups.lz': {'over': ['g', 'z'], 'into': 'h'}, + 'variables.q.where': 'lk != lz', + }, + ('compares lookups keyed by different dimensions',), + id='where-two-lookups-with-different-keys', + ), pytest.param( {'variables.p.where': 'c > flag'}, ('compares two parameters',), id='where-against-a-parameter' ), diff --git a/tests/typesetting/golden/latex.out b/tests/typesetting/golden/latex.out index 4b434197..d862d811 100644 --- a/tests/typesetting/golden/latex.out +++ b/tests/typesetting/golden/latex.out @@ -9,8 +9,8 @@ \paragraph{Sets} \begin{description} -\item[{$\mathcal{T}$}] index $t$ --- \texttt{snapshot} (\texttt{int} coordinates) with $\mathrm{season\_of}: \mathcal{T} \to \mathcal{S}$ -\item[{$\mathcal{G}$}] index $g$ --- \texttt{generator} with $\mathrm{gen\_bus}: \mathcal{G} \to \mathcal{B},\ \mathrm{gen\_tech}: \mathcal{G} \to \mathcal{E}$ +\item[{$\mathcal{T}$}] index $t$ --- \texttt{snapshot} (\texttt{int} coordinates) with $\mathrm{season\_of}: \mathcal{T} \to \mathcal{S},\ \mathrm{gen\_zone}: \mathcal{G} \times \mathcal{T} \to \mathcal{Z}$ +\item[{$\mathcal{G}$}] index $g$ --- \texttt{generator} with $\mathrm{gen\_bus}: \mathcal{G} \to \mathcal{B},\ \mathrm{gen\_tech}: \mathcal{G} \to \mathcal{E},\ \mathrm{gen\_zone}: \mathcal{G} \times \mathcal{T} \to \mathcal{Z}$ \item[{$\mathcal{B}$}] index $b$ --- \texttt{bus} with $\mathrm{zone\_of}: \mathcal{B} \to \mathcal{Z},\ \mathrm{area\_of}: \mathcal{B} \to \mathcal{Z}$ \item[{$\mathcal{Z}$}] index $z$ --- \texttt{zone} \item[{$\mathcal{S}$}] index $s$ --- \texttt{season} @@ -94,6 +94,9 @@ \text{pullback} && \mathit{spill}_{t} & \le \mathrm{zone\_cap}_{\mathrm{zone\_of}(b)} && \forall\, t \in \mathcal{T},\ b \in \mathcal{B} \\ \text{grouped\_twice} && \sum_{g \in \mathcal{G} \,:\, \mathrm{gen\_bus}(g) = b \wedge \mathrm{gen\_tech}(g) = e} p_{t,g} & \le \mathrm{tech\_cap}_{b,e} && \forall\, t \in \mathcal{T},\ b \in \mathcal{B},\ e \in \mathcal{E} \\ \text{pulled\_back\_twice} && \mathit{units}_{g} & \le \mathrm{tech\_cap}_{\mathrm{gen\_bus}(g),\mathrm{gen\_tech}(g)} && \forall\, g \in \mathcal{G} \\ +\text{zonal} && \sum_{g \in \mathcal{G} \,:\, \mathrm{gen\_zone}(g,\ t) = z} p_{t,g} & \le \mathrm{zone\_cap}_{z} && \forall\, t \in \mathcal{T},\ z \in \mathcal{Z} \\ +\text{zonal\_history} && \sum_{t \in \mathcal{T} \,:\, \mathrm{gen\_zone}(g,\ t) = z} p_{t,g} & \le \mathrm{zone\_cap}_{z} && \forall\, g \in \mathcal{G},\ z \in \mathcal{Z} \\ +\text{zonal\_pullback} && p_{t,g} & \le \mathit{spill}_{t} \cdot \mathrm{zone\_cap}_{\mathrm{gen\_zone}(g,\ t)} && \forall\, t \in \mathcal{T},\ g \in \mathcal{G} \,:\, \mathrm{gen\_zone}(g,\ t) = \text{'}\mathrm{north}\text{'} \wedge \mathrm{pos}_{\mathrm{gen\_zone}(g,\ t)}(g) = 0 \\ \text{arithmetic} && \sum_{g \in \mathcal{G}} \left( \frac{p_{t,g}}{2} - \mathrm{cost}_{g} + 10^{-5} \cdot p_{t,g} + 2.5 \times 10^{-7} \cdot \mathrm{cost}_{g} + 0.5 \cdot p_{t,g} \right) & \ge -\left( \sum_{g \in \mathcal{G}} p_{t,g} \right) \cdot \left( -3 \right) && \forall\, t \in \mathcal{T} \\ \text{total} && \sum_{t \in \mathcal{T},\ g \in \mathcal{G}} p_{t,g} & \le \mathrm{budget} \\ \text{scalar} && \mathit{units}_{g} & \le \mathrm{budget} && \forall\, g \in \mathcal{G} \,:\, \mathrm{cost}_{g} \text{ is defined} \\ diff --git a/tests/typesetting/golden/markdown.out b/tests/typesetting/golden/markdown.out index fa7ee0a5..5706fe32 100644 --- a/tests/typesetting/golden/markdown.out +++ b/tests/typesetting/golden/markdown.out @@ -6,8 +6,8 @@ every character a notation escapes, set as text: link\_to, 100% & \#1 costs \$5 | Symbol | Meaning | |---|---| -| $`\mathcal{T}`$ | index $`t`$ — `snapshot` (`int` coordinates) with $`\mathrm{season\_of}: \mathcal{T} \to \mathcal{S}`$ | -| $`\mathcal{G}`$ | index $`g`$ — `generator` with $`\mathrm{gen\_bus}: \mathcal{G} \to \mathcal{B},\ \mathrm{gen\_tech}: \mathcal{G} \to \mathcal{E}`$ | +| $`\mathcal{T}`$ | index $`t`$ — `snapshot` (`int` coordinates) with $`\mathrm{season\_of}: \mathcal{T} \to \mathcal{S},\ \mathrm{gen\_zone}: \mathcal{G} \times \mathcal{T} \to \mathcal{Z}`$ | +| $`\mathcal{G}`$ | index $`g`$ — `generator` with $`\mathrm{gen\_bus}: \mathcal{G} \to \mathcal{B},\ \mathrm{gen\_tech}: \mathcal{G} \to \mathcal{E},\ \mathrm{gen\_zone}: \mathcal{G} \times \mathcal{T} \to \mathcal{Z}`$ | | $`\mathcal{B}`$ | index $`b`$ — `bus` with $`\mathrm{zone\_of}: \mathcal{B} \to \mathcal{Z},\ \mathrm{area\_of}: \mathcal{B} \to \mathcal{Z}`$ | | $`\mathcal{Z}`$ | index $`z`$ — `zone` | | $`\mathcal{S}`$ | index $`s`$ — `season` | @@ -184,6 +184,24 @@ p_{t,g} \le p_{t \boxminus_{0}^{\mathrm{season\_of}(t)} 1,g} \qquad \forall\, t \mathit{units}_{g} \le \mathrm{tech\_cap}_{\mathrm{gen\_bus}(g),\mathrm{gen\_tech}(g)} \qquad \forall\, g \in \mathcal{G} ``` +**`zonal`** + +```math +\sum_{g \in \mathcal{G} \,:\, \mathrm{gen\_zone}(g,\ t) = z} p_{t,g} \le \mathrm{zone\_cap}_{z} \qquad \forall\, t \in \mathcal{T},\ z \in \mathcal{Z} +``` + +**`zonal_history`** + +```math +\sum_{t \in \mathcal{T} \,:\, \mathrm{gen\_zone}(g,\ t) = z} p_{t,g} \le \mathrm{zone\_cap}_{z} \qquad \forall\, g \in \mathcal{G},\ z \in \mathcal{Z} +``` + +**`zonal_pullback`** + +```math +p_{t,g} \le \mathit{spill}_{t} \cdot \mathrm{zone\_cap}_{\mathrm{gen\_zone}(g,\ t)} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} \,:\, \mathrm{gen\_zone}(g,\ t) = \text{'}\mathrm{north}\text{'} \wedge \mathrm{pos}_{\mathrm{gen\_zone}(g,\ t)}(g) = 0 +``` + **`arithmetic`** ```math diff --git a/tests/typesetting/golden/model.yaml b/tests/typesetting/golden/model.yaml index f093df2b..4a49b312 100644 --- a/tests/typesetting/golden/model.yaml +++ b/tests/typesetting/golden/model.yaml @@ -26,6 +26,7 @@ lookups: zone_of: { over: bus, into: zone } area_of: { over: bus, into: zone } # a second map into the same set, to compare against season_of: { over: snapshot, into: season } + gen_zone: { over: [generator, snapshot], into: zone } # a map keyed by two dimensions: a call walks one and joins on the other parameters: p_max: { dims: [generator] } @@ -152,6 +153,16 @@ constraints: pulled_back_twice: # its adjoint, reading one slot through a pair of labels foreach: [generator] expression: units <= at(tech_cap, by=[gen_bus, gen_tech]) + zonal: # a grouping through a two-key map, walked along one key: the condition reads the other, and the row keeps it + foreach: [snapshot, zone] + expression: sum(p, by=gen_zone.generator) <= zone_cap + zonal_history: # the same table walked along its other key + foreach: [generator, zone] + expression: sum(p, by=gen_zone.snapshot) <= zone_cap + zonal_pullback: # its adjoint, reading the slot the row's own snapshot puts the generator in + foreach: [snapshot, generator] + where: "gen_zone == 'north' AND position(generator, by=gen_zone.generator) == 0" + expression: p <= at(spill * zone_cap, by=gen_zone.generator) arithmetic: # division, both unary signs, a sign beside a sign, floats with and without an exponent, bracketing foreach: [snapshot] expression: >- diff --git a/tests/typesetting/golden/typst.out b/tests/typesetting/golden/typst.out index 7f286e31..68285be7 100644 --- a/tests/typesetting/golden/typst.out +++ b/tests/typesetting/golden/typst.out @@ -4,8 +4,8 @@ every character a notation escapes, set as text: link\_to, 100% & \#1 costs \$5 {net} \~ ^ \\ \*star\* \@ref \, and `a_name` in backticks set in code == Sets -/ $cal(T)$: index $t$ --- `snapshot` (`int` coordinates) with $upright("season_of"): cal(T) arrow.r cal(S)$ -/ $cal(G)$: index $g$ --- `generator` with $upright("gen_bus"): cal(G) arrow.r cal(B), upright("gen_tech"): cal(G) arrow.r cal(E)$ +/ $cal(T)$: index $t$ --- `snapshot` (`int` coordinates) with $upright("season_of"): cal(T) arrow.r cal(S), upright("gen_zone"): cal(G) times cal(T) arrow.r cal(Z)$ +/ $cal(G)$: index $g$ --- `generator` with $upright("gen_bus"): cal(G) arrow.r cal(B), upright("gen_tech"): cal(G) arrow.r cal(E), upright("gen_zone"): cal(G) times cal(T) arrow.r cal(Z)$ / $cal(B)$: index $b$ --- `bus` with $upright("zone_of"): cal(B) arrow.r cal(Z), upright("area_of"): cal(B) arrow.r cal(Z)$ / $cal(Z)$: index $z$ --- `zone` / $cal(S)$: index $s$ --- `season` @@ -81,6 +81,9 @@ $ upright("budgeted") & italic("spend")_(t) & <= upright("budget") & forall t in upright("pullback") & italic("spill")_(t) & <= upright("zone_cap")_(upright("zone_of")(b)) & forall t in cal(T), b in cal(B) \ upright("grouped_twice") & sum_(g in cal(G) colon upright("gen_bus")(g) = b and upright("gen_tech")(g) = e) p_(t,g) & <= upright("tech_cap")_(b,e) & forall t in cal(T), b in cal(B), e in cal(E) \ upright("pulled_back_twice") & italic("units")_(g) & <= upright("tech_cap")_(upright("gen_bus")(g),upright("gen_tech")(g)) & forall g in cal(G) \ + upright("zonal") & sum_(g in cal(G) colon upright("gen_zone")(g, t) = z) p_(t,g) & <= upright("zone_cap")_(z) & forall t in cal(T), z in cal(Z) \ + upright("zonal_history") & sum_(t in cal(T) colon upright("gen_zone")(g, t) = z) p_(t,g) & <= upright("zone_cap")_(z) & forall g in cal(G), z in cal(Z) \ + upright("zonal_pullback") & p_(t,g) & <= italic("spill")_(t) dot upright("zone_cap")_(upright("gen_zone")(g, t)) & forall t in cal(T), g in cal(G) colon upright("gen_zone")(g, t) = upright("'north'") and upright("pos")_(upright("gen_zone")(g, t))(g) = 0 \ upright("arithmetic") & sum_(g in cal(G)) (frac(p_(t,g), 2) - upright("cost")_(g) + 10^(-5) dot p_(t,g) + 2.5 times 10^(-7) dot upright("cost")_(g) + 0.5 dot p_(t,g)) & >= -(sum_(g in cal(G)) p_(t,g)) dot (-3) & forall t in cal(T) \ upright("total") & sum_(t in cal(T), g in cal(G)) p_(t,g) & <= upright("budget") \ upright("scalar") & italic("units")_(g) & <= upright("budget") & forall g in cal(G) colon upright("cost")_(g) upright(" is defined") \