From 5e29391bd979a99f5411b56798608f647e0d7f9a Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 15 Sep 2026 08:38:33 +0000 Subject: [PATCH 1/2] feat(language): a where may compare arithmetic over parameters Either side of a comparison in a where string may be an expression, read as an expression is: macros and named expressions expand, every operator keeps its rule, and a variable or a dual is refused. Resolution types it as `ArithmeticComparisonNode` over the core syntax tree for the spec-side readers; lowering rebuilds every mask with `ExpressionComparisonNode` over program expressions, so a program's masks are program vocabulary throughout. The expression form stands aside for the plain shapes, so `p > 0` keeps its node. Docs sentence lengths (n / median / over 25): expressions.md 139 / 16 / 27, reading.md 68 / 15 / 15. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_013HceuCYNepeQX8SdZtiMf1 --- docs/reference/language/expressions.md | 42 ++++++++++- docs/reference/language/reading.md | 5 ++ docs/reference/notation.md | 73 +++++++++++++++++++ src/math_spec/_expression_parser.py | 15 ++-- src/math_spec/_where_parser.py | 81 +++++++++++++++++++-- src/math_spec/dimensions.py | 14 ++-- src/math_spec/exclusivity.py | 15 +++- src/math_spec/lowering.py | 30 ++++++-- src/math_spec/program.py | 88 ++++++++++++++++++++++- src/math_spec/resolution.py | 69 +++++++++++++++++- src/math_spec/typesetting/walk.py | 10 +++ tests/test_lowering.py | 35 +++++++++ tests/test_parser.py | 30 ++++++++ tests/test_validation.py | 99 ++++++++++++++++++++++++-- tests/typesetting/golden/latex.out | 8 ++- tests/typesetting/golden/markdown.out | 31 ++++++++ tests/typesetting/golden/model.yaml | 17 +++++ tests/typesetting/golden/typst.out | 10 ++- tests/typesetting/test_golden.py | 9 ++- tests/typesetting/test_walk.py | 10 +++ 20 files changed, 652 insertions(+), 39 deletions(-) diff --git a/docs/reference/language/expressions.md b/docs/reference/language/expressions.md index 320bf4d4..7efb3a0f 100644 --- a/docs/reference/language/expressions.md +++ b/docs/reference/language/expressions.md @@ -157,10 +157,11 @@ A `where:` is a boolean mask, and true means "this coordinate exists". ```text where_expr ::= atom | "NOT" where_expr | where_expr ("AND"|"OR") where_expr | "(" where_expr ")" -atom ::= NAME | NAME COMPARATOR value | POSITION COMPARATOR INTEGER - | "True" | "False" +atom ::= NAME | NAME COMPARATOR value | expression COMPARATOR expression + | POSITION COMPARATOR INTEGER | "True" | "False" COMPARATOR ::= "<=" | ">=" | "==" | "!=" | "<" | ">" value ::= NUMBER | QUOTED | NAME_OR_STRING +expression ::= the arithmetic grammar above, with no variable and no dual in it POSITION ::= "position" "(" NAME [ "," "by" "=" NAME ] ")" QUOTED ::= "'" chars "'" | '"' chars '"' ``` @@ -175,6 +176,7 @@ QUOTED ::= "'" chars "'" | '"' chars '"' | `name OP value` | dimension | A filter on the frame's own coordinate column | | `name OP value` | lookup | A filter on the lookup's value, so the `over` dimension has to be in the frame. A null compares false | | `name OP name` | two lookups | Legal only where both lookups are over the same dimension and into the same dimension. `from != to` excludes a self-loop | +| `expression OP expression` | arithmetic over parameters | Coordinate by coordinate, over every dimension either side carries. A macro and a named expression expand as in an expression, and every operator keeps its rule, so a `shift` names its `edge=`. A side with no value at a coordinate compares false | | `position(name) OP i` | dimension | Where the row sits along the dimension's own order. `0` is first, and a negative number counts from the end | | `position(name, by=lookup) OP i` | a dimension and a lookup over it | The same, counted within each group the lookup makes | | `AND` `OR` `NOT` | — | Case-insensitive. `NOT` binds tighter than `AND`, and `AND` tighter than `OR` | @@ -216,6 +218,42 @@ boolean parameter instead. Two lookups are the exception, where both lookups share both ends: over one dimension they are two columns of one table, and into one dimension they draw from one label set. +### Arithmetic in a comparison + +Either side of a comparison may be an expression: `p_min <= 0.5 * p_max`, +`sum(p_max, over=generator) >= peak`, `p_max <= at(bus_cap, by=bus_of)`. The +side is read exactly as an [expression](#expressions) is, so a macro and a +named expression expand into it and every operator keeps its own rule. Two +things an expression may carry are refused here, because a mask is built before +either exists: a variable, and a `dual()`. + +A comparison of expressions is checked over every dimension either side +carries. A side whose value is absent at a coordinate compares false there, as +a null does in every other comparison; under a summing operator the absent +term is one fewer. A `shift` says what its vacated positions hold, as it does +everywhere, so a comparison against the previous row names an `edge=` and a +`position()` term keeps the first row out: + +```yaml +dimensions: + snapshot: { dtype: int } +parameters: + load: { dims: [snapshot] } + ramp: { dims: [] } +variables: + shed: { dims: [snapshot], bounds: { lower: 0 } } +constraints: + shed_when_load_jumps: + dims: [snapshot] + where: "load - shift(load, over=snapshot, offset=1, edge=0) > ramp AND position(snapshot) > 0" + expression: shed >= load - ramp +``` + +A case `when:` that compares expressions cannot be proved apart from its +neighbours before the data arrives, so it is refused there with the rewrite: +compare one parameter against a literal, or precompute the test as a boolean +parameter. + ### `position()` `position(dim)` is where the row sits along the dimension's own order, which is diff --git a/docs/reference/language/reading.md b/docs/reference/language/reading.md index 3b6abcc6..75792da9 100644 --- a/docs/reference/language/reading.md +++ b/docs/reference/language/reading.md @@ -109,6 +109,11 @@ questions that every engine would otherwise work out for itself: - `.atoms` gives its leaves, with the connectives removed. - `.dims` gives the dimensions the mask is read at. +A comparison of expressions arrives as an `ExpressionComparisonNode`, whose two +sides are program expressions like a constraint's, and whose `dims` are every +dimension either side carries. Its `names_read` are every parameter and lookup +the sides read, the lookup a grouping joins through included. + A predicate you build yourself answers the same four questions: wrap it in `Mask`, or build it there with `~`, `&` and `|`. A mask folds as it is built: a double negation cancels, and a `True` or `False` is absorbed rather than buried in the diff --git a/docs/reference/notation.md b/docs/reference/notation.md index 10d99b65..ad006bea 100644 --- a/docs/reference/notation.md +++ b/docs/reference/notation.md @@ -112,6 +112,7 @@ parameters: | Symbol | Meaning | |---|---| +| $`\mathrm{spend}^{\mathrm{cap}}`$ | `spend_cap` over $`\mathcal{G}`$ | | $`\mathit{spend}`$ | `spend` over $`\mathcal{T}`$ — what a snapshot's dispatch costs | | $`\mathit{lcoe}`$ | `lcoe` (scalar) | | $`\mathit{marginal\_price}`$ | `marginal_price` over $`\mathcal{T} \times \mathcal{B}`$ | @@ -580,8 +581,80 @@ never: \mathit{slack}_{t} \ge 0 \qquad \forall\, t \in \mathcal{T} \,:\, \bot ``` +#### `margin` + +a mask comparing two expressions, which prints as the arithmetic it is + +```yaml +margin: + dims: [snapshot, generator] + where: "p_max - p_min > cost / 2" + expression: p <= p_max +``` + +```math +p_{t,g} \le \mathrm{p}^{\mathrm{max}}_{g} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} \,:\, \mathrm{p}^{\mathrm{max}}_{g} - \mathrm{p}^{\mathrm{min}}_{g} > \frac{\mathrm{cost}_{g}}{2} +``` + +#### `ramped` + +a translation under a comparison names its edge, a pullback reads through a lookup, and the position keeps the vacated row out + +```yaml +ramped: + dims: [snapshot, bus] + where: "load - shift(load, over=snapshot, offset=1, edge=0) <= at(zone_cap, by=zone_of) AND position(snapshot) > 0" + expression: slack <= load +``` + +```math +\mathit{slack}_{t} \le \mathrm{load}_{t,b} \qquad \forall\, t \in \mathcal{T},\ b \in \mathcal{B} \,:\, \mathrm{load}_{t,b} - \mathrm{load}_{t \boxminus_{0} 1,b} \le \mathrm{zone\_cap}_{\mathrm{zone\_of}(b)} \wedge \mathrm{pos}(t) > 0 +``` + +#### `covered` + +a reduction on a side of a scalar mask, so nothing is left to quantify + +```yaml +covered: + dims: [] + where: "sum(p_max, over=generator) >= budget" + expression: sum(p) <= budget +``` + +```math +\sum_{t \in \mathcal{T},\ g \in \mathcal{G}} p_{t,g} \le \mathrm{budget} \qquad \text{where } \sum_{g \in \mathcal{G}} \mathrm{p}^{\mathrm{max}}_{g} \ge \mathrm{budget} +``` + +#### `capped` + +an expressions: entry on a side, read by the name the file gave it + +```yaml +capped: + dims: [snapshot, generator] + where: "spend_cap > 0 OR NOT is_flexible" + expression: p <= p_max +``` + +```math +p_{t,g} \le \mathrm{p}^{\mathrm{max}}_{g} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} \,:\, \mathrm{spend}^{\mathrm{cap}}_{g} > 0 \vee \neg \mathrm{is\_flexible}_{g} +``` + ### Definitions +#### `spend_cap` + +a data-only entry, so a where may compare it + +```yaml +spend_cap: cost * 2 +``` + +```math +\mathrm{spend}^{\mathrm{cap}}_{g} = \mathrm{cost}_{g} \cdot 2 \qquad \forall\, g \in \mathcal{G} +``` + #### `spend` a plain named expression: its symbol prints where it is used, its body once as a definition diff --git a/src/math_spec/_expression_parser.py b/src/math_spec/_expression_parser.py index 642a9f6a..c90dfb99 100644 --- a/src/math_spec/_expression_parser.py +++ b/src/math_spec/_expression_parser.py @@ -349,8 +349,12 @@ def with_children(node: ArithmeticNode, recurse: Callable[[ArithmeticNode], Arit # --------------------------------------------------------------------------- -def _build_grammar() -> pp.ParserElement: - """``inf`` is a ``pp.Keyword`` rather than a ``pp.Literal``, which would match the prefix of ``inflow``.""" +def _build_grammar() -> tuple[pp.ParserElement, pp.ParserElement]: + """The arithmetic grammar, and the expression grammar that puts one comparison over it. + + ``inf`` is a ``pp.Keyword`` rather than a ``pp.Literal``, which would + match the prefix of ``inflow``. + """ arith = pp.Forward() inf_literal = (pp.Keyword('.inf') | pp.Keyword('inf')).set_parse_action(lambda: NumberNode(float('inf'))) @@ -386,9 +390,10 @@ def _build_grammar() -> pp.ParserElement: arith <<= add_sub comparator = pp.one_of(list(get_args(ComparisonOperator))) - return (arith + pp.Optional(comparator + arith)).set_parse_action( + expression = (arith + pp.Optional(comparator + arith)).set_parse_action( lambda t: ComparisonNode(t[1], t[0], t[2]) if len(t) == 3 else t[0] ) + return arith, expression def _make_func_call(tokens: pp.ParseResults) -> FunctionCallNode: @@ -421,7 +426,9 @@ def _make_power(tokens: pp.ParseResults) -> Any: return items[0] if len(items) == 1 else BinaryOperatorNode('**', items[0], items[2]) -_GRAMMAR = _build_grammar() +#: The arithmetic half on its own, for the where grammar to put a predicate's +#: comparator over — one grammar for what a side may say, wherever it stands. +ARITHMETIC, _GRAMMAR = _build_grammar() #: How deep a tree the language admits. Every pass over an expression recurses, diff --git a/src/math_spec/_where_parser.py b/src/math_spec/_where_parser.py index ea873445..51e65add 100644 --- a/src/math_spec/_where_parser.py +++ b/src/math_spec/_where_parser.py @@ -15,12 +15,23 @@ import pyparsing as pp -from math_spec._expression_parser import NAME, REAL, parse_text +from math_spec._expression_parser import ( + ARITHMETIC, + NAME, + REAL, + FunctionCallNode, + NameNode, + NumberNode, + UnaryOperatorNode, + children, + parse_text, +) from math_spec.program import AndNode, BooleanLiteralNode, NotNode, OrNode, PredicateOperator, where_children if TYPE_CHECKING: from collections.abc import Callable + from math_spec._expression_parser import ArithmeticNode from math_spec.program import WhereNode # --------------------------------------------------------------------------- @@ -49,6 +60,19 @@ class UnresolvedComparisonNode: quoted: bool = False +@dataclass(frozen=True) +class UnresolvedExpressionComparisonNode: + """``expression expression``, both sides still the bare parse — ``resolution.py`` types and judges them. + + The grammar reaches for this only where a side is more than one name or + literal, so the simpler forms keep their own nodes and their own rules. + """ + + left: ArithmeticNode + op: PredicateOperator + right: ArithmeticNode + + @dataclass(frozen=True) class UnresolvedPositionNode: """``position(dim[, by=lookup]) i`` before the names are checked; ``resolution.py`` types it.""" @@ -61,7 +85,9 @@ class UnresolvedPositionNode: #: What resolution rewrites away on the where side — the three nodes whose #: left-hand side is still a name the schema has not been asked about. -UnresolvedWhereNode = UnresolvedNameNode | UnresolvedComparisonNode | UnresolvedPositionNode +UnresolvedWhereNode = ( + UnresolvedNameNode | UnresolvedComparisonNode | UnresolvedExpressionComparisonNode | UnresolvedPositionNode +) # --------------------------------------------------------------------------- @@ -82,6 +108,31 @@ def _position_comparison(tokens: pp.ParseResults) -> UnresolvedPositionNode: return UnresolvedPositionNode(str(dimension), op, at, None if by is None else str(by)) +def _is_plain(node: ArithmeticNode) -> bool: + """Whether *node* is one name or one signed number — a side the simpler comparison forms own.""" + if isinstance(node, UnaryOperatorNode): + return isinstance(node.operand, NumberNode) + return isinstance(node, NameNode | NumberNode) + + +def _reads_arithmetic(tokens: pp.ParseResults) -> bool: + """Whether a comparison needs the expression form at all. + + Two plain sides are ``name literal`` or ``name name``, and a + ``position(...)`` call against a plain side is the position form; each of + those has a node of its own, so this form stands aside for them. + """ + left, _, right = tokens + if _is_plain(left) and _is_plain(right): + return False + return not (isinstance(left, FunctionCallNode) and left.name == 'position' and _is_plain(right)) + + +def _expression_comparison(tokens: pp.ParseResults) -> UnresolvedExpressionComparisonNode: + left, op, right = tokens + return UnresolvedExpressionComparisonNode(left, op, right) + + def _comparison(tokens: pp.ParseResults) -> UnresolvedComparisonNode: """``name literal`` off the tokens the grammar captured, the quoted marker turned into a flag.""" name, op, value = tokens @@ -93,8 +144,11 @@ def _build_where_grammar() -> pp.ParserElement: """Build the pyparsing grammar for where strings. Both quote characters are accepted because YAML already owns one of them. - ``NOT`` binds tightest, then ``AND``, then ``OR``. ``position(...)`` leads - the alternation, since ``position`` would otherwise be read as a bare name. + ``NOT`` binds tightest, then ``AND``, then ``OR``. The three comparison + forms are matched longest-first, so ``p > 2 * q`` is not cut short at + ``p > 2``; the expression form stands aside for the two plain shapes + (:func:`_reads_arithmetic`), so ``p > 0`` keeps the node its dtype rule + is written for. """ where_expr = pp.Forward() @@ -121,14 +175,16 @@ def _build_where_grammar() -> pp.ParserElement: position_comparison = (position_call + comparator + position).set_parse_action(_position_comparison) comparison = (name + comparator + (number | quoted | name)).set_parse_action(_comparison) + expression_comparison = ( + (ARITHMETIC + comparator + ARITHMETIC).add_condition(_reads_arithmetic).add_parse_action(_expression_comparison) + ) # pyrefly: ignore[implicit-any-lambda] existence = name.copy().set_parse_action(lambda t: UnresolvedNameNode(t[0])) atom = ( true_lit | false_lit - | position_comparison - | comparison + | (position_comparison ^ comparison ^ expression_comparison) | existence | (pp.Suppress('(') + where_expr + pp.Suppress(')')) ) @@ -192,6 +248,17 @@ def _named_rewrite(text: str, loc: int) -> str | None: ) +def _nested(node: Any) -> tuple[Any, ...]: + """What a where string nests through: a connective's operands, and the arithmetic under a comparison of expressions.""" + if isinstance(node, UnresolvedExpressionComparisonNode): + return (node.left, node.right) + if isinstance(node, UnresolvedWhereNode): + return () + if isinstance(node, AndNode | OrNode | NotNode | BooleanLiteralNode): + return where_children(node) + return children(node) + + @lru_cache(maxsize=4096) def parse_where(text: str) -> WhereNode | UnresolvedWhereNode: """Parse a where string into an AST, its leaves still unresolved. @@ -208,5 +275,5 @@ def parse_where(text: str) -> WhereNode | UnresolvedWhereNode: """ return cast( 'WhereNode | UnresolvedWhereNode', - parse_text(_WHERE_GRAMMAR, text, 'where string', _named_rewrite, where_children, _DEEP_REWRITE), + parse_text(_WHERE_GRAMMAR, text, 'where string', _named_rewrite, _nested, _DEEP_REWRITE), ) diff --git a/src/math_spec/dimensions.py b/src/math_spec/dimensions.py index 3c07aed5..48b7a4cc 100644 --- a/src/math_spec/dimensions.py +++ b/src/math_spec/dimensions.py @@ -40,8 +40,10 @@ from math_spec.errors import DimensionError from math_spec.operators import BUILTINS from math_spec.program import ( + ArithmeticComparisonNode, DimensionComparisonNode, DimensionPositionNode, + ExpressionComparisonNode, LookupComparisonNode, LookupDefinedNode, LookupPairComparisonNode, @@ -515,17 +517,19 @@ def _check_where_dims( continue match atom: case ParameterDefinedNode() | ParameterComparisonNode(): - noun = 'parameter' + leaf = f"where-parameter '{atom.name}'" case VariableDefinedNode(): - noun = 'variable' + leaf = f"where-variable '{atom.name}'" case DimensionComparisonNode() | DimensionPositionNode(): - noun = 'dimension' + leaf = f"where-dimension '{atom.name}'" case LookupComparisonNode() | LookupPairComparisonNode() | LookupDefinedNode(): - noun = 'lookup' + leaf = f"where-lookup '{atom.name}'" + case ArithmeticComparisonNode() | ExpressionComparisonNode(): + leaf = 'a where-comparison of expressions' case _: assert_never(atom) raise DimensionError( - f"{context}: where-{noun} '{atom.name}' reads dims {outside} outside the frame {sorted(frame)}. " + f'{context}: {leaf} reads dims {outside} outside the frame {sorted(frame)}. ' f'Reducing a mask over an unlisted dim would silently widen it — add the dim to dims:, ' f'or test a name the frame carries.' ) diff --git a/src/math_spec/exclusivity.py b/src/math_spec/exclusivity.py index 450aad92..42ac7e37 100644 --- a/src/math_spec/exclusivity.py +++ b/src/math_spec/exclusivity.py @@ -23,9 +23,11 @@ from math_spec.program import ( AndNode, + ArithmeticComparisonNode, BooleanLiteralNode, DimensionComparisonNode, DimensionPositionNode, + ExpressionComparisonNode, LookupComparisonNode, LookupDefinedNode, LookupPairComparisonNode, @@ -136,7 +138,7 @@ class Subject: a rank is further split by the ``by=`` lookup it is counted within. """ - kind: Literal['param', 'dim', 'rank', 'lookup', 'lookup_pair', 'variable'] + kind: Literal['param', 'expression', 'dim', 'rank', 'lookup', 'lookup_pair', 'variable'] name: str qualifier: str | None = None @@ -189,6 +191,12 @@ def _observe(node: TypedPredicateNode, subject: Subject, values: set[Any], dtype ``position()`` converts the dimension to an integer, so an ordering over a rank is an ordering of integers and every comparator is admitted there. """ + if isinstance(node, ArithmeticComparisonNode | ExpressionComparisonNode): + msg = ( + 'it compares expressions, whose values only the data decides — compare one parameter against a ' + 'literal, or precompute the test as a boolean parameter and test that' + ) + raise Undecidable(msg) if isinstance(node, DimensionPositionNode): values.add(node.position) elif isinstance(node, LookupPairComparisonNode): @@ -224,6 +232,8 @@ def _subject_of(node: TypedPredicateNode) -> Subject: return Subject('lookup', name) case LookupPairComparisonNode(name=name, other=other): return Subject('lookup_pair', name, other) + case ArithmeticComparisonNode() | ExpressionComparisonNode(): + return Subject('expression', 'a comparison of expressions') case _: assert_never(node) @@ -405,6 +415,9 @@ def _atom(node: TypedPredicateNode, cell: dict[Subject, Cell], grid: _Grid) -> b return bool(value) case LookupPairComparisonNode(op=op): return bool(value) if op == '==' else not value + case ArithmeticComparisonNode() | ExpressionComparisonNode(): + msg = 'a comparison of expressions is refused as undecidable before any cell is read' + raise AssertionError(msg) case DimensionPositionNode(op=op, position=position): return _compare(value, op, position) case ParameterComparisonNode(op=op, value=literal) | LookupComparisonNode(op=op, value=literal): diff --git a/src/math_spec/lowering.py b/src/math_spec/lowering.py index d1df7af4..4d6bee23 100644 --- a/src/math_spec/lowering.py +++ b/src/math_spec/lowering.py @@ -113,7 +113,7 @@ def lower_program(expanded: _ExpandedSpec) -> program.Program: lower, upper = _bound_expression(vdef.bounds.lower), _bound_expression(vdef.bounds.upper) variables[vname] = program.VariableDeclaration( tuple(vdef.dims), - where=resolved.variables[vname], + where=_Lowering(expanded, f"variable '{vname}'").mask(resolved.variables[vname]), lower=lower, upper=upper, domain=domain, @@ -129,7 +129,7 @@ def lower_program(expanded: _ExpandedSpec) -> program.Program: lhs=lowering.expr(expression.left), sense=expression.op, rhs=lowering.expr(expression.right), - where=where, + where=lowering.mask(where), ) objective = None @@ -249,13 +249,35 @@ def _cases(self, node: CasesNode) -> program.Cases: Every ``when`` arrives folded from resolution, and an arm that folded to a literal was refused at load — so no literal reaches a region. """ - stated = [program.Mask(arm.when) for arm in node.arms if arm.when is not None] + stated = [self._where(arm.when) for arm in node.arms if arm.when is not None] regions = [] for arm in node.arms: - when = program.Mask(arm.when) if arm.when is not None else _none_of(stated) + when = self._where(arm.when) if arm.when is not None else _none_of(stated) regions.append(program.Region(when, self.expr(arm.value))) return program.Cases(tuple(regions)) + def mask(self, mask: program.Mask | None) -> program.Mask | None: + """*mask* with every comparison of expressions lowered, so a program's masks are program vocabulary throughout. + + Every other predicate node is already the program's own and passes + through; a mask holding none comes back equal to the one handed in. + """ + return None if mask is None else self._where(mask.root) + + def _where(self, node: program.WhereNode) -> program.Mask: + return program.Mask(self._predicate(node)) + + def _predicate(self, node: program.WhereNode) -> program.WhereNode: + if isinstance(node, program.ArithmeticComparisonNode): + return program.ExpressionComparisonNode(self.expr(node.left), node.op, self.expr(node.right), node.dims) + if isinstance(node, program.NotNode): + return program.NotNode(self._predicate(node.operand)) + if isinstance(node, program.AndNode): + return program.AndNode(self._predicate(node.left), self._predicate(node.right)) + if isinstance(node, program.OrNode): + return program.OrNode(self._predicate(node.left), self._predicate(node.right)) + return node + def sum(self, node: FunctionCallNode) -> program.ExpressionNode: """``sum(x)``, ``sum(x, over=d)`` or ``sum(x, by=lookup)``. diff --git a/src/math_spec/program.py b/src/math_spec/program.py index 5478a31a..604af125 100644 --- a/src/math_spec/program.py +++ b/src/math_spec/program.py @@ -25,7 +25,7 @@ from typing import TYPE_CHECKING, Literal, NamedTuple, assert_never, get_args import math_spec.model as _model -from math_spec._expression_parser import ComparisonOperator +from math_spec._expression_parser import ComparisonOperator, LookupNode, ParameterNode, nodes from math_spec._sealed import Sealed from math_spec.errors import did_you_mean @@ -33,12 +33,15 @@ import datetime from collections.abc import Iterator + from math_spec._expression_parser import ArithmeticNode + #: What ``math_spec.program`` promises a consumer, sorted. __all__ = [ 'QUADRATIC_POSITIONS', 'Add', 'AndNode', + 'ArithmeticComparisonNode', 'At', 'AtLeastTwo', 'BooleanLiteralNode', @@ -58,6 +61,7 @@ 'Divide', 'Dual', 'Expression', + 'ExpressionComparisonNode', 'ExpressionDeclaration', 'ExpressionNode', 'FanIn', @@ -1043,6 +1047,38 @@ class ParameterComparisonNode: dims: tuple[str, ...] +@dataclass(frozen=True) +class ExpressionComparisonNode: + """Compare two variable-free expressions, coordinate by coordinate — ``p_min <= 0.5 * p_max``. + + ``dims`` is every dim either side carries. A side whose value is absent at + a coordinate — a parameter row missing, a translation that vacated it — + makes the comparison false there, as a null does in every other + comparison; under a summing operator the absent term is one fewer. + """ + + left: ExpressionNode + op: PredicateOperator + right: ExpressionNode + dims: tuple[str, ...] + + +@dataclass(frozen=True) +class ArithmeticComparisonNode: + """The same comparison as resolution types it, its sides in the core syntax tree. + + What the spec-side readers walk — the typesetter, the dim rules, the + exclusivity check. :func:`~math_spec.lowering.lower_program` rebuilds + every mask with an :class:`ExpressionComparisonNode` in its place, so a + program never carries one. + """ + + left: ArithmeticNode + op: PredicateOperator + right: ArithmeticNode + dims: tuple[str, ...] + + @dataclass(frozen=True) class DimensionComparisonNode: """Compare a dimension's own coordinates against a literal.""" @@ -1124,6 +1160,8 @@ class OrNode: | ParameterDefinedNode | VariableDefinedNode | ParameterComparisonNode + | ExpressionComparisonNode + | ArithmeticComparisonNode | DimensionComparisonNode | LookupComparisonNode | LookupPairComparisonNode @@ -1138,6 +1176,8 @@ class OrNode: #: decide about them. TypedPredicateNode = ( ParameterComparisonNode + | ExpressionComparisonNode + | ArithmeticComparisonNode | ParameterDefinedNode | VariableDefinedNode | DimensionComparisonNode @@ -1198,7 +1238,13 @@ def _atom_dims(atom: TypedPredicateNode) -> frozenset[str]: a branch, rather than a wrong dim set at the first model to use it. """ match atom: - case ParameterComparisonNode() | ParameterDefinedNode() | VariableDefinedNode(): + case ( + ParameterComparisonNode() + | ExpressionComparisonNode() + | ArithmeticComparisonNode() + | ParameterDefinedNode() + | VariableDefinedNode() + ): return frozenset(atom.dims) case DimensionComparisonNode() | DimensionPositionNode(): return frozenset({atom.name}) @@ -1212,7 +1258,8 @@ def _atom_names(atom: TypedPredicateNode) -> frozenset[str]: """One leaf's declarations, its dimension apart — the rule :attr:`Mask.names_read` is the union of. A comparison on a dimension names no declaration — a coordinate is not - data to feed — and a lookup pair names both maps it compares. + data to feed — a lookup pair names both maps it compares, and a comparison + of expressions names every parameter and lookup its sides read. ``assert_never``-closed for the reason :func:`_atom_dims` is: a predicate node added without a reading is a type error at this one branch rather than a name silently dropped at the first model to use it. @@ -1228,12 +1275,47 @@ def _atom_names(atom: TypedPredicateNode) -> frozenset[str]: return frozenset({atom.name}) case LookupPairComparisonNode(): return frozenset({atom.name, atom.other}) + case ExpressionComparisonNode(): + return _names_under(atom.left, atom.right) + case ArithmeticComparisonNode(): + return frozenset( + name + for node in nodes(atom.left, atom.right) + for name in ( + (node.name,) + if isinstance(node, ParameterNode) + else node.names + if isinstance(node, LookupNode) + else () + ) + ) case DimensionComparisonNode() | DimensionPositionNode(): return frozenset() case _: assert_never(atom) +def _names_under(*expressions: ExpressionNode) -> frozenset[str]: + """Every parameter and lookup the data has to supply for *expressions* — what a mask's ``names_read`` promises. + + :func:`parameters_of` alone misses the data an operator reads beside its + operand: the lookup a grouping or a pullback joins through, the one a + translation is partitioned by, and the parameter a named offset or width + is read from. + """ + names: set[str] = set(parameters_of(*expressions)) + for node in walk(*expressions): + if isinstance(node, (GroupSum, At)): + names.update(node.coordinate) + elif isinstance(node, (Translate, Window)): + if node.partition is not None: + names.add(node.partition) + amount = node.offset if isinstance(node, Translate) else node.width + if isinstance(amount, str): + names.add(amount) + return frozenset(names) + + def _conjuncts(where: WhereNode) -> tuple[WhereNode, ...]: """The flatten rule behind :attr:`Mask.conjuncts` — the one home of the split. diff --git a/src/math_spec/resolution.py b/src/math_spec/resolution.py index 4af355b0..b1bb1c9b 100644 --- a/src/math_spec/resolution.py +++ b/src/math_spec/resolution.py @@ -17,6 +17,7 @@ from functools import cached_property from typing import TYPE_CHECKING, Literal, NamedTuple, assert_never, cast +import math_spec.degree as degree from math_spec._expression_parser import ( ArithmeticNode, BinaryOperatorNode, @@ -45,13 +46,15 @@ ) from math_spec._where_parser import ( UnresolvedComparisonNode, + UnresolvedExpressionComparisonNode, UnresolvedNameNode, UnresolvedPositionNode, UnresolvedWhereNode, parse_where, ) +from math_spec.dimensions import dims_of from math_spec.errors import LanguageError, did_you_mean -from math_spec.expansion import parse_and_expand +from math_spec.expansion import expand, parse_and_expand from math_spec.model import NUMERIC_DTYPES from math_spec.operators import ( BUILTINS, @@ -62,6 +65,7 @@ ) from math_spec.program import ( AndNode, + ArithmeticComparisonNode, BooleanLiteralNode, DimensionComparisonNode, DimensionPositionNode, @@ -96,7 +100,7 @@ class Namespace: A name has one kind: model.py refuses one declared under two sections. """ - __slots__ = ('constraints', 'dimensions', 'dtypes', 'leaf_dims', 'lookups', 'parameters', 'variables') + __slots__ = ('constraints', 'dimensions', 'dtypes', 'leaf_dims', 'lookups', 'parameters', 'schema', 'variables') def __init__( self, @@ -107,7 +111,12 @@ def __init__( dtypes: Mapping[str, DeclaredDtype], leaf_dims: Mapping[str, tuple[str, ...]], constraints: Iterable[str], + schema: Spec, ) -> None: + #: The schema the names come from — what a where comparison's sides + #: are expanded and dim-checked against, since those read operators + #: and named expressions that the flat listing above cannot answer for. + self.schema = schema self.variables = frozenset(variables) self.parameters = frozenset(parameters) self.dimensions = frozenset(dimensions) @@ -146,6 +155,7 @@ def of(cls, schema: Spec) -> Namespace: **{v: tuple(vd.dims) for v, vd in schema.variables.items()}, }, schema.constraints, + schema, ) def kind(self, name: str) -> DeclarationKind | None: @@ -634,6 +644,8 @@ def where(self, node: WhereNode | UnresolvedWhereNode) -> WhereNode | Unresolved return self._position(node) if isinstance(node, UnresolvedComparisonNode): return self._comparison(node) + if isinstance(node, UnresolvedExpressionComparisonNode): + return self._expression_comparison(node) if isinstance(node, NotNode): return NotNode(self._child(node.operand)) if isinstance(node, AndNode): @@ -706,9 +718,16 @@ def _position(self, node: UnresolvedPositionNode) -> DimensionPositionNode | Unr return DimensionPositionNode(node.dimension, node.op, node.position, node.by) def _comparison(self, node: UnresolvedComparisonNode) -> WhereNode | UnresolvedWhereNode: - """``name literal``, or the one structural form ``lookup lookup``.""" + """``name literal``, or the one structural form ``lookup lookup``. + + A side that names an ``expressions:`` entry is arithmetic, whatever + the grammar first read it as, and takes the expression path. + """ ns, context = self.ns, self.context value = node.value + if node.name in ns.schema.expressions or (not node.quoted and value in ns.schema.expressions): + right: ArithmeticNode = NameNode(value) if isinstance(value, str) else NumberNode(value) + return self._expression_comparison(UnresolvedExpressionComparisonNode(NameNode(node.name), node.op, right)) if not node.quoted and isinstance(value, str) and (rhs_kind := ns.kind(value)) is not None: if rhs_kind == 'lookup' and ns.kind(node.name) == 'lookup': if (refusal := _lookup_pair_error(context, node, value, ns)) is not None: @@ -744,6 +763,50 @@ def _comparison(self, node: UnresolvedComparisonNode) -> WhereNode | UnresolvedW ) return node + def _expression_comparison( + self, node: UnresolvedExpressionComparisonNode + ) -> ArithmeticComparisonNode | UnresolvedExpressionComparisonNode: + """``expression expression``: each side expanded, typed and held to what a mask may read. + + A side is read as an expression is — macros and named expressions + expand, every operator and dim rule applies — except that it names no + variable and no dual, since a mask is built before either exists. + """ + ns, context = self.ns, self.context + found = len(self.errors) + sides = [] + for side in (node.left, node.right): + try: + expanded = expand(side, ns.schema, context) + except ValueError as e: + self.errors.append(str(e) if str(e).startswith(context) else f'{context}: {e}') + continue + sides.append(self._arith(expanded)) + if len(self.errors) > found: + return node + dims: set[str] = set() + for side in sides: + if degree.carries_variable(side): + self.errors.append( + f'{context}: a where compares expressions, and one side names a variable. A where mask ' + f'is built before variables exist — it may test parameters and dimension coordinates only.' + ) + elif degree.calls_dual(side): + self.errors.append( + f'{context}: a where compares expressions, and one side reads a dual, which only a solve ' + f'produces. A mask is built before it — test the data instead.' + ) + else: + try: + degree.check_expression(side, context, ceiling=1) + dims |= dims_of(side, ns.schema, context) + except LanguageError as e: + self.errors.append(str(e)) + if len(self.errors) > found: + return node + left, right = sides + return ArithmeticComparisonNode(left, node.op, right, tuple(d for d in ns.schema.dimensions if d in dims)) + def _typed_literal( self, node: UnresolvedComparisonNode, dtype: DeclaredDtype ) -> float | str | datetime.date | None: diff --git a/src/math_spec/typesetting/walk.py b/src/math_spec/typesetting/walk.py index 4569052f..b36cdfb6 100644 --- a/src/math_spec/typesetting/walk.py +++ b/src/math_spec/typesetting/walk.py @@ -36,9 +36,11 @@ from math_spec.dimensions import dims_of from math_spec.program import ( AndNode, + ArithmeticComparisonNode, BooleanLiteralNode, DimensionComparisonNode, DimensionPositionNode, + ExpressionComparisonNode, LookupComparisonNode, LookupDefinedNode, LookupPairComparisonNode, @@ -523,6 +525,14 @@ def _where(self, node: WhereNode, ctx: _Context) -> tuple[str, int]: left = ctx.indexed(self.symbols.name[node.name], list(node.dims)) return f'{left} {self._op(_PREDICATES[node.op])} {self._literal(node.value)}', comparison + if isinstance(node, ArithmeticComparisonNode): + left, right = self._expression(node.left, ctx), self._expression(node.right, ctx) + return f'{left} {self._op(_PREDICATES[node.op])} {right}', comparison + + if isinstance(node, ExpressionComparisonNode): + msg = 'a lowered comparison reached the typesetter; it prints the resolved tree, which lowering rebuilds.' + raise AssertionError(msg) + if isinstance(node, DimensionComparisonNode): if isinstance(node.value, int | float): self.noticed.numeric_coordinates.add(node.name) diff --git a/tests/test_lowering.py b/tests/test_lowering.py index f743319f..ec5b480e 100644 --- a/tests/test_lowering.py +++ b/tests/test_lowering.py @@ -33,6 +33,7 @@ DimensionDeclaration, Divide, Dual, + ExpressionComparisonNode, ExpressionNode, Footprint, GroupSum, @@ -392,6 +393,40 @@ def test_a_constraint_where_is_a_mask_like_a_variable_s(): assert c.where == Mask(ParameterComparisonNode('load', '>', 0.0, ('snapshot',))) +def test_a_comparison_of_expressions_lowers_to_program_expressions_on_both_sides(): + """The resolved tree holds the core syntax tree; the program holds the vocabulary a consumer reads, and every mask is rebuilt so.""" + program = to_program( + override( + SHAPES_MODEL, + **{ + 'parameters.zc': {'dims': ['z']}, + 'variables.p.where': 'c <= 0.5 * k', + 'constraints.w': { + 'dims': ['g'], + 'where': 'c <= at(zc, by=lk2) + sum_back(c, over=g, within=2, by=lk2)', + 'expression': 'p <= c', + }, + }, + ) + ) + where = program.variables['p'].where + assert where is not None + assert where.root == ExpressionComparisonNode( + Parameter('c'), '<=', Multiply(Constant(0.5), Parameter('k')), ('g',) + ), 'the sides are lowered as a constraint side is, and the dims are what either side carries' + mask = program.constraints['w'].where + assert mask is not None and isinstance(mask.root, ExpressionComparisonNode) + assert isinstance(mask.root.right, Add) and isinstance(mask.root.right.left, At) + assert mask.names_read == frozenset({'c', 'zc', 'lk2'}), ( + 'the lookup a pullback and a partition read through is data the consumer binds too' + ) + + +def test_a_mask_with_no_arithmetic_is_the_same_mask_after_lowering(dispatch_program): + """Every other predicate node is already the program's own, so lowering hands it through unchanged.""" + assert dispatch_program.variables['p'].where == Mask(P_MAX_POSITIVE) + + def test_a_power_lowers_to_a_node_of_its_own(dispatch_schema): lowered = _Lowering(dispatch_schema, 't').expr(resolved('cost ** cost', dispatch_schema)) assert isinstance(lowered, Power), 'a variable-free power has a plan node of its own' diff --git a/tests/test_parser.py b/tests/test_parser.py index 3748a75f..a104af82 100644 --- a/tests/test_parser.py +++ b/tests/test_parser.py @@ -27,6 +27,7 @@ ) from math_spec._where_parser import ( UnresolvedComparisonNode, + UnresolvedExpressionComparisonNode, UnresolvedNameNode, UnresolvedPositionNode, parse_where, @@ -257,6 +258,35 @@ def test_a_where_string_parses_to_its_node(text, node_type, attrs): assert getattr(node, attr) == expected +@pytest.mark.parametrize( + ('text', 'node_type'), + [ + pytest.param('p > 0', UnresolvedComparisonNode, id='a-name-against-a-literal'), + pytest.param('x >= -1', UnresolvedComparisonNode, id='a-name-against-a-signed-literal'), + pytest.param('a == b', UnresolvedComparisonNode, id='a-name-against-a-name'), + pytest.param('position(t) == -1', UnresolvedPositionNode, id='a-position'), + pytest.param('p > 0.5 * q', UnresolvedExpressionComparisonNode, id='arithmetic-on-the-right'), + pytest.param('(a + b) <= c', UnresolvedExpressionComparisonNode, id='a-bracketed-sum-on-the-left'), + pytest.param('-p < 1', UnresolvedExpressionComparisonNode, id='a-negated-name'), + pytest.param('sum(p, over=g) >= k', UnresolvedExpressionComparisonNode, id='a-reduction'), + pytest.param('position(t) + 1 == 0', UnresolvedExpressionComparisonNode, id='position-inside-arithmetic'), + ], +) +def test_a_comparison_takes_the_expression_form_only_past_the_plain_shapes(text, node_type): + """`p > 0` keeps the node its dtype rule is written for; `p > 2 * q` is not cut short at `p > 2`.""" + assert isinstance(parse_where(text), node_type) + + +def test_a_bracketed_predicate_is_still_a_predicate(): + """`(a > 0) AND b` groups a comparison; only `(a + b) <= c` brackets arithmetic.""" + assert isinstance(parse_where('(a > 0) AND b'), AndNode) + + +def test_a_where_side_is_held_to_the_depth_an_expression_is(): + with pytest.raises(SchemaError, match='nests 121 deep'): + parse_where(' + '.join(['p'] * 120) + ' > 0') + + def test_and_binds_tighter_than_or(): assert parse_where('a OR b AND c') == OrNode( UnresolvedNameNode('a'), AndNode(UnresolvedNameNode('b'), UnresolvedNameNode('c')) diff --git a/tests/test_validation.py b/tests/test_validation.py index 2fcc591a..6e177c4c 100644 --- a/tests/test_validation.py +++ b/tests/test_validation.py @@ -134,9 +134,9 @@ def test_a_nonlinear_entry_is_refused_where_the_math_reads_it(self, patch, fragm is a reported quantity. The constraint and the objective read it and hit the divisor ban at their own ceiling, which is the whole point of grading rather than banning at declaration. The piecewise-link - position is `test_a_link_reading_a_nonlinear_entry_is_refused`; a bound - and a where, which reference no expression at all, are - `test_a_bound_or_where_cannot_name_an_expression`. + position is `test_a_link_reading_a_nonlinear_entry_is_refused`; a bound, + which references no expression at all, and a where, which reads one + as arithmetic, are `test_a_bound_or_where_cannot_name_an_expression`. """ with pytest.raises(LanguageError) as exc: _schema(**_NONLINEAR_ENTRY, **patch) @@ -153,13 +153,13 @@ def test_a_nonlinear_entry_is_refused_where_the_math_reads_it(self, patch, fragm ), pytest.param( {'constraints': {'cap': {'dims': ['g'], 'where': 'bad > 0', 'expression': 'p <= c'}}}, - "'bad' not found", + 'a where compares expressions, and one side names a variable', id='where', ), ], ) def test_a_bound_or_where_cannot_name_an_expression(self, patch, fragment): - """A bound and a where reference parameters/variables, never a named expression, so the name fails to resolve whatever the entry's body.""" + """A bound names a parameter and nothing else; a where reads the entry as arithmetic, and a body carrying a variable is refused there.""" with pytest.raises(LanguageError) as exc: _schema(**_NONLINEAR_ENTRY, **patch) assert fragment in str(exc.value) @@ -745,6 +745,95 @@ def test_a_rule_decided_without_data(self, patch, fragments): assert fragment in message +class TestArithmeticInAWhere: + """What a comparison of expressions may say in a where, decided with no data bound.""" + + @pytest.mark.parametrize( + ('patch', 'where'), + [ + pytest.param({}, 'c <= 0.5 * k', id='arithmetic-on-a-side'), + pytest.param({'macros.half': {'args': ['x'], 'template': 'x / 2'}}, 'c <= half(k)', id='a-macro'), + pytest.param({'expressions.e': 'c * 2'}, 'e > 0', id='a-named-expression-on-the-left'), + pytest.param({'expressions.e': 'c * 2'}, 'k < e', id='a-named-expression-on-the-right'), + pytest.param({'parameters.d': {'dims': ['h']}}, 'c <= at(d, by=lk)', id='a-pullback-through-a-lookup'), + pytest.param( + {}, 'c - shift(c, over=g, offset=1, edge=0) <= k AND position(g) > 0', id='a-translation-with-its-edge' + ), + ], + ) + def test_a_where_comparing_expressions_loads(self, patch, where): + spec = _schema(**patch, **{'variables.p.where': where}) + assert spec.variables['p'].where == where + + def test_a_reduction_on_a_side_leaves_the_frame_it_reduced(self): + spec = _schema( + constraints={'t': {'dims': [], 'where': 'sum(c, over=g) >= k', 'expression': 'sum(p, over=g) <= k'}} + ) + assert list(spec.constraints) == ['t'] + + @pytest.mark.parametrize( + ('patch', 'fragments'), + [ + pytest.param( + {'variables.p.where': 'c > 2 * q'}, + ('one side names a variable', 'built before variables exist'), + id='a-variable-inside-arithmetic', + ), + pytest.param( + {'constraints': {'x': {'dims': ['g'], 'expression': 'p <= c'}}, 'variables.p.where': 'dual(x) * 2 > 0'}, + ('one side reads a dual', 'test the data instead'), + id='a-dual-inside-arithmetic', + ), + pytest.param( + {'variables.p.where': 'tag * 2 > 0'}, + ("'tag' is declared dtype: str, and an expression is arithmetic",), + id='a-label-inside-arithmetic', + ), + pytest.param( + {'variables.p.where': 'c / (k + 1) > 0'}, + ('a divisor must be a single Constant/Parameter factor',), + id='a-divisor-that-adds', + ), + pytest.param( + {'variables.p.where': 'shift(c, over=g, offset=1) <= k'}, + ('shift() over a variable-free expression leaves vacated positions with no value',), + id='a-translation-with-no-edge', + ), + pytest.param( + {'variables.p.where': 'c * nope > 0'}, + ("'nope' not found",), + id='an-unknown-name-inside-arithmetic', + ), + pytest.param( + {'parameters.d': {'dims': ['h']}, 'variables.p.where': 'c > d * 2'}, + ("a where-comparison of expressions reads dims ['h'] outside the frame ['g']",), + id='a-side-outside-the-frame', + ), + ], + ) + def test_a_bad_comparison_of_expressions_is_refused_at_load(self, patch, fragments): + message = _refusal(**patch) + for fragment in fragments: + assert fragment in message + + def test_a_case_comparing_expressions_is_refused_as_undecidable(self): + """Two cases split by arithmetic cannot be proved apart without the numbers, and the rewrite is named.""" + message = _refusal( + expressions={ + 'e': { + 'dims': ['g'], + 'cases': { + 'wide': {'when': 'c > 2 * k', 'expression': 'c'}, + 'narrow': {'when': 'c <= 2 * k', 'expression': 'k'}, + }, + 'otherwise': 0, + } + } + ) + assert 'cannot be told apart before the data arrives: it compares expressions' in message + assert 'precompute the test as a boolean parameter' in message + + class TestTheFrontDoor: def test_a_list_of_models_is_not_a_model(self): """Composition is Python's, not the file's (#30) — and the refusal is the package's own, so the CLI's one except catches it.""" diff --git a/tests/typesetting/golden/latex.out b/tests/typesetting/golden/latex.out index 4b434197..ba3bc86b 100644 --- a/tests/typesetting/golden/latex.out +++ b/tests/typesetting/golden/latex.out @@ -49,6 +49,7 @@ \paragraph{Definitions} \begin{description} +\item[{$\mathrm{spend}^{\mathrm{cap}}$}] \texttt{spend\_cap} over $\mathcal{G}$ \item[{$\mathit{spend}$}] \texttt{spend} over $\mathcal{T}$ --- what a snapshot's dispatch costs \item[{$\mathit{lcoe}$}] \texttt{lcoe} (scalar) \item[{$\mathit{marginal\_price}$}] \texttt{marginal\_price} over $\mathcal{T} \times \mathcal{B}$ @@ -105,11 +106,16 @@ \text{ceiling} && \theta_{b} & \le \infty && \forall\, b \in \mathcal{B} \\ \text{always} && \mathit{spill}_{t} & \ge 0 && \forall\, t \in \mathcal{T} \\ \text{redundant} && \mathit{spill}_{t} & \ge 0 && \forall\, t \in \mathcal{T} \,:\, \mathit{spill}_{t} \text{ exists} \\ -\text{never} && \mathit{slack}_{t} & \ge 0 && \forall\, t \in \mathcal{T} \,:\, \bot +\text{never} && \mathit{slack}_{t} & \ge 0 && \forall\, t \in \mathcal{T} \,:\, \bot \\ +\text{margin} && p_{t,g} & \le \mathrm{p}^{\mathrm{max}}_{g} && \forall\, t \in \mathcal{T},\ g \in \mathcal{G} \,:\, \mathrm{p}^{\mathrm{max}}_{g} - \mathrm{p}^{\mathrm{min}}_{g} > \frac{\mathrm{cost}_{g}}{2} \\ +\text{ramped} && \mathit{slack}_{t} & \le \mathrm{load}_{t,b} && \forall\, t \in \mathcal{T},\ b \in \mathcal{B} \,:\, \mathrm{load}_{t,b} - \mathrm{load}_{t \boxminus_{0} 1,b} \le \mathrm{zone\_cap}_{\mathrm{zone\_of}(b)} \wedge \mathrm{pos}(t) > 0 \\ +\text{covered} && \sum_{t \in \mathcal{T},\ g \in \mathcal{G}} p_{t,g} & \le \mathrm{budget} && \text{where } \sum_{g \in \mathcal{G}} \mathrm{p}^{\mathrm{max}}_{g} \ge \mathrm{budget} \\ +\text{capped} && p_{t,g} & \le \mathrm{p}^{\mathrm{max}}_{g} && \forall\, t \in \mathcal{T},\ g \in \mathcal{G} \,:\, \mathrm{spend}^{\mathrm{cap}}_{g} > 0 \vee \neg \mathrm{is\_flexible}_{g} \end{align} \paragraph{Definitions} \begin{align} +\text{spend\_cap} && \mathrm{spend}^{\mathrm{cap}}_{g} & = \mathrm{cost}_{g} \cdot 2 && \forall\, g \in \mathcal{G} \\ \text{spend} && \mathit{spend}_{t} & = \sum_{g \in \mathcal{G}} p_{t,g} \cdot \mathrm{cost}_{g} && \forall\, t \in \mathcal{T} \\ \text{lcoe} && \mathit{lcoe} & = \frac{\sum_{t \in \mathcal{T},\ g \in \mathcal{G}} p_{t,g} \cdot \mathrm{cost}_{g}}{\sum_{t \in \mathcal{T},\ g \in \mathcal{G}} p_{t,g}} \\ \text{marginal\_price} && \mathit{marginal\_price}_{t,b} & = \lambda_{\mathrm{balance},t,b} && \forall\, t \in \mathcal{T},\ b \in \mathcal{B} \\ diff --git a/tests/typesetting/golden/markdown.out b/tests/typesetting/golden/markdown.out index fa7ee0a5..5b2ec5cc 100644 --- a/tests/typesetting/golden/markdown.out +++ b/tests/typesetting/golden/markdown.out @@ -49,6 +49,7 @@ every character a notation escapes, set as text: link\_to, 100% & \#1 costs \$5 | Symbol | Meaning | |---|---| +| $`\mathrm{spend}^{\mathrm{cap}}`$ | `spend_cap` over $`\mathcal{G}`$ | | $`\mathit{spend}`$ | `spend` over $`\mathcal{T}`$ — what a snapshot's dispatch costs | | $`\mathit{lcoe}`$ | `lcoe` (scalar) | | $`\mathit{marginal\_price}`$ | `marginal_price` over $`\mathcal{T} \times \mathcal{B}`$ | @@ -256,8 +257,38 @@ p_{t,g} \le \mathrm{eta}_{g} \cdot \mathrm{p}^{\mathrm{max}}_{g} \qquad \forall\ \mathit{slack}_{t} \ge 0 \qquad \forall\, t \in \mathcal{T} \,:\, \bot ``` +**`margin`** + +```math +p_{t,g} \le \mathrm{p}^{\mathrm{max}}_{g} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} \,:\, \mathrm{p}^{\mathrm{max}}_{g} - \mathrm{p}^{\mathrm{min}}_{g} > \frac{\mathrm{cost}_{g}}{2} +``` + +**`ramped`** + +```math +\mathit{slack}_{t} \le \mathrm{load}_{t,b} \qquad \forall\, t \in \mathcal{T},\ b \in \mathcal{B} \,:\, \mathrm{load}_{t,b} - \mathrm{load}_{t \boxminus_{0} 1,b} \le \mathrm{zone\_cap}_{\mathrm{zone\_of}(b)} \wedge \mathrm{pos}(t) > 0 +``` + +**`covered`** + +```math +\sum_{t \in \mathcal{T},\ g \in \mathcal{G}} p_{t,g} \le \mathrm{budget} \qquad \text{where } \sum_{g \in \mathcal{G}} \mathrm{p}^{\mathrm{max}}_{g} \ge \mathrm{budget} +``` + +**`capped`** + +```math +p_{t,g} \le \mathrm{p}^{\mathrm{max}}_{g} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} \,:\, \mathrm{spend}^{\mathrm{cap}}_{g} > 0 \vee \neg \mathrm{is\_flexible}_{g} +``` + #### Definitions +**`spend_cap`** + +```math +\mathrm{spend}^{\mathrm{cap}}_{g} = \mathrm{cost}_{g} \cdot 2 \qquad \forall\, g \in \mathcal{G} +``` + **`spend`** ```math diff --git a/tests/typesetting/golden/model.yaml b/tests/typesetting/golden/model.yaml index e1ebe59d..ee738311 100644 --- a/tests/typesetting/golden/model.yaml +++ b/tests/typesetting/golden/model.yaml @@ -82,6 +82,7 @@ sos: type: 2 expressions: + spend_cap: cost * 2 # a data-only entry, so a where may compare it spend: # a plain named expression: its symbol prints where it is used, its body once as a definition description: what a snapshot's dispatch costs expression: sum(p * cost, over=generator) @@ -198,6 +199,22 @@ constraints: dims: [snapshot] where: "false" expression: slack >= 0 + margin: # a mask comparing two expressions, which prints as the arithmetic it is + dims: [snapshot, generator] + where: "p_max - p_min > cost / 2" + expression: p <= p_max + ramped: # a translation under a comparison names its edge, a pullback reads through a lookup, and the position keeps the vacated row out + dims: [snapshot, bus] + where: "load - shift(load, over=snapshot, offset=1, edge=0) <= at(zone_cap, by=zone_of) AND position(snapshot) > 0" + expression: slack <= load + covered: # a reduction on a side of a scalar mask, so nothing is left to quantify + dims: [] + where: "sum(p_max, over=generator) >= budget" + expression: sum(p) <= budget + capped: # an expressions: entry on a side, read by the name the file gave it + dims: [snapshot, generator] + where: "spend_cap > 0 OR NOT is_flexible" + expression: p <= p_max objective: # a sense, a product of two variables, a power over two parameters, a power of one of those, and the summations a scalar objective spells out beside two scalar terms sense: maximize diff --git a/tests/typesetting/golden/typst.out b/tests/typesetting/golden/typst.out index 7f286e31..9aa19a14 100644 --- a/tests/typesetting/golden/typst.out +++ b/tests/typesetting/golden/typst.out @@ -38,6 +38,7 @@ every character a notation escapes, set as text: link\_to, 100% & \#1 costs \$5 / $italic("weight")$: `weight` over $cal(T) times cal(G)$ == Definitions +/ $upright("spend")^(upright("cap"))$: `spend_cap` over $cal(G)$ / $italic("spend")$: `spend` over $cal(T)$ --- what a snapshot's dispatch costs / $italic("lcoe")$: `lcoe` (scalar) / $italic("marginal_price")$: `marginal_price` over $cal(T) times cal(B)$ @@ -92,11 +93,16 @@ $ upright("budgeted") & italic("spend")_(t) & <= upright("budget") & forall t in upright("ceiling") & theta_(b) & <= infinity & forall b in cal(B) \ upright("always") & italic("spill")_(t) & >= 0 & forall t in cal(T) \ upright("redundant") & italic("spill")_(t) & >= 0 & forall t in cal(T) colon italic("spill")_(t) upright(" exists") \ - upright("never") & italic("slack")_(t) & >= 0 & forall t in cal(T) colon bot $ + upright("never") & italic("slack")_(t) & >= 0 & forall t in cal(T) colon bot \ + upright("margin") & p_(t,g) & <= upright("p")^(upright("max"))_(g) & forall t in cal(T), g in cal(G) colon upright("p")^(upright("max"))_(g) - upright("p")^(upright("min"))_(g) > frac(upright("cost")_(g), 2) \ + upright("ramped") & italic("slack")_(t) & <= upright("load")_(t,b) & forall t in cal(T), b in cal(B) colon upright("load")_(t,b) - upright("load")_(t minus.square_(0) 1,b) <= upright("zone_cap")_(upright("zone_of")(b)) and upright("pos")(t) > 0 \ + upright("covered") & sum_(t in cal(T), g in cal(G)) p_(t,g) & <= upright("budget") & upright("where ") sum_(g in cal(G)) upright("p")^(upright("max"))_(g) >= upright("budget") \ + upright("capped") & p_(t,g) & <= upright("p")^(upright("max"))_(g) & forall t in cal(T), g in cal(G) colon upright("spend")^(upright("cap"))_(g) > 0 or not upright("is_flexible")_(g) $ == Definitions #set math.equation(numbering: "(1)") -$ upright("spend") & italic("spend")_(t) & = sum_(g in cal(G)) p_(t,g) dot upright("cost")_(g) & forall t in cal(T) \ +$ upright("spend_cap") & upright("spend")^(upright("cap"))_(g) & = upright("cost")_(g) dot 2 & forall g in cal(G) \ + upright("spend") & italic("spend")_(t) & = sum_(g in cal(G)) p_(t,g) dot upright("cost")_(g) & forall t in cal(T) \ upright("lcoe") & italic("lcoe") & = frac(sum_(t in cal(T), g in cal(G)) p_(t,g) dot upright("cost")_(g), sum_(t in cal(T), g in cal(G)) p_(t,g)) \ upright("marginal_price") & italic("marginal_price")_(t,b) & = lambda_(upright("balance"),t,b) & forall t in cal(T), b in cal(B) \ upright("startup_cost") & upright("startup_cost")_(t,g) & = cases(upright("cost")_(g) & upright("if ") upright("pos")(t) = 0, upright("cost")_(g) dot 2 & upright("if ") upright("pos")(t) > 0 and upright("season_of")(t) = upright("'winter'"), 0 & upright("otherwise")) & forall t in cal(T), g in cal(G) $ diff --git a/tests/typesetting/test_golden.py b/tests/typesetting/test_golden.py index e93b0712..c40e8af1 100644 --- a/tests/typesetting/test_golden.py +++ b/tests/typesetting/test_golden.py @@ -132,17 +132,20 @@ def _rendered_trees() -> Iterator[object]: yield from resolved.expressions.values() -#: What resolution never hands the walk: the three nodes it types away, and the -#: three an expression only carries before names are resolved. The walk raises on +#: What resolution never hands the walk: the four nodes it types away, the three +#: an expression only carries before names are resolved, and the lowered form of +#: a comparison of expressions, which only a program carries. The walk raises on #: each rather than rendering it, so a fixture reaching one would be a bug in #: resolution rather than a case worth committing output for. UNRESOLVED = { 'UnresolvedNameNode', 'UnresolvedComparisonNode', + 'UnresolvedExpressionComparisonNode', 'UnresolvedPositionNode', 'NameNode', 'NameListNode', 'KeywordNode', + 'ExpressionComparisonNode', } #: A dataclass the walk steps *through* rather than renders: an arm has no @@ -192,6 +195,8 @@ def test_the_golden_model_calls_every_operator_in_the_language(): UNREACHABLE = { 'if isinstance(node, UnresolvedNode | KwargNode):', "msg = f'{type(node).__name__} reached the typesetter; resolve the expression first.'", + 'if isinstance(node, ExpressionComparisonNode):', + "msg = 'a lowered comparison reached the typesetter; it prints the resolved tree, which lowering rebuilds.'", 'if not isinstance(node, ComparisonNode):', "msg = f'{context}: expected a comparison, got {type(node).__name__}'", 'raise AssertionError(msg)', diff --git a/tests/typesetting/test_walk.py b/tests/typesetting/test_walk.py index 0db831e7..1a634a71 100644 --- a/tests/typesetting/test_walk.py +++ b/tests/typesetting/test_walk.py @@ -759,3 +759,13 @@ def test_a_string_value_in_a_where_prints_as_a_quoted_label(name: FormatName, fm assert fmt.quoted('gas_ccgt') in text unquoted = text.replace(fmt.quoted('gas_ccgt'), '') assert fmt.prose('gas_ccgt') not in unquoted, 'a string value is data, never words inside math' + + +@EVERY_FORMAT +def test_a_comparison_of_expressions_prints_as_the_arithmetic_it_is(name: FormatName, fmt: Format): + """`cost <= p_max / 2` on a quantifier renders each side as an expression, around the relation.""" + model = override(DISPATCH_MODEL, **{'variables.p.where': 'cost <= p_max / 2'}) + text = typeset(model, name, legend=False) + p_max = fmt.subscript(fmt.superscript(fmt.upright('p'), fmt.upright('max')), ['g']) + cost = fmt.subscript(fmt.upright('cost'), ['g']) + assert f'{cost} {fmt.operators["le"]} {fmt.fraction(p_max, "2")}' in text From 8b84589728e70b7d8f509a596b2bdcb45cd5bdee Mon Sep 17 00:00:00 2001 From: Felix <117816358+FBumann@users.noreply.github.com> Date: Tue, 15 Sep 2026 19:13:47 +0200 Subject: [PATCH 2/2] docs(language): the expressions page says why a case when: cannot compare expressions (#476) --- docs/reference/language/expressions.md | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-) diff --git a/docs/reference/language/expressions.md b/docs/reference/language/expressions.md index 7efb3a0f..7840c1d4 100644 --- a/docs/reference/language/expressions.md +++ b/docs/reference/language/expressions.md @@ -249,10 +249,21 @@ constraints: expression: shed >= load - ramp ``` -A case `when:` that compares expressions cannot be proved apart from its -neighbours before the data arrives, so it is refused there with the rewrite: -compare one parameter against a literal, or precompute the test as a boolean -parameter. +A case `when:` may not compare expressions. The loader proves the cases of a +[`cases:` block](#the-rules-that-keep-the-cases-apart) apart at load: no two of +them may claim one coordinate. It proves that by trying every value the masks +name. A comparison of expressions names no value, because only the data decides +whether `c > 2 * k` holds. There is nothing to try, so the loader refuses the +block: + +> `Named expression 'e'`: cases `wide` and `narrow` cannot be told apart before +> the data arrives: it compares expressions, whose values only the data decides +> — compare one parameter against a literal, or precompute the test as a boolean +> parameter and test that. Two cases claiming one coordinate would give it two +> values, so this is refused the way a proven overlap is. + +A variable's `where` and a constraint's `where` are not held to this, because +neither is proved apart from anything. ### `position()`