feat(language): a parameter says whether its table must cover every coordinate, so a row lost in preparation is not read as a mask

docs/reference/language/declarations.md

@@ -39,8 +39,30 @@ parameters:
 | ------------- | ------------------------------------------------------------ | --------------- |
 | `dims`        | required — the dimensions it is indexed by; `[]` is a scalar |                 |
 | `dtype`       | `float`, `int`, `bool`, `str`                                | default `float` |
+| `coverage`    | `total`, `masked`                                            | default `total` |
 | `description` | free text                                                    | default `null`  |
 
+**`coverage` says whether a missing row was meant.** A table short of a
+coordinate and a table that never had one look identical in the data, and they
+mean opposite things: `total` is the claim that every coordinate the `dims`
+reach has a value, so a row that went missing in preparation is an error rather
+than a mask; `masked` is the file saying the gap is the point — the parameter
+_is_ a mask, and a coordinate it leaves out is [absence](absence.md).
+
+```yaml
+parameters:
+  cost: { dims: [generator] } # total: every generator has one
+  ramp_limit: { dims: [generator], coverage: masked } # no row means no limit
+```
+
+Without it the reading is a consumer's to pick, and two consumers picking
+differently would build different models from one file and one table — so the
+declaration says it and no consumer guesses. **The default is `total`** because
+that is what the other rules already assume: a bound and a divisor
+[refuse absence outright](absence.md), so a parameter reaching either must
+cover its rows, and a file declaring `masked` in those positions is a load
+error naming the rewrite.
+
 **`dtype` is a claim about the values, and the column has to be it.** It
 decides four things — whether the name is a value in an
 [expression](expressions.md) at all, what a `where` comparison is checked

docs/reference/language/dimensions.md

@@ -65,7 +65,10 @@ open. A lookup's map obeys the same rule one axis over, and says so under
 `lookups` below: it is single-valued per label of `over`.
 
 _At most_ once, rather than exactly once: a coordinate with no row is
-[absence](absence.md), which is how a model masks.
+[absence](absence.md), which is how a model masks — and whether a given
+parameter meant to mask is
+[`coverage`](declarations.md), which it declares rather than leaving to be
+inferred from the table.
 
 ## `lookups`
 

examples/pypsa.yaml

@@ -67,9 +67,11 @@ parameters:
   Generator_ramp_limit_up:
     description: most a generator may raise its output between snapshots, per unit of nominal power; no value means no limit
     dims: [generator]
+    coverage: masked
   Generator_ramp_limit_down:
     description: most a generator may lower its output between snapshots, per unit of nominal power; no value means no limit
     dims: [generator]
+    coverage: masked
   Generator_ramp_limit_start_up:
     description: most output in the snapshot a unit starts, per unit of nominal power
     dims: [generator]
@@ -107,6 +109,7 @@ parameters:
   Generator_p_nom_mod:
     description: the module size a build comes in whole numbers of; no value means the build is continuous
     dims: [generator]
+    coverage: masked
   Generator_modules_installed:
     description: >-
       how many whole modules a committable build has in place: `Generator_p_nom
@@ -126,9 +129,11 @@ parameters:
   Link_ramp_limit_up:
     description: most a link may raise its flow between snapshots, per unit of nominal power; no value means no limit
     dims: [link]
+    coverage: masked
   Link_ramp_limit_down:
     description: most a link may lower its flow between snapshots, per unit of nominal power; no value means no limit
     dims: [link]
+    coverage: masked
   Link_p_nom:
     description: nominal power
     dims: [link]

examples/pypsa_linearized_uc.yaml

@@ -72,9 +72,11 @@ parameters:
   Generator_ramp_limit_up:
     description: most a generator may raise its output between snapshots, per unit of nominal power; no value means no limit
     dims: [generator]
+    coverage: masked
   Generator_ramp_limit_down:
     description: most a generator may lower its output between snapshots, per unit of nominal power; no value means no limit
     dims: [generator]
+    coverage: masked
   Generator_ramp_limit_start_up:
     description: most output in the snapshot a unit starts, per unit of nominal power
     dims: [generator]

examples/pypsa_multi_period.yaml

@@ -55,6 +55,7 @@ parameters:
   Carrier_max_growth:
     description: most capacity of a carrier that may be added in a period; no value means no limit
     dims: [carrier]
+    coverage: masked
   Carrier_max_relative_growth:
     description: share of the previous period's additions that may be added on top
     dims: [carrier]

schema/math-spec.schema.json

@@ -356,6 +356,15 @@
       "additionalProperties": false,
       "description": "A declared parameter with dims and dtype.",
       "properties": {
+        "coverage": {
+          "default": "total",
+          "enum": [
+            "total",
+            "masked"
+          ],
+          "title": "Coverage",
+          "type": "string"
+        },
         "description": {
           "anyOf": [
             {

src/math_spec/lowering.py

@@ -17,6 +17,7 @@
 
 import math_spec.program as program
 from math_spec.dimensions import dims_of
+from math_spec.errors import LanguageError
 from math_spec.expression_parser import (
     ArithmeticNode,
     BinaryOperatorNode,
@@ -38,7 +39,7 @@
 from math_spec.validation import to_spec
 
 if TYPE_CHECKING:
-    from collections.abc import Callable
+    from collections.abc import Callable, Mapping
     from pathlib import Path
     from typing import Any
 
@@ -100,7 +101,7 @@ def lower_program(expanded: _ExpandedSpec) -> program.Program:
         for name, how in derivations_of(block, ex).items()
     }
     parameters = {
-        name: program.ParameterDeclaration(tuple(pdef.dims), pdef.dtype, derivations.get(name))
+        name: program.ParameterDeclaration(tuple(pdef.dims), pdef.dtype, derivations.get(name), pdef.coverage)
         for name, pdef in expanded.parameters.items()
     }
 
@@ -164,6 +165,7 @@ def lower_program(expanded: _ExpandedSpec) -> program.Program:
         for sname, sdef in expanded.sos.items()
     }
     expressions = {name: _lower_expression(expanded, ns, name) for name in expanded.expressions}
+    _refuse_a_mask_where_absence_has_no_reading(parameters, variables, constraints, objective, expressions)
     return program.Program(
         parameters=parameters,
         variables=variables,
@@ -370,3 +372,43 @@ def _bound_expression(value: float | str) -> program.ExpressionNode:
     if isinstance(value, str):
         return program.Parameter(value)
     return program.Constant(value)
+
+
+def _refuse_a_mask_where_absence_has_no_reading(
+    parameters: Mapping[str, program.ParameterDeclaration],
+    variables: Mapping[str, program.VariableDeclaration],
+    constraints: Mapping[str, program.ConstraintDeclaration],
+    objective: program.ObjectiveDeclaration | None,
+    expressions: Mapping[str, program.ExpressionNode],
+) -> None:
+    """Refuse ``coverage: masked`` in the two positions rule 8 gives absence no reading.
+
+    A bound and a divisor are the positions where a missing value cannot read
+    as "contributes nothing": an absent bound is no bound rather than an open
+    one, and an absent divisor is no quotient at all. A parameter declaring
+    itself a mask therefore cannot stand in either, and the file says so before
+    any data arrives — where the same fault would otherwise surface as a bind
+    error against whichever rows the data happened to carry.
+
+    Both positions are read off the lowered declarations rather than the file,
+    so a parameter reaching one through a macro or a named expression is caught
+    on the same footing as one written there directly.
+    """
+    bounds = tuple(node for vdef in variables.values() for node in (vdef.lower, vdef.upper))
+    every = (
+        *bounds,
+        *(node for cdef in constraints.values() for node in (cdef.lhs, cdef.rhs)),
+        *((objective.expression,) if objective is not None else ()),
+        *expressions.values(),
+    )
+    positions = dict.fromkeys(program.parameters_of(*bounds), 'a bound')
+    positions |= dict.fromkeys(program.divisor_parameters(*every), 'a divisor')
+    for name, position in sorted(positions.items()):
+        if parameters[name].coverage == 'masked':
+            raise LanguageError(
+                f"parameter '{name}' is declared `coverage: masked`, and stands as {position}. "
+                f'A missing row is absence, and absence has no reading there — an absent bound is '
+                f'no bound rather than an open one, and an absent divisor is no quotient at all. '
+                f"Declare `coverage: total` on '{name}' where its table does carry every "
+                f'coordinate, or move the mask onto the declaration that wants it, as a `where:`.'
+            )

src/math_spec/model.py

@@ -91,6 +91,13 @@ def _reject_unknown_keys(cls, data: Any) -> Any:
 #: half, because a mask names all three kinds and reads the dtype the same way.
 DeclaredDtype = ParameterDtype | DimensionDtype
 
+#: What a parameter's table is required to carry. ``total`` is every coordinate
+#: its ``dims`` reach; ``masked`` says a missing row is deliberate — the
+#: parameter *is* a mask, and a missing row reads as the identity of the
+#: position it stands in. The two are indistinguishable in the data, which is
+#: why the declaration says which was meant rather than a consumer guessing.
+ParameterCoverage = Literal['total', 'masked']
+
 #: The domain a variable may declare.
 VariableDomain = Literal['continuous', 'integer', 'binary']
 
@@ -123,6 +130,7 @@ def _reject_unknown_keys(cls, data: Any) -> Any:
 #: The set form of each vocabulary above, for callers that want membership.
 DIMENSION_DTYPES = frozenset(get_args(DimensionDtype))
 PARAMETER_DTYPES = frozenset(get_args(ParameterDtype))
+PARAMETER_COVERAGE = frozenset(get_args(ParameterCoverage))
 #: The parameter dtypes that stand where a number belongs — a coefficient, a
 #: term, a divisor, a bound. A label selects and a flag masks; neither is one.
 NUMERIC_DTYPES: frozenset[ParameterDtype] = frozenset({'float', 'int'})
@@ -205,6 +213,7 @@ class ParameterBlock(_StrictBlock):
 
     dims: list[str]
     dtype: ParameterDtype = 'float'
+    coverage: ParameterCoverage = 'total'
     description: str | None = None
 
 

src/math_spec/program.py

@@ -78,6 +78,7 @@
     'OrNode',
     'Parameter',
     'ParameterComparisonNode',
+    'ParameterCoverage',
     'ParameterDeclaration',
     'ParameterDefinedNode',
     'ParameterDtype',
@@ -136,6 +137,10 @@
 #: What a parameter's values are (:data:`~math_spec.model.ParameterDtype`).
 ParameterDtype = _model.ParameterDtype
 
+#: Whether a parameter's table must carry every coordinate of its dims
+#: (:data:`~math_spec.model.ParameterCoverage`).
+ParameterCoverage = _model.ParameterCoverage
+
 #: What a masked variable's non-existence means
 #: (:data:`~math_spec.model.VariableAbsence`).
 VariableAbsence = _model.VariableAbsence
@@ -617,6 +622,10 @@ class ParameterDeclaration:
     #: follows: the caller binds a declared parameter, and an emitted one is
     #: built from the block's own breakpoints the way its derivation says.
     derivation: Derivation | None = None
+    #: Whether the table must carry every coordinate of *dims*. ``masked`` says
+    #: a missing row is absence the model means, so the declaration rather than
+    #: the data decides how a short table reads.
+    coverage: ParameterCoverage = 'total'
 
 
 @dataclass(frozen=True)

tests/test_lowering.py

@@ -747,3 +747,12 @@ def test_a_cased_expression_is_readable_by_the_name_the_file_wrote():
     assert isinstance(program.named_expressions['previous'], Cases), (
         'a cased expression reaches the program as the node, not as its fallback arm alone'
     )
+
+
+def test_a_parameter_covers_every_coordinate_unless_it_says_otherwise():
+    """The default is the claim rule 8 already makes of a bound and a divisor:
+    a table carries what its dims reach. ``masked`` is the file saying the
+    missing row was meant."""
+    program = to_program(override(SMALL_MODEL, **{'parameters.k.coverage': 'masked'}))
+    assert program.parameters['c'].coverage == 'total', 'a parameter that says nothing covers its dims'
+    assert program.parameters['k'].coverage == 'masked', 'and one that says so is carried through unchanged'

tests/test_validation.py

@@ -14,6 +14,7 @@
 
 from math_spec._yaml import parse_yaml
 from math_spec.errors import DimensionError, LanguageError, SchemaError
+from math_spec.lowering import to_program
 from math_spec.program import DimensionPositionNode
 from math_spec.resolution import Namespace, where_of
 from math_spec.validation import to_spec
@@ -918,3 +919,75 @@ def test_the_message_names_the_rewrite(self):
 
     def test_an_ordinary_name_still_loads(self):
         assert 'headroom_2' in _schema(**{'parameters.headroom_2': {'dims': ['g']}}).parameters
+
+
+class TestParameterCoverage:
+    """``coverage: masked`` is a claim, and rule 8 gives it two places it cannot stand.
+
+    A bound and a divisor are where a missing value has no reading that
+    contributes nothing, so a parameter calling itself a mask is refused there
+    before any data arrives — rather than binding and failing against whichever
+    rows the caller's table happened to carry.
+    """
+
+    @pytest.mark.parametrize(
+        ('patch', 'position'),
+        [
+            pytest.param(
+                {'parameters.c.coverage': 'masked', 'variables.p.bounds': {'upper': 'c'}},
+                'a bound',
+                id='masked-as-an-upper-bound',
+            ),
+            pytest.param(
+                {'parameters.c.coverage': 'masked', 'variables.p.bounds': {'lower': 'c'}},
+                'a bound',
+                id='masked-as-a-lower-bound',
+            ),
+            pytest.param(
+                {
+                    'parameters.c.coverage': 'masked',
+                    'constraints': {'cap': {'foreach': ['g'], 'expression': 'p / c <= 1'}},
+                },
+                'a divisor',
+                id='masked-as-a-divisor',
+            ),
+            pytest.param(
+                {
+                    'parameters.c.coverage': 'masked',
+                    'expressions': {'scaled': 'p / c'},
+                    'constraints': {'cap': {'foreach': ['g'], 'expression': 'scaled <= 1'}},
+                },
+                'a divisor',
+                id='masked-as-a-divisor-reached-through-a-named-expression',
+            ),
+        ],
+    )
+    def test_a_masked_parameter_is_refused_where_absence_has_no_reading(self, patch, position):
+        with pytest.raises(LanguageError) as exc:
+            to_program(override(SMALL_MODEL, **patch))
+        assert "parameter 'c'" in str(exc.value), 'the message names the parameter that has to change'
+        assert position in str(exc.value), f'the message names the position, which decides the rewrite: {position}'
+        assert 'coverage: total' in str(exc.value), 'the message names the rewrite, not only the fault'
+
+    @pytest.mark.parametrize(
+        ('patch'),
+        [
+            pytest.param(
+                {'constraints': {'cap': {'foreach': ['g'], 'expression': 'p * c <= 1'}}},
+                id='a-coefficient-reads-as-zero',
+            ),
+            pytest.param(
+                {'variables.p.where': 'c > 0'},
+                id='a-where-reads-as-false',
+            ),
+            pytest.param(
+                {'constraints': {'cap': {'foreach': ['g'], 'expression': 'p + c <= 1'}}},
+                id='a-term-reads-as-zero',
+            ),
+        ],
+    )
+    def test_a_masked_parameter_stands_wherever_absence_does_have_a_reading(self, patch):
+        """The guard is the two positions of rule 8 and not a third: a mask is
+        what a sparse table is *for*, so refusing it as a coefficient, a term or
+        a where would refuse the construct the declaration exists to describe."""
+        to_program(override(SMALL_MODEL, **{'parameters.c.coverage': 'masked', **patch}))