spec: rows emptied by absence are dropped silently; revisit strict refusal

[FabianHofmann]

In the long term I would like to have things strict, but for now this is fine. Tracking this so we can readdress it later.

Note

The following content was generated by AI.

Describe the feature you'd like to see

With the move to mathspec 0.2.0 in #958, linopy/spec/builder.py::_check_live no longer refuses a constraint row that absence empties. It drops the row silently. This follows mathspec's documented semantics: absence spread through arithmetic "takes the row with it", and the first row of a bare shift "is not built".

Before (0a725f4), such a row raised SpecDataError ("N row(s) hold no variable term"), because a row that silently disappears no longer binds.

The current state after #958:

row	behaviour
absent (null) row, e.g. x - shift(x) at the first snapshot	dropped silently
bare masked Variable as a term	not checked
present row with no variable, e.g. absence: zero against a binding side	refused

Strict is the long-term goal. Open questions:

• Opt-in strictness, e.g. add_spec(..., strict=True), or strict by default with an explicit edge policy in the spec (shift(..., edge=...))?
• Warn at least with the count of rows dropped per constraint?
• Does mathspec need a language-level way to state "this row may be dropped", so strict engines can tell intended from accidental absence?

Reasons it is relaxed for now: pypsa.yaml, its netcdf round trip and the spec_pypsa benchmark rely on a bare shift dropping the first snapshot.

Minimal example

import pandas as pd
import xarray as xr

import linopy

linopy.options["semantics"] = "v1"

spec = """
dimensions:
  snapshot: { dtype: int }
variables:
  p:
    dims: [snapshot]
    bounds: { lower: 0 }
constraints:
  no_faster_than_before:
    dims: [snapshot]
    expression: p <= shift(p, along=snapshot, offset=1)
objective: { sense: minimize, expression: sum(p) }
"""
sources = xr.Dataset(coords={"snapshot": pd.Index([0, 1, 2], name="snapshot")})
m = linopy.Model.from_spec(spec, sources)
print(m.constraints["no_faster_than_before"].labels.values)
# [-1  1  2]   -> row at snapshot 0 is dropped without error or warning

Tests pinning the current behaviour: test/test_spec_builder.py::test_absence_takes_the_row_with_it and ::test_a_row_the_data_emptied_of_variables_is_refused.

[FabianHofmann]

Note

The following content was generated by AI.

Related hole found while fixing #958 (f186ec8). Since then, arithmetic on present data that turns non-finite is refused (evaluate.check_finite), so a NaN from 0/0 or inf - inf can no longer pass as absence. x + sum(a - b, by=...) is caught. But the difference of two grouped sums is not: the model builds silently, with rhs 0 instead of an error.

import numpy as np
import pandas as pd
import xarray as xr

import linopy

linopy.options["semantics"] = "v1"

spec = """
dimensions:
  generator: { dtype: str }
  bus: { dtype: str }
relations:
  gen_bus: { key: generator, values: bus }
parameters:
  a: { dims: [generator] }
  b: { dims: [generator] }
variables:
  x:
    dims: [bus]
    bounds: { lower: 0 }
constraints:
  row:
    dims: [bus]
    expression: x + sum(a, by=gen_bus, over=generator, into=bus) - sum(b, by=gen_bus, over=generator, into=bus) >= 1
objective: { sense: minimize, expression: sum(x) }
"""
generator = pd.Index(["g1", "g2"], name="generator")
a = xr.DataArray([np.inf, 1.0], coords=[generator])
gen_bus = xr.DataArray(["n1", "n1"], coords=[generator])
sources = {"generator": generator, "a": a, "b": a, "gen_bus": gen_bus, "bus": pd.Index(["n1"], name="bus")}
m = linopy.Model.from_spec(spec, sources)
print(m.constraints["row"].rhs.values)
# prints [0.]; expected SpecDataError (inf - inf at bus='n1')

Likely cause: the grouped sum's fillna(0) (or its reindexing) hides the non-finite operand before the subtraction is checked. Unverified.

[FabianHofmann]

Note

The following content was generated by AI.

The grouped-sum case above is fixed in 81c8eed: data summands a term's constant adds up across a sum or a comparison are now checked too (x + a - a, x + a >= a, sum - sum, shift - shift). Still not caught: data subtracted after a scaled term, e.g. (x + a) * 2 - 2 * a, since summands are not distributed through * or /.