A lookup conditioned on a second dimension: `per:`, so a generator's zone may change by period

[FBumann]

This is the last addition to lookups making them fully capable I think. It makes every mapping sayable, with one dim consumed and one dim generated. SO the mapping says the direction, and operators stay simple.

This is the last addition to lookups making them fully capable, I think. Every function between dimensions becomes sayable — one dim consumed, one generated, any number conditioned on — and many-to-many stays an incidence parameter by design. The mapping fixes the arrow and the verb picks the way along it, so operators stay simple: by= takes a lookup, and nothing else changes.

Note

The following content was generated by AI. Moved here from fluxopt/specsolve#1163, where the design was worked out against both lpspec lanes; the language question is math-spec's.

A lookup today is keyed by over: alone: zone_of: {over: generator, into: zone} gives every generator one zone for the whole model. A map that changes along another dimension — a generator's bidding zone by period, a unit's balancing area by season, a plant's owner by year — has no declaration. #7 already sketches the case (bidding_zone: on: [entity, investment_period]); this is the concrete proposal for it.

Hands on: bidding zones that move between periods

Two generators, two zones, two investment periods. In 2030 g1 bids in A and g2 in B; in 2050 they swap. g1 costs 1, g2 costs 5. Demand: 10/10 in 2030, 30 in A and 10 in B in 2050.

Today the relation is sayable only as a 0/1 membership parameter contracted by an ordinary sum:

dimensions:
  generator: {values: [g1, g2]}
  zone: {values: [A, B]}
  period: {values: [2030, 2050], dtype: int}
parameters:
  cost: {dims: [generator]}
  demand: {dims: [zone, period]}
  in_zone: {dims: [generator, zone, period]}     # 1.0 where the generator bids in that zone that period
variables:
  p: {foreach: [generator, period], bounds: {lower: 0}}
constraints:
  zone_balance:
    foreach: [zone, period]
    expression: sum(p * in_zone, over=generator) >= demand
objective: {sense: minimize, expression: sum(p * cost)}

Run on both lpspec lanes (relational / linopy agree in every row):

in_zone data	objective
the honest map — g1 in A then B, g2 the reverse	220.0	what the model means
the map frozen at 2030	140.0	the only thing a static lookups: entry can say today
g1 given both zones in 2030 (a slip)	170.0	a legal model that says what the modeller did not mean — and nothing can tell

The third row is the point, and it is not a lane bug: (g1, A, 2030) and (g1, B, 2030) are two distinct keys of a parameter, so the file says — correctly, literally — that g1's 2030 output counts toward both zones' balance, and 170.0 is that model's optimum. What is wrong is the intent, one zone per generator per period, which lives only in the modeller's head: a lookup is keyed by over: alone, so the relation had to be flattened into a parameter to gain the period key, and a parameter carries no cardinality claim for the binder to check. What is missing is not a capability but the declaration that turns the slip into a refusal.

Proposed spelling

One new key on a lookup, and one rule:

lookups:
  zone_of: {over: generator, into: zone, per: [period]}

over: is consumed, into: is produced, per: is joined on and passed through. Nothing changes at the call site, and both directions keep working:

sum(p, by=zone_of)             # p[generator, period]   → [zone, period]
at(zone_price, by=zone_of)     # zone_price[zone, period] → [generator, period]  — the price of the zone this generator sat in *that period*

The model above becomes:

lookups:
  zone_of: {over: generator, into: zone, per: [period]}
constraints:
  zone_balance:
    foreach: [zone, period]
    expression: sum(p, by=zone_of) >= demand

with in_zone gone, and its data arriving as a tidy table (generator, period, zone) — the honest admission that this shape is relation-sized, where the single-source case rides free as a column on the dim's index.

Checked at bind: values are labels of into (the existing containment check, verbatim), and at most one row per (over, *per) — the third row above becomes a refusal at bind rather than a legal model. Checked at load: every per: dim is declared, and the operand of sum/at/where carries it (refused otherwise, with the message a frame lacking a dim already gets).

In the math it is the difference between

$$\sum_{g ,:, \mathrm{zone_of}(g) = z} p_{g,p} \qquad\text{and}\qquad \sum_{g ,:, \mathrm{zone_of}(g, p) = z} p_{g,p}$$

— one restriction whose value depends on a dimension that survives the sum.

Why not composite over: [generator, period]

It is not a smaller version of this but a lossy one:

• Recoverability runs one way. With per:, the composite behaviour is a follow-up call, sum(sum(p, by=zone_of), over=period). With composite over:, period is contracted inside the operator, captured by the $\sum$ and unreachable from the $\forall$.
• at() has no coherent reading. at produces the consumed side, so a map keyed (generator, period) → zone would produce both — and period is already a dim of zone_price. Its only sensible resolution is an equi-join on the shared column, i.e. per: reached by name coincidence, behaving as a conditioned map when the operand happens to carry the dim and fanning out to a product when it does not. sum would work, at would be a coin flip, and the two would stop being adjoints.

Where multiplicity lives, position by position:

position	multiple?	why
over:	no	the contraction captures the index
per:	yes	join keys compose; each adds one free variable to the restriction
into:	no	one arrow is one function; two targets are two lookups
by= on the call	yes, already (lpspec by=[zone_of, tech_of])	which maps an aggregation groups through is a fact about that call, not about any map

A list by=[l, m] is a conjunction of restrictions each a function of the contracted dim alone, so no list of static lookups produces a $p$-dependence; per: cannot produce a (zone, tech) grouping either. They are orthogonal.

Relation to #7 and #6

#7's on: [entity, investment_period] under dimensions: is the same relation with the map living on the target dimension. per: keeps it on the lookup, where lpspec's by= collapse already put every fact-about-the-map (over, into, values, dtype), so by='s grammar needs no change to accept one. Whether the home is dimensions: or lookups: is #7's question; the cardinality rule — one target per (over, *per), checked at bind — is this one's, and holds in either home.

Repro (lpspec, both lanes)

import pandas as pd, yaml
from tests.differential import differential   # PYTHONPATH=. in the lpspec checkout

MODEL = yaml.safe_load("""<the yaml above>""")
cost = pd.DataFrame({'generator': ['g1', 'g2'], 'value': [1.0, 5.0]})
demand = pd.DataFrame({'zone': ['A', 'B', 'A', 'B'], 'period': [2030, 2030, 2050, 2050], 'value': [10.0, 10.0, 30.0, 10.0]})
def membership(rows):
    return pd.DataFrame(rows, columns=['generator', 'zone', 'period']).assign(value=1.0)
cases = {
    'honest': [('g1', 'A', 2030), ('g1', 'B', 2050), ('g2', 'B', 2030), ('g2', 'A', 2050)],
    'frozen at 2030': [('g1', 'A', 2030), ('g1', 'A', 2050), ('g2', 'B', 2030), ('g2', 'B', 2050)],
    'g1 in both zones in 2030 (legal, unintended)': [('g1', 'A', 2030), ('g1', 'B', 2030), ('g1', 'B', 2050), ('g2', 'B', 2030), ('g2', 'A', 2050)],
}
for name, rows in cases.items():
    with differential(MODEL, {'cost': cost, 'demand': demand, 'in_zone': membership(rows)}) as run:
        print(name, run.result.objective, run.oracle)
# honest 220.0 220.0 / frozen at 2030 140.0 140.0 / g1 in both zones in 2030 170.0 170.0

lpspec main at d2c9d1d9, math-spec alpha.21.

[FBumann]

Prompt: "#437 was merged / Close all related issues and stale PRs"

Note

The following content was generated by AI.

Shipped by #437, merged as 3284926 and released in v0.0.0-alpha.89 — as a composite key rather than as per:.

A map that varies along a second dimension is a relation whose key names both columns:

relations:
  zone_of: { columns: [generator, period, zone], key: [generator, period] }

sum(p, by=zone_of, over=generator) removes generator, joins on period and lands on zone. The same table walks the other way with over=period, which per: could not do without a second declaration.

The safety point this issue made is the part worth recording as kept: the key is a claim checked when data binds, so a generator on two buses is refused where a 0/1 parameter would have doubled the sum in silence.

per: was the competing proposal (#428, now closed). It lost on capability rather than on spelling — the comparison is in #437's body and the five models are in #453.

---

Generated by Claude Code