feat(language): a relation between dimensions, walked in the direction each call names

[FBumann]

Prompt: "I feel like having a purely relational lookup is much more capable! With the direction per operator…?" — "I want the relation to be well defined (one to one, one to many, many to many etc), and to make the data I track safe and actionable" — "I want the keywords to clearly say what happens" — "columns or dims? Or axes?"

Follow-up (#477): the names @brynpickering proposed in the thread, with by= kept.

Note

The following content was generated by AI.

A relation is a table between dimensions, key: names the columns that are unique together, and every call names which column it removes and which it lands on. Two keywords are renamed with it: shift and sum_back walk their axis with along= rather than over=, and sum_back(within=) becomes window=. sum(x, over=d) is unchanged from main.

#422 is on main, and this is the third proposal on top of it for #275 and #161, beside #428 (per:) and #433 (keys and a dot), which stay open until #275 is decided. It also covers #424, which #436 answers on top of #433.

The feature in one example

On main a lookup is an arrow. over: is the dimension it starts at and into: the one it lands on. sum(p, by=gen_bus) follows the arrow, at(x, by=gen_bus) follows it backwards, and those are the only two directions there are.

lookups:
  gen_bus: { over: generator, into: bus }

Here a relation is a table with one column per dimension it relates. key: names the columns that hold each combination once. The table has no direction. Each call says which column leaves the frame and which arrives.

relations:
  gen_bus:    { columns: [generator, bus], key: generator }                     # one bus per generator
  zone_of:    { columns: [generator, period, zone], key: [generator, period] }  # a generator's zone, per period
  ends:       { columns: {line: line, bus0: bus, bus1: bus}, key: line }        # a line's two ends, one table
  connection: { columns: [generator, bus] }                                     # no key: many-to-many

With p over [generator, period], f over [line, period] and price over [zone, period]:

sum(p, by=gen_bus, over=generator, into=bus)                           # [generator, period] → [bus, period]; main's sum(p, by=gen_bus), both sides defaulted
sum(p, by=zone_of, over=generator, into=zone)                          # [generator, period] → [zone, period]; period is joined on
sum(p, by=zone_of, over=period, into=zone)                             # [generator, period] → [generator, zone]; the same table the other way
at(price, by=zone_of, over=zone, into=generator)                       # [zone, period] → [generator, period]; each generator's zone price
sum(f, by=ends, over=line, into=bus1) - sum(f, by=ends, over=line, into=bus0)  # [line, period] → [bus, period]; a nodal balance through one table
sum(p, by=connection, over=generator, into=bus)                        # [generator, period] → [bus, period]; a bare relation, both ends named

A side the declaration decides may be left out. gen_bus has one key column and one value column, so sum(p, by=gen_bus) and at(price, by=gen_bus) are complete. zone_of has one value column, so into=zone may go, and over= stays because the key has two columns. ends has one key column, so over=line may go. connection has no key, so both stay.

A model on main with a one-key, one-value lookup keeps every by= call and every sum(over=) as it is. The declaration changes once: {over: generator, into: bus} becomes {columns: [generator, bus], key: generator}. A shift or sum_back changes over= to along=. The data it binds is unchanged.

What the key means

key: is the database word: the columns that identify a row. key: generator says each generator appears once, so the other columns are a function of it. key: [generator, period] says each pair appears once, and neither column need be unique on its own. The claim is checked when data is bound, so a generator on two buses is refused where a 0/1 parameter would have doubled the sum in silence (#161). The thread asked whether a key must be unique: yes, and the bind check is what enforces it.

Each cardinality is one declaration:

to say	write
many-to-one	{columns: [generator, bus], key: generator}
one-to-many	{columns: [generator, bus], key: generator} — the same table, walked the other way: sum(p, by=gen_bus) collects a bus's generators, at(x, by=gen_bus) reads a generator's bus
many-to-many	{columns: [generator, bus]}, no key
one-to-one	sayable — #424's self-map is {columns: {snapshot: snapshot, rep: snapshot}, key: snapshot} — but key: cannot narrow the bind check to it. The check stays many-to-one.

The key also decides which walks the table allows. This is the rule a reader most needs, because it answers "what can be removed":

a call	needs	because
sum(by=)	the key not wholly inside the produced and joined columns	a sum adds rows up. Walked to the key it finds one row per coordinate, which is a read, so it is refused toward at
at(by=)	the key inside the produced and joined columns	a read is one value per coordinate
shift, sum_back, position with by=	a key column over the walked dimension	a coordinate is in one group, or it has no neighbour
where: "zone_of == 'A'"	a key, and the column compared a value column	a comparison is one value per coordinate
where: connection (bare)	nothing	a row exists, or it does not

So sum may remove a key column (over=generator on gen_bus) or a value column (over=zone on connection), but sum(p, by=gen_bus, over=bus, into=generator) is refused: with generator the key, each generator finds one row and nothing is added. The rewrite is at.

A relation with no key: is a bare relation. sum walks it with both ends named, a bare where tests it, and nothing else touches it. Its columns not walked are joined on. That is what many-to-many can say, and all it can say.

The keywords

Five keywords are about relations. Each means one thing wherever it appears.

keyword	what it names	on sum	on at	on shift, sum_back, position
by=	the relation the call walks	optional	required	optional
over=	what leaves the frame: a column of by=, or a dimension when there is no by=	the key by default	a value by default	—
into=	what arrives: a column of by=	a value by default	the key by default	—
along=	the ordered axis walked and kept	—	—	required on shift and sum_back; position(d, …) takes the axis positionally
within=	the value columns the group is made of	—	—	all value columns by default

A default is taken from the declaration where it leaves one candidate. A table with two key columns or two value columns has no default on that side, and the call names it. The refusal lists the candidates. into= and within= without a by= are refused. over= without one names a dimension of the operand, which is main's sum(x, over=d).

over= and into= say the frame changes. along= and within= say it does not.

The renames

Three renames land with the feature. All three change models that load on main today. sum(x, over=d) is not one of them: its 235 call sites are untouched.

on main	here	why
shift(x, over=d), sum_back(x, over=d)	shift(x, along=d), sum_back(x, along=d)	over= on sum means removed, and on shift it meant walked and kept. sum keeps the word every reader says, and the operators that keep the axis take along=, which also carries that the axis is ordered. 226 call sites in src/, tests/, docs/, examples/.
sum_back(x, over=d, within=n)	sum_back(x, along=d, window=n)	within= now names a partition's group. The node was always Window and its field width. 71 call sites.
lookups: {l: {over: a, into: b}}	relations: {l: {columns: [a, b], key: a}}	the declaration is the feature. relations:, because a keyless table looks nothing up. columns: and not dims:, because a relation may name one dimension twice (ends, a self-map) and a frame never does. 27 files.

Counts are git grep on origin/main at 5656b11. per= was the other candidate for the group keyword and lost because per is this language's word for a frame: a constraint is one row per coordinate.

from=, to=, consume=, produce= and lookups: appear in this PR's commit history and nowhere on main. They were earlier spellings of over=, into=, within= and relations:, renamed inside the PR after the thread's comments, the last round in #477. The diff against main shows the final spelling only.

What is new

Nothing sayable on main is unsayable here. New:

• a map that varies along a second dimension (zone_of), walked from either key column
• a table with several value columns, walked to one, or landed on their product: sum(p, by=gen_bt, into=[bus, technology])
• a line's two ends, and a self-map (A lookup from a dimension into itself, so representative snapshots are sayable #424), through named columns
• a masked sum: sum(load * p, by=gen_bus) where the operand already carries bus joins on it, where main refused it
• one calendar table partitioning at day, week and season: shift(x, along=snapshot, by=cal, within=week)
• a many-to-many relation as a keyless relation, where main needs a parameter of ones
• two where forms: ends.bus0 != ends.bus1, and a bare where: connection
• a new refusal: a sum that walks to the key, since it reads rather than sums

How to review this

• The semantics are in src/math_spec/resolution.py (_walk, _partition_walk, _relation_ref), model.py (RelationBlock, the declaration rules) and program.py (RelationDeclaration, Walk). Those three files are 958 of the 1548 lines changed under src/.
• The rules a modeller meets are docs/reference/language/dimensions.md under relations, and each has a refusal case in tests/test_validation.py.
• The renames are four commits, each mechanical and readable on its own: 684fff5 (within=, window=), cf7ca1b (consume=, produce=), 1baa338 (columns:), 3d261e4 (feat(language): relations replace lookups, walked with over= and into= rather than consume= and produce= #477: relations:, over=, into=, along=).
• examples/ changes are the renames only: 138 lines out, 138 in.

The whole model, which loads on the branch, with the frame the loader reports for each row

dimensions:
  generator: {}
  zone: {}
  bus: {}
  technology: {}
  line: {}
  period: { dtype: int }

relations:
  gen_bus:    { columns: [generator, bus], key: generator }
  gen_bt:     { columns: [generator, bus, technology], key: generator }
  zone_of:    { columns: [generator, period, zone], key: [generator, period] }
  ends:       { columns: {line: line, bus0: bus, bus1: bus}, key: line }
  connection: { columns: [generator, bus] }

parameters:
  cost:     { dims: [generator] }
  demand:   { dims: [zone, period] }
  price:    { dims: [zone, period] }
  load:     { dims: [bus, period] }
  tech_cap: { dims: [bus, technology] }

variables:
  p: { dims: [generator, period], bounds: { lower: 0 } }
  f: { dims: [line, period] }

constraints:
  zone_balance: # remove generator, join period, land on zone → [zone, period]
    dims: [zone, period]
    expression: sum(p, by=zone_of, over=generator) >= demand

  history: # the same table, removed along its other key column → [generator, zone]
    dims: [generator, zone]
    expression: sum(p, by=zone_of, over=period) <= 100

  capped_revenue: # at removes the value column, joins period, lands on generator
    dims: [generator, period]
    expression: at(price, by=zone_of, into=generator) * p <= 1000

  nodal: # one table for both ends of a line
    dims: [bus, period]
    expression: sum(p, by=gen_bus) + sum(f, by=ends, into=bus1) - sum(f, by=ends, into=bus0) == load

  by_bus_and_tech: # one table landed on a product, in one join
    dims: [bus, technology, period]
    expression: sum(p, by=gen_bt, into=[bus, technology]) <= tech_cap

  reachable: # a bare relation, walked with both ends named
    dims: [bus, period]
    expression: sum(p, by=connection, over=generator, into=bus) <= 2 * load

  no_loop: # two columns of one table compared
    dims: [line, period]
    where: "ends.bus0 != ends.bus1"
    expression: f <= 10

  first_in_zone: # a partition walks the key column over period and groups by the value columns
    dims: [generator, period]
    where: "position(period, by=zone_of) == 0 AND zone_of == 'A'"
    expression: p <= 10

  within_bus: # grouped by one named value column of a two-value table
    dims: [generator, period]
    expression: p <= shift(p, along=generator, offset=1, edge=0, by=gen_bt, within=bus)

  parallel: # grouped by a pair of buses: lines between the same two ends are neighbours
    dims: [line, period]
    expression: f <= shift(f, along=line, offset=1, edge=0, by=ends)

objective: { sense: minimize, expression: sum(p * cost) }

$$\sum_{g \in \mathcal{G} : \mathrm{zone_of}(g, e) = z} p_{g,e} \ge \mathrm{demand}_{z,e} \qquad \sum_{g \in \mathcal{G} : (g, b) \in \mathrm{connection}} p_{g,e} \le 2 \cdot \mathrm{load}_{b,e} \qquad p_{g,e} \le p_{g \boxminus_{0}^{\mathrm{gen_bt.bus}(g)} 1,, e}$$

What each walk does to the frame

call	removed	landed on	joined on and kept	result
sum(p, by=gen_bus)	generator (the key)	bus (the value)	—	bus, period
at(x, by=gen_bus)	bus (the value)	generator (the key)	—	generator, …
sum(p, by=zone_of, over=generator)	generator	zone	period	zone, period
sum(p, by=zone_of, over=period)	period	zone	generator	generator, zone
sum(p, by=zone_of, over=[generator, period])	both keys	zone	—	zone
sum(q, by=zone_of, over=zone, into=generator)	zone (a value column)	generator	period	refused: one row per coordinate, a read; use at
at(price, by=zone_of, into=generator)	zone	generator	period	generator, period
sum(p, by=gen_bt, into=[bus, technology])	generator	both values	—	bus, technology, period
at(tech_cap, by=gen_bt, over=[bus, technology])	both values	generator	—	generator
sum(f, by=ends, into=bus1)	line	bus1	— (bus0 is not read)	bus, period
sum(p, by=connection, over=generator, into=bus)	generator	bus	—	bus, period
sum(load * p, by=gen_bus) with load[snapshot, bus]	generator	bus, already carried, so joined on too	—	snapshot, bus: a masked sum
shift(x, along=period, by=zone_of)	walks period	the group: zone	generator	unchanged; groups are (generator, zone)
shift(x, along=generator, by=gen_bt, within=bus)	walks generator	the group: bus	—	unchanged; groups are buses
shift(f, along=line, by=ends)	walks line	the group: (bus0, bus1)	—	unchanged; groups are pairs of buses
position(generator, by=gen_bt, within=[bus, technology]) == 0	walks generator	the group: both	—	first generator of each bus-and-technology

The ten rules, each decided at load

#	Rule	Refused with
1	columns: names at least two columns, each over a declared dimension, each column name once; a list names columns after their dimensions, a mapping gives names where two columns share a dimension; a column named like a dimension is over that dimension	"has 1 column(s)", "names dimension 'bus' twice under 'columns:'. Give the two columns roles: columns: {bus0: bus, bus1: bus}", "names column 'bus' after dimension 'bus', but the column is over 'line'"
2	key: names columns the relation has, each once, one per dimension, and not all of them	"has key column 'z', which is not one of its columns", "has two key columns over 'bus' … no frame carries a dimension twice", "has every column in its key, so the key determines nothing"
3	over= and into= name columns of the relation by= names — one each or a list each — with no column on both sides, none twice, and no two over one dimension; sum and at take both, a partition takes within= alone. An into= or within= with no by= is refused; an over= with no by= names a dimension	"over= and into= both name ['h']", "into=['h', 'h'] names a column twice", "into=['bus0', 'bus1'] names two columns over ['bus'], and the operand carries each dimension once", "shift(within=) names a column of a relation, and no by= names the relation"
4	A side the call leaves unsaid is taken from the declaration where it has exactly one candidate	"'zone_of' has 2 key columns (['generator', 'period']), and the call has to say which over= names"; "'gen_bt' has 2 value columns (['bus', 'technology']), and the call has to say which into= names"; "'rel' declares no key, so nothing says which column sum walks"
5	The other key columns are joined on: the operand carries each of their dimensions once; a value column not walked is not read; a produced dimension the operand already carries is joined on too; a sum or at lands on each dimension once	"joins on ['period'] (columns ['period'] of 'zone_of'), which the expression does not carry", "produces ['bus'] more than once"
6	at reads one value, so the key lies inside the produced columns and the joined columns; a bare relation is never read by at. The mirror holds for sum: a sum whose key lies there sums one term per coordinate, so it too is refused, toward at	"at reads one value per coordinate, and 'rel' is not single-valued in ['g'] at the columns the operand fixes"; "this sum walks to the key ['g'], so each coordinate has one term and nothing is added up — that is a read, which is at()'s"
7	A partition walks the one key column over the dimension it walks, joins on the other key columns, and groups by the value columns within= names — all of them where it names none; the group lands nothing, so it may pair two columns over one dimension; within= naming a key column is refused, and a bare relation partitions nothing	"'rel' declares no key, so no coordinate is in exactly one group", "has no key column over 'snapshot'", "within=['g'] names a key column of 'lz', and a partition groups by value columns"
8	A by= list walks each relation by its declared arrow; every entry removes the same dimensions and no two land on the same one	"a list walks each relation by its declared key and value, so a column keyword has nothing to name"
9	A where comparison reads a value column of a keyed relation at its key, named where the key determines several; two compared columns are over one dimension and their relations keyed over the same dimensions; a bare name tests that a row exists	"compares a column of 'rel', which declares no key", "'g' is a key column of 'lk', which the frame supplies rather than reads"
10	At bind: one column per declared column, named after it; every value a label of its dimension; a keyed table single-valued per key tuple	the consumer's DataError

Capability against #428 and #433, on the same base

"Yes" is sayable in one call with the language checking its shape; "workaround" is sayable another way. The same comparison runs on five loadable models in #453.

capability	main	#428 per:	#433 keys and dot	#437 relations
a map keyed by one dimension, walked both ways	yes	yes	yes	yes
a map that varies along a second dimension	no	yes	yes	yes
the same table walked from its other key	no	no, a second declaration	yes	yes
a table with several value columns, walked to one	no, one lookup per value	no	no	yes
landing on a product of two value columns	workaround: two tables and by=[…]	workaround	workaround	yes, into=[…]
reading a two-column slot at once	workaround	workaround	workaround	yes, over=[…]
removing two key columns at once	workaround: nested sums	workaround	workaround	yes, over=[…]
spreading a coarse quantity onto its keys (a value removed, the key landed on)	no	no	no	yes, via at (a sum here is refused, since it reads)
a masked sum, the produced dim already carried	refused	refused	refused	yes, joined on
a line's two ends in one table	no	no	no	yes
a self-map (representative snapshots)	no	no	no, #436 adds it	yes, by role columns
one calendar table partitioning at day, week and season	no	no	no	yes, within=
an unweighted many-to-many relation	workaround: an int parameter of ones	same	same	yes, a keyless relation
a weighted many-to-many relation	a parameter	same	same	same, a parameter
single-valuedness checked at bind	per over	per (over, *per)	per key tuple	per key tuple, and absent by choice
rules to state	5	7	7	10
call-site syntax beyond by=	none	none	a dot	over=, into=, within=
existing declarations rewritten	none	none	none	all, once

Nothing sayable in an earlier column is unsayable in a later one. The last three rows are the price.

Decisions taken, and why

The declaration makes the cardinality claim; the call makes the direction. key: says what is single-valued per what, which is a fact about the data, checked at bind. Which way a call walks is a fact about the call. What the key buys is not an arrow but a licence: a read, a partition and a comparison each need one value per coordinate, and only a key promises it. A bare relation is walked by sum alone, with both ends named, and tested by a bare where.

The keywords are the words a modeller already says. "Sum over generator" is standard mathematics, so over= is what a sum or at removes, and it composes with by=: sum(p, by=zone_of, over=generator). into= is where the result lands. along= is the axis shift and sum_back walk and keep, and it carries that the axis is ordered. The earlier spellings consume= and produce= named the walk's effect in the program's own words, and lost in the thread because they are energy-system vocabulary: a modeller writing a balance constraint should not meet consume= as syntax. Each of over= and into= takes a list, because a walk between two sets of columns needs two names. by= stays, because on a partition it reads as the group-by it is, and via= would suggest a path where there is none.

One rule went with it. sum used to refuse over= and by= together — "a lookup carries its own dimensions, so by= leaves over= nothing to add". They compose instead: by= names the table and over= names what leaves the frame. at_most_one_of is now unused on every built-in.

A partition takes within=, and sum_back's length is window=. The columns a result carries and the columns a walk groups by are two different things, and the frame changes in the first and not the second. A partition contains, so within= is the word; it held sum_back's length, which becomes window=, the name the program always used (the node is Window, its field width). per= was the cheaper candidate and lost because per is this language's word for a frame — a constraint is one row per coordinate — and a keyword renamed to stop implying the result gains a dimension should not imply one per bus.

The declaration says columns:, not over:. The reference has called them columns since #422 — "a table with one column per dimension it relates", "its value columns", "a key has one column per dimension" — and the schema key was the last place that disagreed. dims: would have been wrong beside parameters:, where dims: means a frame and a frame is a product of distinct dimensions; a relation's columns may name one dimension twice, which is what a self-map and a line's two ends need. With over: gone from the declaration, over= means one thing: the column a sum or at removes.

The block is relations:. A lookup promises a function, and here that holds only with key:. A keyless table such as connection: {columns: [generator, bus]} looks nothing up; it says which pairs exist, and a bare where: connection tests membership. relations: is right in both cases, and key: is what turns a relation into a function. The class names follow (RelationBlock, RelationDeclaration, Program.relations).

key: stays. The thread asked whether a modeller reads it as the database word. The reference opens the section with "the columns that are unique together", which is the claim, and the candidate unique: is an adjective the prose would still have to give a noun.

A produced dimension the operand already carries is joined on. #422 refused sum(load * p, by=gen_bus) as needing bus twice. Under a relational reading each term is restricted to the row where the generator's bus is the row's bus, a masked sum. The refusal is gone.

A sum toward the key is refused. With the key inside the produced and joined columns, each coordinate finds one row and nothing is added. That is at's read, and the refusal names it. The fan-out acceptance case in test_dimensions.py is dropped, and a-sum-that-walks-to-the-key-is-a-read in test_validation.py fails with DID NOT RAISE LanguageError when the guard is deleted.

Only key columns are joined on. Walking ends from line into bus1 must not join on bus0: a value column not walked is not read.

Roles are essential, not sugar. A self-map and a line's two ends are tables with two columns over one dimension, so columns need names of their own. columns: {bus0: bus, bus1: bus} is coroa's mapping form, taken whole.

Deferred. Several candidate keys (one-to-one): key: would widen to a list of keys, and each rule keeps its shape. piecewise: over: and sos: over: name a declaration's breakpoint axis, not a call keyword, and are untouched.

What consumers see, and where it lands

RelationDeclaration(name, columns, key), with columns as (role, dimension) pairs in declared order, is where a relation's roles, value columns and column dimensions live, and it is the same object the resolver reads and the program carries. Walk(relation, consumed, produced, joined) holds that declaration and three role tuples. GroupSum(operand, walks) and At(operand, walks) hold their walks alone, so a node cannot carry a coordinate without its walk; GroupSum.over and GroupSum.into are the dims that over= and into= name. Every partition is one Walk whose consumed is the key column over the walked dimension and whose produced is the group columns — Translate.partition, Window.partition and DimensionPositionNode.partition alike. Program.relations is a dict by name. The two relation-only where forms name.col OP value and name.a OP name.b are new grammar, as is a grouped position(…).

• operators.py: a role kwarg kind and a dimension_or_role one, which is what lets over= name a dimension alone and a column beside by=; kind_of takes the call's by= to decide. at_most_one_of is now unused.
• _expression_parser.py: no dotted names; over=/into= are ordinary kwargs, a name or a bracketed list.
• _where_parser.py: name.column on either side of a comparison; position(d, by=l[, within=c | [c, …]]).
• model.py: RelationBlock with columns (list or mapping) and key; rules 1 and 2. The parsed (role, dimension) pairs are RelationBlock.pairs, since columns is now the field.
• resolution.py: _walk (rules 3, 4, 6), _partition_walk (rule 7), _relation_ref (rules 5 and 8), _relation_column and the pair rule (rule 9).
• dimensions.py, lowering.py: sum reduces what over= names; the partitions are unchanged.
• piecewise.py: the three constraints the expansion writes are sums, and carry the keyword.
• program.py, separability.py, exclusivity.py: the declaration, the walk, and a rank subject that carries its partition.
• typesetting: a keyed read prints as name(key…) or name.col(key…), a bare relation as (…) ∈ name; the legend prints → for a keyed table and ⊆ for a bare one.
• docs and tests: the relations section rewritten around the relation, the key and the walk; every declaration rule and call-site rule above has a refusal case in test_validation.py, every walk in the frame table a dims case in test_dimensions.py, the lowering test covers every node, and the golden model prints in three formats.

Verified

CI is green on 3d261e4, the current head, with #477 merged in. That covers pyrefly, typos, taplo, zizmor, mkdocs build --strict and compile-tex.

pixi.sh is blocked in the sessions that built this, so every local run is a uv venv on Python 3.13 with the pinned ruff. On 3d261e4: pytest -q -n auto gives 1270 passed, 1 skipped. ruff check and ruff format --check clean on the pinned 0.16.1, prettier --check clean on the pinned 3.9.3, typos and reuse lint clean, every generator re-run including the schema, which lists columns as the relation's required key. The whole model above loads through to_spec on 3d261e4, and the refused row of the frame table is refused with the sentence rule 6 quotes.

Coverage moved with the rules: over-and-by-together asserted the rule that is gone and its case is deleted, replaced by a dims inference (by-and-over-compose) that sum(p, by=gen_bz, over=generator, into=bus) reaches the same frame as the defaulted call; from-without-by is into-without-by, since an over= with no by= is a dimension.

The conflict with main from #429's foreach: to dims: rename is resolved in fec2851.

Why

#433 answered coroa's design by keeping the lookup a function and letting the call pick which key it walks. The objection was that a table has no direction until something walks it, and the direction belongs to the verb. This PR takes that seriously: the declaration keeps the one claim only the data can be held to, and every direction is the call's.

🤖 Generated with Claude Code

https://claude.ai/code/session_01FD5LpGRzAWdi5sKWXDdnHC
https://claude.ai/code/session_014DGTkrKPqMWnTnhzmASgCn
https://claude.ai/code/session_01FPJEkH4kkG9onVCpYkBTM7

[read-the-docs-community]

Documentation build overview

📚 math-spec | 🛠️ Build #34575351 | 📁 Comparing 3d261e4 against latest (62564a2)

  🔍 Preview build  

48 files changed · + 6 added · ± 39 modified · - 3 deleted

+ Added

• about/limits/index.html
• about/what-counts-as-public-api/index.html
• howto/check/index.html
• howto/installation/index.html
• howto/print/index.html
• howto/regimes/index.html

± Modified

• index.html
• CHANGELOG/index.html
• contributing/index.html
• examples/index.html
• about/what-counts-as-language/index.html
• examples/commitment/index.html
• examples/dispatch/index.html
• examples/operators/index.html
• examples/pypsa/index.html
• examples/pypsa_linearized_uc/index.html
• and 29 more...

- Deleted

• installation/index.html
• about/ceiling/index.html
• about/what-counts-as-a-verb/index.html

[FBumann]

@brynpickering @coroa @FabianHofmann I think I found the most capable and complex form of lookups with this.
The key idea is that a lookup only provides a mapping and information about wether columns are unique or not (key).

Everything else is determined where its used: operator(from=..., into=...)

This makes it much more obvious how a lookup is used in an operator and it can be reused in multiple places.
And the spec can validate that for example at() can only use akey column!

Happy to discuss naming etc.

[brynpickering]

@FBumann I can see the power that this has to define pretty much anything we want but with the risk that it is quite confusing for users to both read and write the YAML math.

• key as the term used in the lookups doesn't feel quite right. Although I understand it as a dictionary key, it requires some Python understanding to get that syntax
• We could converge on dims or foreach instead of over to align with other blocks, since it's defining the dimensions involved in the lookups?
• You say one-to-one isn't something that is available; how does this work with A lookup from a dimension into itself, so representative snapshots are sayable #424?

[FBumann]

@bryn I iterated with claude on the "key" concept, because i wasnt fully convinced either. But I am now.

• The key defined which combination of columns is the "primary key", like in a database. This one needs to be unique.
  The others can be duplicates. THis allwos to define every data contract needed (one to many, many to one, many to many etc).
• Yes, probably. I also have an issue chnaging foreach to `dims everywhere feat(language): a variable, a constraint and a cased expression declare their shape as dims, as a parameter does #429
• one to one is sayable, but key cant narrow the the data contract to it. Thats the only limitation. A lookup from a dimension into itself, so representative snapshots are sayable #424 is sayable! Only the data contract isnt super strict.

About confusion: I think its much less confusing and surprising than the alternative: having the lookup directional and having "magic operators" whose behaviour depends on how the lookups direction is set. But Id like to discuss this based on examples

[FabianHofmann]

This feels very generalized and neat! I really like this

---

---

having the unique keys/key-combinations defined is a clever way to deal with things. I would also suggest to think about argument names further. In particular, I find the from keyword-attribute in the sum function weird. Why not over?

[FBumann]

@FabianHofmann I dodnt oterate on the keywords per operator. But I think its a key feature of this proposal to be able to choose more intuitive names per operator!
Ill work on it tomorrow. You can also propose sth if you want to

[FabianHofmann]

🎉