[FBumann]

Prompt: "Lets implement this proposal." — the proposal being, in the author's words:

direction	consumes	produces	operand over	disambiguate with
forward, sum	key	value	key	over=R.keycol or by=R.valuecol
backward, at	value	key	value	param[R.valuecol]

Scoped by two follow-ups: "Change at back to at()", so the backward row is at(param, by=R.valuecol) and the bracket form is not in this PR; and "Should we start from 509? Or from a less buggy branch?" — it starts from #493, which is the base branch. The dot syntax is #509's, by @coroa.

Note

The following content was generated by AI.

What this changes

A relational call names a column after the relation, and every keyword keeps one kind of value. into= is gone, and so is by= on a partition.

before	now	lands on
sum(p, by=zone_of, over=generator, into=zone)	sum(p, over=zone_of.generator)	[zone, period]
sum(p, by=slot_of, over=generator, into=bus)	sum(p, over=slot_of.generator, by=slot_of.bus)	[bus, period]
sum(p, by=slot_of, over=[generator, period], into=bus)	sum(p, by=slot_of.bus)	[bus]
sum(p, by=slot_of, over=generator, into=[bus, technology])	sum(p, over=slot_of.generator, by=slot_of.[bus, technology])	[bus, technology, period]
sum(p, by=slot_of, into=[bus, technology])	sum(p, by=slot_of), or written out, by=slot_of.[bus, technology]	[bus, technology]
sum(f, by=ends, over=line, into=bus1)	sum(f, by=ends.bus1)	[bus]
at(cap, by=ends, over=bus0)	at(cap, by=ends.bus0)	[line]
shift(x, along=t, by=cal, within=week)	shift(x, along=cal.t, within=week)	[t]
position(t, by=cal)	position(cal.t)	a mask over t
sum(p, by=gen_bus), at(price, by=gen_bus), sum(p, by=[gen_bus, gen_tech])	unchanged	[bus, period], [generator], [bus, technology, period]

Every row operates on these declarations:

dimensions:
  generator: { dtype: str }
  period: { dtype: int }
  zone: { dtype: str }
  bus: { dtype: str }
  technology: { dtype: str }
  line: { dtype: str }
  t: { dtype: int }
  day: { dtype: int }
  week: { dtype: int }

relations:
  zone_of: { key: [generator, period], value: zone } # 2 keys, 1 value
  slot_of: { key: [generator, period], value: [bus, technology] } # 2 keys, 2 values
  ends: { key: line, value: { bus0: bus, bus1: bus } } # 2 values over one dimension
  cal: { key: t, value: [day, week] } # a calendar, grouped by either column
  gen_bus: { key: generator, value: bus }
  gen_tech: { key: generator, value: technology }

parameters:
  cap: { dims: [bus] }
  price: { dims: [bus] }

variables:
  p: { dims: [generator, period], bounds: { lower: 0 } }
  f: { dims: [line] }
  x: { dims: [t] }

The rules, as the docs now state them:

• Forward. sum(x, by=rel) consumes every key column and lands on every value column. over=rel.k narrows the key columns consumed, the rest are joined on. by=rel.v narrows the value columns landed on. Both compose, and both name one relation. A side left unsaid is the whole side.
• A list of columns goes after the dot; a list of relations goes in brackets. by=slot_of.[bus, technology] lands on two columns of one relation. by=[gen_bus, gen_tech] groups through two relations at once, each walked whole, and its members are bare names, so by=[slot_of.bus, slot_of.technology] does not parse.
• Backward. at(x, by=rel) reads every value column at the key and lands on the whole key. by=rel.v narrows the value columns read.
• Partition. shift(x, along=rel.k), sum_back(x, along=rel.k) and position(rel.k) slide along the key column k and group by every value column. within=v narrows the value columns the group is made of. Nothing lands anywhere.
• A relation with two columns over one dimension needs the column named. ends holds bus0 and bus1, both over bus, and a frame carries bus once. So sum(f, by=ends) and at(cap, by=ends) are refused, and the call names one: by=ends.bus1.
• A bare relation is summed with the consumed columns named, over=rel.k, and lands on the rest of its key.
• over= beside by= names a column, never a dimension. A dimension is reduced by a second sum.

A bare by=rel on a two-key relation now consumes both keys rather than being refused, which is the proposal's default. The sum-walks-to-the-key refusal is gone, because the shape cannot be written any more.

The relation before a dot is a child node, so a macro formal binds there: sum(x, over=r.generator, by=r.bus), at(x, by=r.bus) and shift(x, along=r.snapshot, …) all take r= at the call. The resolved tree keeps the shape the program reads today, so lowering, the dim check and the typesetter are unchanged, and the golden .out files move only in one legend line.

Why

Two keywords for one direction, with the table named in a third, made the relational calls hard to read, and over= meant a dimension in one call and a column in the next. Writing the column after the relation shows what is consumed and what is landed on, lets the two compose (the one case #509 could not write), and keeps every guard main already has: distinct columns over one dimension, the list rules, the joined-dimension check and formal binding.

Refusals, each naming the rewrite

• sum(p, by=slot_of, over=period): beside by=, over= names the key columns the sum consumes, written after the relation: over=slot_of.<key column>. To reduce a dimension as well, sum twice: sum(sum(…, by=slot_of), over=period).
• sum(p, over=slot_of.generator, by=gen_bus.bus): names two relations, and one sum walks one relation. Name the same relation on both sides, or group through several with by=[slot_of, gen_bus].
• sum(p, over=slot_of.bus): over=slot_of.bus names ['bus'], which is not a key column of 'slot_of' — its key columns are ['generator', 'period']. Name key columns here, and the other side with by=slot_of.<value column>.
• sum(f, by=ends): lands on ['bus0', 'bus1'], two columns over ['bus'], and a frame carries each dimension once, so nothing says which column its coordinate is read at. Name one: by=ends.bus0.
• sum(x, by=conn) on a bare relation: nothing says which columns the sum consumes and which it lands on. Write over=conn.<column> among […] to consume some, or declare the columns it lands on under value:.
• at(q, by=lk.g): by=lk.g names ['g'], which is not a value column of 'lk' — its value columns are ['h'].
• shift(x, along=t, within=week): along= names a dimension, not a relation. Write along=<relation>.<key column> to slide inside its groups, or drop within=.
• shift(x, along=cal.week): 'week' is a value column of 'cal', and a partition slides along a key column — its key is ['t'].
• position(cal): To count inside the groups of a relation, write position(cal.<key column>).
• a formal bound to an expression before a dot: names columns of an expression, and columns belong to a relation.
• into=, over= on at, by= on a partition: the closed call shape.

Guards and their tests

Each of the nineteen new guards in resolution.py was neutralised in turn and its test_validation.py case run; every one failed without its guard. The list: over= beside a bracketed by=, a plain dimension beside by=, two relations, the key side and the value side of a sum, both bare-relation refusals, distinct columns consumed and landed on, the read's bare, side and distinct checks, within= beside a plain dimension, along= naming several columns, the partition's bare, key-column, unknown-column and key-column-in-within checks, and position on a name that is not a relation.

Coverage moved: a-sum-that-walks-to-the-key-is-a-read is deleted, since the shape cannot be written. by-a-two-key-relation-without-from became a-plain-dimension-beside-by, since a bare by= on a two-key relation now loads.

Verified

pixi is not installable in this session (the installer is blocked by the proxy with a 403), so the gates ran from a uv venv on Python 3.13 with the pinned ruff==0.16.1 and pyrefly==1.2.0:

• pytest -q -n 4: 1302 passed, 1 skipped (the whole suite, test_docs.py included).
• ruff check, ruff format --check: clean. pyrefly check: 0 errors. prettier --check on docs, examples and the README: clean.
• Generated files regenerated with their tools: tests.typesetting.golden, tools.notation, tools.spec_math, tools.gallery, tools.home_math, tools.schema. The .out files move in one legend line, shift(along=relation.d) for shift(by=relation); the math is unchanged.
• mkdocs build: no link or anchor warnings. --strict aborts here on fetching docs.python.org/objects.inv through the proxy, which is the environment, not a page.

Not run: pixi run ci as one command, typos, reuse lint, zizmor, taplo, compile-tex.

Deliberately not done: the bracket form param[R.valuecol] (the follow-up the author named), a dotted member inside by=[a, b] (a parse error, as before), and Walk.is_function_read, which is removed since nothing calls it.

🤖 Generated with Claude Code

https://claude.ai/code/session_018uvytQ8L5Kz5s5tZYpkwpp

[read-the-docs-community]

Documentation build overview

📚 math-spec | 🛠️ Build #34607865 | 📁 Comparing b55dc42 against latest (bc1d9d4)

  🔍 Preview build  

11 files changed · ± 11 modified

± Modified

• CHANGELOG/index.html
• examples/operators/index.html
• reference/notation/index.html
• reference/language/dimensions/index.html
• reference/language/expressions/index.html
• reference/language/operators/index.html
• reference/math_spec/model/index.html
• reference/math_spec/operators/index.html
• reference/math_spec/program/index.html
• reference/math_spec/resolution/index.html
• and 1 more...

[coroa]

I'd never mix by and over in sum. You either state what you want to sum, or what you want to keep!

[FBumann]

Closed in favor of #540

Will revisit for syntax discussions