[coroa]

Prompt: I'd like to change how relations work. [...] sum(ex, over=rel.k1) [...] sum(ex, by=rel.v1) [...] at is in principle an indexing operation, so i'd like to write it as such [...] price[zone_of]. [...] Then implement it. [...] commit early [and on a] new branch for the new implementations PR.

Note

The following content was generated by AI.

Implements the dot syntax PR #506 documents. Stacked on that docs PR. Every relation operator joins the operand on shared dims, then reduces; the retired at()/into=/partition-by=/"walk" are gone.

The design, the diff, and how it was verified

The surface

• sum(x, over=rel.k) sums key column k away; the value and other keys ride in.
• sum(x, by=rel.v) groups onto value column v; the whole key is summed away.
• x[rel] indexes: reads the value at the key, replacing at(x, by=rel). The operand is a name or (expr); x[a, b] indexes through several relations at once.
• shift/sum_back/position take along=rel.k / position(rel.k); within= stays; position has no bare form.
• The dot picks columns; a bare rel is all of them.

Three commits

1. feat(parser) — the grammar reads x[rel] and rel.col as unresolved nodes.
2. feat(language) — resolution rewrites the index to an internal index operator (at renamed); both sum forms and the index build one Join whose dim rule is uniformly (inner - consumed) | landed. The dead is_function_read guard on the key-join is dropped — no sum can be degenerate now that indexing is its own operator. Examples, the golden model, PyPSA models, and every fixture asserting the retired spelling migrated; the generated tables/pages regenerate from them.
3. chore — the internal Walk structure renamed to Join (you asked for the "walk" vocabulary gone); no behaviour change. The AST-traversal walk() is a different concept and stays.

Verification

• pixi run pytest — 1280 passed.
• pixi run pyrefly check src/ — 0 errors.
• pixi run ruff check — clean.
• pixi run compile-tex — 31 documents.
• pixi run docs-build — --strict, clean.
• All doc generators current; the schema regenerated with one corrected description.

Deliberately not done

• The program node for indexing is still named At internally (not Index) — it is not "walk" vocabulary, so renaming it was out of scope; say the word and it is a one-commit follow-up.
• Multi-relation index members are bare (mirroring sum(by=[a, b])); a dotted member in a list is not wired.

Departure from a Part 2 default: the branch lives in the primary checkout, not the ../wt/ worktree AGENTS.md prescribes, because the sandbox permits writes only inside the repo directory.

🤖 Generated with Claude Code

[read-the-docs-community]

Documentation build overview

📚 math-spec | 🛠️ Build #34627853 | 📁 Comparing 8936e13 against latest (479b11b)

  🔍 Preview build  

16 files changed · ± 16 modified

± Modified

• examples/operators/index.html
• examples/pypsa/index.html
• examples/pypsa_linearized_uc/index.html
• examples/pypsa_losses/index.html
• examples/pypsa_multi_period/index.html
• examples/pypsa_quadratic/index.html
• examples/pypsa_stochastic/index.html
• reference/notation/index.html
• reference/language/dimensions/index.html
• reference/language/expressions/index.html
• and 6 more...

[FBumann]

@coroa I have a few opinions:

1. I still dont like the new at() syntax. Could we stay with at() for now for the sake of only worrying about trelations? var[relation.key] is an easy drop in for at(var, single_kw) in a follow up. This reduces the surface we need to argue about, and the diff.

2. This notation has a shortcoming when summing over a relation with multiple values:

main:  sum(p, by=slot_of, over=generator, into=bus) <= cap

509: sum(p, over=slot_of.generator) <= cap
# REFUSED: the expression carries dims ['technology'] that are not in its dims: ['bus', 'period']
#   over= lands on every value column, so technology comes along

#509:  sum(p, by=slot_of.bus) <= cap

Assumes:

dimensions:
  generator:  { dtype: str }
  period:     { dtype: int }
  bus:        { dtype: str }
  technology: { dtype: str }

relations:
  slot_of: { key: [generator, period], value: [bus, technology] }

variables:
  p: { dims: [generator, period], bounds: { lower: 0 } }

---

As a side note for now:

I found a few bugs. Can you confirm they are bugs? Or am i misunderstanding a design choice as a bug?

• Two value columns over one dimension. cap[ends.[bus0, bus1]] and bare cap[ends] read bus1 only. _relation_node dedups the consumed dims, so the collision is gone before any check. Mirror on the sum side: sum(f, over=ends.line) lands on both bus columns unrefused. Main refuses both.
• Two relations on different keys. price[zone_of, tech_of] loads and the math has an unbound s. The index path in _relation_ref returns early with the first join's key only, and skips the repeated-landing check, so pz[zone_of, zone2] passes too.
• Dotted member in an index list. u[ends.bus0, gen_bus] drops the dot and reads both bus columns. _index keeps only r.name. The PR says "not wired", but it is accepted, not refused.
• Macro formal as a relation. sum(x, by=r) works, but x[r] and sum(x, by=r.bus1) are refused at the call site, and sum(x, over=r.line) crashes with KeyError: 'r'. The template check accepts the formal, but expansion never substitutes it: IndexNode.relations and ColumnRefNode.name are not children.

[coroa]

I really don't like issues, too much scattered information.

Re 1: I'm fine with separating at/indexing out. It does not change much. I do like indexing better.

Re 2: I guess the implementation is buggy then.

dimensions:
  generator:  { dtype: str }
  period:     { dtype: int }
  bus:        { dtype: str }
  technology: { dtype: str }

relations:
  slot_of: { key: [generator, period], value: [bus, technology] }

parameters:
  cap: { dims: [period, bus] }   # was that your assumption, as well?

variables:
  p: { dims: [generator, period], bounds: { lower: 0 } }

sum(p, by=slot_of.[period, bus]) <= cap

is what i imagine you should be saying for this case, and then my assumption that by can only name value columns was wrong. and it should be able to include key columns.

[FBumann]

Closed in favor of #540

Will revisit for syntax discussions