feat(program): a sum through a relation is a sum over the axes its join opens

[FBumann]

Prompt: Look into 638. Will the conceptual change in 605 work with it ? Or even improve it? … Do a new branch from 605 and do a stacked PR onto 638. … Join, then mean. THats conceptually the best we can have. Can we get there again? … Do it

Note

The following content was generated by AI.

#605's join and group-by, on #638's resolver. A Join opens an axis named relation.column for each column it drops. sum(by=) is a Sum over those axes, and at() is the bare Join. Each node's frame is its own. #605 is unchanged.

What this changes

The branch starts at #605's head (53ba4a2) and merges #638's head (3270e48). The diff against #638 is #605's change, carried onto the resolver that builds program nodes. It keeps #605's two nodes and adds one thing: the axes a join opens.

• Direction, GroupSum and Pullback are gone. Join(operand, columns) is the one relation node. JoinColumns holds the joined and grouped roles, with dropped, added, kept, one_row_per_group and axes derived.
• A join's frame is the operand's dims, less the dims joined on, plus the dims grouped by, plus one axis per column joined on and not grouped by. The axis is named for the relation's column, for example zone_of.generator, and not for its dimension. A dimension name holds no dot, so an axis never meets one. The spelling is the one a where: already reads a column with.
• A lookup opens no axis. Where the grouped columns hold the key, they determine every other column. at(x, by=R, over=a, into=b) lowers to the bare Join.
• A grouped sum is the group-by after the join. sum(x, by=R, over=a, into=b) lowers to Sum(Join(x, cols), over=('R.a',)). Sum.over names dimensions, or the axes of the Join under it.
• The node type again says what happens. fan_in(Join) is one-to-one and Sum is many-to-one, and degree counts Sum and WindowSum as reductions. Separability reports a Sum over a join's axes as a grouping, and a bare Join as a coordinate read. The walk prints a Sum over a join's axes with the domain condition, and a bare Join at the leaves.
• PulledBackPredicate.direction is .columns. dimensions.pulled_back_dims is join_dims, one frame rule for an expression and a predicate. Partition.group is Partition.grouped, as in feat(language): a relation call is a join on the columns it names, and a sum through a relation is a sum over that join #605.
• feat(language): a relation call is a join on the columns it names, and a sum through a relation is a sum over that join #605's refusal wording, relations page and linear-maps page come along. The linear-maps page gains two sentences on the axis. contributing.md names the axes in its node table, and reading.md names JoinColumns on PulledBackPredicate.

Why

#638 makes dims_of, the walk and separability read program nodes, one node at a time. #605's Sum(Join(x), over=dropped_dims) names the join's dropped columns by their dimension, and two cases break:

1. A map into its own dimension. Golden representative sums through rep_of: { key: snapshot, values: { rep: snapshot } }. The join drops snapshot and adds it again, so the two columns share one name. The Sum over snapshot then takes away the dim the row keeps. With the axis, the join gives {snapshot, rep_of.snapshot} and the sum gives {snapshot}.
2. A sum over a lookup. sum(at(q, by=zone_of, over=zone, into=u), over=u) has feat(language): a relation call is a join on the columns it names, and a sum through a relation is a sum over that join #605's grouped-sum shape. Separability then reports "groups u into u" and drops the lookup's read of zone. Now it is a Sum over the dim u, and a grouped sum is a Sum over zone_of.u, so the names tell them apart.

With the join and the reduction separate again, another reduction can follow the same join. A Mean over a join's axes is one new node. It is not in this PR.

Guards, tests, gates, what was not done

Guards and the tests that fail without them

Each guard was deleted in turn, the suite run, and the tree restored.

guard	caught by
the axes in join_dims' frame	test_dimensions.py frame cases, and the golden and gallery pages
a lookup opens no axis	test_dim_inference[its-lookup-keeps-the-joined-key-too], [a-from-list-reads-two-value-columns-at-once]
the walk's grouped-sum dispatch	the gallery pages in test_docs.py, and the golden files
separability's grouping arm	test_a_grouping_that_sums_the_axis_away_couples_it
separability counting only an axis-free Join as a lookup	test_a_grouping_that_sums_the_axis_away_couples_it, after it gained the coupled and undecided asserts; before, this guard survived
the frame of a map into its own dimension	new test_a_join_opens_an_axis_for_the_column_it_drops_and_the_sum_over_it_closes_it

I checked two things against #605's code in a separate worktree:

• test_a_sum_over_a_lookup_is_a_sum_and_a_lookup_not_a_grouping fails there with 'groups u into u — …' != 'sums over u — …'.
• With feat(language): a relation call is a join on the columns it names, and a sum through a relation is a sum over that join #605's shape and a join that does not open axes, test_dim_inference[a-map-into-its-own-dimension-keeps-the-frame] is the one dims test that fails.

Degree has two new cases: two lookups multiplied are one term each, and a grouped sum is a reduction.

No guard refuses a join axis in a declaration's frame. Resolution always builds the Sum over a grouping Join, so a file cannot reach one.

Coverage that moved

• tests/test_lowering.py: the lowering cases, the zonal and history asserts, and the fan-in table expect Sum(Join(...), ('relation.column',)).
• tests/typesetting/test_golden.py: the operator census reads at off a Join that opens no axis. CARRIERS names JoinColumns.
• tests/test_validation.py: feat(language): a where may read a predicate through a relation with at() #634's two predicate at() refusals assert feat(language): a relation call is a join on the columns it names, and a sum through a relation is a sum over that join #605's wording ("joins on", "groups by").

Gates

Pixi is not reachable here (pixi.sh refused), so the gates ran from a uv venv on Python 3.12 with the pinned ruff and pyrefly.

gate	result
pytest -q -n auto	1570 passed
ruff check, ruff format --check	clean
pyrefly check	0 errors
tools.schema, tests.typesetting.golden	re-run, no diff: the schema and every golden output are #638's
generated pages (tests/test_docs.py)	current
typos, reuse lint	clean
prettier --check on the changed pages	clean
mkdocs build --strict	clean, with the docs.python.org inventory dropped for the run (the proxy refuses it)
compile-tex, zizmor	not run: tectonic and zizmor are not installed here

No check runs are on this PR. The workflows may not run on a base branch other than main.

History

• 1e4282d merges refactor(language): an expression resolves straight into the program's own nodes #638 into feat(language): a relation call is a join on the columns it names, and a sum through a relation is a sum over that join #605 with the one-node design. 0455e93 replaces that design with the join axes. The PR diff against refactor(language): an expression resolves straight into the program's own nodes #638 is the second design alone.
• refactor(language): an expression resolves straight into the program's own nodes #638 was rebased while this was in progress. The branch merges its current head 3270e48, not the earlier 540d412. The two trees differ only in CHANGELOG.md and the release manifest, and this merge takes refactor(language): an expression resolves straight into the program's own nodes #638's copies of those.

Not done

• Mean, or any reduction but Sum. The design admits it. No file writes one yet.
• feat(language): a relation call is a join on the columns it names, and a sum through a relation is a sum over that join #605 is untouched and still open. This PR carries all of its commits, so merging this one makes feat(language): a relation call is a join on the columns it names, and a sum through a relation is a sum over that join #605 redundant.
• fluxopt/lpspec is not updated. It reads GroupSum, Pullback and Direction today. With this shape it runs the join for a Join, and the terminal group-by for the Sum over its axes.
• feat(language): a piecewise curve applies to the members that have one, and ties as many expressions as the data says #569's piecewise walk still says consumed and produced. It is not touched here.

🤖 Generated with Claude Code

https://claude.ai/code/session_013M6RG2cnsrFCEbaUuU8KyJ

[read-the-docs-community]

Documentation build overview

📚 math-spec | 🛠️ Build #34716798 | 📁 Comparing 5f85607 against latest (fdb22f9)

  🔍 Preview build  

31 files changed · + 1 added · ± 30 modified

+ Added

• about/relations-as-linear-maps/index.html

± Modified

• contributing/index.html
• about/what-counts-as-language/index.html
• examples/operators/index.html
• howto/declare-a-column/index.html
• reference/notation/index.html
• reference/reading/index.html
• reference/language/errors/index.html
• reference/language/expressions/index.html
• reference/language/named/index.html
• reference/language/operators/index.html
• and 20 more...

[FBumann]

Prompt: Merge them and update the PR description. And close 641

Note

The following content was generated by AI.

Closed as superseded. #664 carries this PR's change, moved onto main, together with the relation[column] syntax. This PR's body keeps the design and the guard table, and #664 links to it. The branch is kept.

---

Generated by Claude Code