Skip to content

feat(program): a grouped sum and a pullback say which dimensions their walks join on - #488

Merged
FBumann merged 1 commit into
mainfrom
claude/mathspec-relations-issues-tsf1fa
Sep 16, 2026
Merged

feat(program): a grouped sum and a pullback say which dimensions their walks join on#488
FBumann merged 1 commit into
mainfrom
claude/mathspec-relations-issues-tsf1fa

Conversation

@FBumann

@FBumann FBumann commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Prompt: "If needed, we can improve mathspec too" — asked while lpspec caught up with the relations feature (fluxopt/specsolve#1659).

Note

The following content was generated by AI.

What this changes

GroupSum.joined and At.joined, beside over and into: the dimensions of the key columns the walks neither consume nor produce, each once, in walk order. The rule is one private helper the two properties share. test_lowering.py pins it on the conditioned map (zone_of keyed by [generator, snapshot], walked from either key column and read back through at).

Why

A consumer joining a relation's table keys the join on the consumed columns and every joined column (the GroupSum docstring already says so). over and into are read off the node; the third set was not, so both of lpspec's lanes derived it from the walks with a copy of their own (compiler.joined_dims, builder._joined_dims), and fluxopt/specsolve#1650 carried the same two copies. Two consumers answering that separately is the bug what counts as language names, so the answer moves here, once. The copies downstream go with the pin bump after this releases.

Verified

ruff check · ruff format --check · pyrefly check (0 errors) · pytest -q -n 4: 1255 passed, 5 failed, all five test_docs.py cases needing mkdocs, which this uv venv lacks — they fail identically on the unchanged tree. No pixi here, so pixi run ci was not run.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DT1r52zPp9BdnrzFdp1xoY

Generated by Claude Code

@claude
feat(program): a grouped sum and a pullback say which dimensions thei…
07ad518 
…r walks join on

`GroupSum.joined` and `At.joined` sit beside `over` and `into`: the
dimensions of the key columns the walks neither consume nor produce, each
once, which the operand carries and the result keeps. A consumer joining a
relation's table reads all three off the node rather than deriving the
third from the walks itself.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DT1r52zPp9BdnrzFdp1xoY
@read-the-docs-community

read-the-docs-community Bot commented Sep 16, 2026

Copy link
Copy Markdown

Documentation build overview

📚 math-spec | 🛠️ Build #34583919 | 📁 Comparing 07ad518 against latest (c8aea0e)

  🔍 Preview build  

3 files changed
± CHANGELOG/index.html
± contributing/index.html
± reference/math_spec/program/index.html

@FBumann
FBumann merged commit dd05088 into main Sep 16, 2026
5 checks passed
@energy-models-release-please energy-models-release-please Bot mentioned this pull request Sep 16, 2026
Merged
This was referenced Sep 16, 2026
Merged
Merged
FBumann added a commit to fluxopt/specsolve that referenced this pull request Sep 16, 2026
@FBumann @claude
feat(engine): the relational lane builds every shape of relation the …
88914c4 
…language admits (#1659)

> **Prompt:** "Let's catch up and support everything from mathspecs
relations feature. If the linopy lane is to hard, leave it stale. The
relational lane is what we care about. Also take the time to potentially
refactor the code about relations, as the concept has changed a bit"

> [!NOTE]
> The following content was generated by AI.

The relational lane builds every relation shape the language admits: a
key of several columns, several value columns, a bare relation, a
partition by a conditioned map, a self-map, and two columns over one
dimension. The eager lane keeps the single-valued map and refuses the
rest, naming the lane that builds it.

Closes #1654, #1655, #1656 and #1653. Supersedes #1650, whose eager-lane
half is carried here. #1657 (the self-map fix, #1652) landed on `main`
meanwhile and is merged in: the general walk carries its relational
half, and the eager lane keeps its `operator_at` fix.

<details><summary>What moved</summary>

- **The transport.** A relation is attached as the table it declares,
one column per column under its own name. `sources.py` checks every
column against its dimension, one row per key tuple (every column, for a
bare relation), and a null anywhere; `attaching.py` casts every column
to its dimension's `Enum`. The archive stores the table as supplied, so
`supplied()` no longer renames.
- **The compiler** reads the plan's `Walk`. `_walked` puts consumed and
joined columns under their dimensions and produced ones under a landing
name until the consumed column is dropped, which is what a self-map
needs. `_remap_fragment` joins on consumed, joined, and any produced
dimension the operand already carries (the masked sum). `_empty_groups`
and `_pulled_back_presences` carry the joined dimensions and several
produced ones. `partitioned` takes the `Walk` and ranks inside `(joined
dims…, group columns…)`.
- **Partitions** (`reindex.py`) key every side, edge and presence by the
dimension and the partition's joined dimensions; a per-group offset or
width is read under the group column. **Predicates** read a relation at
every key dimension, a bare existence at every column, and a grouped
position at the rest of the key.
- **`lpspec/relations.py` is deleted.** Its accessors assumed the
two-column map; the language's `Walk` carries the fact, and
`lanes.lowered` refuses no relation any more.
- **The eager lane** takes #1650's multi-key support (one array per
relation over its key's product, `groupby` on value and condition),
reads a relation's key columns by role so a self-map's two columns stay
apart, and refuses the other three shapes in `linopy/loader.py` with a
`LaneError` naming the relational lane.
- **The parity tables** (`differential/pypsa/tables`) are regenerated
with `parity.py` against the pinned corpus: every rung still matches
PyPSA, and the only change is each relation table's header, which
`tidy_sources` no longer renames.
- **Docs:** `architecture.md` (module rows, the `GroupSum` row, the
relational-lane paragraph), `reference/data.md` (relation rows),
`about/linopy.md` (a third wall between the lanes).

</details>

<details><summary>Coverage that moved</summary>

- `tests/test_relation_shapes.py` (new): every shape on the relational
lane, each optimum hand-derived and the written LP file re-solved to it;
the masked sum; a bare relation holding a pair twice. The self-map walks
live in `main`'s `test_self_map.py`, which is differential; this module
keeps the self-map partition.
- `tests/test_conditioned_relations.py` (new): #1650's differential
cases for the multi-key map, the eager lane's three refusals, and the
door checks.
- `test_label_coords.py`: the two refusal cases became "passes check"
cases.
`test_sum_by_relations.test_a_hand_built_walk_onto_two_columns_is_refused`
is deleted, the shape now building in the new module.
- Door messages reworded for the general shape, regexes moved with them
(`test_self_map.py`'s included): `maps 1 key(s) more than once:
generator='g1'`, `has value(s) in 'generator' that are not 'generator'
labels`, `null in 'bus'`, `is a relation with a column over 'g'`.

</details>

<details><summary>Mutation table</summary>

Taken with `python -m tools.mutate` on 419870d, before the merge; the
tree came back clean. The two-columns-over-one-dimension guard was
retired by the merge, the eager lane building the shape since #1657.

| mutation | result |
|---|---|
| the eager lane refuses a bare relation (`loader.py:40-41`) |
**caught** |
| the eager lane refuses a key determining several columns
(`loader.py:42-48`) | **caught** |
| the eager lane refuses a partition by a conditioned map
(`loader.py:53-60`) | **caught** |
| a relation holding a row twice is refused (`sources.py:341-352`) |
**caught** |

</details>

<details><summary>Verified</summary>

`ruff check` · `ruff format --check` · `pyrefly check` · `pytest -q -n
4` on the merged head 4d133ea: the suite reaches the same 10 failures
as `main` (3e825f0) in this environment, all of them a solver that is
not installed (`gurobi`, `xpress`). `differential/pypsa/parity.py`
against math-spec at v0.0.0-alpha.89: every rung matches. Run in a `uv`
venv with the `linopy` extra rather than pixi (no pixi here).

**Not run:** `docs-build` and `test-floors`, no pixi environment here.

</details>

## What I did not do

- **The eager lane is left at the single-valued map**, as asked. The
three other shapes are refused there, not built; their tests use the LP
file as the second opinion.
- **No new example or gallery page.** A page for a line-ends table is a
`docs:` PR of its own.
- **Two lanes still each answer "which dimensions do a node's walks join
on"** (`compiler.joined_dims`, `builder._joined_dims`).
energy-models/mathspec#488 puts `joined` on `GroupSum` and `At`; the
two copies go with the pin bump once it releases.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01DT1r52zPp9BdnrzFdp1xoY

---------

Co-authored-by: Claude <noreply@anthropic.com>
FBumann added a commit to fluxopt/specsolve that referenced this pull request Sep 16, 2026
@FBumann @claude
chore(engine): a relation's table is read through one module, as the …
ef08322 
…join and the grouping a walk names (#1660)

> **Prompt:** "Please review everything about relations in lpspec. Do a
stacked pr refactoring the code to make it more readable and have better
seams and types" — and, on the first cut: "I fell like this lpspec pr
adds a lot of new names which are things we already have conceptually",
which took `Ends` and `keyed` back out; "Axis is a separability concept
I think", which took the word out; and "Do Order", which named the type
by the language's word.

> [!NOTE]
> The following content was generated by AI.

Stacked on #1659. No consumer can tell the difference: the same queries,
read through one module and two types instead of a dozen loose
functions, in the language's own words — a walk's `consumed`, `joined`
and `produced` dimensions, and the order a shift counts along.

**What the review found.** After #1659, the relational lane spelled the
same three facts several times over:

- **a `Walk`'s roles became columns in four places** —
`compiler._walked`, `compiler.partitioned`, `predicates.join_relation`
and `reindex._group_columns`;
- **the walk join was written twice** — `_remap_fragment` and
`_pulled_back_presences.pulled` both keyed on consumed, joined and
carried-produced dims and landed the rest under their names;
- **a partition's group key was recomputed in three places**, and
`reindex._edge` rebuilt the ranked table a second time per shift;
`reindex._Walk` had come to hold a `program.Walk` as a field, under the
same name.

<details><summary>What moved</summary>

- **`relational/engines/polars/relations.py`** (new) holds the
role-to-column translation once:
- `mapping` and `walk_join` — the table a group or a pullback joins
against, and the one join that trades the consumed dims for the produced
ones (keyed on consumed, joined, and a produced dim the operand already
carries) in a single select, which is what a self-map needs.
`_remap_fragment` is two lines and the pullback's presence step one.
- `Grouping` — the walked dimension ranked inside its groups, with
`key`, `keys`, `placed()` and `column_of()`; replaces
`compiler.partitioned`, `reindex._grouped` and `reindex._group_columns`.
- `joined_dims` — the one property a node lacks until
energy-models/mathspec#488 releases; the pin bump deletes it.
`consumed_dims`, `produced_dims` and `landing` are gone from the
compiler.
- `GROUP_RANK`, `GROUP_SIZE`, `group_column` and `landing` live here,
out of `fragments.py` and `compiler.py`.
- **`reindex.py`**: `_Walk` is `_Order` — one dimension's own order, or
each group's own, as an operator counts along it: the positions and the
two keyed sides of the remap — built once per operator and holding its
`Grouping`; the acyclic edge is `_Edge`, one type with `keys`,
`coordinates()`, `filled()` and `vacated_of()`, in place of `_edge`,
`_filled_edge`, `_vacated`, `edge_keys` and `offset_dims`, and the named
offset's table is read once. The prose says dimension where it said
axis.
- **`predicates.py`** reads a grouped position through `Grouping`; the
relation read keeps its body.
- **`compiler.py`** keeps the expression walk only.

</details>

<details><summary>Verified</summary>

`ruff check` · `ruff format --check` · `pyrefly check` · `pytest -q -n
4` on the head: 3772 passed, the same 10 failures as the base (`gurobi`,
`xpress` not installed here). No new guard, so no mutation table.

**Departed from one default:** the pass is not fewer lines. Six files,
+417 −332 against the base, and the surplus is the docstrings on the two
types; what it ends with is fewer concepts and one home per fact.

</details>

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01DT1r52zPp9BdnrzFdp1xoY

---------

Co-authored-by: Claude <noreply@anthropic.com>
@FBumann FBumann mentioned this pull request Sep 16, 2026
Merged
FBumann added a commit to fluxopt/specsolve that referenced this pull request Sep 16, 2026
@FBumann @claude
build(deps): math-spec 0.0.0-alpha.92, which says the dimensions a wa…
afe924e 
…lk joins on (#1663)

> **Prompt:** "488 merged and released. Bump mathspec version!"

> [!NOTE]
> The following content was generated by AI.

The pin follows math-spec 0.0.0-alpha.92, the release carrying
energy-models/mathspec#488: `GroupSum.joined` and `At.joined`. The two
copies of that answer here — `relations.joined_dims` on the relational
lane and `builder._joined_dims` on the eager one — are deleted, and both
lanes read it off the node. The follow-up #1659 and #1660 named.

<details><summary>What moved</summary>

- `pyproject.toml` and `uv.lock`: the pin, `v0.0.0-alpha.89` to
`v0.0.0-alpha.92`.
- `relations.py`: `landed` and `walk_join` take the node rather than its
walks, and read `node.joined`; `joined_dims` is gone.
- `compiler.py`: `_remap_fragment` takes the node; `_empty_groups` and
`_pulled_back_presences` read `g.joined` and `a.joined`.
- `linopy/builder.py`: the grouped sum passes `node.joined`;
`_joined_dims` is gone.

Five files, +26 −40.

</details>

<details><summary>Verified</summary>

`ruff check` · `ruff format --check` · `pyrefly check` · `pytest -q -n
4` on 309422e, with alpha.92 installed: 3772 passed, the same 10
failures as `main` (`gurobi`, `xpress` not installed here). Run in a
`uv` venv with the `linopy` extra rather than pixi.

**Not run:** `docs-build` and `test-floors`, no pixi environment here.
The parity job checks out the corpus at the pinned tag, so it runs
against alpha.92 in CI.

</details>

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01DT1r52zPp9BdnrzFdp1xoY

---
_Generated by [Claude
Code](https://claude.ai/code/session_01DT1r52zPp9BdnrzFdp1xoY)_

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@brynpickering brynpickering Awaiting requested review from brynpickering brynpickering is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@FBumann @claude