feat(engine): the relational lane builds every shape of relation the language admits by FBumann · Pull Request #1659 · fluxopt/specsolve

FBumann · 2026-09-16T06:10:04Z

Prompt: "Let's catch up and support everything from mathspecs relations feature. If the linopy lane is to hard, leave it stale. The relational lane is what we care about. Also take the time to potentially refactor the code about relations, as the concept has changed a bit"

Note

The following content was generated by AI.

The relational lane builds every relation shape the language admits: a key of several columns, several value columns, a bare relation, a partition by a conditioned map, a self-map, and two columns over one dimension. The eager lane keeps the single-valued map and refuses the rest, naming the lane that builds it.

Closes #1654, #1655, #1656 and #1653. Supersedes #1650, whose eager-lane half is carried here. #1657 (the self-map fix, #1652) landed on main meanwhile and is merged in: the general walk carries its relational half, and the eager lane keeps its operator_at fix.

What moved

The transport. A relation is attached as the table it declares, one column per column under its own name. sources.py checks every column against its dimension, one row per key tuple (every column, for a bare relation), and a null anywhere; attaching.py casts every column to its dimension's Enum. The archive stores the table as supplied, so supplied() no longer renames.
The compiler reads the plan's Walk. _walked puts consumed and joined columns under their dimensions and produced ones under a landing name until the consumed column is dropped, which is what a self-map needs. _remap_fragment joins on consumed, joined, and any produced dimension the operand already carries (the masked sum). _empty_groups and _pulled_back_presences carry the joined dimensions and several produced ones. partitioned takes the Walk and ranks inside (joined dims…, group columns…).
Partitions (reindex.py) key every side, edge and presence by the dimension and the partition's joined dimensions; a per-group offset or width is read under the group column. Predicates read a relation at every key dimension, a bare existence at every column, and a grouped position at the rest of the key.
lpspec/relations.py is deleted. Its accessors assumed the two-column map; the language's Walk carries the fact, and lanes.lowered refuses no relation any more.
The eager lane takes feat(data): a relation may be keyed by several columns, so a generator's zone can change by period #1650's multi-key support (one array per relation over its key's product, groupby on value and condition), reads a relation's key columns by role so a self-map's two columns stay apart, and refuses the other three shapes in linopy/loader.py with a LaneError naming the relational lane.
The parity tables (differential/pypsa/tables) are regenerated with parity.py against the pinned corpus: every rung still matches PyPSA, and the only change is each relation table's header, which tidy_sources no longer renames.
Docs: architecture.md (module rows, the GroupSum row, the relational-lane paragraph), reference/data.md (relation rows), about/linopy.md (a third wall between the lanes).

Coverage that moved

tests/test_relation_shapes.py (new): every shape on the relational lane, each optimum hand-derived and the written LP file re-solved to it; the masked sum; a bare relation holding a pair twice. The self-map walks live in main's test_self_map.py, which is differential; this module keeps the self-map partition.
tests/test_conditioned_relations.py (new): feat(data): a relation may be keyed by several columns, so a generator's zone can change by period #1650's differential cases for the multi-key map, the eager lane's three refusals, and the door checks.
test_label_coords.py: the two refusal cases became "passes check" cases. test_sum_by_relations.test_a_hand_built_walk_onto_two_columns_is_refused is deleted, the shape now building in the new module.
Door messages reworded for the general shape, regexes moved with them (test_self_map.py's included): maps 1 key(s) more than once: generator='g1', has value(s) in 'generator' that are not 'generator' labels, null in 'bus', is a relation with a column over 'g'.

Mutation table

Taken with python -m tools.mutate on 419870d, before the merge; the tree came back clean. The two-columns-over-one-dimension guard was retired by the merge, the eager lane building the shape since #1657.

mutation	result
the eager lane refuses a bare relation (`loader.py:40-41`)	caught
the eager lane refuses a key determining several columns (`loader.py:42-48`)	caught
the eager lane refuses a partition by a conditioned map (`loader.py:53-60`)	caught
a relation holding a row twice is refused (`sources.py:341-352`)	caught

Verified

ruff check · ruff format --check · pyrefly check · pytest -q -n 4 on the merged head 4d133ea: the suite reaches the same 10 failures as main (3e825f0) in this environment, all of them a solver that is not installed (gurobi, xpress). differential/pypsa/parity.py against math-spec at v0.0.0-alpha.89: every rung matches. Run in a uv venv with the linopy extra rather than pixi (no pixi here).

Not run: docs-build and test-floors, no pixi environment here.

What I did not do

The eager lane is left at the single-valued map, as asked. The three other shapes are refused there, not built; their tests use the LP file as the second opinion.
No new example or gallery page. A page for a line-ends table is a docs: PR of its own.
Two lanes still each answer "which dimensions do a node's walks join on" (compiler.joined_dims, builder._joined_dims). feat(program): a grouped sum and a pullback say which dimensions their walks join on energy-models/mathspec#488 puts joined on GroupSum and At; the two copies go with the pin bump once it releases.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DT1r52zPp9BdnrzFdp1xoY

…language admits A relation is attached as the table it declares, one column per column under its own name, and every walk reads the plan's Walk: a key of several columns joins on the rest of the key, several value columns land on a product of dimensions, a bare relation fans a member out to every target, a partition groups by the value at the rest of its key, and a self-map or two columns over one dimension land under a name of their own until the consumed column is dropped. The eager lane keeps the single-valued map, now keyed by any number of columns, and refuses the other shapes at its door with the relational lane named. lpspec.relations is retired: the language's Walk carries the fact it accessed, and nothing refuses a relation at lowering any more. Closes #1652, #1654, #1655, #1656 and their parent #1653; supersedes #1650. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DT1r52zPp9BdnrzFdp1xoY

read-the-docs-community · 2026-09-16T06:12:14Z

Documentation build overview

📚 lpspec | 🛠️ Build #34584098 | 📁 Comparing 8c64e8c against latest (3e825f0)

🔍 Preview build

11 files changed · ± 11 modified

± Modified

… walk The relational compiler keeps the branch's Walk-based join, which lands a self-map's produced column under a name of its own. The eager lane keeps main's fix to operator_at, reads a relation's key columns by role so a self-map's two columns stay apart, and no longer refuses two columns over one dimension. test_self_map.py stays as the differential oracle for the shape; the relational-only copies of its cases are dropped. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DT1r52zPp9BdnrzFdp1xoY

codspeed · 2026-09-16T06:19:10Z

Merging this PR will not alter performance

✅ 24 untouched benchmarks
⏩ 58 skipped benchmarks¹

_{Comparing claude/mathspec-relations-issues-tsf1fa (8c64e8c) with main (2f96905)²}

58 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
No successful run was found on main (3e825f0) during the generation of this report, so 2f96905 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

…er their own names Regenerated with differential/pypsa/parity.py against math-spec at the pinned tag; every rung still matches PyPSA, and the only change is the header of each relation table, which tidy_sources no longer renames. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DT1r52zPp9BdnrzFdp1xoY

Written by tools.ladder; the 38 changed lines are the relation table headers the pages embed byte for byte. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DT1r52zPp9BdnrzFdp1xoY

…join and the grouping a walk names (#1660) > **Prompt:** "Please review everything about relations in lpspec. Do a stacked pr refactoring the code to make it more readable and have better seams and types" — and, on the first cut: "I fell like this lpspec pr adds a lot of new names which are things we already have conceptually", which took `Ends` and `keyed` back out; "Axis is a separability concept I think", which took the word out; and "Do Order", which named the type by the language's word. > [!NOTE] > The following content was generated by AI. Stacked on #1659. No consumer can tell the difference: the same queries, read through one module and two types instead of a dozen loose functions, in the language's own words — a walk's `consumed`, `joined` and `produced` dimensions, and the order a shift counts along. **What the review found.** After #1659, the relational lane spelled the same three facts several times over: - **a `Walk`'s roles became columns in four places** — `compiler._walked`, `compiler.partitioned`, `predicates.join_relation` and `reindex._group_columns`; - **the walk join was written twice** — `_remap_fragment` and `_pulled_back_presences.pulled` both keyed on consumed, joined and carried-produced dims and landed the rest under their names; - **a partition's group key was recomputed in three places**, and `reindex._edge` rebuilt the ranked table a second time per shift; `reindex._Walk` had come to hold a `program.Walk` as a field, under the same name. <details><summary>What moved</summary> - **`relational/engines/polars/relations.py`** (new) holds the role-to-column translation once: - `mapping` and `walk_join` — the table a group or a pullback joins against, and the one join that trades the consumed dims for the produced ones (keyed on consumed, joined, and a produced dim the operand already carries) in a single select, which is what a self-map needs. `_remap_fragment` is two lines and the pullback's presence step one. - `Grouping` — the walked dimension ranked inside its groups, with `key`, `keys`, `placed()` and `column_of()`; replaces `compiler.partitioned`, `reindex._grouped` and `reindex._group_columns`. - `joined_dims` — the one property a node lacks until energy-models/mathspec#488 releases; the pin bump deletes it. `consumed_dims`, `produced_dims` and `landing` are gone from the compiler. - `GROUP_RANK`, `GROUP_SIZE`, `group_column` and `landing` live here, out of `fragments.py` and `compiler.py`. - **`reindex.py`**: `_Walk` is `_Order` — one dimension's own order, or each group's own, as an operator counts along it: the positions and the two keyed sides of the remap — built once per operator and holding its `Grouping`; the acyclic edge is `_Edge`, one type with `keys`, `coordinates()`, `filled()` and `vacated_of()`, in place of `_edge`, `_filled_edge`, `_vacated`, `edge_keys` and `offset_dims`, and the named offset's table is read once. The prose says dimension where it said axis. - **`predicates.py`** reads a grouped position through `Grouping`; the relation read keeps its body. - **`compiler.py`** keeps the expression walk only. </details> <details><summary>Verified</summary> `ruff check` · `ruff format --check` · `pyrefly check` · `pytest -q -n 4` on the head: 3772 passed, the same 10 failures as the base (`gurobi`, `xpress` not installed here). No new guard, so no mutation table. **Departed from one default:** the pass is not fewer lines. Six files, +417 −332 against the base, and the surplus is the docstrings on the two types; what it ends with is fewer concepts and one home per fact. </details> 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01DT1r52zPp9BdnrzFdp1xoY --------- Co-authored-by: Claude <noreply@anthropic.com>

🤖 I have created a release *beep* *boop* --- ## [0.0.1-alpha.336](v0.0.1-alpha.335...v0.0.1-alpha.336) (2026-09-16) ### Features * **engine:** the relational lane builds every shape of relation the language admits ([#1659](#1659)) ([88914c4](88914c4)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: fluxopt-release-bot[bot] <307443024+fluxopt-release-bot[bot]@users.noreply.github.com>

…lk joins on (#1663) > **Prompt:** "488 merged and released. Bump mathspec version!" > [!NOTE] > The following content was generated by AI. The pin follows math-spec 0.0.0-alpha.92, the release carrying energy-models/mathspec#488: `GroupSum.joined` and `At.joined`. The two copies of that answer here — `relations.joined_dims` on the relational lane and `builder._joined_dims` on the eager one — are deleted, and both lanes read it off the node. The follow-up #1659 and #1660 named. <details><summary>What moved</summary> - `pyproject.toml` and `uv.lock`: the pin, `v0.0.0-alpha.89` to `v0.0.0-alpha.92`. - `relations.py`: `landed` and `walk_join` take the node rather than its walks, and read `node.joined`; `joined_dims` is gone. - `compiler.py`: `_remap_fragment` takes the node; `_empty_groups` and `_pulled_back_presences` read `g.joined` and `a.joined`. - `linopy/builder.py`: the grouped sum passes `node.joined`; `_joined_dims` is gone. Five files, +26 −40. </details> <details><summary>Verified</summary> `ruff check` · `ruff format --check` · `pyrefly check` · `pytest -q -n 4` on 309422e, with alpha.92 installed: 3772 passed, the same 10 failures as `main` (`gurobi`, `xpress` not installed here). Run in a `uv` venv with the `linopy` extra rather than pixi. **Not run:** `docs-build` and `test-floors`, no pixi environment here. The parity job checks out the corpus at the pinned tag, so it runs against alpha.92 in CI. </details> 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01DT1r52zPp9BdnrzFdp1xoY --- _Generated by [Claude Code](https://claude.ai/code/session_01DT1r52zPp9BdnrzFdp1xoY)_ Co-authored-by: Claude <noreply@anthropic.com>

FBumann mentioned this pull request Sep 16, 2026

feat(program): a grouped sum and a pullback say which dimensions their walks join on energy-models/mathspec#488

Merged

claude added 2 commits September 16, 2026 06:25

FBumann mentioned this pull request Sep 16, 2026

chore(engine): a relation's table is read through one module, as the join and the grouping a walk names #1660

Merged

FBumann added this pull request to stack #1661 September 16, 2026 07:13

FBumann merged commit 88914c4 into main Sep 16, 2026
12 checks passed

FBumann deleted the claude/mathspec-relations-issues-tsf1fa branch September 16, 2026 07:13

fluxopt-release-bot Bot mentioned this pull request Sep 16, 2026

chore(main): release 0.0.1-alpha.336 #1662

Merged

FBumann mentioned this pull request Sep 16, 2026

build(deps): math-spec 0.0.0-alpha.92, which says the dimensions a walk joins on #1663

Merged

Navigation Menu

feat(engine): the relational lane builds every shape of relation the language admits - #1659

feat(engine): the relational lane builds every shape of relation the language admits#1659
FBumann merged 4 commits into
mainfrom
claude/mathspec-relations-issues-tsf1fa

FBumann commented Sep 16, 2026 •

edited

Loading

Uh oh!

read-the-docs-community Bot commented Sep 16, 2026 •

edited

Loading

Uh oh!

codspeed Bot commented Sep 16, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

FBumann commented Sep 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What I did not do

Uh oh!

read-the-docs-community Bot commented Sep 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Documentation build overview

Uh oh!

codspeed Bot commented Sep 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Footnotes

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

FBumann commented Sep 16, 2026 •

edited

Loading

read-the-docs-community Bot commented Sep 16, 2026 •

edited

Loading

codspeed Bot commented Sep 16, 2026 •

edited

Loading