Skip to content

fix(data): a relation whose two columns are over one dimension builds - #1657

Merged
FBumann merged 2 commits into
mainfrom
fix/self-map-walk-column
Sep 16, 2026
Merged

FBumann merged 2 commits into
mainfrom
fix/self-map-walk-column

Conversation

@FBumann

@FBumann FBumann commented Sep 16, 2026

Copy link
Copy Markdown
Collaborator

Prompt: Then let's fix the bug first

Note

The following content was generated by AI.

Closes #1652. A self-map — rep_of: {columns: {snapshot: snapshot, rep: snapshot}, key: snapshot}, the language's own example of the form — passed check and then raised: polars' DuplicateError relationally, linopy's coordinate mismatch eagerly, and on sum(by=) the two lanes disagreed, one raising where the other built.

The walk named its value column after the dimension it lands on, which for a self-map is the name the key column already holds. The mapping table names its columns by side now — __walk key__, __walk value {relation}__ — and the join names its sides (left_on / right_on), so the landing is aliased to its dimension at the end rather than colliding at the start. On the eager lane the pullback puts the fine labels back as the coordinate: a row lands at the snapshot that read a value, not at the one it read.

What the probe says, before and after
lps.check relational eager
at(price, by=rep_of, over=rep) before passes DuplicateError ValueError: Coordinate mismatch
sum(p, by=rep_of, over=snapshot, into=rep) before passes DuplicateError builds, unchecked
both, after passes builds builds, and the two agree
Mutation table

Baseline for this environment: 10 failures, all gurobi/xpress not installed.

mutation result
the one-value-column-per-relation guard (compiler.py:703-706) caught
the walk selects its landing by the dimension's own name (compiler.py:845) caught — 218 failed
the eager pullback keeps the labels it read (operators.py:120) caught — 11 failed, 1 beyond the baseline
the join renames the mapping onto the fragment's dims instead of naming both sides (compiler.py:846) not caught — recorded rather than hidden: it is an equivalent implementation, since what the bug needed was the landing to keep a name of its own, which the row above is the guard for

The first through python -m tools.mutate, which reported the tree clean afterwards; the rest by hand (changed lines, which the tool does not express), each with the same three precautions.

Verified

ruff check · ruff format --check · pyrefly check · pytest -q -n 4 — the suite reaches the same 10 failures as main in this environment, all a solver that is not installed. Run in a uv venv rather than pixi (no pixi here).

Not run: pixi run test-floors and test-bench — no pixi environment here.

The bug was reproduced first: tests/test_self_map.py landed as two strict xfails naming #1652 (commit 0324845), which XPASS on the fix and lose their markers in the commit that makes them pass. What was wrong is in their docstrings. The where: case and the duplicate-key refusal pass beside them, which is what said the data path was never the problem.

Coverage that moved: test_sum_by_relations.test_a_hand_built_walk_onto_two_columns_is_refused met a zip crash inside the mapping builder; the guard it probes is now a named assertion that fires before any relation is read, and the test matches that.

What I did not do

🤖 Generated with Claude Code

https://claude.ai/code/session_014pdMgEVkGBfh1f2AyiCSKA


Generated by Claude Code

Two columns over one dimension: the pullback and the group are strict
xfails naming #1652, the where and the door check pass beside them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014pdMgEVkGBfh1f2AyiCSKA
A self-map — `rep_of: {columns: {snapshot: snapshot, rep: snapshot}, key:
snapshot}` — passed check and then raised: the walk named its value column
after the dimension it lands on, which is the name the key already holds.
The mapping table names its columns by side now, and the eager pullback puts
the fine labels back as the coordinate rather than the ones it read.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014pdMgEVkGBfh1f2AyiCSKA
@codspeed

codspeed Bot commented Sep 16, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 24 untouched benchmarks
⏩ 58 skipped benchmarks1


Comparing fix/self-map-walk-column (ff191bf) with main (1fd8926)2

Open in CodSpeed

Footnotes

  1. 58 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

  2. No successful run was found on main (28cceaf) during the generation of this report, so 1fd8926 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

@FBumann
FBumann merged commit 2f96905 into main Sep 16, 2026
12 checks passed
@FBumann
FBumann deleted the fix/self-map-walk-column branch September 16, 2026 05:59
fluxopt-release-bot Bot added a commit that referenced this pull request Sep 16, 2026
🤖 I have created a release *beep* *boop*
---


##
[0.0.1-alpha.335](v0.0.1-alpha.334...v0.0.1-alpha.335)
(2026-09-16)


### Bug Fixes

* **data:** a relation whose two columns are over one dimension builds
([#1657](#1657))
([2f96905](2f96905))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: fluxopt-release-bot[bot] <307443024+fluxopt-release-bot[bot]@users.noreply.github.com>
FBumann pushed a commit that referenced this pull request Sep 16, 2026
… walk

The relational compiler keeps the branch's Walk-based join, which lands a
self-map's produced column under a name of its own. The eager lane keeps
main's fix to operator_at, reads a relation's key columns by role so a
self-map's two columns stay apart, and no longer refuses two columns over
one dimension. test_self_map.py stays as the differential oracle for the
shape; the relational-only copies of its cases are dropped.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DT1r52zPp9BdnrzFdp1xoY
FBumann added a commit that referenced this pull request Sep 16, 2026
…language admits (#1659)

> **Prompt:** "Let's catch up and support everything from mathspecs
relations feature. If the linopy lane is to hard, leave it stale. The
relational lane is what we care about. Also take the time to potentially
refactor the code about relations, as the concept has changed a bit"

> [!NOTE]
> The following content was generated by AI.

The relational lane builds every relation shape the language admits: a
key of several columns, several value columns, a bare relation, a
partition by a conditioned map, a self-map, and two columns over one
dimension. The eager lane keeps the single-valued map and refuses the
rest, naming the lane that builds it.

Closes #1654, #1655, #1656 and #1653. Supersedes #1650, whose eager-lane
half is carried here. #1657 (the self-map fix, #1652) landed on `main`
meanwhile and is merged in: the general walk carries its relational
half, and the eager lane keeps its `operator_at` fix.

<details><summary>What moved</summary>

- **The transport.** A relation is attached as the table it declares,
one column per column under its own name. `sources.py` checks every
column against its dimension, one row per key tuple (every column, for a
bare relation), and a null anywhere; `attaching.py` casts every column
to its dimension's `Enum`. The archive stores the table as supplied, so
`supplied()` no longer renames.
- **The compiler** reads the plan's `Walk`. `_walked` puts consumed and
joined columns under their dimensions and produced ones under a landing
name until the consumed column is dropped, which is what a self-map
needs. `_remap_fragment` joins on consumed, joined, and any produced
dimension the operand already carries (the masked sum). `_empty_groups`
and `_pulled_back_presences` carry the joined dimensions and several
produced ones. `partitioned` takes the `Walk` and ranks inside `(joined
dims…, group columns…)`.
- **Partitions** (`reindex.py`) key every side, edge and presence by the
dimension and the partition's joined dimensions; a per-group offset or
width is read under the group column. **Predicates** read a relation at
every key dimension, a bare existence at every column, and a grouped
position at the rest of the key.
- **`lpspec/relations.py` is deleted.** Its accessors assumed the
two-column map; the language's `Walk` carries the fact, and
`lanes.lowered` refuses no relation any more.
- **The eager lane** takes #1650's multi-key support (one array per
relation over its key's product, `groupby` on value and condition),
reads a relation's key columns by role so a self-map's two columns stay
apart, and refuses the other three shapes in `linopy/loader.py` with a
`LaneError` naming the relational lane.
- **The parity tables** (`differential/pypsa/tables`) are regenerated
with `parity.py` against the pinned corpus: every rung still matches
PyPSA, and the only change is each relation table's header, which
`tidy_sources` no longer renames.
- **Docs:** `architecture.md` (module rows, the `GroupSum` row, the
relational-lane paragraph), `reference/data.md` (relation rows),
`about/linopy.md` (a third wall between the lanes).

</details>

<details><summary>Coverage that moved</summary>

- `tests/test_relation_shapes.py` (new): every shape on the relational
lane, each optimum hand-derived and the written LP file re-solved to it;
the masked sum; a bare relation holding a pair twice. The self-map walks
live in `main`'s `test_self_map.py`, which is differential; this module
keeps the self-map partition.
- `tests/test_conditioned_relations.py` (new): #1650's differential
cases for the multi-key map, the eager lane's three refusals, and the
door checks.
- `test_label_coords.py`: the two refusal cases became "passes check"
cases.
`test_sum_by_relations.test_a_hand_built_walk_onto_two_columns_is_refused`
is deleted, the shape now building in the new module.
- Door messages reworded for the general shape, regexes moved with them
(`test_self_map.py`'s included): `maps 1 key(s) more than once:
generator='g1'`, `has value(s) in 'generator' that are not 'generator'
labels`, `null in 'bus'`, `is a relation with a column over 'g'`.

</details>

<details><summary>Mutation table</summary>

Taken with `python -m tools.mutate` on 419870d, before the merge; the
tree came back clean. The two-columns-over-one-dimension guard was
retired by the merge, the eager lane building the shape since #1657.

| mutation | result |
|---|---|
| the eager lane refuses a bare relation (`loader.py:40-41`) |
**caught** |
| the eager lane refuses a key determining several columns
(`loader.py:42-48`) | **caught** |
| the eager lane refuses a partition by a conditioned map
(`loader.py:53-60`) | **caught** |
| a relation holding a row twice is refused (`sources.py:341-352`) |
**caught** |

</details>

<details><summary>Verified</summary>

`ruff check` · `ruff format --check` · `pyrefly check` · `pytest -q -n
4` on the merged head 4d133ea: the suite reaches the same 10 failures
as `main` (3e825f0) in this environment, all of them a solver that is
not installed (`gurobi`, `xpress`). `differential/pypsa/parity.py`
against math-spec at v0.0.0-alpha.89: every rung matches. Run in a `uv`
venv with the `linopy` extra rather than pixi (no pixi here).

**Not run:** `docs-build` and `test-floors`, no pixi environment here.

</details>

## What I did not do

- **The eager lane is left at the single-valued map**, as asked. The
three other shapes are refused there, not built; their tests use the LP
file as the second opinion.
- **No new example or gallery page.** A page for a line-ends table is a
`docs:` PR of its own.
- **Two lanes still each answer "which dimensions do a node's walks join
on"** (`compiler.joined_dims`, `builder._joined_dims`).
energy-models/mathspec#488 puts `joined` on `GroupSum` and `At`; the
two copies go with the pin bump once it releases.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01DT1r52zPp9BdnrzFdp1xoY

---------

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

a self-map passes check and then crashes in the build with polars' own error

2 participants