Skip to content

refac!: the math is one file per feature, and each feature adds its own term to the ledger - #342

Closed
FBumann wants to merge 5 commits into
chore/specsolve-mathspec-0.2from
claude/fluxopt-mathspec-732-yesoe9
Closed

FBumann wants to merge 5 commits into
chore/specsolve-mathspec-0.2from
claude/fluxopt-mathspec-732-yesoe9

Conversation

@FBumann

@FBumann FBumann commented Sep 26, 2026 •

Copy link
Copy Markdown
Collaborator

Prompt: "I want to explore how much easier and better the yaml stuff will become as we use the new mathspec feature in fluxopt. Use mathspec 742 in a new branch from the tip of the large stack in fluxopt" — then "I want this as two stacked PRs"

(Intent: to be written by the author.)

Note

The following content was generated by AI.

Description

program.yaml becomes ten fragments under math/program/, one per feature, composed by mathspec's merge. effects.yaml declares the ledger's two halves as empty sums (energy-models/mathspec#742), and each feature adds its own term. The composed math is the same.

File Declares Adds to the ledger
effects.yaml effect_total, the limits, the objective, and effect_temporal/effect_lump as sums with no body —
flows.yaml rate, the carrier balance, flow hours flow_hour_costs → temporal
status.yaml on/off, durations, carry-over status_costs → temporal
sizing.yaml size, size_built sizing_costs → lump
investment.yaml build timing investment_costs → lump
storage.yaml level, recurrence, capacity storage_costs → lump
converters.yaml, piecewise.yaml, ramps.yaml, status_sizing.yaml their rows —

What gets better. The ledger no longer lists every feature. A feature brings its term, and a caller brings a new one in a file of its own, with no shipped file edited (see the new test). Each term is one expression that sums the feature's contributions, so a result reports each feature's share of every effect by name. Each file loads and typesets alone. The cross-feature rules (status × sizing, ramps) are files you can see.

What it costs. 896 → 1076 lines. The 11 dimensions are declared 44 times. There are 57 given: entries, mostly shared flags and envelopes; status.yaml alone reads 11. The flag coupling (has_sizing read by status, has_invest by sizing) is now visible, but it is not gone.

Type of Change

  • Bug fix
  • New feature
  • Documentation update
  • Code refactoring

Related Issues

Stacked on #341.

Testing

  • I have tested my changes
  • Existing tests still pass
Equivalence, gates, what was not done

Equivalence. The merged spec against the ported monolith of #341, declaration by declaration, with descriptions ignored:

dimensions   11 = 11
relations     6 = 6
parameters   93 = 93
variables    18 = 18
constraints  86 = 86   identical text
sos           1 = 1
expressions  14 → 19   + flow_hour_costs, status_costs, sizing_costs, investment_costs, storage_costs
effect_temporal = flow_hour_costs + status_costs
effect_lump     = investment_costs + sizing_costs + storage_costs
objective    identical; nothing is left under given:

Changes. PROGRAM is the fragment directory, and program() merges it. Package data ships math/program/*.yaml. test_data's ledger check follows the terms one level down. The new tests/math/test_fragments.py checks three things: every fragment loads alone, the merge leaves nothing given, and a grid fee in a fragment of its own triples a unit-cost bill.

Gates (Python 3.12, pandas 3.0.6 with pyarrow, and pandas 2):

pytest tests -n 8        717 passed, 74 skipped
benchmark smoke test     41 passed (it builds through program(), so it reads the fragments)
ruff check / format      clean

Not done. docs/design/*.md still describe program.yaml.

What mathspec could take from this. Reading shared dimensions and parameters without restating them would remove most of the added lines.

Checklist

  • My code follows the project style
  • I have updated documentation if needed
  • I have added tests for new functionality (if applicable)

🤖 Generated with Claude Code

https://claude.ai/code/session_01UVhM31okkc7zrhYj7DncVs

…s own term to the ledger

program.yaml is split into ten fragments under math/program/, which
`program()` composes with mathspec's `merge`. effects.yaml declares
`effect_temporal` and `effect_lump` as sums with a frame and no body
(energy-models/mathspec#742); flows, status, sizing, investment and storage
each add one named term to them. A fragment reads what another declares under
`given:`, and each loads and prints on its own.

The merged spec declares the same 93 parameters, 18 variables and 86
constraints as the monolith, with identical text. The only new names are the
five per-feature terms, which a result now reports as well.

PROGRAM is now the fragment directory. tests/math/test_fragments.py holds the
fragments to that shape, and shows a fee added in a file of its own reaching
the objective with no shipped file edited. test_data's ledger check follows
the terms one level down.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UVhM31okkc7zrhYj7DncVs
@github-actions

Copy link
Copy Markdown
Contributor

Benchmark hint

benchmark/test_reference.py under pytest-benchmem — quarter-year horizon; peak = memray allocator high-water, as on CodSpeed. The label columns are each model's size: the temporal grid (timesteps × periods), element stats (components/flows/effects/series) and the measured solver model (variables/binaries/constraints). Shared-runner walltime is noisy: a hint, not a gate.

test_reference_build

name time (s) mean peak (MiB) mean timesteps periods components flows effects series variables constraints
test_reference_build[district_heating] (run) 1.165 8.52 2,190 1 8 15 2 6 35,043 26,281
test_reference_build[energy_transition] (run) 2.443 75.36 2,190 8 15 25 3 10 490,592 245,320
test_reference_build[green_city] (run) 1.237 10.47 2,190 1 15 25 3 10 61,326 37,237
test_reference_build[industry_park] (run) 1.376 13.87 2,190 1 9 16 2 5 76,657 105,112
test_reference_build[stress] (run) 10.02 112.76 136 16 83 204 30 270 503,056 1,179,906
Replicate locally / compare against chore/specsolve-mathspec-0.2
uvx --from 'pytest-benchmem[plot]' benchmem sweep fluxopt \
    git+https://github.com/fluxopt/fluxopt@chore/specsolve-mathspec-0.2 \
    git+https://github.com/fluxopt/fluxopt@claude/fluxopt-mathspec-732-yesoe9 \
    --suite benchmark/ --memory
uvx --from 'pytest-benchmem[plot]' benchmem compare .benchmarks/sweep/*.json

One fresh venv per ref, checkout untouched; covers the archetype, IO and reference-system benchmarks. Alternative flows: benchmark/README.md.

@FBumann
FBumann marked this pull request as ready for review September 29, 2026 07:00
FBumann added a commit that referenced this pull request Sep 29, 2026
_(Intent: to be written by the author.)_

> [!NOTE]
> The following content was generated by AI.

## Description

The stack #274 → #357 in one PR, so it lands with one merge and one CI
run instead of 62. Each commit below was already reviewed in its own PR.
This PR adds only the merges that bring the stack up to date with
`main`, plus one doc fix.

**Included PRs, bottom to top:** #274, #275, #289, #290, #291, #293,
#294, #295, #296, #297, #298, #299, #300, #301, #302, #303, #304, #305,
#306, #307, #308, #310, #311, #312, #313, #314, #315, #316, #317, #318,
#319, #320, #321, #322, #323, #324, #325, #326, #327, #328, #329, #330,
#331, #332, #333, #336, #341, #342, #343, #344, #345, #346, #347, #348,
#349, #350, #351, #352, #353, #354, #355, #357.

**How `main` was brought in:** main was merged into #274 and then
carried up the stack one branch at a time. The branch that first ran
into each conflict got the fix:
- `pyproject.toml` (#290, #341): kept the stack's dependency block and
applied main's `linopy==0.9.0` pin. #354 later drops `linopy` and
`netcdf4`, and that wins.
- `README.md` (#319, #351): kept the stack's text and restyled the code
blocks the way ruff 0.16 formats them (main's lint bump).
- `benchmark/uv.lock` (#341): regenerated with `uv lock`.
- ruff 0.16 also reformats four notebooks (#351). The same formatting is
in 19ab587.
- #213 is already reverted on `main` (#356). `main`'s tree equals the
revert commit inside the stack (ef32b05), so the merge with `main` takes
the stack's tree unchanged.
- #357 on top: the ruff `T201` ignores keep main's
`benchmark/flixopt_stress.py` and swap `src/fluxopt/benchmark.py` for
`benchmark/reference.py`. The two `python -m fluxopt.benchmark stress`
commands that main's #271 added (`benchmark/flixopt_stress.py`,
`docs/benchmark.md`) now point at `reference.py`.

These merge commits were never pushed to the stack branches, so those
PRs still show conflicts on GitHub. Once this merges, they can be
closed.

## Type of Change
- [ ] Bug fix
- [x] New feature
- [x] Documentation update
- [x] Code refactoring

## Related Issues
Supersedes #274 … #357 (the list above).

## Testing
- [x] I have tested my changes
- [x] Existing tests still pass

<details><summary>Gates</summary>

On this branch:

```text
pytest tests -n auto                    695 passed, 72 skipped
benchmark/ pytest --benchmark-disable   31 passed
ruff check / format                     clean
pyrefly check                           0 errors
```

At the branches where a conflict was resolved (before #357):

```text
#290 chore/pin-lpspec                700 passed, 83 skipped   lint ok
#341 chore/specsolve-mathspec-0.2    704 passed, 74 skipped   lint ok
#351 refac/specsolve-api             710 passed, 72 skipped   lint ok
#354 refac/drop-dead-weight          704 passed, 72 skipped   lint ok
#355 refac/timestamps-only           703 passed, 72 skipped   lint ok
```

#274 and #319 could not be tested: their pinned git dependencies no
longer resolve (`farkas` is now published as `specsolve`). That affects
only those intermediate states, not this branch. The docs build and
notebooks were left to CI.

</details>

## Checklist
- [x] My code follows the project style
- [x] I have updated documentation if needed
- [x] I have added tests for new functionality (if applicable)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_0182CG6AhGfcNcWshCsR8Rt5

---
_Generated by [Claude
Code](https://claude.ai/code/session_0182CG6AhGfcNcWshCsR8Rt5)_

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

FBumann commented Sep 29, 2026

Copy link
Copy Markdown
Collaborator Author

Note

The following content was generated by AI.

Landed in #358 (squash-merged into main as 4c0f220).


Generated by Claude Code

@FBumann FBumann closed this Sep 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants