Repository navigation
feat(api): a solve or a sweep asked for outputs={'variable_basis', 'constraint_basis'} reads the basis it ended on, in the same words on every solver - #1876
Merged
FBumann merged 6 commits intoOct 8, 2026
Conversation
…onstraint_basis'} reads the basis it ended on, in the same words on every solver Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GLBvwp6NU7YjszFz5QMRf1
…onstraint ends on none Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GLBvwp6NU7YjszFz5QMRf1
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GLBvwp6NU7YjszFz5QMRf1
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GLBvwp6NU7YjszFz5QMRf1
FBumann
added this pull request to stack #1874
October 7, 2026 07:18
Merging this PR will not alter performance
Comparing Footnotes
|
…ck' into claude/festive-carson-8x46sq-basis
This was referenced Oct 7, 2026
FBumann
removed this pull request from stack #1874
October 8, 2026 08:06
FBumann
added this pull request to stack #1886
October 8, 2026 08:07
…x46sq-basis Brings main through the stack: polars 2.0, the 0.7.0 release and load_inputs. The collects this branch added go through collected. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014wLrXpeSGDqoBFxyrs8NEz
FBumann
added a commit
that referenced
this pull request
Oct 8, 2026
…hed by coordinate, so a model that gained a cut starts from where it was (#1877) > **Prompt:** Add stacked PRs about the new things we discussed The discussion it refers to began with: "What about basis, warmstaer initial solution etc? What kind of things would be reasonable to add to boost sovler performance?" > [!NOTE] > The following content was generated by AI. `solve(start=earlier_answer)` starts an LP from the basis an earlier answer ended on. The basis is matched by coordinate, so a master that gained a cut starts where it was. On the Benders bench this saved 66–73% of the simplex iterations. This is #1869 part 3 and closes the gap #382 left. <details><summary>What it is</summary> - **Matched by coordinate** (`readback.matched_basis`). A coordinate both builds hold keeps its status. A column only the new build holds starts at a bound. A row only the new build holds starts basic, so a gained cut enters without moving the vertex. A declaration whose dims changed is new. - **Made takeable.** `_counted` makes it one basic entry per row: a surplus leaves from the last basic columns, and a shortfall is made up by the last nonbasic rows. `settled` (now shared with the read side) then puts every nonbasic column at a bound the new build has. Each solver repairs a singular basis itself. - **Any answer, any solver**: live, from `load_result`, or from an archive, and read on one solver to warm another, because #1876's statuses are one vocabulary. The answer must carry both basis outputs. The refusal names the `outputs=` to solve it with, before the solver loads. - **A mixed-integer model is refused**: it has no basis to start from. The PR stacked on this one makes `start=` on a MIP take values instead. - **The sink family is simpler.** `WarmStart`, `warm_start()`, and the refusal of another solver's statuses are gone. A member now implements `_warm(basis)` in its own statuses and `_start(values)` for an incumbent, and the base checks spans. No caller used `warm_start()` outside tests and the bench. </details> <details><summary>Measured: bench/warm_payoff.py, the Benders master</summary> Method: `python -m bench.warm_payoff xs s m --wall`, HiGHS, at `ecdefd1`. At every rebuild the master is solved cold, and with `start=` the previous started answer (`keep='nothing'` both, so each solve has a fresh solver). Simplex iterations are read off HiGHS and are deterministic. Wall seconds count the whole `solve` call, including reading the basis back and matching it. | generators | rebuilds | iterations cold | iterations started | saved | wall cold | wall started | |---|---|---|---|---|---|---| | 6 | 28 | 112 | 35 | 68.8% | 0.099 s | 0.339 s | | 100 | 34 | 283 | 97 | 65.7% | 0.164 s | 0.465 s | | 1,000 | 47 | 856 | 235 | 72.5% | 0.809 s | 0.839 s | The wall columns were taken at load averages between 1.7 and 2.0 on 4 cores. They are a direction, not a measurement: below 1,000 generators the extra work of a started solve (reading the basis back and matching it, a polars join per declaration, about 9 ms a solve here) costs more than the iterations save. A run at 10,000 generators reached the 30-minute limit of a background job before it printed, so it is not in the table. A wall measurement on an idle machine is still owed before the how-to claims time. The conclusion that goes into the warm-start how-to is the iteration count only. </details> <details><summary>Verification</summary> Gates: `ruff check .`, `ruff format --check .`, `pyrefly check`, the full suite (`pytest -n auto`, with HiGHS, Gurobi and Xpress installed), `pytest bench/test_harness.py -k started_master`, `test_docstring_links.py`, `test_assertions.py`, `zensical build --strict`. `tests/test_start.py` covers the following, on every sink unless a row says otherwise: - An unchanged model started from its own answer does no simplex work. - A model that gained or lost snapshots takes fewer iterations than cold and reaches the same objective. - A constraint gained: the #382 case. - HiGHS's basis starts Gurobi and Xpress. - A saved answer and an archived one start a solve. - A declaration whose dims changed starts as new. - The refusals: an answer without its basis, one that ended on no basis, a MIP, a closed answer. - `settled` and `_counted`, case by case. `tests/test_warm_start.py`'s sink half moved onto `warm(basis)` and `start(values)`. Its test of the other-solver refusal is gone with the refusal, and `test_start.py` asserts the opposite now. Mutation table, from `python -m tools.mutate` against the full suite, except the rows marked by hand, which were taken with its three precautions (a clean tree, `git checkout --` to restore, `__pycache__` dropped on both sides): | guard | result | |---|---| | a mixed-integer model takes no basis (`engine.py:228-229`) | **caught** | | a declaration whose dims changed is new (`readback.py:362-363`) | **caught** | | an answer without its basis refused (`result.py:660-665`) | **caught** | | `sps.solve` checks the start before the build (`api.py:563-564`) | **caught** | | a basis that does not span the model refused (`base.py:171-172`) | **caught** | | a column at equal bounds is `fixed` (`base.py:332`) | **caught** | | a nonbasic column moves to a bound it has (`base.py:333`) | **caught** | | a surplus leaves from the columns (by hand: `if surplus > 0` → `if False`) | **caught** (1 failed) | | a shortfall enters as rows (by hand: `elif surplus < 0` → `elif False`) | **caught** (1 failed) | | Xpress: a nonbasic row names its slack's side (by hand: rows mapped as columns) | **still green**. A probe was added (`test_a_basis_is_loaded_in_the_row_statuses_xpress_itself_reports`), and it is green too: Xpress puts a nonbasic slack at its one finite bound when it loads the basis, whichever side it was given. So the side cannot be observed. The mapping stays the one Xpress reports, and the probe holds the rest of the round trip. | The tree was clean after each run. </details> <details><summary>Deliberately not done</summary> - No `start=` on `solve_over`: `keep='progress'` already carries a sweep's work from slice to slice. A sweep that rebuilds between slices is a separate case. - No repair choice beyond the count. Which basic column leaves is arbitrary, and each solver repairs a singular basis. A smarter choice waits for a model that shows it matters. </details> 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01GLBvwp6NU7YjszFz5QMRf1 --- _Generated by [Claude Code](https://claude.ai/code/session_01GLBvwp6NU7YjszFz5QMRf1)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
FBumann
added a commit
that referenced
this pull request
Oct 8, 2026
…is statuses, and a sweep from an earlier sweep or from the slice before it (#1882) > **Prompt:** Review the R stack with 1878. Look for room for simplification, refactoring, better naming, better seams and if better encapsulation. > **Prompt:** Id like to unify the start for lp and mip. But we need to find a simple and intuitive api. Users need to be able to pass basis values easily too! > **Prompt:** I think start on a sweep is prefectly reasonable! And it makes no sense for it to not exist if its availlable on regular solve! > [!NOTE] > The following content was generated by AI. `start=` is one shape for an LP and a mixed-integer model: an earlier answer, or tables keyed by reader (`primal`, `variable_basis`, `constraint_basis`). `solve_over` also takes an earlier `Sweep` or `'previous'`, cut by the axis as a source is. `outputs={'basis'}` asks for the basis. <details><summary>The API</summary> ```python start: Result | Start | None # solve, Model.solve start: Result | Sweep | Start | Literal['previous'] | None # solve_over ``` - **`Start` is a `TypedDict`** in `specsolve.types`: `{'primal': {name: table}, 'variable_basis': {...}, 'constraint_basis': {...}}`. Each table takes every shape a parameter's source takes (#1878), read by the same reader. Basis tables are read as strings and held to `basic`, `at_lower`, `at_upper`, `fixed` and `superbasic`. - **An LP starts from a basis where one is given, else from values. A mixed-integer model starts from values.** An LP answer solved without `outputs={'basis'}` now starts a solve from its primal rather than being refused. - **Values for an LP follow each solver's `lp_values`.** HiGHS uses a complete primal and warns of a partial one. Gurobi warns of either. Xpress warns of a complete one and refuses a partial one, which `loadLPSol` cannot take. A warning asks the user to report a gain at the issue tracker. - **A sweep's start is cut by its axis, as a source is.** A table over the sliced dimension gives each slice its rows: a scenario its scenario's, a window the coordinates it covers, lookahead included, over its local index. A table without it reaches every slice whole. An earlier sweep is its answer, so an archive without kept windows starts a sweep too. Each `Slice` carries the cut it applied to its sources, and the start goes through the same function. - **A slice its start leaves no row starts from nothing, with a warning.** That covers a table short of the slice's key, and an earlier sweep whose slice left no values. A missing start costs time, never the answer, so it no longer refuses the sweep. This changed after review: "For empty tables we should warn, not error". - **`'previous'`** starts each slice from the answer of the one before it, matched by the slice model's own coordinates (local `t` for a window). Each slice is solved with its basis for the next to start from, whether or not `outputs=` asks for it; the answer, the spill and the archive keep only what `outputs=` asks for. The first slice, and a slice after one that left no values, starts cold. Refused under an executor. A slice read back off a spill starts the next from what the spill holds. (#1884 changes this.) - **`start=` combines with `keep='solver'` only.** `keep='progress'` and `keep='nothing'` also say what the solve begins from, so the two together are refused, on `solve` and `solve_over`. - **`outputs={'basis'}`** replaces `{'variable_basis', 'constraint_basis'}`. The readers keep their names, and the files stay under `variable_basis/` and `constraint_basis/`. </details> <details><summary>From the review of #1865–#1878</summary> - The basis vocabulary (`BASIS_STATUSES`, the `BASIS` dtype, the codes) moved from the sinks to `answer_layout`, beside the outputs read back in it. - The engine matches a start to a `Basis` or to values and branches on it, rather than returning a closure. It reads an earlier answer through `Result._start` rather than its private frames. - `Result` and `Sweep` dispatch an output kind explicitly rather than by `getattr`, and the output readers share one path. - Each sink's write-back table names the status it maps rather than listing bare codes. - The spill manifest check compares the key first, then the outputs. - `Result.dual` no longer says slacks are not read. </details> <details><summary>Values for an LP: the probe behind <code>lp_values</code></summary> A random 300×400 LP (5% density, seed 0), solved once cold and once started from its own optimum, outside specsolve. This is the best case for a values start, one run each, and it was not taken on a machine checked for idleness. | | cold | from the full primal | from half the columns | |---|---|---|---| | HiGHS `setSolution` (simplex iterations) | 661 | 0 | 661 | | Gurobi `PStart` | 877 | 877 | — | | Xpress `loadLPSol` | 760 | 760 | refused: needs every column | Separately, on HiGHS, `DISPATCH` at 40 snapshots started from a `variable_basis` alone (rows left at their default) took 40 iterations against 29 cold. The test asserts only that such a start is taken. </details> <details><summary>Mutation tables</summary> `python -m tools.mutate`, scoped to `tests/test_start.py` (and `test_basis`, `test_warm_start`, `test_outputs` for the first group), rather than the full suite. A guard the subset catches, the full suite catches too. | guard | result | |---|---| | a key that names no reader (`sources.py`) | **caught** | | a name its reader has no declaration for (`sources.py`) | **caught** | | a basis status outside the five words (`sources.py`) | **caught** | | `lp_values` refused (`engine.py`) | **caught** | | `lp_values` no_gain warns (`engine.py`, body → `pass`) | **caught** | | nothing the model starts from (`engine.py`) | **caught** | | a start that lands nowhere (`readback.py`) | **caught** | | start beside keep progress or nothing (`result.py`) | **caught** | | a word other than `previous` (`strategy.py`) | **caught** | | `previous` under an executor (`strategy.py`) | **caught** | | a table over the local index alone (`strategy.py`) | **caught** | | the warning for a slice its start leaves no row (`strategy.py:407-413`) | **caught** | | by hand: that slice's empty start passed on rather than none | **caught** (3 failed) | | a slice after one with no values starts cold (`strategy.py`) | **caught** | | a word on one solve (`api.py`, taken before the merge in `engine.py`) | **caught** | | a constraint named as a variable keeps its own basis (by hand: both halves in one directory) | **caught** | | a slice keeps only the outputs asked for (by hand: every output the result holds copied) | **caught** | Three guards first survived, and each got a purpose-built probe before the table was taken: the up-front check on a sweep's start and the up-front windows check, which now spy on the build, and the outputs filter, whose test now looks for a basis in the spill. The tree was clean after every run. </details> <details><summary>Verification</summary> - `ruff check .`, `ruff format --check .`, `pyrefly check`, and the full suite (`pytest -n 4`, with HiGHS, Gurobi and Xpress): 5301 passed, 0 failed, on the tree of the last source change. The collected tests (5867) are the same as at the commit before it. An earlier run on the merge reported 5311 passed; I have not found why the two counts differ. - On ef01170 (the empty-start warning): `pixi run check` gave 5329 passed, 566 skipped, 1 xfailed. `docs-test` gave 112 passed. The new test was a strict `xfail` first, and failed on the refusal. - `pixi run test-bench`: 263 passed. An earlier version of this line said it passed, which was wrong: it had last run before the `keep`/`start=` rule, which refused `bench/warm_payoff.py`'s warm arm (`keep='nothing'` beside `start=`). CI caught it. The warm arm now asks for `keep='solver'` and runs first after each update, where the gained cut loads a fresh solver anyway, and an assertion on `Result.kept` holds both arms to a fresh solver. - `pixi run -e docs docs-test` and `docs-build --strict` pass. - #1878's branch moved while this was open (any-shape values). It is merged in here, not rebased, and its tests now pass their tables under `'primal'`. </details> <details><summary>Deliberately not done, and departures</summary> - **One PR rather than a stack.** The review refactors, the basis output, the unified start and the sweep start all reshape #1876–#1878 before release, and each builds on the one before. - **`keep` is unchanged** apart from the refusal beside `start=`. Whether `start=` replaces `keep='progress'`, and which alignment `'previous'` uses for windows, needs a measurement first: a follow-up. - **No dual values in a start.** Gurobi's `PStart` showed no gain, and duals would need the sign conversion reversed per solver. Measure first. - **The other PRs' changelog lines are unchanged.** #1876's line still reads `outputs={'variable_basis', 'constraint_basis'}`, and its title would need the same edit. </details> 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_014wLrXpeSGDqoBFxyrs8NEz --------- Co-authored-by: Claude <noreply@anthropic.com>
This was referenced Oct 8, 2026
Merged
FBumann
added a commit
that referenced
this pull request
Oct 9, 2026
> **Prompt:** Prepare the latest release of specsolve. > > **Prompt:** Im wondering if we bumped the layout stuff multiple times in this release. We should only increment once since the last release! > [!NOTE] > The following content was generated by AI. Releases 0.8.0 on merge. The answer layout rose twice since 0.7.0, and the new inputs layout once after it was added. Each now rises once: `ANSWER_LAYOUT` 3 → 4, `INPUTS_LAYOUT` 1. <details><summary>The layout numbers</summary> `v0.7.0..origin/main` on `archive_layout.py` and `relational/answer_layout.py`: | PR | change | |---|---| | #1883 | `LAYOUT` renamed `ANSWER_LAYOUT` (3); `INPUTS_LAYOUT = 1` added | | #1865 | `ANSWER_LAYOUT` 3 → 4 | | #1904 | `ANSWER_LAYOUT` 4 → 5 | | #1913 | `INPUTS_LAYOUT` 1 → 2 | No release wrote the intermediate numbers, so this PR sets `ANSWER_LAYOUT = 4` and `INPUTS_LAYOUT = 1`. The `format.json` example in `Result.save`'s docstring follows. A 0.7.0 answer stamps 3 and a 0.7.0 archive has no root stamp, so both stay refused. The cost is on unreleased files only. An answer written from `main` between #1865 and #1904 stamps 4 and is no longer refused, though it still carries `model_digest`. An archive written from `main` between #1883 and #1913 stamps inputs 1 and is no longer refused, though it may hold `sources.parquet`. No test asserts a layout number; the refusal tests use 0, 99 and no stamp. </details> <details><summary>Release notes and gates</summary> - `## Upcoming version` → `## 0.8.0 (2026-10-09)`, with a paragraph, a **Breaks** list and the lines grouped under Solving, Reading an answer, Archives, Data and Examples. An empty `## Upcoming version` stays above it. All 21 lines are kept; #1876's line says `outputs={'basis'}`, the spelling #1882 gave it. - **Minor, not patch**: #1884 removes `keep=` and `Result.kept`, #1913 removes the digests, and the layout refuses 0.7.0 files. - `python -m tools.changelog check` → `releases 0.8.0 on merge`. - `pixi run check`: 5523 passed, 566 skipped, 4 xfailed. - Not run: `pixi run docs-test`. - Departure: the layout fix rides in this PR rather than its own, because the release must not ship the doubled numbers. It is its own commit. </details> 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01SUzrD57WeiMZtKUnw7wsG8 --------- Co-authored-by: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The discussion it refers to began with: "What about basis, warmstaer initial solution etc? What kind of things would be reasonable to add to boost sovler performance?"
Note
The following content was generated by AI.
outputs={'variable_basis', 'constraint_basis'}makes a solve or a sweep carry the basis status it ended on. The words are the same on HiGHS, Gurobi and Xpress. This is #1869 part 1, and the warm start stacked on it reads it.What it is
BASIS_STATUSES), read back as anEnum:basic,at_lower,at_upper,fixed,superbasic. A row's bound is its right-hand side, so a binding<=row isat_upper, a binding>=rowat_lower, and a binding==rowfixed. A column at equal bounds isfixed, whichever bound its solver names._basis()maps the solver's own codes to the shared ones. It reads only basic or not, and which bound a column is at.Solver.run(handoff, basis=True)then reads each nonbasic status against the model (settled): a row's bound from its sense,fixedfrom equal bounds. That is also why all three sinks agree, even though Gurobi says only "nonbasic" for a row and Xpress names a row's slack.basis=Trueonly whereoutputsnames one of the two.NO_BASIS, which names the solver options that give one. A sweep raises it only where some slice solved. Otherwise it gives its "nothing to read" message.Result.variable_basis/constraint_basis, the same onSweep,kind=on the bridges,<output>/<name>.parquetin a save, a spill and an archive, and a catalog row per declaration. NoLAYOUTbump.Verification
Gates:
ruff check .,ruff format --check .,pyrefly check, the full suite (pytest -n auto, with HiGHS, Gurobi and Xpress installed),zensical build --strict,test_docstring_links.pyandtest_docs_math.py.The local runs above were taken while one test of the base,
test_a_solution_is_read_back_in_label_order_without_sorting_for_it, failed there, so they deselect it. The base has fixed it since, it is merged in, and CI runs the whole suite.tests/test_basis.pycovers the following:superbasicappears, read in the same words on every sink.Enumdtype.Mutation table, from
python -m tools.mutateagainst the full suite (that one test deselected throughPYTEST_ADDOPTS), except one row taken by hand with the same three precautions:base.py:271-272)fixed(base.py:274)base.py:277)result.py:666-667)sweep.py:733-734)highs.py:244-245)xpress.py:140-141)try/except SolverErrorremoved)The tree was clean after each run. One guard is not in the table:
runreads a basis only after a run that left a primal. Reading one after a failed run would be harmless and invisible, because the reader refuses on the status first.Deliberately not done
_basishas no guard for a quadratic constraint: Gurobi refusesVBasisthere itself, which the new test shows.fixedread from a solver for a row: every row is single-sided, so its nonbasic status follows from its sense.🤖 Generated with Claude Code
https://claude.ai/code/session_01GLBvwp6NU7YjszFz5QMRf1