Repository navigation
feat(api): a mixed-integer solve can start from values, an earlier answer's or any source a parameter takes, matched by coordinate - #1878
Merged
FBumann merged 10 commits intoOct 8, 2026
Conversation
…swer's or a table per variable, matched by coordinate Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GLBvwp6NU7YjszFz5QMRf1
…carson-8x46sq-mipstart
…is documented Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GLBvwp6NU7YjszFz5QMRf1
…carson-8x46sq-mipstart # Conflicts: # docs/howto/warm-start.md
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GLBvwp6NU7YjszFz5QMRf1
Merging this PR will not alter performance
Comparing Footnotes
|
FBumann
added this pull request to stack #1874
October 7, 2026 08:46
…r's source takes, read and checked as one is Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GLBvwp6NU7YjszFz5QMRf1
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GLBvwp6NU7YjszFz5QMRf1
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GLBvwp6NU7YjszFz5QMRf1
…row are installed Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GLBvwp6NU7YjszFz5QMRf1
FBumann
pushed a commit
that referenced
this pull request
Oct 7, 2026
… source takes #1878 gained values in any shape a parameter's source takes, read by the parameter reader. Here that reader serves every table of a Start, under all three readers: read_start in sources.py reads primal tables as numbers and basis tables as strings held to the five words, against the build that solves. The engine takes the start already read. A sweep checks names before a slice is built, cuts any table-shaped start by its axis, and passes a shape that is not a table, such as one number, to every slice whole. start='previous' hands each slice the Result before it, so its frames are matched by coordinate rather than read as a caller's table. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014wLrXpeSGDqoBFxyrs8NEz
FBumann
removed this pull request from stack #1874
October 8, 2026 08:06
FBumann
added this pull request to stack #1886
October 8, 2026 08:07
…46sq-mipstart Brings main through the stack: polars 2.0, the 0.7.0 release and load_inputs. The collects this branch added go through collected. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014wLrXpeSGDqoBFxyrs8NEz
FBumann
added a commit
that referenced
this pull request
Oct 8, 2026
…is statuses, and a sweep from an earlier sweep or from the slice before it (#1882) > **Prompt:** Review the R stack with 1878. Look for room for simplification, refactoring, better naming, better seams and if better encapsulation. > **Prompt:** Id like to unify the start for lp and mip. But we need to find a simple and intuitive api. Users need to be able to pass basis values easily too! > **Prompt:** I think start on a sweep is prefectly reasonable! And it makes no sense for it to not exist if its availlable on regular solve! > [!NOTE] > The following content was generated by AI. `start=` is one shape for an LP and a mixed-integer model: an earlier answer, or tables keyed by reader (`primal`, `variable_basis`, `constraint_basis`). `solve_over` also takes an earlier `Sweep` or `'previous'`, cut by the axis as a source is. `outputs={'basis'}` asks for the basis. <details><summary>The API</summary> ```python start: Result | Start | None # solve, Model.solve start: Result | Sweep | Start | Literal['previous'] | None # solve_over ``` - **`Start` is a `TypedDict`** in `specsolve.types`: `{'primal': {name: table}, 'variable_basis': {...}, 'constraint_basis': {...}}`. Each table takes every shape a parameter's source takes (#1878), read by the same reader. Basis tables are read as strings and held to `basic`, `at_lower`, `at_upper`, `fixed` and `superbasic`. - **An LP starts from a basis where one is given, else from values. A mixed-integer model starts from values.** An LP answer solved without `outputs={'basis'}` now starts a solve from its primal rather than being refused. - **Values for an LP follow each solver's `lp_values`.** HiGHS uses a complete primal and warns of a partial one. Gurobi warns of either. Xpress warns of a complete one and refuses a partial one, which `loadLPSol` cannot take. A warning asks the user to report a gain at the issue tracker. - **A sweep's start is cut by its axis, as a source is.** A table over the sliced dimension gives each slice its rows: a scenario its scenario's, a window the coordinates it covers, lookahead included, over its local index. A table without it reaches every slice whole. An earlier sweep is its answer, so an archive without kept windows starts a sweep too. Each `Slice` carries the cut it applied to its sources, and the start goes through the same function. - **A slice its start leaves no row starts from nothing, with a warning.** That covers a table short of the slice's key, and an earlier sweep whose slice left no values. A missing start costs time, never the answer, so it no longer refuses the sweep. This changed after review: "For empty tables we should warn, not error". - **`'previous'`** starts each slice from the answer of the one before it, matched by the slice model's own coordinates (local `t` for a window). Each slice is solved with its basis for the next to start from, whether or not `outputs=` asks for it; the answer, the spill and the archive keep only what `outputs=` asks for. The first slice, and a slice after one that left no values, starts cold. Refused under an executor. A slice read back off a spill starts the next from what the spill holds. (#1884 changes this.) - **`start=` combines with `keep='solver'` only.** `keep='progress'` and `keep='nothing'` also say what the solve begins from, so the two together are refused, on `solve` and `solve_over`. - **`outputs={'basis'}`** replaces `{'variable_basis', 'constraint_basis'}`. The readers keep their names, and the files stay under `variable_basis/` and `constraint_basis/`. </details> <details><summary>From the review of #1865–#1878</summary> - The basis vocabulary (`BASIS_STATUSES`, the `BASIS` dtype, the codes) moved from the sinks to `answer_layout`, beside the outputs read back in it. - The engine matches a start to a `Basis` or to values and branches on it, rather than returning a closure. It reads an earlier answer through `Result._start` rather than its private frames. - `Result` and `Sweep` dispatch an output kind explicitly rather than by `getattr`, and the output readers share one path. - Each sink's write-back table names the status it maps rather than listing bare codes. - The spill manifest check compares the key first, then the outputs. - `Result.dual` no longer says slacks are not read. </details> <details><summary>Values for an LP: the probe behind <code>lp_values</code></summary> A random 300×400 LP (5% density, seed 0), solved once cold and once started from its own optimum, outside specsolve. This is the best case for a values start, one run each, and it was not taken on a machine checked for idleness. | | cold | from the full primal | from half the columns | |---|---|---|---| | HiGHS `setSolution` (simplex iterations) | 661 | 0 | 661 | | Gurobi `PStart` | 877 | 877 | — | | Xpress `loadLPSol` | 760 | 760 | refused: needs every column | Separately, on HiGHS, `DISPATCH` at 40 snapshots started from a `variable_basis` alone (rows left at their default) took 40 iterations against 29 cold. The test asserts only that such a start is taken. </details> <details><summary>Mutation tables</summary> `python -m tools.mutate`, scoped to `tests/test_start.py` (and `test_basis`, `test_warm_start`, `test_outputs` for the first group), rather than the full suite. A guard the subset catches, the full suite catches too. | guard | result | |---|---| | a key that names no reader (`sources.py`) | **caught** | | a name its reader has no declaration for (`sources.py`) | **caught** | | a basis status outside the five words (`sources.py`) | **caught** | | `lp_values` refused (`engine.py`) | **caught** | | `lp_values` no_gain warns (`engine.py`, body → `pass`) | **caught** | | nothing the model starts from (`engine.py`) | **caught** | | a start that lands nowhere (`readback.py`) | **caught** | | start beside keep progress or nothing (`result.py`) | **caught** | | a word other than `previous` (`strategy.py`) | **caught** | | `previous` under an executor (`strategy.py`) | **caught** | | a table over the local index alone (`strategy.py`) | **caught** | | the warning for a slice its start leaves no row (`strategy.py:407-413`) | **caught** | | by hand: that slice's empty start passed on rather than none | **caught** (3 failed) | | a slice after one with no values starts cold (`strategy.py`) | **caught** | | a word on one solve (`api.py`, taken before the merge in `engine.py`) | **caught** | | a constraint named as a variable keeps its own basis (by hand: both halves in one directory) | **caught** | | a slice keeps only the outputs asked for (by hand: every output the result holds copied) | **caught** | Three guards first survived, and each got a purpose-built probe before the table was taken: the up-front check on a sweep's start and the up-front windows check, which now spy on the build, and the outputs filter, whose test now looks for a basis in the spill. The tree was clean after every run. </details> <details><summary>Verification</summary> - `ruff check .`, `ruff format --check .`, `pyrefly check`, and the full suite (`pytest -n 4`, with HiGHS, Gurobi and Xpress): 5301 passed, 0 failed, on the tree of the last source change. The collected tests (5867) are the same as at the commit before it. An earlier run on the merge reported 5311 passed; I have not found why the two counts differ. - On ef01170 (the empty-start warning): `pixi run check` gave 5329 passed, 566 skipped, 1 xfailed. `docs-test` gave 112 passed. The new test was a strict `xfail` first, and failed on the refusal. - `pixi run test-bench`: 263 passed. An earlier version of this line said it passed, which was wrong: it had last run before the `keep`/`start=` rule, which refused `bench/warm_payoff.py`'s warm arm (`keep='nothing'` beside `start=`). CI caught it. The warm arm now asks for `keep='solver'` and runs first after each update, where the gained cut loads a fresh solver anyway, and an assertion on `Result.kept` holds both arms to a fresh solver. - `pixi run -e docs docs-test` and `docs-build --strict` pass. - #1878's branch moved while this was open (any-shape values). It is merged in here, not rebased, and its tests now pass their tables under `'primal'`. </details> <details><summary>Deliberately not done, and departures</summary> - **One PR rather than a stack.** The review refactors, the basis output, the unified start and the sweep start all reshape #1876–#1878 before release, and each builds on the one before. - **`keep` is unchanged** apart from the refusal beside `start=`. Whether `start=` replaces `keep='progress'`, and which alignment `'previous'` uses for windows, needs a measurement first: a follow-up. - **No dual values in a start.** Gurobi's `PStart` showed no gain, and duals would need the sign conversion reversed per solver. Measure first. - **The other PRs' changelog lines are unchanged.** #1876's line still reads `outputs={'variable_basis', 'constraint_basis'}`, and its title would need the same edit. </details> 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_014wLrXpeSGDqoBFxyrs8NEz --------- Co-authored-by: Claude <noreply@anthropic.com>
FBumann
added a commit
that referenced
this pull request
Oct 8, 2026
…, and when it paid on the benchmark models (#1879) > **Prompt:** Add stacked PRs about the new things we discussed The discussion it refers to began with: "What about basis, warmstaer initial solution etc? What kind of things would be reasonable to add to boost sovler performance?" > [!NOTE] > The following content was generated by AI. The solvers how-to now says how to leave crossover off on each sink and what that costs: no basis, so no `start=`. On HiGHS at rung `m` it paid only on `transport`, by 14%. `bench/crossover.py` measures this. <details><summary>What it is</summary> - `docs/howto/solvers.md` gains "Solve an LP by interior point without crossover": the option on each sink, what the answer carries (duals, no basis), and the conclusion below with this PR's number. - `bench/crossover.py` builds a case's rung once and solves it three ways, each from a fresh solver: HiGHS's own choice (simplex on these LPs), interior point with crossover, and interior point without. It prints solve seconds (`diagnostics().seconds['solve']`, the solve alone and not the build), whether the answer has duals, and the objective's gap relative to simplex. It is not an arm and writes no results file. `bench/test_harness.py` checks that the three ways reach one optimum on `dispatch` `2xs`. - No `src/` change. A solve that would pick crossover itself, when no `outputs=` needs a basis, was the idea in the discussion. These numbers do not support it on HiGHS. </details> <details><summary>Measured: HiGHS, rung m, solve seconds</summary> Method: `python -m bench.crossover <cases> --sizes m`, each case built once and solved three ways with `keep='nothing'`. HiGHS 1.15, at `fcea065` for the first run and `6908ec1` for the others: the bench script changed between them, `src/` did not. Each run started only after the 1-minute load average fell below 0.5 on 4 cores, and nothing else ran on the box. `transport` and `nodal` were run three times, `dispatch` and `storage` twice; each cell is the range over those runs. | case | simplex | interior point with crossover | without crossover | |---|---|---|---| | dispatch | 3.5–3.8 s | 5.5–7.1 s | 5.4–6.3 s | | nodal | 2.5–2.8 s | 1.1–1.3 s | 32.6–34.1 s | | transport | 24.2–25.1 s | 24.3–24.5 s | 20.8–21.5 s | | storage | 48.6–50.6 s | 131.0–133.7 s | 121.7–123.3 s | On `transport`, without crossover is fastest: 21.2 s against 24.7 s for simplex, the means of the three runs, 14% less. On `nodal` it took 28 times as long as interior point with crossover: 33.2 s against 1.18 s, the means. Every answer had duals. The largest objective gap to simplex was 6.7e-9 relative (`dispatch`, without crossover), so the how-to says 7e-9. At rung `s` (one run), the fastest way was the same as at `m` on every case except `transport`. There simplex was fastest: 0.86 s against 1.25 s without crossover. <details><summary>Full output</summary> ``` dispatch s: simplex 0.276 ipm 0.414 no crossover 0.396 dispatch m: simplex 3.783 ipm 7.125 no crossover 6.257 nodal s: simplex 0.167 ipm 0.093 no crossover 0.837 nodal m: simplex 2.729 ipm 1.256 no crossover 34.071 transport s: simplex 0.863 ipm 1.363 no crossover 1.247 transport m: simplex 25.104 ipm 24.309 no crossover 20.794 storage s: simplex 1.248 ipm 2.855 no crossover 2.446 storage m: simplex 50.602 ipm 133.702 no crossover 121.679 second and third runs, with the relative gap: transport m (9.76456e+08): simplex 24.891 / 24.185, ipm 24.461 / 24.263 (+1.2e-16), no crossover 21.502 / 21.312 (+1.6e-11) nodal m (3.07336e+09): simplex 2.541 / 2.766, ipm 1.154 / 1.126 (0), no crossover 33.049 / 32.627 (+4.2e-12) dispatch m (2.17995e+09): simplex 3.488, ipm 5.496 (-4.4e-16), no crossover 5.414 (+6.7e-09) storage m (9.62692e+08): simplex 48.648, ipm 131.049 (-5.0e-16), no crossover 123.328 (+3.1e-09) ``` </details> </details> <details><summary>Verification</summary> Gates: `ruff check .`, `ruff format --check .`, `zensical build --strict`, and `pytest bench/test_harness.py -k crossover_bench`. The `src/` tree is #1878's, whose full suite is green. </details> <details><summary>Deliberately not done</summary> - Gurobi and Xpress are not measured: their bundled licences stop well below rung `m`. - Rung `l` is not measured: `storage` at `m` already takes over two minutes per way. - No choice of method in `src/`. The numbers say HiGHS's default is the right default on these models. </details> 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01GLBvwp6NU7YjszFz5QMRf1 --------- Co-authored-by: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The discussion it refers to began with: "What about basis, warmstaer initial solution etc? What kind of things would be reasonable to add to boost sovler performance?" A follow-up set the shape of the values: "Why start: Result | Mapping[str, pl.DataFrame] ? Why not str, Sources?"
Note
The following content was generated by AI.
start=on a mixed-integer model gives the solver values as its first incumbent. They come from an earlier answer, or per variable in any shape a parameter's source takes. They are matched by coordinate and may be partial. This is #1869 part 2.What it is
start=takesResult | Mapping[str, Source].sources.read_valuesreads each through the parameter reader over the variable's dims, so a start takes a table (polars, pandas, Arrow), a parquet path, a{label: value}map, a sequence in label order, or one number for every coordinate. It is checked the same way: a missing dim column, a label the dimension lacks, a coordinate twice, a null, a sequence of the wrong length. The reader's refusals now name what they read (kind):parameter 'cost'as before,start= for variable 'take'here. One message changed for parameters too: "A missing row is a zero coefficient" became "A missing row means no value there", which is true of both.Model.solve, not the engine, becausetest_engine_is_isolatedkeepsspecsolve.sourcesout ofrelational/. The engine takes tidy frames per variable.readback.matched_values), on the placement feat(api): an LP solve can start from an earlier answer's basis, matched by coordinate, so a model that gained a cut starts from where it was #1877 introduced, now shared by a basis and values (_place). A variable or a coordinate the start leaves out is NaN, and the solver completes it.setSolution's sparse form, Gurobi byGRB.UNDEFINED, and Xpress byaddMipSolover the columns given.sps.solveno longer checks a start before the build: whether it needs a basis or values depends on the model, which the build decides. feat(api): an LP solve can start from an earlier answer's basis, matched by coordinate, so a model that gained a cut starts from where it was #1877's test of the earlier check moved to "before the solver loads".Why not a bare path for the whole start
A path is ambiguous where the rest of
start=is not. It could be a saved answer or an archive, which have two loaders. An archive can also hold a sweep, which has no single answer to start from.start=sps.load_result(path)is one explicit line. A path per variable works, through the parameter reader.How a start is observed
Each sink is stopped at its root node with presolve, heuristics and (Gurobi, Xpress) cuts off, so any incumbent it returns is one it was given. On the 12-item knapsack fixture, all three sinks give the same numbers:
0.0, for every itemIn the partial case each solver filled in the rest itself. The test asserts that the named items stay in, not the 35.
Verification
Gates:
ruff check .,ruff format --check .,pyrefly check, the full suite (pytest -n auto, with HiGHS, Gurobi and Xpress installed: 5371 passed),zensical build --strict.New in
tests/test_start.py, on every sink:And, parametrized, the refusals:
test_a_hint_the_solver_refuses_is_loud_not_a_silent_cold_startnow stubs the sparsesetSolution.Mutation table, from
python -m tools.mutateagainst the full suite, except the rows marked by hand, which were taken with its three precautions:api.py:394-395)engine.py:260-264)engine.py, theisinstance(start, Result)refusal)UNDEFINED(by hand: the mapping removed)The tree was clean after each run. The earlier rows for the table's own name and column checks are gone with those checks: the parameter reader does both now, under its own tests.
Deliberately not done
start=onsolve_over, as in feat(api): an LP solve can start from an earlier answer's basis, matched by coordinate, so a model that gained a cut starts from where it was #1877.🤖 Generated with Claude Code
https://claude.ai/code/session_01GLBvwp6NU7YjszFz5QMRf1