Skip to content

feat(api): an LP solve can start from an earlier answer's basis, matched by coordinate, so a model that gained a cut starts from where it was - #1877

Merged
FBumann merged 8 commits into
claude/festive-carson-8x46sq-basisfrom
claude/festive-carson-8x46sq-warm
Oct 8, 2026
Merged

FBumann merged 8 commits into
claude/festive-carson-8x46sq-basisfrom
claude/festive-carson-8x46sq-warm

Conversation

@FBumann

@FBumann FBumann commented Oct 7, 2026

Copy link
Copy Markdown
Collaborator

Prompt: Add stacked PRs about the new things we discussed

The discussion it refers to began with: "What about basis, warmstaer initial solution etc? What kind of things would be reasonable to add to boost sovler performance?"

Note

The following content was generated by AI.

solve(start=earlier_answer) starts an LP from the basis an earlier answer ended on. The basis is matched by coordinate, so a master that gained a cut starts where it was. On the Benders bench this saved 66–73% of the simplex iterations. This is #1869 part 3 and closes the gap #382 left.

What it is
  • Matched by coordinate (readback.matched_basis). A coordinate both builds hold keeps its status. A column only the new build holds starts at a bound. A row only the new build holds starts basic, so a gained cut enters without moving the vertex. A declaration whose dims changed is new.
  • Made takeable. _counted makes it one basic entry per row: a surplus leaves from the last basic columns, and a shortfall is made up by the last nonbasic rows. settled (now shared with the read side) then puts every nonbasic column at a bound the new build has. Each solver repairs a singular basis itself.
  • Any answer, any solver: live, from load_result, or from an archive, and read on one solver to warm another, because feat(api): a solve or a sweep asked for outputs={'variable_basis', 'constraint_basis'} reads the basis it ended on, in the same words on every solver #1876's statuses are one vocabulary. The answer must carry both basis outputs. The refusal names the outputs= to solve it with, before the solver loads.
  • A mixed-integer model is refused: it has no basis to start from. The PR stacked on this one makes start= on a MIP take values instead.
  • The sink family is simpler. WarmStart, warm_start(), and the refusal of another solver's statuses are gone. A member now implements _warm(basis) in its own statuses and _start(values) for an incumbent, and the base checks spans. No caller used warm_start() outside tests and the bench.
Measured: bench/warm_payoff.py, the Benders master

Method: python -m bench.warm_payoff xs s m --wall, HiGHS, at ecdefd1. At every rebuild the master is solved cold, and with start= the previous started answer (keep='nothing' both, so each solve has a fresh solver). Simplex iterations are read off HiGHS and are deterministic. Wall seconds count the whole solve call, including reading the basis back and matching it.

generators rebuilds iterations cold iterations started saved wall cold wall started
6 28 112 35 68.8% 0.099 s 0.339 s
100 34 283 97 65.7% 0.164 s 0.465 s
1,000 47 856 235 72.5% 0.809 s 0.839 s

The wall columns were taken at load averages between 1.7 and 2.0 on 4 cores. They are a direction, not a measurement: below 1,000 generators the extra work of a started solve (reading the basis back and matching it, a polars join per declaration, about 9 ms a solve here) costs more than the iterations save. A run at 10,000 generators reached the 30-minute limit of a background job before it printed, so it is not in the table. A wall measurement on an idle machine is still owed before the how-to claims time. The conclusion that goes into the warm-start how-to is the iteration count only.

Verification

Gates: ruff check ., ruff format --check ., pyrefly check, the full suite (pytest -n auto, with HiGHS, Gurobi and Xpress installed), pytest bench/test_harness.py -k started_master, test_docstring_links.py, test_assertions.py, zensical build --strict.

tests/test_start.py covers the following, on every sink unless a row says otherwise:

  • An unchanged model started from its own answer does no simplex work.
  • A model that gained or lost snapshots takes fewer iterations than cold and reaches the same objective.
  • A constraint gained: the Warm starts: rebuild-over-edit is only cheap if the solve is warm #382 case.
  • HiGHS's basis starts Gurobi and Xpress.
  • A saved answer and an archived one start a solve.
  • A declaration whose dims changed starts as new.
  • The refusals: an answer without its basis, one that ended on no basis, a MIP, a closed answer.
  • settled and _counted, case by case.

tests/test_warm_start.py's sink half moved onto warm(basis) and start(values). Its test of the other-solver refusal is gone with the refusal, and test_start.py asserts the opposite now.

Mutation table, from python -m tools.mutate against the full suite, except the rows marked by hand, which were taken with its three precautions (a clean tree, git checkout -- to restore, __pycache__ dropped on both sides):

guard result
a mixed-integer model takes no basis (engine.py:228-229) caught
a declaration whose dims changed is new (readback.py:362-363) caught
an answer without its basis refused (result.py:660-665) caught
sps.solve checks the start before the build (api.py:563-564) caught
a basis that does not span the model refused (base.py:171-172) caught
a column at equal bounds is fixed (base.py:332) caught
a nonbasic column moves to a bound it has (base.py:333) caught
a surplus leaves from the columns (by hand: if surplus > 0 → if False) caught (1 failed)
a shortfall enters as rows (by hand: elif surplus < 0 → elif False) caught (1 failed)
Xpress: a nonbasic row names its slack's side (by hand: rows mapped as columns) still green. A probe was added (test_a_basis_is_loaded_in_the_row_statuses_xpress_itself_reports), and it is green too: Xpress puts a nonbasic slack at its one finite bound when it loads the basis, whichever side it was given. So the side cannot be observed. The mapping stays the one Xpress reports, and the probe holds the rest of the round trip.

The tree was clean after each run.

Deliberately not done
  • No start= on solve_over: keep='progress' already carries a sweep's work from slice to slice. A sweep that rebuilds between slices is a separate case.
  • No repair choice beyond the count. Which basic column leaves is arbitrary, and each solver repairs a singular basis. A smarter choice waits for a model that shows it matters.

🤖 Generated with Claude Code

https://claude.ai/code/session_01GLBvwp6NU7YjszFz5QMRf1


Generated by Claude Code

claude added 7 commits October 7, 2026 07:18
…hed by coordinate, so a model that gained a cut starts from where it was

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GLBvwp6NU7YjszFz5QMRf1
…is' into claude/festive-carson-8x46sq-warm

# Conflicts:
#	src/specsolve/api.py
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GLBvwp6NU7YjszFz5QMRf1
@read-the-docs-community

read-the-docs-community Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

@codspeed

codspeed Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 67 untouched benchmarks
⏩ 307 skipped benchmarks1


Comparing claude/festive-carson-8x46sq-warm (70506b4) with claude/festive-carson-8x46sq-basis (452789c)

Open in CodSpeed

Footnotes

  1. 307 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@FBumann
FBumann added this pull request to stack #1874 October 7, 2026 08:46
@FBumann
FBumann removed this pull request from stack #1874 October 8, 2026 08:06
@FBumann
FBumann added this pull request to stack #1886 October 8, 2026 08:07
…x46sq-warm

Brings main through the stack: polars 2.0, the 0.7.0 release and
load_inputs. The collects this branch added go through collected.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014wLrXpeSGDqoBFxyrs8NEz
@FBumann
FBumann merged commit d5e88c2 into main Oct 8, 2026
13 checks passed
@FBumann
FBumann deleted the claude/festive-carson-8x46sq-warm branch October 8, 2026 14:16
FBumann added a commit that referenced this pull request Oct 8, 2026
…swer's or any source a parameter takes, matched by coordinate (#1878)

> **Prompt:** Add stacked PRs about the new things we discussed

The discussion it refers to began with: "What about basis, warmstaer
initial solution etc? What kind of things would be reasonable to add to
boost sovler performance?" A follow-up set the shape of the values: "Why
start: Result | Mapping[str, pl.DataFrame] ? Why not str, Sources?"

> [!NOTE]
> The following content was generated by AI.

`start=` on a mixed-integer model gives the solver values as its first
incumbent. They come from an earlier answer, or per variable in any
shape a parameter's source takes. They are matched by coordinate and may
be partial. This is #1869 part 2.

<details><summary>What it is</summary>

- **One keyword, decided by the model**: an LP starts from an earlier
answer's basis (#1877); a mixed-integer model starts from values. An
earlier answer (live, saved or archived) gives its primal. `start=`
takes `Result | Mapping[str, Source]`.
- **Values enter as a parameter does**: `sources.read_values` reads each
through the parameter reader over the variable's dims, so a start takes
a table (polars, pandas, Arrow), a parquet path, a `{label: value}` map,
a sequence in label order, or one number for every coordinate. It is
checked the same way: a missing dim column, a label the dimension lacks,
a coordinate twice, a null, a sequence of the wrong length. The reader's
refusals now name what they read (`kind`): `parameter 'cost'` as before,
`start= for variable 'take'` here. One message changed for parameters
too: "A missing row is a zero coefficient" became "A missing row means
no value there", which is true of both.
- **Read in `Model.solve`, not the engine**, because
`test_engine_is_isolated` keeps `specsolve.sources` out of
`relational/`. The engine takes tidy frames per variable.
- **Matched by coordinate** (`readback.matched_values`), on the
placement #1877 introduced, now shared by a basis and values (`_place`).
A variable or a coordinate the start leaves out is NaN, and the solver
completes it.
- **Partial starts reach every solver**: HiGHS by `setSolution`'s sparse
form, Gurobi by `GRB.UNDEFINED`, and Xpress by `addMipSol` over the
columns given.
- **Refused before the solver loads**: a name that is no variable (with
the near miss), anything a parameter over the same dims is refused for,
a start that places no value at all (an earlier answer of another model,
say), and values for an LP.
- **`sps.solve` no longer checks a start before the build**: whether it
needs a basis or values depends on the model, which the build decides.
#1877's test of the earlier check moved to "before the solver loads".

</details>

<details><summary>Why not a bare path for the whole start</summary>

A path is ambiguous where the rest of `start=` is not. It could be a
saved answer or an archive, which have two loaders. An archive can also
hold a sweep, which has no single answer to start from.
`start=sps.load_result(path)` is one explicit line. A path per variable
works, through the parameter reader.

</details>

<details><summary>How a start is observed</summary>

Each sink is stopped at its root node with presolve, heuristics and
(Gurobi, Xpress) cuts off, so any incumbent it returns is one it was
given. On the 12-item knapsack fixture, all three sinks give the same
numbers:

| | HiGHS | Gurobi | Xpress |
|---|---|---|---|
| no start | no solution | no solution | no solution |
| a full start worth 10 | 10 | 10 | 10 |
| a partial start naming two items | 35 | 35 | 35 |
| one number, `0.0`, for every item | 0 | 0 | 0 |

In the partial case each solver filled in the rest itself. The test
asserts that the named items stay in, not the 35.

</details>

<details><summary>Verification</summary>

Gates: `ruff check .`, `ruff format --check .`, `pyrefly check`, the
full suite (`pytest -n auto`, with HiGHS, Gurobi and Xpress installed:
5371 passed), `zensical build --strict`.

New in `tests/test_start.py`, on every sink:
- The control: no incumbent at the root without a start.
- A full start is the incumbent, as a polars frame, a pandas frame, a
parquet path, a map and a sequence.
- One number starts every coordinate.
- A partial start is completed, and keeps what it names.
- An earlier answer starts the solve at its optimum.

And, parametrized, the refusals:
- a misspelled variable, a missing dim column, a label the dimension
lacks, a coordinate twice, and a sequence of the wrong length;
- an earlier answer of another model, which places nothing;
- values for an LP.

`test_a_hint_the_solver_refuses_is_loud_not_a_silent_cold_start` now
stubs the sparse `setSolution`.

Mutation table, from `python -m tools.mutate` against the full suite,
except the rows marked by hand, which were taken with its three
precautions:

| guard | result |
|---|---|
| a start naming no variable refused (`api.py:394-395`) | **caught** |
| a start placing nothing refused (`engine.py:260-264`) | **caught** |
| values for an LP refused (`engine.py`, the `isinstance(start, Result)`
refusal) | **caught**, taken before the move to the reader |
| HiGHS: only the given values cross (by hand: the NaN filter removed) |
**caught** (1 failed) |
| Xpress: only the given values cross (by hand: the NaN filter removed)
| **caught** (1 failed) |
| Gurobi: a value not given is `UNDEFINED` (by hand: the mapping
removed) | **caught** (1 failed) |

The tree was clean after each run. The earlier rows for the table's own
name and column checks are gone with those checks: the parameter reader
does both now, under its own tests.

</details>

<details><summary>Deliberately not done</summary>

- A coordinate the model's dimension holds but the variable's mask
leaves out is dropped without a word, as is a value outside a variable's
bounds: a start is a hint, and the solver repairs it.
- No `start=` on `solve_over`, as in #1877.

</details>

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01GLBvwp6NU7YjszFz5QMRf1

---------

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants