perf(highs): handing a basis to HiGHS takes milliseconds rather than a second at a million columns

[FBumann]

Prompt: yes, stack both PRs on 1882

Note

The following content was generated by AI.

The HiGHS sink converted each basis status with a Python call. Now numpy picks each status. On dispatch m, 6 re-solves that start from a basis and read one back take 21.2 s instead of 30.0 s, with the same iterations.

Benchmark

HiGHS, base 9b046ce (the head of #1882) against 6ecab89. Each process builds the case and solves it with outputs={'basis'}. It then re-solves it after 6 updates that scale load by 1 + 0.02·sin(k), each with start= the previous answer and outputs={'basis'}. Each re-solve so reads, matches and writes a basis. The time is the wall time of the 6 solve() calls. One process per run, 3 rounds, the order alternating each round. Load average 0.4 to 1.1. Both sides took the same iterations and reached the same objectives in every round.

case	base (s)	this PR (s)	change	simplex iterations, both
dispatch s (100 × 1000)	2.668, 2.287, 2.222 (median 2.287)	1.474, 1.786, 1.694 (median 1.694)	−26%	3700
storage s	5.013, 5.375, 4.843 (median 5.013)	4.092, 4.270, 3.967 (median 4.092)	−18%	10639
dispatch m (100 × 10000)	29.011, 30.076, 29.997 (median 29.997)	21.252, 20.712, 21.241 (median 21.241)	−29%	37123

A first run started at load average 8.5, just after the test suite, and is not counted. Its medians were within 0.2 s of these.

Microbenchmark on highspy 1.15.1, 1M statuses, one run:

step	before	after
statuses into HighsBasis.col_status (_warm)	1.04 s (a HighsBasisStatus(int) call each)	0.064 s (numpy index into the five members)
integers out of the returned list (_basis)	0.44 s (int() each)	0.19 s (np.fromiter over .value)
the highspy getter that makes the list	0.9 to 2.0 s	unchanged: highspy has no numpy path

On 200k codes, building the BASIS series by gather instead of from strings took 0.003 s against 0.12 s.

A profile of dispatch s at the base, per re-solve: write 0.158 s, read 0.093 s, match 0.034 s, status table 0.027 s. With this change, the write falls to 0.009 s. The read stays at 0.092 s, which is mostly the highspy getter.

Verification

• Full suite (pytest -n 4, with HiGHS, Gurobi and Xpress): 5301 passed, 566 skipped, 1 xfailed.
• ruff check ., ruff format --check . and pyrefly check: 0 errors.
• Not run: test-bench and docs-test. This PR changes no file under bench/ or docs/, and only private docstrings.

Deliberately not done

• No change to Gurobi or Xpress. Their sinks already pass numpy arrays.
• No way round the highspy getter. writeBasis to a file took 0.165 s for 200k columns before any parsing, so it does not pay. A numpy accessor in highspy would remove the rest.
• No mutation table. The change adds no guard. The existing basis tests on HiGHS (test_basis, test_start, test_warm_start) cover the round trip.
• This PR is stacked on feat(api): a solve or a sweep can start from tables of values and basis statuses, and a sweep from an earlier sweep or from the slice before it #1882 beside refactor(api): keep= is gone, and a re-solve carries on from where the last solve ended unless start=None says otherwise #1884, not on it. The code changes do not overlap. Both add their CHANGELOG.md line after feat(api): a solve or a sweep can start from tables of values and basis statuses, and a sweep from an earlier sweep or from the slice before it #1882's, so the second to merge has a one-line conflict there.

🤖 Generated with Claude Code

https://claude.ai/code/session_014wLrXpeSGDqoBFxyrs8NEz

[read-the-docs-community]

Documentation build overview

📚 specsolve | 🛠️ Build #34999730 | 📁 Comparing 333d1b5 against latest (5adfc58)

  🔍 Preview build  

9 files changed · ± 9 modified

± Modified

• tables/index.html
• about/architecture/index.html
• about/changelog/index.html
• howto/archiving/index.html
• howto/warm-start/index.html
• reference/api/index.html
• reference/glossary/index.html
• reference/internals/index.html
• reference/sweeps/index.html

[codspeed]

Merging this PR will not alter performance

✅ 67 untouched benchmarks
⏩ 307 skipped benchmarks¹

---

_{Comparing claude/wizardly-lamport-e3g6om-highs-basis (333d1b5) with claude/wizardly-lamport-e3g6om (9b046ce)}

Footnotes

1. 307 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩