feat(api): every answer holds an optional output only when outputs= asks for it, and a sweep can hold the activity too by FBumann · Pull Request #1865 · fluxopt/specsolve

FBumann · 2026-10-06T21:50:02Z

Prompt: Let’s design the api no properly. How should users configure this? Is this maintainable and stable? What should be optional, what not?

Two follow-ups set scope: "This should be properly typed etc! And immutable..." and "I think Output CAN live in .types!" A review of the stack added: "Can it be simpler? Or a smaller diff?"

Note

The following content was generated by AI.

outputs= on solve, Model.solve and solve_over says what an answer holds beyond the primal, duals and declared expressions. Activity is the first output: it is now opt-in everywhere, and a sweep can hold it. LAYOUT goes 3 → 4.

The rule, and how it is built

One rule for every shape an answer takes: a live, saved, scanned or archived Result, and a Sweep held in memory, run on a thread or process pool, spilled, saved, scanned or archived. Each holds what the solve asked for. Reading anything else refuses with one message: Solve again with outputs={'activity'}.
Typed and immutable. specsolve.types.Output = Literal['activity'] is what a caller writes. OUTPUTS: Mapping[Output, Per] is the one table the code reads: a MappingProxyType giving per variable or per constraint. checked_outputs returns a frozenset[Output]. The checks run before any build. They refuse a bare string, an unknown name, and a core kind such as primal.
The table drives everything: KINDS, the readers' kind=, the spill, Sweep.save, the archive's answer/ and its catalog (_answered_under). Each kind's reader is found by the kind's name (expression → evaluate), so the bridges hold no dispatch table. Adding an output is a table row, one case in Engine._output, a reader on Result and Sweep, and a test.
On disk: format.json gains "outputs": [...], so a reader can tell "not asked for" from "not defined". A spill's sweep.json records them too. A spill_to= directory solved with other outputs is refused rather than resumed, because its slices hold only what they were solved with. A name in the stamp that this package does not know is dropped, so a later output needs no LAYOUT bump. AGENTS.md now says so in one sentence.
Result holds _outputs: Mapping[Output, Mapping[str, LazyFrame]] in place of _activities. Engine._output lays out each output asked for and nothing else, and a sweep slice collects those frames as the result holds them.

Breaks

Result.activity refuses unless the solve passed outputs={'activity'}, and save() writes no activity/ by default.
LAYOUT 3 → 4: an answer, spill or archive written by 0.6 is refused by name.
specsolve.types gains Output. tests/test_architecture.py and architecture.md now allow Literal vocabularies in types.

Coverage moved: tests that read activity now ask for it (test_activity.py, conftest.assert_agrees_with_highs, test_cases.py, the sink tests, differential.differential(outputs=)). test_a_saved_solution_carries_the_activities became …_it_was_asked_for and also loads the answer back. The default-save listing in test_api.py no longer has activity/. test_a_solution_is_read_back_in_label_order_without_sorting_for_it no longer reads activity through _read_back. Activity is laid out by the same per-constraint helper as the duals, so the dual's assertion covers it.

The bench read arm now measures the default answer. test_read reads every primal and dual into frames, or Result.saves, with no activity. So CodSpeed's test_read[*] benchmarks move with this PR: the default answer no longer carries activity. That move is this PR's effect, not noise. Only specsolve has a read arm, so no cross-library comparison changes. The PyPSA parity harness still asks for activity, because it counts rows per block through activity().

Why activity is opt-in: its share of a saved answer

Measured with Result.save on the bench models, rung m, HiGHS, at 79258b1. Each figure is activity's bytes against everything save() wrote:

model	activity	whole answer
nodal	0.8 MB	1.7 MB
sector	0.8 MB	1.5 MB
transport	1.1 MB	2.5 MB
storage	0.1 MB	0.8 MB
dispatch	0.1 MB	0.4 MB
fleet	0.1 MB	0.3 MB

Verification

Gates: ruff check ., ruff format --check ., pyrefly check, the full suite (pytest -n auto, with HiGHS, Gurobi and Xpress installed), pytest bench/test_harness.py, zensical build --strict, and test_docs_math.py + test_docstring_links.py with the docs group installed.

The review simplification (909967d, 3ab29cb, 7e84f24) was gated at 7e84f24 with ruff check ., ruff format --check ., pyrefly check (0 errors), and the full suite with the dev group, Gurobi and Xpress installed: 5087 passed, 564 skipped, 1 xfailed. The docs build and bench/test_harness.py were not re-run.

tests/test_outputs.py (81 cases) covers the following:

every request refusal, on all three verbs, and before any model is built;
every result shape and every sweep shape, each one asked for activity and one not;
the refusal through the reader, to_pandas and to_dataset, and the read through to_pandas where pandas is installed (CI's floors job has none, so that check skips there);
no activity/ written anywhere by default;
the stamp on a result, a spill and a saved sweep;
the refused and the accepted spill resume;
a stamp naming an output from a later version;
a window sweep's stitched and per-window activity, live and archived.

Mutation table, python -m tools.mutate against the full suite, unless marked by hand. Line numbers are at the commit each row was taken on:

guard	result
a bare string refused (`answer_layout.py:63-67`)	caught
a name that is no output refused (`answer_layout.py:69-74`)	caught
a result refuses an output not asked for (`result.py:575-576`)	caught
a result names the frames of an output (`result.py:626-627`)	caught
a result saves its outputs (`result.py:701-703`)	caught
a spill solved with other outputs refused (`sweep.py:179-185`)	caught
a sweep refuses an output not asked for (`sweep.py:374-375`)	caught
the refusal on every sweep reader (`sweep.py:383`)	caught
the refusal on the sweep bridges (`sweep.py:629`)	caught
a slice carries its outputs (`strategy.py:610-612`, re-taken at `3ab29cb`)	caught
`solve` checks outputs before the build (`api.py:543`)	still green at first → probe `test_a_misspelled_output_is_refused_before_anything_is_built` added → caught
`solve_over` checks before a slice (`strategy.py:279`, by hand: `asked = frozenset(outputs)`)	caught (2 failed)

Tree clean after each run. Not tabled:

Engine._output laying out only what was asked for. Laying out more is invisible to every reader, since _outputs decides what is readable.
The slice skipping an output its result left empty (strategy.py:611). With activity alone it is unreachable, because activity travels with the primal. feat(api): a solve or a sweep asked for outputs={'reduced_cost'} reads each variable's reduced cost, with the same sign on every solver #1856 makes it reachable and tables it.

Deliberately not done

dual_ray is still live-only on a Result. Making it an output (saved, and per slice in a sweep) is a follow-up PR.
Reduced costs (#1856) and slack (#1868) are stacked on this as outputs, in their own PRs.

🤖 Generated with Claude Code

https://claude.ai/code/session_01GLBvwp6NU7YjszFz5QMRf1
https://claude.ai/code/session_01UFX6qbCYnXKTVkucgxEqfS

…sks for it, and a sweep can hold the activity too Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GLBvwp6NU7YjszFz5QMRf1

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GLBvwp6NU7YjszFz5QMRf1

read-the-docs-community · 2026-10-06T21:51:09Z

Documentation build overview

📚 specsolve | 🛠️ Build #35014709 | 📁 Comparing b136129 against latest (7af455a)

🔍 Preview build

9 files changed · ± 9 modified

± Modified

codspeed · 2026-10-06T21:56:03Z

Merging this PR will not alter performance

✅ 67 untouched benchmarks
⏩ 307 skipped benchmarks¹

_{Comparing claude/festive-carson-8x46sq-outputs (b136129) with main (1487031)}

307 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GLBvwp6NU7YjszFz5QMRf1

…-8x46sq-outputs # Conflicts: # CHANGELOG.md

…s no activity Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GLBvwp6NU7YjszFz5QMRf1

… is found by its kind's name The engine computes each output asked for in one place instead of threading the activity through the read-back. A sweep slice collects the frames the result already holds rather than re-reading them per name. Result and Sweep find a kind's reader by its name, so a new output adds no dispatch entry. The read-only wrappers around three module tables are dropped. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UFX6qbCYnXKTVkucgxEqfS

FBumann · 2026-10-07T06:37:04Z

Note

The following content was generated by AI.

suite (and with it ci) fails on 909967d: one test still calls Engine._read_back with the activity argument that commit removed.

tests/test_relational.py::TestThePositionalHandoff::test_a_solution_is_read_back_in_label_order_without_sorting_for_it
TypeError: Engine._read_back() takes 4 positional arguments but 5 were given
1 failed, 5086 passed, 564 skipped, 1 xfailed

Fix, and why it is not pushed from this session

In tests/test_relational.py, near line 1242: call model._engine._read_back(primal, dual, None) and unpack primals, duals, _. Then drop the activity assertion, or move it to Engine._output('activity', …), where activity is now laid out.

Not pushed here: another session is pushing to this stack right now (909967d, and a merge into #1856 in progress), so a second push would race it. Separately, the branch conflicts with main in CHANGELOG.md since the 0.6.2 release (#1875).

Generated by Claude Code

…-8x46sq-outputs # Conflicts: # CHANGELOG.md

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UFX6qbCYnXKTVkucgxEqfS

…ow that activity is laid out as an output Activity is laid out by the same per-constraint helper as the duals, so the dual's label-order assertion covers it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UFX6qbCYnXKTVkucgxEqfS

Brings main's polars 2.0, the 0.7.0 release and load_inputs. An answer's stamp keeps the outputs it carries beside ANSWER_LAYOUT, and the collects this branch added go through collected. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014wLrXpeSGDqoBFxyrs8NEz

…s each variable's reduced cost, with the same sign on every solver (#1856) > **Prompt:** Lets work on this as a pr from main …of a plan whose first item was: "Reduced costs computed from duals, with a cross-check test against each solver." A follow-up set scope: "Reduced cost must work for sweeps the same way as it does for regular solves!" It now builds on #1865's `outputs=`. > [!NOTE] > The following content was generated by AI. `outputs={'reduced_cost'}` makes a solve or a sweep carry each variable's reduced cost, computed from the duals, so its sign is `dual`'s on every sink. A result, a sweep, their saves, spills and archives all read it. <details><summary>What it is, and what it refuses</summary> - **The value** is the rate at which the optimal objective rises as the variable's binding bound rises. That is `dual`'s convention moved from a row's right-hand side to a column's bound, so one rule holds for both senses, every comparator, and either side a term is written on. A variable strictly between its bounds reads zero. - **How it is computed**: ∇f(x) − Σ yᵣ∇gᵣ(x), in one numpy pass over the CSR matrix (`readback.reduced_costs`). The pass includes a quadratic objective's gradient and a quadratic row's gradient (Gurobi with `QCPDual`). No sink changes. - **An output under #1865's rule.** `Result.reduced_cost` and `Sweep.reduced_cost`, `kind='reduced_cost'` on the bridges, `reduced_cost/<name>.parquet` in a save, a spill and an archive, and a catalog row per variable. No `LAYOUT` bump: an older answer reads it as not asked for. - **Where it is refused**: wherever `dual` refuses, with the duals' own reason (mixed-integer, a quadratic row without `QCPDual`, a run with no basis), through the reader and the bridges alike. That reason is saved once in `reasons.parquet`. `PRICED = {'dual', 'reduced_cost'}` is the one home of "exists exactly where the duals do", read by `Result._carried` and the sweep readers. It is also refused when not asked for, for a closed result, and with `NoSolutionError` when there are no values. - **Computed only when asked**, so a default solve pays neither the time nor the memory. </details> <details><summary>Verification</summary> Gates: `ruff check .`, `ruff format --check .`, `pyrefly check`, the full suite (`pytest -n auto`, with HiGHS, Gurobi and Xpress installed), `zensical build --strict`, and `test_docs_math.py` + `test_docstring_links.py` with the docs group installed. The review fix (`768ae3e`, `9ebc0a2`) was gated at `6a03f5d` with `ruff check .`, `ruff format --check .`, `pyrefly check` (0 errors), and the full suite with the dev group, Gurobi and Xpress installed: 5151 passed, 566 skipped, 1 xfailed. The docs build was not re-run. `tests/test_reduced_cost.py` has three oracles: - A finite difference: raise the binding bound by 1e-3, solve again, and the objective moves at the reduced cost. Run on every sink, for six ways of writing the balance (min/max × `>=`/`<=` reversed/`==`). - Each solver's own value: HiGHS `col_dual`, Gurobi `RC`, Xpress `getRedCosts`. All three agree with the computed value for both senses, so no sink needs a sign conversion. - A hand-derived optimum. On top of those: quadratic objectives (HiGHS, Gurobi) and quadratic rows (Gurobi `QCPDual`); a save that loads back; an integer answer, live and loaded, giving the duals' reason through the reader and through `to_dataset`; and a two-scenario sweep held, spilled and archived. In that sweep each slice is priced at its own marginal generator, and an integer sweep gives the duals' reason. The `to_dataset` case was a bug found in review: a mixed-integer answer's bridge listed no names and failed unpacking them (`ValueError`). It was reproduced as a strict `xfail` (`768ae3e`), seen failing live and loaded, then fixed by giving the duals' reason where any output is read (`9ebc0a2`). Mutation table, `python -m tools.mutate` against the full suite, unless marked by hand. Line numbers are at the commit each row was taken on: | guard | result | |---|---| | the objective cost term (`readback.py:164`) | **caught** | | the rows weighted by their duals (`readback.py:165`) | **caught** | | the quadratic objective gradient (`readback.py:167`) | **caught** | | the quadratic rows weighted by their duals (`readback.py:168`) | **caught** | | the gradient on the left column of a pair (`readback.py:170`) | **caught** | | the gradient on the right column of a pair (`readback.py:171`) | **caught** | | a priced output gives the duals' reason (`result.py:605-606`, at `9ebc0a2`) | **caught** | | a slice skips an output its result left empty (`strategy.py:611`, at `3fcba30`, by hand: `if laid:` → `if True:`) | **caught** (36 failed) | | a sweep gives the duals' reason for a priced kind (`sweep.py:391`, by hand: `kind in PRICED` → `kind == 'dual'`) | **caught** (3 failed) | Tree clean after each run. </details> <details><summary>Cost, only when asked</summary> Method: a dispatch LP with 200,000 snapshots × 10 generators (2,000,000 columns, 200,000 rows, 2,000,000 nonzeros) on HiGHS. One build, at `28aa7d6`. `Engine._answered` was timed with and without `outputs={'reduced_cost'}`, alternating, median of 7. The solve is the median of 3. Three runs, load average under 1.1 with nothing else running. | | run 1 | run 2 | run 3 | |---|---|---|---| | read-back, no outputs | 0.1 ms | 0.1 ms | 0.1 ms | | read-back, `outputs={'reduced_cost'}` | 54.8 ms | 66.9 ms | 55.5 ms | | HiGHS solve | 20.68 s | 20.27 s | 20.29 s | Asking costs about 0.3% of the solve on this model. Not asking costs nothing. These numbers predate the review simplification, which changes where the computation is called from and not what it computes. They were not re-run. </details> <details><summary>Deliberately not done</summary> - No language construct (`reduced_cost(x)` in a spec). - The name `reduced_cost` was discussed and kept: it is the name HiGHS, Gurobi, Xpress and JuMP use. Its sign is stated in the docstring, because JuMP's convention (relaxing the bound) flips the sign at a lower bound. - Slack is the next output, in its own PR stacked on this one. </details> 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01GLBvwp6NU7YjszFz5QMRf1 https://claude.ai/code/session_01UFX6qbCYnXKTVkucgxEqfS --------- Co-authored-by: Claude <noreply@anthropic.com>

…ar each constraint is from binding (#1868) > **Prompt:** Then, we can continue with slack …the second item of a plan whose text was: "Slack: nearly free. It's right-hand side minus row activity, and you already read activity." > [!NOTE] > The following content was generated by AI. `outputs={'slack'}` makes a solve or a sweep carry each constraint's distance to binding. It is non-negative wherever the row holds, however the row is written, and readable wherever the activity is. <details><summary>What it is</summary> - **The value**: `rhs − lhs` for `<=`, `lhs − rhs` for `>=`, and `−|rhs − lhs|` for `==`, which holds only at zero. A binding row reads 0. A violated one, such as an incumbent inside the solver's tolerance, reads below 0 by how much. - **Why not plain `rhs − activity`**, which the plan named and Gurobi's `Slack` is: that sign depends on which side the model writes a term on. `x >= 1` and `1 <= x` would read opposite signs. The distance to binding reads the same for both, and its sign answers "does the row hold?" on every comparator. For a `<=` row the two agree. - **How it is computed**: from the activity the solver returns and the built model's right-hand sides (`readback.slacks`, one numpy pass over the rows). So it exists wherever the activity does, a mixed-integer incumbent included. No sink changes. - **An output under #1865's rule**, stacked on #1856: `Result.slack`, `Sweep.slack`, `kind='slack'` on the bridges, `slack/<name>.parquet` in a save, a spill and an archive, and a catalog row per constraint. No `LAYOUT` bump. After the stack's review simplification, the engine side is one `case` in `Engine._output`. </details> <details><summary>Verification</summary> Gates: `ruff check .`, `ruff format --check .`, `pyrefly check`, the full suite (`pytest -n auto`, with HiGHS, Gurobi and Xpress installed), and `zensical build --strict` with the docs group installed. After merging the simplified #1856, the branch was gated at `f8489f2` with `ruff check .`, `ruff format --check .`, `pyrefly check` (0 errors), and the full suite with the dev group, Gurobi and Xpress installed: 5168 passed, 566 skipped, 1 xfailed. The docs build was not re-run. `tests/test_slack.py` covers the following: - A hand-derived optimum on every sink, with each row written three ways: variables on the left, variables on the right, constants moved across. All give the same slacks. - An equality off its right-hand side, on either side, reading `−0.5`. - A mixed-integer incumbent, which has a slack. - A save that loads back. - The refusal when not asked for. - A sweep held, spilled and archived. Mutation table: `python -m tools.mutate` for the first row, the rest by hand with its three precautions (a clean tree, `git checkout --` to restore, `__pycache__` dropped on both sides). Full suite each time. | guard | result | |---|---| | the slack is laid out per constraint (`engine.py:324-330`, re-taken at `8129926`) | **caught** | | `<=` reads `rhs − lhs` (by hand: → `lhs − rhs`) | **caught** (10 failed) | | `>=` reads `lhs − rhs` (by hand: → `rhs − lhs`) | **caught** (10 failed) | | `==` reads `−\|rhs − lhs\|` (by hand: → `rhs − lhs`) | **caught** (1 failed, the purpose-built residual test) | Tree clean after each run. </details> <details><summary>Deliberately not done</summary> - No cross-check against each solver's native slack. The activity it is computed from is already the solver's own number, checked by `test_activity.py`, and the natives disagree on sign: Gurobi gives `rhs − lhs` on every comparator. - No `dual_ray` output, which is still live-only. That is next, if wanted. </details> 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01GLBvwp6NU7YjszFz5QMRf1 https://claude.ai/code/session_01UFX6qbCYnXKTVkucgxEqfS --------- Co-authored-by: Claude <noreply@anthropic.com>

…onstraint_basis'} reads the basis it ended on, in the same words on every solver (#1876) > **Prompt:** Add stacked PRs about the new things we discussed The discussion it refers to began with: "What about basis, warmstaer initial solution etc? What kind of things would be reasonable to add to boost sovler performance?" > [!NOTE] > The following content was generated by AI. `outputs={'variable_basis', 'constraint_basis'}` makes a solve or a sweep carry the basis status it ended on. The words are the same on HiGHS, Gurobi and Xpress. This is #1869 part 1, and the warm start stacked on it reads it. <details><summary>What it is</summary> - **One vocabulary** (`BASIS_STATUSES`), read back as an `Enum`: `basic`, `at_lower`, `at_upper`, `fixed`, `superbasic`. A row's bound is its right-hand side, so a binding `<=` row is `at_upper`, a binding `>=` row `at_lower`, and a binding `==` row `fixed`. A column at equal bounds is `fixed`, whichever bound its solver names. - **Two outputs, not one**, because a constraint may share a variable's name (mathspec's flat namespace leaves constraints out). Each output holds one frame per declaration, as every other output does. - **How each member reads its basis**: `_basis()` maps the solver's own codes to the shared ones. It reads only basic or not, and which bound a column is at. `Solver.run(handoff, basis=True)` then reads each nonbasic status against the model (`settled`): a row's bound from its sense, `fixed` from equal bounds. That is also why all three sinks agree, even though Gurobi says only "nonbasic" for a row and Xpress names a row's slack. - **Read only when asked**: the engine passes `basis=True` only where `outputs` names one of the two. - **Where there is no basis** — a mixed-integer model, a quadratic constraint, interior point with crossover off — the reader raises `NO_BASIS`, which names the solver options that give one. A sweep raises it only where some slice solved. Otherwise it gives its "nothing to read" message. - An output under #1865's rule, stacked on #1868: `Result.variable_basis` / `constraint_basis`, the same on `Sweep`, `kind=` on the bridges, `<output>/<name>.parquet` in a save, a spill and an archive, and a catalog row per declaration. No `LAYOUT` bump. </details> <details><summary>Verification</summary> Gates: `ruff check .`, `ruff format --check .`, `pyrefly check`, the full suite (`pytest -n auto`, with HiGHS, Gurobi and Xpress installed), `zensical build --strict`, `test_docstring_links.py` and `test_docs_math.py`. The local runs above were taken while one test of the base, `test_a_solution_is_read_back_in_label_order_without_sorting_for_it`, failed there, so they deselect it. The base has fixed it since, it is merged in, and CI runs the whole suite. `tests/test_basis.py` covers the following: - The vertex of a five-variable, four-row LP, worked out by hand so that every status but `superbasic` appears, read in the same words on every sink. - The `Enum` dtype. - A save that loads back. - The refusal when not asked for, per output. - No basis after a mixed-integer solve and after interior point without crossover, on each sink. - No basis with a quadratic constraint, on Gurobi. - A sweep held, on a process pool, spilled and archived. - A sweep whose every slice ended on no basis. - A basis a member reads short, refused. Mutation table, from `python -m tools.mutate` against the full suite (that one test deselected through `PYTEST_ADDOPTS`), except one row taken by hand with the same three precautions: | guard | result | |---|---| | a basis that does not span the model refused (`base.py:271-272`) | **caught** | | a column at equal bounds reads `fixed` (`base.py:274`) | **caught** | | a nonbasic row reads its bound off its sense (`base.py:277`) | **caught** | | a result that ended on no basis says so (`result.py:666-667`) | **caught** | | a sweep whose slices ended on no basis says so (`sweep.py:733-734`) | **caught**, on a second run; the first was interrupted by the load of another run on the box and proved nothing | | HiGHS: an invalid basis is none (`highs.py:244-245`) | **caught** | | Xpress: no basis after a mixed-integer solve (`xpress.py:140-141`) | **caught** | | Xpress: no basis after barrier without crossover (by hand: the `try`/`except SolverError` removed) | **caught** (1 failed) | The tree was clean after each run. One guard is not in the table: `run` reads a basis only after a run that left a primal. Reading one after a failed run would be harmless and invisible, because the reader refuses on the status first. </details> <details><summary>Deliberately not done</summary> - Gurobi's `_basis` has no guard for a quadratic constraint: Gurobi refuses `VBasis` there itself, which the new test shows. - No `fixed` read from a solver for a row: every row is single-sided, so its nonbasic status follows from its sense. </details> 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01GLBvwp6NU7YjszFz5QMRf1 --------- Co-authored-by: Claude <noreply@anthropic.com>

…is statuses, and a sweep from an earlier sweep or from the slice before it (#1882) > **Prompt:** Review the R stack with 1878. Look for room for simplification, refactoring, better naming, better seams and if better encapsulation. > **Prompt:** Id like to unify the start for lp and mip. But we need to find a simple and intuitive api. Users need to be able to pass basis values easily too! > **Prompt:** I think start on a sweep is prefectly reasonable! And it makes no sense for it to not exist if its availlable on regular solve! > [!NOTE] > The following content was generated by AI. `start=` is one shape for an LP and a mixed-integer model: an earlier answer, or tables keyed by reader (`primal`, `variable_basis`, `constraint_basis`). `solve_over` also takes an earlier `Sweep` or `'previous'`, cut by the axis as a source is. `outputs={'basis'}` asks for the basis. <details><summary>The API</summary> ```python start: Result | Start | None # solve, Model.solve start: Result | Sweep | Start | Literal['previous'] | None # solve_over ``` - **`Start` is a `TypedDict`** in `specsolve.types`: `{'primal': {name: table}, 'variable_basis': {...}, 'constraint_basis': {...}}`. Each table takes every shape a parameter's source takes (#1878), read by the same reader. Basis tables are read as strings and held to `basic`, `at_lower`, `at_upper`, `fixed` and `superbasic`. - **An LP starts from a basis where one is given, else from values. A mixed-integer model starts from values.** An LP answer solved without `outputs={'basis'}` now starts a solve from its primal rather than being refused. - **Values for an LP follow each solver's `lp_values`.** HiGHS uses a complete primal and warns of a partial one. Gurobi warns of either. Xpress warns of a complete one and refuses a partial one, which `loadLPSol` cannot take. A warning asks the user to report a gain at the issue tracker. - **A sweep's start is cut by its axis, as a source is.** A table over the sliced dimension gives each slice its rows: a scenario its scenario's, a window the coordinates it covers, lookahead included, over its local index. A table without it reaches every slice whole. An earlier sweep is its answer, so an archive without kept windows starts a sweep too. Each `Slice` carries the cut it applied to its sources, and the start goes through the same function. - **A slice its start leaves no row starts from nothing, with a warning.** That covers a table short of the slice's key, and an earlier sweep whose slice left no values. A missing start costs time, never the answer, so it no longer refuses the sweep. This changed after review: "For empty tables we should warn, not error". - **`'previous'`** starts each slice from the answer of the one before it, matched by the slice model's own coordinates (local `t` for a window). Each slice is solved with its basis for the next to start from, whether or not `outputs=` asks for it; the answer, the spill and the archive keep only what `outputs=` asks for. The first slice, and a slice after one that left no values, starts cold. Refused under an executor. A slice read back off a spill starts the next from what the spill holds. (#1884 changes this.) - **`start=` combines with `keep='solver'` only.** `keep='progress'` and `keep='nothing'` also say what the solve begins from, so the two together are refused, on `solve` and `solve_over`. - **`outputs={'basis'}`** replaces `{'variable_basis', 'constraint_basis'}`. The readers keep their names, and the files stay under `variable_basis/` and `constraint_basis/`. </details> <details><summary>From the review of #1865–#1878</summary> - The basis vocabulary (`BASIS_STATUSES`, the `BASIS` dtype, the codes) moved from the sinks to `answer_layout`, beside the outputs read back in it. - The engine matches a start to a `Basis` or to values and branches on it, rather than returning a closure. It reads an earlier answer through `Result._start` rather than its private frames. - `Result` and `Sweep` dispatch an output kind explicitly rather than by `getattr`, and the output readers share one path. - Each sink's write-back table names the status it maps rather than listing bare codes. - The spill manifest check compares the key first, then the outputs. - `Result.dual` no longer says slacks are not read. </details> <details><summary>Values for an LP: the probe behind <code>lp_values</code></summary> A random 300×400 LP (5% density, seed 0), solved once cold and once started from its own optimum, outside specsolve. This is the best case for a values start, one run each, and it was not taken on a machine checked for idleness. | | cold | from the full primal | from half the columns | |---|---|---|---| | HiGHS `setSolution` (simplex iterations) | 661 | 0 | 661 | | Gurobi `PStart` | 877 | 877 | — | | Xpress `loadLPSol` | 760 | 760 | refused: needs every column | Separately, on HiGHS, `DISPATCH` at 40 snapshots started from a `variable_basis` alone (rows left at their default) took 40 iterations against 29 cold. The test asserts only that such a start is taken. </details> <details><summary>Mutation tables</summary> `python -m tools.mutate`, scoped to `tests/test_start.py` (and `test_basis`, `test_warm_start`, `test_outputs` for the first group), rather than the full suite. A guard the subset catches, the full suite catches too. | guard | result | |---|---| | a key that names no reader (`sources.py`) | **caught** | | a name its reader has no declaration for (`sources.py`) | **caught** | | a basis status outside the five words (`sources.py`) | **caught** | | `lp_values` refused (`engine.py`) | **caught** | | `lp_values` no_gain warns (`engine.py`, body → `pass`) | **caught** | | nothing the model starts from (`engine.py`) | **caught** | | a start that lands nowhere (`readback.py`) | **caught** | | start beside keep progress or nothing (`result.py`) | **caught** | | a word other than `previous` (`strategy.py`) | **caught** | | `previous` under an executor (`strategy.py`) | **caught** | | a table over the local index alone (`strategy.py`) | **caught** | | the warning for a slice its start leaves no row (`strategy.py:407-413`) | **caught** | | by hand: that slice's empty start passed on rather than none | **caught** (3 failed) | | a slice after one with no values starts cold (`strategy.py`) | **caught** | | a word on one solve (`api.py`, taken before the merge in `engine.py`) | **caught** | | a constraint named as a variable keeps its own basis (by hand: both halves in one directory) | **caught** | | a slice keeps only the outputs asked for (by hand: every output the result holds copied) | **caught** | Three guards first survived, and each got a purpose-built probe before the table was taken: the up-front check on a sweep's start and the up-front windows check, which now spy on the build, and the outputs filter, whose test now looks for a basis in the spill. The tree was clean after every run. </details> <details><summary>Verification</summary> - `ruff check .`, `ruff format --check .`, `pyrefly check`, and the full suite (`pytest -n 4`, with HiGHS, Gurobi and Xpress): 5301 passed, 0 failed, on the tree of the last source change. The collected tests (5867) are the same as at the commit before it. An earlier run on the merge reported 5311 passed; I have not found why the two counts differ. - On ef01170 (the empty-start warning): `pixi run check` gave 5329 passed, 566 skipped, 1 xfailed. `docs-test` gave 112 passed. The new test was a strict `xfail` first, and failed on the refusal. - `pixi run test-bench`: 263 passed. An earlier version of this line said it passed, which was wrong: it had last run before the `keep`/`start=` rule, which refused `bench/warm_payoff.py`'s warm arm (`keep='nothing'` beside `start=`). CI caught it. The warm arm now asks for `keep='solver'` and runs first after each update, where the gained cut loads a fresh solver anyway, and an assertion on `Result.kept` holds both arms to a fresh solver. - `pixi run -e docs docs-test` and `docs-build --strict` pass. - #1878's branch moved while this was open (any-shape values). It is merged in here, not rebased, and its tests now pass their tables under `'primal'`. </details> <details><summary>Deliberately not done, and departures</summary> - **One PR rather than a stack.** The review refactors, the basis output, the unified start and the sweep start all reshape #1876–#1878 before release, and each builds on the one before. - **`keep` is unchanged** apart from the refusal beside `start=`. Whether `start=` replaces `keep='progress'`, and which alignment `'previous'` uses for windows, needs a measurement first: a follow-up. - **No dual values in a start.** Gurobi's `PStart` showed no gain, and duals would need the sign conversion reversed per solver. Measure first. - **The other PRs' changelog lines are unchanged.** #1876's line still reads `outputs={'variable_basis', 'constraint_basis'}`, and its title would need the same edit. </details> 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_014wLrXpeSGDqoBFxyrs8NEz --------- Co-authored-by: Claude <noreply@anthropic.com>

> **Prompt:** Prepare the latest release of specsolve. > > **Prompt:** Im wondering if we bumped the layout stuff multiple times in this release. We should only increment once since the last release! > [!NOTE] > The following content was generated by AI. Releases 0.8.0 on merge. The answer layout rose twice since 0.7.0, and the new inputs layout once after it was added. Each now rises once: `ANSWER_LAYOUT` 3 → 4, `INPUTS_LAYOUT` 1. <details><summary>The layout numbers</summary> `v0.7.0..origin/main` on `archive_layout.py` and `relational/answer_layout.py`: | PR | change | |---|---| | #1883 | `LAYOUT` renamed `ANSWER_LAYOUT` (3); `INPUTS_LAYOUT = 1` added | | #1865 | `ANSWER_LAYOUT` 3 → 4 | | #1904 | `ANSWER_LAYOUT` 4 → 5 | | #1913 | `INPUTS_LAYOUT` 1 → 2 | No release wrote the intermediate numbers, so this PR sets `ANSWER_LAYOUT = 4` and `INPUTS_LAYOUT = 1`. The `format.json` example in `Result.save`'s docstring follows. A 0.7.0 answer stamps 3 and a 0.7.0 archive has no root stamp, so both stay refused. The cost is on unreleased files only. An answer written from `main` between #1865 and #1904 stamps 4 and is no longer refused, though it still carries `model_digest`. An archive written from `main` between #1883 and #1913 stamps inputs 1 and is no longer refused, though it may hold `sources.parquet`. No test asserts a layout number; the refusal tests use 0, 99 and no stamp. </details> <details><summary>Release notes and gates</summary> - `## Upcoming version` → `## 0.8.0 (2026-10-09)`, with a paragraph, a **Breaks** list and the lines grouped under Solving, Reading an answer, Archives, Data and Examples. An empty `## Upcoming version` stays above it. All 21 lines are kept; #1876's line says `outputs={'basis'}`, the spelling #1882 gave it. - **Minor, not patch**: #1884 removes `keep=` and `Result.kept`, #1913 removes the digests, and the layout refuses 0.7.0 files. - `python -m tools.changelog check` → `releases 0.8.0 on merge`. - `pixi run check`: 5523 passed, 566 skipped, 4 xfailed. - Not run: `pixi run docs-test`. - Departure: the layout fix rides in this PR rather than its own, because the release must not ship the doubled numbers. It is its own commit. </details> 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01SUzrD57WeiMZtKUnw7wsG8 --------- Co-authored-by: Claude <noreply@anthropic.com>

claude added 3 commits October 6, 2026 21:20

feat(api): every answer holds an optional output only when outputs= a…

b47778e

…sks for it, and a sweep can hold the activity too Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GLBvwp6NU7YjszFz5QMRf1

test: a misspelled output is refused before a model is built

fd6dade

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GLBvwp6NU7YjszFz5QMRf1

chore: the changelog names optional outputs

958c3e3

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GLBvwp6NU7YjszFz5QMRf1

FBumann force-pushed the claude/festive-carson-8x46sq-outputs branch from 239dfe1 to 958c3e3 Compare October 6, 2026 21:50

FBumann mentioned this pull request Oct 6, 2026

feat(api): a solve or a sweep asked for outputs={'reduced_cost'} reads each variable's reduced cost, with the same sign on every solver #1856

Merged

test: the bridge check on an output skips where pandas is not installed

87a5804

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GLBvwp6NU7YjszFz5QMRf1

FBumann mentioned this pull request Oct 6, 2026

feat(api): a solve or a sweep asked for outputs={'slack'} reads how far each constraint is from binding #1868

Merged

Merge remote-tracking branch 'origin/main' into claude/festive-carson…

8256863

…-8x46sq-outputs # Conflicts: # CHANGELOG.md

This was referenced Oct 6, 2026

A solve reads its basis status by coordinate, and a new process can warm-start from an archived basis #1869

Closed

Tracking: lessons from the MOSDEX and Rima papers #1872

Open

FBumann added this pull request to stack #1874 October 7, 2026 06:01

claude added 3 commits October 7, 2026 06:03

Merge remote-tracking branch 'origin/main' into claude/festive-carson…

0b304e3

…-8x46sq-outputs # Conflicts: # CHANGELOG.md

chore(bench): the read benchmark reads the default answer, which hold…

de61b09

…s no activity Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GLBvwp6NU7YjszFz5QMRf1

claude added 3 commits October 7, 2026 06:40

Merge remote-tracking branch 'origin/main' into claude/festive-carson…

fda5013

…-8x46sq-outputs # Conflicts: # CHANGELOG.md

chore(api): the tables an answer's layout is read from stay read-only

3ab29cb

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UFX6qbCYnXKTVkucgxEqfS

FBumann removed this pull request from stack #1874 October 8, 2026 08:06

FBumann added this pull request to stack #1886 October 8, 2026 08:07

FBumann merged commit f8ea6ce into main Oct 8, 2026
13 checks passed

FBumann deleted the claude/festive-carson-8x46sq-outputs branch October 8, 2026 14:16

FBumann mentioned this pull request Oct 9, 2026

chore: release 0.8.0 #1918

Merged

Navigation Menu

Repository navigation

feat(api): every answer holds an optional output only when outputs= asks for it, and a sweep can hold the activity too - #1865

feat(api): every answer holds an optional output only when outputs= asks for it, and a sweep can hold the activity too#1865
FBumann merged 12 commits into
mainfrom
claude/festive-carson-8x46sq-outputs

FBumann commented Oct 6, 2026 •

edited

Loading

Uh oh!

read-the-docs-community Bot commented Oct 6, 2026 •

edited

Loading

Uh oh!

codspeed Bot commented Oct 6, 2026 •

edited

Loading

Uh oh!

FBumann commented Oct 7, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

FBumann commented Oct 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

read-the-docs-community Bot commented Oct 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Documentation build overview

Uh oh!

codspeed Bot commented Oct 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Footnotes

Uh oh!

FBumann commented Oct 7, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

FBumann commented Oct 6, 2026 •

edited

Loading

read-the-docs-community Bot commented Oct 6, 2026 •

edited

Loading

codspeed Bot commented Oct 6, 2026 •

edited

Loading