[FBumann]

Prompt: Let’s implement this new cli module!! As a pr

Prompt (scope): I want full determinism. We will add incremental manifests etc at some later point...

Note

The following content was generated by AI.

A YAML manifest of solve / solve_over arguments, one entry per run, read by sps.load_manifest and by a new specsolve command (check, solve, list) behind a cli extra. Data comes from Excel, parquet and CSV files.

Method, gate output, alternatives

Changes, file by file

• src/specsolve/manifest.py (new): load_manifest(path) -> Manifest; Manifest.runs[name] is a Run with .arguments (exactly the keyword arguments of solve or solve_over) and .solve(**overrides). A closed pydantic schema (pydantic arrives through mathspec, as does pyyaml). The manifest's own keys are exactly manifest, runs, from, archive (a directory: <archive>/<run>), sources (a list of locations) and workers; every other key is an argument name of solve / solve_over, and the axis arguments are the axis classes' own field names. What only solve_over takes is read off the two signatures at import, not listed by hand. A duplicate YAML key is refused (PyYAML would keep the last). Every refusal names the file and the run; an unknown key names the near miss and the keys valid there.
• src/specsolve/cli.py (new): Typer, imported only when the command runs, so main without Typer exits with pip install 'specsolve[cli]' rather than a traceback.
• src/specsolve/__init__.py, types.py, tests/test_architecture.py: load_manifest in PUBLIC_API ("run it from a file"); Manifest and Run in specsolve.types.
• pyproject.toml: extra cli = ["typer>=0.12", "fastexcel>=0.9"]; [project.scripts] specsolve = "specsolve.cli:main"; the dev feature installs the extra beside gurobi and xpress, and the docs feature installs it because the how-to page runs the command.
• examples/manifest/: the how-to's study: specsolve.yaml, two workbooks, a CSV directory and a parquet file, over examples/dispatch.yaml. The workbooks were written once with xlsxwriter through uv run --script; nothing in the tree regenerates them.
• docs/howto/manifest.md (new, How-to nav): runs specsolve check, solve, solve base (the skip) and list for real during the build, then reads an archive and calls Run.solve from Python.
• docs/reference/manifest.md (new, Reference nav): the one home of the keys, inheritance, source locations, the Excel convention, the load-time refusals and the command.
• docs/reference/api.md, docs/reference/glossary.md (Manifest, Run), docs/about/architecture.md (module map, the fifteen unfenced modules, sixteen names), mkdocs.yml.
• tests/test_manifest.py, tests/test_cli.py (new); tests/test_architecture.py: one fence test, parametrized: typer only in cli.py, fastexcel imported nowhere.

Tests

Loading (defaults, run keys replacing them, from, a run's sources replacing what it inherits, a later location replacing a table, paths relative to the manifest from another working directory); arguments equal to a hand-written solve_over call, and the same objectives from both; workers builds a spawn pool that Run.solve shuts down; a workbook reads one table per sheet; 17 load refusals and 3 location refusals parametrized with ids; the fastexcel and typer install hints; the signature test (every key is an argument of solve / solve_over or a manifest-own key; each axis class's constructor takes exactly its fields); CLI through typer.testing.CliRunner on a copy of examples/manifest: solve two runs then skip them, list, check reporting a missing load per run, an unknown run (exit 2), a failed run followed by one that solves, an infeasible run (exit 1). No bug, so no xfail.

Mutation table

mutation	result
from cycle (manifest.py:207-209)	caught
from naming no run (manifest.py:217-221)	caught
duplicate key (manifest.py:246-249)	caught
run name with a separator (manifest.py:288-289)	caught
no spec (manifest.py:291-292)	caught
workers without axis (manifest.py:299-300)	caught
solve_over-only key without axis (manifest.py:301-305)	caught
axis names one known class (manifest.py:329-333)	caught
unknown axis argument (manifest.py:338-339)	caught
missing axis argument (manifest.py:341-342)	caught
one table twice in a directory (manifest.py:367-368)	caught
missing location (manifest.py:370-371)	caught
unknown location form (manifest.py:376-378)	caught
xlsx without fastexcel (manifest.py:387-388)	caught
main without typer (cli.py:37-38, body → pass)	caught
existing archive skipped (cli.py:69-71)	caught
not optimal fails (cli.py:80)	caught
unknown run exits 2 (cli.py:114-116)	caught
unknown key refused (extra='forbid' → 'ignore', by hand)	caught
axis value refused by its class (try/except removed, by hand)	caught

Each row deletes the guard and runs the whole suite (tools.mutate, or by hand with git checkout -- and __pycache__ dropped on both sides). Tree clean after each. Line numbers are those of 79477a4; 6f57835 removed one line from the resolve, which moves the rows below line 222 up by one.

Gates

• pixi run check (lint, format-check, typecheck, test): pass — 5046 passed, 567 skipped, 1 xfailed, on 6f57835 after merging main.
• pixi run -e docs docs-build: pass, "No issues found"; the how-to page's commands ran during the build.
• pixi run -e docs docs-test: 112 passed.

Floors: typer==0.12.0 and fastexcel==0.9.0 with polars==1.30, numpy==1.26, highspy==1.13, mathspec==0.2.1 in a scratch uv venv: tests/test_cli.py and tests/test_manifest.py pass (38). typer 0.12.0 refuses a list[str] | None argument annotation, so the commands annotate list[str].

Not run: pixi run test-floors (the floors environment installs no extras, so the CLI tests skip there and the manifest tests run). pixi lock: this repository has no pixi.lock (pyproject.toml says why), so there was nothing to lock. The changelog line is under ## Upcoming version. LAYOUT: nothing an archive writes changed, so no LAYOUT bump and no interaction with #1835.

Decisions the brief left open

• Default manifest name: specsolve.yaml in the working directory.
• [MANIFEST] [RUN]... is told apart by suffix: the first argument is the manifest where it ends in .yaml or .yml, so specsolve solve base solves the run base.
• Time per run is the command's own wall clock around the whole run (read, build, solve, archive), not a field of the archive: Record holds no duration, and Metrics is not on a Result. Status and objective come from the returned record; a sweep prints its slice count and termination conditions, since a sum of objectives means nothing for a scenario sweep.
• check attaches every slice of a sweep with tidy; what solve_over refuses of the model as a whole (window separability) is reported by solve. Missing files are found when data is read, not at load, so check reports them per run and list works before the data exists.
• Not manifest keys: executor, workers_share_fs, spill_to. workers_share_fs has no meaning for a local pool; spill_to shared as a default would put two sweeps in one directory, which resumes the second from the first's slices when their keys match. All three remain overrides of Run.solve.
• A parquet location stays a path (workers read it themselves); a CSV file and a sheet are read into a DataFrame. A directory reads only .parquet and .csv, and refuses load.csv beside load.parquet.
• The error type is SpecsolveError for manifest refusals (DataError for locations); no new error class.
• polars warns from_arrow(...) will return a Series from inside read_excel where pyarrow is absent; _workbook silences that one warning.

Sources replace, they do not add

A run that sets sources reads exactly those locations. A run that sets none reads its parent's, or the defaults'. This is the same rule as every other key, so a run's line shows all it reads. Within one list, a later location still replaces an earlier table of the same name, which is how high_gas changes cost. The cost is repetition: high_gas and scenarios list the default locations again.

This also unblocks a rolling-horizon run beside plain solves: it no longer receives an inherited table its spec does not declare. A rolling run still needs its own spec over a local index (into='t') and its own data, so the example does not add one. EachWindow is in the reference and the load tests.

Commit 6f57835 changes the resolve, the example manifest, both doc pages and three tests: the inheritance test now asserts replacement and that a run with no sources reads its parent's, and two tests that leaned on adding now list their full sources.

Deliberately not done

Incremental manifests (a run that adds to or drops from what it inherits), --jobs across runs, init, write, zip archives, datarecord sources, nested or matrix runs, export adapters. tests/test_doc_examples.py does not track the two new pages: the manifest YAML is not a spec, and the how-to's Python runs during the build.

🤖 Generated with Claude Code

https://claude.ai/code/session_01GS5ucQvLdLgdP1vaFy7kUS

[codspeed]

Merging this PR will degrade performance by 14.76%

⚡ 4 improved benchmarks
❌ 11 regressed benchmarks
✅ 68 untouched benchmarks
🆕 10 new benchmarks
⏩ 307 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Benchmark	BASE	HEAD	Efficiency
❌	test_window[sector-s-specsolve-highs-shape]	4.1 MB	6.1 MB	-33.55%
❌	test_window[sector-s-specsolve-highs-cold]	4.1 MB	6.2 MB	-33.21%
❌	test_window[sector-s-specsolve-highs-values]	4.2 MB	6.2 MB	-32.41%
❌	test_window[sector-s-specsolve-highs-one]	4.2 MB	6.1 MB	-32.25%
❌	test_emit[sector-s-specsolve-highs]	4.2 MB	6.1 MB	-32.21%
❌	test_window[sector-s-specsolve-highs-coefficient]	4.2 MB	6.1 MB	-31.77%
❌	test_emit[sector-s-specsolve-lp]	4.6 MB	6.1 MB	-24.53%
❌	test_window[transport-s-specsolve-highs-values]	11.3 MB	13.1 MB	-13.71%
❌	test_window[transport-s-specsolve-highs-one]	11.3 MB	13.1 MB	-13.71%
❌	test_window[storage-s-specsolve-highs-values]	9.3 MB	10.5 MB	-11.52%
❌	test_window[storage-s-specsolve-highs-one]	9.3 MB	10.5 MB	-11.52%
⚡	test_window[dispatch-s-specsolve-highs-values]	9.1 MB	6.8 MB	+34.08%
⚡	test_window[dispatch-s-specsolve-highs-one]	9.1 MB	6.8 MB	+34.08%
⚡	test_read[profiled-s-specsolve-parquet]	6.6 MB	5.7 MB	+15.59%
⚡	test_window[fleet-s-specsolve-highs-values]	12 MB	11.3 MB	+6.02%
🆕	test_emit[milp-s-specsolve-highs]	N/A	29.5 MB	N/A
🆕	test_emit[milp-s-specsolve-in-memory-highs]	N/A	29.5 MB	N/A
🆕	test_emit[milp-s-specsolve-in-memory-lp]	N/A	47.7 MB	N/A
🆕	test_emit[milp-s-specsolve-lp]	N/A	47.7 MB	N/A
🆕	test_read[milp-s-specsolve-frames]	N/A	1.5 MB	N/A
...	...	...	...	...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing feat/manifest-cli (c7ac1e6) with main (89306c6)²}

Footnotes

1. 307 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

2. No successful run was found on main (f709000) during the generation of this report, so 89306c6 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[read-the-docs-community]

Documentation build overview

📚 specsolve | 🛠️ Build #35072922 | 📁 Comparing c7ac1e6 against latest (44fe66e)

  🔍 Preview build  

59 files changed · + 3 added · ± 55 modified · - 1 deleted

+ Added

• howto/manifest/index.html
• howto/small-models/index.html
• reference/manifest/index.html

± Modified

• index.html
• about/index.html
• examples/index.html
• sweep/index.html
• tables/index.html
• about/architecture/index.html
• about/benchmarks/index.html
• about/changelog/index.html
• about/linopy/index.html
• about/roadmap/index.html
• and 45 more...

- Deleted

• about/benchmarks-scaling.html

[FBumann]

Prompt: Should start: monsay not just be a path instead? to the archive?

Note

The following content was generated by AI.

solve and solve_over gain start= in the stack ending at #1878 (branch claude/wizardly-lamport-e3g6om sits on top of it). A manifest start: key would map onto the Python forms like this:

start: previous                       # each slice starts from the slice before it
start: {archive: runs/monday.zip}     # an archive anywhere, read with load_archive
start: {run: monday}                  # a run of this manifest, read from <archive>/monday
start: {primal: [data/start/], constraint_basis: [data/basis/]}   # tables, read like sources

Rules

• A plain string is only previous. A reference to another run is written {run: …}, so a run named previous cannot clash.
• A path is always an archive, read with load_archive: a ResultArchive gives its Result, a SweepArchive its Sweep. load_archive refuses anything else with a LayoutError.
• {run: …} resolves against the manifest's archive: key, and can check that the named run runs earlier.
• The mapping keys run, archive, primal, variable_basis and constraint_basis are a closed set. run and archive exclude each other and the table keys.
• Each table location is read as a sources: location is: one file per declaration, named by it. For a sweep, solve_over then cuts each table by the axis, as it cuts a source.

---

Generated by Claude Code