Spec.expand() lowers a formulation on request, the typesetter prints the block it was handed, and sos: becomes a formulation #579

Prompt: "Im thinking about lpspec calling mathspec to reformulate sos. So lpspec decides depending on solver, but the logic is in a public method in mathspec."

and, which widened it: "Would this mean a user could also reformulate a piecewise into regular variables/constraints? Would be great for review, teaching and testing"

Settled in discussion, each a line that changed the work: "Should piecewise rather join the typesetting, and only optionally expand?" · "Expand sounds like the better method than reformulate, because it's one directional" · "an expand() method which learns what to expand sounds good as long as there are no options per kind" · "no public ExpandedSpec" · "Is there the possibility that a user needs to provide different sources to a reformulated spec? That would lead to a bad api in lpspec"

What changes

A formulation is lowered when the user asks, and printed as what it states until then. sos: becomes the second formulation.

spec.expand()              # every formulation, as plain variables and constraints
spec.expand('sos')         # one kind; 'piecewise' is the other
typeset(spec)              # a curve prints as a curve, a set as a set
typeset(spec.expand())     # the same model, as the rows a solver holds

expand returns a plain Spec. There is no public expanded type, no per-kind option, and no solver name anywhere in this repository. The engine decides whether; the language decides how.

The design, settled

One verb, expand. After this change sos: is a formulation in limits.md's own sense, so both kinds share the word the reference already uses. reformulate is dropped.

spec.expand(*kinds). No argument expands everything. The kinds are 'piecewise' and 'sos', a closed literal; any other string is refused naming both. The order is fixed rather than chosen: piecewise before sos, because a method: sos2 block emits a set and a set never emits a curve. A spec with nothing left to expand comes back as itself. The signature breaks the day a kind needs an option, so options are refused until something needs one.

The typesetter prints the spec it was handed. Today typeset expands every model (typesetting/__init__.py:90, stated as a rule in docs/reference/typeset.md), and three of the sidecar symbol tables exist only to rename cost_curve_lam to λ. Instead a piecewise: block prints as one line, the way sos: already prints as set membership beside its variable (walk.py:830): the link expressions lie on the curve through the breakpoints, quantified over the block's frame. convex prints the hull, lp the bounded link's side, activity a factor on the statement, points the admitted set. The expanded print needs no flag, since an expanded spec has no block left to print, and typeset stops calling the expander.

No public expanded type. _ExpandedSpec is deleted. The record of which emitted parameters are derived rides in a pydantic private attribute on the returned Spec, the pattern Spec._expansion already uses (model.py:707). Lowering reads it when present. A private attribute survives in process and is dropped by model_dump, so to_yaml refuses on a spec whose record is non-empty, naming typeset as the reader's route. For sos the record is empty and the file round-trips whole.

sos: → binaries, spelled as method: adjacency already spells it. SOS1: a binary per member, x <= M * seg, sum(seg, over=d) <= 1. SOS2: x <= M * (seg + shift(seg, along=d, offset=1, edge=0)), sum(seg, over=d) <= 1. The emitted names follow the piecewise convention exactly: <block>_seg, <block>_pick, <block>_adjacency. Then method: adjacency is defined as sos2 followed by expand('sos'), so the adjacency spelling lives once. The one visible change: the pick row becomes <= 1, which the convexity row makes equivalent to today's == 1.

The coefficient is language, and the rule fixes a divergence. M is the block's bound: when declared, and the member's own upper otherwise, a literal or a parameter name read straight off BoundsBlock. lpspec today takes the tighter of big_m and the numeric upper bound (relational/sinks/sos.py:117), so where big_m is smaller HiGHS solves a different feasible set from gurobi. big_m: is renamed bound:, because it is now a coefficient a reader sees printed beside upper rather than a hint an engine acts on.

Bounds are decided at load, so nothing depends on #471. The rewrite is exact for a member with a non-negative lower bound and a finite coefficient. So: a literal lower of at least zero, and either bound: or a finite upper. Anything else is refused at load, naming bounds: {lower: 0} or bound:. A parameter-valued lower is refused in this cut; #471 later admits it as an emitted assumption.

The invariant that keeps lpspec's API honest

The same sources bind a spec and its expansion. An expanded set adds no parameter: the coefficient is a literal or a parameter the file already declares. An expanded curve emits only parameters it derives (piecewise.py:92), and in process those ride in the private record. Stated as a rule of what a formulation may do, one sentence in limits.md: a formulation emits variables and constraints, and any parameter it emits it derives. Indicator and semi-continuous both pass it.

Pinned without data: for every example, the parameters without a derivation in to_program(spec) equal those in to_program(spec.expand()).

Scope, as three PRs

• A piecewise: block prints as the curve it states. A curve and a hull operator in the format table beside sos_set, a _piecewise line beside _sos, typeset no longer expands, the piecewise sidecar symbol tables go, golden output and the notation page regenerated. Stands alone.
• Spec.expand() public, _ExpandedSpec gone. The derived-parameter record as a private attribute; to_yaml refuses on a non-empty one; tests/test_public_surface.py grows to pin Spec's methods; the sources invariant test.
• sos: is a formulation. The emitter; big_m: → bound:; the load-time bound rule; method: adjacency defined through it; limits.md (sos: stops being "the only declaration that adds neither a column nor a row", and piecewise: stops being the only formulation); the reference's sos section; the name-for-name test of method: sos2 plus expand('sos') against method: adjacency.

Unblocks fluxopt/specsolve#1702, which waits on the third.

Deliberately not proposed: anything solver-aware here; an inexact rewrite; per-kind options on expand; a to_spec(model, expand=…) kwarg, since the front door's contract is the schema as the file declares it; and piping the selector through lpspec's verbs, where lps.solve(spec.expand('sos'), sources) already composes.

Version

v0.0.0-alpha.106

🤖 Generated with Claude Code

https://claude.ai/code/session_013PTRgxu6Ma4vYm5dSrbdEK