feat(spec): a spec reports how its optimum moves with a parameter, which for a fixed decision is a Benders cut

[FBumann]

Prompt: "Then port 749 as a stacked pr onto it and close 748 and 749 as superseded"

Then: "What doesn't work yet? And why?", "Take this loudly on the or description!", and "Add an xfail test!"

Then: "i removed the stack. Retarget both and rebase them."

Note

The following content was generated by AI.

What this changes

spec.sensitivity(*names) reports <name>_sensitivity: the rise of the optimal objective per unit of a parameter, written over dual(). Of a variable that spec.fix made a parameter, it is the slope of a Benders cut. Stacked on #824 (itself on #810), and it supersedes #749.

fix and sensitivity now combine on a masked variable. On #810, fix gives the parameter the variable's missing: absent and adds no where: to a row. spec.fix('cap').sensitivity('cap') on a cap that exists only where flag writes 2.0 + dual(within), and outside flag both cap and within are absent, so a missing rate there is right. The strict expected failure that pinned this is now a plain test.

Formula, provenance, guards, gates, what was not done

The formula

Σ_c dual(c) · ∂(rhs − lhs)/∂q, plus the objective's own ∂f/∂q. It is one expression for every comparator and under either sense, because #751 (on main) defines dual(c) as the rate at which the optimal objective rises with the right side of c. On #824's Benders monolith:

cap_sensitivity  = invest + sum(dual(capacity) * avail, over=snapshot)
load_sensitivity = dual(balance)

#749 checked these against finite differences on HiGHS: cap[wind] 67.5 and cap[gas] 30.0, derived and measured, also with balance as == and under maximize. That check was a scratch script, because mathspec has no solver dependency, and I did not run it again.

What it carries, and what it refuses

• It carries arithmetic, sum(over=) and named expressions from the row down to the parameter, and sums back over each dimension the parameter does not carry. On main, over= takes several dims. sum(p * cost, over=[snapshot, generator]) gives sum(p, over=snapshot) for cost over generator.
• A composed objective works after merge. A term that a fragment adds with adds_to: is read through the named sum.
• It refuses, naming the operator:
  • shift, at, sum(by=), sum_back or a case between the row and the parameter;
  • the parameter as a divisor;
  • a sum over a dim that the factor outside it also carries.
• It refuses a parameter that bounds a variable. That price is a reduced cost, and no expression reads one.
• It refuses a row with a where: that reads the parameter pointwise. Where the mask deletes the row, the dual is absent and would take the whole rate with it, although the parameter may still be read elsewhere. fix no longer writes such a where:, so this refusal now meets only a mask the file wrote itself.

Provenance

This is #749's diff on top of #748, applied to #824, and now rebased onto #824 as stacked on #810. The rebase replayed its three commits without a conflict; one commit on top turns the expected failure into a test. Changed in the first port:

• A given parameter is refused with a message that names it as given and says to ask on the merged spec.
• Docstring cross-references use the full mathspec.spec.Spec path.
• Raises: lists the two refusals it left out: nothing reads the parameter, and the entry is already declared.
• named.md says what the call reports.

Guards, measured before the rebase

Guard deleted	tests/test_sensitivity.py
the left side enters negated	2 failed
a negation flips the seed	2 failed
sum back over the dims the parameter lacks	10 failed
the objective's own rate	9 failed
a bound refuses	1 failed
a masked pointwise row refuses	1 failed
a sum over a carried dim refuses	1 failed
a divisor refuses	1 failed
an entry already declared refuses	1 failed
a given parameter is named as given	1 failed

test_the_rate_of_a_fixed_masked_variable fails without #824's copy of missing:.

Gates

On d3b7d30:

• pixi run lint: pass.
• pixi run test: 2713 passed.
• docs-build: not run on this head. On feat(language)!: missing: says what a missing row means, a table short of a row is refused unless the file says otherwise, and a variable's absence: is now missing: #810 and feat(spec): a decided variable becomes a supplied number in one call, so a subproblem is not a second file #824 the only warning is pathlib.Path, which needs docs.python.org/3/objects.inv; the proxy here refuses it. This PR's docs change is one paragraph in named.md.
• compile-tex: not run. tectonic cannot download its bundle here. The diff changes no typeset output.

Pushed with --force-with-lease, at your instruction, against the AGENTS.md rule.

Not done

• shift, at, sum(by=) and cases on the path. The adjoint of each is an operator the language has (sum(by=) is the adjoint of at), except sum_back, whose adjoint would be sum_forward #49's sum_forward.
• A variable that multiplies the parameter makes the formula read that variable's solved value. This is allowed, and not checked numerically.
• No how-to. The method is in the API reference, and named.md links to it under dual().

Why

A Benders loop needs the cut slope, and a modeller reads the marginal value of capacity or load. Both are this derivative. A driver that writes it by hand has to get the dual's sign right, and #749 measured the old sign giving 112.5 where the true value is 67.5.

[read-the-docs-community]

Documentation build overview

📚 mathspec | 🛠️ Build #34895624 | 📁 Comparing 3e25e16 against latest (34c68fd)

  🔍 Preview build  

39 files changed · ± 39 modified

± Modified

• changelog/index.html
• examples/pypsa/index.html
• examples/pypsa_linearized_uc/index.html
• howto/pin-a-variable/index.html
• reference/api/index.html
• reference/glossary/index.html
• reference/language/index.html
• reference/notation/index.html
• reference/program/index.html
• reference/reading/index.html
• and 29 more...