diff --git a/.claude/skills/docs-writing/SKILL.md b/.claude/skills/docs-writing/SKILL.md index d5d20122..d3d6a801 100644 --- a/.claude/skills/docs-writing/SKILL.md +++ b/.claude/skills/docs-writing/SKILL.md @@ -65,7 +65,8 @@ not need, in three groups: - **Building on math-spec** is for someone who writes a tool against `Spec` and `Program`: an engine such as specsolve, a renderer, a checker. - **Contributing** is for someone who changes math-spec itself. -- **Proofs of concept** holds the PyPSA pages. They stay in `docs/examples/`, +- **Proofs of concept** holds the notation page, which renders the typesetting + test model, and the PyPSA pages. The PyPSA pages stay in `docs/examples/`, where `tools/gallery.py` writes them. A page in Development keeps the folder of its kind. diff --git a/README.md b/README.md index 7037afc7..30249386 100644 --- a/README.md +++ b/README.md @@ -28,10 +28,8 @@ math-spec reads that file, checks everything that can be checked without data, and hands the result on: to an engine that builds and solves the model, or to the typesetter that prints it as LaTeX, Typst or Markdown. It builds nothing and solves nothing itself. Every tool reads the file through the same checked syntax -tree, so an engine and a renderer cannot disagree about what the file means; that -is the [test](docs/about/what-counts-as-language.md) for what belongs here. - -Three properties follow: +tree, so an engine and a renderer cannot disagree about what the file means +([what counts as language](docs/about/what-counts-as-language.md)). - **Nothing is guessed.** A misspelled name, a `where` string on an undeclared parameter, a constraint whose dimensions do not match its `dims`: each fails @@ -42,7 +40,8 @@ Three properties follow: composition of them goes in `macros:` ([the limits](docs/about/limits.md)). - **The file is the document.** `to_latex(spec)` prints the model as equations from the file alone, so the math you publish is the math you solve - ([typeset](docs/reference/typeset.md)). + ([typeset](docs/reference/typeset.md)). The file diffs in review, and no + Python state changes what it means. @@ -102,13 +101,10 @@ objective: -That file is a complete model. Nothing outside it changes what it means. - ### The math it prints -Here is that model as math, printed from the file above and nothing else. No -data, no solver, and no second copy of the equations to keep in step. Markdown -is one of three formats, so GitHub renders it here. +The typesetter prints the file above as math, with no data and no solver. +Markdown is one of three formats, and GitHub renders it here. -Each format is one call, and the file is read and checked once: +Each format is one call: ```python import math_spec as ms @@ -273,108 +269,27 @@ ms.to_latex(spec) # amsmath align ms.to_typst(spec) # compiles without a TeX toolchain ``` -Those symbols are the file's own names: `load` prints as $`\mathrm{load}_t`$, -and `capacity` as $`\mathrm{capacity}_g`$. Nothing had to be set up for -that. Pass `symbols='dispatch.symbols.yaml'` and the typesetter prints -$`\ell_t`$ and $`\bar p_g`$ instead, above a legend that defines them. The -first folded block shows it. The table can be a dict, a `SymbolTable`, or a -path to YAML. A key that names nothing in the model is an error, and nothing -in a table changes what the file means. - -Or from a shell, beside `pdflatex` in a Makefile: - -```bash -python -m math_spec latex dispatch.yaml --symbols dispatch.symbols.yaml --standalone -o dispatch.tex -python -m math_spec typst dispatch.yaml --standalone -o dispatch.typ -python -m math_spec markdown dispatch.yaml -``` - -### `Spec` and `Program` - - - -Whatever is wrong with a model is wrong when it loads, not when it solves: - -```python -import math_spec as ms - -spec = ms.to_spec('dispatch.yaml') # schema, names, dimensions, degree: all checked here -sorted(spec.variables) # ['dispatch'] +A [symbol table](docs/reference/typeset.md#symbol-tables) gives the names their +conventional spelling, as in the first folded block. +[Print a model as math](docs/howto/print.md) does the same from a shell. +`to_spec` returns a `Spec`, and `spec.program` the model it builds +([reading a loaded model](docs/reference/reading.md#spec-and-program)). -program = spec.program # names typed, operators resolved to nodes -sorted(program.constraints) # ['power_balance'] -``` +## Documentation -Neither needs data or a solver, so a repository of models compiles in CI with -nothing bound to any of them. **A `Spec` holds the file as written, and a -`Program` holds the model it builds**, with every macro expanded and every curve -kept as the block it is. -[`spec.expand()`](docs/reference/reading.md#formulations-written-out) writes the -curves out as rows. - - - -[Reading a loaded model](docs/reference/reading.md) says what a tool gets -from each, and [the file and the program](docs/about/file-and-program.md) says -why there are two. - -## Why - -- **Declarative math.** A file is readable without knowing any implementation, - and no Python state changes what it means. It diffs in review and travels as a - research artefact. -- **Fail early, fail loud.** Nothing falls back silently, and an error names the - problem and its rewrite. A model that does not load does not print either. -- **One flat namespace, ten rules.** A collision is a load error naming both - declarations, position decides which kinds of name are legal, and a name's kind - is fixed at load. The [ten rules](docs/reference/language/index.md) are one - principle in ten positions. -- **A closed operator set.** `sum`, `sum_back`, `at` and `shift`, with the - arithmetic and `where` grammars. A composition of them goes in `macros:`, so - every engine expands it the same way. -- **A finite language.** An operator joins the language only if each output row - reads a bounded number of input rows, and a file cannot add one. Math the - language cannot express is refused, with the rewrite named. - -## Docs - -Start with [the language](https://math-spec.readthedocs.io/latest/reference/language/): -the ten rules, and the pages that give the exact ones. Then -[every construct as math](https://math-spec.readthedocs.io/latest/reference/notation/), -which prints all of it beside the notation the typesetter gives it, and -[typeset the math](https://math-spec.readthedocs.io/latest/reference/typeset/) -for how to print your own. Why the language is shaped this way, what may enter -it, and who owns a rule once it is in are under -[about](https://math-spec.readthedocs.io/latest/about/limits/). To work on it, -read [CONTRIBUTING.md](CONTRIBUTING.md). +The documentation is at . ## Installation -This project is managed by [pixi](https://pixi.prefix.dev/). To develop against -it: - - - -```bash -git clone https://github.com/energy-models/math-spec -cd math-spec - -pixi run pre-commit-install -pixi run test -``` - - - -Releases are on the alpha stream, and **nothing is published yet**. The publish -job is off until the project leaves it, so `pip install math-spec` is what the -first release will look like, not what today does. Install from a checkout or a -git reference until then; see [RELEASING.md](RELEASING.md). +Nothing is published yet. +[Installation](docs/howto/installation.md) gives the command that installs from +git, and [contributing](docs/contributing.md#setting-up-a-development-environment) +sets up a development clone. ## Prior art Every file under `src/` was written in [specsolve](https://github.com/fluxopt/specsolve) -and extracted here, so that the language and the syntax tree a tool reads it -through are a dependency rather than one engine's internals. The keys themselves, +and extracted here. The keys themselves, which are YAML math, a block per component, `dims:` and a `where:` string, come from [Calliope](https://github.com/calliope-project/calliope). [linopy](https://github.com/PyPSA/linopy) supplies the vocabulary that @@ -387,17 +302,12 @@ Alpha, pre-1.0. -**Breaking changes land without a deprecation cycle.** When a construct is named -wrong, a default is wrong, or a permissive input hides a silent wrong answer, it -is fixed rather than aliased. A compatibility shim for every earlier spelling -would defeat the point of a small language. - -Pin an exact version if you depend on this, and read the +**Breaking changes land without a deprecation cycle.** Pin an exact version if +you depend on this, and read the [changelog](https://github.com/energy-models/math-spec/blob/main/CHANGELOG.md) -before upgrading. What exists is tested: every construct the language has -round-trips through the schema, the parsers and all three typeset formats, and -the LaTeX is compiled rather than eyeballed. It is the accepted YAML that is not -yet frozen, not the behaviour. +before upgrading. Every construct round-trips through the schema, the parsers +and all three typeset formats, and the LaTeX is compiled. The accepted YAML is +not yet frozen. diff --git a/docs/about/file-and-program.md b/docs/about/file-and-program.md index de719059..dda0ffc1 100644 --- a/docs/about/file-and-program.md +++ b/docs/about/file-and-program.md @@ -6,8 +6,8 @@ SPDX-License-Identifier: CC-BY-4.0 # The file and the program This page explains why a loaded model is two objects, and which one each tool -reads. Read it before you write a tool that reads models. You need none of it -to write a model. +reads. You need none of it to write a model. +[Reading a loaded model](../reference/reading.md) is the reference for both. ```text file ── to_spec ──▶ Spec ── .program ──▶ Program @@ -17,29 +17,10 @@ file ── to_spec ──▶ Spec ── .program ──▶ Program ## Two states -**A `Spec` is the file as written.** `to_spec` reads the YAML into a `Spec` -and checks every rule that needs no data. The spec keeps the file's own -spelling: an expression is a string, a bound is a number or a parameter's -name, and a macro is its template. - -**A `Program` is what the file means.** `spec.program` holds every declaration -of the file, section for section, with every name typed and every operator -resolved to a node. The macros are expanded into the trees. A -[`piecewise:`](../reference/language/piecewise.md) block stays one curve, and a -`sos:` block stays one set. Every description is there. - -Lowering builds the program once, while the spec loads. A spec in hand has -already passed every rule, and `spec.program` returns the same object on every -ask. - -| | `Spec` | `Program` | -| ---------------- | ------------------------ | ------------------------------------------- | -| An expression | the text the file wrote | a typed tree of nodes | -| A macro | its template | expanded into every tree that calls it | -| A curve | the block as written | one `PiecewiseDeclaration`, its links typed | -| A set | the block as written | one `SosDeclaration` | -| A description | as written | on each declaration | -| Written back out | `to_yaml()`, `to_dict()` | not at all: trees do not give the text back | +**A `Spec` is the file as written**, checked against every rule that needs no +data. **A `Program` is what the file means**: every name typed, every operator +resolved to a node, every macro expanded, and each `piecewise:` or `sos:` block +kept as one declaration. ## Which tool reads which @@ -50,18 +31,6 @@ ask. | An engine that builds rows | the program of `spec.expand()` | a solver takes rows | | A tool that rewrites files | the `Spec` | only the spec holds the text and the macros | -**The typesetter never reads the spec.** A `Program` handed to it prints the -same as the spec it came from. - -**A program's `footprint`, `separability` and `roots` describe the rows that -program holds.** A curve still on the program is not a row, so it counts once -it is written out. An engine asks these of the program of the expansion, which -is the one it builds. - -**`advice` reads a curve's links as the rows they state.** Its notes are -claims, and a curve that holds a variable keeps that variable out of the -unbounded note. So advice on the spec and advice on its expansion agree. - ## Why the split falls here - **A reader after load needs one typed object.** Printing a model needs the @@ -77,6 +46,3 @@ unbounded note. So advice on the spec and advice on its expansion agree. - **The program does not hold its spec.** Nothing reads the file from a program, and two objects that own each other form a cycle. A tool handed a bare `Program` has the model, not the file. - -[Reading a loaded model](../reference/reading.md) is the reference for both -objects: their fields, the nodes, and the questions a program answers. diff --git a/docs/about/limits.md b/docs/about/limits.md index bf60e285..bee23da3 100644 --- a/docs/about/limits.md +++ b/docs/about/limits.md @@ -17,12 +17,11 @@ costs to add. - **A macro** is a template with arguments, written in the file under `macros:`. Adding one costs nothing: it uses only operators that exist, so no engine has - to change. Most requests turn out to be a macro - ([macros](../reference/language/named.md#macros)). + to change ([macros](../reference/language/named.md#macros)). - **A primitive** is an operator built into the language: `sum`, `sum_back`, - `at`, `shift`, and the `where` comparisons. A file cannot add one. Adding one - here is the expensive kind: every engine that builds models has to implement - it, and the typesetter has to print it in LaTeX, Typst and Markdown. + `at`, `shift`, and the `where` comparisons. Adding one is the expensive kind: + every engine that builds models has to implement it, and the typesetter has + to print it in LaTeX, Typst and Markdown. - **A formulation** is a block that states ordinary variables and constraints rather than being one. `piecewise:` and `sos:` are the two. It costs as much as a primitive to build, but composes as freely as a macro. It emits variables, @@ -39,14 +38,11 @@ instead. **A macro must be able to call it.** Everything a modeller might pass in goes in the value of a keyword argument, such as `over=snapshot`. -**An operator may read the whole table. It pays one full pass over the data.** -`sum(p, over=g)` reads one row per generator, and `shift(p, along=t, offset=1)` -reads the row before. Each reads a bounded number of rows per output row, so an -engine builds the model one chunk of rows at a time. An operator that reads -every row to produce one row costs one full pass before any chunk builds, and a -request for such an operator names that price. - -**An operator that calls itself is refused.** Nothing bounds how far it expands. +**An operator names its price in rows read.** `sum(p, over=g)` reads one row +per generator, and `shift(p, along=t, offset=1)` reads the row before. Each +reads a bounded number of rows per output row, so an engine builds the model +one chunk of rows at a time. An operator that calls itself is refused, because +nothing bounds how far it expands. | The operator | Allowed? | | ------------------------------------------------ | ----------------------------------------------- | @@ -57,11 +53,9 @@ request for such an operator names that price. | reads every row | yes, at one full pass before any chunk builds | | calls itself | no, and the message names what to write instead | -**Degree is not a third test.** `p * q` at one coordinate is a join of a table -with itself, so the objective and the constraints take it. A product of two -sums, `sum(x, over=i) * sum(y, over=j)`, is refused, because the file does not -say how many terms either sum has. `x[i] * y[j] * a[i, j]` is allowed, because -the table `a` says which pairs exist. +Degree is not a test for a new primitive. The +[product rule](../reference/language/expressions.md#where-a-product-of-two-variables-is-allowed) +holds for every operator. A new primitive is finished when lowering builds it, the typesetter prints it in all three formats, and an engine's build of a model that uses it matches @@ -69,43 +63,16 @@ the same model written out by hand. ### Three kinds of refusal -| The language refuses it because… | Examples | Can it change? | -| ------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------ | -| **one solver cannot take it** | indicator constraints; a quadratic constraint. `sos:` was in this group, and entered: a solver with sets takes it as one, and a model for a solver without is written out first | yes, solver by solver | -| **the file would stop being the artifact** | arbitrary Python, whose content no loader can check and no typesetter can print | no | -| **this project puts the work elsewhere** | data preparation such as resampling; helpers for one domain; Python that decides which declarations exist | it could; this project does not want it to | - -Three things never appear inside one model: an `if`, a loop, and a set of -declarations that depends on the data. A dimension computed before the model -loads is fine: a cycle basis for Kirchhoff's voltage law is a graph algorithm -run in data preparation, and its result arrives as a parameter. What no model -can hold is work that needs the solver's answer before it can write the next -row, such as cuts added during a solve. A tool can still loop over models: a -rolling horizon and Benders decomposition each build a model, solve it, and -build the next. - -### Solver capability - -Whether an engine can build the operator is one question. Whether a given -solver then accepts the result is a second one, and the language does not -answer it. If it did, one solver's limits would be written into the language, -and every other solver would inherit them. - -- HiGHS has no special-ordered sets. Gurobi does. An engine handing a model to - Gurobi passes the set through; one handing it to HiGHS refuses it, and the - author writes the set out with `spec.expand('sos')` first. -- A quadratic constraint is accepted by some solvers only when it is convex, - and convexity depends on the numbers, which the file does not have. - -So `sos:` entered the language on the first question alone. Each engine then -decides whether it takes a set, and the language decides what a set is written -out as. +| The language refuses it because… | Examples | Can it change? | +| ------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------ | +| **one solver cannot take it** | indicator constraints; a quadratic constraint ([solver capability](what-counts-as-language.md#what-each-tool-decides-for-itself)) | yes, solver by solver | +| **the file would stop being the artifact** | arbitrary Python, whose content no loader can check and no typesetter can print | no | +| **this project puts the work elsewhere** | data preparation such as resampling; helpers for one domain; Python that decides which declarations exist | it could; this project does not want it to | ## What counts as data preparation -From inside a model, a column you computed in pandas and a column the language -could have derived look the same: a parameter arrives, and a constraint reads -it. One sentence tells them apart: +A column computed in pandas and a column the language could derive look the +same inside a model. One sentence tells them apart: > Data preparation computes what the model cannot know. The language derives what > it can from data the model already has. @@ -124,8 +91,8 @@ it. ## Deliberate non-primitives -What has been asked for and refused, with the reason and what to write instead. -That another tool has a feature is not by itself a reason to add it. +Each row is a request the language refuses, with the reason and what to write +instead. | Request | Why refused | Instead | | ------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | @@ -137,21 +104,5 @@ That another tool has a feature is not by itself a reason to add it. | `**` with a variable in the base or the exponent | the exponent would decide the degree, and `to_spec` reads no data | `x * x` for a square. `**` over parameters and numbers is allowed | | Normalisation, `x / sum(x)` | dividing by a variable is not a polynomial, and no solver takes it | write the ratio as a constraint, or fix the denominator | | An `if`, a loop, or declarations that depend on the data | `to_spec` could no longer read the file without the data | `where:` masks and `dims:` dimensions. A tool may loop over models | -| A Python API for building models | the model is the file you review and diff | YAML, or a `dict` with the same keys ([below](#composition-component-libraries)) | +| A Python API for building models | the model is the file you review and diff | YAML, or a `dict` with the same keys, merged before `to_spec` | | A `where` comparing a relation column against the dimension it maps into | the relation already pairs the two, and a mask over the pair is the same fact in a bigger shape | place the quantity with `sum(by=)`, or read it with `at(by=)` ([operators](../reference/language/operators.md#sum)) | - -## Composition (component libraries) - -A component library is a set of templates, such as a boiler, a battery and a -line, that agree on how ports and flows are named. You merge the templates you -need into one file, wire the components together with a connectivity table in -the data, and close the system with one `sum(by=)` balance. - -The topology is data. Adding a second battery is a row in a table, so the file -grows with the number of component _types_. - -Merging happens before `to_spec`. Every function here takes a `dict` as well as -a path, so a model assembled in Python is checked exactly as a file is, and -`Spec.to_yaml()` writes the file a reviewer reads. A `dict` may hold only what a -file may hold, so the file itself states no composition. A template names no -sibling, and no key says which fragment wins where two declare a `p`. diff --git a/docs/about/what-counts-as-language.md b/docs/about/what-counts-as-language.md index f0f52271..2a407302 100644 --- a/docs/about/what-counts-as-language.md +++ b/docs/about/what-counts-as-language.md @@ -54,11 +54,5 @@ So the boundary runs both ways: one, the rule goes into the language, once. - The language must not state a rule about what one tool can _build_. -A file that every tool accepts can still be a file that one engine cannot -build. Accepting and building are different steps. - -## How this differs from the limits - -[The limits](limits.md) answer a different question: which operators and blocks -may be added to the language at all. This page answers who decides a rule once -the operator or block exists. +[The limits](limits.md) say which operators and blocks may be added to the +language at all. diff --git a/docs/about/what-counts-as-public-api.md b/docs/about/what-counts-as-public-api.md index 28ec779c..6f0c19a0 100644 --- a/docs/about/what-counts-as-public-api.md +++ b/docs/about/what-counts-as-public-api.md @@ -19,13 +19,8 @@ A function may join the public API when both of these hold: on a page of this reference, so somebody could rewrite the function in another language from the pages alone and get the same answer. -## Where a new feature lands - -When the language gained piecewise-linear curves, it gained a `piecewise:` key -in the YAML. A key in the file -shows up in a git diff, the typesetter prints it as math, and an engine written -in another language can read it. So wherever a feature can be a key in the -file, it is one. +Wherever a feature can be a key in the file, it is one: a key shows up in a git +diff, the typesetter prints it, and an engine in another language reads it. ## What every function keeps @@ -35,26 +30,10 @@ file, it is one. - **A value or an error, and nothing between.** `to_spec` either returns a `Spec` or raises an error that names the rewrite. `advice()` is separate: it talks about a file the language accepts, and changes nothing. -- **Safe to call again.** `spec.program` is one object, however often it is - asked for. - **Nothing is written out unasked.** A `piecewise:` or `sos:` block stays the block until a caller calls - [`spec.expand()`](../reference/reading.md#formulations-written-out). An - engine that writes curves out at its own door makes that choice for its - users, not for the language. - -## Three things a function never decides - -- What one solver or file format can take. That is the engine's question. -- How the numbers bind to the names. That is the engine's too. -- Which solver runs. - -## What this refuses + [`spec.expand()`](../reference/reading.md#formulations-written-out). -| Asked for | Why | -| ------------------------------------------------ | ------------------------------------------------------------------- | -| A Python API for building models | The model is the file you review and diff | -| A hook, a callback, a registry, a plugin | Cannot be diffed, printed or read from another language | -| A function that binds data or calls a solver | Needs more than the file | -| A setting that changes what a file means | Two callers would read one file two ways | -| A function whose answer a declaration could give | A declaration can be diffed, printed and read from another language | +What a solver or file format can take, how the numbers bind to the names, and +which solver runs are each engine's to decide +([what counts as language](what-counts-as-language.md#what-each-tool-decides-for-itself)). diff --git a/docs/contributing.md b/docs/contributing.md index 1c39cf1e..be4b4328 100644 --- a/docs/contributing.md +++ b/docs/contributing.md @@ -54,7 +54,7 @@ stale anchor fails it. `pixi run docs-serve` builds the site and serves it at ??? question "I have updated the README.md" The home page includes named sections of the README rather than a copy: the - badges, the model, the development install and the status note. A section + badges, the model and the status note. A section is delimited in the README by `:::md ` and `:::md `, and `docs/index.md` pulls it in with `:::md --8<-- "README.md:name"`. Edit inside the markers, and the site diff --git a/docs/examples/index.md b/docs/examples/index.md index a7828713..d90386e5 100644 --- a/docs/examples/index.md +++ b/docs/examples/index.md @@ -18,5 +18,4 @@ Every model is a file under `examples/` in the repository. The PyPSA parity pages, from [PyPSA in one file](pypsa.md) on, are a proof of concept. They sit in the Development section. -The math on these pages is printed by the typesetter from the file above it. See -[Typeset the math](../reference/typeset.md) to print your own. +[Typeset the math](../reference/typeset.md) prints your own. diff --git a/docs/first-model.md b/docs/first-model.md index a2fbabbe..810e5bd5 100644 --- a/docs/first-model.md +++ b/docs/first-model.md @@ -6,40 +6,12 @@ SPDX-License-Identifier: CC-BY-4.0 # Your first model In this lesson you write a least-cost dispatch model one block at a time, check -it, and print it as math. You finish with the model on the -[home page](index.md) in a file of your own. - -## Installation - -Install math-spec as [installation](howto/installation.md) says. Then run the -command-line interface: - -```bash -python -m math_spec --help -``` - -It prints its four commands: - -```text -usage: python -m math_spec [-h] {check,latex,markdown,typst} ... - -positional arguments: - {check,latex,markdown,typst} - check load a model, and print what the language advises - latex render a model as latex - markdown render a model as markdown - typst render a model as typst - -options: - -h, --help show this help message and exit -``` +it, and print it as math. [Install math-spec](howto/installation.md) first. ## Dimensions Make a file `dispatch.yaml` with a description and two -[dimensions](reference/language/dimensions.md). A dimension is an axis the -model runs over. Here `snapshot` holds the dispatch periods and `generator` -holds the generating units. +[dimensions](reference/language/dimensions.md), the axes the model runs over: ```yaml title="dispatch.yaml" description: Least-cost dispatch of a generator fleet against an hourly load. @@ -55,74 +27,21 @@ Check the file: python -m math_spec check dispatch.yaml ``` -The check accepts the file and prints two lines of advice. Nothing uses the -dimensions yet: +The check accepts the file, and advises that nothing uses the dimensions yet: ```text dimension 'snapshot' is never used: nothing is indexed by it, nothing aggregates into it, and no relation has a column over it. Remove it — or keep it knowingly, if the declarations that use it are still to be written. dimension 'generator' is never used: nothing is indexed by it, nothing aggregates into it, and no relation has a column over it. Remove it — or keep it knowingly, if the declarations that use it are still to be written. ``` -## Parameters - -Add three [parameters](reference/language/declarations.md#parameters). A -parameter is data the model expects. The file gives its name and its -dimensions, and no values. - -```yaml title="dispatch.yaml" hl_lines="7-10" -description: Least-cost dispatch of a generator fleet against an hourly load. - -dimensions: - snapshot: { dtype: int, description: dispatch periods } - generator: { description: generating units } - -parameters: - capacity: { dims: [generator], description: installed capacity } - load: { dims: [snapshot], description: demand to be met } - cost: { dims: [generator], description: marginal cost } -``` - -Print the file as Markdown: - -```bash -python -m math_spec markdown dispatch.yaml -``` - -It prints Markdown: a table of sets and a table of parameters. Rendered, the -output reads: - -!!! example "Rendered output" - - Least-cost dispatch of a generator fleet against an hourly load. - - #### Sets - - | Symbol | Meaning | - |---|---| - | $`\mathcal{T}`$ | index $`t`$ — `snapshot` — dispatch periods | - | $`\mathcal{G}`$ | index $`g`$ — `generator` — generating units | - - #### Parameters +## Parameters and a variable - | Symbol | Meaning | - |---|---| - | $`\mathrm{capacity}`$ | `capacity` over $`\mathcal{G}`$ — installed capacity | - | $`\mathrm{load}`$ | `load` over $`\mathcal{T}`$ — demand to be met | - | $`\mathrm{cost}`$ | `cost` over $`\mathcal{G}`$ — marginal cost | - -## Variable - -Add one [variable](reference/language/declarations.md#variables). A variable is -a decision the solver makes. The [`where:`](reference/language/absence.md) line -leaves out every generator with no capacity. - -```yaml title="dispatch.yaml" hl_lines="12-17" -description: Least-cost dispatch of a generator fleet against an hourly load. - -dimensions: - snapshot: { dtype: int, description: dispatch periods } - generator: { description: generating units } +Add three [parameters](reference/language/declarations.md#parameters), the data +the model expects, and one [variable](reference/language/declarations.md#variables), +the decision the solver makes. The `where:` line leaves out every generator with +no capacity. +```yaml parameters: capacity: { dims: [generator], description: installed capacity } load: { dims: [snapshot], description: demand to be met } @@ -136,15 +55,12 @@ variables: bounds: { lower: 0, upper: capacity } ``` -Print the file again. `--no-legend` leaves out the tables, so only the math -prints: +Print the math. `--no-legend` leaves out the tables of symbols: ```bash python -m math_spec markdown --no-legend dispatch.yaml ``` -The variable prints as its bounds: - !!! example "Rendered output" Least-cost dispatch of a generator fleet against an hourly load. @@ -157,89 +73,13 @@ The variable prints as its bounds: 0 \le \mathit{dispatch}_{t,g} \le \mathrm{capacity}_{g} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} \,:\, \mathrm{capacity}_{g} > 0 ``` -## Constraint +## Constraint and objective -Add one [constraint](reference/language/declarations.md#constraints). A -constraint is a rule the variables obey. This one makes the generators meet the -load in every snapshot. - -```yaml title="dispatch.yaml" hl_lines="19-22" -description: Least-cost dispatch of a generator fleet against an hourly load. - -dimensions: - snapshot: { dtype: int, description: dispatch periods } - generator: { description: generating units } - -parameters: - capacity: { dims: [generator], description: installed capacity } - load: { dims: [snapshot], description: demand to be met } - cost: { dims: [generator], description: marginal cost } - -variables: - dispatch: - description: output of a generator in a snapshot - dims: [snapshot, generator] - where: "capacity > 0" - bounds: { lower: 0, upper: capacity } - -constraints: - power_balance: - dims: [snapshot] - expression: sum(dispatch, over=generator) == load -``` - -Print the math again: - -```bash -python -m math_spec markdown --no-legend dispatch.yaml -``` - -The constraint prints above the bounds: - -!!! example "Rendered output" - - Least-cost dispatch of a generator fleet against an hourly load. - - #### Subject to - - **`power_balance`** - - ```math - \sum_{g \in \mathcal{G}} \mathit{dispatch}_{t,g} = \mathrm{load}_{t} \qquad \forall\, t \in \mathcal{T} - ``` - - #### Variable domains - - **`dispatch`** - - ```math - 0 \le \mathit{dispatch}_{t,g} \le \mathrm{capacity}_{g} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} \,:\, \mathrm{capacity}_{g} > 0 - ``` - -## Objective - -Add the [objective](reference/language/declarations.md#objective). The -objective is the one number the solver minimises. - -```yaml title="dispatch.yaml" hl_lines="24-26" -description: Least-cost dispatch of a generator fleet against an hourly load. - -dimensions: - snapshot: { dtype: int, description: dispatch periods } - generator: { description: generating units } - -parameters: - capacity: { dims: [generator], description: installed capacity } - load: { dims: [snapshot], description: demand to be met } - cost: { dims: [generator], description: marginal cost } - -variables: - dispatch: - description: output of a generator in a snapshot - dims: [snapshot, generator] - where: "capacity > 0" - bounds: { lower: 0, upper: capacity } +Add one [constraint](reference/language/declarations.md#constraints), which +meets the load in every snapshot, and the +[objective](reference/language/declarations.md#objective): +```yaml constraints: power_balance: dims: [snapshot] @@ -250,55 +90,43 @@ objective: expression: sum(dispatch * cost) ``` -The model is complete. Check it: - -```bash -python -m math_spec check dispatch.yaml -``` - -The check prints nothing and exits with status 0. The language accepts the -model. - -## An undeclared name - -Change `load` to `loads` in the constraint: +Check the file again. The check prints nothing and exits with status 0. -```yaml title="dispatch.yaml" hl_lines="22" -description: Least-cost dispatch of a generator fleet against an hourly load. +??? note "The whole file" -dimensions: - snapshot: { dtype: int, description: dispatch periods } - generator: { description: generating units } + ```yaml title="dispatch.yaml" + description: Least-cost dispatch of a generator fleet against an hourly load. -parameters: - capacity: { dims: [generator], description: installed capacity } - load: { dims: [snapshot], description: demand to be met } - cost: { dims: [generator], description: marginal cost } + dimensions: + snapshot: { dtype: int, description: dispatch periods } + generator: { description: generating units } -variables: - dispatch: - description: output of a generator in a snapshot - dims: [snapshot, generator] - where: "capacity > 0" - bounds: { lower: 0, upper: capacity } + parameters: + capacity: { dims: [generator], description: installed capacity } + load: { dims: [snapshot], description: demand to be met } + cost: { dims: [generator], description: marginal cost } -constraints: - power_balance: - dims: [snapshot] - expression: sum(dispatch, over=generator) == loads + variables: + dispatch: + description: output of a generator in a snapshot + dims: [snapshot, generator] + where: "capacity > 0" + bounds: { lower: 0, upper: capacity } -objective: - sense: minimize - expression: sum(dispatch * cost) -``` + constraints: + power_balance: + dims: [snapshot] + expression: sum(dispatch, over=generator) == load -Check the file: + objective: + sense: minimize + expression: sum(dispatch * cost) + ``` -```bash -python -m math_spec check dispatch.yaml -``` +## An undeclared name -The check refuses the file. It prints this message and exits with status 1: +Change `load` to `loads` in the constraint, and check the file. The check +refuses it and exits with status 1: ```text Constraint 'power_balance': 'loads' not found. @@ -307,7 +135,7 @@ Constraint 'power_balance': 'loads' not found. Check for typos, or ensure 'loads' is declared. ``` -Change `loads` back to `load`. The check prints nothing again. +Change `loads` back to `load`. ## The math @@ -317,9 +145,6 @@ Print the whole model: python -m math_spec markdown dispatch.yaml ``` -It prints the description, the tables, the objective, the constraint and the -bounds: - !!! example "Rendered output" Least-cost dispatch of a generator fleet against an hourly load. @@ -372,10 +197,5 @@ bounds: ## Where to next - [The language](reference/language/index.md) gives every rule a file obeys. -- [The glossary](reference/glossary.md) defines each word the pages use in a - fixed sense. - [Examples](examples/index.md) shows larger models beside the math they print. -- [Print a model as math](howto/print.md) prints LaTeX and Typst, and gives - each name its own symbol. -- [Check a model without data](howto/check.md) runs the check over every model - in CI. +- [Print a model as math](howto/print.md) prints LaTeX and Typst. diff --git a/docs/howto/check.md b/docs/howto/check.md index 1cee3616..a4dbc4cc 100644 --- a/docs/howto/check.md +++ b/docs/howto/check.md @@ -21,12 +21,10 @@ machine and in CI. ``` Advice prints on stdout and exits with status 0. A model the language - accepts with nothing to advise prints nothing. A `piecewise:` or `sos:` - block is read as the rows it states, so the answer is the one its - expansion gets, with nothing expanded. + accepts with nothing to advise prints nothing. ```text - Variable 'slack' makes this model unbounded: no constraint names it, and bounds.lower is -inf, which is the direction a +slack term improves a minimize objective in. No data can change that, so the solve would answer `unbounded` and name nothing. + Variable 'slack' makes this model unbounded: no constraint names it, and bounds.lower is open, which is the direction a +slack term improves a minimize objective in. No data can change that, so the solve would answer `unbounded` and name nothing. Give it a finite bounds.lower, or the constraint that was meant to define it. ``` diff --git a/docs/howto/declare-a-column.md b/docs/howto/declare-a-column.md index 12c8c173..8b0fd62a 100644 --- a/docs/howto/declare-a-column.md +++ b/docs/howto/declare-a-column.md @@ -11,16 +11,16 @@ Decide whether a column of your data is a [parameter](../reference/language/declarations.md#parameters). What decides is what the math does with the column. -| The column… | is declared as | because | -| ------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- | -| is an axis: something is indexed by it, or an aggregation lands terms on it | a `dimension` | its members are the coordinate set every table over it is reindexed onto | -| has one value per member of a dimension, or per tuple of several — a generator's bus, a line's two ends, a generator's zone by period | a `relation` with that `key` | it is a map every operator reads, and its values are checked against the dimensions they name | -| relates members of two dimensions many-to-many, with nothing to weigh — which buses a generator may connect to | a bare `relation`, with no `values:` | `sum` reads it with both ends named, and a bare `where` tests it. Nothing reads it, because there is no one value to read | -| relates members of two dimensions many-to-many, with a weight per pair — a link's efficiency to each bus, a cycle's lines | a `parameter` over both | the weight is the data, its row set is the relation, and the aggregation is `sum(w * x, over=a)` | -| is a label set the model only selects on or counts within — a period, a season, a zone | a `dimension`, and a `relation` onto it | its labels are checked, at the cost of one line and one table | -| scales terms — a coefficient, a bound, an offset | a `parameter` (`float` or `int`) | arithmetic is over numbers ([dtype](../reference/language/declarations.md#parameters)) | -| is a per-row attribute the math only selects on — a fuel, a constraint's sense | a `str` parameter | it names rows rather than scaling them, and no set is declared to check its values against | -| is a mask | a `bool` parameter | a bare name in a `where` is its own answer | +| The column… | is declared as | +| ------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------- | +| is an axis: something is indexed by it, or an aggregation lands terms on it | a `dimension` | +| has one value per member of a dimension, or per tuple of several — a generator's bus, a line's two ends, a generator's zone by period | a `relation` with that `key` | +| relates members of two dimensions many-to-many, with nothing to weigh — which buses a generator may connect to | a bare `relation`, with no `values:` | +| relates members of two dimensions many-to-many, with a weight per pair — a link's efficiency to each bus, a cycle's lines | a `parameter` over both | +| is a label set the model only selects on or counts within — a period, a season, a zone | a `dimension`, and a `relation` onto it | +| scales terms — a coefficient, a bound, an offset | a `parameter` (`float` or `int`) | +| is a per-row attribute the math only selects on — a fuel, a constraint's sense | a `str` parameter | +| is a mask | a `bool` parameter | Two rules decide the cases the table does not list: diff --git a/docs/howto/installation.md b/docs/howto/installation.md index 1e743f2d..c68e0d37 100644 --- a/docs/howto/installation.md +++ b/docs/howto/installation.md @@ -5,46 +5,15 @@ SPDX-License-Identifier: CC-BY-4.0 # Installation -!!! warning "Not published yet" +`math-spec` needs Python 3.12 or above. Nothing is published yet, so install it +from git: - math-spec is on the alpha stream, and nothing is published yet. The - commands below are what the first release will look like. Until then, - install from a checkout or a git reference. - -`math-spec` needs Python 3.12 or above. Install it into a dedicated -environment: - -=== "pixi" - - ``` bash - pixi add --pypi math_spec - ``` - -=== "uv" - - ``` bash - uv add math_spec - ``` - -=== "conda" - - ``` bash - conda create -n math-spec "python>=3.12" "pip" - conda activate math-spec - pip install math_spec - ``` - -=== "pip" - - ``` bash - pip install math_spec - ``` - -To develop against a clone instead: - ---8<-- "README.md:docs-install-dev" +```bash +pip install git+https://github.com/energy-models/math-spec +``` -[Contributing](../contributing.md) has the rest. +To develop against a clone instead, follow +[contributing](../contributing.md#setting-up-a-development-environment). ## Editor completion and offline checking diff --git a/docs/howto/print.md b/docs/howto/print.md index d94bc409..f50f02ce 100644 --- a/docs/howto/print.md +++ b/docs/howto/print.md @@ -8,8 +8,7 @@ SPDX-License-Identifier: CC-BY-4.0 Turn a model file into the math a paper would print, from the file alone, and keep the document current as the file changes. -1. **Print Markdown first** and read it. It is the quickest way to see that - the YAML says what you meant: +1. **Print Markdown first** and read it: ```bash python -m math_spec markdown model.yaml @@ -47,9 +46,7 @@ keep the document current as the file changes. `notation: typst` or none. Without `--standalone` the output is a fragment to `\input` or `#include` into a paper. -4. **Print the rows a solver holds** with `--expand`, where the model states a - curve or a set and the reader wants the formulation rather than the - construct: +4. **Print the rows a curve or a set states** with `--expand`: ```bash python -m math_spec markdown model.yaml --expand @@ -65,6 +62,5 @@ keep the document current as the file changes. python -m math_spec latex $< --symbols model.symbols.yaml --standalone -o $@ ``` -The options each renderer takes, what a symbol table may say, and how one -declaration is printed on its own are under -[typeset the math](../reference/typeset.md). +[Typeset the math](../reference/typeset.md) lists every option and what a +symbol table may say. diff --git a/docs/howto/regimes.md b/docs/howto/regimes.md index 538df345..d1108859 100644 --- a/docs/howto/regimes.md +++ b/docs/howto/regimes.md @@ -81,8 +81,7 @@ the recipe needs no second model file. expression: dispatch <= available ``` - The loader proves at load that no two cases can hold at one coordinate, - and `otherwise:` takes every coordinate they leave. + `otherwise:` takes every coordinate the cases leave. 4. **Check it** with `python -m math_spec check model.yaml`. A pair of masks that can both hold, or a case with no `otherwise:`, is refused there with diff --git a/docs/howto/see-an-expansion.md b/docs/howto/see-an-expansion.md index e9904090..c3e98759 100644 --- a/docs/howto/see-an-expansion.md +++ b/docs/howto/see-an-expansion.md @@ -35,10 +35,9 @@ The command line prints the expansion as math rather than as YAML. Pass ## 2. Read a set -The `sos:` block below says that at most one `p` is nonzero. Its expansion -adds one binary per member, a row that picks at most one binary, and a row that -holds an unpicked member at zero. The coefficient `10.0` is the upper bound of -`p`. +Compare the tabs. The `sos:` block below says that at most one `p` is nonzero. +[What a set is written out as](../reference/language/piecewise.md#what-a-set-is-written-out-as) +names each row the expansion adds. @@ -138,21 +137,11 @@ holds an unpicked member at zero. The coefficient `10.0` is the upper bound of -Every name the expansion adds starts with the name of the block, so `pick_seg` -is the binary of the set `pick`. - ## 3. Read a curve -The `piecewise:` block below ties `x` and `y` to a curve through the -breakpoints in `x_bp` and `y_bp`. A `method: sos2` curve states a set, so it -writes out in two steps. Compare the tabs from left to right: - -- **`expand('piecewise')` writes the curve out and leaves its set.** It adds a - weight per breakpoint and one link row per tied variable. An `sos:` block - over the weights keeps at most two neighbouring weights nonzero. -- **`expand()` writes the set out too.** The `sos:` block becomes one binary - per segment and the rows that keep the two nonzero weights next to each - other. +Compare the tabs from left to right. `expand('piecewise')` writes the +`method: sos2` curve below out and leaves the set it states. `expand()` writes +the set out too. @@ -439,9 +428,7 @@ writes out in two steps. Compare the tabs from left to right: -The [`assumptions:`](../reference/language/assumptions.md) rows state what -the curve needs of its data. [`Spec.expand()`](../reference/reading.md#formulations-written-out) lists what -the call accepts. -[Writing a formulation out](../reference/language/piecewise.md#writing-a-formulation-out) +the call accepts, and +[writing a formulation out](../reference/language/piecewise.md#writing-a-formulation-out) says what each block emits. diff --git a/docs/index.md b/docs/index.md index ec317ff7..5edb449f 100644 --- a/docs/index.md +++ b/docs/index.md @@ -188,10 +188,8 @@ call. ## Install it ---8<-- "README.md:docs-install-dev" - -Or as a dependency, once the project leaves the alpha stream. See -[installation](howto/installation.md) for every package manager. +Nothing is published yet. [Installation](howto/installation.md) gives the +command that installs from git. !!! warning "Alpha, pre-1.0" diff --git a/docs/reference/glossary.md b/docs/reference/glossary.md index 0e437a3b..2218a6d9 100644 --- a/docs/reference/glossary.md +++ b/docs/reference/glossary.md @@ -5,119 +5,43 @@ SPDX-License-Identifier: CC-BY-4.0 # Glossary -This page gives the one meaning of each word these docs use in a fixed sense, -and links the page that owns it. The rest hang off one distinction: - -> A **spec** is the file as written. A **program** is what the file means. -> Neither holds a number: the data arrives later, in the tool that builds the -> model. - -```text -model.yaml ── to_spec ──▶ Spec ── .program ──▶ Program ──▶ typesetter, advice, an engine - │ - └── .expand() ──▶ Spec of the rows -``` +This page defines the words these docs use in a fixed sense and that no single +reference page owns. A construct, such as a parameter or a macro, is defined on +its [language page](language/index.md). ## The file and what reads it **Spec** -: The file as written, checked: what `to_spec` returns. It keeps the file's own -spelling, its macros and its descriptions, and writes itself back out with -`to_yaml()` ([the file and the program](../about/file-and-program.md#two-states)). +: The file as written, checked: what `to_spec` returns +([reading a loaded model](reading.md#spec-and-program)). **Program** -: What the file means: `spec.program`. Every name is typed, every macro is -expanded, and every operator is a node. The typesetter and `advice` read it -([reading a loaded model](reading.md#spec-and-program)). +: What the file means, `spec.program`: every name typed, every macro expanded, +every operator a node. **Load** -: What `to_spec` does: parse the file and check every rule that needs no data. -"Refused at load" means `to_spec` raises, before any data exists. +: What `to_spec` does. "Refused at load" means `to_spec` raises, before any +data exists. **Bind** -: What a consumer does when it puts data on a program. "When the data binds" is -the first moment a rule about numbers can be checked, and the language checks -none of them itself. +: What a consumer does when it puts data on a program. A rule about numbers can +be checked only then, and the language checks none itself. **Consumer** : A tool that reads a spec: an **engine** that binds data and builds the rows a -solver takes, a **renderer** such as the typesetter, or a **checker** in CI. -A consumer may refuse a model for a reason of its own, and may not give the -file a second meaning +solver takes, a **renderer** such as the typesetter, or a **checker** ([what counts as language](../about/what-counts-as-language.md)). -**Typesetter** -: The part of this package that prints a program as math: `to_latex`, -`to_typst` and `to_markdown` ([typeset the math](typeset.md)). - -**Symbol table** -: A mapping from each name and dimension in the file to the symbol it prints -as. With none, the symbols are **derived** from the names -([symbol tables](typeset.md#symbol-tables)). - -**Legend** -: The table of sets, parameters, variables and definitions that the typesetter -prints above the math ([options](typeset.md#options)). - -## Declarations - **Declaration** -: One named entry under one of the eleven top-level keys: one dimension, one -parameter, one constraint. The objective is the one declaration with no name -([file shape](language/file.md)). +: One named entry under one of the top-level keys: one dimension, one +parameter, one constraint ([file shape](language/file.md)). -**Dimension** -: An axis of the model, such as `snapshot` or `generator`. Declarations are -indexed by it, and `sum` reduces over it. The docs also say _axis_ for it, -and `dims` is the key that lists them ([dimensions](language/dimensions.md)). +## Coordinates **Label** : One member of a dimension, `wind` say. The labels arrive with the data, in the order that `shift`, `sum_back` and `position()` count along. -**Relation** -: A table that maps one dimension onto another: a generator's bus, a -snapshot's period. Its **key** is the columns unique per row, and its -**values** are what the key determines. A **bare relation** has no values, -so it may be many-to-many ([relations](language/relations.md)). - -**Parameter** -: A name for data the model reads, with its dimensions and its `dtype`. It -declares a shape and nothing more. A `bool` parameter is a mask, and a `str` -parameter is a label; neither may stand in arithmetic -([parameters](language/declarations.md#parameters)). - -**Variable** -: What the solver decides: one column per coordinate of its `dims`. Its -`domain` is `continuous`, `integer` or `binary`. It is unbounded on each side -the file does not bound ([variables](language/declarations.md#variables)). - -**Constraint** -: One rule, built as one row per coordinate of its `dims` -([constraints](language/declarations.md#constraints)). - -**Named expression** -: A quantity the file names once, under `expressions:`. The math may read it, -and a solve may report it ([named expressions](language/named.md)). - -**Cases** -: A named expression that takes a different body in each region of its frame. -Each **case** has a `when:` mask that claims coordinates, and `otherwise:` -holds the value at the rest. No two cases may claim one coordinate -([cases](language/named.md#cases)). - -**Macro** -: A template with arguments, under `macros:`. It is substituted into each -expression that calls it before anything reads the expression. Its arguments -are its **formals** ([macros](language/named.md#macros)). - -**Assumption** -: A fact the data has to meet, written as a predicate under `assumptions:`. The -language types it and prints it; a consumer that has the data checks it -([assumptions](language/assumptions.md)). - -## Coordinates and rows - **Coordinate** : One point of a declaration's dimensions: one generator in one snapshot. A variable has one column at each coordinate it is built at, and a constraint @@ -128,118 +52,38 @@ has one row. must fit inside the frame they sit in ([how dimensions combine](language/expressions.md#how-dimensions-combine)). -**Dimension set** -: The dimensions an expression carries. `a + b` carries those of `a` and `b` -together, and `sum(x, over=d)` carries those of `x` less `d`. - -**Scalar** -: A declaration or an expression with no dimensions, `dims: []`. The objective -is scalar. - -**Degree** -: How many variables multiply together in one term. The objective and the -constraints stop at 2, and everything beside them stays at 1 -([where a product of two variables is allowed](language/expressions.md#where-a-product-of-two-variables-is-allowed)). - **Group** : The labels that one value of a relation column collects. `within=` keeps a `shift`, a `sum_back` or a `position()` inside each group. ## Masks and absence -**Where** -: A predicate on a declaration that says which of its coordinates exist. Its -grammar is the [where grammar](language/expressions.md#where-strings). - -**Mask** -: A `where` once the program holds it, and the coordinates it admits. A `bool` -parameter is a mask on its own ([nodes and masks](reading.md#nodes-and-masks)). - -**Predicate** -: A true-or-false expression in the where grammar: the body of a `where:`, a -case's `when:`, or an assumption's `holds:`. +**Mask** · **predicate** +: A predicate is a true-or-false expression in the +[where grammar](language/expressions.md#where-strings). A mask is a predicate +on a declaration, and the coordinates it admits. **Absence** -: No value at a coordinate: a variable masked out has no column there, and a -row that reads it is not built. Inside a `sum` an absent term is one term fewer -([absence](language/absence.md)). The `absence:` key on a variable chooses -between this reading, `undefined`, and `zero` -([what a missing coordinate means](language/absence.md#what-a-missing-coordinate-means)). +: No value at a coordinate. A masked-out variable has no column there, and a +row that reads it is not built ([absence](language/absence.md)). **Missing row** : A coordinate that a parameter's table has no row for. It is not absence: it reads as `0` in arithmetic and as false in a `where` ([what creates absence](language/absence.md#what-creates-absence)). -**Edge** -: The coordinates that a `shift` or a `sum_back` reaches past the start of its -dimension. Without `edge=`, a `shift` leaves the vacated coordinate absent, -and a `sum_back` window stops short ([`shift`](language/operators.md#shift)). - -## Operators - -**Operator** -: One of `sum`, `sum_back`, `at` and `shift`, plus `dual` in a reported -expression. The set is closed: a file cannot add one -([operators](language/operators.md)). +## Kinds of construct **Primitive** -: A construct built into the language, which every engine has to implement and -the typesetter has to print: the operators and the `where` comparisons. A -request for a new construct is a macro, a primitive or a formulation, or it is -refused ([how a new construct enters](../about/limits.md#how-a-new-construct-enters)). - -**Consumed** · **produced** -: The relation columns that `sum(by=)` and `at(by=)` take away (`over=`) and put -in their place (`into=`) -([how a relation is used](language/relations.md#how-a-relation-is-used)). - -**In the math** · **reported** -: A named expression is in the math when the objective, a constraint or a -`piecewise:` link reaches it. Otherwise it is reported: a solve computes it -from the solution, and no degree limit applies -([reported expressions](language/named.md#reported-expressions)). - -**Row dual** -: `dual(c)`: the shadow price a solve puts on each row of constraint `c`. Only -a reported expression may read one -([reading a constraint's dual](language/named.md#reading-a-constraints-dual)). - -## Formulations +: A construct built into the language, which every engine implements and the +typesetter prints: the operators and the `where` comparisons. **Formulation** -: A block that states ordinary variables and constraints rather than being one. -`piecewise:` and `sos:` are the two -([piecewise curves and SOS](language/piecewise.md)). - -**Curve** -: A `piecewise:` entry: two or more expressions tied to one piecewise-linear -curve. Its **breakpoints** are the corners, one per label of the dimension -named by `over:`. Each **link** pairs an expression with the parameter that -holds its breakpoint values. `method:` says how the curve is written out -([`piecewise`](language/piecewise.md#piecewise)). - -**Set** -: An `sos:` entry, a special-ordered set: of the members of a variable along -one dimension, at most one (`type: 1`) or two neighbours (`type: 2`) may be -non-zero ([`sos`](language/piecewise.md#sos)). - -**Expand** -: Write each formulation out as the variables and constraints it states. -`spec.expand('piecewise')` writes the curves out and `spec.expand()` writes -the sets out too. Each returns a new spec, and nothing expands a model unasked -([writing a formulation out](language/piecewise.md#writing-a-formulation-out)). - -## Checks and refusals - -**Load error** -: An exception `to_spec` raises. Each is a `MathSpecError`, and the message -names the rewrite ([which error you get](language/errors.md#which-error-you-get)). +: A block that states ordinary variables and constraints rather than being +one: `piecewise:` and `sos:` ([piecewise curves and SOS](language/piecewise.md)). -**Advice** -: A warning about a file that loads: a dimension nothing uses, or a variable -the objective pushes towards a bound it does not have -([what `advice` warns about](language/errors.md#what-advice-warns-about)). +A request for a new construct is a macro, a primitive or a formulation, or it +is refused ([how a new construct enters](../about/limits.md#how-a-new-construct-enters)). ## Words with two senses diff --git a/docs/reference/language/absence.md b/docs/reference/language/absence.md index 196539eb..f4d1a755 100644 --- a/docs/reference/language/absence.md +++ b/docs/reference/language/absence.md @@ -6,8 +6,7 @@ SPDX-License-Identifier: CC-BY-4.0 # Absence and `where` A `where:` does not set a variable to zero. It leaves the variable **unbuilt** -at the masked coordinates: no column, and no value. Every rule on this page -follows from that. +at the masked coordinates: no column, and no value. ```yaml dimensions: @@ -23,8 +22,7 @@ variables: With `capacity = {wind: 10, gas: 5, old: 0}`, the model has `dispatch[wind]` and `dispatch[gas]`. There is no `dispatch[old]`. -The [grammar](expressions.md#where-strings) says what a `where:` may contain. -This page says what the mask means for the rows that are built. +The [grammar](expressions.md#where-strings) says what a `where:` may hold. ## What creates absence @@ -41,7 +39,8 @@ in a `where`. Where no such value exists, loading is refused. There are four such positions: a divisor, a `bounds:` entry, the whole constant side of a comparison, and a -[`piecewise:`](piecewise.md) breakpoint. +[`piecewise:`](piecewise.md) breakpoint. For a bound only where the data has +one, supply the bound in the data, where `inf` is a value, or mask the variable. ## How absence travels @@ -64,9 +63,8 @@ constraints: expression: sum(x, over=g) + sum(y, over=g) >= 1 # x[old] is back in ``` -`each` has no row at `old`. `total` sums the summand wherever the summand -exists, so `x[old]` goes away with `y[old]`. `split` sums each operand over its -own domain, so `x[old]` counts. The two are different constraints. +`total` sums the summand wherever the summand exists. `split` sums each +operand over its own domain. The two are different constraints. Beside a parameter, the rule reads the other way: @@ -91,12 +89,9 @@ there instead, write `where: rel_max` on the constraint. ## What a missing coordinate means -By default a masked coordinate has **no value**. A store that is not there has no -state of charge, so a row that needs that state is not built. - -Some quantities are **zero** outside their mask. A reservoir with no inflow spills -nothing, and a model like that wants its row. The variable says which reading -applies: +By default a masked coordinate has **no value**, and a row that needs it is not +built. Some quantities are **zero** outside their mask, and the variable says +which reading applies: ```yaml variables: @@ -121,26 +116,12 @@ a storage with inflow and no store, there is no row. ## Rows with no variable terms A missing parameter row can leave a row with nothing to decide, such as -`0 == load` at a bus with no generator. Such a row is not built, and the engine -reports it. An expression that names no variable _in the file_ is refused at +`0 == load` at a bus with no generator. Such a row is not built. An expression that names no variable _in the file_ is refused at load. ## Reported values -A [reported expression](named.md#reported-expressions) is arithmetic over -solved numbers, and it inherits their absence by the rule above. Through -arithmetic, a null spreads: `cost / delivered` has no value wherever either -operand is masked. Out of a summing operator, it does not. - -A quotient whose divisor solved to zero is absent in the same way. `dual(c)` has -no value at a row that `c`'s `where:` leaves unbuilt. - -## Asking for the opposite reading - -| You want | You write | -| ---------------------------------------------- | -------------------------------------------------------------------------------------------- | -| the row kept, the masked variable read as zero | `absence: zero` on the variable | -| the row dropped where a parameter has no data | `where: capacity` on the constraint | -| a vacated shift position to contribute | `shift(x, along=d, offset=n, edge=0)` | -| to test whether a variable exists here | its bare name in a `where` | -| a bound only where the data has one | supply the bound, because `inf` is a value, or mask the variable. These are different models | +A [reported expression](named.md#reported-expressions) inherits the absence of +the solved numbers it reads, by the rules above. A quotient whose divisor solved +to zero is absent too. A deleted row has +[no dual](named.md#reading-a-constraints-dual). diff --git a/docs/reference/language/assumptions.md b/docs/reference/language/assumptions.md index 23e84655..78712e9d 100644 --- a/docs/reference/language/assumptions.md +++ b/docs/reference/language/assumptions.md @@ -6,10 +6,8 @@ SPDX-License-Identifier: CC-BY-4.0 # Assumptions `assumptions:` states what the model expects of the data it is bound to. The -language reads no data, so it checks nothing here. It types the predicate, -carries it on the program, and prints it in the -[typeset document](../typeset.md). The consumer that binds the numbers runs -each one, and refuses the data that fails it. +language types each predicate and prints it in the +[typeset document](../typeset.md). The consumer that binds the numbers runs it. ```yaml dimensions: @@ -48,9 +46,6 @@ predicate. `bounds_do_not_cross: "p_min <= p_max"` above is the short form of `bounds_do_not_cross: { holds: "p_min <= p_max" }`. -A `description:` says why the rule is there. The sentence a consumer refuses -with quotes it, so a failure names the columns and the reason. - There is no `dims:`. The predicate holds at every coordinate of the product of the dimensions its two masks name. A predicate narrower than that broadcasts, as it does in any `where`. @@ -90,14 +85,12 @@ assumptions: description: the first snapshot has no predecessor to ramp from ``` -A `where:` narrows which coordinates are checked. A parameter supplied only -where it applies takes one, so the rows it has no value at are not held to the -predicate. +A parameter supplied only where it applies takes a `where:`, so the rows it has +no value at are not checked. ## What the loader refuses -**A predicate the connectives already decide.** It reads no data, so it is -either a claim about nothing or a claim no data can meet: +**A predicate the connectives already decide:** > `Assumption 'sound'`: the predicate `'c > 0 OR true'` folds to true, so it > assumes nothing of the data. Delete it, or name a parameter it constrains. @@ -105,27 +98,28 @@ either a claim about nothing or a claim no data can meet: A `where:` the connectives decide is refused the same way: one that folds to true narrows nothing, and one that folds to false checks the entry on no row. -**A variable.** An assumption is about the numbers the caller binds, and a -variable is what the solver decides from them: +**A variable:** > `Assumption 'sound'`: variable `'p'` stands in what the assumption assumes, > and an assumption is about the data — a variable is what the solver decides > from it. Name a parameter, or state the rule as a constraint. -A rule that binds a decision is a [constraint](declarations.md#constraints). A constraint whose sides carry no variable is refused, and its message names this section. ## What a curve assumes -A [`piecewise:`](piecewise.md) block puts its own conditions on the numbers. -Its breakpoints increase along the curve, and the shape is the one its -`method:` is exact for. The language derives both from the method and the sign -on its links, not from anything else the file writes, and carries them beside -the written ones under the name a refusal quotes. A `method: convex` block -called `curve` adds `curve_increasing` and `curve_curvature`. +A [`piecewise:`](piecewise.md) block `curve` adds its own assumptions, derived +from its `method:`, its `points:` and the sign on its links. They print under +the same _Assumptions_ heading as the written ones. + +| Entry | Added for | Holds | +| ------------------- | ----------------- | ----------------------------------------------------------------------------------------------------- | +| `curve_complete` | every block | every values parameter has a row at every breakpoint the curve runs through | +| `curve_increasing` | `convex`, `lp` | the pinned link's breakpoints (the first link's, when both are pinned) strictly increase along `over` | +| `curve_curvature` | `convex`, `lp` | with a `>=` link the curve is convex, with `<=` concave; with both links pinned it bends one way only | +| `curve_breakpoints` | `lp` | each curve has at least two breakpoints | +| `curve_contiguous` | a block `points:` | the marked breakpoints are one consecutive run of at least one | -Both kinds print under one _Assumptions_ heading, because a reader checking -the data against the document checks all of them. [Reading a loaded model](../reading.md#what-the-data-has-to-satisfy) says how a consumer runs them. diff --git a/docs/reference/language/declarations.md b/docs/reference/language/declarations.md index 4414b405..bcf388d5 100644 --- a/docs/reference/language/declarations.md +++ b/docs/reference/language/declarations.md @@ -10,8 +10,7 @@ text that the [typeset](../typeset.md#descriptions) legend prints. ## `parameters` -A parameter declares a shape and nothing more. The engine that builds the model -supplies the numbers, by name, from its own tables. +A parameter declares a shape. The numbers arrive by name with the data. ```yaml dimensions: @@ -73,17 +72,11 @@ variables: | `absence` | `undefined` or `zero`: what a masked-out coordinate means ([absence](absence.md#what-a-missing-coordinate-means)) | default `undefined` | | `description` | free text | default `null` | -!!! warning "A bound you omit leaves the variable unbounded on that side" - - You write non-negativity. The language does not assume it. - -An open side is `null`, as every other field a file may leave open is. A bound -is never infinite: `.inf` and `-.inf` are refused, with `null` named as the -rewrite. +An open side is `null`. A bound is never infinite: `.inf` and `-.inf` are +refused, with `null` named as the rewrite. A bound is a name or a number: `upper: capacity` is accepted, -and `upper: -rating` is refused. Ship the negated column as data. The dimensions of -a bound parameter are a subset of the variable's. +and `upper: -rating` is refused. Ship the negated column as data. Equal bounds pin a variable ([fix a quantity](../../howto/pin-a-variable.md)). A pinned variable is still a variable. @@ -119,8 +112,7 @@ The dimensions of the expression must **equal** its `dims` At least one side of the comparator carries a variable. A comparison between numbers and parameters alone is refused at load. -`dims: []` gives one scalar row, for a rule such as a system-wide budget. A -scalar variable may not carry a `where`; put the condition on the constraints +`dims: []` gives one scalar row. A scalar variable may not carry a `where`; put the condition on the constraints that use it. Two regimes of one rule are two blocks, each under its own `where:` diff --git a/docs/reference/language/errors.md b/docs/reference/language/errors.md index e13b0411..65d41499 100644 --- a/docs/reference/language/errors.md +++ b/docs/reference/language/errors.md @@ -10,12 +10,8 @@ SPDX-License-Identifier: CC-BY-4.0 `to_spec` binds no data. Before it returns a `Spec`, it parses the file, resolves every name, checks every dimension rule and every degree, and reads every `where` string and every macro template, including the templates that -nothing calls. A `piecewise:` block is checked as written, against every rule -its expansion would be held to, and stays a block. - -Anything the language refuses is refused there, so a repository of models -validates in CI with no data and no solver. An array that does not bind, or a -solver exception, comes from the tool that builds and solves the model. +nothing calls. A `piecewise:` block is checked against every rule its expansion +would be held to. Anything the language refuses is refused there. Every message names what went wrong and what to do about it: @@ -39,7 +35,7 @@ loads. ```text Variable 'slack' makes this model unbounded: no constraint names it, and -bounds.lower is -inf, which is the direction a +slack term improves a minimize +bounds.lower is open, which is the direction a +slack term improves a minimize objective in. No data can change that, so the solve would answer `unbounded` and name nothing. Give it a finite bounds.lower, or the constraint that was meant to define it. @@ -51,34 +47,16 @@ variable with no constraint row. ## Which error you get -| | | -| ---------------- | -------------------------------------------------------------------------------------------------------------------------------- | -| `MathSpecError` | The root. Everything below is an instance of it | -| `LanguageError` | Something in the model: a construct outside the language, a dimension set that does not compose, or a name that nothing declares | -| `SchemaError` | Something in the file: an unknown key, a malformed declaration, or a bad symbol table | -| `DimensionError` | Dimensions that disagree, such as a constraint whose expression does not equal its `dims` | +| Class | Subclass of | | +| ---------------- | --------------- | -------------------------------------------------------------------------------------------------------------------------------- | +| `MathSpecError` | `ValueError` | The root | +| `LanguageError` | `MathSpecError` | Something in the model: a construct outside the language, a dimension set that does not compose, or a name that nothing declares | +| `SchemaError` | `LanguageError` | Something in the file: an unknown key, a malformed declaration, or a bad symbol table | +| `DimensionError` | `LanguageError` | Dimensions that disagree, such as a constraint whose expression does not equal its `dims` | -Every one of these is reproducible from the YAML alone. An engine that binds -numbers or calls a solver adds its own errors below `MathSpecError`. +Every one of these is reproducible from the YAML alone. ## What the language will not express -Each of these was asked for and refused, and [the limits](../../about/limits.md) -gives the reasons. - -| Not in the language | Instead | -| ------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `variable * variable` in a bound or a `piecewise:` link | The objective and the constraints take it. Everywhere else, use a parameter coefficient ([expressions](expressions.md#where-a-product-of-two-variables-is-allowed)) | -| `sum(x, over=d) * sum(y, over=d)` | Multiply before you reduce, or constrain a variable to equal the reduction | -| degree 3 (`x * y * z`) | A variable constrained to equal one product, multiplied by the third | -| `**` with a variable in it | `x * x` for a square. Over variable-free operands `**` is in the language | -| arithmetic in `bounds:` | A name or a number. Ship the derived column as data | -| time-series processing (resample, cluster, interpolate, align), file IO, units | Data preparation. Pass a parameter | -| indicator constraints | `sos:` is where that landed ([piecewise](piecewise.md#sos)) | -| multi-objective | There is one `objective:` block. Weight the goals into one expression | -| arbitrary array operations (`merge`, `reindex`, `apply_ufunc`) | Data preparation | -| filling a missing value (`.fillna`) | Data preparation, or a `where` if the coordinate should not exist. Inside the language, only `shift(..., edge=)` fills ([absence](absence.md)) | - -The language has no escape hatch. Math it cannot express is a gap in the -language, and a gap closes as a macro, a primitive or a formulation -([the limits](../../about/limits.md)). +What the language refuses, and what to write instead, is in +[the limits](../../about/limits.md#deliberate-non-primitives). diff --git a/docs/reference/language/expressions.md b/docs/reference/language/expressions.md index 61d03a99..f32cebe5 100644 --- a/docs/reference/language/expressions.md +++ b/docs/reference/language/expressions.md @@ -23,12 +23,11 @@ NUMBER ::= integer | float | "inf" | ".inf" - Operators bind in this order, highest first: `**`, then unary `+` and `-`, then `*` and `/`, then binary `+` and `-`. So `-x ** 2` is `-(x ** 2)`, as in - Python. Parentheses override precedence. + Python. - A float may carry an exponent, as in `1e5` or `2.5e-3`. - The same keyword twice in one call is an error. -- An expression nests at most 100 levels deep, and so does a `where:` string. - With every named expression it reads written in, an expression nests at most - 300 levels deep. +- An expression and a `where:` string nest at most 100 levels deep, and at most + 300 with every named expression they read written in. ## Where a product of two variables is allowed @@ -44,9 +43,7 @@ bound it: - **Everything beside the math stays affine.** A bound is one number per column, and a `piecewise:` link is affine. -A [named expression](named.md) is held to the limit of the place that reads -it. One that nothing in the math reads is [reported](named.md#reported-expressions), -and no degree limit applies to it. +A [reported expression](named.md#reported-expressions) is not held to these. `/` needs a divisor that carries no variable and is a single factor. @@ -56,8 +53,6 @@ refused: bind the factor itself as a parameter. Write `x * x` for a square. ## Name resolution -A name is a letter or an underscore, followed by letters, digits or underscores. - One flat namespace covers dimensions, relations, parameters, variables, named expressions, macros and the built-in operators. A collision is a load error that names both declarations, and nothing shadows anything. @@ -74,12 +69,11 @@ Position decides which kinds of name are legal: | the `edge` key of `shift` | `'wrap'` in quotes, or a bare number | | `dual` argument (`dual(c)`) | a constraint. It resolves against the constraints alone ([named expressions](named.md#reading-a-constraints-dual)) | -A bare word in the value of a keyword argument is a name to resolve, which is -why `wrap` is quoted. A keyword's key is never a name. +A bare word in the value of a keyword argument is a name to resolve. A +keyword's key is never a name. -Constraints and assumptions sit outside the flat namespace, because no -expression names either, so a model may name a constraint or an assumption -after a variable. The objective has no name at all. +Constraints and assumptions sit outside the flat namespace, so a constraint or +an assumption may share a variable's name. ## How dimensions combine @@ -98,9 +92,8 @@ The dimension set of every expression is known before any data binds: | `shift(x, along=d, offset=n)` | `dims(x)` | error if `d ∉ dims(x)` | | `sum_back(x, along=d, window=n)` | `dims(x)` | error if `d ∉ dims(x)` | -A binary operator takes the **union** of the two dimension sets, so an outer -product is allowed. The declaration's own dimensions are its **frame**, and a -declaration may not disagree with its expression: +An outer product is allowed. The declaration's own dimensions are its +**frame**, and a declaration may not disagree with its expression: - A constraint requires `dims(lhs) ∪ dims(rhs)` to **equal** its `dims`. - An objective must carry **no dimensions**. Write the sums that reduce it. @@ -130,38 +123,31 @@ COLUMNS ::= NAME | "[" NAME { "," NAME } "]" QUOTED ::= "'" chars "'" | '"' chars '"' ``` -| Written as | Names a… | Meaning | -| --------------------------------------------- | -------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `name` (bare) | parameter | The value is defined here. A `bool` is its own answer. A `str` is defined wherever the table has a row. A number has to have a row and be finite | -| `name` (bare) | variable | The variable exists at this coordinate | -| `name` (bare) | relation | A row exists, read at the relation's key. A relation may be [partial](relations.md#the-data-contract), and this selects the labels that do map | -| `name` (bare) | dimension | A load error. It would be true everywhere | -| `name OP value` | parameter | Element-wise, and a null compares false | -| `name OP value` | dimension | A filter on the frame's own coordinate column | -| `name OP value`, `name.col OP value` | relation | A filter on a value column, read at the relation's key. Name the column where the key determines several | -| `name OP name`, `name.a OP name.b` | two relation columns | Legal where both relations are keyed over the same dimensions and both columns are over one dimension. `ends.bus0 != ends.bus1` excludes a self-loop | -| `expression OP expression` | arithmetic over parameters | Coordinate by coordinate, over every dimension either side carries ([arithmetic in a comparison](#arithmetic-in-a-comparison)). A side with no value at a coordinate compares false | -| `position(name) OP i` | dimension | Where the row sits along the dimension's own order. `0` is first, and a negative number counts from the end | -| `position(name, by=relation, within=c)` | dimension | The same, counted within each group the relation makes | -| `count(where_expr, over=name) OP i` | a predicate | How many coordinates along the dimension the predicate admits ([counting what a predicate admits](#counting-what-a-predicate-admits)) | -| `shift(where_expr, along=name, offset=i)` | a predicate | The predicate read `i` coordinates back, and false where that vacates | -| `at(where_expr, by=relation, over=a, into=b)` | a predicate | The predicate read through the relation ([reading a predicate through a relation](#reading-a-predicate-through-a-relation)), and false where the relation has no row | -| `AND` `OR` `NOT` | — | Case-insensitive. `NOT` binds tighter than `AND`, and `AND` tighter than `OR` | -| `True` / `False` | — | `True` is the same as no `where`; `False` gives a declaration with no rows. A [case `when:`](named.md#the-rules-that-keep-the-cases-apart) may not fold to either | - -The dimensions of the mask must not exceed the frame it sits in. A bare name -that is not declared is a load error. - -!!! warning "Defined is not the same as non-zero" - - A bare parameter name is true wherever the table has a row, and a row - holding `0.0` is a row. Where you mean non-zero, write `where: "inflow != 0"`. +| Written as | Names a… | Meaning | +| --------------------------------------------- | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `name` (bare) | parameter | The value is defined here. A `bool` is its own answer. A `str` is defined wherever the table has a row. A number has to have a row and be finite, and `0.0` is a row: write `inflow != 0` for non-zero | +| `name` (bare) | variable | The variable exists at this coordinate | +| `name` (bare) | relation | A row exists, read at the relation's key. A relation may be [partial](relations.md#the-data-contract), and this selects the labels that do map | +| `name` (bare) | dimension | A load error. It would be true everywhere | +| `name OP value` | parameter | Element-wise, and a null compares false | +| `name OP value` | dimension | A filter on the frame's own coordinate column | +| `name OP value`, `name.col OP value` | relation | A filter on a value column, read at the relation's key. Name the column where the key determines several | +| `name OP name`, `name.a OP name.b` | two relation columns | Legal where both relations are keyed over the same dimensions and both columns are over one dimension. `ends.bus0 != ends.bus1` excludes a self-loop | +| `expression OP expression` | arithmetic over parameters | Coordinate by coordinate, over every dimension either side carries ([arithmetic in a comparison](#arithmetic-in-a-comparison)). A side with no value at a coordinate compares false | +| `position(name) OP i` | dimension | Where the row sits along the dimension's own order. `0` is first, and a negative number counts from the end | +| `position(name, by=relation, within=c)` | dimension | The same, counted within each group the relation makes | +| `count(where_expr, over=name) OP i` | a predicate | How many coordinates along the dimension the predicate admits ([counting what a predicate admits](#counting-what-a-predicate-admits)) | +| `shift(where_expr, along=name, offset=i)` | a predicate | The predicate read `i` coordinates back, and false where that vacates | +| `at(where_expr, by=relation, over=a, into=b)` | a predicate | The predicate read through the relation ([reading a predicate through a relation](#reading-a-predicate-through-a-relation)), and false where the relation has no row | +| `AND` `OR` `NOT` | — | Case-insensitive. `NOT` binds tighter than `AND`, and `AND` tighter than `OR` | +| `True` / `False` | — | `True` is the same as no `where`; `False` gives a declaration with no rows | + +A bare name that is not declared is a load error. ### Counting what a predicate admits `count(, over=)` is how many coordinates along that -dimension the predicate is true at. It is the one place a predicate is read as -a number, and it is compared against a whole number: +dimension the predicate is true at: ```yaml dimensions: @@ -186,25 +172,19 @@ objective: $$\lvert \{ b \in \mathcal{B} \thinspace : \thinspace \mathrm{points}_{g,b} \} \rvert \ge 2 \qquad \forall\thinspace g \in \mathcal{G}$$ -The dimension counted over is **removed**, as a `sum(over=)` removes it, so -what is left is one number per remaining coordinate — one per generator above. -The count therefore states a fact about each group without naming the group. -Counting along a dimension the predicate does not read is a load error. +The dimension counted over is **removed**, as a `sum(over=)` removes it: one +count per generator above. Counting along a dimension the predicate does not +read is a load error. -The comparison takes a whole number on the right. A count is a number of -coordinates, so a fraction and a parameter are both load errors, and so is a -comparison a count can never fail or never meet: `>= 0`, `< 0`, or any -negative number. +The right-hand side is a whole number. A fraction, a parameter, and a +comparison a count can never fail or never meet (`>= 0`, `< 0`, or any +negative number) are load errors. ### Reading a predicate at the previous coordinate `shift(, along=, offset=)` reads the predicate `offset` coordinates back. It is **false** where the translation vacates, and -it takes no `edge=`: the arithmetic `shift` needs one because no number is -neutral, and false is what a missing row already means in a mask. - -The two together name the start of a run — a coordinate the mask admits whose -neighbour before it the mask does not: +it takes no `edge=`. With `count`, it names the start of a run: ```yaml where: "count(points AND NOT shift(points, along=bp, offset=1), over=bp) == 1" @@ -213,16 +193,15 @@ where: "count(points AND NOT shift(points, along=bp, offset=1), over=bp) == 1" That reads: the marked breakpoints are one consecutive run. A negative `offset` reads forwards. `by=`, `within=` and `edge='wrap'` are not -in this form; where you need a grouped or cyclic translation, compare the -arithmetic one instead. +in this form; for a grouped or cyclic translation, compare the arithmetic +`shift`. ### Reading a predicate through a relation `at(, by=, over=, into=)` reads a predicate over coarse coordinates at fine ones, as [`at`](operators.md#at) reads an array. It -is true at a coordinate where the relation has a row and the predicate holds at -the coordinate that row maps to. It is **false** where the relation has no row, -which is what a missing row already means in a mask. +is true where the relation has a row and the predicate holds at the coordinate +that row maps to, and **false** where the relation has no row. ```yaml dimensions: @@ -245,53 +224,40 @@ objective: $$0 \le \mathit{rate}_{f} \le \mathrm{cap}_{f} \qquad \forall\thinspace f \in \mathcal{F} \thinspace : \thinspace \mathrm{has\_curve}_{\mathrm{converter\_of}(f)}$$ -The consumed dimension goes and the produced one arrives, so the mask above is -over `flow` alone. The rules are those of `at` in an expression: `by=`, -`over=` and `into=` are all written, the read lands on the relation's key, and -the predicate carries every dimension the read consumes. The read maps one -dimension onto another and adds none, so a mask still may not widen its frame. - -A parameter compared as arithmetic reads through a relation too: -`at(cap, by=bus_of, over=bus, into=generator) > 0`. The predicate form reads -what arithmetic cannot: whether a row is defined, a `bool`, a variable's -existence, and any connective over them. +The mask above is over `flow` alone. The rules are those of `at` in an +expression: `by=`, `over=` and `into=` are all written, the read lands on the +relation's key, and the predicate carries every dimension the read consumes. ### The right-hand side of a comparison A bare name on the right is read as a string label when the model does not declare it. A declared name there is a load error. -Quote a label that is not an identifier, and quote a date: `'combined-cycle'`, -`'IT-north'`, `'2030-01-01'`. A quoted word is never read as a declaration. +Quote a label that is not an identifier, such as `'combined-cycle'`. A quoted +word is never read as a declaration. A comparison is checked against the declared `dtype`. A `datetime` dimension is -compared against a quoted ISO date such as `snapshot > '2030-01-01'` or +compared against a quoted ISO date such as `'2030-01-01'` or `'2030-01-01T06:00'`, and a number against it is a load error. -String labels compare bytewise, whatever order the dimension declared them in. -A label the dimension does not carry compares equal to nothing, so the mask is -false there. +String labels compare bytewise, whatever the dimension's order. A label the +dimension does not carry compares equal to nothing. Comparing two dimensions is not in the language. Precompute a boolean parameter -instead. Two parameters compare as [arithmetic](#arithmetic-in-a-comparison). +instead. ### Arithmetic in a comparison Either side of a comparison may be an expression over parameters: -`p_min <= 0.5 * p_max`, `sum(p_max, over=generator) >= peak`, +`p_min <= 0.5 * p_max`, or `p_max <= at(bus_cap, by=bus_of, over=bus, into=generator)`. The side is read as -an [expression](#expressions) is. A macro and a named expression expand into it, -and every operator keeps its own rule. Two things an expression may carry are -refused here, because a mask is built before either exists: a variable, and a -`dual()`. A relation column and a quoted label are compared on their own, and -are not read in arithmetic. - -The comparison is checked over every dimension either side carries, and those -dimensions must not exceed the frame. A side whose value is absent at a -coordinate compares false there, as a null does in every other comparison. -Under a summing operator the absent term is one fewer. A `shift` says what its -vacated positions hold, as it does everywhere. So a comparison against the -previous row names an `edge=`, and a `position()` term keeps the first row out: +an [expression](#expressions) is, macros and named expressions included. A +variable and a `dual()` are refused. A relation column and a quoted label are +compared on their own, and are not read in arithmetic. + +A side absent at a coordinate compares false there, and under a summing operator the absent term +is one fewer. A comparison against the previous row gives its `shift` an +`edge=`, and a `position()` term keeps the first row out: ```yaml dimensions: @@ -308,29 +274,13 @@ constraints: expression: shed >= load - ramp ``` -A case `when:` may not compare expressions. The loader proves the cases of a -[`cases:` block](named.md#the-rules-that-keep-the-cases-apart) apart at load, -by trying every value the masks name. A comparison of expressions names no -value, because only the data decides whether `c > 2 * k` holds, so the loader -refuses the case, whether or not the block has a second one: - -> `Named expression 'e'`: case `wide` cannot be told apart before the data -> arrives: it compares expressions, whose values only the data decides — compare -> one parameter against a literal, or precompute the test as a boolean parameter -> and test that. The `otherwise` is its negation, and only the data says where -> that falls, so this is refused the way a proven overlap is. - -A comparison with a number on both sides, such as `2 < 1`, is refused -everywhere: it is decided before any data arrives, and a `where` tests data. - -A variable's `where` and a constraint's `where` are not held to this, because -neither is proved apart from anything. +A [case `when:`](named.md#the-rules-that-keep-the-cases-apart) may not compare +expressions. A comparison with a number on both sides, such as `2 < 1`, is +refused everywhere. ### `position()` -`position(dim)` is where the row sits along the dimension's own order, which is -the order `shift` steps along. A boundary written with it survives a relabelling of -the index: +`position(dim)` counts along the order `shift` steps along, not the label: ```yaml dimensions: @@ -346,11 +296,10 @@ constraints: expression: soc == soc_initial ``` -`-1` is the last position, and `-2` the one before it. A position that no -coordinate occupies is an error when the data binds. +A position that no coordinate occupies is an error when the data binds. -`by=` counts inside each group that a relation makes. That gives one seeded row -per period, however long each period is: +`by=` counts inside each group a [partition](relations.md#partitions) makes, so +each period gets one seeded row: ```yaml dimensions: @@ -368,8 +317,3 @@ constraints: where: "position(snapshot, by=period_of, within=period) == 0" expression: soc == at(soc_initial, by=period_of, over=period, into=snapshot) ``` - -The relation must have a key column over the dimension being counted, and -`within=` names the value columns the groups are made of -([partitions](relations.md#partitions)). A coordinate the relation sends -nowhere is in no group. diff --git a/docs/reference/language/index.md b/docs/reference/language/index.md index 785fde55..39738432 100644 --- a/docs/reference/language/index.md +++ b/docs/reference/language/index.md @@ -36,29 +36,13 @@ objective: expression: sum(dispatch * cost) # an objective is one number, so the sum is written ``` -That file is a complete model. The pages below give the exact rules, and the -[glossary](../glossary.md) defines each word they use in a fixed sense. - -## The pages - -| | | -| ----------------------------------------------------------------------- | ------------------------------------------------------------------------- | -| [File shape](file.md) | the eleven keys, `version` and `description` | -| [Dimensions](dimensions.md) | the axes | -| [Relations](relations.md) | the maps from one axis onto another | -| [Parameters, variables, constraints and the objective](declarations.md) | the four blocks that carry the math | -| [Expressions](expressions.md) | the arithmetic grammar, the `where` grammar, and how dimensions combine | -| [Named expressions and macros](named.md) | quantities named once, templates with arguments, and what a solve reports | -| [Operators](operators.md) | `sum`, `sum_back`, `at` and `shift` | -| [Absence and `where`](absence.md) | which rows are built, and which are not | -| [Piecewise curves and SOS](piecewise.md) | `piecewise:` and `sos:` | -| [Assumptions](assumptions.md) | what the model expects of the data it is bound to | -| [Errors and limits](errors.md) | what fails when, and what the language will not express | +That file is a complete model. The pages of this section give the exact rules, +and the [glossary](../glossary.md) defines each word they use in a fixed sense. ## The ten rules -`to_spec` checks everything it can without data, and refuses the file with a -message that names the fix. These are the rules it checks. +`to_spec` refuses a file that breaks one of these rules, with a message that +names the fix. | # | Rule | | | --- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------- | diff --git a/docs/reference/language/named.md b/docs/reference/language/named.md index b1f08820..96894d29 100644 --- a/docs/reference/language/named.md +++ b/docs/reference/language/named.md @@ -11,8 +11,8 @@ before anything reads it. ## `expressions` -A named expression is a quantity the model names once. A constraint or the -objective may use it, and the engine can report its value after a solve: +A constraint or the objective may use a named expression, and a solve may +report its value: ```yaml dimensions: @@ -28,8 +28,8 @@ expressions: description: CO2 released, the quantity a cap would bound ``` -Write it as a bare string, or as a mapping when it carries a `description:`. Its -dimensions follow from its body, so there is no `dims:`. +It is a bare string, or a mapping with a `description:`. Its body decides its +dimensions, and there is no `dims:`. Where the objective or a constraint names it, the body is substituted there, and the [degree limit](expressions.md#where-a-product-of-two-variables-is-allowed) @@ -87,7 +87,7 @@ A named expression carries **exactly one** of `expression:` and `cases:`. > two `when:` strings by the negation of the other, or drop the wider one and > let `otherwise:` carry that region. - That is why `boundary` above says `committable and`. The cases carry no order. + The cases carry no order. - **A `when:` must be a question the data answers.** `True`, `False`, and a mask that folds to one of them, such as `committable OR True`, are refused. @@ -96,14 +96,16 @@ A named expression carries **exactly one** of `expression:` and `cases:`. against `position(snapshot) == -1` pick the same row on an axis with one member. Count from one end only. +- **A `when:` may not compare expressions**, such as `c > 2 * k`, even in a + block with one case. Precompute the test as a boolean parameter. + - **Each `when:` and each value sits inside the frame.** A narrower case broadcasts as a parameter with fewer dimensions does. -Claiming a coordinate is not the same as having a value there. The `otherwise:` -above carries no `edge=`, so its `shift` has no value at the first snapshot, and -`previous_status` is whole there only because a case claims every unit at that -snapshot. To close such a hole, widen a `when`, give the `shift` an `edge=`, or -set `absence: zero` on the masked variable. +A claimed coordinate can still have no value: the `otherwise:` above has none +at the first snapshot, where a case claims every unit. To close such a hole, +widen a `when`, give the `shift` an `edge=`, or set `absence: zero` on the +masked variable. `cases:` is not accepted inside a `macros:` template. @@ -127,18 +129,15 @@ expressions: objective: { sense: minimize, expression: system_cost } ``` -`system_cost` is in the math: the objective uses it, so the solver sees its body. -`delivered` and `lcoe` are reported: nothing in the math uses them, so the -engine computes them from the solution after the solve. +`system_cost` is in the math. `delivered` and `lcoe` are reported. An entry is in the math when the objective, a constraint or a `piecewise:` link reaches it, directly or through another entry or a macro. A bound and a `where` name no entry. -A reported body is built by no solver, so **no degree limit applies to it**: -it may divide by a variable, raise one to a power, and multiply two sums. A -comparison stays out. A constraint that later names such an entry reads its -body, and is refused there under the constraint's own name. +**No degree limit applies to a reported entry**: it may divide by a variable, +raise one to a power, and multiply two sums. A comparison stays out. A +constraint that names such an entry is refused under the constraint's own name. ### Reading a constraint's dual @@ -152,14 +151,11 @@ Constraint 'd': a dual exists only after a solve; the math cannot read one — keep the entry that carries it out of constraints, the objective, bounds and where. ``` -`c` [resolves against the constraints alone](expressions.md#name-resolution). - `dual(c)` is the rate at which the optimal objective improves as `c` is relaxed in the direction its comparator points, under the model's own `minimize` or `maximize`. -A row that `c`'s `where:` deletes has no dual. Where the solver returns no dual, -as for a model with integer variables, the engine reports no value. +A row that `c`'s `where:` deletes has no dual. ## `macros` @@ -183,6 +179,6 @@ macros: - Every template is held at load to every rule a call site is, whether or not it is called. A formal is left for the call site to bind. -Anything composed out of the [built-in operators](operators.md) belongs here. -What the language cannot express is under -[what the language will not express](errors.md#what-the-language-will-not-express). +A composition of the [built-in operators](operators.md) belongs here. What +the language will not express is in +[the limits](../../about/limits.md#deliberate-non-primitives). diff --git a/docs/reference/language/operators.md b/docs/reference/language/operators.md index eeec1f3d..1ca80623 100644 --- a/docs/reference/language/operators.md +++ b/docs/reference/language/operators.md @@ -27,23 +27,14 @@ in a reported expression, are all of them. A composition of them goes in | `sum_back(array, along=dim, window=p, edge='wrap')` | The window reaches around the axis, instead of stopping short at its start | | `sum_back(array, along=dim, window=n, by=relation, within=c)` | The window stays inside each group that the relation's column `c` makes | -`array` is any expression with the right dimension set, so each operator reads a -parameter as readily as a variable. Dimension arguments are name-checked at -load. [Every operator as math](#every-operator-as-math) shows how each row prints. +`array` is any expression with the right dimension set, a parameter or a +variable. [Every operator as math](#every-operator-as-math) shows how each row +prints. ## `sum` -`sum(x, over=d)` adds up `x` along `d`, and `d` is gone from the result. - -`sum(x)` names no dimension and reduces every dimension `x` carries, so its -result is a scalar. - -An operand that is already scalar, and an `over=` naming a dimension the -operand does not carry, are both errors. - -`sum(x, by=l, over=a, into=b)` sums through a [relation](relations.md), -consuming column `a` and landing the result on column `b`. A nodal balance is -one `sum(by=)` per kind of component: +`sum(x)` on a scalar is an error. A nodal balance is one `sum(by=)` per kind of +component: ```yaml dimensions: @@ -69,32 +60,20 @@ constraints: == load ``` -The same `f` is summed twice through two relations, once as inflow and once as -outflow. What the call reads and what its result carries are on +What a call through a relation reads and carries is on [how a relation is used](relations.md#how-a-relation-is-used). -A group with no members contributes nothing, and a member whose relation value -is null belongs to no group. - ## `at` -`at(x, by=l, over=a, into=b)` reads the relation the other way. It consumes a -value column and produces the key, so it reads one coarse value once for each -fine label that points at it ([reads](relations.md#aggregates-and-reads)). - -`at` reads a variable as readily as a parameter. One decision taken per bus, read -once by every line that touches the bus, is `at(decision, by=line_bus, over=bus, into=line)`. - -A fine label whose relation value is null reads nothing, and its row is absent. +`at` reads one coarse value once for each fine label that points at it +([reads](relations.md#aggregates-and-reads)). One decision per bus, read by +every line that touches the bus, is +`at(decision, by=line_bus, over=bus, into=line)`. ## `sum_back` -`sum_back(x, along=d, window=n)` is the sum of the last `n` positions along `d`, -ending at the position being written. It states a minimum up time, a rolling -budget or a delivery horizon. A width of `1` is `x` itself. - -The dimension **survives**: `sum_back` leaves one value per position, and each -value reads a window of its own. +`sum_back` states a minimum up time, a rolling budget or a delivery horizon. +The dimension **survives**, and a width of `1` is `x` itself. ```yaml dimensions: @@ -116,22 +95,18 @@ constraints: objective: { sense: minimize, expression: sum(on) } ``` -`window=` takes a number or the name of an integer parameter. A named width is `dtype: int`, and does not vary along the -dimension being summed. +A named width is `dtype: int`, and does not vary along the dimension being +summed. -`edge=` takes `'wrap'` or nothing. A window that reaches past the start of the -axis is **short**, so no row is lost. `edge='wrap'` makes the window -reach around the axis. A number here is a load error. +`edge=` takes `'wrap'` or nothing, and a number is a load error. Without it, a +window that reaches past the start of the axis is **short**, and no row is lost. -`by=` keeps the window inside each group that a relation makes. The relation -obeys the rules given for [`shift(by=)`](#translation-within-groups). +`by=` takes a [partition](relations.md#partitions). ## `shift` -`shift(x, along=d, offset=n)` moves values along one dimension by `n` positions, -counted in the dimension's **declared order**. The value at each coordinate -becomes the value that stood `n` places before it. `edge=` says what stands -where nothing moved in. +`shift` counts positions in the dimension's **declared order**. `edge=` says +what stands where nothing moved in. ```yaml dimensions: @@ -149,32 +124,21 @@ constraints: expression: soc == shift(soc, along=snapshot, offset=1, edge='wrap') + charge * eta - discharge ``` -`edge='wrap'` makes the store cyclic: the first snapshot reads the last. - -`edge=` has three settings: - -- **Bare.** The vacated coordinate is [absent](absence.md), so the row it - would have fed is not built. State the initial condition in a block of its - own ([a rule that differs by regime](../../howto/regimes.md)). -- **`'wrap'`.** The translation is cyclic, so nothing is vacated. -- **A number.** That number stands where the slot was vacated, and the row - survives: `0` in a sum, and `1` in a product. +`edge='wrap'` makes the store cyclic: the first snapshot reads the last. Bare, +the row the vacated coordinate would have fed is not built; state the initial +condition in a block of its own ([a rule that differs by regime](../../howto/regimes.md)). -Two rules hold across all three: +Two rules hold for `edge=`: - **Over a variable, the only numeric edge is `0`.** - **A bare `shift` over an expression with no variable is a load error.** The error names the rewrites: `edge='wrap'`, `edge=0`, or `edge=0` together with a `where` that excludes the vacated coordinate. -`shift` reads parameters too. `shift(dt, along=t, offset=1, edge=0)` is the -previous snapshot's duration. - ### Translation within groups -`by=` partitions the axis the operator steps along, so the neighbour of a coordinate -is the coordinate before it in its own group. A group can be a season, an -investment period or a representative day: +`by=` takes a [partition](relations.md#partitions), and the neighbour of a +coordinate is the one before it in its own group, such as a season: ```yaml dimensions: @@ -193,19 +157,12 @@ constraints: objective: { sense: minimize, expression: sum(soc) } ``` -Every `edge=` setting then applies one group at a time. Bare, the first -coordinate of each group is vacated and its row drops. `edge='wrap'` closes each -group onto its own last coordinate. `edge=v` puts `v` at the edge of each group. - -`by=` takes a relation with a key column over the dimension being stepped -along, and `within=` names the value columns the group is made of -([partitions](relations.md#partitions)). A coordinate the relation sends -nowhere is in no group, so its row drops under every `edge=`. +Every `edge=` setting then applies one group at a time. A coordinate in no +group drops under every `edge=`. ### A parameter as offset -`offset=` may name an integer parameter instead of a number. Then each entity is -reached by its own offset: a construction lead time, a transit time, or any delay +An offset per entity is a construction lead time, a transit time, or any delay the data carries as a column: ```yaml @@ -226,14 +183,13 @@ constraints: objective: { sense: minimize, expression: sum(order) } ``` -Three rules hold, and breaking any of them is a load error that names its -rewrite: +Each of these is a load error: -- **The parameter is `dtype: int`.** -- **The parameter does not vary along the dimension being translated.** -- **The parameter varies only over dimensions the shift can read.** Those are - the dimensions of the shifted expression, and the dimension a - [`by=`](#translation-within-groups) relation groups into, so `offset=lead` +- **The parameter is not `dtype: int`.** +- **The parameter varies along the dimension being translated.** +- **The parameter varies over a dimension the shift cannot read.** The shift + reads the dimensions of the shifted expression, and the dimension a + [`by=`](#translation-within-groups) relation groups into: `offset=lead` with `lead: {dims: [period]}` under `by=period_of` gives one lag per period. The sign travels in the values: `offset=-lead` is refused. @@ -243,8 +199,7 @@ The sign travels in the values: `offset=-lead` is refused. Each row is generated from one model in [`examples/operators/`](https://github.com/energy-models/math-spec/tree/main/examples/operators), printed by the [typesetter](../typeset.md). The models themselves are on -[One construct per model](../../examples/operators.md), and the rest of the -language prints on [Every construct, as math](../notation.md). +[One construct per model](../../examples/operators.md). | Operator | Renders as | diff --git a/docs/reference/language/piecewise.md b/docs/reference/language/piecewise.md index dda8384c..e20bf72e 100644 --- a/docs/reference/language/piecewise.md +++ b/docs/reference/language/piecewise.md @@ -5,13 +5,10 @@ SPDX-License-Identifier: CC-BY-4.0 # Piecewise curves and SOS -Two blocks state shapes that no `expression:` can, because an expression is -affine. `piecewise:` states a curve through breakpoints. `sos:` states a family -of variables of which only one, or only two neighbours, may be non-zero. - -Both are **formulations**: each states plain variables and constraints rather -than being one, and [`spec.expand()`](#writing-a-formulation-out) writes them -out. +`piecewise:` states a curve through breakpoints. `sos:` states a family of +variables of which only one, or only two neighbours, may be non-zero. Both are +**formulations**: each states plain variables and constraints, and +[`spec.expand()`](#writing-a-formulation-out) writes them out. ## `piecewise` @@ -52,25 +49,10 @@ piecewise: | `activity` | a binary variable that gates the curve ([below](#activity)) | default `null` | | `points` | how far each curve runs, where the curves are not all the same length ([below](#points)) | default `null` | -A block states plain variables and constraints: one weight per breakpoint in -`[0, 1]`, one row making the weights sum to 1, and one row per link tying its -expression to the weighted breakpoints. The block stays one curve until -[`spec.expand()`](#writing-a-formulation-out) writes these rows out. - -The breakpoint order is the declared order of `over`. A curve whose breakpoints -decrease in that order is refused when the data binds. - -Every condition this page says is checked "when the data binds" is an -[assumption](assumptions.md) that the `method:` implies, named after the block. -It prints beside the file's own assumptions, and the consumer that binds the -numbers runs it. - -!!! warning "A values parameter short of a row does not build a shorter curve" - - The missing row reads as a breakpoint at the origin. Every block states - `_complete` for this, whatever its `method:`, so the table is - refused when the data binds and the refusal names `points:` as the way to - say how far a curve runs. +A block states one weight per breakpoint in `[0, 1]`, a row making the weights +sum to 1, and a row per link tying its expression to the weighted breakpoints. +The breakpoint order is the declared order of `over`. What a block assumes of +its numbers is on [what a curve assumes](assumptions.md#what-a-curve-assumes). ### `activity` @@ -92,9 +74,10 @@ instead, put `absence: zero` on the gate. ### `points` -A curve with fewer breakpoints than the dimension holds says so with `points:`. -Name one of the block's own values parameters, and the curve is as long as that -parameter has rows: +A values parameter short of a row does not build a shorter curve: the missing +row reads as a breakpoint at the origin. A curve with fewer breakpoints than the +dimension holds says so with `points:`. Name one of the block's own values +parameters, and the curve is as long as that parameter has rows: ```yaml piecewise: @@ -106,12 +89,9 @@ piecewise: - [op_cost, bp_y] ``` -The other links are still read against the parameter you named, so a row missing -from `bp_y` is refused. Where the length is its own data, name a boolean -parameter instead. - -The marked breakpoints must be consecutive. They need not start at the head of -the axis. A gap, or a curve with no points, is refused when the data binds. +A row missing from `bp_y` is still refused. Where the length is its own data, +name a boolean parameter instead. The marked breakpoints are one consecutive +run, anywhere on the axis. ### `method` @@ -124,20 +104,8 @@ the axis. A gap, or a curve with no points, is refused when the data binds. | `convex` | nothing | the hull, which is a pure linear program | | `lp` | no weights at all: one row per segment line, plus two rows holding the domain | the curve as its own lines | -`adjacency` and `sos2` state the same restriction and reach the same optimum. -They differ in what the solver is handed: `adjacency` **is** `sos2` with the set -written out, so the two emit the same rows under the same names. - -`convex` is a different model: the weights range over the hull the breakpoints -span rather than over the curve itself. It takes exactly two links and no -`activity:`. - -A bounded link binds from one side, and that side is the part of the hull the -weights are driven onto. `>=` requires a convex curve and `<=` a concave one. -With both links pinned the weights reach the whole hull. What drives them -within it is the rest of the model rather than the block, so the curve must -bend one way only. Each of the three conditions is checked against the -breakpoint values when the data binds. +`convex` takes exactly two links and no `activity:`. The shape it needs is an +[assumption](assumptions.md#what-a-curve-assumes). `lp` states the curve as its segment lines. It needs **exactly two links**, one of them bounded with `<=` or `>=`, and no `activity:`: @@ -152,14 +120,7 @@ piecewise: - [op_cost, bp_y, ">="] # cost bounded below by the curve ``` -The bounded link decides the shape, as it does under `convex` above. The two -domain rows hold the pinned link inside the breakpoint range: under `points:`, -each sits where the mask holds and does not one breakpoint outward, which is -the first and the last breakpoint of each curve. - -`links:` is a list, so the number of expressions a block ties is written in the -file. Where that number is data, write the formulation out -([a curve by hand](../../howto/curve-by-hand.md)). +Where the number of links is data, write the formulation out ([a curve by hand](../../howto/curve-by-hand.md)). ## `sos` @@ -174,22 +135,18 @@ sos: type: 1 # 1: at most one non-zero; 2: at most two, and consecutive ``` -`type: 1` is a choice: at most one member is non-zero. `type: 2` is an -interpolation: at most two members are non-zero, and they are **consecutive**. - A set is over **one** variable, and a variable holds **one** set. A second block naming the same variable is a load error. -Membership belongs to the variable. Its `where` decides which coordinates exist, -so a masked-out member is not in the set. The order is the declared order of -the `along` dimension. +A member the variable's `where` masks out is not in the set. The order is the +declared order of the `along` dimension. ### What a set is written out as `spec.expand('sos')` states the set as binaries: one per member for `type: 1`, one per segment for `type: 2`. A member the binaries do not admit is held at -zero, from above and from below. The names are the block's own, and the rows are -these, for a set `s` over variable `x` along `d`, writing `admitted` for +zero, from above and from below. For a set `s` over variable `x` along `d`, +writing `admitted` for `(s_seg)` at `type: 1` and `(s_seg + shift(s_seg, along=d, offset=1, edge=0))` at `type: 2`: @@ -200,33 +157,10 @@ at `type: 2`: | `s_nonzero` (`type: 1`), `s_adjacency` (`type: 2`) | `x <= upper * admitted` | | the same name plus `_below` | `x >= lower * admitted`, where `lower` is not `0` | -Each coefficient is read off the member's own `bounds:`. A binary member's are -`0` and `1`, from its domain. A row multiplies by its coefficient rather than -reading it, so a bound the data carries is a coefficient like any other: -`bounds: {lower: floor, upper: cap}` states `x >= floor * admitted` and -`x <= cap * admitted`. - -Two coefficients are left out rather than printed, because the row would state -what another row already does: a `1` above, and a `lower` of `0`, which the -variable's own bound states. - -So each side needs a coefficient, and a model is refused at load without one: - -- `bounds.lower`, a number or a parameter. An omitted lower bound leaves the - member free below zero, which no row can pull back. -- `bounds.upper`, a number or a parameter, or `domain: binary`. - -The set carries no coefficient of its own. A number below the member's bound -would cap a picked member the set does not cap, and one above it is a looser -row than the bound already states, so there is no value of such a key that -states the set and nothing else. - -A positive `bounds.lower` loads and is infeasible, as it is on a solver that -takes the set: an unpicked member has to be `0`, and its own bound says it is -above that. - -A name the expansion writes that the file already declares is refused at load -too. +`upper` and `lower` are the member's own `bounds:`, a number or a parameter; +a binary member's are `0` and `1`. A model is refused at load where a member +has no `bounds.lower`, or no `bounds.upper` and no `domain: binary`. A name the +expansion writes that the file already declares is refused at load too. ## Writing a formulation out @@ -237,11 +171,7 @@ shows a model before and after. - **Every name written out starts with the name of the block.** The weights of the curve `curve` are `curve_lam`. -- **A curve writes out the rows its [`method`](#method) adds.** A - `method: sos2` curve writes out an `sos:` block, and a set writes out as - [binaries](#what-a-set-is-written-out-as). - **No formulation emits a parameter.** The same data binds a model and its - expansion. A curve under `points:` puts its rows on `where:` predicates over - the mask the file named. + expansion. - **The assumptions a `method:` implies become `assumptions:` entries** with the same names. diff --git a/docs/reference/language/relations.md b/docs/reference/language/relations.md index 63ab8950..8651caa3 100644 --- a/docs/reference/language/relations.md +++ b/docs/reference/language/relations.md @@ -39,17 +39,15 @@ mapping form names them: `{bus0: bus, bus1: bus}`. ### Cardinalities -| intention | written | cardinality | -| ------------------------------------------------------ | ----------------------------------------------------- | ------------------------------------------- | -| each generator has one bus | `{key: generator, values: bus}` | many-to-one | -| a bus has several generators | the same table, read the other way | one-to-many | -| a generator may connect to several buses | `{key: [generator, bus]}`, no `values:` | many-to-many | -| a generator has one zone in each period | `{key: [generator, period], values: zone}` | many-to-one, keyed by a pair | -| a snapshot has a month, a week and a weekday | `{key: snapshot, values: [month, week, weekday]}` | many-to-one, several values | -| a line has two ends, both buses | `{key: line, values: {bus0: bus, bus1: bus}}` | many-to-one, two columns over one dimension | -| a snapshot has a representative snapshot | `{key: snapshot, values: {rep: snapshot}}` | many-to-one, onto itself | -| a snapshot has neighbours | `{key: {from: snapshot, to: snapshot}}`, no `values:` | many-to-many, onto itself | -| each generator has one bus, and each bus one generator | not a claim the language has | one-to-one | +| intention | written | cardinality | +| -------------------------------------------- | ----------------------------------------------------- | ------------------------------------------- | +| each generator has one bus | `{key: generator, values: bus}` | many-to-one | +| a generator may connect to several buses | `{key: [generator, bus]}`, no `values:` | many-to-many | +| a generator has one zone in each period | `{key: [generator, period], values: zone}` | many-to-one, keyed by a pair | +| a snapshot has a month, a week and a weekday | `{key: snapshot, values: [month, week, weekday]}` | many-to-one, several values | +| a line has two ends, both buses | `{key: line, values: {bus0: bus, bus1: bus}}` | many-to-one, two columns over one dimension | +| a snapshot has a representative snapshot | `{key: snapshot, values: {rep: snapshot}}` | many-to-one, onto itself | +| a snapshot has neighbours | `{key: {from: snapshot, to: snapshot}}`, no `values:` | many-to-many, onto itself | A key that determines a value holds one column per dimension, so `{key: {bus0: bus, bus1: bus}, values: line}` is refused. A bare relation may @@ -72,8 +70,8 @@ column per declared column, named after it. ## How a relation is used -The declaration fixes no direction. A call names the columns it reads, and a -key column it names at neither end is **joined on**. +The declaration fixes no direction. A key column a call names at neither end +is **joined on**. | kind | what it does | written as | | --------- | ------------------------------------------- | ----------------------------------------------------- | @@ -85,8 +83,7 @@ key column it names at neither end is **joined on**. Four rules hold for every use: 1. **A call names every column it reads.** `sum(p, by=gen_bus)` is refused. -2. **A value column the call does not name is not read.** So adding one to the - relation changes no call. +2. **A value column the call does not name is not read.** 3. **The key is fixed.** To change it, declare a new relation. 4. **A dimension the relation does not name passes through** to the result. @@ -116,11 +113,8 @@ bare `connection: { key: [generator, bus] }` and `p` over `[generator, period]`: ``` - **A sum consumes at least one key column, and lands on any column it does not - consume.** Either end may name a value column beside a key one. `connection` - has only key columns, and the sum above consumes one and lands on the other. -- **A read consumes value columns, and lands on the key.** Its result carries - every key column — named in `into=`, or joined on — and whatever else the - operand carries that the read does not consume. + consume.** Either end may name a value column beside a key one. +- **A read consumes value columns, and lands on the key.** - **`over=` and `into=` name different columns**, and neither names two columns over one dimension. @@ -131,7 +125,7 @@ and `position(d, by=l, within=c)` step along the key column over `d`, join on the other key columns, and group by the value columns `within=` names. The frame does not change. `within=` is written whenever `by=` is. It may name two columns over one dimension, may not name a key column, and a bare relation -partitions nothing. +partitions nothing. A coordinate the relation sends nowhere is in no group. ### Tests diff --git a/docs/reference/reading.md b/docs/reference/reading.md index d42caf77..d419cdab 100644 --- a/docs/reference/reading.md +++ b/docs/reference/reading.md @@ -6,12 +6,7 @@ SPDX-License-Identifier: CC-BY-4.0 # Reading a loaded model This page is for whoever writes an engine that builds models, a renderer, or a -checker. You need none of it to write a model. A tool reads the model through -two objects, and one door: - -```text -to_spec → Spec → .program → Program -``` +checker. A tool reads the model through two objects, `Spec` and `Program`. ## `Spec` and `Program` @@ -77,7 +72,7 @@ sorted(rows.variables) # ['cost', 'curve_lam', 'p'] program built when the model loaded, so every ask on one model returns one object. A `piecewise:` block is a curve under `program.piecewise`, typed, and a `sos:` block is a set under `program.sos`. Every parameter the program declares -is one the file declared, and the engine binds each from its data. +is one the file declared. ## Formulations written out @@ -102,30 +97,23 @@ spec.expand('sos') is spec # True declares the same dimensions and parameters, so the same data binds both. - **A model with nothing to write out comes back as itself.** So does an expansion asked for the same kinds again. -- **The spec keeps no expansion.** A second call builds it again, so a caller - that needs it twice holds the result. -- **The expansion is a model like any other.** `to_yaml()` writes it, and its - `program` holds the rows and no curve. -- **Nothing expands a model unasked.** A consumer that builds rows reads the - program of `spec.expand('piecewise')` if it takes a set, and of - `spec.expand()` if it does not. It refuses a curve it finds on a program, in - its own words, naming the call: +- **The spec keeps no expansion.** A second call builds it again. +- **Nothing expands a model unasked.** A program holds its curves until + `expand()` writes them out. The expansion is a model like any other: + `to_yaml()` writes it, and its `program` holds the rows and no curve. ```python -def rows_of(program): - if program.piecewise: - raise ValueError(f"{sorted(program.piecewise)} are curves; pass spec.expand('piecewise')") - return program - - -rows_of(rows) is rows # True +sorted(rows.piecewise) # [] ``` ## What the data has to satisfy -`program.assumptions` holds every fact the numbers have to meet, by the name a -refusal quotes. The engine, which has the numbers, runs each one and raises -`assumption_message` where it fails: +`program.assumptions` maps a name to an `Assumption`: each entry the file +declared, and each one a curve's method derives +([what a curve assumes](language/assumptions.md#what-a-curve-assumes)). An +`Assumption` carries a `predicate` and the `where` it is checked under, both +masks, and the `description` a refusal ends with. `assumption_message` returns +the message for an assumption the data does not meet: ```python from math_spec.program import Assumption, assumption_message @@ -138,19 +126,10 @@ written = assumption_message('cost_is_never_negative', program.assumptions['cost written # "assumption 'cost_is_never_negative' does not hold for the data bound to 'bp_y' — a negative cost is a gain the objective would chase" ``` -One kind stands in that mapping. An `Assumption` carries a predicate as two masks — -`predicate`, and the `where` it is checked under — and the sentence a refusal -trails under `description`. What a `piecewise:` block's method implies about -its breakpoints is written in the same language and stands beside what the -file wrote: `expand()` emits those entries, and a model that still declares -the block derives the same text at load. So a consumer reads one kind, and a -condition a method adds later is a row in that mapping rather than a case to -handle. - ## Nodes and masks -You never build a node yourself. The node classes are exported so that you can -test one with `isinstance` and read its fields. `children()` walks an expression +The node classes live in `math_spec.program`, for `isinstance` tests and field +reads. `children()` walks an expression node's operands, and `where_children()` walks a predicate's. `walk()` yields every node under an expression, parents first. `walk_regions()` yields each node with the `cases:` regions it stands inside, outermost first. @@ -158,7 +137,7 @@ with the `cases:` regions it stands inside, outermost first. A `Named` stands where an `expressions:` entry is used. Its `body` is the entry's expression, the same object that `program.expressions[name].expression` holds, and its value is the body's value. `children()` steps into the body, so -a walk reads through it; a renderer prints the name where the file wrote it. +a walk reads through it. Every `where` arrives as a `Mask`. Its `.root` is the resolved predicate. The mask also answers four questions: @@ -175,30 +154,26 @@ the sides read, the relation a grouping reads through included. A name compared against a literal does not arrive this way. `p_max > 5` is a `ParameterComparison` and `1 * p_max > 5` is an `ExpressionComparison`, though -both mask the same coordinates. Match both where you read a comparison over -parameters. +both mask the same coordinates. Three predicates read another predicate rather than a declaration. A `CountComparison` carries the mask it counts and the dimension it counts away. A `TranslatedPredicate` carries the mask it reads at a neighbouring coordinate. A `PulledBackPredicate` carries the mask it reads through a relation, and the `Direction` it reads in. Each holds that mask as a `Mask`, -where a connective holds a bare predicate: the walk recurses through a -connective and stops at these, so read the field where you need what is -inside. `.names_read` and `.dims` already see through all three, and the -relation a `PulledBackPredicate` reads is in its `.names_read`. +where a connective holds a bare predicate, so the walk recurses through a +connective and stops at these. `.names_read` and `.dims` see through all three, +and the relation a `PulledBackPredicate` reads is in its `.names_read`. -A predicate you build yourself answers the same four questions: wrap it in -`Mask`, or build it there with `~`, `&` and `|`. A mask folds as it is built, -so a boolean literal stands at a mask's root or nowhere. A `Region`'s `when` -arrives as a `Mask` too. The node classes live in `math_spec.program`. +`Mask(predicate)` answers the same four questions of any resolved predicate, +and `~`, `&` and `|` combine masks into a mask. A mask folds as it is built, so a boolean literal +stands at a mask's root or nowhere. A `Region`'s `when` is a `Mask` too. ## Asking what a program uses `program.footprint` says which of the language's constructs one model uses. -It answers for the rows the program holds, and a curve still on the program is -not a row. Ask it of the rows a solver takes, since a curve written out uses -more of the language than the block did: +It answers for the rows the program holds. A curve still on the program is not +a row, so its constructs count on the program of the expansion: ```python footprint = rows.footprint @@ -209,11 +184,10 @@ sorted(footprint.sos_types) # [] sorted(kind.__name__ for kind in footprint.kinds) # ['Constant', 'Multiply', 'Parameter', 'Sum', 'Variable'] ``` -Every field is a set. An empty field means this model does not use the -construct. The footprint says what the model uses. Whether your solver or -file format can take a construct is your question -([what a solver can take](../about/limits.md#solver-capability)). Whether a -quadratic form is convex is not reported, because it depends on the numbers. +Every field is a set, and an empty field means the model does not use the +construct. Whether a solver takes a construct is the engine's question +([what counts as language](../about/what-counts-as-language.md#what-each-tool-decides-for-itself)). +Convexity is not reported: it depends on the numbers. ## Asking whether an axis can be cut @@ -261,5 +235,5 @@ about whether the windowed answer equals the whole-horizon answer. that data as a file. Both round-trip, so `to_spec(spec.to_dict()) == spec`. `to_yaml()` writes every value and omits every absence. `domain: continuous` is -written out. A `null`, an infinite bound and an empty section are left out. +written out. A `null` and an empty section are left out. `dims: []` is written, because it says the declaration is a scalar. diff --git a/docs/reference/typeset.md b/docs/reference/typeset.md index 06052608..4004f726 100644 --- a/docs/reference/typeset.md +++ b/docs/reference/typeset.md @@ -19,17 +19,13 @@ print(ms.to_markdown(spec)) # renders as-is on GitHub ``` Each function takes a path, the YAML, a mapping, a `Spec` or a `Program`, and -prints the program: the one a spec holds, or the one it was handed. The same -three formats come from a shell: - -```bash -python -m math_spec latex model.yaml --symbols model.symbols.yaml --standalone -o model.tex -python -m math_spec typst model.yaml --standalone -o model.typ -python -m math_spec markdown model.yaml -``` +prints the program: the one a spec holds, or the one it was handed. +From a shell, `python -m math_spec latex model.yaml` prints the same, and +`typst` or `markdown` in place of `latex` picks the format. [Print a model as math](../howto/print.md) is the recipe, and -[every construct, as math](notation.md) shows what each construct prints. +[every operator as math](language/operators.md#every-operator-as-math) shows +what each operator prints. ## Options @@ -49,9 +45,9 @@ a flag. The [Python API](api.md#typesetting) gives each signature. - The model's `description:` opens the document. - A `piecewise:` block prints as one line: the curve it states, over the frame - it states one curve per coordinate of. - [Printing what a formulation states](#printing-what-a-formulation-states) - prints its rows instead. + it states one curve per coordinate of. To print its rows, print + [`spec.expand()`](reading.md#formulations-written-out) or pass `--expand` + ([see an expansion](../howto/see-an-expansion.md)). - An [`assumptions:`](language/assumptions.md) entry prints under an **Assumptions** heading, last, beside what each curve assumes of its breakpoints. A model that assumes nothing of its data prints no such @@ -107,54 +103,14 @@ expressions it uses are substituted. A cased expression prints by symbol, and a second call with its name prints its block. A name that is none of the four kinds is refused with the near miss. A name -declared as two of them, such as a constraint and a variable, is refused too, -because one line can print only one of them. - -## Printing what a formulation states - -To print the variables and constraints that a `piecewise:` or `sos:` block -states, print [`spec.expand()`](reading.md#formulations-written-out): - -```python -ms.to_latex(spec) # the curve, and the set beside its variable -ms.to_latex(spec.expand()) # the weights, the convexity row, the binaries -ms.to_latex(spec.expand('sos')) # the curves as curves, the sets as binaries -``` - -The command line spells it `--expand`: - -```bash -python -m math_spec latex model.yaml --expand --symbols model.symbols.yaml -``` - -One symbol table serves both, because a name a formulation emits counts as -declared — which is what lets `_lam` print as $\lambda$ in the expansion -and the same table render the file it came from. +declared as two of them, such as a constraint and a variable, is refused too. ## Symbol tables With no table, the symbols are **derived** from the names in the file, such as $\mathrm{load}_t$ and $\mathrm{capacity}_g$. A symbol table makes the output -conventional: - -```python -symbols = { - 'notation': 'latex', - 'dimensions': { - 'snapshot': {'index': 's', 'set': '\\mathcal{S}'}, - 'generator': {'index': 'g', 'set': '\\mathcal{G}'}, - }, - 'names': { - 'cost': 'c', - 'load': '\\ell', - 'capacity': '\\bar p', - }, -} - -ms.to_latex('dispatch.yaml', symbols=symbols) -``` - -Pass a dict, a path to a YAML file, or a `ms.SymbolTable`. As a file: +conventional. Pass a path to a YAML file, the same keys as a dict, or a +`ms.SymbolTable`: ```yaml # dispatch.symbols.yaml @@ -168,6 +124,10 @@ names: capacity: "\\bar p" ``` +```python +ms.to_latex('dispatch.yaml', symbols='dispatch.symbols.yaml') +``` + | Section | | | ------------ | ------------------------------------------------------------------------------- | | `notation` | **Required.** `latex` or `typst`: the language the entries are written in | @@ -178,5 +138,4 @@ Every spelling is printed as you wrote it, and nothing translates notation, so rendering a LaTeX table as Typst is refused. A key that names nothing in the model, and nothing a formulation of it emits, is an error with the near miss. -Nothing in a symbol table changes what the file means. What a declaration _is_ -stays in its own `description:`. +Nothing in a symbol table changes what the file means. diff --git a/mkdocs.yml b/mkdocs.yml index ecc96d37..7d7b25de 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -56,7 +56,6 @@ nav: - Assumptions: reference/language/assumptions.md - Absence and where: reference/language/absence.md - Errors and limits: reference/language/errors.md - - Every construct, as math: reference/notation.md - Typeset the math: reference/typeset.md - Python API: reference/api.md - Glossary: reference/glossary.md @@ -71,8 +70,9 @@ nav: # Everything a model writer does not need, grouped by reader: whoever # writes a tool against `Spec` and `Program` (an engine such as specsolve, a # renderer, a checker), whoever changes math-spec itself, and the proofs of - # concept. The PyPSA pages stay in `docs/examples/`, where `tools/gallery.py` - # writes them. `docs/static/hooks.py` appends one page per module under + # concept: the notation page, which renders the typesetting test model, and + # the PyPSA pages. The PyPSA pages stay in `docs/examples/`, where + # `tools/gallery.py` writes them. `docs/static/hooks.py` appends one page per module under # `src/math_spec/` to Contributing, as `Modules`. - Development: - Building on math-spec: @@ -83,6 +83,7 @@ nav: - contributing.md - What counts as public API: about/what-counts-as-public-api.md - Proofs of concept: + - Every construct, as math: reference/notation.md - PyPSA in one file: examples/pypsa.md - PyPSA, the quadratic class: examples/pypsa_quadratic.md - PyPSA, the relaxed commitment: examples/pypsa_linearized_uc.md