diff --git a/.claude/skills/docs-writing/SKILL.md b/.claude/skills/docs-writing/SKILL.md index cd0e83b2..1e62e0b4 100644 --- a/.claude/skills/docs-writing/SKILL.md +++ b/.claude/skills/docs-writing/SKILL.md @@ -45,19 +45,31 @@ Two questions decide it, and they work on a paragraph as well as a page: 1. Does it inform **action** or **cognition**? 2. Does it serve **acquiring** a skill or **applying** one? -| Kind | Informs | Serves | Answers | Nav section · folder | +| Kind | Informs | Serves | Answers | Section · folder | | ----------- | --------- | ------- | ----------------------------------------------------- | -------------------------------------------------------------- | | Tutorial | action | acquire | "Get me a first file that loads and prints" | Tutorials · `docs/` | | How-to | action | apply | "I have this task" | How-to guides · `docs/howto/` | | Reference | cognition | apply | "What exactly does X accept, and what does it print?" | Reference · `docs/reference/`, model pages in `docs/examples/` | | Explanation | cognition | acquire | "Why is it like this?" | About · `docs/about/` | -The nav and the tree are both arranged by kind. A new page goes in the folder -of its kind and under the nav section of the same name; the first tutorial -opens the `Tutorials:` section, above the how-to guides. The model pages sit at -the end of the Reference section, after the pages a reader looks things up in. -A worked example is neither a tutorial nor a how-to: it teaches no path and -names no task, it shows that the language says a model. +The nav and the tree are both arranged by kind, for someone who writes a +model. A new page goes in the folder of its kind and under the nav section of +the same name. The model pages sit at the end of the Reference section, after +the pages a reader looks things up in. A worked example is neither a tutorial +nor a how-to: it teaches no path and names no task, it shows that the language +says a model. + +The Development section, last in the nav, holds every page a model writer does +not need, in three groups: + +- **Building on math-spec** is for someone who writes a tool against `Spec` + and `Program`: an engine such as specsolve, a renderer, a checker. +- **Contributing** is for someone who changes math-spec itself. +- **Proofs of concept** holds the notation page, which renders the typesetting + test model, and the PyPSA pages. The PyPSA pages stay in `docs/examples/`, + where `tools/gallery.py` writes them. + +A page in Development keeps the folder of its kind. Each kind has one job, and one thing it must not do: @@ -80,16 +92,19 @@ math the typesetter prints from it. The block is written by `tools/gallery.py` between `` and ``; the paragraph is the only prose on the page, and it says what the model is and the one or two things worth reading for, which the `description:` line in the file does not. -The PyPSA pages add a generated block per rung, holding the reference script -and what PyPSA solved it to. Every model is a file under `examples/`, loaded -by the suite and compiled by the LaTeX gate, and `tests/test_docs.py` holds -each block to its generator byte for byte. The catalogue in -`docs/examples/index.md` is hand-written: one bullet per page, saying why a -reader would open it. - -**The Python API pages are built, not written.** mkdocs renders -`reference/math_spec/` from the docstrings at build time, so their prose is -the docstring rules in `AGENTS.md`. +The PyPSA pages, in the Development section, add a generated block per rung, +holding the reference script and what PyPSA solved it to. Every model is a +file under `examples/`, loaded by the suite and compiled by the LaTeX gate, +and `tests/test_docs.py` holds each block to its generator byte for byte. The catalogue in +`docs/examples/index.md` is hand-written: one bullet per page in the Examples +section, saying why a reader would open it. + +**The Python API is rendered from the docstrings.** +`docs/reference/api.md` holds one `:::` entry per name in `math_spec.__all__`, +and mkdocstrings renders each from its docstring. `docs/static/hooks.py` +renders one page per module under `src/math_spec/`, and puts them in the +Contributing group of the Development section as `Modules`. The prose of both is the docstring rules in +`AGENTS.md`. Mixing kinds is the most common failure. Rationale inside a reference section makes the rules unskimmable, and rules inside an explanation page make the @@ -98,11 +113,11 @@ survives into an explanation page is the part a user needs to make decisions. The language is documented here and only here. A page says what a file may contain, what it means, what the loader refuses, and what the typesetter -prints from it. What a consumer does with a spec — the data it binds, how it +prints from it. What a consumer does with a spec — the data it attaches, how it solves, what it reads back — is that consumer's page, not this tree's ([what counts as language](../../../docs/about/what-counts-as-language.md)). A rule about a consumer says only what the file guarantees it -([reading a loaded model](../../../docs/reference/language/reading.md)). +([reading a loaded model](../../../docs/reference/reading.md)). Answer the two questions before starting. If a page needs two kinds, it is two sections with two headings, or two pages. @@ -179,7 +194,7 @@ not - **Gloss house vocabulary at first use** — _spec_, _program_, _declaration_, _dimension_, _coordinate_, _frame_, _relation_, _absence_, _macro_, _named - expression_, _reported expression_, _escape_. One clause with a concrete + expression_, _reported expression_. One clause with a concrete instance: "one point of it, one generator in one snapshot, is a coordinate". - **Gloss every acronym and domain term at first use**, in parentheses, six words or fewer. @@ -204,7 +219,7 @@ The bar, and it is checkable: 1. **One idea per sentence.** Median at or under 20 words; over 25 is where a newcomer re-reads. 2. **Active voice, with a real subject.** "The loader refuses it before any - data binds", not "the refusal comes before any data binds". An abstract + data is attached", not "the refusal comes before any data is attached". An abstract noun as subject is the single biggest reason technical prose reads expert-only. 3. **State the rule in things, then in abstractions.** "One generator at one @@ -307,7 +322,7 @@ PY - **Anything that duplicates another page.** One fact, one home; link instead. A second copy drifts silently. The README is pulled into `docs/index.md` as snippets, so a sentence that appears on both is edited once, in the README. -- **A rule of an engine.** How a spec is bound to data, solved, or read back +- **A rule of an engine.** How data is attached to a spec, how it is solved, or read back is a consumer's page. Here a consumer is named only for what the file guarantees it. - **Generated content.** The model and its math on every example page and the diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 68092c85..034804e5 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -65,8 +65,10 @@ When opening a pull request, please provide a clear summary of your changes! decides where it goes, in the nav and in the tree**: a tutorial (`docs/`), a how-to guide (`docs/howto/`), reference (`docs/reference/`, and the model pages in `docs/examples/`) or explanation (`docs/about/`) — the four kinds of -[Diátaxis](https://diataxis.fr) — and one page is one kind. The rules each kind -has to meet, and the sentence-level bar, are in +[Diátaxis](https://diataxis.fr) — and one page is one kind. A page a model +writer does not need goes under Development in the nav: building on +math-spec, contributing, or a proof of concept. The rules each kind has to meet, and the sentence-level +bar, are in [the docs-writing skill](https://github.com/energy-models/math-spec/blob/main/.claude/skills/docs-writing/SKILL.md). Every page needs a `nav:` entry in `mkdocs.yml`, links inside `docs/` are relative, and a link outside it is the full GitHub URL; `pixi run docs-build` diff --git a/README.md b/README.md index e1354e88..30249386 100644 --- a/README.md +++ b/README.md @@ -28,10 +28,8 @@ math-spec reads that file, checks everything that can be checked without data, and hands the result on: to an engine that builds and solves the model, or to the typesetter that prints it as LaTeX, Typst or Markdown. It builds nothing and solves nothing itself. Every tool reads the file through the same checked syntax -tree, so an engine and a renderer cannot disagree about what the file means; that -is the [test](docs/about/what-counts-as-language.md) for what belongs here. - -Three properties follow: +tree, so an engine and a renderer cannot disagree about what the file means +([what counts as language](docs/about/what-counts-as-language.md)). - **Nothing is guessed.** A misspelled name, a `where` string on an undeclared parameter, a constraint whose dimensions do not match its `dims`: each fails @@ -42,7 +40,8 @@ Three properties follow: composition of them goes in `macros:` ([the limits](docs/about/limits.md)). - **The file is the document.** `to_latex(spec)` prints the model as equations from the file alone, so the math you publish is the math you solve - ([typeset](docs/reference/typeset.md)). + ([typeset](docs/reference/typeset.md)). The file diffs in review, and no + Python state changes what it means. @@ -102,13 +101,10 @@ objective: -That file is a complete model. Nothing outside it changes what it means. - ### The math it prints -Here is that model as math, printed from the file above and nothing else. No -data, no solver, and no second copy of the equations to keep in step. Markdown -is one of three formats, so GitHub renders it here. +The typesetter prints the file above as math, with no data and no solver. +Markdown is one of three formats, and GitHub renders it here. -Each format is one call, and the file is read and checked once: +Each format is one call: ```python import math_spec as ms @@ -273,107 +269,27 @@ ms.to_latex(spec) # amsmath align ms.to_typst(spec) # compiles without a TeX toolchain ``` -Those symbols are the file's own names: `load` prints as $`\mathrm{load}_t`$, -and `capacity` as $`\mathrm{capacity}_g`$. Nothing had to be set up for -that. Pass `symbols='dispatch.symbols.yaml'` and the typesetter prints -$`\ell_t`$ and $`\bar p_g`$ instead, above a legend that defines them. The -first folded block shows it. The table can be a dict, a `SymbolTable`, or a -path to YAML. A key that names nothing in the model is an error, and nothing -in a table changes what the file means. - -Or from a shell, beside `pdflatex` in a Makefile: - -```bash -python -m math_spec latex dispatch.yaml --symbols dispatch.symbols.yaml --standalone -o dispatch.tex -python -m math_spec typst dispatch.yaml --standalone -o dispatch.typ -python -m math_spec markdown dispatch.yaml -``` - -### `Spec` and `Program` - - - -Whatever is wrong with a model is wrong when it loads, not when it solves: - -```python -import math_spec as ms - -spec = ms.to_spec('dispatch.yaml') # schema, names, dimensions, degree: all checked here -sorted(spec.variables) # ['dispatch'] +A [symbol table](docs/reference/typeset.md#symbol-tables) gives the names their +conventional spelling, as in the first folded block. +[Print a model as math](docs/howto/print.md) does the same from a shell. +`to_spec` returns a `Spec`, and `spec.program` the model it builds +([reading a loaded model](docs/reference/reading.md#spec-and-program)). -program = spec.expand().program # curves expanded, names typed, operators resolved to nodes -sorted(program.constraints) # ['power_balance'] -``` +## Documentation -Neither needs data or a solver, so a repository of models compiles in CI with -nothing bound to any of them. **A `Spec` holds the file as written, and a -`Program` holds the model it builds**, with every macro expanded and every curve -kept as the block it is. `spec.expand()` turns each curve into its variables and -constraints; an engine that builds rows reads that model's `Program`. - - - -[Reading a loaded model](docs/reference/reading.md) says what a tool gets -from each, and [the file and the program](docs/about/file-and-program.md) says -why there are two. - -## Why - -- **Declarative math.** A file is readable without knowing any implementation, - and no Python state changes what it means. It diffs in review and travels as a - research artefact. -- **Fail early, fail loud.** Nothing falls back silently, and an error names the - problem and its rewrite. A model that does not load does not print either. -- **One flat namespace, ten rules.** A collision is a load error naming both - declarations, position decides which kinds of name are legal, and a name's kind - is fixed at load. The [ten rules](docs/reference/language/index.md) are one - principle in ten positions. -- **A closed operator set.** `sum`, `sum_back`, `at` and `shift`, with the - arithmetic and `where` grammars. A composition of them goes in `macros:`, so - every engine expands it the same way. -- **A finite language.** An operator joins the language only if each output row - reads a bounded number of input rows, and a file cannot add one. Math the - language cannot express is refused, with the rewrite named. - -## Docs - -Start with [the language](https://math-spec.readthedocs.io/latest/reference/language/): -the ten rules, and the pages that give the exact ones. Then -[every construct as math](https://math-spec.readthedocs.io/latest/reference/notation/), -which prints all of it beside the notation the typesetter gives it, and -[typeset the math](https://math-spec.readthedocs.io/latest/reference/typeset/) -for how to print your own. Why the language is shaped this way, what may enter -it, and who owns a rule once it is in are under -[about](https://math-spec.readthedocs.io/latest/about/limits/). To work on it, -read [CONTRIBUTING.md](CONTRIBUTING.md). +The documentation is at . ## Installation -This project is managed by [pixi](https://pixi.prefix.dev/). To develop against -it: - - - -```bash -git clone https://github.com/energy-models/math-spec -cd math-spec - -pixi run pre-commit-install -pixi run test -``` - - - -Releases are on the alpha stream, and **nothing is published yet**. The publish -job is off until the project leaves it, so `pip install math-spec` is what the -first release will look like, not what today does. Install from a checkout or a -git reference until then; see [RELEASING.md](RELEASING.md). +Nothing is published yet. +[Installation](docs/howto/installation.md) gives the command that installs from +git, and [contributing](docs/contributing.md#setting-up-a-development-environment) +sets up a development clone. ## Prior art Every file under `src/` was written in [specsolve](https://github.com/fluxopt/specsolve) -and extracted here, so that the language and the syntax tree a tool reads it -through are a dependency rather than one engine's internals. The keys themselves, +and extracted here. The keys themselves, which are YAML math, a block per component, `dims:` and a `where:` string, come from [Calliope](https://github.com/calliope-project/calliope). [linopy](https://github.com/PyPSA/linopy) supplies the vocabulary that @@ -386,17 +302,12 @@ Alpha, pre-1.0. -**Breaking changes land without a deprecation cycle.** When a construct is named -wrong, a default is wrong, or a permissive input hides a silent wrong answer, it -is fixed rather than aliased. A compatibility shim for every earlier spelling -would defeat the point of a small language. - -Pin an exact version if you depend on this, and read the +**Breaking changes land without a deprecation cycle.** Pin an exact version if +you depend on this, and read the [changelog](https://github.com/energy-models/math-spec/blob/main/CHANGELOG.md) -before upgrading. What exists is tested: every construct the language has -round-trips through the schema, the parsers and all three typeset formats, and -the LaTeX is compiled rather than eyeballed. It is the accepted YAML that is not -yet frozen, not the behaviour. +before upgrading. Every construct round-trips through the schema, the parsers +and all three typeset formats, and the LaTeX is compiled. The accepted YAML is +not yet frozen. diff --git a/docs/about/file-and-program.md b/docs/about/file-and-program.md index fc359b03..86b6d377 100644 --- a/docs/about/file-and-program.md +++ b/docs/about/file-and-program.md @@ -6,8 +6,8 @@ SPDX-License-Identifier: CC-BY-4.0 # The file and the program This page explains why a loaded model is two objects, and which one each tool -reads. Read it before you write a tool that reads models. You need none of it -to write a model. +reads. You need none of it to write a model. +[Reading a loaded model](../reference/reading.md) is the reference for both. ```text file ── to_spec ──▶ Spec ── .program ──▶ Program @@ -17,63 +17,19 @@ file ── to_spec ──▶ Spec ── .program ──▶ Program ## Two states -**A `Spec` is the file as written.** `to_spec` reads the YAML into a `Spec` -and checks every rule that needs no data. The spec keeps the file's own -spelling: an expression is a string, a bound is a number or a parameter's -name, and a macro is its template. - -**A `Program` is what the file means.** `spec.program` holds every declaration -of the file, section for section, with every name typed and every operator -resolved to a node. The macros are expanded into the trees. A -[`piecewise:`](../reference/language/piecewise.md) block stays one curve, and a -`sos:` block stays one set. Every description is there. - -Lowering builds the program once, while the spec loads. A spec in hand has -already passed every rule, and `spec.program` returns the same object on every -ask. - -| | `Spec` | `Program` | -| ---------------- | ------------------------ | ------------------------------------------- | -| An expression | the text the file wrote | a typed tree of nodes | -| A macro | its template | expanded into every tree that calls it | -| A curve | the block as written | one `PiecewiseDeclaration`, its links typed | -| A set | the block as written | one `SosDeclaration` | -| A description | as written | on each declaration | -| Written back out | `to_yaml()`, `to_dict()` | not at all: trees do not give the text back | - -## The rows - -A curve or a set stands for plain variables and constraints. -`spec.expand('piecewise')` writes each curve out as those rows, and -`spec.expand()` writes the sets out too. Each returns a new `Spec`, checked as -any other, with a program of its own. It is a different model from the one it -came from, and the two do not compare equal. The spec keeps no expansion, so a -caller that needs the rows twice holds the result. - -**Nothing in the package expands a model unasked.** Each tool reads the model -as it arrives. A caller that wants the rows asks for them, and -[see what a curve or a set expands to](../howto/see-an-expansion.md) shows how. +**A `Spec` is the file as written**, checked against every rule that needs no +data. **A `Program` is what the file means**: every name typed, every operator +resolved to a node, every macro expanded, and each `piecewise:` or `sos:` block +kept as one declaration. ## Which tool reads which -| Tool | Reads | Because | -| -------------------------- | -------------------------------------------------------------- | ------------------------------------------------------------- | -| The typesetter | `spec.program`, or a `Program` handed to it | it prints each curve as the curve the file states | -| `advice` | `spec.program` | its notes are about the model the author wrote | -| An engine that builds rows | `spec.expand('piecewise').program`, or `spec.expand().program` | a solver takes rows, and the engine knows which sets it takes | -| A tool that rewrites files | the `Spec` | only the spec holds the text and the macros | - -**The typesetter never reads the spec.** A `Program` handed to it prints the -same as the spec it came from. - -**A program's `footprint`, `separability` and `roots` describe the rows that -program holds.** A curve still on the program is not a row, so it counts once -it is written out. An engine asks these of the program of the expansion, which -is the one it builds. - -**`advice` reads a curve's links as the rows they state.** Its notes are -claims, and a curve that holds a variable keeps that variable out of the -unbounded note. So advice on the spec and advice on its expansion agree. +| Tool | Reads | Because | +| -------------------------- | ------------------------------------------- | ------------------------------------------------- | +| The typesetter | `spec.program`, or a `Program` handed to it | it prints each curve as the curve the file states | +| `advice` | `spec.program` | its notes are about the model the author wrote | +| An engine that builds rows | the program of `spec.expand()` | a solver takes rows | +| A tool that rewrites files | the `Spec` | only the spec holds the text and the macros | ## Why the split falls here @@ -82,13 +38,11 @@ unbounded note. So advice on the spec and advice on its expansion agree. all three, so no reader parses text again or reads two objects. - **The program keeps the model the author wrote.** A curve is one declaration to print and one to explain. Its rows are one formulation of it, so the rows - are a second model that a caller asks for. + are a second model, which a caller asks for with + [`spec.expand()`](../reference/api.md#math_spec.Spec.expand). - **The spec keeps the text.** A tool that rewrites a model needs the file as written: `to_yaml()` writes it back, and `expand()` rewrites it. A tree does not give the text back. - **The program does not hold its spec.** Nothing reads the file from a program, and two objects that own each other form a cycle. A tool handed a bare `Program` has the model, not the file. - -[Reading a loaded model](../reference/reading.md) is the reference for both -objects: their fields, the nodes, and the questions a program answers. diff --git a/docs/about/limits.md b/docs/about/limits.md index 5e5a1878..1c98b2ee 100644 --- a/docs/about/limits.md +++ b/docs/about/limits.md @@ -17,20 +17,17 @@ costs to add. - **A macro** is a template with arguments, written in the file under `macros:`. Adding one costs nothing: it uses only operators that exist, so no engine has - to change. Most requests turn out to be a macro - ([macros](../reference/language/named.md#macros)). + to change ([macros](../reference/language/named.md#macros)). - **A primitive** is an operator built into the language: `sum`, `sum_back`, - `at`, `shift`, and the `where` comparisons. A file cannot add one. Adding one - here is the expensive kind: every engine that builds models has to implement - it, and the typesetter has to print it in LaTeX, Typst and Markdown. + `at`, `shift`, and the `where` comparisons. Adding one is the expensive kind: + every engine that builds models has to implement it, and the typesetter has + to print it in LaTeX, Typst and Markdown. - **A formulation** is a block that states ordinary variables and constraints rather than being one. `piecewise:` and `sos:` are the two. It costs as much as - a primitive to build, but composes as freely as a macro. A formulation emits - variables and constraints, states what it assumes of the data as ordinary - assumptions, and emits no parameter — so the same data binds a model and its - expansion, and + a primitive to build, but composes as freely as a macro. It emits variables, + constraints and assumptions, and no parameter, so [`spec.expand()`](../reference/language/piecewise.md#writing-a-formulation-out) - needs no source a reader has to supply. + writes it out with the data the model already attaches. A request that is none of the three is refused, and the [table of refusals](#deliberate-non-primitives) records it with what to write @@ -41,14 +38,11 @@ instead. **A macro must be able to call it.** Everything a modeller might pass in goes in the value of a keyword argument, such as `over=snapshot`. -**An operator may read the whole table. It pays one full pass over the data.** -`sum(p, over=g)` reads one row per generator, and `shift(p, along=t, offset=1)` -reads the row before. Each reads a bounded number of rows per output row, so an -engine builds the model one chunk of rows at a time. An operator that reads -every row to produce one row costs one full pass before any chunk builds, and a -request for such an operator names that price. - -**An operator that calls itself is refused.** Nothing bounds how far it expands. +**An operator names its price in rows read.** `sum(p, over=g)` reads one row +per generator, and `shift(p, along=t, offset=1)` reads the row before. Each +reads a bounded number of rows per output row, so an engine builds the model +one chunk of rows at a time. An operator that calls itself is refused, because +nothing bounds how far it expands. | The operator | Allowed? | | ------------------------------------------------ | ----------------------------------------------- | @@ -59,11 +53,9 @@ request for such an operator names that price. | reads every row | yes, at one full pass before any chunk builds | | calls itself | no, and the message names what to write instead | -**Degree is not a third test.** `p * q` at one coordinate is a join of a table -with itself, so the objective and the constraints take it. A product of two -sums, `sum(x, over=i) * sum(y, over=j)`, is refused, because the file does not -say how many terms either sum has. `x[i] * y[j] * a[i, j]` is allowed, because -the table `a` says which pairs exist. +Degree is not a test for a new primitive. The +[product rule](../reference/language/expressions.md#where-a-product-of-two-variables-is-allowed) +holds for every operator. A new primitive is finished when lowering builds it, the typesetter prints it in all three formats, and an engine's build of a model that uses it matches @@ -71,63 +63,36 @@ the same model written out by hand. ### Three kinds of refusal -| The language refuses it because… | Examples | Can it change? | -| ------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------ | -| **one solver cannot take it** | indicator constraints; a quadratic constraint. `sos:` was in this group, and entered: a solver with sets takes it as one, and a model for a solver without is written out first | yes, solver by solver | -| **the file would stop being the artifact** | arbitrary Python, whose content no loader can check and no typesetter can print | no | -| **this project puts the work elsewhere** | data preparation such as resampling; helpers for one domain; Python that decides which declarations exist | it could; this project does not want it to | - -Three things never appear inside one model: an `if`, a loop, and a set of -declarations that depends on the data. A dimension computed before the model -loads is fine: a cycle basis for Kirchhoff's voltage law is a graph algorithm -run in data preparation, and its result arrives as a parameter. What no model -can hold is work that needs the solver's answer before it can write the next -row, such as cuts added during a solve. A tool can still loop over models: a -rolling horizon and Benders decomposition each build a model, solve it, and -build the next. - -### Solver capability - -Whether an engine can build the operator is one question. Whether a given -solver then accepts the result is a second one, and the language does not -answer it. If it did, one solver's limits would be written into the language, -and every other solver would inherit them. - -- HiGHS has no special-ordered sets. Gurobi does. An engine handing a model to - Gurobi passes the set through; one handing it to HiGHS refuses it, and the - author writes the set out with `spec.expand('sos')` first. -- A quadratic constraint is accepted by some solvers only when it is convex, - and convexity depends on the numbers, which the file does not have. - -So `sos:` entered the language on the first question alone. Each engine then -decides whether it takes a set, and the language decides what a set is written -out as. +| The language refuses it because… | Examples | Can it change? | +| ------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------ | +| **one solver cannot take it** | indicator constraints; a quadratic constraint ([solver capability](what-counts-as-language.md#what-each-tool-decides-for-itself)) | yes, solver by solver | +| **the file would stop being the artifact** | arbitrary Python, whose content no loader can check and no typesetter can print | no | +| **this project puts the work elsewhere** | data preparation such as resampling; helpers for one domain; Python that decides which declarations exist | it could; this project does not want it to | ## What counts as data preparation -From inside a model, a column you computed in pandas and a column the language -could have derived look the same: a parameter arrives, and a constraint reads -it. One sentence tells them apart: +A column computed in pandas and a column the language could derive look the +same inside a model. One sentence tells them apart: > Data preparation computes what the model cannot know. The language derives what > it can from data the model already has. A cycle basis is the first kind. It needs the network's topology, which only the data has, so `cycle_incidence` arrives as a parameter. A minimum up time is the -second kind. `min_up_time` is a column the model already binds, so +second kind. `min_up_time` is a column the model already attaches, so `sum_back(window=min_up_time)` reads the width off the column and you ship no window mask. Checking a column is neither. `p_min <= p_max` is a rule two consumers must not answer differently, so the rule is [language](../reference/language/assumptions.md) and the check is the -consumer's. The file states the predicate, and whoever binds the numbers runs +consumer's. The file states the predicate, and whoever attaches the numbers runs it. ## Deliberate non-primitives -What has been asked for and refused, with the reason and what to write instead. -That another tool has a feature is not by itself a reason to add it. +Each row is a request the language refuses, with the reason and what to write +instead. | Request | Why refused | Instead | | ------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | @@ -139,21 +104,5 @@ That another tool has a feature is not by itself a reason to add it. | `**` with a variable in the base or the exponent | the exponent would decide the degree, and `to_spec` reads no data | `x * x` for a square. `**` over parameters and numbers is allowed | | Normalisation, `x / sum(x)` | dividing by a variable is not a polynomial, and no solver takes it | write the ratio as a constraint, or fix the denominator | | An `if`, a loop, or declarations that depend on the data | `to_spec` could no longer read the file without the data | `where:` masks and `dims:` dimensions. A tool may loop over models | -| A Python API for building models | the model is the file you review and diff | YAML, or a `dict` with the same keys ([below](#composition-component-libraries)) | +| A Python API for building models | the model is the file you review and diff | YAML, or a `dict` with the same keys, merged before `to_spec` | | A `where` comparing a relation column against the dimension it maps into | the relation already pairs the two, and a mask over the pair is the same fact in a bigger shape | place the quantity with `sum(by=)`, or read it with `at(by=)` ([operators](../reference/language/operators.md#sum)) | - -## Composition (component libraries) - -A component library is a set of templates, such as a boiler, a battery and a -line, that agree on how ports and flows are named. You merge the templates you -need into one file, wire the components together with a connectivity table in -the data, and close the system with one `sum(by=)` balance. - -The topology is data. Adding a second battery is a row in a table, so the file -grows with the number of component _types_. - -Merging happens before `to_spec`. Every function here takes a `dict` as well as -a path, so a model assembled in Python is checked exactly as a file is, and -`Spec.to_yaml()` writes the file a reviewer reads. A `dict` may hold only what a -file may hold, so the file itself states no composition. A template names no -sibling, and no key says which fragment wins where two declare a `p`. diff --git a/docs/about/what-counts-as-language.md b/docs/about/what-counts-as-language.md index f0f52271..2a407302 100644 --- a/docs/about/what-counts-as-language.md +++ b/docs/about/what-counts-as-language.md @@ -54,11 +54,5 @@ So the boundary runs both ways: one, the rule goes into the language, once. - The language must not state a rule about what one tool can _build_. -A file that every tool accepts can still be a file that one engine cannot -build. Accepting and building are different steps. - -## How this differs from the limits - -[The limits](limits.md) answer a different question: which operators and blocks -may be added to the language at all. This page answers who decides a rule once -the operator or block exists. +[The limits](limits.md) say which operators and blocks may be added to the +language at all. diff --git a/docs/about/what-counts-as-public-api.md b/docs/about/what-counts-as-public-api.md index 6ede6a37..e2ba29c4 100644 --- a/docs/about/what-counts-as-public-api.md +++ b/docs/about/what-counts-as-public-api.md @@ -19,13 +19,8 @@ A function may join the public API when both of these hold: on a page of this reference, so somebody could rewrite the function in another language from the pages alone and get the same answer. -## Where a new feature lands - -When the language gained piecewise-linear curves, it gained a `piecewise:` key -in the YAML. A key in the file -shows up in a git diff, the typesetter prints it as math, and an engine written -in another language can read it. So wherever a feature can be a key in the -file, it is one. +Wherever a feature can be a key in the file, it is one: a key shows up in a git +diff, the typesetter prints it, and an engine in another language reads it. ## What every function keeps @@ -35,27 +30,10 @@ file, it is one. - **A value or an error, and nothing between.** `to_spec` either returns a `Spec` or raises an error that names the rewrite. `advice()` is separate: it talks about a file the language accepts, and changes nothing. -- **Safe to call again.** `spec.program` is one object, however often it is - asked for. -- **Nothing is written out unasked.** A `piecewise:` or `sos:` block is the - block until a caller writes it out with `spec.expand(...)`. No door, verb or - check expands a model on the caller's behalf. - [The file and the program](file-and-program.md) says which tool reads the - block and which reads the rows. An engine that writes curves out at its own - door makes that choice for its users, not for the language. - -## Three things a function never decides - -- What one solver or file format can take. That is the engine's question. -- How the numbers bind to the names. That is the engine's too. -- Which solver runs. - -## What this refuses - -| Asked for | Why | -| ------------------------------------------------ | ------------------------------------------------------------------- | -| A Python API for building models | The model is the file you review and diff | -| A hook, a callback, a registry, a plugin | Cannot be diffed, printed or read from another language | -| A function that binds data or calls a solver | Needs more than the file | -| A setting that changes what a file means | Two callers would read one file two ways | -| A function whose answer a declaration could give | A declaration can be diffed, printed and read from another language | +- **Nothing is written out unasked.** A `piecewise:` or `sos:` block stays the + block until a caller calls + [`spec.expand()`](../reference/api.md#math_spec.Spec.expand). + +What a solver or file format can take, how the numbers attach to the names, and +which solver runs are each engine's to decide +([what counts as language](what-counts-as-language.md#what-each-tool-decides-for-itself)). diff --git a/docs/contributing.md b/docs/contributing.md index 271bbed5..be4b4328 100644 --- a/docs/contributing.md +++ b/docs/contributing.md @@ -54,7 +54,7 @@ stale anchor fails it. `pixi run docs-serve` builds the site and serves it at ??? question "I have updated the README.md" The home page includes named sections of the README rather than a copy: the - badges, the model, the development install and the status note. A section + badges, the model and the status note. A section is delimited in the README by `:::md ` and `:::md `, and `docs/index.md` pulls it in with `:::md --8<-- "README.md:name"`. Edit inside the markers, and the site @@ -90,8 +90,9 @@ stale anchor fails it. `pixi run docs-serve` builds the site and serves it at - My Page: my-page.md ``` - The Python API pages are generated from the docstrings, so a new class or - module appears in the next build. + The module pages under Development are generated from the docstrings, so a + new module appears in the next build. A new public name also needs its own + `:::` entry on the [Python API](reference/api.md) page. ## Naming across the layers diff --git a/docs/examples/index.md b/docs/examples/index.md index e446a64c..d90386e5 100644 --- a/docs/examples/index.md +++ b/docs/examples/index.md @@ -14,9 +14,8 @@ Every model is a file under `examples/` in the repository. by region, so a single inequality covers both regimes. - [One construct per model](operators.md) declares each operator in the smallest file that can, and prints the equation beside it. -- [PyPSA in one file](pypsa.md) states the model `n.optimize()` builds, one - declaration at a time. PyPSA's name for each row sits beside the YAML and the - equation. -The math on these pages is printed by the typesetter from the file above it. See -[Typeset the math](../reference/typeset.md) to print your own. +The PyPSA parity pages, from [PyPSA in one file](pypsa.md) on, are a proof of +concept. They sit in the Development section. + +[Typeset the math](../reference/typeset.md) prints your own. diff --git a/docs/examples/pypsa.md b/docs/examples/pypsa.md index f36ed251..b3714973 100644 --- a/docs/examples/pypsa.md +++ b/docs/examples/pypsa.md @@ -25,7 +25,7 @@ family PyPSA numbers per segment or scenario. Each rung's banner states what PyPSA solved its reference network to. -> Every rung's network is `spine.build()` plus the rung's own `n.add` calls, data inline; a keyword not passed is PyPSA's default. A banner states what PyPSA solved the rung to; how an engine binds the network to the file, and what it makes of it, is that engine's own record. +> Every rung's network is `spine.build()` plus the rung's own `n.add` calls, data inline; a keyword not passed is PyPSA's default. A banner states what PyPSA solved the rung to; how an engine attaches the network to the file, and what it makes of it, is that engine's own record.
The shared spine, spine.py diff --git a/docs/first-model.md b/docs/first-model.md new file mode 100644 index 00000000..810e5bd5 --- /dev/null +++ b/docs/first-model.md @@ -0,0 +1,201 @@ + + +# Your first model + +In this lesson you write a least-cost dispatch model one block at a time, check +it, and print it as math. [Install math-spec](howto/installation.md) first. + +## Dimensions + +Make a file `dispatch.yaml` with a description and two +[dimensions](reference/language/dimensions.md), the axes the model runs over: + +```yaml title="dispatch.yaml" +description: Least-cost dispatch of a generator fleet against an hourly load. + +dimensions: + snapshot: { dtype: int, description: dispatch periods } + generator: { description: generating units } +``` + +Check the file: + +```bash +python -m math_spec check dispatch.yaml +``` + +The check accepts the file, and advises that nothing uses the dimensions yet: + +```text +dimension 'snapshot' is never used: nothing is indexed by it, nothing aggregates into it, and no relation has a column over it. Remove it — or keep it knowingly, if the declarations that use it are still to be written. +dimension 'generator' is never used: nothing is indexed by it, nothing aggregates into it, and no relation has a column over it. Remove it — or keep it knowingly, if the declarations that use it are still to be written. +``` + +## Parameters and a variable + +Add three [parameters](reference/language/declarations.md#parameters), the data +the model expects, and one [variable](reference/language/declarations.md#variables), +the decision the solver makes. The `where:` line leaves out every generator with +no capacity. + +```yaml +parameters: + capacity: { dims: [generator], description: installed capacity } + load: { dims: [snapshot], description: demand to be met } + cost: { dims: [generator], description: marginal cost } + +variables: + dispatch: + description: output of a generator in a snapshot + dims: [snapshot, generator] + where: "capacity > 0" + bounds: { lower: 0, upper: capacity } +``` + +Print the math. `--no-legend` leaves out the tables of symbols: + +```bash +python -m math_spec markdown --no-legend dispatch.yaml +``` + +!!! example "Rendered output" + + Least-cost dispatch of a generator fleet against an hourly load. + + #### Variable domains + + **`dispatch`** + + ```math + 0 \le \mathit{dispatch}_{t,g} \le \mathrm{capacity}_{g} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} \,:\, \mathrm{capacity}_{g} > 0 + ``` + +## Constraint and objective + +Add one [constraint](reference/language/declarations.md#constraints), which +meets the load in every snapshot, and the +[objective](reference/language/declarations.md#objective): + +```yaml +constraints: + power_balance: + dims: [snapshot] + expression: sum(dispatch, over=generator) == load + +objective: + sense: minimize + expression: sum(dispatch * cost) +``` + +Check the file again. The check prints nothing and exits with status 0. + +??? note "The whole file" + + ```yaml title="dispatch.yaml" + description: Least-cost dispatch of a generator fleet against an hourly load. + + dimensions: + snapshot: { dtype: int, description: dispatch periods } + generator: { description: generating units } + + parameters: + capacity: { dims: [generator], description: installed capacity } + load: { dims: [snapshot], description: demand to be met } + cost: { dims: [generator], description: marginal cost } + + variables: + dispatch: + description: output of a generator in a snapshot + dims: [snapshot, generator] + where: "capacity > 0" + bounds: { lower: 0, upper: capacity } + + constraints: + power_balance: + dims: [snapshot] + expression: sum(dispatch, over=generator) == load + + objective: + sense: minimize + expression: sum(dispatch * cost) + ``` + +## An undeclared name + +Change `load` to `loads` in the constraint, and check the file. The check +refuses it and exits with status 1: + +```text +Constraint 'power_balance': 'loads' not found. + Variables: ['dispatch'] + Parameters: ['capacity', 'cost', 'load'] +Check for typos, or ensure 'loads' is declared. +``` + +Change `loads` back to `load`. + +## The math + +Print the whole model: + +```bash +python -m math_spec markdown dispatch.yaml +``` + +!!! example "Rendered output" + + Least-cost dispatch of a generator fleet against an hourly load. + + #### Sets + + | Symbol | Meaning | + |---|---| + | $`\mathcal{T}`$ | index $`t`$ — `snapshot` — dispatch periods | + | $`\mathcal{G}`$ | index $`g`$ — `generator` — generating units | + + #### Parameters + + | Symbol | Meaning | + |---|---| + | $`\mathrm{capacity}`$ | `capacity` over $`\mathcal{G}`$ — installed capacity | + | $`\mathrm{load}`$ | `load` over $`\mathcal{T}`$ — demand to be met | + | $`\mathrm{cost}`$ | `cost` over $`\mathcal{G}`$ — marginal cost | + + #### Variables + + | Symbol | Meaning | + |---|---| + | $`\mathit{dispatch}`$ | `dispatch` over $`\mathcal{T} \times \mathcal{G}`$ — output of a generator in a snapshot | + + Upright is what the model is given — a parameter such as $`\mathrm{capacity}`$, a coordinate map, a label — and italic is what the solver chooses, such as $`\mathit{dispatch}`$. An index is italic too, being what a quantifier chooses, and a set is script. + + #### Objective + + ```math + \min \sum_{t \in \mathcal{T},\ g \in \mathcal{G}} \mathit{dispatch}_{t,g} \cdot \mathrm{cost}_{g} + ``` + + #### Subject to + + **`power_balance`** + + ```math + \sum_{g \in \mathcal{G}} \mathit{dispatch}_{t,g} = \mathrm{load}_{t} \qquad \forall\, t \in \mathcal{T} + ``` + + #### Variable domains + + **`dispatch`** + + ```math + 0 \le \mathit{dispatch}_{t,g} \le \mathrm{capacity}_{g} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} \,:\, \mathrm{capacity}_{g} > 0 + ``` + +## Where to next + +- [The language](reference/language/index.md) gives every rule a file obeys. +- [Examples](examples/index.md) shows larger models beside the math they print. +- [Print a model as math](howto/print.md) prints LaTeX and Typst. diff --git a/docs/howto/check.md b/docs/howto/check.md index 96a506ff..a4dbc4cc 100644 --- a/docs/howto/check.md +++ b/docs/howto/check.md @@ -21,12 +21,10 @@ machine and in CI. ``` Advice prints on stdout and exits with status 0. A model the language - accepts with nothing to advise prints nothing. A `piecewise:` or `sos:` - block is read as the rows it states, so the answer is the one its - expansion gets, with nothing expanded. + accepts with nothing to advise prints nothing. ```text - Variable 'slack' makes this model unbounded: no constraint names it, and bounds.lower is -inf, which is the direction a +slack term improves a minimize objective in. No data can change that, so the solve would answer `unbounded` and name nothing. + Variable 'slack' makes this model unbounded: no constraint names it, and bounds.lower is open, which is the direction a +slack term improves a minimize objective in. No data can change that, so the solve would answer `unbounded` and name nothing. Give it a finite bounds.lower, or the constraint that was meant to define it. ``` @@ -38,8 +36,9 @@ machine and in CI. ``` 3. **Ask from Python** where the check is one step of a longer script. - `to_spec` raises a `MathSpecError` for anything the language refuses, and - `advice` returns what it would print: + [`to_spec`](../reference/api.md#loading) raises a `MathSpecError` for + anything the language refuses, and [`advice`](../reference/api.md#advice) + returns what it would print: ```python import math_spec as ms diff --git a/docs/howto/curve-by-hand.md b/docs/howto/curve-by-hand.md index caa05314..a24d58e2 100644 --- a/docs/howto/curve-by-hand.md +++ b/docs/howto/curve-by-hand.md @@ -26,7 +26,7 @@ file, so it cannot say this. The formulation written out can. ```yaml sos: - on_one_segment: { variable: weight, over: bp, type: 2 } + on_one_segment: { variable: weight, along: bp, type: 2 } ``` 3. **Write the convexity row, and one row per flow.** The row per flow is where diff --git a/docs/howto/declare-a-column.md b/docs/howto/declare-a-column.md index 12c8c173..8b0fd62a 100644 --- a/docs/howto/declare-a-column.md +++ b/docs/howto/declare-a-column.md @@ -11,16 +11,16 @@ Decide whether a column of your data is a [parameter](../reference/language/declarations.md#parameters). What decides is what the math does with the column. -| The column… | is declared as | because | -| ------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- | -| is an axis: something is indexed by it, or an aggregation lands terms on it | a `dimension` | its members are the coordinate set every table over it is reindexed onto | -| has one value per member of a dimension, or per tuple of several — a generator's bus, a line's two ends, a generator's zone by period | a `relation` with that `key` | it is a map every operator reads, and its values are checked against the dimensions they name | -| relates members of two dimensions many-to-many, with nothing to weigh — which buses a generator may connect to | a bare `relation`, with no `values:` | `sum` reads it with both ends named, and a bare `where` tests it. Nothing reads it, because there is no one value to read | -| relates members of two dimensions many-to-many, with a weight per pair — a link's efficiency to each bus, a cycle's lines | a `parameter` over both | the weight is the data, its row set is the relation, and the aggregation is `sum(w * x, over=a)` | -| is a label set the model only selects on or counts within — a period, a season, a zone | a `dimension`, and a `relation` onto it | its labels are checked, at the cost of one line and one table | -| scales terms — a coefficient, a bound, an offset | a `parameter` (`float` or `int`) | arithmetic is over numbers ([dtype](../reference/language/declarations.md#parameters)) | -| is a per-row attribute the math only selects on — a fuel, a constraint's sense | a `str` parameter | it names rows rather than scaling them, and no set is declared to check its values against | -| is a mask | a `bool` parameter | a bare name in a `where` is its own answer | +| The column… | is declared as | +| ------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------- | +| is an axis: something is indexed by it, or an aggregation lands terms on it | a `dimension` | +| has one value per member of a dimension, or per tuple of several — a generator's bus, a line's two ends, a generator's zone by period | a `relation` with that `key` | +| relates members of two dimensions many-to-many, with nothing to weigh — which buses a generator may connect to | a bare `relation`, with no `values:` | +| relates members of two dimensions many-to-many, with a weight per pair — a link's efficiency to each bus, a cycle's lines | a `parameter` over both | +| is a label set the model only selects on or counts within — a period, a season, a zone | a `dimension`, and a `relation` onto it | +| scales terms — a coefficient, a bound, an offset | a `parameter` (`float` or `int`) | +| is a per-row attribute the math only selects on — a fuel, a constraint's sense | a `str` parameter | +| is a mask | a `bool` parameter | Two rules decide the cases the table does not list: diff --git a/docs/howto/installation.md b/docs/howto/installation.md index 1e743f2d..c68e0d37 100644 --- a/docs/howto/installation.md +++ b/docs/howto/installation.md @@ -5,46 +5,15 @@ SPDX-License-Identifier: CC-BY-4.0 # Installation -!!! warning "Not published yet" +`math-spec` needs Python 3.12 or above. Nothing is published yet, so install it +from git: - math-spec is on the alpha stream, and nothing is published yet. The - commands below are what the first release will look like. Until then, - install from a checkout or a git reference. - -`math-spec` needs Python 3.12 or above. Install it into a dedicated -environment: - -=== "pixi" - - ``` bash - pixi add --pypi math_spec - ``` - -=== "uv" - - ``` bash - uv add math_spec - ``` - -=== "conda" - - ``` bash - conda create -n math-spec "python>=3.12" "pip" - conda activate math-spec - pip install math_spec - ``` - -=== "pip" - - ``` bash - pip install math_spec - ``` - -To develop against a clone instead: - ---8<-- "README.md:docs-install-dev" +```bash +pip install git+https://github.com/energy-models/math-spec +``` -[Contributing](../contributing.md) has the rest. +To develop against a clone instead, follow +[contributing](../contributing.md#setting-up-a-development-environment). ## Editor completion and offline checking diff --git a/docs/howto/pin-a-variable.md b/docs/howto/pin-a-variable.md index 8eb01c0d..adfc3ab0 100644 --- a/docs/howto/pin-a-variable.md +++ b/docs/howto/pin-a-variable.md @@ -26,7 +26,7 @@ between the two. The data does. 2. **Write every rule against the variable.** `rate - relmax * size <= 0` is one equation whether `size` is chosen or given. -3. **Pin it in the data where it is given.** Bind `size_min` and `size_max` to +3. **Pin it in the data where it is given.** Attach `size_min` and `size_max` as the same value for a plant whose size is fixed. Equal bounds pin a variable ([variables](../reference/language/declarations.md#variables)). diff --git a/docs/howto/print.md b/docs/howto/print.md index d94bc409..f50f02ce 100644 --- a/docs/howto/print.md +++ b/docs/howto/print.md @@ -8,8 +8,7 @@ SPDX-License-Identifier: CC-BY-4.0 Turn a model file into the math a paper would print, from the file alone, and keep the document current as the file changes. -1. **Print Markdown first** and read it. It is the quickest way to see that - the YAML says what you meant: +1. **Print Markdown first** and read it: ```bash python -m math_spec markdown model.yaml @@ -47,9 +46,7 @@ keep the document current as the file changes. `notation: typst` or none. Without `--standalone` the output is a fragment to `\input` or `#include` into a paper. -4. **Print the rows a solver holds** with `--expand`, where the model states a - curve or a set and the reader wants the formulation rather than the - construct: +4. **Print the rows a curve or a set states** with `--expand`: ```bash python -m math_spec markdown model.yaml --expand @@ -65,6 +62,5 @@ keep the document current as the file changes. python -m math_spec latex $< --symbols model.symbols.yaml --standalone -o $@ ``` -The options each renderer takes, what a symbol table may say, and how one -declaration is printed on its own are under -[typeset the math](../reference/typeset.md). +[Typeset the math](../reference/typeset.md) lists every option and what a +symbol table may say. diff --git a/docs/howto/regimes.md b/docs/howto/regimes.md index 538df345..d1108859 100644 --- a/docs/howto/regimes.md +++ b/docs/howto/regimes.md @@ -81,8 +81,7 @@ the recipe needs no second model file. expression: dispatch <= available ``` - The loader proves at load that no two cases can hold at one coordinate, - and `otherwise:` takes every coordinate they leave. + `otherwise:` takes every coordinate the cases leave. 4. **Check it** with `python -m math_spec check model.yaml`. A pair of masks that can both hold, or a case with no `otherwise:`, is refused there with diff --git a/docs/howto/see-an-expansion.md b/docs/howto/see-an-expansion.md index 8a90a4d0..8b72c2b7 100644 --- a/docs/howto/see-an-expansion.md +++ b/docs/howto/see-an-expansion.md @@ -12,8 +12,8 @@ concept of a set. ## 1. Write the formulation out -`expand()` returns the same math with its formulations stated as plain -declarations. `to_yaml()` prints the result as a file. +`expand()` writes each formulation out as plain declarations, and `to_yaml()` +prints the result as a file. === "Python" @@ -35,10 +35,9 @@ The command line prints the expansion as math rather than as YAML. Pass ## 2. Read a set -The `sos:` block below says that at most one `p` is nonzero. Its expansion -adds one binary per member, a row that picks at most one binary, and a row that -holds an unpicked member at zero. The coefficient `10.0` is the upper bound of -`p`. +Compare the tabs. The `sos:` block below says that at most one `p` is nonzero. +[What a set is written out as](../reference/language/piecewise.md#what-a-set-is-written-out-as) +names each row the expansion adds. @@ -138,21 +137,11 @@ holds an unpicked member at zero. The coefficient `10.0` is the upper bound of -Every name the expansion adds starts with the name of the block, so `pick_seg` -is the binary of the set `pick`. - ## 3. Read a curve -The `piecewise:` block below ties `x` and `y` to a curve through the -breakpoints in `x_bp` and `y_bp`. A `method: sos2` curve states a set, so it -writes out in two steps. Compare the tabs from left to right: - -- **`expand('piecewise')` writes the curve out and leaves its set.** It adds a - weight per breakpoint and one link row per tied variable. An `sos:` block - over the weights keeps at most two neighbouring weights nonzero. -- **`expand()` writes the set out too.** The `sos:` block becomes one binary - per segment and the rows that keep the two nonzero weights next to each - other. +Compare the tabs from left to right. `expand('piecewise')` writes the +`method: sos2` curve below out and leaves the set it states. `expand()` writes +the set out too. @@ -257,7 +246,7 @@ writes out in two steps. Compare the tabs from left to right: description: >- piecewise 'curve': every breakpoint the curve runs through needs a row in 'x_bp', 'y_bp' — a missing row is read as a zero rather than as a shorter - curve, so it sits the curve on the origin. Bind the rows, or declare + curve, so it sits the curve on the origin. Attach the rows, or declare points: to say how far the curve runs. ``` @@ -364,7 +353,7 @@ writes out in two steps. Compare the tabs from left to right: description: >- piecewise 'curve': every breakpoint the curve runs through needs a row in 'x_bp', 'y_bp' — a missing row is read as a zero rather than as a shorter - curve, so it sits the curve on the origin. Bind the rows, or declare + curve, so it sits the curve on the origin. Attach the rows, or declare points: to say how far the curve runs. ``` @@ -439,7 +428,7 @@ writes out in two steps. Compare the tabs from left to right: -The [`assumptions:`](../reference/language/assumptions.md) rows state what -the curve needs of its data. What -`expand()` accepts, and what each `method:` emits, is under -[piecewise curves and SOS](../reference/language/piecewise.md#writing-a-formulation-out). +[`Spec.expand()`](../reference/api.md#math_spec.Spec.expand) lists what +the call accepts, and +[writing a formulation out](../reference/language/piecewise.md#writing-a-formulation-out) +says what each block emits. diff --git a/docs/index.md b/docs/index.md index 46c5afeb..5edb449f 100644 --- a/docs/index.md +++ b/docs/index.md @@ -175,6 +175,8 @@ call. ## Where to next +- [Your first model](first-model.md): write the file above one block at a + time, check it and print it. - [The language](reference/language/index.md): what a file may contain, and what it means. - [Examples](examples/index.md): whole models, each beside the math it prints. @@ -186,10 +188,8 @@ call. ## Install it ---8<-- "README.md:docs-install-dev" - -Or as a dependency, once the project leaves the alpha stream. See -[installation](howto/installation.md) for every package manager. +Nothing is published yet. [Installation](howto/installation.md) gives the +command that installs from git. !!! warning "Alpha, pre-1.0" diff --git a/docs/reference/api.md b/docs/reference/api.md new file mode 100644 index 00000000..c76f5ee2 --- /dev/null +++ b/docs/reference/api.md @@ -0,0 +1,134 @@ + + +# Python API + +This page documents every name that `import math_spec` exports, grouped by +task. + + + +## Loading + +The module `math_spec.program` holds the node and declaration classes of a +loaded model. [Reading a loaded model](reading.md) documents them. + +::: math_spec.to_spec + options: + show_root_heading: true + show_root_toc_entry: true + heading_level: 3 + +::: math_spec.Spec + options: + show_root_heading: true + show_root_toc_entry: true + heading_level: 3 + +## Typesetting + +::: math_spec.to_latex + options: + show_root_heading: true + show_root_toc_entry: true + heading_level: 3 + +::: math_spec.to_typst + options: + show_root_heading: true + show_root_toc_entry: true + heading_level: 3 + +::: math_spec.to_markdown + options: + show_root_heading: true + show_root_toc_entry: true + heading_level: 3 + +::: math_spec.typeset + options: + show_root_heading: true + show_root_toc_entry: true + heading_level: 3 + +::: math_spec.typeset_declaration + options: + show_root_heading: true + show_root_toc_entry: true + heading_level: 3 + +::: math_spec.FORMATS + options: + show_root_heading: true + show_root_toc_entry: true + heading_level: 3 + +::: math_spec.SymbolTable + options: + show_root_heading: true + show_root_toc_entry: true + heading_level: 3 + +## Advice + +::: math_spec.advice + options: + show_root_heading: true + show_root_toc_entry: true + heading_level: 3 + +::: math_spec.Advice + options: + show_root_heading: true + show_root_toc_entry: true + heading_level: 3 + +::: math_spec.AdviceKind + options: + show_root_heading: true + show_root_toc_entry: true + heading_level: 3 + +## Errors + +::: math_spec.MathSpecError + options: + show_root_heading: true + show_root_toc_entry: true + heading_level: 3 + +::: math_spec.LanguageError + options: + show_root_heading: true + show_root_toc_entry: true + heading_level: 3 + +::: math_spec.SchemaError + options: + show_root_heading: true + show_root_toc_entry: true + heading_level: 3 + +::: math_spec.DimensionError + options: + show_root_heading: true + show_root_toc_entry: true + heading_level: 3 + +## Names + +::: math_spec.BUILTIN_NAMES + options: + show_root_heading: true + show_root_toc_entry: true + heading_level: 3 + +::: math_spec.did_you_mean + options: + show_root_heading: true + show_root_toc_entry: true + heading_level: 3 + + diff --git a/docs/reference/glossary.md b/docs/reference/glossary.md new file mode 100644 index 00000000..ad451ac4 --- /dev/null +++ b/docs/reference/glossary.md @@ -0,0 +1,103 @@ + + +# Glossary + +This page defines the words these docs use in a fixed sense and that no single +reference page owns. A construct, such as a parameter or a macro, is defined on +its [language page](language/index.md). + +## The file and what reads it + +**Spec** +: The file as written, checked: what `to_spec` returns +([reading a loaded model](reading.md#spec-and-program)). + +**Program** +: What the file means, `spec.program`: every name typed, every macro expanded, +every operator a node. + +**Load** +: What `to_spec` does. "Refused at load" means `to_spec` raises, before any +data exists. + +**Attach** +: What a consumer does when it puts data on a program. A rule about numbers can +be checked only then, and the language checks none itself. The docs never say +"bind" for it, so that **bound** means one thing: a lower or upper limit on a +variable ([variables](language/declarations.md#variables)). + +**Consumer** +: A tool that reads a spec: an **engine** that attaches data and builds the rows a +solver takes, a **renderer** such as the typesetter, or a **checker** +([what counts as language](../about/what-counts-as-language.md)). + +**Declaration** +: One named entry under one of the top-level keys: one dimension, one +parameter, one constraint ([file shape](language/file.md)). + +## Coordinates + +**Label** +: One member of a dimension, `wind` say. The labels arrive with the data, in +the order that `shift`, `sum_back` and `position()` count along. + +**Coordinate** +: One point of a declaration's dimensions: one generator in one snapshot. A +variable has one column at each coordinate it is built at, and a constraint +has one row. + +**Frame** +: A declaration's own dimensions. An expression, a mask and a bound parameter +must fit inside the frame they sit in +([how dimensions combine](language/expressions.md#how-dimensions-combine)). + +**Group** +: The labels that one value of a relation column collects. `within=` keeps a +`shift`, a `sum_back` or a `position()` inside each group. + +## Masks and absence + +**Mask** · **predicate** +: A predicate is a true-or-false expression in the +[where grammar](language/expressions.md#where-strings). A mask is a predicate +on a declaration, and the coordinates it admits. + +**Absence** +: No value at a coordinate. A masked-out variable has no column there, and a +row that reads it is not built ([absence](language/absence.md)). + +**Missing row** +: A coordinate that a parameter's table has no row for. It is not absence: it +reads as `0` in arithmetic and as false in a `where` +([what creates absence](language/absence.md#what-creates-absence)). + +## Kinds of construct + +**Primitive** +: A construct built into the language, which every engine implements and the +typesetter prints: the operators and the `where` comparisons. + +**Formulation** +: A block that states ordinary variables and constraints rather than being +one: `piecewise:` and `sos:` ([piecewise curves and SOS](language/piecewise.md)). + +A request for a new construct is a macro, a primitive or a formulation, or it +is refused ([how a new construct enters](../about/limits.md#how-a-new-construct-enters)). + +## Words with two senses + +These words mean two things in these docs. The sentence around each one says +which. + +| Word | One sense | The other sense | +| -------- | ------------------------------------------------------- | ------------------------------------------------------------- | +| row | a constraint at one coordinate | one line of a parameter's or a relation's table | +| column | a variable at one coordinate | one column of a data table or a relation | +| set | an `sos:` entry | the set symbol of a dimension, $\mathcal{G}$, in the legend | +| regime | one case of a [`cases:`](language/named.md#cases) block | one of two constraints, each under its own `where:` | +| domain | a variable's `continuous`, `integer` or `binary` | the rows that hold a curve's link inside its breakpoint range | +| program | `spec.program`, the typed model | a linear or quadratic program, the problem a solver takes | +| the rows | the constraint rows of a model | the expanded model: the spec a formulation is written out as | diff --git a/docs/reference/language/absence.md b/docs/reference/language/absence.md index 196539eb..f4d1a755 100644 --- a/docs/reference/language/absence.md +++ b/docs/reference/language/absence.md @@ -6,8 +6,7 @@ SPDX-License-Identifier: CC-BY-4.0 # Absence and `where` A `where:` does not set a variable to zero. It leaves the variable **unbuilt** -at the masked coordinates: no column, and no value. Every rule on this page -follows from that. +at the masked coordinates: no column, and no value. ```yaml dimensions: @@ -23,8 +22,7 @@ variables: With `capacity = {wind: 10, gas: 5, old: 0}`, the model has `dispatch[wind]` and `dispatch[gas]`. There is no `dispatch[old]`. -The [grammar](expressions.md#where-strings) says what a `where:` may contain. -This page says what the mask means for the rows that are built. +The [grammar](expressions.md#where-strings) says what a `where:` may hold. ## What creates absence @@ -41,7 +39,8 @@ in a `where`. Where no such value exists, loading is refused. There are four such positions: a divisor, a `bounds:` entry, the whole constant side of a comparison, and a -[`piecewise:`](piecewise.md) breakpoint. +[`piecewise:`](piecewise.md) breakpoint. For a bound only where the data has +one, supply the bound in the data, where `inf` is a value, or mask the variable. ## How absence travels @@ -64,9 +63,8 @@ constraints: expression: sum(x, over=g) + sum(y, over=g) >= 1 # x[old] is back in ``` -`each` has no row at `old`. `total` sums the summand wherever the summand -exists, so `x[old]` goes away with `y[old]`. `split` sums each operand over its -own domain, so `x[old]` counts. The two are different constraints. +`total` sums the summand wherever the summand exists. `split` sums each +operand over its own domain. The two are different constraints. Beside a parameter, the rule reads the other way: @@ -91,12 +89,9 @@ there instead, write `where: rel_max` on the constraint. ## What a missing coordinate means -By default a masked coordinate has **no value**. A store that is not there has no -state of charge, so a row that needs that state is not built. - -Some quantities are **zero** outside their mask. A reservoir with no inflow spills -nothing, and a model like that wants its row. The variable says which reading -applies: +By default a masked coordinate has **no value**, and a row that needs it is not +built. Some quantities are **zero** outside their mask, and the variable says +which reading applies: ```yaml variables: @@ -121,26 +116,12 @@ a storage with inflow and no store, there is no row. ## Rows with no variable terms A missing parameter row can leave a row with nothing to decide, such as -`0 == load` at a bus with no generator. Such a row is not built, and the engine -reports it. An expression that names no variable _in the file_ is refused at +`0 == load` at a bus with no generator. Such a row is not built. An expression that names no variable _in the file_ is refused at load. ## Reported values -A [reported expression](named.md#reported-expressions) is arithmetic over -solved numbers, and it inherits their absence by the rule above. Through -arithmetic, a null spreads: `cost / delivered` has no value wherever either -operand is masked. Out of a summing operator, it does not. - -A quotient whose divisor solved to zero is absent in the same way. `dual(c)` has -no value at a row that `c`'s `where:` leaves unbuilt. - -## Asking for the opposite reading - -| You want | You write | -| ---------------------------------------------- | -------------------------------------------------------------------------------------------- | -| the row kept, the masked variable read as zero | `absence: zero` on the variable | -| the row dropped where a parameter has no data | `where: capacity` on the constraint | -| a vacated shift position to contribute | `shift(x, along=d, offset=n, edge=0)` | -| to test whether a variable exists here | its bare name in a `where` | -| a bound only where the data has one | supply the bound, because `inf` is a value, or mask the variable. These are different models | +A [reported expression](named.md#reported-expressions) inherits the absence of +the solved numbers it reads, by the rules above. A quotient whose divisor solved +to zero is absent too. A deleted row has +[no dual](named.md#reading-a-constraints-dual). diff --git a/docs/reference/language/assumptions.md b/docs/reference/language/assumptions.md index 23e84655..e3952034 100644 --- a/docs/reference/language/assumptions.md +++ b/docs/reference/language/assumptions.md @@ -5,11 +5,9 @@ SPDX-License-Identifier: CC-BY-4.0 # Assumptions -`assumptions:` states what the model expects of the data it is bound to. The -language reads no data, so it checks nothing here. It types the predicate, -carries it on the program, and prints it in the -[typeset document](../typeset.md). The consumer that binds the numbers runs -each one, and refuses the data that fails it. +`assumptions:` states what the model expects of the data attached to it. The +language types each predicate and prints it in the +[typeset document](../typeset.md). The consumer that attaches the numbers runs it. ```yaml dimensions: @@ -48,9 +46,6 @@ predicate. `bounds_do_not_cross: "p_min <= p_max"` above is the short form of `bounds_do_not_cross: { holds: "p_min <= p_max" }`. -A `description:` says why the rule is there. The sentence a consumer refuses -with quotes it, so a failure names the columns and the reason. - There is no `dims:`. The predicate holds at every coordinate of the product of the dimensions its two masks name. A predicate narrower than that broadcasts, as it does in any `where`. @@ -90,14 +85,12 @@ assumptions: description: the first snapshot has no predecessor to ramp from ``` -A `where:` narrows which coordinates are checked. A parameter supplied only -where it applies takes one, so the rows it has no value at are not held to the -predicate. +A parameter supplied only where it applies takes a `where:`, so the rows it has +no value at are not checked. ## What the loader refuses -**A predicate the connectives already decide.** It reads no data, so it is -either a claim about nothing or a claim no data can meet: +**A predicate the connectives already decide:** > `Assumption 'sound'`: the predicate `'c > 0 OR true'` folds to true, so it > assumes nothing of the data. Delete it, or name a parameter it constrains. @@ -105,27 +98,28 @@ either a claim about nothing or a claim no data can meet: A `where:` the connectives decide is refused the same way: one that folds to true narrows nothing, and one that folds to false checks the entry on no row. -**A variable.** An assumption is about the numbers the caller binds, and a -variable is what the solver decides from them: +**A variable:** > `Assumption 'sound'`: variable `'p'` stands in what the assumption assumes, > and an assumption is about the data — a variable is what the solver decides > from it. Name a parameter, or state the rule as a constraint. -A rule that binds a decision is a [constraint](declarations.md#constraints). A constraint whose sides carry no variable is refused, and its message names this section. ## What a curve assumes -A [`piecewise:`](piecewise.md) block puts its own conditions on the numbers. -Its breakpoints increase along the curve, and the shape is the one its -`method:` is exact for. The language derives both from the method and the sign -on its links, not from anything else the file writes, and carries them beside -the written ones under the name a refusal quotes. A `method: convex` block -called `curve` adds `curve_increasing` and `curve_curvature`. +A [`piecewise:`](piecewise.md) block `curve` adds its own assumptions, derived +from its `method:`, its `points:` and the sign on its links. They print under +the same _Assumptions_ heading as the written ones. + +| Entry | Added for | Holds | +| ------------------- | ----------------- | ----------------------------------------------------------------------------------------------------- | +| `curve_complete` | every block | every values parameter has a row at every breakpoint the curve runs through | +| `curve_increasing` | `convex`, `lp` | the pinned link's breakpoints (the first link's, when both are pinned) strictly increase along `over` | +| `curve_curvature` | `convex`, `lp` | with a `>=` link the curve is convex, with `<=` concave; with both links pinned it bends one way only | +| `curve_breakpoints` | `lp` | each curve has at least two breakpoints | +| `curve_contiguous` | a block `points:` | the marked breakpoints are one consecutive run of at least one | -Both kinds print under one _Assumptions_ heading, because a reader checking -the data against the document checks all of them. [Reading a loaded model](../reading.md#what-the-data-has-to-satisfy) says how a consumer runs them. diff --git a/docs/reference/language/declarations.md b/docs/reference/language/declarations.md index 4414b405..bcf388d5 100644 --- a/docs/reference/language/declarations.md +++ b/docs/reference/language/declarations.md @@ -10,8 +10,7 @@ text that the [typeset](../typeset.md#descriptions) legend prints. ## `parameters` -A parameter declares a shape and nothing more. The engine that builds the model -supplies the numbers, by name, from its own tables. +A parameter declares a shape. The numbers arrive by name with the data. ```yaml dimensions: @@ -73,17 +72,11 @@ variables: | `absence` | `undefined` or `zero`: what a masked-out coordinate means ([absence](absence.md#what-a-missing-coordinate-means)) | default `undefined` | | `description` | free text | default `null` | -!!! warning "A bound you omit leaves the variable unbounded on that side" - - You write non-negativity. The language does not assume it. - -An open side is `null`, as every other field a file may leave open is. A bound -is never infinite: `.inf` and `-.inf` are refused, with `null` named as the -rewrite. +An open side is `null`. A bound is never infinite: `.inf` and `-.inf` are +refused, with `null` named as the rewrite. A bound is a name or a number: `upper: capacity` is accepted, -and `upper: -rating` is refused. Ship the negated column as data. The dimensions of -a bound parameter are a subset of the variable's. +and `upper: -rating` is refused. Ship the negated column as data. Equal bounds pin a variable ([fix a quantity](../../howto/pin-a-variable.md)). A pinned variable is still a variable. @@ -119,8 +112,7 @@ The dimensions of the expression must **equal** its `dims` At least one side of the comparator carries a variable. A comparison between numbers and parameters alone is refused at load. -`dims: []` gives one scalar row, for a rule such as a system-wide budget. A -scalar variable may not carry a `where`; put the condition on the constraints +`dims: []` gives one scalar row. A scalar variable may not carry a `where`; put the condition on the constraints that use it. Two regimes of one rule are two blocks, each under its own `where:` diff --git a/docs/reference/language/errors.md b/docs/reference/language/errors.md index e13b0411..579cbdb6 100644 --- a/docs/reference/language/errors.md +++ b/docs/reference/language/errors.md @@ -7,15 +7,11 @@ SPDX-License-Identifier: CC-BY-4.0 ## What `to_spec` checks -`to_spec` binds no data. Before it returns a `Spec`, it parses the file, +`to_spec` attaches no data. Before it returns a `Spec`, it parses the file, resolves every name, checks every dimension rule and every degree, and reads every `where` string and every macro template, including the templates that -nothing calls. A `piecewise:` block is checked as written, against every rule -its expansion would be held to, and stays a block. - -Anything the language refuses is refused there, so a repository of models -validates in CI with no data and no solver. An array that does not bind, or a -solver exception, comes from the tool that builds and solves the model. +nothing calls. A `piecewise:` block is checked against every rule its expansion +would be held to. Anything the language refuses is refused there. Every message names what went wrong and what to do about it: @@ -39,7 +35,7 @@ loads. ```text Variable 'slack' makes this model unbounded: no constraint names it, and -bounds.lower is -inf, which is the direction a +slack term improves a minimize +bounds.lower is open, which is the direction a +slack term improves a minimize objective in. No data can change that, so the solve would answer `unbounded` and name nothing. Give it a finite bounds.lower, or the constraint that was meant to define it. @@ -51,34 +47,16 @@ variable with no constraint row. ## Which error you get -| | | -| ---------------- | -------------------------------------------------------------------------------------------------------------------------------- | -| `MathSpecError` | The root. Everything below is an instance of it | -| `LanguageError` | Something in the model: a construct outside the language, a dimension set that does not compose, or a name that nothing declares | -| `SchemaError` | Something in the file: an unknown key, a malformed declaration, or a bad symbol table | -| `DimensionError` | Dimensions that disagree, such as a constraint whose expression does not equal its `dims` | +| Class | Subclass of | | +| ---------------- | --------------- | -------------------------------------------------------------------------------------------------------------------------------- | +| `MathSpecError` | `ValueError` | The root | +| `LanguageError` | `MathSpecError` | Something in the model: a construct outside the language, a dimension set that does not compose, or a name that nothing declares | +| `SchemaError` | `LanguageError` | Something in the file: an unknown key, a malformed declaration, or a bad symbol table | +| `DimensionError` | `LanguageError` | Dimensions that disagree, such as a constraint whose expression does not equal its `dims` | -Every one of these is reproducible from the YAML alone. An engine that binds -numbers or calls a solver adds its own errors below `MathSpecError`. +Every one of these is reproducible from the YAML alone. ## What the language will not express -Each of these was asked for and refused, and [the limits](../../about/limits.md) -gives the reasons. - -| Not in the language | Instead | -| ------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `variable * variable` in a bound or a `piecewise:` link | The objective and the constraints take it. Everywhere else, use a parameter coefficient ([expressions](expressions.md#where-a-product-of-two-variables-is-allowed)) | -| `sum(x, over=d) * sum(y, over=d)` | Multiply before you reduce, or constrain a variable to equal the reduction | -| degree 3 (`x * y * z`) | A variable constrained to equal one product, multiplied by the third | -| `**` with a variable in it | `x * x` for a square. Over variable-free operands `**` is in the language | -| arithmetic in `bounds:` | A name or a number. Ship the derived column as data | -| time-series processing (resample, cluster, interpolate, align), file IO, units | Data preparation. Pass a parameter | -| indicator constraints | `sos:` is where that landed ([piecewise](piecewise.md#sos)) | -| multi-objective | There is one `objective:` block. Weight the goals into one expression | -| arbitrary array operations (`merge`, `reindex`, `apply_ufunc`) | Data preparation | -| filling a missing value (`.fillna`) | Data preparation, or a `where` if the coordinate should not exist. Inside the language, only `shift(..., edge=)` fills ([absence](absence.md)) | - -The language has no escape hatch. Math it cannot express is a gap in the -language, and a gap closes as a macro, a primitive or a formulation -([the limits](../../about/limits.md)). +What the language refuses, and what to write instead, is in +[the limits](../../about/limits.md#deliberate-non-primitives). diff --git a/docs/reference/language/expressions.md b/docs/reference/language/expressions.md index 61d03a99..6bc34806 100644 --- a/docs/reference/language/expressions.md +++ b/docs/reference/language/expressions.md @@ -23,12 +23,11 @@ NUMBER ::= integer | float | "inf" | ".inf" - Operators bind in this order, highest first: `**`, then unary `+` and `-`, then `*` and `/`, then binary `+` and `-`. So `-x ** 2` is `-(x ** 2)`, as in - Python. Parentheses override precedence. + Python. - A float may carry an exponent, as in `1e5` or `2.5e-3`. - The same keyword twice in one call is an error. -- An expression nests at most 100 levels deep, and so does a `where:` string. - With every named expression it reads written in, an expression nests at most - 300 levels deep. +- An expression and a `where:` string nest at most 100 levels deep, and at most + 300 with every named expression they read written in. ## Where a product of two variables is allowed @@ -44,20 +43,16 @@ bound it: - **Everything beside the math stays affine.** A bound is one number per column, and a `piecewise:` link is affine. -A [named expression](named.md) is held to the limit of the place that reads -it. One that nothing in the math reads is [reported](named.md#reported-expressions), -and no degree limit applies to it. +A [reported expression](named.md#reported-expressions) is not held to these. `/` needs a divisor that carries no variable and is a single factor. `**` needs a base and an exponent that both carry no variable and neither of which adds. `growth ** period` is allowed, and `(1 + rate) ** period` is -refused: bind the factor itself as a parameter. Write `x * x` for a square. +refused: declare the factor itself as a parameter. Write `x * x` for a square. ## Name resolution -A name is a letter or an underscore, followed by letters, digits or underscores. - One flat namespace covers dimensions, relations, parameters, variables, named expressions, macros and the built-in operators. A collision is a load error that names both declarations, and nothing shadows anything. @@ -74,16 +69,15 @@ Position decides which kinds of name are legal: | the `edge` key of `shift` | `'wrap'` in quotes, or a bare number | | `dual` argument (`dual(c)`) | a constraint. It resolves against the constraints alone ([named expressions](named.md#reading-a-constraints-dual)) | -A bare word in the value of a keyword argument is a name to resolve, which is -why `wrap` is quoted. A keyword's key is never a name. +A bare word in the value of a keyword argument is a name to resolve. A +keyword's key is never a name. -Constraints and assumptions sit outside the flat namespace, because no -expression names either, so a model may name a constraint or an assumption -after a variable. The objective has no name at all. +Constraints and assumptions sit outside the flat namespace, so a constraint or +an assumption may share a variable's name. ## How dimensions combine -The dimension set of every expression is known before any data binds: +The dimension set of every expression is known before any data is attached: | Node | Dim set | Error | | -------------------------------- | --------------------------------- | -------------------------------------------------------------------------------- | @@ -98,9 +92,8 @@ The dimension set of every expression is known before any data binds: | `shift(x, along=d, offset=n)` | `dims(x)` | error if `d ∉ dims(x)` | | `sum_back(x, along=d, window=n)` | `dims(x)` | error if `d ∉ dims(x)` | -A binary operator takes the **union** of the two dimension sets, so an outer -product is allowed. The declaration's own dimensions are its **frame**, and a -declaration may not disagree with its expression: +An outer product is allowed. The declaration's own dimensions are its +**frame**, and a declaration may not disagree with its expression: - A constraint requires `dims(lhs) ∪ dims(rhs)` to **equal** its `dims`. - An objective must carry **no dimensions**. Write the sums that reduce it. @@ -130,38 +123,31 @@ COLUMNS ::= NAME | "[" NAME { "," NAME } "]" QUOTED ::= "'" chars "'" | '"' chars '"' ``` -| Written as | Names a… | Meaning | -| --------------------------------------------- | -------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `name` (bare) | parameter | The value is defined here. A `bool` is its own answer. A `str` is defined wherever the table has a row. A number has to have a row and be finite | -| `name` (bare) | variable | The variable exists at this coordinate | -| `name` (bare) | relation | A row exists, read at the relation's key. A relation may be [partial](relations.md#the-data-contract), and this selects the labels that do map | -| `name` (bare) | dimension | A load error. It would be true everywhere | -| `name OP value` | parameter | Element-wise, and a null compares false | -| `name OP value` | dimension | A filter on the frame's own coordinate column | -| `name OP value`, `name.col OP value` | relation | A filter on a value column, read at the relation's key. Name the column where the key determines several | -| `name OP name`, `name.a OP name.b` | two relation columns | Legal where both relations are keyed over the same dimensions and both columns are over one dimension. `ends.bus0 != ends.bus1` excludes a self-loop | -| `expression OP expression` | arithmetic over parameters | Coordinate by coordinate, over every dimension either side carries ([arithmetic in a comparison](#arithmetic-in-a-comparison)). A side with no value at a coordinate compares false | -| `position(name) OP i` | dimension | Where the row sits along the dimension's own order. `0` is first, and a negative number counts from the end | -| `position(name, by=relation, within=c)` | dimension | The same, counted within each group the relation makes | -| `count(where_expr, over=name) OP i` | a predicate | How many coordinates along the dimension the predicate admits ([counting what a predicate admits](#counting-what-a-predicate-admits)) | -| `shift(where_expr, along=name, offset=i)` | a predicate | The predicate read `i` coordinates back, and false where that vacates | -| `at(where_expr, by=relation, over=a, into=b)` | a predicate | The predicate read through the relation ([reading a predicate through a relation](#reading-a-predicate-through-a-relation)), and false where the relation has no row | -| `AND` `OR` `NOT` | — | Case-insensitive. `NOT` binds tighter than `AND`, and `AND` tighter than `OR` | -| `True` / `False` | — | `True` is the same as no `where`; `False` gives a declaration with no rows. A [case `when:`](named.md#the-rules-that-keep-the-cases-apart) may not fold to either | - -The dimensions of the mask must not exceed the frame it sits in. A bare name -that is not declared is a load error. - -!!! warning "Defined is not the same as non-zero" - - A bare parameter name is true wherever the table has a row, and a row - holding `0.0` is a row. Where you mean non-zero, write `where: "inflow != 0"`. +| Written as | Names a… | Meaning | +| --------------------------------------------- | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `name` (bare) | parameter | The value is defined here. A `bool` is its own answer. A `str` is defined wherever the table has a row. A number has to have a row and be finite, and `0.0` is a row: write `inflow != 0` for non-zero | +| `name` (bare) | variable | The variable exists at this coordinate | +| `name` (bare) | relation | A row exists, read at the relation's key. A relation may be [partial](relations.md#the-data-contract), and this selects the labels that do map | +| `name` (bare) | dimension | A load error. It would be true everywhere | +| `name OP value` | parameter | Element-wise, and a null compares false | +| `name OP value` | dimension | A filter on the frame's own coordinate column | +| `name OP value`, `name.col OP value` | relation | A filter on a value column, read at the relation's key. Name the column where the key determines several | +| `name OP name`, `name.a OP name.b` | two relation columns | Legal where both relations are keyed over the same dimensions and both columns are over one dimension. `ends.bus0 != ends.bus1` excludes a self-loop | +| `expression OP expression` | arithmetic over parameters | Coordinate by coordinate, over every dimension either side carries ([arithmetic in a comparison](#arithmetic-in-a-comparison)). A side with no value at a coordinate compares false | +| `position(name) OP i` | dimension | Where the row sits along the dimension's own order. `0` is first, and a negative number counts from the end | +| `position(name, by=relation, within=c)` | dimension | The same, counted within each group the relation makes | +| `count(where_expr, over=name) OP i` | a predicate | How many coordinates along the dimension the predicate admits ([counting what a predicate admits](#counting-what-a-predicate-admits)) | +| `shift(where_expr, along=name, offset=i)` | a predicate | The predicate read `i` coordinates back, and false where that vacates | +| `at(where_expr, by=relation, over=a, into=b)` | a predicate | The predicate read through the relation ([reading a predicate through a relation](#reading-a-predicate-through-a-relation)), and false where the relation has no row | +| `AND` `OR` `NOT` | — | Case-insensitive. `NOT` binds tighter than `AND`, and `AND` tighter than `OR` | +| `True` / `False` | — | `True` is the same as no `where`; `False` gives a declaration with no rows | + +A bare name that is not declared is a load error. ### Counting what a predicate admits `count(, over=)` is how many coordinates along that -dimension the predicate is true at. It is the one place a predicate is read as -a number, and it is compared against a whole number: +dimension the predicate is true at: ```yaml dimensions: @@ -186,25 +172,19 @@ objective: $$\lvert \{ b \in \mathcal{B} \thinspace : \thinspace \mathrm{points}_{g,b} \} \rvert \ge 2 \qquad \forall\thinspace g \in \mathcal{G}$$ -The dimension counted over is **removed**, as a `sum(over=)` removes it, so -what is left is one number per remaining coordinate — one per generator above. -The count therefore states a fact about each group without naming the group. -Counting along a dimension the predicate does not read is a load error. +The dimension counted over is **removed**, as a `sum(over=)` removes it: one +count per generator above. Counting along a dimension the predicate does not +read is a load error. -The comparison takes a whole number on the right. A count is a number of -coordinates, so a fraction and a parameter are both load errors, and so is a -comparison a count can never fail or never meet: `>= 0`, `< 0`, or any -negative number. +The right-hand side is a whole number. A fraction, a parameter, and a +comparison a count can never fail or never meet (`>= 0`, `< 0`, or any +negative number) are load errors. ### Reading a predicate at the previous coordinate `shift(, along=, offset=)` reads the predicate `offset` coordinates back. It is **false** where the translation vacates, and -it takes no `edge=`: the arithmetic `shift` needs one because no number is -neutral, and false is what a missing row already means in a mask. - -The two together name the start of a run — a coordinate the mask admits whose -neighbour before it the mask does not: +it takes no `edge=`. With `count`, it names the start of a run: ```yaml where: "count(points AND NOT shift(points, along=bp, offset=1), over=bp) == 1" @@ -213,16 +193,15 @@ where: "count(points AND NOT shift(points, along=bp, offset=1), over=bp) == 1" That reads: the marked breakpoints are one consecutive run. A negative `offset` reads forwards. `by=`, `within=` and `edge='wrap'` are not -in this form; where you need a grouped or cyclic translation, compare the -arithmetic one instead. +in this form; for a grouped or cyclic translation, compare the arithmetic +`shift`. ### Reading a predicate through a relation `at(, by=, over=, into=)` reads a predicate over coarse coordinates at fine ones, as [`at`](operators.md#at) reads an array. It -is true at a coordinate where the relation has a row and the predicate holds at -the coordinate that row maps to. It is **false** where the relation has no row, -which is what a missing row already means in a mask. +is true where the relation has a row and the predicate holds at the coordinate +that row maps to, and **false** where the relation has no row. ```yaml dimensions: @@ -245,53 +224,40 @@ objective: $$0 \le \mathit{rate}_{f} \le \mathrm{cap}_{f} \qquad \forall\thinspace f \in \mathcal{F} \thinspace : \thinspace \mathrm{has\_curve}_{\mathrm{converter\_of}(f)}$$ -The consumed dimension goes and the produced one arrives, so the mask above is -over `flow` alone. The rules are those of `at` in an expression: `by=`, -`over=` and `into=` are all written, the read lands on the relation's key, and -the predicate carries every dimension the read consumes. The read maps one -dimension onto another and adds none, so a mask still may not widen its frame. - -A parameter compared as arithmetic reads through a relation too: -`at(cap, by=bus_of, over=bus, into=generator) > 0`. The predicate form reads -what arithmetic cannot: whether a row is defined, a `bool`, a variable's -existence, and any connective over them. +The mask above is over `flow` alone. The rules are those of `at` in an +expression: `by=`, `over=` and `into=` are all written, the read lands on the +relation's key, and the predicate carries every dimension the read consumes. ### The right-hand side of a comparison A bare name on the right is read as a string label when the model does not declare it. A declared name there is a load error. -Quote a label that is not an identifier, and quote a date: `'combined-cycle'`, -`'IT-north'`, `'2030-01-01'`. A quoted word is never read as a declaration. +Quote a label that is not an identifier, such as `'combined-cycle'`. A quoted +word is never read as a declaration. A comparison is checked against the declared `dtype`. A `datetime` dimension is -compared against a quoted ISO date such as `snapshot > '2030-01-01'` or +compared against a quoted ISO date such as `'2030-01-01'` or `'2030-01-01T06:00'`, and a number against it is a load error. -String labels compare bytewise, whatever order the dimension declared them in. -A label the dimension does not carry compares equal to nothing, so the mask is -false there. +String labels compare bytewise, whatever the dimension's order. A label the +dimension does not carry compares equal to nothing. Comparing two dimensions is not in the language. Precompute a boolean parameter -instead. Two parameters compare as [arithmetic](#arithmetic-in-a-comparison). +instead. ### Arithmetic in a comparison Either side of a comparison may be an expression over parameters: -`p_min <= 0.5 * p_max`, `sum(p_max, over=generator) >= peak`, +`p_min <= 0.5 * p_max`, or `p_max <= at(bus_cap, by=bus_of, over=bus, into=generator)`. The side is read as -an [expression](#expressions) is. A macro and a named expression expand into it, -and every operator keeps its own rule. Two things an expression may carry are -refused here, because a mask is built before either exists: a variable, and a -`dual()`. A relation column and a quoted label are compared on their own, and -are not read in arithmetic. - -The comparison is checked over every dimension either side carries, and those -dimensions must not exceed the frame. A side whose value is absent at a -coordinate compares false there, as a null does in every other comparison. -Under a summing operator the absent term is one fewer. A `shift` says what its -vacated positions hold, as it does everywhere. So a comparison against the -previous row names an `edge=`, and a `position()` term keeps the first row out: +an [expression](#expressions) is, macros and named expressions included. A +variable and a `dual()` are refused. A relation column and a quoted label are +compared on their own, and are not read in arithmetic. + +A side absent at a coordinate compares false there, and under a summing operator the absent term +is one fewer. A comparison against the previous row gives its `shift` an +`edge=`, and a `position()` term keeps the first row out: ```yaml dimensions: @@ -308,29 +274,13 @@ constraints: expression: shed >= load - ramp ``` -A case `when:` may not compare expressions. The loader proves the cases of a -[`cases:` block](named.md#the-rules-that-keep-the-cases-apart) apart at load, -by trying every value the masks name. A comparison of expressions names no -value, because only the data decides whether `c > 2 * k` holds, so the loader -refuses the case, whether or not the block has a second one: - -> `Named expression 'e'`: case `wide` cannot be told apart before the data -> arrives: it compares expressions, whose values only the data decides — compare -> one parameter against a literal, or precompute the test as a boolean parameter -> and test that. The `otherwise` is its negation, and only the data says where -> that falls, so this is refused the way a proven overlap is. - -A comparison with a number on both sides, such as `2 < 1`, is refused -everywhere: it is decided before any data arrives, and a `where` tests data. - -A variable's `where` and a constraint's `where` are not held to this, because -neither is proved apart from anything. +A [case `when:`](named.md#the-rules-that-keep-the-cases-apart) may not compare +expressions. A comparison with a number on both sides, such as `2 < 1`, is +refused everywhere. ### `position()` -`position(dim)` is where the row sits along the dimension's own order, which is -the order `shift` steps along. A boundary written with it survives a relabelling of -the index: +`position(dim)` counts along the order `shift` steps along, not the label: ```yaml dimensions: @@ -346,11 +296,10 @@ constraints: expression: soc == soc_initial ``` -`-1` is the last position, and `-2` the one before it. A position that no -coordinate occupies is an error when the data binds. +A position that no coordinate occupies is an error when the data is attached. -`by=` counts inside each group that a relation makes. That gives one seeded row -per period, however long each period is: +`by=` counts inside each group a [partition](relations.md#partitions) makes, so +each period gets one seeded row: ```yaml dimensions: @@ -368,8 +317,3 @@ constraints: where: "position(snapshot, by=period_of, within=period) == 0" expression: soc == at(soc_initial, by=period_of, over=period, into=snapshot) ``` - -The relation must have a key column over the dimension being counted, and -`within=` names the value columns the groups are made of -([partitions](relations.md#partitions)). A coordinate the relation sends -nowhere is in no group. diff --git a/docs/reference/language/index.md b/docs/reference/language/index.md index bab5b195..39738432 100644 --- a/docs/reference/language/index.md +++ b/docs/reference/language/index.md @@ -36,28 +36,13 @@ objective: expression: sum(dispatch * cost) # an objective is one number, so the sum is written ``` -That file is a complete model. The pages below give the exact rules. - -## The pages - -| | | -| ----------------------------------------------------------------------- | ------------------------------------------------------------------------- | -| [File shape](file.md) | the eleven keys, `version` and `description` | -| [Dimensions](dimensions.md) | the axes | -| [Relations](relations.md) | the maps from one axis onto another | -| [Parameters, variables, constraints and the objective](declarations.md) | the four blocks that carry the math | -| [Expressions](expressions.md) | the arithmetic grammar, the `where` grammar, and how dimensions combine | -| [Named expressions and macros](named.md) | quantities named once, templates with arguments, and what a solve reports | -| [Operators](operators.md) | `sum`, `sum_back`, `at` and `shift` | -| [Absence and `where`](absence.md) | which rows are built, and which are not | -| [Piecewise curves and SOS](piecewise.md) | `piecewise:` and `sos:` | -| [Assumptions](assumptions.md) | what the model expects of the data it is bound to | -| [Errors and limits](errors.md) | what fails when, and what the language will not express | +That file is a complete model. The pages of this section give the exact rules, +and the [glossary](../glossary.md) defines each word they use in a fixed sense. ## The ten rules -`to_spec` checks everything it can without data, and refuses the file with a -message that names the fix. These are the rules it checks. +`to_spec` refuses a file that breaks one of these rules, with a message that +names the fix. | # | Rule | | | --- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------- | diff --git a/docs/reference/language/named.md b/docs/reference/language/named.md index b1f08820..96894d29 100644 --- a/docs/reference/language/named.md +++ b/docs/reference/language/named.md @@ -11,8 +11,8 @@ before anything reads it. ## `expressions` -A named expression is a quantity the model names once. A constraint or the -objective may use it, and the engine can report its value after a solve: +A constraint or the objective may use a named expression, and a solve may +report its value: ```yaml dimensions: @@ -28,8 +28,8 @@ expressions: description: CO2 released, the quantity a cap would bound ``` -Write it as a bare string, or as a mapping when it carries a `description:`. Its -dimensions follow from its body, so there is no `dims:`. +It is a bare string, or a mapping with a `description:`. Its body decides its +dimensions, and there is no `dims:`. Where the objective or a constraint names it, the body is substituted there, and the [degree limit](expressions.md#where-a-product-of-two-variables-is-allowed) @@ -87,7 +87,7 @@ A named expression carries **exactly one** of `expression:` and `cases:`. > two `when:` strings by the negation of the other, or drop the wider one and > let `otherwise:` carry that region. - That is why `boundary` above says `committable and`. The cases carry no order. + The cases carry no order. - **A `when:` must be a question the data answers.** `True`, `False`, and a mask that folds to one of them, such as `committable OR True`, are refused. @@ -96,14 +96,16 @@ A named expression carries **exactly one** of `expression:` and `cases:`. against `position(snapshot) == -1` pick the same row on an axis with one member. Count from one end only. +- **A `when:` may not compare expressions**, such as `c > 2 * k`, even in a + block with one case. Precompute the test as a boolean parameter. + - **Each `when:` and each value sits inside the frame.** A narrower case broadcasts as a parameter with fewer dimensions does. -Claiming a coordinate is not the same as having a value there. The `otherwise:` -above carries no `edge=`, so its `shift` has no value at the first snapshot, and -`previous_status` is whole there only because a case claims every unit at that -snapshot. To close such a hole, widen a `when`, give the `shift` an `edge=`, or -set `absence: zero` on the masked variable. +A claimed coordinate can still have no value: the `otherwise:` above has none +at the first snapshot, where a case claims every unit. To close such a hole, +widen a `when`, give the `shift` an `edge=`, or set `absence: zero` on the +masked variable. `cases:` is not accepted inside a `macros:` template. @@ -127,18 +129,15 @@ expressions: objective: { sense: minimize, expression: system_cost } ``` -`system_cost` is in the math: the objective uses it, so the solver sees its body. -`delivered` and `lcoe` are reported: nothing in the math uses them, so the -engine computes them from the solution after the solve. +`system_cost` is in the math. `delivered` and `lcoe` are reported. An entry is in the math when the objective, a constraint or a `piecewise:` link reaches it, directly or through another entry or a macro. A bound and a `where` name no entry. -A reported body is built by no solver, so **no degree limit applies to it**: -it may divide by a variable, raise one to a power, and multiply two sums. A -comparison stays out. A constraint that later names such an entry reads its -body, and is refused there under the constraint's own name. +**No degree limit applies to a reported entry**: it may divide by a variable, +raise one to a power, and multiply two sums. A comparison stays out. A +constraint that names such an entry is refused under the constraint's own name. ### Reading a constraint's dual @@ -152,14 +151,11 @@ Constraint 'd': a dual exists only after a solve; the math cannot read one — keep the entry that carries it out of constraints, the objective, bounds and where. ``` -`c` [resolves against the constraints alone](expressions.md#name-resolution). - `dual(c)` is the rate at which the optimal objective improves as `c` is relaxed in the direction its comparator points, under the model's own `minimize` or `maximize`. -A row that `c`'s `where:` deletes has no dual. Where the solver returns no dual, -as for a model with integer variables, the engine reports no value. +A row that `c`'s `where:` deletes has no dual. ## `macros` @@ -183,6 +179,6 @@ macros: - Every template is held at load to every rule a call site is, whether or not it is called. A formal is left for the call site to bind. -Anything composed out of the [built-in operators](operators.md) belongs here. -What the language cannot express is under -[what the language will not express](errors.md#what-the-language-will-not-express). +A composition of the [built-in operators](operators.md) belongs here. What +the language will not express is in +[the limits](../../about/limits.md#deliberate-non-primitives). diff --git a/docs/reference/language/operators.md b/docs/reference/language/operators.md index eeec1f3d..1ca80623 100644 --- a/docs/reference/language/operators.md +++ b/docs/reference/language/operators.md @@ -27,23 +27,14 @@ in a reported expression, are all of them. A composition of them goes in | `sum_back(array, along=dim, window=p, edge='wrap')` | The window reaches around the axis, instead of stopping short at its start | | `sum_back(array, along=dim, window=n, by=relation, within=c)` | The window stays inside each group that the relation's column `c` makes | -`array` is any expression with the right dimension set, so each operator reads a -parameter as readily as a variable. Dimension arguments are name-checked at -load. [Every operator as math](#every-operator-as-math) shows how each row prints. +`array` is any expression with the right dimension set, a parameter or a +variable. [Every operator as math](#every-operator-as-math) shows how each row +prints. ## `sum` -`sum(x, over=d)` adds up `x` along `d`, and `d` is gone from the result. - -`sum(x)` names no dimension and reduces every dimension `x` carries, so its -result is a scalar. - -An operand that is already scalar, and an `over=` naming a dimension the -operand does not carry, are both errors. - -`sum(x, by=l, over=a, into=b)` sums through a [relation](relations.md), -consuming column `a` and landing the result on column `b`. A nodal balance is -one `sum(by=)` per kind of component: +`sum(x)` on a scalar is an error. A nodal balance is one `sum(by=)` per kind of +component: ```yaml dimensions: @@ -69,32 +60,20 @@ constraints: == load ``` -The same `f` is summed twice through two relations, once as inflow and once as -outflow. What the call reads and what its result carries are on +What a call through a relation reads and carries is on [how a relation is used](relations.md#how-a-relation-is-used). -A group with no members contributes nothing, and a member whose relation value -is null belongs to no group. - ## `at` -`at(x, by=l, over=a, into=b)` reads the relation the other way. It consumes a -value column and produces the key, so it reads one coarse value once for each -fine label that points at it ([reads](relations.md#aggregates-and-reads)). - -`at` reads a variable as readily as a parameter. One decision taken per bus, read -once by every line that touches the bus, is `at(decision, by=line_bus, over=bus, into=line)`. - -A fine label whose relation value is null reads nothing, and its row is absent. +`at` reads one coarse value once for each fine label that points at it +([reads](relations.md#aggregates-and-reads)). One decision per bus, read by +every line that touches the bus, is +`at(decision, by=line_bus, over=bus, into=line)`. ## `sum_back` -`sum_back(x, along=d, window=n)` is the sum of the last `n` positions along `d`, -ending at the position being written. It states a minimum up time, a rolling -budget or a delivery horizon. A width of `1` is `x` itself. - -The dimension **survives**: `sum_back` leaves one value per position, and each -value reads a window of its own. +`sum_back` states a minimum up time, a rolling budget or a delivery horizon. +The dimension **survives**, and a width of `1` is `x` itself. ```yaml dimensions: @@ -116,22 +95,18 @@ constraints: objective: { sense: minimize, expression: sum(on) } ``` -`window=` takes a number or the name of an integer parameter. A named width is `dtype: int`, and does not vary along the -dimension being summed. +A named width is `dtype: int`, and does not vary along the dimension being +summed. -`edge=` takes `'wrap'` or nothing. A window that reaches past the start of the -axis is **short**, so no row is lost. `edge='wrap'` makes the window -reach around the axis. A number here is a load error. +`edge=` takes `'wrap'` or nothing, and a number is a load error. Without it, a +window that reaches past the start of the axis is **short**, and no row is lost. -`by=` keeps the window inside each group that a relation makes. The relation -obeys the rules given for [`shift(by=)`](#translation-within-groups). +`by=` takes a [partition](relations.md#partitions). ## `shift` -`shift(x, along=d, offset=n)` moves values along one dimension by `n` positions, -counted in the dimension's **declared order**. The value at each coordinate -becomes the value that stood `n` places before it. `edge=` says what stands -where nothing moved in. +`shift` counts positions in the dimension's **declared order**. `edge=` says +what stands where nothing moved in. ```yaml dimensions: @@ -149,32 +124,21 @@ constraints: expression: soc == shift(soc, along=snapshot, offset=1, edge='wrap') + charge * eta - discharge ``` -`edge='wrap'` makes the store cyclic: the first snapshot reads the last. - -`edge=` has three settings: - -- **Bare.** The vacated coordinate is [absent](absence.md), so the row it - would have fed is not built. State the initial condition in a block of its - own ([a rule that differs by regime](../../howto/regimes.md)). -- **`'wrap'`.** The translation is cyclic, so nothing is vacated. -- **A number.** That number stands where the slot was vacated, and the row - survives: `0` in a sum, and `1` in a product. +`edge='wrap'` makes the store cyclic: the first snapshot reads the last. Bare, +the row the vacated coordinate would have fed is not built; state the initial +condition in a block of its own ([a rule that differs by regime](../../howto/regimes.md)). -Two rules hold across all three: +Two rules hold for `edge=`: - **Over a variable, the only numeric edge is `0`.** - **A bare `shift` over an expression with no variable is a load error.** The error names the rewrites: `edge='wrap'`, `edge=0`, or `edge=0` together with a `where` that excludes the vacated coordinate. -`shift` reads parameters too. `shift(dt, along=t, offset=1, edge=0)` is the -previous snapshot's duration. - ### Translation within groups -`by=` partitions the axis the operator steps along, so the neighbour of a coordinate -is the coordinate before it in its own group. A group can be a season, an -investment period or a representative day: +`by=` takes a [partition](relations.md#partitions), and the neighbour of a +coordinate is the one before it in its own group, such as a season: ```yaml dimensions: @@ -193,19 +157,12 @@ constraints: objective: { sense: minimize, expression: sum(soc) } ``` -Every `edge=` setting then applies one group at a time. Bare, the first -coordinate of each group is vacated and its row drops. `edge='wrap'` closes each -group onto its own last coordinate. `edge=v` puts `v` at the edge of each group. - -`by=` takes a relation with a key column over the dimension being stepped -along, and `within=` names the value columns the group is made of -([partitions](relations.md#partitions)). A coordinate the relation sends -nowhere is in no group, so its row drops under every `edge=`. +Every `edge=` setting then applies one group at a time. A coordinate in no +group drops under every `edge=`. ### A parameter as offset -`offset=` may name an integer parameter instead of a number. Then each entity is -reached by its own offset: a construction lead time, a transit time, or any delay +An offset per entity is a construction lead time, a transit time, or any delay the data carries as a column: ```yaml @@ -226,14 +183,13 @@ constraints: objective: { sense: minimize, expression: sum(order) } ``` -Three rules hold, and breaking any of them is a load error that names its -rewrite: +Each of these is a load error: -- **The parameter is `dtype: int`.** -- **The parameter does not vary along the dimension being translated.** -- **The parameter varies only over dimensions the shift can read.** Those are - the dimensions of the shifted expression, and the dimension a - [`by=`](#translation-within-groups) relation groups into, so `offset=lead` +- **The parameter is not `dtype: int`.** +- **The parameter varies along the dimension being translated.** +- **The parameter varies over a dimension the shift cannot read.** The shift + reads the dimensions of the shifted expression, and the dimension a + [`by=`](#translation-within-groups) relation groups into: `offset=lead` with `lead: {dims: [period]}` under `by=period_of` gives one lag per period. The sign travels in the values: `offset=-lead` is refused. @@ -243,8 +199,7 @@ The sign travels in the values: `offset=-lead` is refused. Each row is generated from one model in [`examples/operators/`](https://github.com/energy-models/math-spec/tree/main/examples/operators), printed by the [typesetter](../typeset.md). The models themselves are on -[One construct per model](../../examples/operators.md), and the rest of the -language prints on [Every construct, as math](../notation.md). +[One construct per model](../../examples/operators.md). | Operator | Renders as | diff --git a/docs/reference/language/piecewise.md b/docs/reference/language/piecewise.md index f216e3f5..034abd2c 100644 --- a/docs/reference/language/piecewise.md +++ b/docs/reference/language/piecewise.md @@ -5,13 +5,10 @@ SPDX-License-Identifier: CC-BY-4.0 # Piecewise curves and SOS -Two blocks state shapes that no `expression:` can, because an expression is -affine. `piecewise:` states a curve through breakpoints. `sos:` states a family -of variables of which only one, or only two neighbours, may be non-zero. - -Both are **formulations**: each states plain variables and constraints rather -than being one, and [`spec.expand()`](#writing-a-formulation-out) writes them -out. +`piecewise:` states a curve through breakpoints. `sos:` states a family of +variables of which only one, or only two neighbours, may be non-zero. Both are +**formulations**: each states plain variables and constraints, and +[`spec.expand()`](#writing-a-formulation-out) writes them out. ## `piecewise` @@ -52,30 +49,10 @@ piecewise: | `activity` | a binary variable that gates the curve ([below](#activity)) | default `null` | | `points` | how far each curve runs, where the curves are not all the same length ([below](#points)) | default `null` | -A block states plain variables and constraints: one weight per breakpoint in -`[0, 1]`, one row making the weights sum to 1, and one row per link tying its -expression to the weighted breakpoints. A `Program` holds the block as one -curve, and the [typeset output](../typeset.md) prints the curve itself. -[`spec.expand()`](#writing-a-formulation-out) writes the rows into a model of -their own, which is the model a consumer that builds rows reads. - -The breakpoint order is the declared order of `over`. A curve whose breakpoints -decrease in that order is refused when the data binds. - -Every condition this page says is checked "when the data binds" is an -[assumption](assumptions.md), written in the same grammar as one the file -states. The `method:` implies it rather than the file writing it, so -[`expand()`](#writing-a-formulation-out) writes it into `assumptions:` under -the block's own name, and a model that still declares the block derives the -same text when it loads. Both print under one heading, and the consumer that -binds the numbers runs them. - -!!! warning "A values parameter short of a row does not build a shorter curve" - - The missing row reads as a breakpoint at the origin. Every block states - `_complete` for this, whatever its `method:`, so the table is - refused when the data binds and the refusal names `points:` as the way to - say how far a curve runs. +A block states one weight per breakpoint in `[0, 1]`, a row making the weights +sum to 1, and a row per link tying its expression to the weighted breakpoints. +The breakpoint order is the declared order of `over`. What a block assumes of +its numbers is on [what a curve assumes](assumptions.md#what-a-curve-assumes). ### `activity` @@ -97,9 +74,10 @@ instead, put `absence: zero` on the gate. ### `points` -A curve with fewer breakpoints than the dimension holds says so with `points:`. -Name one of the block's own values parameters, and the curve is as long as that -parameter has rows: +A values parameter short of a row does not build a shorter curve: the missing +row reads as a breakpoint at the origin. A curve with fewer breakpoints than the +dimension holds says so with `points:`. Name one of the block's own values +parameters, and the curve is as long as that parameter has rows: ```yaml piecewise: @@ -111,12 +89,9 @@ piecewise: - [op_cost, bp_y] ``` -The other links are still read against the parameter you named, so a row missing -from `bp_y` is refused. Where the length is its own data, name a boolean -parameter instead. - -The marked breakpoints must be consecutive. They need not start at the head of -the axis. A gap, or a curve with no points, is refused when the data binds. +A row missing from `bp_y` is still refused. Where the length is its own data, +name a boolean parameter instead. The marked breakpoints are one consecutive +run, anywhere on the axis. ### `method` @@ -129,20 +104,8 @@ the axis. A gap, or a curve with no points, is refused when the data binds. | `convex` | nothing | the hull, which is a pure linear program | | `lp` | no weights at all: one row per segment line, plus two rows holding the domain | the curve as its own lines | -`adjacency` and `sos2` state the same restriction and reach the same optimum. -They differ in what the solver is handed: `adjacency` **is** `sos2` with the set -written out, so the two emit the same rows under the same names. - -`convex` is a different model: the weights range over the hull the breakpoints -span rather than over the curve itself. It takes exactly two links and no -`activity:`. - -A bounded link binds from one side, and that side is the part of the hull the -weights are driven onto. `>=` requires a convex curve and `<=` a concave one. -With both links pinned the weights reach the whole hull. What drives them -within it is the rest of the model rather than the block, so the curve must -bend one way only. Each of the three conditions is checked against the -breakpoint values when the data binds. +`convex` takes exactly two links and no `activity:`. The shape it needs is an +[assumption](assumptions.md#what-a-curve-assumes). `lp` states the curve as its segment lines. It needs **exactly two links**, one of them bounded with `<=` or `>=`, and no `activity:`: @@ -157,14 +120,7 @@ piecewise: - [op_cost, bp_y, ">="] # cost bounded below by the curve ``` -The bounded link decides the shape, as it does under `convex` above. The two -domain rows hold the pinned link inside the breakpoint range: under `points:`, -each sits where the mask holds and does not one breakpoint outward, which is -the first and the last breakpoint of each curve. - -`links:` is a list, so the number of expressions a block ties is written in the -file. Where that number is data, write the formulation out -([a curve by hand](../../howto/curve-by-hand.md)). +Where the number of links is data, write the formulation out ([a curve by hand](../../howto/curve-by-hand.md)). ## `sos` @@ -179,22 +135,18 @@ sos: type: 1 # 1: at most one non-zero; 2: at most two, and consecutive ``` -`type: 1` is a choice: at most one member is non-zero. `type: 2` is an -interpolation: at most two members are non-zero, and they are **consecutive**. - A set is over **one** variable, and a variable holds **one** set. A second block naming the same variable is a load error. -Membership belongs to the variable. Its `where` decides which coordinates exist, -so a masked-out member is not in the set. The order is the declared order of -the `along` dimension. +A member the variable's `where` masks out is not in the set. The order is the +declared order of the `along` dimension. ### What a set is written out as `spec.expand('sos')` states the set as binaries: one per member for `type: 1`, one per segment for `type: 2`. A member the binaries do not admit is held at -zero, from above and from below. The names are the block's own, and the rows are -these, for a set `s` over variable `x` along `d`, writing `admitted` for +zero, from above and from below. For a set `s` over variable `x` along `d`, +writing `admitted` for `(s_seg)` at `type: 1` and `(s_seg + shift(s_seg, along=d, offset=1, edge=0))` at `type: 2`: @@ -205,63 +157,21 @@ at `type: 2`: | `s_nonzero` (`type: 1`), `s_adjacency` (`type: 2`) | `x <= upper * admitted` | | the same name plus `_below` | `x >= lower * admitted`, where `lower` is not `0` | -Each coefficient is read off the member's own `bounds:`. A binary member's are -`0` and `1`, from its domain. A row multiplies by its coefficient rather than -reading it, so a bound the data carries is a coefficient like any other: -`bounds: {lower: floor, upper: cap}` states `x >= floor * admitted` and -`x <= cap * admitted`. - -Two coefficients are left out rather than printed, because the row would state -what another row already does: a `1` above, and a `lower` of `0`, which the -variable's own bound states. - -So each side needs a coefficient, and a model is refused at load without one: - -- `bounds.lower`, a number or a parameter. An omitted lower bound leaves the - member free below zero, which no row can pull back. -- `bounds.upper`, a number or a parameter, or `domain: binary`. - -The set carries no coefficient of its own. A number below the member's bound -would cap a picked member the set does not cap, and one above it is a looser -row than the bound already states, so there is no value of such a key that -states the set and nothing else. - -A positive `bounds.lower` loads and is infeasible, as it is on a solver that -takes the set: an unpicked member has to be `0`, and its own bound says it is -above that. - -A name the expansion writes that the file already declares is refused at load -too. +`upper` and `lower` are the member's own `bounds:`, a number or a parameter; +a binary member's are `0` and `1`. A model is refused at load where a member +has no `bounds.lower`, or no `bounds.upper` and no `domain: binary`. A name the +expansion writes that the file already declares is refused at load too. ## Writing a formulation out -`Spec.expand()` returns the same math with its formulations stated as plain -variables and constraints: - -```python -from math_spec import to_spec - -spec = to_spec('curve.yaml') -spec.expand() # every formulation -spec.expand('sos') # only the sets -spec.expand('piecewise') # only the curves -``` - -[See what a curve or a set expands to](../../howto/see-an-expansion.md) shows -a model before and after, as whole files. - -- **The kinds are `'piecewise'` and `'sos'`, and no argument means both.** Any - other string is refused, naming the two. Curves go first whatever order they - are asked in, because a `method: sos2` curve states a set and no set states a - curve. -- **A model with nothing to write out is the model that comes back.** So is a - second call with the same kinds. -- **The same data binds a model and its expansion.** Neither a set nor a curve - emits a parameter. A curve under `points:` sits its rows on `where:` - predicates over the mask the file named, and the expansion is a file like any - other: `to_yaml()` writes it, and loading it back changes nothing. -- **`spec.program` writes nothing out.** The program mirrors the model: a - curve the model still declares is under `program.piecewise`, typed, and - `spec.expand('piecewise').program` carries its rows instead. A consumer - building rows reads the expansion's program, and refuses a curve it finds on - a program; one that cannot take a set reads `spec.expand().program`. +Writing a formulation out replaces the block with the variables and constraints +it states. [`Spec.expand()`](../api.md#math_spec.Spec.expand) is the +call, and [see what a curve or a set expands to](../../howto/see-an-expansion.md) +shows a model before and after. + +- **Every name written out starts with the name of the block.** The weights of + the curve `curve` are `curve_lam`. +- **No formulation emits a parameter.** The same data attaches to a model and its + expansion. +- **The assumptions a `method:` implies become `assumptions:` entries** with + the same names. diff --git a/docs/reference/language/relations.md b/docs/reference/language/relations.md index 63ab8950..2568b47c 100644 --- a/docs/reference/language/relations.md +++ b/docs/reference/language/relations.md @@ -39,17 +39,15 @@ mapping form names them: `{bus0: bus, bus1: bus}`. ### Cardinalities -| intention | written | cardinality | -| ------------------------------------------------------ | ----------------------------------------------------- | ------------------------------------------- | -| each generator has one bus | `{key: generator, values: bus}` | many-to-one | -| a bus has several generators | the same table, read the other way | one-to-many | -| a generator may connect to several buses | `{key: [generator, bus]}`, no `values:` | many-to-many | -| a generator has one zone in each period | `{key: [generator, period], values: zone}` | many-to-one, keyed by a pair | -| a snapshot has a month, a week and a weekday | `{key: snapshot, values: [month, week, weekday]}` | many-to-one, several values | -| a line has two ends, both buses | `{key: line, values: {bus0: bus, bus1: bus}}` | many-to-one, two columns over one dimension | -| a snapshot has a representative snapshot | `{key: snapshot, values: {rep: snapshot}}` | many-to-one, onto itself | -| a snapshot has neighbours | `{key: {from: snapshot, to: snapshot}}`, no `values:` | many-to-many, onto itself | -| each generator has one bus, and each bus one generator | not a claim the language has | one-to-one | +| intention | written | cardinality | +| -------------------------------------------- | ----------------------------------------------------- | ------------------------------------------- | +| each generator has one bus | `{key: generator, values: bus}` | many-to-one | +| a generator may connect to several buses | `{key: [generator, bus]}`, no `values:` | many-to-many | +| a generator has one zone in each period | `{key: [generator, period], values: zone}` | many-to-one, keyed by a pair | +| a snapshot has a month, a week and a weekday | `{key: snapshot, values: [month, week, weekday]}` | many-to-one, several values | +| a line has two ends, both buses | `{key: line, values: {bus0: bus, bus1: bus}}` | many-to-one, two columns over one dimension | +| a snapshot has a representative snapshot | `{key: snapshot, values: {rep: snapshot}}` | many-to-one, onto itself | +| a snapshot has neighbours | `{key: {from: snapshot, to: snapshot}}`, no `values:` | many-to-many, onto itself | A key that determines a value holds one column per dimension, so `{key: {bus0: bus, bus1: bus}, values: line}` is refused. A bare relation may @@ -61,7 +59,7 @@ The data for `gen_bus` arrives under the key `gen_bus`, as a table with one column per declared column, named after it. - **One row per key tuple.** A generator on two buses is refused when the data - binds. + is attached. - **Every value is a label of its dimension.** A value that matches none is refused. - **A partial map is the rows it has.** A generator in no row sits on no bus, @@ -72,8 +70,8 @@ column per declared column, named after it. ## How a relation is used -The declaration fixes no direction. A call names the columns it reads, and a -key column it names at neither end is **joined on**. +The declaration fixes no direction. A key column a call names at neither end +is **joined on**. | kind | what it does | written as | | --------- | ------------------------------------------- | ----------------------------------------------------- | @@ -85,8 +83,7 @@ key column it names at neither end is **joined on**. Four rules hold for every use: 1. **A call names every column it reads.** `sum(p, by=gen_bus)` is refused. -2. **A value column the call does not name is not read.** So adding one to the - relation changes no call. +2. **A value column the call does not name is not read.** 3. **The key is fixed.** To change it, declare a new relation. 4. **A dimension the relation does not name passes through** to the result. @@ -116,11 +113,8 @@ bare `connection: { key: [generator, bus] }` and `p` over `[generator, period]`: ``` - **A sum consumes at least one key column, and lands on any column it does not - consume.** Either end may name a value column beside a key one. `connection` - has only key columns, and the sum above consumes one and lands on the other. -- **A read consumes value columns, and lands on the key.** Its result carries - every key column — named in `into=`, or joined on — and whatever else the - operand carries that the read does not consume. + consume.** Either end may name a value column beside a key one. +- **A read consumes value columns, and lands on the key.** - **`over=` and `into=` name different columns**, and neither names two columns over one dimension. @@ -131,7 +125,7 @@ and `position(d, by=l, within=c)` step along the key column over `d`, join on the other key columns, and group by the value columns `within=` names. The frame does not change. `within=` is written whenever `by=` is. It may name two columns over one dimension, may not name a key column, and a bare relation -partitions nothing. +partitions nothing. A coordinate the relation sends nowhere is in no group. ### Tests diff --git a/docs/reference/notation.md b/docs/reference/notation.md index 0187bc3d..f589a712 100644 --- a/docs/reference/notation.md +++ b/docs/reference/notation.md @@ -5,35 +5,24 @@ SPDX-License-Identifier: CC-BY-4.0 # Every construct, as math -[Typesetting](typeset.md) prints a model the way a paper prints it. This page -prints _all_ of it: every construct the language has, beside the math the -typesetter gives it, so the notation can be read as the one system it has to -be — two constructs that mean different things looking different, a symbol -introduced where it is defined and used where it is meant. - -It is generated by `pixi run python -m tools.notation`, almost all of it from one -model: +This page shows every construct of the language beside the math that +[the typesetter](typeset.md) prints for it. Use it to find how a construct +prints, or which construct printed a symbol. + +Each section shows the YAML of one construct, then its equation. Most fragments +come from one test model, [`tests/typesetting/golden/model.yaml`](https://github.com/energy-models/math-spec/blob/main/tests/typesetting/golden/model.yaml), -which is not a sensible optimisation problem and is not trying to be: it is the -one file that carries every construct at once, and three checks in -`tests/typesetting/test_typeset.py` hold it to the language — every operator a format -spells, every node kind the parsers produce, every line of the walk. So _every_ -here is asserted rather than promised, and a construct added to the language -arrives on this page or CI goes red. The curves are the exception, one real -model per `method:`, for the reason the section gives. - -Two things this page is not. It is not the operator reference — what each -operator _does_ is [Operators](language/operators.md), which renders the same -math one row per call shape. And it is not a tutorial: the models under -`examples/` are the ones written to be read. - -The symbols below are **derived** from the names in the file, which is what a -model prints with no setup, so you see $\mathrm{load}_{t}$ rather than $\ell_t$. -A [symbol table](typeset.md#symbol-tables) replaces every symbol, and changes -nothing else on this page. +which holds every construct and is not a sensible model. The curves come from +the example models that their section names. What each operator does is on +[Operators](language/operators.md). + +The symbols are **derived** from the names in the file, so you see +$\mathrm{load}_{t}$ rather than $\ell_t$. A +[symbol table](typeset.md#symbol-tables) replaces the symbols and changes +nothing else. -### The legend +### Legend A dimension, a relation and a parameter declare no equation; what they print is the legend every model opens with. @@ -152,966 +141,790 @@ $`\mathrm{pos}_{\mathrm{relation}(t)}(t)`$ counts within the group a relation pu $`\lvert \mathcal{T} \rvert`$ denotes the size of the set being counted along, and a position counted from the end prints against it — $`\lvert \mathcal{T} \rvert - 1`$ is the last position, one less than the size because the first is $`0`$. -### The objective +### Variable domains -#### `objective` +#### Lower and upper bounds -a sense, a product of two variables, a power over two parameters, a power of one of those, and the summations a scalar objective spells out beside two scalar terms +both bounds, and a where with all three connectives ```yaml -sense: maximize -expression: sum(p * cost) + sum(p * p * cost) + sum(p * cost * growth ** lead) + sum(p * (growth ** lead) ** 2) + sum(p * p_max) - reserve + -headroom +variables: + p: + dims: [snapshot, generator] + where: "p_max > 0 AND NOT is_flexible OR p_min > 0" + bounds: { lower: p_min, upper: p_max } ``` ```math -\max \sum_{t \in \mathcal{T},\ g \in \mathcal{G}} p_{t,g} \cdot \mathrm{cost}_{g} + \sum_{t \in \mathcal{T},\ g \in \mathcal{G}} p_{t,g} \cdot p_{t,g} \cdot \mathrm{cost}_{g} + \sum_{t \in \mathcal{T},\ g \in \mathcal{G}} p_{t,g} \cdot \mathrm{cost}_{g} \cdot \mathrm{growth}^{\mathrm{lead}_{g}} + \sum_{t \in \mathcal{T},\ g \in \mathcal{G}} p_{t,g} \cdot \left( \mathrm{growth}^{\mathrm{lead}_{g}} \right)^{2} + \sum_{t \in \mathcal{T},\ g \in \mathcal{G}} p_{t,g} \cdot \mathrm{p}^{\mathrm{max}}_{g} - \mathit{reserve} - \mathit{headroom} +\mathrm{p}^{\mathrm{min}}_{g} \le p_{t,g} \le \mathrm{p}^{\mathrm{max}}_{g} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} \,:\, \mathrm{p}^{\mathrm{max}}_{g} > 0 \wedge \neg \mathrm{is\_flexible}_{g} \vee \mathrm{p}^{\mathrm{min}}_{g} > 0 ``` -### Constraints +#### Lower bound only -#### `budgeted` +```yaml +variables: + spill: + dims: [snapshot] + bounds: { lower: 0 } +``` -names the plain expression: its symbol prints here, its definition once below +```math +\mathit{spill}_{t} \ge 0 \qquad \forall\, t \in \mathcal{T} +``` + +#### Upper bound only ```yaml -budgeted: - dims: [snapshot] - expression: spend <= budget +variables: + slack: + dims: [snapshot] + bounds: { upper: 100 } ``` ```math -\mathit{spend}_{t} \le \mathrm{budget} \qquad \forall\, t \in \mathcal{T} +\mathit{slack}_{t} \le 100 \qquad \forall\, t \in \mathcal{T} ``` -#### `starts` - -names the cased expression: its symbol prints here, its block once below +#### Unbounded variable ```yaml -starts: - dims: [snapshot, generator] - expression: p <= startup_cost +variables: + theta: + dims: [bus] ``` ```math -p_{t,g} \le \mathrm{startup\_cost}_{t,g} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} +\theta_{b} \in \mathbb{R} \qquad \forall\, b \in \mathcal{B} ``` -#### `balance` +#### Binary domain -sum over a relation +a binary domain, which is a set rather than a pair of bounds ```yaml -balance: - dims: [snapshot, bus] - expression: sum(p, by=gen_bus, over=generator, into=bus) + spill - slack == load +variables: + on: + dims: [snapshot, generator] + domain: binary ``` ```math -\sum_{g \in \mathcal{G} \,:\, \mathrm{gen\_bus}(g) = b} p_{t,g} + \mathit{spill}_{t} - \mathit{slack}_{t} = \mathrm{load}_{t,b} \qquad \forall\, t \in \mathcal{T},\ b \in \mathcal{B} +\mathit{on}_{t,g} \in \{0, 1\} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} ``` -#### `ramp` +#### Bounded integer domain -roll (cyclic) and shift (acyclic) in one equation +an integer domain, which is both: bounds, and where the values live ```yaml -ramp: - dims: [snapshot, generator] - expression: p - shift(p, along=snapshot, offset=1, edge='wrap') <= shift(p, along=snapshot, offset=1) + p_max +variables: + units: + dims: [generator] + domain: integer + bounds: { lower: 0, upper: 10 } ``` ```math -p_{t,g} - p_{t \ominus 1,g} \le p_{t - 1,g} + \mathrm{p}^{\mathrm{max}}_{g} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} +0 \le \mathit{units}_{g} \le 10, \mathit{units}_{g} \in \mathbb{Z} \qquad \forall\, g \in \mathcal{G} ``` -#### `edges` +#### Unbounded integer domain -the two translations `ramp` leaves out: a fill, and forwards +integer with neither bound: the domain is the whole line ```yaml -edges: - dims: [snapshot, generator] - expression: >- - shift(p, along=snapshot, offset=1, edge=0) - <= shift(p, along=snapshot, offset=-1, edge=0) + p_max +variables: + spare: + dims: [generator] + domain: integer ``` ```math -p_{t \boxminus_{0} 1,g} \le p_{t \boxplus_{0} 1,g} + \mathrm{p}^{\mathrm{max}}_{g} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} +\mathit{spare}_{g} \in \mathbb{Z} \qquad \forall\, g \in \mathcal{G} ``` -#### `ahead` +#### Scalar variable -the cyclic translation forwards, which is a fourth symbol again +an empty dims: a scalar declaration, whose line carries no quantifier ```yaml -ahead: - dims: [snapshot, generator] - expression: p <= shift(p, along=snapshot, offset=-1, edge='wrap') +variables: + reserve: + dims: [] + bounds: { lower: 0 } ``` ```math -p_{t,g} \le p_{t \oplus 1,g} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} +\mathit{reserve} \ge 0 ``` -#### `composed` +#### Scalar variable with a condition -two steps of one policy are one step; a zero step is none at all +scalar too, but masked, so the condition stands with no set beside it ```yaml -composed: - dims: [snapshot, generator] - expression: shift(shift(p, along=snapshot, offset=1), along=snapshot, offset=1) <= shift(p_max, along=generator, offset=0) +variables: + headroom: + dims: [] + where: "budget" + bounds: { lower: 0 } ``` ```math -p_{t - 2,g} \le \mathrm{p}^{\mathrm{max}}_{g} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} +\mathit{headroom} \ge 0 \qquad \text{where } \mathrm{budget} \text{ is defined} ``` -#### `uncomposed` +#### Variable in a special ordered set -a named offset under a numbered one stays two steps, not their sum +the family a sos runs along ```yaml -uncomposed: - dims: [snapshot, generator] - expression: shift(shift(p, along=snapshot, offset=lead, edge=0), along=snapshot, offset=1) <= p_max +variables: + weight: + dims: [snapshot, generator] + bounds: { lower: 0, upper: 1 } ``` ```math -p_{\left( t - 1 \right) \boxminus_{0} \mathrm{lead},g} \le \mathrm{p}^{\mathrm{max}}_{g} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} +0 \le \mathit{weight}_{t,g} \le 1 \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} ``` -#### `crossed` +#### Second axis of a curve -two dimensions translated at one leaf, each with its own policy +a curve's second axis ```yaml -crossed: - dims: [snapshot, generator] - expression: shift(shift(p, along=snapshot, offset=1, edge='wrap'), along=generator, offset=-1) <= p_max +variables: + fuel: + dims: [snapshot, generator] + bounds: { lower: 0 } ``` ```math -p_{t \ominus 1,g + 1} \le \mathrm{p}^{\mathrm{max}}_{g} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} +\mathit{fuel}_{t,g} \ge 0 \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} ``` -#### `lead_time` +#### Third axis of a curve -an offset the data carries, so it prints as a symbol rather than a number +its third, so one curve ties three expressions ```yaml -lead_time: - dims: [snapshot, generator] - expression: shift(p, along=snapshot, offset=lead, edge=0) <= p_max +variables: + heat: + dims: [snapshot, generator] + bounds: { lower: 0 } ``` ```math -p_{t \boxminus_{0} \mathrm{lead},g} \le \mathrm{p}^{\mathrm{max}}_{g} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} +\mathit{heat}_{t,g} \ge 0 \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} ``` -#### `in_season` +#### Variable bounded by a curve -a translation partitioned by a relation: the group rides on the operator +bounded by a curve rather than pinned to it ```yaml -in_season: - dims: [snapshot, generator] - expression: p <= shift(p, along=snapshot, offset=1, edge='wrap', by=season_of, within=season) +variables: + op_cost: + dims: [snapshot, generator] + bounds: { lower: 0 } ``` ```math -p_{t,g} \le p_{t \ominus^{\mathrm{season\_of}(t)} 1,g} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} +\mathit{op\_cost}_{t,g} \ge 0 \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} ``` -#### `held_in_season` +#### Binary variable with a condition -the same group, with a fill: each season's opening row is kept and given a zero +a gate not every unit has, so the curve it gates is ungated where it does not exist ```yaml -held_in_season: - dims: [snapshot, generator] - expression: p <= shift(p, along=snapshot, offset=1, edge=0, by=season_of, within=season) +variables: + warm: + dims: [snapshot, generator] + domain: binary + where: "is_flexible" ``` ```math -p_{t,g} \le p_{t \boxminus_{0}^{\mathrm{season\_of}(t)} 1,g} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} +\mathit{warm}_{t,g} \in \{0, 1\} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} \,:\, \mathrm{is\_flexible}_{g} ``` -#### `window` +### Objective -a trailing window of fixed width +#### Products and powers in the objective + +a sense, a product of two variables, a power over two parameters, a power of one of those, and the summations a scalar objective spells out beside two scalar terms ```yaml -window: - dims: [snapshot, generator] - expression: sum_back(on, along=snapshot, window=3) <= units +objective: + sense: maximize + expression: sum(p * cost) + sum(p * p * cost) + sum(p * cost * growth ** lead) + sum(p * (growth ** lead) ** 2) + sum(p * p_max) - reserve + -headroom ``` ```math -\sum_{t' \in \mathcal{T} \,:\, 0 \le t - t' < 3} \mathit{on}_{t',g} \le \mathit{units}_{g} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} +\max \sum_{t \in \mathcal{T},\ g \in \mathcal{G}} p_{t,g} \cdot \mathrm{cost}_{g} + \sum_{t \in \mathcal{T},\ g \in \mathcal{G}} p_{t,g} \cdot p_{t,g} \cdot \mathrm{cost}_{g} + \sum_{t \in \mathcal{T},\ g \in \mathcal{G}} p_{t,g} \cdot \mathrm{cost}_{g} \cdot \mathrm{growth}^{\mathrm{lead}_{g}} + \sum_{t \in \mathcal{T},\ g \in \mathcal{G}} p_{t,g} \cdot \left( \mathrm{growth}^{\mathrm{lead}_{g}} \right)^{2} + \sum_{t \in \mathcal{T},\ g \in \mathcal{G}} p_{t,g} \cdot \mathrm{p}^{\mathrm{max}}_{g} - \mathit{reserve} - \mathit{headroom} ``` -#### `history` +### Relations -the same window, its width in the data and its edge wrapped +#### Sum through a relation ```yaml -history: - dims: [snapshot, generator] - expression: sum_back(on, along=snapshot, window=min_up, edge='wrap') <= units +constraints: + balance: + dims: [snapshot, bus] + expression: sum(p, by=gen_bus, over=generator, into=bus) + spill - slack == load ``` ```math -\sum_{t' \in \mathcal{T} \,:\, 0 \le t \ominus t' < \mathrm{min\_up}} \mathit{on}_{t',g} \le \mathit{units}_{g} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} +\sum_{g \in \mathcal{G} \,:\, \mathrm{gen\_bus}(g) = b} p_{t,g} + \mathit{spill}_{t} - \mathit{slack}_{t} = \mathrm{load}_{t,b} \qquad \forall\, t \in \mathcal{T},\ b \in \mathcal{B} ``` -#### `seasonal_window` +#### Sum over every dimension -a window partitioned by a relation: the group rides on the operator +a sum naming no dim, whose domain is the one place the dims it took are said ```yaml -seasonal_window: - dims: [snapshot, generator] - expression: sum_back(on, along=snapshot, window=3, by=season_of, within=season) <= units +constraints: + total: + dims: [] + expression: sum(p) <= budget ``` ```math -\sum_{t' \in \mathcal{T} \,:\, 0 \le t -^{\mathrm{season\_of}(t)} t' < 3} \mathit{on}_{t',g} \le \mathit{units}_{g} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} +\sum_{t \in \mathcal{T},\ g \in \mathcal{G}} p_{t,g} \le \mathrm{budget} ``` -#### `pullback` +#### `at` through a relation at(), which re-indexes through a relation instead of an offset ```yaml -pullback: - dims: [snapshot, bus] - expression: spill <= at(zone_cap, by=zone_of, over=zone, into=bus) +constraints: + pullback: + dims: [snapshot, bus] + expression: spill <= at(zone_cap, by=zone_of, over=zone, into=bus) ``` ```math \mathit{spill}_{t} \le \mathrm{zone\_cap}_{\mathrm{zone\_of}(b)} \qquad \forall\, t \in \mathcal{T},\ b \in \mathcal{B} ``` -#### `grouped_once` +#### Sum into two value columns one table read to two value columns: the domain carries a condition per column ```yaml -grouped_once: - dims: [snapshot, bus, technology] - expression: sum(p, by=gen_bt, into=[bus, technology], over=generator) <= tech_cap +constraints: + grouped_once: + dims: [snapshot, bus, technology] + expression: sum(p, by=gen_bt, into=[bus, technology], over=generator) <= tech_cap ``` ```math \sum_{g \in \mathcal{G} \,:\, \mathrm{gen\_bt.bus}(g) = b \wedge \mathrm{gen\_bt.technology}(g) = e} p_{t,g} \le \mathrm{tech\_cap}_{b,e} \qquad \forall\, t \in \mathcal{T},\ b \in \mathcal{B},\ e \in \mathcal{E} ``` -#### `pulled_back_once` +#### `at` through two value columns its adjoint, reading one slot through two columns of one table ```yaml -pulled_back_once: - dims: [generator] - expression: units <= at(tech_cap, by=gen_bt, over=[bus, technology], into=generator) +constraints: + pulled_back_once: + dims: [generator] + expression: units <= at(tech_cap, by=gen_bt, over=[bus, technology], into=generator) ``` ```math \mathit{units}_{g} \le \mathrm{tech\_cap}_{\mathrm{gen\_bt.bus}(g),\mathrm{gen\_bt.technology}(g)} \qquad \forall\, g \in \mathcal{G} ``` -#### `within_bus` +#### Shift within one value column a partition grouped by one named value column of a two-value table, and a position within both ```yaml -within_bus: - dims: [generator] - where: "position(generator, by=gen_bt, within=[bus, technology]) == 0" - expression: units <= shift(units, along=generator, offset=1, edge=0, by=gen_bt, within=bus) +constraints: + within_bus: + dims: [generator] + where: "position(generator, by=gen_bt, within=[bus, technology]) == 0" + expression: units <= shift(units, along=generator, offset=1, edge=0, by=gen_bt, within=bus) ``` ```math \mathit{units}_{g} \le \mathit{units}_{g \boxminus_{0}^{\mathrm{gen\_bt.bus}(g)} 1} \qquad \forall\, g \in \mathcal{G} \,:\, \mathrm{pos}_{\left( \mathrm{gen\_bt.bus}(g),\ \mathrm{gen\_bt.technology}(g) \right)}(g) = 0 ``` -#### `relational` +#### Sum through a bare relation a sum through a bare relation: the domain is a row of the relation rather than a function's value ```yaml -relational: - dims: [snapshot, bus] - expression: sum(p, by=connection, over=generator, into=bus) <= load +constraints: + relational: + dims: [snapshot, bus] + expression: sum(p, by=connection, over=generator, into=bus) <= load ``` ```math \sum_{g \in \mathcal{G} \,:\, \left( g,\ b \right) \in \mathrm{connection}} p_{t,g} \le \mathrm{load}_{t,b} \qquad \forall\, t \in \mathcal{T},\ b \in \mathcal{B} ``` -#### `connected` +#### Bare relation as a condition a bare relation as a where: the row of the frame has to be a member of the relation ```yaml -connected: - dims: [snapshot, generator, bus] - where: "connection" - expression: p <= load +constraints: + connected: + dims: [snapshot, generator, bus] + where: "connection" + expression: p <= load ``` ```math p_{t,g} \le \mathrm{load}_{t,b} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G},\ b \in \mathcal{B} \,:\, \left( g,\ b \right) \in \mathrm{connection} ``` -#### `representative` +#### Map into its own dimension a map into its own dimension, read both ways: the frame is unchanged and the index is primed ```yaml -representative: - dims: [snapshot] - expression: sum(spill, by=rep_of, over=snapshot, into=rep) <= at(spill, by=rep_of, over=rep, into=snapshot) +constraints: + representative: + dims: [snapshot] + expression: sum(spill, by=rep_of, over=snapshot, into=rep) <= at(spill, by=rep_of, over=rep, into=snapshot) ``` ```math \sum_{t' \in \mathcal{T} \,:\, \mathrm{rep\_of}(t') = t} \mathit{spill}_{t'} \le \mathit{spill}_{\mathrm{rep\_of}(t)} \qquad \forall\, t \in \mathcal{T} ``` -#### `zonal` +#### Sum through a two-key map a grouping through a two-key map, consuming one key: the condition reads the other, and the row keeps it ```yaml -zonal: - dims: [snapshot, zone] - expression: sum(p, by=gen_zone, over=generator, into=zone) <= zone_cap +constraints: + zonal: + dims: [snapshot, zone] + expression: sum(p, by=gen_zone, over=generator, into=zone) <= zone_cap ``` ```math \sum_{g \in \mathcal{G} \,:\, \mathrm{gen\_zone}(g,\ t) = z} p_{t,g} \le \mathrm{zone\_cap}_{z} \qquad \forall\, t \in \mathcal{T},\ z \in \mathcal{Z} ``` -#### `zonal_history` +#### Sum over the other key of a two-key map the same table consuming its other key ```yaml -zonal_history: - dims: [generator, zone] - expression: sum(p, by=gen_zone, over=snapshot, into=zone) <= zone_cap +constraints: + zonal_history: + dims: [generator, zone] + expression: sum(p, by=gen_zone, over=snapshot, into=zone) <= zone_cap ``` ```math \sum_{t \in \mathcal{T} \,:\, \mathrm{gen\_zone}(g,\ t) = z} p_{t,g} \le \mathrm{zone\_cap}_{z} \qquad \forall\, g \in \mathcal{G},\ z \in \mathcal{Z} ``` -#### `zonal_membership` +#### Sum between the two keys of a map the same table read between its two key columns: no value column is read, so the domain asks only that the row is there ```yaml -zonal_membership: - dims: [snapshot] - expression: sum(units, by=gen_zone, over=generator, into=snapshot) <= budget +constraints: + zonal_membership: + dims: [snapshot] + expression: sum(units, by=gen_zone, over=generator, into=snapshot) <= budget ``` ```math \sum_{g \in \mathcal{G} \,:\, \mathrm{gen\_zone}(g,\ t) \text{ is defined}} \mathit{units}_{g} \le \mathrm{budget} \qquad \forall\, t \in \mathcal{T} ``` -#### `zonal_pullback` +#### `at` through a two-key map its adjoint, reading the slot the row's own snapshot puts the generator in ```yaml -zonal_pullback: - dims: [snapshot, generator] - where: "gen_zone == 'north' AND position(generator, by=gen_zone, within=zone) == 0" - expression: p <= at(spill * zone_cap, by=gen_zone, into=generator, over=zone) +constraints: + zonal_pullback: + dims: [snapshot, generator] + where: "gen_zone == 'north' AND position(generator, by=gen_zone, within=zone) == 0" + expression: p <= at(spill * zone_cap, by=gen_zone, into=generator, over=zone) ``` ```math p_{t,g} \le \mathit{spill}_{t} \cdot \mathrm{zone\_cap}_{\mathrm{gen\_zone}(g,\ t)} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} \,:\, \mathrm{gen\_zone}(g,\ t) = \text{'}\mathrm{north}\text{'} \wedge \mathrm{pos}_{\mathrm{gen\_zone}(g,\ t)}(g) = 0 ``` -#### `arithmetic` +### Arithmetic and literals + +#### Signs, division and number literals division, both unary signs, a sign beside a sign, floats with and without an exponent, bracketing ```yaml -arithmetic: - dims: [snapshot] - expression: >- - sum(p / 2 + -cost - -1e-5 * p + 2.5e-7 * cost + 0.5 * p, over=generator) - >= -sum(+p, over=generator) * -3 +constraints: + arithmetic: + dims: [snapshot] + expression: >- + sum(p / 2 + -cost - -1e-5 * p + 2.5e-7 * cost + 0.5 * p, over=generator) + >= -sum(+p, over=generator) * -3 ``` ```math \sum_{g \in \mathcal{G}} \left( \frac{p_{t,g}}{2} - \mathrm{cost}_{g} + 10^{-5} \cdot p_{t,g} + 2.5 \times 10^{-7} \cdot \mathrm{cost}_{g} + 0.5 \cdot p_{t,g} \right) \ge -\left( \sum_{g \in \mathcal{G}} p_{t,g} \right) \cdot \left( -3 \right) \qquad \forall\, t \in \mathcal{T} ``` -#### `total` - -a sum naming no dim, whose domain is the one place the dims it took are said - -```yaml -total: - dims: [] - expression: sum(p) <= budget -``` - -```math -\sum_{t \in \mathcal{T},\ g \in \mathcal{G}} p_{t,g} \le \mathrm{budget} -``` - -#### `scalar` - -a parameter over nothing, and a mask that is a bare parameter - -```yaml -scalar: - dims: [generator] - where: "cost" - expression: units <= budget -``` - -```math -\mathit{units}_{g} \le \mathrm{budget} \qquad \forall\, g \in \mathcal{G} \,:\, \mathrm{cost}_{g} \text{ is defined} -``` - -#### `running` - -a mask on a variable's existence, and one on a dimension's label - -```yaml -running: - dims: [snapshot, bus] - where: "theta AND snapshot >= 3" - expression: theta <= load -``` - -```math -\theta_{b} \le \mathrm{load}_{t,b} \qquad \forall\, t \in \mathcal{T},\ b \in \mathcal{B} \,:\, \theta_{b} \text{ exists} \wedge t \ge 3 -``` - -#### `first` - -a position in a dimension, and the same position within a group - -```yaml -first: - dims: [snapshot, generator] - where: "position(snapshot) == 0 OR position(snapshot, by=season_of, within=season) == 0" - expression: on == 1 -``` - -```math -\mathit{on}_{t,g} = 1 \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} \,:\, \mathrm{pos}(t) = 0 \vee \mathrm{pos}_{\mathrm{season\_of}(t)}(t) = 0 -``` - -#### `last` - -the same two counted from the end, which print against a size rather than as themselves - -```yaml -last: - dims: [snapshot, generator] - where: "position(snapshot) == -1 OR position(snapshot, by=season_of, within=season) == -1" - expression: on == 0 -``` - -```math -\mathit{on}_{t,g} = 0 \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} \,:\, \mathrm{pos}(t) = \lvert \mathcal{T} \rvert - 1 \vee \mathrm{pos}_{\mathrm{season\_of}(t)}(t) = \lvert \mathcal{T}_{\mathrm{season\_of}(t)} \rvert - 1 -``` - -#### `northern` - -a relation compared to a label, to another relation, and to nothing - -```yaml -northern: - dims: [snapshot, bus] - where: "zone_of == 'north' AND zone_of != area_of AND zone_of" - expression: slack <= load -``` - -```math -\mathit{slack}_{t} \le \mathrm{load}_{t,b} \qquad \forall\, t \in \mathcal{T},\ b \in \mathcal{B} \,:\, \mathrm{zone\_of}(b) = \text{'}\mathrm{north}\text{'} \wedge \mathrm{zone\_of}(b) \neq \mathrm{area\_of}(b) \wedge \mathrm{zone\_of}(b) \text{ is defined} -``` - -#### `efficiency` +#### Greek parameter name a Greek-named parameter, which is given — so the convention wins and it prints as the word ```yaml -efficiency: - dims: [snapshot, generator] - expression: p <= eta * p_max +constraints: + efficiency: + dims: [snapshot, generator] + expression: p <= eta * p_max ``` ```math p_{t,g} \le \mathrm{eta}_{g} \cdot \mathrm{p}^{\mathrm{max}}_{g} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} ``` -#### `ceiling` +#### Infinity literal the infinity literal, which is the one way infinity prints ```yaml -ceiling: - dims: [bus] - expression: theta <= inf +constraints: + ceiling: + dims: [bus] + expression: theta <= inf ``` ```math \theta_{b} \le \infty \qquad \forall\, b \in \mathcal{B} ``` -#### `always` - -a mask that is only the constant true, which the language says is no mask at all — so none prints - -```yaml -always: - dims: [snapshot] - where: "true" - expression: spill >= 0 -``` - -```math -\mathit{spill}_{t} \ge 0 \qquad \forall\, t \in \mathcal{T} -``` - -#### `redundant` - -the same constant *inside* a mask, where it is what the file says and prints - -```yaml -redundant: - dims: [snapshot] - where: "True AND spill" - expression: spill >= 0 -``` - -```math -\mathit{spill}_{t} \ge 0 \qquad \forall\, t \in \mathcal{T} \,:\, \mathit{spill}_{t} \text{ exists} -``` - -#### `never` - -the other constant mask, which says the rows are none and is worth seeing - -```yaml -never: - dims: [snapshot] - where: "false" - expression: slack >= 0 -``` - -```math -\mathit{slack}_{t} \ge 0 \qquad \forall\, t \in \mathcal{T} \,:\, \bot -``` - -#### `margin` - -a mask comparing two expressions, which prints as the arithmetic it is - -```yaml -margin: - dims: [snapshot, generator] - where: "p_max - p_min > cost / 2" - expression: p <= p_max -``` - -```math -p_{t,g} \le \mathrm{p}^{\mathrm{max}}_{g} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} \,:\, \mathrm{p}^{\mathrm{max}}_{g} - \mathrm{p}^{\mathrm{min}}_{g} > \frac{\mathrm{cost}_{g}}{2} -``` - -#### `ramped` - -a translation under a comparison names its edge, a pullback reads through a relation, and the position keeps the vacated row out - -```yaml -ramped: - dims: [snapshot, bus] - where: "load - shift(load, along=snapshot, offset=1, edge=0) <= at(zone_cap, by=zone_of, over=zone, into=bus) AND position(snapshot) > 0" - expression: slack <= load -``` - -```math -\mathit{slack}_{t} \le \mathrm{load}_{t,b} \qquad \forall\, t \in \mathcal{T},\ b \in \mathcal{B} \,:\, \mathrm{load}_{t,b} - \mathrm{load}_{t \boxminus_{0} 1,b} \le \mathrm{zone\_cap}_{\mathrm{zone\_of}(b)} \wedge \mathrm{pos}(t) > 0 -``` - -#### `covered` - -a reduction on a side of a scalar mask, so nothing is left to quantify - -```yaml -covered: - dims: [] - where: "sum(p_max, over=generator) >= budget" - expression: sum(p) <= budget -``` - -```math -\sum_{t \in \mathcal{T},\ g \in \mathcal{G}} p_{t,g} \le \mathrm{budget} \qquad \text{where } \sum_{g \in \mathcal{G}} \mathrm{p}^{\mathrm{max}}_{g} \ge \mathrm{budget} -``` - -#### `counted` - -a count of the coordinates a predicate admits, which reduces one dim away - -```yaml -counted: - dims: [bus] - where: "count(tech_cap > 0, over=technology) >= 2" - expression: theta <= budget -``` +### Named expressions -```math -\theta_{b} \le \mathrm{budget} \qquad \forall\, b \in \mathcal{B} \,:\, \lvert \{ e \in \mathcal{E} \,:\, \mathrm{tech\_cap}_{b,e} > 0 \} \rvert \ge 2 -``` +#### Plain expression in a constraint -#### `counted_here` - -the same count along a dim the frame carries, so the set takes a primed dummy +names the plain expression: its symbol prints here, its definition once below ```yaml -counted_here: - dims: [bus, technology] - where: "count(tech_cap > 0, over=technology) >= 2" - expression: theta <= tech_cap +constraints: + budgeted: + dims: [snapshot] + expression: spend <= budget ``` ```math -\theta_{b} \le \mathrm{tech\_cap}_{b,e} \qquad \forall\, b \in \mathcal{B},\ e \in \mathcal{E} \,:\, \lvert \{ e' \in \mathcal{E} \,:\, \mathrm{tech\_cap}_{b,e'} > 0 \} \rvert \ge 2 +\mathit{spend}_{t} \le \mathrm{budget} \qquad \forall\, t \in \mathcal{T} ``` -#### `run_start` +#### Cased expression in a constraint -a predicate read one coordinate back, which is false where the translation vacates +names the cased expression: its symbol prints here, its block once below ```yaml -run_start: - dims: [snapshot, bus] - where: "load AND NOT shift(load, along=snapshot, offset=1)" - expression: slack <= load +constraints: + starts: + dims: [snapshot, generator] + expression: p <= startup_cost ``` ```math -\mathit{slack}_{t} \le \mathrm{load}_{t,b} \qquad \forall\, t \in \mathcal{T},\ b \in \mathcal{B} \,:\, \mathrm{load}_{t,b} \text{ is defined} \wedge \neg \left( \mathrm{load}_{t - 1,b} \text{ is defined} \right) +p_{t,g} \le \mathrm{startup\_cost}_{t,g} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} ``` -#### `zoned` +#### Plain named expression -a predicate read through a relation: a bus is held only where its zone has a cap at all +a plain named expression: its symbol prints where it is used, its body once as a definition ```yaml -zoned: - dims: [bus] - where: "at(zone_cap, by=zone_of, over=zone, into=bus)" - expression: theta <= budget +expressions: + spend: + expression: sum(p * cost, over=generator) ``` ```math -\theta_{b} \le \mathrm{budget} \qquad \forall\, b \in \mathcal{B} \,:\, \mathrm{zone\_cap}_{\mathrm{zone\_of}(b)} \text{ is defined} +\mathit{spend}_{t} = \sum_{g \in \mathcal{G}} p_{t,g} \cdot \mathrm{cost}_{g} \qquad \forall\, t \in \mathcal{T} ``` -#### `capped` +#### Expression defined by cases -an expressions: entry on a side, read by the name the file gave it +a quantity defined by region: no two cases overlap, and `otherwise` is the rest ```yaml -capped: - dims: [snapshot, generator] - where: "spend_cap > 0 OR NOT is_flexible" - expression: p <= p_max +expressions: + startup_cost: + dims: [snapshot, generator] + cases: + opening: { when: "position(snapshot) == 0", expression: cost } + winter: { when: "position(snapshot) > 0 and season_of == 'winter'", expression: cost * 2 } + otherwise: 0 ``` ```math -p_{t,g} \le \mathrm{p}^{\mathrm{max}}_{g} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} \,:\, \mathrm{spend}^{\mathrm{cap}}_{g} > 0 \vee \neg \mathrm{is\_flexible}_{g} +\mathrm{startup\_cost}_{t,g} = \begin{cases} \mathrm{cost}_{g} & \text{if } \mathrm{pos}(t) = 0 \\ \mathrm{cost}_{g} \cdot 2 & \text{if } \mathrm{pos}(t) > 0 \wedge \mathrm{season\_of}(t) = \text{'}\mathrm{winter}\text{'} \\ 0 & \text{otherwise} \end{cases} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} ``` -### Definitions - -#### `spend_cap` +#### Data-only expression a data-only entry, so a where may compare it ```yaml -spend_cap: cost * 2 +expressions: + spend_cap: cost * 2 ``` ```math \mathrm{spend}^{\mathrm{cap}}_{g} = \mathrm{cost}_{g} \cdot 2 \qquad \forall\, g \in \mathcal{G} ``` -#### `spend` +#### Named expression in a condition -a plain named expression: its symbol prints where it is used, its body once as a definition +an expressions: entry on a side, read by the name the file gave it ```yaml -spend: - expression: sum(p * cost, over=generator) +constraints: + capped: + dims: [snapshot, generator] + where: "spend_cap > 0 OR NOT is_flexible" + expression: p <= p_max ``` ```math -\mathit{spend}_{t} = \sum_{g \in \mathcal{G}} p_{t,g} \cdot \mathrm{cost}_{g} \qquad \forall\, t \in \mathcal{T} +p_{t,g} \le \mathrm{p}^{\mathrm{max}}_{g} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} \,:\, \mathrm{spend}^{\mathrm{cap}}_{g} > 0 \vee \neg \mathrm{is\_flexible}_{g} ``` -#### `lcoe` +#### Reported expression nothing in the math reads it, so its divisor may carry a variable ```yaml -lcoe: sum(p * cost) / sum(p) +expressions: + lcoe: sum(p * cost) / sum(p) ``` ```math \mathit{lcoe} = \frac{\sum_{t \in \mathcal{T},\ g \in \mathcal{G}} p_{t,g} \cdot \mathrm{cost}_{g}}{\sum_{t \in \mathcal{T},\ g \in \mathcal{G}} p_{t,g}} ``` -#### `marginal_price` +#### Dual of a constraint the row dual of a constraint, the one builtin only an entry the math never reads may call ```yaml -marginal_price: dual(balance) +expressions: + marginal_price: dual(balance) ``` ```math \mathit{marginal\_price}_{t,b} = \lambda_{\mathrm{balance},t,b} \qquad \forall\, t \in \mathcal{T},\ b \in \mathcal{B} ``` -#### `startup_cost` - -a quantity defined by region: no two cases overlap, and `otherwise` is the rest - -```yaml -startup_cost: - dims: [snapshot, generator] - cases: - opening: { when: "position(snapshot) == 0", expression: cost } - winter: { when: "position(snapshot) > 0 and season_of == 'winter'", expression: cost * 2 } - otherwise: 0 -``` - -```math -\mathrm{startup\_cost}_{t,g} = \begin{cases} \mathrm{cost}_{g} & \text{if } \mathrm{pos}(t) = 0 \\ \mathrm{cost}_{g} \cdot 2 & \text{if } \mathrm{pos}(t) > 0 \wedge \mathrm{season\_of}(t) = \text{'}\mathrm{winter}\text{'} \\ 0 & \text{otherwise} \end{cases} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} -``` - -### Variable domains - -#### `p` - -both bounds, and a where with all three connectives - -```yaml -p: - dims: [snapshot, generator] - where: "p_max > 0 AND NOT is_flexible OR p_min > 0" - bounds: { lower: p_min, upper: p_max } -``` - -```math -\mathrm{p}^{\mathrm{min}}_{g} \le p_{t,g} \le \mathrm{p}^{\mathrm{max}}_{g} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} \,:\, \mathrm{p}^{\mathrm{max}}_{g} > 0 \wedge \neg \mathrm{is\_flexible}_{g} \vee \mathrm{p}^{\mathrm{min}}_{g} > 0 -``` +### Shifts -#### `spill` +#### Cyclic and acyclic shift -lower only +roll (cyclic) and shift (acyclic) in one equation ```yaml -spill: - dims: [snapshot] - bounds: { lower: 0 } +constraints: + ramp: + dims: [snapshot, generator] + expression: p - shift(p, along=snapshot, offset=1, edge='wrap') <= shift(p, along=snapshot, offset=1) + p_max ``` ```math -\mathit{spill}_{t} \ge 0 \qquad \forall\, t \in \mathcal{T} +p_{t,g} - p_{t \ominus 1,g} \le p_{t - 1,g} + \mathrm{p}^{\mathrm{max}}_{g} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} ``` -#### `slack` +#### Filled and forward shifts -upper only +the two translations `ramp` leaves out: a fill, and forwards ```yaml -slack: - dims: [snapshot] - bounds: { upper: 100 } +constraints: + edges: + dims: [snapshot, generator] + expression: >- + shift(p, along=snapshot, offset=1, edge=0) + <= shift(p, along=snapshot, offset=-1, edge=0) + p_max ``` ```math -\mathit{slack}_{t} \le 100 \qquad \forall\, t \in \mathcal{T} +p_{t \boxminus_{0} 1,g} \le p_{t \boxplus_{0} 1,g} + \mathrm{p}^{\mathrm{max}}_{g} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} ``` -#### `theta` +#### Cyclic forward shift -unbounded +the cyclic translation forwards, which is a fourth symbol again ```yaml -theta: - dims: [bus] +constraints: + ahead: + dims: [snapshot, generator] + expression: p <= shift(p, along=snapshot, offset=-1, edge='wrap') ``` ```math -\theta_{b} \in \mathbb{R} \qquad \forall\, b \in \mathcal{B} +p_{t,g} \le p_{t \oplus 1,g} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} ``` -#### `on` +#### Composed shifts -a binary domain, which is a set rather than a pair of bounds +two steps of one policy are one step; a zero step is none at all ```yaml -on: - dims: [snapshot, generator] - domain: binary +constraints: + composed: + dims: [snapshot, generator] + expression: shift(shift(p, along=snapshot, offset=1), along=snapshot, offset=1) <= shift(p_max, along=generator, offset=0) ``` ```math -\mathit{on}_{t,g} \in \{0, 1\} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} +p_{t - 2,g} \le \mathrm{p}^{\mathrm{max}}_{g} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} ``` -#### `units` +#### Nested shifts that do not compose -an integer domain, which is both: bounds, and where the values live +a named offset under a numbered one stays two steps, not their sum ```yaml -units: - dims: [generator] - domain: integer - bounds: { lower: 0, upper: 10 } +constraints: + uncomposed: + dims: [snapshot, generator] + expression: shift(shift(p, along=snapshot, offset=lead, edge=0), along=snapshot, offset=1) <= p_max ``` ```math -0 \le \mathit{units}_{g} \le 10, \mathit{units}_{g} \in \mathbb{Z} \qquad \forall\, g \in \mathcal{G} +p_{\left( t - 1 \right) \boxminus_{0} \mathrm{lead},g} \le \mathrm{p}^{\mathrm{max}}_{g} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} ``` -#### `spare` +#### Shift along two dimensions -integer with neither bound: the domain is the whole line +two dimensions translated at one leaf, each with its own policy ```yaml -spare: - dims: [generator] - domain: integer +constraints: + crossed: + dims: [snapshot, generator] + expression: shift(shift(p, along=snapshot, offset=1, edge='wrap'), along=generator, offset=-1) <= p_max ``` ```math -\mathit{spare}_{g} \in \mathbb{Z} \qquad \forall\, g \in \mathcal{G} +p_{t \ominus 1,g + 1} \le \mathrm{p}^{\mathrm{max}}_{g} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} ``` -#### `reserve` +#### Shift by a parameter offset -an empty dims: a scalar declaration, whose line carries no quantifier +an offset the data carries, so it prints as a symbol rather than a number ```yaml -reserve: - dims: [] - bounds: { lower: 0 } +constraints: + lead_time: + dims: [snapshot, generator] + expression: shift(p, along=snapshot, offset=lead, edge=0) <= p_max ``` ```math -\mathit{reserve} \ge 0 +p_{t \boxminus_{0} \mathrm{lead},g} \le \mathrm{p}^{\mathrm{max}}_{g} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} ``` -#### `headroom` +#### Cyclic shift within a group -scalar too, but masked, so the condition stands with no set beside it +a translation partitioned by a relation: the group rides on the operator ```yaml -headroom: - dims: [] - where: "budget" - bounds: { lower: 0 } +constraints: + in_season: + dims: [snapshot, generator] + expression: p <= shift(p, along=snapshot, offset=1, edge='wrap', by=season_of, within=season) ``` ```math -\mathit{headroom} \ge 0 \qquad \text{where } \mathrm{budget} \text{ is defined} +p_{t,g} \le p_{t \ominus^{\mathrm{season\_of}(t)} 1,g} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} ``` -#### `weight` +#### Filled shift within a group -the family a sos runs along +the same group, with a fill: each season's opening row is kept and given a zero ```yaml -weight: - dims: [snapshot, generator] - bounds: { lower: 0, upper: 1 } +constraints: + held_in_season: + dims: [snapshot, generator] + expression: p <= shift(p, along=snapshot, offset=1, edge=0, by=season_of, within=season) ``` ```math -0 \le \mathit{weight}_{t,g} \le 1 \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} -``` - -#### `fuel` - -a curve's second axis - -```yaml -fuel: - dims: [snapshot, generator] - bounds: { lower: 0 } +p_{t,g} \le p_{t \boxminus_{0}^{\mathrm{season\_of}(t)} 1,g} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} ``` -```math -\mathit{fuel}_{t,g} \ge 0 \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} -``` +### Trailing windows -#### `heat` +#### Window of fixed width -its third, so one curve ties three expressions +a trailing window of fixed width ```yaml -heat: - dims: [snapshot, generator] - bounds: { lower: 0 } +constraints: + window: + dims: [snapshot, generator] + expression: sum_back(on, along=snapshot, window=3) <= units ``` ```math -\mathit{heat}_{t,g} \ge 0 \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} +\sum_{t' \in \mathcal{T} \,:\, 0 \le t - t' < 3} \mathit{on}_{t',g} \le \mathit{units}_{g} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} ``` -#### `op_cost` +#### Window with a parameter width -bounded by a curve rather than pinned to it +the same window, its width in the data and its edge wrapped ```yaml -op_cost: - dims: [snapshot, generator] - bounds: { lower: 0 } +constraints: + history: + dims: [snapshot, generator] + expression: sum_back(on, along=snapshot, window=min_up, edge='wrap') <= units ``` ```math -\mathit{op\_cost}_{t,g} \ge 0 \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} +\sum_{t' \in \mathcal{T} \,:\, 0 \le t \ominus t' < \mathrm{min\_up}} \mathit{on}_{t',g} \le \mathit{units}_{g} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} ``` -#### `warm` +#### Window within a group -a gate not every unit has, so the curve it gates is ungated where it does not exist +a window partitioned by a relation: the group rides on the operator ```yaml -warm: - dims: [snapshot, generator] - domain: binary - where: "is_flexible" +constraints: + seasonal_window: + dims: [snapshot, generator] + expression: sum_back(on, along=snapshot, window=3, by=season_of, within=season) <= units ``` ```math -\mathit{warm}_{t,g} \in \{0, 1\} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} \,:\, \mathrm{is\_flexible}_{g} +\sum_{t' \in \mathcal{T} \,:\, 0 \le t -^{\mathrm{season\_of}(t)} t' < 3} \mathit{on}_{t',g} \le \mathit{units}_{g} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} ``` -### Curves +### Piecewise curves A curve prints as the curve it states, over the frame the block builds one per coordinate of, and its expansion prints the rows that curve stands for. One row per `method:`, each from the model named under it, so the symbols in this section are that model's. -#### `economies_of_scale` +#### Adjacency method -**`method: adjacency`** — a binary per segment, and a row making the two nonzero weights neighbours, in `examples/ports/transport_pwl.yaml`. +`method: adjacency` — a binary per segment, and a row making the two nonzero weights neighbours, in `examples/ports/transport_pwl.yaml`. Rendered with the sidecar symbol table `examples/symbols/transport_pwl.yaml`, which is what the breakpoints print as: @@ -1126,11 +939,12 @@ names: ``` ```yaml -economies_of_scale: - over: bp - links: - - [shipment, bp_x] - - [scaled, bp_y] +piecewise: + economies_of_scale: + over: bp + links: + - [shipment, bp_x] + - [scaled, bp_y] ``` ```math @@ -1171,9 +985,9 @@ Written out by `spec.expand()`: \mathrm{x}_{b} \text{ is defined} \wedge \mathrm{y}_{b} \text{ is defined} \qquad \forall\, b \in \mathcal{B} ``` -#### `cost_curve` +#### SOS2 method -**`method: sos2`** — the same weights, restricted by a set the solver branches on (the sos rules), in `examples/sos.yaml`. +`method: sos2` — the same weights, restricted by a set the solver branches on (the sos rules), in `examples/sos.yaml`. Rendered with the sidecar symbol table `examples/symbols/sos.yaml`, which is what the breakpoints print as: @@ -1187,12 +1001,13 @@ names: ``` ```yaml -cost_curve: - over: bp - links: - - [dispatch, bp_x] - - [op_cost, bp_y] - method: sos2 +piecewise: + cost_curve: + over: bp + links: + - [dispatch, bp_x] + - [op_cost, bp_y] + method: sos2 ``` ```math @@ -1225,9 +1040,9 @@ Written out by `spec.expand()`: \mathrm{x}_{g,b} \text{ is defined} \wedge \mathrm{y}_{g,b} \text{ is defined} \qquad \forall\, g \in \mathcal{G},\ b \in \mathcal{B} ``` -#### `cost_curve` +#### Convex method -**`method: convex`** — nothing — the weights range over the hull, which is a pure LP, in `examples/piecewise.yaml`. +`method: convex` — nothing — the weights range over the hull, which is a pure LP, in `examples/piecewise.yaml`. Rendered with the sidecar symbol table `examples/symbols/piecewise.yaml`, which is what the breakpoints print as: @@ -1241,12 +1056,13 @@ names: ``` ```yaml -cost_curve: - over: bp - links: - - [dispatch, bp_x] - - [op_cost, bp_y] - method: convex +piecewise: + cost_curve: + over: bp + links: + - [dispatch, bp_x] + - [op_cost, bp_y] + method: convex ``` ```math @@ -1283,9 +1099,9 @@ Written out by `spec.expand()`: \lvert \{ b \in \mathcal{B} \,:\, \left( \mathrm{y}_{g,b} - \mathrm{y}_{g,b \boxminus_{0} 1} \right) \cdot \left( \mathrm{x}_{g,b \boxplus_{0} 1} - \mathrm{x}_{g,b} \right) > \left( \mathrm{y}_{g,b \boxplus_{0} 1} - \mathrm{y}_{g,b} \right) \cdot \left( \mathrm{x}_{g,b} - \mathrm{x}_{g,b \boxminus_{0} 1} \right) \wedge \mathrm{pos}(b) > 0 \wedge \mathrm{pos}(b) \neq \lvert \mathcal{B} \rvert - 1 \} \rvert = 0 \vee \lvert \{ b \in \mathcal{B} \,:\, \left( \mathrm{y}_{g,b} - \mathrm{y}_{g,b \boxminus_{0} 1} \right) \cdot \left( \mathrm{x}_{g,b \boxplus_{0} 1} - \mathrm{x}_{g,b} \right) < \left( \mathrm{y}_{g,b \boxplus_{0} 1} - \mathrm{y}_{g,b} \right) \cdot \left( \mathrm{x}_{g,b} - \mathrm{x}_{g,b \boxminus_{0} 1} \right) \wedge \mathrm{pos}(b) > 0 \wedge \mathrm{pos}(b) \neq \lvert \mathcal{B} \rvert - 1 \} \rvert = 0 \qquad \forall\, g \in \mathcal{G} ``` -#### `cost_curve` +#### LP method -**`method: lp`** — no weights at all — one row per segment line, plus the two rows holding the domain, in `examples/piecewise_lp.yaml`. +`method: lp` — no weights at all — one row per segment line, plus the two rows holding the domain, in `examples/piecewise_lp.yaml`. Rendered with the sidecar symbol table `examples/symbols/piecewise_lp.yaml`, which is what the breakpoints print as: @@ -1298,12 +1114,13 @@ names: ``` ```yaml -cost_curve: - over: bp - links: - - [dispatch, bp_x] - - [op_cost, bp_y, ">="] - method: lp +piecewise: + cost_curve: + over: bp + links: + - [dispatch, bp_x] + - [op_cost, bp_y, ">="] + method: lp ``` ```math @@ -1340,19 +1157,20 @@ Written out by `spec.expand()`: \lvert \{ b \in \mathcal{B} \,:\, \mathrm{x}_{g,b} \text{ is defined} \} \rvert \ge 2 \qquad \forall\, g \in \mathcal{G} ``` -### Sets carried to the solver +### Special ordered sets A set prints beside the variable it restricts, because it restricts that variable rather than adding a row of its own. Under it are the rows it is written out as. -#### `adjacent` +#### Special ordered set of type 2 at most two adjacent members nonzero, one set per snapshot ```yaml -adjacent: - variable: weight - along: generator - type: 2 +sos: + adjacent: + variable: weight + along: generator + type: 2 ``` ```math @@ -1373,107 +1191,357 @@ Written out by `spec.expand()`: \mathit{adjacent\_seg}_{t,g} \in \{0, 1\} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} ``` -### What the data has to satisfy +### Assumptions -#### `bounds_do_not_cross` +#### Two parameters compared two parameters, which is arithmetic like any other ```yaml -bounds_do_not_cross: "p_min <= p_max" +assumptions: + bounds_do_not_cross: "p_min <= p_max" ``` ```math \mathrm{p}^{\mathrm{min}}_{g} \le \mathrm{p}^{\mathrm{max}}_{g} \qquad \forall\, g \in \mathcal{G} ``` -#### `efficiency_is_a_fraction` +#### Connective in an assumption a connective, so the line has no relation to align on ```yaml -efficiency_is_a_fraction: "eta > 0 AND eta <= 1" +assumptions: + efficiency_is_a_fraction: "eta > 0 AND eta <= 1" ``` ```math \mathrm{eta}_{g} > 0 \wedge \mathrm{eta}_{g} \le 1 \qquad \forall\, g \in \mathcal{G} ``` -#### `lead_times_are_short` +#### Parameter compared to a literal one parameter against a literal ```yaml -lead_times_are_short: "lead <= 3" +assumptions: + lead_times_are_short: "lead <= 3" ``` ```math \mathrm{lead}_{g} \le 3 \qquad \forall\, g \in \mathcal{G} ``` -#### `zones_agree` +#### Two relations compared two maps into one set, compared row by row ```yaml -zones_agree: "zone_of == area_of" +assumptions: + zones_agree: "zone_of == area_of" ``` ```math \mathrm{zone\_of}(b) = \mathrm{area\_of}(b) \qquad \forall\, b \in \mathcal{B} ``` -#### `budget_covers_the_peak` +#### Reduction in an assumption a reduction on a side, leaving nothing to quantify ```yaml -budget_covers_the_peak: "sum(p_max, over=generator) >= budget" +assumptions: + budget_covers_the_peak: "sum(p_max, over=generator) >= budget" ``` ```math \sum_{g \in \mathcal{G}} \mathrm{p}^{\mathrm{max}}_{g} \ge \mathrm{budget} ``` -#### `ramps_are_gentle` +#### Shift in an assumption a translation inside arithmetic, and a position keeping the vacated row out ```yaml -ramps_are_gentle: - holds: "load - shift(load, along=snapshot, offset=1, edge=0) <= budget" - where: "position(snapshot) > 0" +assumptions: + ramps_are_gentle: + holds: "load - shift(load, along=snapshot, offset=1, edge=0) <= budget" + where: "position(snapshot) > 0" ``` ```math \mathrm{load}_{t,b} - \mathrm{load}_{t \boxminus_{0} 1,b} \le \mathrm{budget} \qquad \forall\, t \in \mathcal{T},\ b \in \mathcal{B} \,:\, \mathrm{pos}(t) > 0 ``` -#### `flexible_units_have_headroom` +#### Assumption with a boolean condition a bare bool parameter as the where ```yaml -flexible_units_have_headroom: - holds: "p_min < p_max" - where: "is_flexible" +assumptions: + flexible_units_have_headroom: + holds: "p_min < p_max" + where: "is_flexible" ``` ```math \mathrm{p}^{\mathrm{min}}_{g} < \mathrm{p}^{\mathrm{max}}_{g} \qquad \forall\, g \in \mathcal{G} \,:\, \mathrm{is\_flexible}_{g} ``` -#### `northern_demand_is_real` +#### Assumption with a relation condition a relation comparison as the where, over a frame two dims wide ```yaml -northern_demand_is_real: - holds: "load >= 0" - where: "zone_of == 'north'" +assumptions: + northern_demand_is_real: + holds: "load >= 0" + where: "zone_of == 'north'" ``` ```math \mathrm{load}_{t,b} \ge 0 \qquad \forall\, t \in \mathcal{T},\ b \in \mathcal{B} \,:\, \mathrm{zone\_of}(b) = \text{'}\mathrm{north}\text{'} ``` + +### Where conditions + +#### Parameter as a condition + +a parameter over nothing, and a mask that is a bare parameter + +```yaml +constraints: + scalar: + dims: [generator] + where: "cost" + expression: units <= budget +``` + +```math +\mathit{units}_{g} \le \mathrm{budget} \qquad \forall\, g \in \mathcal{G} \,:\, \mathrm{cost}_{g} \text{ is defined} +``` + +#### Variable and label conditions + +a mask on a variable's existence, and one on a dimension's label + +```yaml +constraints: + running: + dims: [snapshot, bus] + where: "theta AND snapshot >= 3" + expression: theta <= load +``` + +```math +\theta_{b} \le \mathrm{load}_{t,b} \qquad \forall\, t \in \mathcal{T},\ b \in \mathcal{B} \,:\, \theta_{b} \text{ exists} \wedge t \ge 3 +``` + +#### Position in a dimension + +a position in a dimension, and the same position within a group + +```yaml +constraints: + first: + dims: [snapshot, generator] + where: "position(snapshot) == 0 OR position(snapshot, by=season_of, within=season) == 0" + expression: on == 1 +``` + +```math +\mathit{on}_{t,g} = 1 \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} \,:\, \mathrm{pos}(t) = 0 \vee \mathrm{pos}_{\mathrm{season\_of}(t)}(t) = 0 +``` + +#### Position counted from the end + +the same two counted from the end, which print against a size rather than as themselves + +```yaml +constraints: + last: + dims: [snapshot, generator] + where: "position(snapshot) == -1 OR position(snapshot, by=season_of, within=season) == -1" + expression: on == 0 +``` + +```math +\mathit{on}_{t,g} = 0 \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} \,:\, \mathrm{pos}(t) = \lvert \mathcal{T} \rvert - 1 \vee \mathrm{pos}_{\mathrm{season\_of}(t)}(t) = \lvert \mathcal{T}_{\mathrm{season\_of}(t)} \rvert - 1 +``` + +#### Relation compared to a label + +a relation compared to a label, to another relation, and to nothing + +```yaml +constraints: + northern: + dims: [snapshot, bus] + where: "zone_of == 'north' AND zone_of != area_of AND zone_of" + expression: slack <= load +``` + +```math +\mathit{slack}_{t} \le \mathrm{load}_{t,b} \qquad \forall\, t \in \mathcal{T},\ b \in \mathcal{B} \,:\, \mathrm{zone\_of}(b) = \text{'}\mathrm{north}\text{'} \wedge \mathrm{zone\_of}(b) \neq \mathrm{area\_of}(b) \wedge \mathrm{zone\_of}(b) \text{ is defined} +``` + +#### Constant true condition + +a mask that is only the constant true, which the language says is no mask at all — so none prints + +```yaml +constraints: + always: + dims: [snapshot] + where: "true" + expression: spill >= 0 +``` + +```math +\mathit{spill}_{t} \ge 0 \qquad \forall\, t \in \mathcal{T} +``` + +#### Constant true inside a condition + +the same constant *inside* a mask, where it is what the file says and prints + +```yaml +constraints: + redundant: + dims: [snapshot] + where: "True AND spill" + expression: spill >= 0 +``` + +```math +\mathit{spill}_{t} \ge 0 \qquad \forall\, t \in \mathcal{T} \,:\, \mathit{spill}_{t} \text{ exists} +``` + +#### Constant false condition + +the other constant mask, which says the rows are none and is worth seeing + +```yaml +constraints: + never: + dims: [snapshot] + where: "false" + expression: slack >= 0 +``` + +```math +\mathit{slack}_{t} \ge 0 \qquad \forall\, t \in \mathcal{T} \,:\, \bot +``` + +#### Comparison of two expressions + +a mask comparing two expressions, which prints as the arithmetic it is + +```yaml +constraints: + margin: + dims: [snapshot, generator] + where: "p_max - p_min > cost / 2" + expression: p <= p_max +``` + +```math +p_{t,g} \le \mathrm{p}^{\mathrm{max}}_{g} \qquad \forall\, t \in \mathcal{T},\ g \in \mathcal{G} \,:\, \mathrm{p}^{\mathrm{max}}_{g} - \mathrm{p}^{\mathrm{min}}_{g} > \frac{\mathrm{cost}_{g}}{2} +``` + +#### Shift and pullback in a condition + +a translation under a comparison names its edge, a pullback reads through a relation, and the position keeps the vacated row out + +```yaml +constraints: + ramped: + dims: [snapshot, bus] + where: "load - shift(load, along=snapshot, offset=1, edge=0) <= at(zone_cap, by=zone_of, over=zone, into=bus) AND position(snapshot) > 0" + expression: slack <= load +``` + +```math +\mathit{slack}_{t} \le \mathrm{load}_{t,b} \qquad \forall\, t \in \mathcal{T},\ b \in \mathcal{B} \,:\, \mathrm{load}_{t,b} - \mathrm{load}_{t \boxminus_{0} 1,b} \le \mathrm{zone\_cap}_{\mathrm{zone\_of}(b)} \wedge \mathrm{pos}(t) > 0 +``` + +#### Reduction in a scalar condition + +a reduction on a side of a scalar mask, so nothing is left to quantify + +```yaml +constraints: + covered: + dims: [] + where: "sum(p_max, over=generator) >= budget" + expression: sum(p) <= budget +``` + +```math +\sum_{t \in \mathcal{T},\ g \in \mathcal{G}} p_{t,g} \le \mathrm{budget} \qquad \text{where } \sum_{g \in \mathcal{G}} \mathrm{p}^{\mathrm{max}}_{g} \ge \mathrm{budget} +``` + +#### Count over a dimension + +a count of the coordinates a predicate admits, which reduces one dim away + +```yaml +constraints: + counted: + dims: [bus] + where: "count(tech_cap > 0, over=technology) >= 2" + expression: theta <= budget +``` + +```math +\theta_{b} \le \mathrm{budget} \qquad \forall\, b \in \mathcal{B} \,:\, \lvert \{ e \in \mathcal{E} \,:\, \mathrm{tech\_cap}_{b,e} > 0 \} \rvert \ge 2 +``` + +#### Count along a dimension of the frame + +the same count along a dim the frame carries, so the set takes a primed dummy + +```yaml +constraints: + counted_here: + dims: [bus, technology] + where: "count(tech_cap > 0, over=technology) >= 2" + expression: theta <= tech_cap +``` + +```math +\theta_{b} \le \mathrm{tech\_cap}_{b,e} \qquad \forall\, b \in \mathcal{B},\ e \in \mathcal{E} \,:\, \lvert \{ e' \in \mathcal{E} \,:\, \mathrm{tech\_cap}_{b,e'} > 0 \} \rvert \ge 2 +``` + +#### Predicate at the previous coordinate + +a predicate read one coordinate back, which is false where the translation vacates + +```yaml +constraints: + run_start: + dims: [snapshot, bus] + where: "load AND NOT shift(load, along=snapshot, offset=1)" + expression: slack <= load +``` + +```math +\mathit{slack}_{t} \le \mathrm{load}_{t,b} \qquad \forall\, t \in \mathcal{T},\ b \in \mathcal{B} \,:\, \mathrm{load}_{t,b} \text{ is defined} \wedge \neg \left( \mathrm{load}_{t - 1,b} \text{ is defined} \right) +``` + +#### Predicate through a relation + +a predicate read through a relation: a bus is held only where its zone has a cap at all + +```yaml +constraints: + zoned: + dims: [bus] + where: "at(zone_cap, by=zone_of, over=zone, into=bus)" + expression: theta <= budget +``` + +```math +\theta_{b} \le \mathrm{budget} \qquad \forall\, b \in \mathcal{B} \,:\, \mathrm{zone\_cap}_{\mathrm{zone\_of}(b)} \text{ is defined} +``` diff --git a/docs/reference/reading.md b/docs/reference/reading.md index 54211619..42cc7395 100644 --- a/docs/reference/reading.md +++ b/docs/reference/reading.md @@ -6,22 +6,17 @@ SPDX-License-Identifier: CC-BY-4.0 # Reading a loaded model This page is for whoever writes an engine that builds models, a renderer, or a -checker. You need none of it to write a model. A tool reads the model through -two objects, and one door: - -```text -to_spec → Spec → .program → Program -``` +checker. A tool reads the model through two objects, `Spec` and `Program`. ## `Spec` and `Program` A `Spec` holds the file as written: its `macros:`, its descriptions, and a `piecewise:` block as one block. A `Program` holds the model the file builds: every macro expanded, every name typed, every operator resolved to a node, and -every dimension and degree rule already checked. A curve stays one curve there; -`spec.expand('piecewise')` turns it into the variables and constraints it -stands for. [The file and the program](../about/file-and-program.md) says why -the two are split, and which tool reads which. +every dimension and degree rule already checked. A curve stays one curve there +until [`spec.expand()`](#formulations-written-out) writes it out. +[The file and the program](../about/file-and-program.md) says why the two are +split, and which tool reads which. The curve below [expands](language/piecewise.md) into a weight per breakpoint, a convexity row and one row per link: @@ -76,52 +71,29 @@ sorted(rows.variables) # ['cost', 'curve_lam', 'p'] `to_spec` takes a path, the YAML, a mapping or a `Spec`. `spec.program` is the program built when the model loaded, so every ask on one model returns one object. A `piecewise:` block is a curve under `program.piecewise`, typed, and a -`sos:` block is a set under `program.sos`. `spec.expand('piecewise')` is the -model with each curve written out as rows, and `spec.expand()` writes the sets -out too. - -Nothing in the package expands a model unasked. A consumer that builds rows -calls `spec.expand('piecewise')` at its own door. A consumer that cannot take a -curve refuses it in its own words, naming that call: - -```python -def rows_of(program): - if program.piecewise: - raise ValueError(f"{sorted(program.piecewise)} are curves; pass spec.expand('piecewise')") - return program - - -rows_of(rows) is rows # True -``` +`sos:` block is a set under `program.sos`. Every parameter the program declares +is one the file declared. ## Formulations written out -`Spec.expand()` returns a `Spec` whose formulations — `piecewise:` and `sos:` — -are stated as the variables and constraints they stand for. It is the same math, -bound by the same data, and it is what to print for a reader who wants the rows -rather than the curve: +A program holds each curve and each set as one declaration until +[`Spec.expand()`](api.md#math_spec.Spec.expand) writes it out. An engine that +builds rows reads the program of `spec.expand('piecewise')` if it takes a set, +and the program of `spec.expand()` if it does not. The program of an expansion +holds no curve: ```python -sorted(spec.expand().variables) # ['cost', 'curve_lam', 'p'] -sorted(spec.expand().constraints) # ['curve_convexity', 'curve_link0', 'curve_link1', 'target'] -spec.expand() == spec.expand() # True +sorted(rows.piecewise) # [] ``` -A consumer that takes a set reads the program of `spec.expand('piecewise')`, -and one that does not reads the program of `spec.expand()`. The -[piecewise page](language/piecewise.md#what-a-set-is-written-out-as) says what a -set is written out as. - -Every parameter the program declares is one the file declared, and the engine -binds each from its data. The program of an expansion keeps no curve: the -rows, the weights and the conditions the method states are declarations like -any other. - ## What the data has to satisfy -`program.assumptions` holds every fact the numbers have to meet, by the name a -refusal quotes. The engine, which has the numbers, runs each one and raises -`assumption_message` where it fails: +`program.assumptions` maps a name to an `Assumption`: each entry the file +declared, and each one a curve's method derives +([what a curve assumes](language/assumptions.md#what-a-curve-assumes)). An +`Assumption` carries a `predicate` and the `where` it is checked under, both +masks, and the `description` a refusal ends with. `assumption_message` returns +the message for an assumption the data does not meet: ```python from math_spec.program import Assumption, assumption_message @@ -129,24 +101,15 @@ from math_spec.program import Assumption, assumption_message sorted(program.assumptions) # ['cost_is_never_negative', 'curve_complete', 'curve_curvature', 'curve_increasing'] isinstance(program.assumptions['curve_increasing'], Assumption) # True message = assumption_message('curve_increasing', program.assumptions['curve_increasing']) -message # "assumption 'curve_increasing' does not hold for the data bound to 'bp_x' — piecewise 'curve': method: convex requires strictly increasing breakpoints in 'bp_x' along 'bp'" +message # "assumption 'curve_increasing' does not hold for the data attached to 'bp_x' — piecewise 'curve': method: convex requires strictly increasing breakpoints in 'bp_x' along 'bp'" written = assumption_message('cost_is_never_negative', program.assumptions['cost_is_never_negative']) -written # "assumption 'cost_is_never_negative' does not hold for the data bound to 'bp_y' — a negative cost is a gain the objective would chase" +written # "assumption 'cost_is_never_negative' does not hold for the data attached to 'bp_y' — a negative cost is a gain the objective would chase" ``` -One kind stands in that mapping. An `Assumption` carries a predicate as two masks — -`predicate`, and the `where` it is checked under — and the sentence a refusal -trails under `description`. What a `piecewise:` block's method implies about -its breakpoints is written in the same language and stands beside what the -file wrote: `expand()` emits those entries, and a model that still declares -the block derives the same text at load. So a consumer reads one kind, and a -condition a method adds later is a row in that mapping rather than a case to -handle. - ## Nodes and masks -You never build a node yourself. The node classes are exported so that you can -test one with `isinstance` and read its fields. `children()` walks an expression +The node classes live in `math_spec.program`, for `isinstance` tests and field +reads. `children()` walks an expression node's operands, and `where_children()` walks a predicate's. `walk()` yields every node under an expression, parents first. `walk_regions()` yields each node with the `cases:` regions it stands inside, outermost first. @@ -154,7 +117,7 @@ with the `cases:` regions it stands inside, outermost first. A `Named` stands where an `expressions:` entry is used. Its `body` is the entry's expression, the same object that `program.expressions[name].expression` holds, and its value is the body's value. `children()` steps into the body, so -a walk reads through it; a renderer prints the name where the file wrote it. +a walk reads through it. Every `where` arrives as a `Mask`. Its `.root` is the resolved predicate. The mask also answers four questions: @@ -171,30 +134,26 @@ the sides read, the relation a grouping reads through included. A name compared against a literal does not arrive this way. `p_max > 5` is a `ParameterComparison` and `1 * p_max > 5` is an `ExpressionComparison`, though -both mask the same coordinates. Match both where you read a comparison over -parameters. +both mask the same coordinates. Three predicates read another predicate rather than a declaration. A `CountComparison` carries the mask it counts and the dimension it counts away. A `TranslatedPredicate` carries the mask it reads at a neighbouring coordinate. A `PulledBackPredicate` carries the mask it reads through a relation, and the `Direction` it reads in. Each holds that mask as a `Mask`, -where a connective holds a bare predicate: the walk recurses through a -connective and stops at these, so read the field where you need what is -inside. `.names_read` and `.dims` already see through all three, and the -relation a `PulledBackPredicate` reads is in its `.names_read`. +where a connective holds a bare predicate, so the walk recurses through a +connective and stops at these. `.names_read` and `.dims` see through all three, +and the relation a `PulledBackPredicate` reads is in its `.names_read`. -A predicate you build yourself answers the same four questions: wrap it in -`Mask`, or build it there with `~`, `&` and `|`. A mask folds as it is built, -so a boolean literal stands at a mask's root or nowhere. A `Region`'s `when` -arrives as a `Mask` too. The node classes live in `math_spec.program`. +`Mask(predicate)` answers the same four questions of any resolved predicate, +and `~`, `&` and `|` combine masks into a mask. A mask folds as it is built, so a boolean literal +stands at a mask's root or nowhere. A `Region`'s `when` is a `Mask` too. ## Asking what a program uses `program.footprint` says which of the language's constructs one model uses. -It answers for the rows the program holds, and a curve still on the program is -not a row. Ask it of the rows a solver takes, since a curve written out uses -more of the language than the block did: +It answers for the rows the program holds. A curve still on the program is not +a row, so its constructs count on the program of the expansion: ```python footprint = rows.footprint @@ -205,11 +164,10 @@ sorted(footprint.sos_types) # [] sorted(kind.__name__ for kind in footprint.kinds) # ['Constant', 'Multiply', 'Parameter', 'Sum', 'Variable'] ``` -Every field is a set. An empty field means this model does not use the -construct. The footprint says what the model uses. Whether your solver or -file format can take a construct is your question -([what a solver can take](../about/limits.md#solver-capability)). Whether a -quadratic form is convex is not reported, because it depends on the numbers. +Every field is a set, and an empty field means the model does not use the +construct. Whether a solver takes a construct is the engine's question +([what counts as language](../about/what-counts-as-language.md#what-each-tool-decides-for-itself)). +Convexity is not reported: it depends on the numbers. ## Asking whether an axis can be cut @@ -257,5 +215,5 @@ about whether the windowed answer equals the whole-horizon answer. that data as a file. Both round-trip, so `to_spec(spec.to_dict()) == spec`. `to_yaml()` writes every value and omits every absence. `domain: continuous` is -written out. A `null`, an infinite bound and an empty section are left out. +written out. A `null` and an empty section are left out. `dims: []` is written, because it says the declaration is a scalar. diff --git a/docs/reference/typeset.md b/docs/reference/typeset.md index cb8d2231..09fbe747 100644 --- a/docs/reference/typeset.md +++ b/docs/reference/typeset.md @@ -6,7 +6,7 @@ SPDX-License-Identifier: CC-BY-4.0 # Typeset the math `to_latex`, `to_typst` and `to_markdown` print a model as the equations it -stands for, from the file alone. No data binds, and no solver runs. +stands for, from the file alone. No data is attached, and no solver runs. ```python import math_spec as ms @@ -19,22 +19,18 @@ print(ms.to_markdown(spec)) # renders as-is on GitHub ``` Each function takes a path, the YAML, a mapping, a `Spec` or a `Program`, and -prints the program: the one a spec holds, or the one it was handed. The same -three formats come from a shell: - -```bash -python -m math_spec latex model.yaml --symbols model.symbols.yaml --standalone -o model.tex -python -m math_spec typst model.yaml --standalone -o model.typ -python -m math_spec markdown model.yaml -``` +prints the program: the one a spec holds, or the one it was handed. +From a shell, `python -m math_spec latex model.yaml` prints the same, and +`typst` or `markdown` in place of `latex` picks the format. [Print a model as math](../howto/print.md) is the recipe, and -[every construct, as math](notation.md) shows what each construct prints. +[every operator as math](language/operators.md#every-operator-as-math) shows +what each operator prints. ## Options The three functions take the same keywords, and the command line spells each as -a flag. +a flag. The [Python API](api.md#typesetting) gives each signature. | | | | | -------------------- | ---------------------- | ------------------------------------------------------------------------------------------------------------------------------- | @@ -49,9 +45,9 @@ a flag. - The model's `description:` opens the document. - A `piecewise:` block prints as one line: the curve it states, over the frame - it states one curve per coordinate of. To print the variables and constraints - it stands for instead, print - [`spec.expand()`](language/piecewise.md#writing-a-formulation-out). + it states one curve per coordinate of. To print its rows, print + [`spec.expand()`](api.md#math_spec.Spec.expand) or pass `--expand` + ([see an expansion](../howto/see-an-expansion.md)). - An [`assumptions:`](language/assumptions.md) entry prints under an **Assumptions** heading, last, beside what each curve assumes of its breakpoints. A model that assumes nothing of its data prints no such @@ -107,57 +103,14 @@ expressions it uses are substituted. A cased expression prints by symbol, and a second call with its name prints its block. A name that is none of the four kinds is refused with the near miss. A name -declared as two of them, such as a constraint and a variable, is refused too, -because one line can print only one of them. - -## Printing what a formulation states - -A `piecewise:` block and a `sos:` block each state variables and constraints -([formulations](language/piecewise.md#writing-a-formulation-out)). Printing -those rows is printing a different model, so it is -[`expand()`](language/piecewise.md#writing-a-formulation-out) that produces it -and not an option on the render: - -```python -ms.to_latex(spec) # the curve, and the set beside its variable -ms.to_latex(spec.expand()) # the weights, the convexity row, the binaries -ms.to_latex(spec.expand('sos')) # the curves as curves, the sets as binaries -``` - -A shell cannot compose that, so the command line spells it as a flag: - -```bash -python -m math_spec latex model.yaml --expand --symbols model.symbols.yaml -``` - -One symbol table serves both, because a name a formulation emits counts as -declared — which is what lets `_lam` print as $\lambda$ in the expansion -and the same table render the file it came from. +declared as two of them, such as a constraint and a variable, is refused too. ## Symbol tables With no table, the symbols are **derived** from the names in the file, such as $\mathrm{load}_t$ and $\mathrm{capacity}_g$. A symbol table makes the output -conventional: - -```python -symbols = { - 'notation': 'latex', - 'dimensions': { - 'snapshot': {'index': 's', 'set': '\\mathcal{S}'}, - 'generator': {'index': 'g', 'set': '\\mathcal{G}'}, - }, - 'names': { - 'cost': 'c', - 'load': '\\ell', - 'capacity': '\\bar p', - }, -} - -ms.to_latex('dispatch.yaml', symbols=symbols) -``` - -Pass a dict, a path to a YAML file, or a `ms.SymbolTable`. As a file: +conventional. Pass a path to a YAML file, the same keys as a dict, or a +`ms.SymbolTable`: ```yaml # dispatch.symbols.yaml @@ -171,6 +124,10 @@ names: capacity: "\\bar p" ``` +```python +ms.to_latex('dispatch.yaml', symbols='dispatch.symbols.yaml') +``` + | Section | | | ------------ | ------------------------------------------------------------------------------- | | `notation` | **Required.** `latex` or `typst`: the language the entries are written in | @@ -181,5 +138,4 @@ Every spelling is printed as you wrote it, and nothing translates notation, so rendering a LaTeX table as Typst is refused. A key that names nothing in the model, and nothing a formulation of it emits, is an error with the near miss. -Nothing in a symbol table changes what the file means. What a declaration _is_ -stays in its own `description:`. +Nothing in a symbol table changes what the file means. diff --git a/docs/static/hooks.py b/docs/static/hooks.py index db580209..d99cd015 100644 --- a/docs/static/hooks.py +++ b/docs/static/hooks.py @@ -118,7 +118,7 @@ def _py_to_md(filepath: Path, api_nav: dict, config: dict) -> File: def _update_nav(api_nav: dict, config: dict) -> None: - """Update mkdocs navigation tree with the Python API sub-tree. + """Append the per-module API pages to the Contributing group of Development, as `Modules`. Mkdocs navigation is composed of lists of dictionaries. Lists nesting defines navigation nesting, dictionary keys are the page names, and values are the pointers to markdown files. @@ -127,18 +127,8 @@ def _update_nav(api_nav: dict, config: dict) -> None: api_nav (dict): Python API navigation tree. config (dict): mkdocs config dictionary (in which `nav` can be found). """ - api_reference_nav = {'Python API': [*api_nav.pop('top_level'), *[{k: v} for k, v in api_nav.items()]]} - reference = _get_nav_list(config['nav'], 'Reference') - reference.insert(_index_of(reference, 'Examples'), api_reference_nav) - - -def _index_of(nav: list[dict | str], ref: str) -> int: - """Where the entry titled `ref` sits in `nav`, or the end when there is none. - - The Examples entry is the model pages, which the nav keeps last in the - Reference section; the API is a lookup page and goes before them. - """ - return next((i for i, idx in enumerate(nav) if isinstance(idx, dict) and set(idx.keys()) == {ref}), len(nav)) + modules_nav = {'Modules': [*api_nav.pop('top_level'), *[{k: v} for k, v in api_nav.items()]]} + _get_nav_list(_get_nav_list(config['nav'], 'Development'), 'Contributing').append(modules_nav) def _get_nav_list(nav: list[dict | str], ref: str) -> list: diff --git a/mkdocs.yml b/mkdocs.yml index f3e7e9bd..7d7b25de 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -26,13 +26,13 @@ validation: nav: - Home: index.md # Arranged by what a page is for — tutorial, how-to guide, reference, - # explanation (https://diataxis.fr). `.claude/skills/docs-writing/SKILL.md` - # says how one kind is told from another, and the folders under `docs/` - # follow the same split. There is no tutorial yet; the first one opens a - # `Tutorials:` section here, above the how-to guides. The model pages are - # reference in their own form and sit at the end of the Reference section, - # after the pages a reader looks things up in; `docs/static/hooks.py` puts - # the Python API in front of them. + # explanation (https://diataxis.fr) — for someone who writes a model. + # `.claude/skills/docs-writing/SKILL.md` says how one kind is told from + # another, and the folders under `docs/` follow the same split. The model + # pages are reference in their own form and sit at the end of the Reference + # section, after the pages a reader looks things up in. + - Tutorials: + - Your first model: first-model.md - How-to guides: - Installation: howto/installation.md - Check a model without data: howto/check.md @@ -56,29 +56,40 @@ nav: - Assumptions: reference/language/assumptions.md - Absence and where: reference/language/absence.md - Errors and limits: reference/language/errors.md - - Every construct, as math: reference/notation.md - Typeset the math: reference/typeset.md - - Reading a loaded model: reference/reading.md + - Python API: reference/api.md + - Glossary: reference/glossary.md - Examples: - examples/index.md - Least-cost dispatch: examples/dispatch.md - Unit commitment: examples/commitment.md - One construct per model: examples/operators.md + - About: + - The limits: about/limits.md + - Changelog: CHANGELOG.md + # Everything a model writer does not need, grouped by reader: whoever + # writes a tool against `Spec` and `Program` (an engine such as specsolve, a + # renderer, a checker), whoever changes math-spec itself, and the proofs of + # concept: the notation page, which renders the typesetting test model, and + # the PyPSA pages. The PyPSA pages stay in `docs/examples/`, where + # `tools/gallery.py` writes them. `docs/static/hooks.py` appends one page per module under + # `src/math_spec/` to Contributing, as `Modules`. + - Development: + - Building on math-spec: + - Reading a loaded model: reference/reading.md + - The file and the program: about/file-and-program.md + - What counts as language: about/what-counts-as-language.md + - Contributing: + - contributing.md + - What counts as public API: about/what-counts-as-public-api.md + - Proofs of concept: + - Every construct, as math: reference/notation.md - PyPSA in one file: examples/pypsa.md - PyPSA, the quadratic class: examples/pypsa_quadratic.md - PyPSA, the relaxed commitment: examples/pypsa_linearized_uc.md - PyPSA, the lossy lines: examples/pypsa_losses.md - PyPSA, the two-stage class: examples/pypsa_stochastic.md - PyPSA, the multi-period class: examples/pypsa_multi_period.md - # Everything a reader does not need in order to write a model: the design - # arguments, how to contribute, what changed. - - About: - - The file and the program: about/file-and-program.md - - The limits: about/limits.md - - What counts as language: about/what-counts-as-language.md - - What counts as public API: about/what-counts-as-public-api.md - - Contributing: contributing.md - - Changelog: CHANGELOG.md theme: name: material diff --git a/schema/math-spec.schema.json b/schema/math-spec.schema.json index 78f55535..399403c9 100644 --- a/schema/math-spec.schema.json +++ b/schema/math-spec.schema.json @@ -4,7 +4,7 @@ "anyOf": [ { "additionalProperties": false, - "description": "What the model assumes of its data: a predicate every coordinate it is checked at has to satisfy.\n\nWritten in YAML as a bare where string, or as a mapping once it carries a\n``where:`` or a ``description:``, and serialised back to whichever form it\nwas written in::\n\n assumptions:\n efficiency_is_a_fraction: \"efficiency > 0 AND efficiency <= 1\"\n bounds_do_not_cross:\n holds: \"p_min <= p_max\"\n where: \"p_min\"\n description: a unit with no minimum is unconstrained below\n\nThe language decides nothing about the numbers, so the consumer binding\nthe data checks it, and refuses the data where it does not hold.", + "description": "What the model assumes of its data: a predicate every coordinate it is checked at has to satisfy.\n\nWritten in YAML as a bare where string, or as a mapping once it carries a\n``where:`` or a ``description:``, and serialised back to whichever form it\nwas written in::\n\n assumptions:\n efficiency_is_a_fraction: \"efficiency > 0 AND efficiency <= 1\"\n bounds_do_not_cross:\n holds: \"p_min <= p_max\"\n where: \"p_min\"\n description: a unit with no minimum is unconstrained below\n\nThe language decides nothing about the numbers, so the consumer attaching\nthe data checks it, and refuses the data where it does not hold.", "properties": { "description": { "anyOf": [ @@ -133,7 +133,7 @@ }, "DimensionBlock": { "additionalProperties": false, - "description": "A declared dimension, and the dtype its coordinates must be.\n\nA dimension is an axis and nothing else: it declares that the axis exists\nand what its coordinates are typed as, never which coordinates there are \u2014\nthose are data, and arrive at bind time. The maps its members carry \u2014 a\ngenerator's bus, a snapshot's period \u2014 are top-level ``relations:``\n(:class:`RelationBlock`), keyed by their own name.", + "description": "A declared dimension, and the dtype its coordinates must be.\n\nA dimension is an axis and nothing else: it declares that the axis exists\nand what its coordinates are typed as, never which coordinates there are \u2014\nthose are data, and arrive when the data is attached. The maps its members carry \u2014 a\ngenerator's bus, a snapshot's period \u2014 are top-level ``relations:``\n(:class:`RelationBlock`), keyed by their own name.", "properties": { "description": { "anyOf": [ @@ -504,7 +504,7 @@ }, "RelationBlock": { "additionalProperties": false, - "description": "A named relation between dimensions: the columns a row is keyed by, and the columns that key determines.\n\nEach side is a dimension, a list of them, or a mapping of column name to\ndimension where two columns share one. ``key:`` is the claim the language\nchecks at bind: one row per key tuple, so every ``values:`` column is a\nfunction of it. A relation with no ``values:`` is **bare** \u2014 every column is\nin its key, a row is its own identity, and nothing reads it::\n\n relations:\n gen_bus: {key: generator, values: bus}\n gen_bt: {key: [generator], values: [bus, technology]}\n zone_of: {key: [generator, period], values: zone}\n ends: {key: line, values: {bus0: bus, bus1: bus}}\n connection: {key: [generator, bus]}\n\nAn operator reads the table in the direction the call names\n(``over=``, ``into=``), joining on the other key columns; the\ndeclaration fixes no direction. The map itself is data, and arrives at bind\ntime under the relation's name, one column per role.", + "description": "A named relation between dimensions: the columns a row is keyed by, and the columns that key determines.\n\nEach side is a dimension, a list of them, or a mapping of column name to\ndimension where two columns share one. ``key:`` is the claim the language\nchecks when the data is attached: one row per key tuple, so every ``values:`` column is a\nfunction of it. A relation with no ``values:`` is **bare** \u2014 every column is\nin its key, a row is its own identity, and nothing reads it::\n\n relations:\n gen_bus: {key: generator, values: bus}\n gen_bt: {key: [generator], values: [bus, technology]}\n zone_of: {key: [generator, period], values: zone}\n ends: {key: line, values: {bus0: bus, bus1: bus}}\n connection: {key: [generator, bus]}\n\nAn operator reads the table in the direction the call names\n(``over=``, ``into=``), joining on the other key columns; the\ndeclaration fixes no direction. The map itself is data, and arrives with the rest of it,\nunder the relation's name, one column per role.", "properties": { "description": { "anyOf": [ @@ -681,7 +681,7 @@ }, "$schema": "https://json-schema.org/draft/2020-12/schema", "additionalProperties": false, - "description": "The declared math \u2014 one YAML file, or one dict, validated. Nothing here has seen data.\n\nA ``Spec`` that exists has passed the whole language: constructing one by\nany route \u2014 ``to_spec``, :meth:`model_validate`, the constructor \u2014 runs\nevery load-time check, expression pass included, and raises\n:class:`~math_spec.errors.LanguageError` on a model the language refuses.\nHolding one is the proof, so nothing downstream checks it again.\n\nThe API is the eleven declaration sections plus ``version`` and\n``description``, three ways back out \u2014 :meth:`to_dict` for the model as\ndata, :meth:`to_yaml` for the file a reviewer reads, :meth:`expand` for the\nsame math with its formulations written out \u2014 and :attr:`program`, the\nmodel typed, which every reader after load walks. Everything else on this\nclass is pydantic's, not a contract this package keeps.", + "description": "The declared math \u2014 one YAML file, or one dict, validated. Nothing here has seen data.\n\nA ``Spec`` that exists has passed the whole language: constructing one by\nany route \u2014 ``to_spec``, :meth:`model_validate`, the constructor \u2014 runs\nevery load-time check, expression pass included, and raises\n:class:`~math_spec.errors.LanguageError` on a model the language refuses.\nHolding one is the proof, so nothing downstream checks it again.\n\nThe API is the eleven declaration sections plus ``version`` and\n``description``, three ways back out \u2014 :meth:`to_dict` for the model as\ndata, :meth:`to_yaml` for the file a reviewer reads, :meth:`expand` for the\nmodel with its formulations written out as plain rows \u2014 and :attr:`program`, the\nmodel typed, which every reader after load walks. Everything else on this\nclass is pydantic's, not a contract this package keeps.", "properties": { "assumptions": { "additionalProperties": { diff --git a/src/math_spec/_expression_resolver.py b/src/math_spec/_expression_resolver.py index 2d0f7423..bc05b669 100644 --- a/src/math_spec/_expression_resolver.py +++ b/src/math_spec/_expression_resolver.py @@ -384,7 +384,7 @@ def _amount(self, value: ArithmeticNode, operator: str, key: str) -> int | str | if (dtype := self.ns.dtypes[bare.name]) != 'int': self.errors.append( f"{self.context}: {operator}({key}={bare.name}) counts positions, but '{bare.name}' is declared " - f'dtype: {dtype}. A count of positions is integral — declare it dtype: int, which binds only an ' + f'dtype: {dtype}. A count of positions is integral — declare it dtype: int, which accepts only an ' f'integer column, so a fractional {words.noun} has nowhere to arrive from.' ) return None @@ -676,7 +676,7 @@ def not_a_number(name: str, dtype: str, context: str) -> str: ) return ( f"{context}: '{name}' is declared dtype: {dtype}, and an expression is arithmetic — " - f'only dtype: float and dtype: int bind a column it can be done to. {instead}' + f'only dtype: float and dtype: int accept a column it can be done to. {instead}' ) diff --git a/src/math_spec/degree.py b/src/math_spec/degree.py index ef5b1fe0..7ebe0d07 100644 --- a/src/math_spec/degree.py +++ b/src/math_spec/degree.py @@ -75,7 +75,7 @@ def check_binary(node: Multiply | Divide | Power, context: str, *, ceiling: int) raise LanguageError( f'{where}a base and an exponent must each be a single Constant/Parameter factor, ' f'not a sum — addition does not distribute over `**`, so `(1 + rate) ** period` is ' - f'refused where `growth ** period` is not. Bind the factor itself.' + f'refused where `growth ** period` is not. Declare the factor itself as a parameter.' ) return if isinstance(node, Divide): diff --git a/src/math_spec/dimensions.py b/src/math_spec/dimensions.py index 9ee479e4..b63b1954 100644 --- a/src/math_spec/dimensions.py +++ b/src/math_spec/dimensions.py @@ -4,7 +4,7 @@ """Static dim-set checking — a type system whose type is a set of dim names. -Every node's dim set is computable before any data is bound, so this pass runs +Every node's dim set is computable before any data is attached, so this pass runs at load on the resolved tree. The per-node rules are the "Dim algebra" table in ``docs/reference/language/expressions.md``; a constraint's two sides together must equal its ``dims``, and a where or a bound may not exceed the frame. diff --git a/src/math_spec/model.py b/src/math_spec/model.py index 119eb8db..dccd6feb 100644 --- a/src/math_spec/model.py +++ b/src/math_spec/model.py @@ -139,7 +139,7 @@ class RelationBlock(_StrictBlock): Each side is a dimension, a list of them, or a mapping of column name to dimension where two columns share one. ``key:`` is the claim the language - checks at bind: one row per key tuple, so every ``values:`` column is a + checks when the data is attached: one row per key tuple, so every ``values:`` column is a function of it. A relation with no ``values:`` is **bare** — every column is in its key, a row is its own identity, and nothing reads it:: @@ -152,8 +152,8 @@ class RelationBlock(_StrictBlock): An operator reads the table in the direction the call names (``over=``, ``into=``), joining on the other key columns; the - declaration fixes no direction. The map itself is data, and arrives at bind - time under the relation's name, one column per role. + declaration fixes no direction. The map itself is data, and arrives with the rest of it, + under the relation's name, one column per role. """ _label: ClassVar[str] = 'a relation declaration' @@ -195,7 +195,7 @@ class DimensionBlock(_StrictBlock): A dimension is an axis and nothing else: it declares that the axis exists and what its coordinates are typed as, never which coordinates there are — - those are data, and arrive at bind time. The maps its members carry — a + those are data, and arrive when the data is attached. The maps its members carry — a generator's bus, a snapshot's period — are top-level ``relations:`` (:class:`RelationBlock`), keyed by their own name. """ @@ -465,7 +465,7 @@ class AssumptionBlock(_StrictBlock): where: "p_min" description: a unit with no minimum is unconstrained below - The language decides nothing about the numbers, so the consumer binding + The language decides nothing about the numbers, so the consumer attaching the data checks it, and refuses the data where it does not hold. """ @@ -712,7 +712,7 @@ class Spec(_StrictBlock): The API is the eleven declaration sections plus ``version`` and ``description``, three ways back out — :meth:`to_dict` for the model as data, :meth:`to_yaml` for the file a reviewer reads, :meth:`expand` for the - same math with its formulations written out — and :attr:`program`, the + model with its formulations written out as plain rows — and :attr:`program`, the model typed, which every reader after load walks. Everything else on this class is pydantic's, not a contract this package keeps. """ @@ -825,11 +825,13 @@ def expand(self, *kinds: Formulation) -> Spec: """This model with its formulations written out as plain variables and constraints. A formulation states rows rather than being one — ``piecewise:`` states - a curve, ``sos:`` states which members of a family may be nonzero — and - expanding one writes those rows under names prefixed with the block's - own, then drops the block. The math is the same afterwards, and so is - the data that binds it: neither a set nor a curve emits a parameter, - and a curve's rows sit on ``where`` predicates over the file's own. + a curve, ``sos:`` states which members of a family may be nonzero. + Expanding one writes those rows under names prefixed with the block's + own, and drops the block. The result is a different model: it declares + more variables and constraints, so it does not compare equal to this + one. It declares the same dimensions and parameters, so the same data + attaches to both. Nothing is cached, so a second call builds the + expansion again. Args: kinds: Which formulations to write out — ``'piecewise'``, @@ -839,9 +841,10 @@ def expand(self, *kinds: Formulation) -> Spec: curve. Returns: - The model those blocks wrote out, or this one where it declares - none of them. It is a model like any other: :meth:`to_yaml` writes - it, and the file binds the same data as the one it came from. + The model with those blocks written out, or this same object where + it declares none of them, so an expansion asked for the same kinds + again returns itself. It is a model like any other: :meth:`to_yaml` + writes it, and :attr:`program` holds its rows. Raises: ValueError: *kinds* names something that is not a formulation. diff --git a/src/math_spec/operators.py b/src/math_spec/operators.py index cfa31e9e..d7e95e2a 100644 --- a/src/math_spec/operators.py +++ b/src/math_spec/operators.py @@ -4,8 +4,8 @@ """The closed set of built-in operators and their call shapes. -One home for each signature: a composition is a macro, and math the language -cannot say is a declared ``escape:``. +One home for each signature: a composition is a macro, and the set is closed +to a file. """ from __future__ import annotations @@ -230,6 +230,6 @@ def unknown_operator_message(name: str) -> str: return ( f"Unknown operator '{name}'.\n" f'Available: {sorted(BUILTIN_NAMES)}\n' - f"Define '{name}' as a macro under 'macros:' if it composes built-ins; " - f'if the math is not sayable in the language, use a declared escape.' + f"Define '{name}' as a macro under 'macros:' if it composes built-ins. " + f'A file cannot add an operator: see docs/about/limits.md.' ) diff --git a/src/math_spec/piecewise.py b/src/math_spec/piecewise.py index a3a0a6be..37149b08 100644 --- a/src/math_spec/piecewise.py +++ b/src/math_spec/piecewise.py @@ -118,9 +118,9 @@ def assumptions_of(block: str, pw: PiecewiseDeclaration) -> dict[str, Assumption f'{_quoted(link.values for link in pw.links)} — a missing row is read as a zero rather than as a ' f'shorter curve, so it sits the curve on the origin. ' + ( - f"Bind the rows, or narrow points: '{mask}' to where the curve runs." + f"Attach the rows, or narrow points: '{mask}' to where the curve runs." if mask is not None - else 'Bind the rows, or declare points: to say how far the curve runs.' + else 'Attach the rows, or declare points: to say how far the curve runs.' ), ) curvature = _curvature_required(pw) diff --git a/src/math_spec/program.py b/src/math_spec/program.py index 2122947b..4554900f 100644 --- a/src/math_spec/program.py +++ b/src/math_spec/program.py @@ -431,11 +431,11 @@ def children(expression: Expression) -> tuple[Expression, ...]: class RelationDeclaration: """One declared relation: a table over its ``columns``, single-valued per ``key``. - ``columns`` binds each role to its dimension in the order the table + ``columns`` maps each role to its dimension in the order the table carries them, the key's roles first; ``key`` is the roles a row is identified by, and :attr:`values` the rest — every role is a key role for a bare relation, which is one with no value columns. Every value is - checked at bind to be a label of its column's dimension, and the table to + checked when the data is attached to be a label of its column's dimension, and the table to have one row per key tuple — which keeps a mistyped label from silently dropping its terms in the join that places them, and is what lets ``at`` read one value. @@ -474,7 +474,7 @@ class Direction: The declaration fixes no direction; the call does, and this is the one it named. ``name`` is the relation's, as :attr:`Program.relations` keys it. ``consumed``, ``produced`` and ``joined`` are *roles* — column names of - ``relation``, which binds every role to its dimension and names the key. + ``relation``, which maps every role to its dimension and names the key. ``joined`` is the key roles the call did not name (every role, for a bare relation): the join keys on them, and a value role left unnamed is not read. @@ -487,7 +487,7 @@ class Direction: joined: tuple[str, ...] def dim(self, role: str) -> str: - """The dimension *role* is bound to.""" + """The dimension *role* ranges over.""" return self.relation.dim(role) @property @@ -509,7 +509,7 @@ class Partition: ``name`` is the relation's, as :attr:`Program.relations` keys it. ``along``, ``group`` and ``joined`` are *roles* — column names of - ``relation``, which binds every role to its dimension and names the key. + ``relation``, which maps every role to its dimension and names the key. ``along`` is the one key column over the dimension stepped along, and the frame keeps it. ``group`` is the value columns ``within=`` named, read at the row's key. ``joined`` is the other key columns, whose @@ -524,7 +524,7 @@ class Partition: joined: tuple[str, ...] def dim(self, role: str) -> str: - """The dimension *role* is bound to.""" + """The dimension *role* ranges over.""" return self.relation.dim(role) @property @@ -555,7 +555,7 @@ class Assumption: ``predicate`` is true at every coordinate of its frame — the product of every dim the two masks name — that ``where`` admits, a missing row reading as false as it does in any mask. Nothing here is decidable at - load: both sides are the data's, which is why the consumer binding it + load: both sides are the data's, which is why the consumer attaching it checks. """ @@ -568,7 +568,7 @@ class Assumption: def assumption_message(name: str, assumption: Assumption) -> str: - """The sentence a consumer raises when the data bound to *assumption*, called *name*, fails it. + """The sentence a consumer raises when the data attached to *assumption*, called *name*, fails it. The language's own wording, so every consumer refuses in the same words; a consumer appends the coordinates it saw. Where the file wrote a @@ -577,16 +577,16 @@ def assumption_message(name: str, assumption: Assumption) -> str: why the rule is there. """ read = ', '.join(f"'{n}'" for n in sorted(assumption.predicate.names_read)) - sentence = f"assumption '{name}' does not hold for the data bound to {read}" + sentence = f"assumption '{name}' does not hold for the data attached to {read}" return f'{sentence} — {assumption.description}' if assumption.description else sentence @dataclass(frozen=True) class ParameterDeclaration: - """Shape declaration; data is bound at execution time by name. + """Shape declaration; data is attached at execution time by name. ``dtype`` is what the declaration claims the values are, and a consumer - binding data refuses a column that is not it — so the *declaration* is + attaching data refuses a column that is not it — so the *declaration* is what is read, rather than whatever the column happens to hold. """ @@ -888,7 +888,7 @@ class Program: #: What the data has to satisfy for the answer to mean anything, by the #: name a refusal quotes: every ``assumptions:`` entry the file wrote, then #: what each ``piecewise:`` block's method assumes of its breakpoints. The - #: language decides none of it, so the consumer binding the data checks + #: language decides none of it, so the consumer attaching the data checks #: each and refuses with :func:`assumption_message`. assumptions: Mapping[str, Assumption] = Sealed({}) #: Declared ``expressions:``, each saying whether the math reads it. None diff --git a/src/math_spec/resolution.py b/src/math_spec/resolution.py index d2bbe02d..898b26af 100644 --- a/src/math_spec/resolution.py +++ b/src/math_spec/resolution.py @@ -394,7 +394,7 @@ def resolve_constraint_text( f'Got: {text!r}\n' f'A constraint is a claim about a decision, and a comparison of numbers and parameters ' f'is settled before the solve — no consumer builds a row for it. Name the variable it should ' - f'bound, or state the fact under `assumptions:`, where the consumer binding the data checks it.' + f'bound, or state the fact under `assumptions:`, where the consumer attaching the data checks it.' ) return None return left, ast.op, right diff --git a/src/math_spec/typesetting/README.md b/src/math_spec/typesetting/README.md index ddf19103..f4c52ba3 100644 --- a/src/math_spec/typesetting/README.md +++ b/src/math_spec/typesetting/README.md @@ -5,7 +5,7 @@ SPDX-License-Identifier: MIT # `typesetting/` — the model, printed -This package is a consumer of the program. It builds no model and binds no +This package is a consumer of the program. It builds no model and attaches no data. It walks a program, the one a loaded `Spec` holds or one handed to it, and prints it. diff --git a/tests/expand/curve-activity/after.yaml b/tests/expand/curve-activity/after.yaml index 39de860c..4ec42085 100644 --- a/tests/expand/curve-activity/after.yaml +++ b/tests/expand/curve-activity/after.yaml @@ -41,5 +41,5 @@ assumptions: description: >- piecewise 'curve': every breakpoint the curve runs through needs a row in 'x_bp', 'y_bp' — a missing row is read as a zero rather than as a shorter - curve, so it sits the curve on the origin. Bind the rows, or declare + curve, so it sits the curve on the origin. Attach the rows, or declare points: to say how far the curve runs. diff --git a/tests/expand/curve-adjacency-points/after.yaml b/tests/expand/curve-adjacency-points/after.yaml index d3892213..0dd6a7d6 100644 --- a/tests/expand/curve-adjacency-points/after.yaml +++ b/tests/expand/curve-adjacency-points/after.yaml @@ -43,7 +43,7 @@ assumptions: description: >- piecewise 'curve': every breakpoint the curve runs through needs a row in 'x_bp', 'y_bp' — a missing row is read as a zero rather than as a shorter - curve, so it sits the curve on the origin. Bind the rows, or narrow + curve, so it sits the curve on the origin. Attach the rows, or narrow points: 'x_bp' to where the curve runs. curve_contiguous: holds: count(x_bp AND NOT shift(x_bp, along=bp, offset=1), over=bp) == 1 diff --git a/tests/expand/curve-adjacency/after.yaml b/tests/expand/curve-adjacency/after.yaml index fcad551f..317493b6 100644 --- a/tests/expand/curve-adjacency/after.yaml +++ b/tests/expand/curve-adjacency/after.yaml @@ -40,5 +40,5 @@ assumptions: description: >- piecewise 'curve': every breakpoint the curve runs through needs a row in 'x_bp', 'y_bp' — a missing row is read as a zero rather than as a shorter - curve, so it sits the curve on the origin. Bind the rows, or declare + curve, so it sits the curve on the origin. Attach the rows, or declare points: to say how far the curve runs. diff --git a/tests/expand/curve-convex/after.yaml b/tests/expand/curve-convex/after.yaml index 7d1cc4e1..4d69388a 100644 --- a/tests/expand/curve-convex/after.yaml +++ b/tests/expand/curve-convex/after.yaml @@ -30,7 +30,7 @@ assumptions: description: >- piecewise 'curve': every breakpoint the curve runs through needs a row in 'x_bp', 'y_bp' — a missing row is read as a zero rather than as a shorter - curve, so it sits the curve on the origin. Bind the rows, or declare + curve, so it sits the curve on the origin. Attach the rows, or declare points: to say how far the curve runs. curve_increasing: where: position(bp) > 0 diff --git a/tests/expand/curve-lp-points/after.yaml b/tests/expand/curve-lp-points/after.yaml index db719efc..9a24e7b0 100644 --- a/tests/expand/curve-lp-points/after.yaml +++ b/tests/expand/curve-lp-points/after.yaml @@ -33,7 +33,7 @@ assumptions: description: >- piecewise 'curve': every breakpoint the curve runs through needs a row in 'x_bp', 'y_bp' — a missing row is read as a zero rather than as a shorter - curve, so it sits the curve on the origin. Bind the rows, or narrow + curve, so it sits the curve on the origin. Attach the rows, or narrow points: 'x_bp' to where the curve runs. curve_increasing: where: x_bp AND shift(x_bp, along=bp, offset=1) diff --git a/tests/expand/curve-lp/after.yaml b/tests/expand/curve-lp/after.yaml index 3c9564c3..0c7f627b 100644 --- a/tests/expand/curve-lp/after.yaml +++ b/tests/expand/curve-lp/after.yaml @@ -32,7 +32,7 @@ assumptions: description: >- piecewise 'curve': every breakpoint the curve runs through needs a row in 'x_bp', 'y_bp' — a missing row is read as a zero rather than as a shorter - curve, so it sits the curve on the origin. Bind the rows, or declare + curve, so it sits the curve on the origin. Attach the rows, or declare points: to say how far the curve runs. curve_increasing: where: position(bp) > 0 diff --git a/tests/expand/curve-sos2-piecewise/after.yaml b/tests/expand/curve-sos2-piecewise/after.yaml index 0c66ddea..2efdb329 100644 --- a/tests/expand/curve-sos2-piecewise/after.yaml +++ b/tests/expand/curve-sos2-piecewise/after.yaml @@ -36,5 +36,5 @@ assumptions: description: >- piecewise 'curve': every breakpoint the curve runs through needs a row in 'x_bp', 'y_bp' — a missing row is read as a zero rather than as a shorter - curve, so it sits the curve on the origin. Bind the rows, or declare + curve, so it sits the curve on the origin. Attach the rows, or declare points: to say how far the curve runs. diff --git a/tests/expand/curve-sos2/after.yaml b/tests/expand/curve-sos2/after.yaml index fcad551f..317493b6 100644 --- a/tests/expand/curve-sos2/after.yaml +++ b/tests/expand/curve-sos2/after.yaml @@ -40,5 +40,5 @@ assumptions: description: >- piecewise 'curve': every breakpoint the curve runs through needs a row in 'x_bp', 'y_bp' — a missing row is read as a zero rather than as a shorter - curve, so it sits the curve on the origin. Bind the rows, or declare + curve, so it sits the curve on the origin. Attach the rows, or declare points: to say how far the curve runs. diff --git a/tests/test_dimensions.py b/tests/test_dimensions.py index b286b362..70432814 100644 --- a/tests/test_dimensions.py +++ b/tests/test_dimensions.py @@ -2,7 +2,7 @@ # # SPDX-License-Identifier: MIT -"""Dim sets are a type system, checked before any data is bound.""" +"""Dim sets are a type system, checked before any data is attached.""" from __future__ import annotations diff --git a/tests/test_expand.py b/tests/test_expand.py index e5d6fce9..a04d90d6 100644 --- a/tests/test_expand.py +++ b/tests/test_expand.py @@ -2,7 +2,7 @@ # # SPDX-License-Identifier: MIT -"""`Spec.expand`: what it takes, what comes back, and what still binds it. +"""`Spec.expand`: what it takes, what comes back, and what data still attaches to it. The kinds are a closed pair and the result is a plain `Spec`, so the claims here are about the verb rather than about either formulation — those are in @@ -122,7 +122,7 @@ def test_an_expansion_declares_exactly_the_parameters_the_file_declared(): schema = schema_of(MASKED) expanded = schema.expand() - assert expanded.parameters == schema.parameters, 'a curve emits no parameter, so the same data binds both' + assert expanded.parameters == schema.parameters, 'a curve emits no parameter, so the same data attaches to both' assert schema_of(expanded.to_yaml()).to_dict() == expanded.to_dict(), ( 'the expansion is a file like any other, and loading it back changes nothing' ) @@ -161,3 +161,18 @@ def test_the_same_sources_bind_a_model_and_its_expansion(model): written_out = set(spec.expand().program.parameters) assert written_out == supplied, 'writing a formulation out asks for data the model it came from did not' + + +def test_an_expansion_is_a_different_model_and_has_nothing_left_to_write_out(): + """What `expand()` returns: a new model, which a second expansion hands back unchanged.""" + spec = schema_of(CURVE) + expanded = spec.expand() + + assert expanded != spec, 'the expansion declares more rows, so it is a different model' + assert expanded.expand() is expanded, 'an expansion has no formulation left, so it comes back as itself' + + +def test_a_model_with_no_formulation_expands_to_itself(): + spec = schema_of(DISPATCH_MODEL) + + assert spec.expand() is spec, 'nothing to write out returns the same object, not a copy' diff --git a/tests/test_expansion.py b/tests/test_expansion.py index e18b91f4..78bec488 100644 --- a/tests/test_expansion.py +++ b/tests/test_expansion.py @@ -320,13 +320,20 @@ def test_a_template_is_held_to_the_rules_a_call_site_is(template, match): schema_of(SMALL_MODEL, macros={'m': {'args': ['x'], 'template': template}}) -@pytest.mark.parametrize('fragment', ['my_python_helper', 'macros:', 'escape']) +@pytest.mark.parametrize('fragment', ['my_python_helper', 'macros:', 'docs/about/limits.md']) def test_an_unknown_operator_is_refused_at_load_with_the_rewrite(fragment): with pytest.raises(LanguageError) as exc: schema(constraints={'c': {'dims': ['snapshot'], 'expression': 'my_python_helper(p) <= load'}}) assert fragment in str(exc.value) +def test_an_unknown_operator_names_no_construct_the_language_lacks(): + """The refusal told the author to "use a declared escape", and the schema has no `escape:` key.""" + with pytest.raises(LanguageError) as exc: + schema(constraints={'c': {'dims': ['snapshot'], 'expression': 'my_python_helper(p) <= load'}}) + assert 'escape' not in str(exc.value), 'the message points at a key the closed schema refuses' + + @pytest.mark.parametrize( ('formals', 'template'), [ diff --git a/tests/test_lowering.py b/tests/test_lowering.py index 6ddd5834..9dc0667a 100644 --- a/tests/test_lowering.py +++ b/tests/test_lowering.py @@ -417,7 +417,7 @@ def test_a_comparison_of_expressions_lowers_to_program_expressions_on_both_sides assert mask is not None and isinstance(mask.root, ExpressionComparison) assert isinstance(mask.root.right, Add) and isinstance(mask.root.right.left, Pullback) assert mask.names_read == frozenset({'c', 'zc', 'lk2'}), ( - 'the relation a pullback and a partition read through is data the consumer binds too' + 'the relation a pullback and a partition read through is data the consumer attaches too' ) @@ -434,7 +434,7 @@ def test_a_predicate_a_leaf_carries_is_lowered_like_any_other_mask(): assert mask.root.predicate.root == ExpressionComparison( Parameter('c'), '<=', Multiply(Constant(0.5), Parameter('k')), ('g',) ), 'the counted predicate is rebuilt, not handed through with the resolved comparison still in it' - assert mask.names_read == frozenset({'c', 'k'}), 'what the counted predicate reads is data the consumer binds' + assert mask.names_read == frozenset({'c', 'k'}), 'what the counted predicate reads is data the consumer attaches' def test_a_translated_predicate_keeps_what_it_reads_in_reach(): @@ -458,7 +458,7 @@ def test_a_translated_predicate_keeps_what_it_reads_in_reach(): def test_a_predicate_read_through_a_relation_is_lowered_and_keeps_the_relation_in_reach(): - """The comparison under the read is rebuilt, and the relation is data the consumer binds as well as the operand.""" + """The comparison under the read is rebuilt, and the relation is data the consumer attaches as well as the operand.""" program = to_spec( override( SHAPES_MODEL, @@ -482,7 +482,7 @@ def test_a_predicate_read_through_a_relation_is_lowered_and_keeps_the_relation_i def test_assumptions_carry_the_file_s_entries_and_the_curves_behind_them(): - """One mapping holds every fact about the data, so a consumer binding it has one loop and one refusal. + """One mapping holds every fact about the data, so a consumer attaching it has one loop and one refusal. The file's entries come first, in the order it wrote them; each ``piecewise:`` block's conditions follow under the name a refusal quotes. @@ -511,7 +511,7 @@ def test_an_assumption_lowers_both_of_its_masks(): Mask(ParameterDefined('flag', ('g',))), ), 'the arithmetic side is a program expression, and the where is the mask the file wrote' assert assumption_message('sound', assumption) == ( - "assumption 'sound' does not hold for the data bound to 'c', 'k'" + "assumption 'sound' does not hold for the data attached to 'c', 'k'" ), 'the refusal names what the consumer bound, so it can say which column is wrong' @@ -528,7 +528,7 @@ def test_an_assumption_refuses_in_the_words_the_file_wrote(): assert assumption.description == reason, 'the program carries it, so a consumer needs no second read of the file' assert assumption_message('sound', assumption) == ( - f"assumption 'sound' does not hold for the data bound to 'c', 'k' \N{EM DASH} {reason}" + f"assumption 'sound' does not hold for the data attached to 'c', 'k' \N{EM DASH} {reason}" ), 'the sentence trails what the author wrote' @@ -551,7 +551,7 @@ def test_a_cased_side_reads_the_data_its_regions_are_decided_by(): where = program.variables['p'].where assert where is not None assert where.names_read == frozenset({'c', 'k', 'flag', 'lk2'}), ( - 'the flag and the relation decide which region applies, so the consumer binds them too' + 'the flag and the relation decide which region applies, so the consumer attaches them too' ) diff --git a/tests/test_piecewise.py b/tests/test_piecewise.py index a470e5a9..a5d84bb1 100644 --- a/tests/test_piecewise.py +++ b/tests/test_piecewise.py @@ -4,7 +4,7 @@ """`piecewise:` expansion, judged at the door that decides it. -Every claim here is one `to_spec` or `Spec.expand` reaches with no data bound: +Every claim here is one `to_spec` or `Spec.expand` reaches with no data attached: which declarations a curve emits, which names it may not collide with, which methods exist, and which gates a block will accept. """ @@ -360,7 +360,7 @@ def test_any_affine_expression_is_a_legal_link(link): ], ) def test_a_malformed_block_is_refused(model, patch, match): - """Schema-level arity rules and the expansion's own preconditions, before any data is bound. + """Schema-level arity rules and the expansion's own preconditions, before any data is attached. Refused rather than fallen back from: a method written down is a formulation chosen. """ @@ -652,7 +652,7 @@ def test_every_check_has_a_sentence(suffix): name = f'cost_curve_{suffix}' assert name in assumptions, 'the fixture is the block that assumes everything' message = assumption_message(name, assumptions[name]) - assert message.startswith(f"assumption '{name}' does not hold for the data bound to "), ( + assert message.startswith(f"assumption '{name}' does not hold for the data attached to "), ( 'the refusal names the columns a consumer has to look at before it says why' ) assert "— piecewise 'cost_curve':" in message, 'and trails the sentence the method implies' diff --git a/tests/test_reading_page.py b/tests/test_reading_page.py index 909569e0..7f932ecf 100644 --- a/tests/test_reading_page.py +++ b/tests/test_reading_page.py @@ -55,6 +55,6 @@ def test_the_page_shows_the_declarations_the_expansion_emits(tmp_path, monkeypat exec(compile(code, str(PAGE), 'exec'), namespace) claims.extend(_claims(code)) - assert len(claims) == 22, 'every `expression # value` line on the page is checked; one without one is not' + assert len(claims) == 19, 'every `expression # value` line on the page is checked; one without one is not' for expression, claimed in claims: assert eval(expression, namespace) == claimed, f'reading.md says `{expression}` is {claimed}' diff --git a/tests/test_separability.py b/tests/test_separability.py index e4dd817c..8555eb50 100644 --- a/tests/test_separability.py +++ b/tests/test_separability.py @@ -2,7 +2,7 @@ # # SPDX-License-Identifier: MIT -"""Whether a horizon may be built in windows, asked before any data binds. +"""Whether a horizon may be built in windows, asked before any data is attached. The verdict is what a rolling-horizon or myopic driver needs and cannot currently get: a model with an annual budget windows into feasible pieces whose @@ -98,7 +98,7 @@ def test_a_reach_only_data_can_say_names_what_says_it(patch, reach): than refusing the model, so a driver holding the data knows what to read and `resolved` knows how to fold it.""" verdict = _verdict(**patch) - assert not verdict.windowable, 'undecided until data binds' + assert not verdict.windowable, 'undecided until data is attached' assert verdict.undecided == (reach,), 'the report names what the driver has to read, once' assert not verdict.coupled, 'and nothing structural ties the axis' @@ -153,7 +153,7 @@ def test_a_read_through_a_relation_is_undecided_on_the_axis_it_reads(): """`at(cap, by=zone_of, over=zone, into=u)` reads `zone` at whatever coordinate the relation chooses, so how far that reaches along `zone` is the relation's data to say.""" verdict = _verdict('zone', **_rows('p - at(cap, by=zone_of, over=zone, into=u) <= 0')) - assert not verdict.windowable and not verdict.coupled, 'undecided until the relation binds' + assert not verdict.windowable and not verdict.coupled, 'undecided until the relation is attached' assert verdict.undecided == (Reach("constraint 'k'", 'zone_of', 'coordinate'),), ( 'the report names the relation a driver has to read' ) diff --git a/tests/test_sos.py b/tests/test_sos.py index cd1c059a..c2d742d5 100644 --- a/tests/test_sos.py +++ b/tests/test_sos.py @@ -4,7 +4,7 @@ """`sos:` as a formulation: what a set is written out as, and what it may not lose. -Every claim here is one `Spec.expand` reaches with no data bound — which +Every claim here is one `Spec.expand` reaches with no data attached — which declarations a set emits, which coefficient links them, and that the adjacency method is the same rows under the same names. """ diff --git a/tests/test_validation.py b/tests/test_validation.py index e4d61ce3..cdb35ab2 100644 --- a/tests/test_validation.py +++ b/tests/test_validation.py @@ -2,7 +2,7 @@ # # SPDX-License-Identifier: MIT -"""What `to_spec` refuses with no data bound, and how it says so.""" +"""What `to_spec` refuses with no data attached, and how it says so.""" from __future__ import annotations @@ -602,7 +602,7 @@ class TestAWhereSideIsReadInResolution: """The grammar hands a comparison's sides over as arithmetic, and the language decides here what a side may be. A ``position()`` call is held to its shape, a literal is the expression grammar's, and - everything else on a side is a comparison of expressions, decided with no data bound. + everything else on a side is a comparison of expressions, decided with no data attached. """ @pytest.mark.parametrize( @@ -997,22 +997,22 @@ def test_a_read_given_an_edge_is_not_told_about_translations(self): assert 'translation' not in str(caught.value) def test_a_read_lands_on_the_dims_it_produces_and_reads_the_relation(self): - """The mask is over what the relation maps onto, and a consumer binds the relation as well as the operand.""" + """The mask is over what the relation maps onto, and a consumer attaches the relation as well as the operand.""" mask = where_of("at(h == 'north', by=lk, over=h, into=g)", Namespace(_schema()), 'probe') assert mask is not None assert sorted(mask.dims) == ['g'], "'h' is read at lk(g), so g is all the mask is over" - assert mask.names_read == frozenset({'lk'}), 'the relation is data a consumer binds, the label is not' + assert mask.names_read == frozenset({'lk'}), 'the relation is data a consumer attaches, the label is not' def test_a_count_reduces_the_dim_it_counts_along_away(self): """The count is one number per remaining coordinate, so a claim about each group needs no word for the group.""" mask = where_of('count(q, over=h) >= 2', Namespace(_schema()), 'probe') assert mask is not None assert sorted(mask.dims) == ['g'], "'q' is read over g and h, and h is counted away" - assert mask.names_read == frozenset({'q'}), 'a consumer binds what the counted predicate reads' + assert mask.names_read == frozenset({'q'}), 'a consumer attaches what the counted predicate reads' class TestRulesDecidedWithoutData: - """Every refusal the schema or the resolver makes with no data bound, one row each.""" + """Every refusal the schema or the resolver makes with no data attached, one row each.""" @pytest.mark.parametrize( ('patch', 'fragments'), @@ -1597,7 +1597,7 @@ class TestAssumptions: Everything here is about the data, so nothing in it is decided at load but the shape of the predicate: the entry is refused where the connectives already settle it, and where it names a variable, which is what the solver - decides rather than what the caller binds. + decides rather than what the caller attaches. """ @pytest.mark.parametrize( @@ -1859,7 +1859,7 @@ def test_the_schema_itself_states_the_shape_of_a_case(self, cases: dict[str, Any to_spec(_cased(cases)) def test_two_cases_may_not_claim_one_coordinate(self): - """Proved before any data binds, so the arms are read apart rather than in order.""" + """Proved before any data is attached, so the arms are read apart rather than in order.""" cases = { 'gas': {'when': "generator == 'gas'", 'expression': 'p_max'}, 'opening': {'when': 'position(snapshot) == 0', 'expression': 'p_max * 2'}, diff --git a/tests/typesetting/test_cli.py b/tests/typesetting/test_cli.py index 873a50cd..c62ea407 100644 --- a/tests/typesetting/test_cli.py +++ b/tests/typesetting/test_cli.py @@ -180,4 +180,4 @@ def test_no_verb_binds_data(): banned = {'--source', '--coords', '--data'} for name, verb in _verbs().items(): flags = {option for action in verb._actions for option in action.option_strings} - assert not (flags & banned), f'{name} binds data: {sorted(flags & banned)}' + assert not (flags & banned), f'{name} attaches data: {sorted(flags & banned)}' diff --git a/tools/gallery.py b/tools/gallery.py index 74fba0f5..d772374c 100644 --- a/tools/gallery.py +++ b/tools/gallery.py @@ -158,7 +158,7 @@ def spine_block() -> str: """The shared spine, shown once.""" return ( "> Every rung's network is `spine.build()` plus the rung's own `n.add` calls, data inline; a keyword not" - " passed is PyPSA's default. A banner states what PyPSA solved the rung to; how an engine binds the network to" + " passed is PyPSA's default. A banner states what PyPSA solved the rung to; how an engine attaches the network to" " the file, and what it makes of it, is that engine's own record.\n" '\n' '
\n' diff --git a/tools/notation.py b/tools/notation.py index ef5ac0a8..c0e6ceb9 100644 --- a/tools/notation.py +++ b/tools/notation.py @@ -10,7 +10,8 @@ The source is ``tests/typesetting/golden/model.yaml``, the one model that carries every construct — ``tests/typesetting/test_golden.py`` holds it to the language, and this tool emits a row for every declaration in it. The fixture's -own case-label comments become the captions. +own case-label comments become the captions; :data:`FAMILIES` gives each row +its heading and its place on the page. """ from __future__ import annotations @@ -39,33 +40,135 @@ #: saying what a method is for reads against a model that had a reason to #: choose it. PIECEWISE = { - 'adjacency': ROOT / 'examples' / 'ports' / 'transport_pwl.yaml', - 'sos2': ROOT / 'examples' / 'sos.yaml', - 'convex': ROOT / 'examples' / 'piecewise.yaml', - 'lp': ROOT / 'examples' / 'piecewise_lp.yaml', + 'adjacency': ('Adjacency method', ROOT / 'examples' / 'ports' / 'transport_pwl.yaml'), + 'sos2': ('SOS2 method', ROOT / 'examples' / 'sos.yaml'), + 'convex': ('Convex method', ROOT / 'examples' / 'piecewise.yaml'), + 'lp': ('LP method', ROOT / 'examples' / 'piecewise_lp.yaml'), } BEGIN, END = '', '' -#: The blocks that declare math, in the order the page walks them, and the -#: heading each gets. ``dimensions``, ``relations`` and ``parameters`` are absent -#: on purpose: they declare no equation, and what they print is the legend, -#: which the page shows once as a legend rather than a row at a time. -SECTIONS = { - 'objective': 'The objective', - 'constraints': 'Constraints', - 'expressions': 'Definitions', - 'variables': 'Variable domains', - 'piecewise': 'Curves', - 'sos': 'Sets carried to the solver', - 'assumptions': 'What the data has to satisfy', +#: The blocks of the fixture that declare math. ``dimensions``, ``relations`` +#: and ``parameters`` are absent on purpose: they declare no equation, and what +#: they print is the legend, which the page shows once rather than a row at a +#: time. +BLOCKS = ('objective', 'constraints', 'expressions', 'variables', 'piecewise', 'sos', 'assumptions') + +#: The page's sections in the order of the language reference, and in each the +#: fixture's declarations under the construct they show. The declaration name +#: is the fixture's and no reader searches for it, so it stays in the YAML and +#: the heading names the construct. Within a section the order is the file's +#: wherever a caption reads against the row above it ("its adjoint", "the same +#: window"). Every declaration of the fixture is here exactly once, or the +#: tool refuses to write the page. +FAMILIES: dict[str, dict[str, str]] = { + 'Variable domains': { + 'p': 'Lower and upper bounds', + 'spill': 'Lower bound only', + 'slack': 'Upper bound only', + 'theta': 'Unbounded variable', + 'on': 'Binary domain', + 'units': 'Bounded integer domain', + 'spare': 'Unbounded integer domain', + 'reserve': 'Scalar variable', + 'headroom': 'Scalar variable with a condition', + 'weight': 'Variable in a special ordered set', + 'fuel': 'Second axis of a curve', + 'heat': 'Third axis of a curve', + 'op_cost': 'Variable bounded by a curve', + 'warm': 'Binary variable with a condition', + }, + 'Objective': { + 'objective': 'Products and powers in the objective', + }, + 'Relations': { + 'balance': 'Sum through a relation', + 'total': 'Sum over every dimension', + 'pullback': '`at` through a relation', + 'grouped_once': 'Sum into two value columns', + 'pulled_back_once': '`at` through two value columns', + 'within_bus': 'Shift within one value column', + 'relational': 'Sum through a bare relation', + 'connected': 'Bare relation as a condition', + 'representative': 'Map into its own dimension', + 'zonal': 'Sum through a two-key map', + 'zonal_history': 'Sum over the other key of a two-key map', + 'zonal_membership': 'Sum between the two keys of a map', + 'zonal_pullback': '`at` through a two-key map', + }, + 'Arithmetic and literals': { + 'arithmetic': 'Signs, division and number literals', + 'efficiency': 'Greek parameter name', + 'ceiling': 'Infinity literal', + }, + 'Named expressions': { + 'budgeted': 'Plain expression in a constraint', + 'starts': 'Cased expression in a constraint', + 'spend': 'Plain named expression', + 'startup_cost': 'Expression defined by cases', + 'spend_cap': 'Data-only expression', + 'capped': 'Named expression in a condition', + 'lcoe': 'Reported expression', + 'marginal_price': 'Dual of a constraint', + }, + 'Shifts': { + 'ramp': 'Cyclic and acyclic shift', + 'edges': 'Filled and forward shifts', + 'ahead': 'Cyclic forward shift', + 'composed': 'Composed shifts', + 'uncomposed': 'Nested shifts that do not compose', + 'crossed': 'Shift along two dimensions', + 'lead_time': 'Shift by a parameter offset', + 'in_season': 'Cyclic shift within a group', + 'held_in_season': 'Filled shift within a group', + }, + 'Trailing windows': { + 'window': 'Window of fixed width', + 'history': 'Window with a parameter width', + 'seasonal_window': 'Window within a group', + }, + 'Piecewise curves': {}, + 'Special ordered sets': { + 'adjacent': 'Special ordered set of type 2', + }, + 'Assumptions': { + 'bounds_do_not_cross': 'Two parameters compared', + 'efficiency_is_a_fraction': 'Connective in an assumption', + 'lead_times_are_short': 'Parameter compared to a literal', + 'zones_agree': 'Two relations compared', + 'budget_covers_the_peak': 'Reduction in an assumption', + 'ramps_are_gentle': 'Shift in an assumption', + 'flexible_units_have_headroom': 'Assumption with a boolean condition', + 'northern_demand_is_real': 'Assumption with a relation condition', + }, + 'Where conditions': { + 'scalar': 'Parameter as a condition', + 'running': 'Variable and label conditions', + 'first': 'Position in a dimension', + 'last': 'Position counted from the end', + 'northern': 'Relation compared to a label', + 'always': 'Constant true condition', + 'redundant': 'Constant true inside a condition', + 'never': 'Constant false condition', + 'margin': 'Comparison of two expressions', + 'ramped': 'Shift and pullback in a condition', + 'covered': 'Reduction in a scalar condition', + 'counted': 'Count over a dimension', + 'counted_here': 'Count along a dimension of the frame', + 'run_start': 'Predicate at the previous coordinate', + 'zoned': 'Predicate through a relation', + }, } +#: Declarations whose caption says nothing its heading does not, so the page +#: prints the heading alone. +NAMED_BY_HEADING = frozenset({'spill', 'slack', 'theta', 'balance'}) + class Declaration: - """One block of the fixture: its name, its YAML, and the caption beside it.""" + """One block of the fixture: its name, the block it sits in, its YAML, and the caption beside it.""" - def __init__(self, name: str, lines: list[str], caption: str) -> None: - self.name, self.lines, self.caption = name, lines, caption + def __init__(self, name: str, block: str, lines: list[str], caption: str) -> None: + self.name, self.block, self.lines, self.caption = name, block, lines, caption def field(self, key: str) -> str: """One scalar the block declares — ``''`` where it declares no such key.""" @@ -76,15 +179,16 @@ def field(self, key: str) -> str: @property def yaml(self) -> str: - """The block as written, dedented, with the caption comment removed. + """The declaration under the key of its block, with the caption comment removed. - Dedented because a fragment is read on its own: two spaces of leading - indent are what the block's position in the file costs it, and every - line of every row would carry them. + The key is kept because the heading names the construct rather than + the block, and a section mixes blocks: the fragment is where a reader + sees whether the row is a constraint, an expression or a variable. """ - kept = [line.removeprefix(' ') for line in self.lines if not _described(line, self.lines)] - body = '\n'.join(kept) - return re.sub(r'[ ]+#[^\n]*', '', body, count=1) if self.caption else body + kept = [line for line in self.lines if not _described(line, self.lines)] + kept[0] = re.sub(r'[ ]+#.*$', '', kept[0]) + key = [] if self.block == 'objective' else [f'{self.block}:'] + return '\n'.join([*key, *kept]) def _described(line: str, lines: list[str]) -> bool: @@ -111,14 +215,14 @@ def declarations(text: str) -> dict[str, list[Declaration]]: Scanned rather than parsed by a YAML reader: the comments are the captions, and a reader that keeps them is a dependency this repo does not have. """ - found: dict[str, list[Declaration]] = {section: [] for section in SECTIONS} + found: dict[str, list[Declaration]] = {section: [] for section in BLOCKS} section, current = None, None for line in text.splitlines(): if match := re.match(r'^(\w+):', line): - section = match[1] if match[1] in SECTIONS else None + section = match[1] if match[1] in BLOCKS else None current = None if section == 'objective': - current = Declaration('objective', [], _caption(line)) + current = Declaration('objective', section, [line], _caption(line)) found[section].append(current) continue if section is None: @@ -129,7 +233,7 @@ def declarations(text: str) -> dict[str, list[Declaration]]: current.lines.append(line) continue if match := re.match(r'^ (\w+):', line): - current = Declaration(match[1], [line], _caption(line)) + current = Declaration(match[1], section, [line], _caption(line)) found[section].append(current) elif current is not None and line.strip(): current.lines.append(line) @@ -187,10 +291,10 @@ def preamble(text: str) -> str: def block() -> str: - """The page's generated half: the legend, then every declaration in turn.""" + """The page's generated half: the legend, then every construct in its family.""" rendered = to_markdown(MODEL, numbered=False) parts = [ - '### The legend', + '### Legend', 'A dimension, a relation and a parameter declare no equation; what they ' 'print is the legend every model opens with.', f'```yaml\n{preamble(MODEL.read_text())}\n```', @@ -198,10 +302,21 @@ def block() -> str: ] printed = equations(rendered) written = equations(to_markdown(to_spec(MODEL).expand('sos'), numbered=False)) - for section, title in SECTIONS.items(): - parts.append(f'### {title}') - found = declarations(MODEL.read_text())[section] - if section == 'piecewise': + found = { + one.name: one + for section, ones in declarations(MODEL.read_text()).items() + if section != 'piecewise' + for one in ones + } + placed = [name for rows in FAMILIES.values() for name in rows] + twice = sorted({name for name in placed if placed.count(name) > 1}) + assert set(placed) == set(found) and not twice, ( + f'every declaration of the fixture has one heading in FAMILIES: missing ' + f'{sorted(set(found) - set(placed))}, not in the fixture {sorted(set(placed) - set(found))}, twice {twice}' + ) + for family, rows in FAMILIES.items(): + parts.append(f'### {family}') + if family == 'Piecewise curves': parts.append( 'A curve prints as the curve it states, over the frame the block builds one per coordinate of, ' 'and its expansion prints the rows that curve stands for. One row per `method:`, each from the ' @@ -209,15 +324,22 @@ def block() -> str: ) parts += _curves() continue - if section == 'sos': + if family == 'Special ordered sets': parts.append( 'A set prints beside the variable it restricts, because it restricts that variable rather than ' 'adding a row of its own. Under it are the rows it is written out as.' ) - parts += [f'{_row(one, printed)}\n\n{_written_out(one.name, written)}' for one in found] + parts += [ + f'{_row(found[name], heading, printed)}\n\n{_written_out(name, written)}' + for name, heading in rows.items() + ] continue - parts += [_row(one, printed) for one in found] - return '\n\n'.join(parts) + parts += [_row(found[name], heading, printed) for name, heading in rows.items()] + page = '\n\n'.join(parts) + headings = re.findall(r'^#{3,4} (.+)$', page, re.MULTILINE) + shared = sorted({heading for heading in headings if headings.count(heading) > 1}) + assert not shared, f'two sections share a heading, so one anchor is lost: {shared}' + return page def _curves() -> list[str]: @@ -228,7 +350,7 @@ def _curves() -> list[str]: ``sos2`` keeps the set and for ``adjacency`` is the binaries that set states. """ rows = [] - for method, source in PIECEWISE.items(): + for method, (heading, source) in PIECEWISE.items(): table = sidecar_for(source) spec = to_spec(source) stated = equations(to_markdown(spec, symbols=table, numbered=False)) @@ -240,12 +362,12 @@ def _curves() -> list[str]: ] assert found, f'{source.name} declares no piecewise block with method: {method}' for block in found: - row = _row(block, stated) - caption = ( - f'**`method: {method}`** \N{EM DASH} {PIECEWISE_METHODS[method]}, in `{source.relative_to(ROOT)}`.' - ) + row = _row(block, heading, stated) + caption = f'`method: {method}` \N{EM DASH} {PIECEWISE_METHODS[method]}, in `{source.relative_to(ROOT)}`.' derived = [math for label, math in stated.items() if label.startswith(f'{block.name} ')] - assumed = '\n\n'.join(['What the method assumes of the numbers bound to it:', *derived]) if derived else '' + assumed = ( + '\n\n'.join(['What the method assumes of the numbers attached to it:', *derived]) if derived else '' + ) rows.append( row.replace('\n\n', f'\n\n{caption}\n\n{_table_shown(table)}', 1) + f'\n\n{_written_out(block.name, written)}' @@ -286,12 +408,13 @@ def _table_shown(table: Path | None) -> str: ) -def _row(declaration: Declaration, printed: dict[str, str]) -> str: - """One construct: what it says, what it is for, and what it prints.""" - caption = f'{declaration.caption}\n\n' if declaration.caption else '' +def _row(declaration: Declaration, heading: str, printed: dict[str, str]) -> str: + """One construct: what it is called, what it is for, what it says, and what it prints.""" + shown = declaration.caption and declaration.name not in NAMED_BY_HEADING + caption = f'{declaration.caption}\n\n' if shown else '' assert declaration.name in printed, f'{declaration.name} declares math and the walk printed none of it' math = printed[declaration.name] - return f'#### `{declaration.name}`\n\n{caption}```yaml\n{declaration.yaml}\n```\n\n{math}' + return f'#### {heading}\n\n{caption}```yaml\n{declaration.yaml}\n```\n\n{math}' def rendered_page(page: str) -> str: