docs(limits): a whole-table operator is priced at a barrier rather than refused

AGENTS.md

@@ -81,7 +81,9 @@ answer here is the mistake.
   the ones nothing calls. Where a file does not determine the answer, loading
   fails, and the message names the rewrite.
 - **Triage a new construct first: is it a primitive, a macro, or refused?**
-  A primitive is admissible when it is relational and local. Read the
+  A primitive is admissible when it is relational. Locality prices a primitive
+  rather than barring it. An operator that reads the whole table costs one full
+  pass over the data. Read the
   deliberate non-primitives in
   [limits.md](docs/about/limits.md) first. The argument for admitting one of
   those is the argument that page is already making, not a new argument.

docs/about/limits.md

@@ -40,21 +40,29 @@ the value of a keyword argument, such as `over=snapshot`, never in the key. A
 macro can write `over=d` and let the caller supply `d`. It could not do that if
 the dimension were the keyword itself.
 
-**Each output row reads a bounded number of input rows.** `sum(p, over=g)` reads
-one row per generator. `shift(p, over=t, offset=1)` reads one row, the one
-before it. `x * y * a` reads the rows of `a` that pair an `x` with a `y`. An
-operator that reads the whole table to produce one row, or that calls itself, is
-refused, because an engine cannot then build the model one chunk of rows at a
-time. Reading only the coordinate labels does not count: "the last snapshot"
-looks at the list of snapshots, not at the data, and is allowed.
+**An operator may read the whole table. It pays one full pass over the data.**
+`sum(p, over=g)` reads one row per generator. `shift(p, over=t, offset=1)` reads
+one row, the one before it. `x * y * a` reads the rows of `a` that pair an `x`
+with a `y`. Each reads a bounded number of rows per output row, so an engine
+builds the model one chunk of rows at a time.
+
+One kind of operator reads every row to produce one row. The engine then reads
+the whole table before it builds any chunk, and the chunks stop being
+independent. That is the price, and a request for such an operator names it.
+Reading only the coordinate labels costs nothing. "The last snapshot" looks at
+the list of snapshots, not at the data.
+
+**An operator that calls itself is refused.** Nothing bounds how far it expands,
+so no number of passes over the data is enough.
 
 | The operator                                         | Allowed?                                        |
 | ---------------------------------------------------- | ----------------------------------------------- |
 | filters rows on a column they already carry          | yes                                             |
 | joins each row against a parameter or a lookup table | yes                                             |
 | reads a fixed number of neighbouring rows            | yes                                             |
 | reads only the coordinate labels                     | yes                                             |
-| reads every row, or calls itself                     | no, and the message names what to write instead |
+| reads every row                                      | yes, at one full pass before any chunk builds   |
+| calls itself                                         | no, and the message names what to write instead |
 
 **Degree is not a third test.** `p * q` at one coordinate is a join of a table
 with itself, so the objective and the constraints take it. Two things limit the
@@ -74,11 +82,11 @@ the same model written out by hand.
 
 ### Three kinds of refusal
 
-| The language refuses it because…                         | Examples                                                                                                                                                            | Can it change?                                                                                                |
-| -------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- |
-| **one solver cannot take it**                            | indicator constraints (#220); a quadratic constraint. `sos:` was in this group, and entered: a solver with sets takes it as one, and a solver without gets binaries | yes, solver by solver                                                                                         |
-| **no engine could build it one chunk of rows at a time** | an operator that reads a whole table; arbitrary Python                                                                                                              | not today. A capped block of Python for this is planned as [#38](https://github.com/fluxopt/lpspec/issues/38) |
-| **this project puts the work elsewhere**                 | data preparation such as resampling; helpers for one domain; Python that decides which declarations exist                                                           | it could; this project does not want it to                                                                    |
+| The language refuses it because…           | Examples                                                                                                                                                            | Can it change?                             |
+| ------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------ |
+| **one solver cannot take it**              | indicator constraints (#220); a quadratic constraint. `sos:` was in this group, and entered: a solver with sets takes it as one, and a solver without gets binaries | yes, solver by solver                      |
+| **the file would stop being the artifact** | arbitrary Python, whose content no loader can check and no typesetter can print                                                                                     | no                                         |
+| **this project puts the work elsewhere**   | data preparation such as resampling; helpers for one domain; Python that decides which declarations exist                                                           | it could; this project does not want it to |
 
 Three things never appear inside one model: an `if`, a loop, and a set of
 declarations that depends on the data. `dims: [snapshot]` does not know how

docs/reference/language/errors.md

@@ -91,7 +91,7 @@ and [the limits](../../about/limits.md) gives the reasons.
 | time-series processing (resample, cluster, interpolate, align), file IO, units | Data preparation. Pass a parameter                                                                                                                                  |
 | indicator constraints                                                          | What a solver can take is a question of its own, and `sos:` is where it landed ([#220](https://github.com/fluxopt/lpspec/issues/220))                               |
 | multi-objective                                                                | There is one `objective:` block. Weight the goals into one expression                                                                                               |
-| arbitrary array operations (`merge`, `reindex`, `apply_ufunc`)                 | Data preparation. The closed operator set is what lets a build stream its terms                                                                                     |
+| arbitrary array operations (`merge`, `reindex`, `apply_ufunc`)                 | Data preparation. The operator set is closed so that every tool reads the file the same way                                                                         |
 | filling a missing value (`.fillna`)                                            | Data preparation, or a `where` if the coordinate should not exist. Inside the language, only `shift(..., edge=)` fills ([absence](absence.md))                      |
 | schema migrations                                                              | —                                                                                                                                                                   |
 
@@ -101,7 +101,7 @@ The arrays it holds would build the same model, but an `expression:` string and
 reviewer could read. A library that wants a file passes a `dict` with the file's
 keys to `to_spec`, and calls `to_yaml()`.
 
-For math the language cannot express, a block of Python named in the file, with
-a cap on how many rows and columns it may emit, is planned as
-[#38](https://github.com/fluxopt/lpspec/issues/38). It has not shipped, and no
-key for it exists yet.
+The language has no escape hatch. Math it cannot express is a gap in the
+language, and a gap closes as a macro, a primitive or a formulation
+([the limits](../../about/limits.md)). Where the table above has a row, that row
+names what to write instead.