fix: Restate RFC 0005 coercion as satisfaction then conversion - #175
Conversation
This change started on the implementation side: we identified that the coercion code had become too complicated to understand and review properly, and evaluated how we might improve it. The improvement was to restructure coercion as two explicitly ordered steps — first check whether the result already satisfies the target, then convert it if not. Working through that split surfaced adjustments we wanted to make in the specification itself, restated here. Because EXPR isn't yet widely deployed in production, we believe this is still a good time to make a change like this. The coercion rules applied the scalar rule only "when the target types have a single scalar type (without counting `nulltype` or `list[T]`)" and the list rule only when there was a single list type, prescribing no coercion at all for a target with two or more candidates of the same shape. That leaves reachable targets undefined: a target built from several candidate signatures can carry two scalar candidates (`zfill`, `int`, `float`, and `bool` each have a `float | int | string` parameter position), and implementations coerce there rather than reporting an ambiguity. The RFC also listed `range_expr` → `string` and `range_expr` → `list[int]` as rules whose conditions both hold for a `list[int] | string` target, with no stated winner. Restate the section in the two steps an implementation actually performs: - Satisfaction. If the result's type already satisfies the target it is used unchanged. Spell out the relation, including that a union target needs one member satisfied and that `list[T]` is covariant in `T`, so `list[int]` satisfies `list[any]` and `list[int | string]`. Note it is directional and therefore not the symmetric matching used to bind type variables — using one for the other accepts a `list[T1]` target by binding `T1` and discarding the binding — and that a result's type is never itself a union, since union constraints on unresolved values are decomposed first. - Conversion. Otherwise convert toward one of the target's destinations, a union contributing each member, first success winning. This replaces the single-candidate conditions and makes a union accept at least what each member accepts on its own. Destinations are ordered non-list before list, and within each group by a per-result-type preference table set by two principles: a value prefers to stay within its own kind, so a number remains a number before it becomes text, and a conversion that can fail is attempted before one that always succeeds, since a universal fallback attempted first would make every destination after it unreachable. So `int` prefers `float` over `string`; `float` prefers `int` (exact wholes) over `string`; `string` prefers `int`, then `float`, then the selective `bool` and `range_expr` parses, then `path`, which every string trivially satisfies; and a list source orders list destinations by its element type's preference, recursively. This makes the choice fully deterministic — `5` against `float | string` is `5.0`, `"5"` against `int | float` is `5` — where a first-draft of this rewrite had left same-shape order unspecified, letting the same template produce different jobs on different conforming implementations. The non-list-first level resolves the `range_expr` overlap: against `list[int] | string` the result is the canonical string `"1-5"`, whose cost does not depend on the range size. Add `string` → `bool` (the same case-insensitive spellings as RFC 0006's explicit `bool()` conversion) and `string` → `range_expr` to the conversion list. Both are non-destructive parses that succeed only for strings that unambiguously denote a value of the target type, in the same spirit as `string` → `int` and `string` → `float`, and they slot directly into the ordering principles — after the numeric parses, before the universal `path` fallback — so `"true"` against a `bool | path` target is `Bool(true)`. Since satisfaction runs first, the conversions no longer need their "when the target types do not include ..." conditions; those were restating the first step. State that `nulltype` is never a destination, so a `string` whose text is `"null"` does not become `null`, and that the type-variable rule holds at any nesting depth: an implementation must reject a `list` destination whose element type mentions an unbound type variable rather than binding the variable and discarding the binding. Also sharpen the unresolved-value narrowing: against a union target the constraint narrows to the union of every destination with a type-level rule, rather than betting on any one of them, because the type level cannot see the payload that decides which destination wins. The narrowed constraint thus always satisfies the target and always describes the concrete result — an `unresolved[float]` narrows to `unresolved[int | string]` against `int | string`, covering both the 3.0 payload that lands on `int` and the 3.5 payload that falls through to `string`. For a non-union target exactly one destination exists, so the constraint is exactly the type evaluation will produce. Matching user-facing language in the wiki's Expression Language page. The openjd-rs implementation matches this text, with every stated example pinned by a test. Signed-off-by: Mark <399551+mwiebe@users.noreply.github.com>
b53c27f to
aaaa197
Compare
|
@uberware this PR is tweaking the EXPR RFC / spec slightly about coercion, would love your eyes and opinions on it if you are able to look. (I'm planning to merge this, and iterate on any follow-ups) |
|
Happy to — thanks for the ping. Read the merged text closely; the two-step split is a real improvement and I'm implementing against it rather than the old wording. Notes below, in descending order of how much I think they matter. The satisfaction/conversion split resolves something I had implemented by accident. My evaluator got the "never convert what the target already admits" outcome by asking the old negated-inclusion questions ( 1. Does a failed list destination charge operations for what it materialized? This is the one question I'd most like pinned down. "A destination that fails is not an error so long as a later one succeeds", combined with element-wise 2. The other rules in the table pick between values an implementation already had to produce. These two make expressions valid in positions where they previously weren't — a format string resolving to 3. Is the Non-list-before-list makes 4. Unresolved narrowing — agreed, and worth keeping the invariant explicit. Narrowing to the union of every destination rather than betting on one is the right call, and the sentence that satisfaction narrows to the source type ( Where my implementation stands, for whatever it's worth as a second data point. Three gaps against the merged text, all of them me accepting less than the spec now requires: multi-candidate scalar union targets (rejected today), Two things unrelated to the rewrite, since I have your attention: The corpus offer is still open, and coercion is where it's thickest. I asked on OpenJobDescription/openjd-rs#291 where you'd want the files, but I posted that comment after that issue had already been closed, so I suspect it was never seen — my fault for replying into a closed thread. The offer stands unchanged: 1,063 differential cases with 133 adjudicated value divergences and 248 operation-count divergences, each with written reasoning, offered under MIT-0. Tell me where to put them (an issue on either repo, a PR, a gist) and they go out the same day. The second question from that comment also still stands: is Two housekeeping items you may not have linked up:
|
Description of the change. What is being added or fixed?
This change was identified while working on the Rust implementation. The
coercion code was too complicated to understand and review properly,
and we evaluated how to improve it. The change was to make coercion
run as two steps — first check whether the result already satisfies
the target, then convert it if not. Working through that split surfaced
adjustments we wanted to make in the specification itself, which this
PR contains. Because EXPR isn't yet widely deployed in production,
we believe this is still a good time to make a change like this.
Each field in a template gives the expression it contains a target type —
the type of result the field expects. If the expression's result has a
different type, it may be converted losslessly: an
intresult in a fieldthat expects a string becomes that string. RFC 0005's "Implicit Type
Coercion" section defines these conversions, and this PR updates that
section. The coercion applied while resolving function calls — what makes
1 + 2.0promote the1to1.0— is a separate mechanism and is nottouched.
A target type can offer a choice: an optional string field accepts a string
or null, and a command-line argument accepts a string, a list of strings, or
null. The current wording has two gaps there:
its kind (one plain type, or one list type). If it offers two — e.g.
float | int | string, which several built-in functions accept — thespec doesn't prescribe what to do.
1-5can become the string
"1-5"or the list[1, 2, 3, 4, 5], and for atarget accepting both, the spec doesn't say which you get.
This amendment states the procedure implementations should follow:
intfitsint | string, so it stays anintrather than becoming a string.and use the first conversion that works.
The order the types are tried in is specified by a small table with
two rules behind it: a number prefers to stay a number before becoming text,
and conversions that always succeed (anything can become a string; any
string is a valid path) are tried last. Non-list types come before list types.
So
5againstfloat | stringbecomes5.0,"5"againstint | floatbecomes
5while"5.0"becomes5.0, and1-5against "list of ints or string"becomes the string
"1-5"— settling gap 2. Trying each offered type settlesgap 1, makes the outcome identical across conforming implementations,
and guarantees that adding another accepted type to a field never breaks
a conversion that used to work; the old wording accidentally implied otherwise.
Other clarifications included:
..." condition. Step 1 covers all of those, so it is stated once.
field because it is already null; a string containing the text
"null"stays a string.
T,T1, ...) arenever valid conversion targets. That was already the rule; it now
explicitly applies at any nesting depth, e.g.
list[T1].against these same rules at the type level. The spec now says what
validation may claim about the eventual result: a type the target accepts,
though not necessarily which of several offered types the actual value
will land on, since that can depend on the value itself.
Two conversions are added to the spec's list —
string → bool(the samecase-insensitive spellings as the explicit
bool()conversion) andstring → range_expr. They are non-destructive parses that succeed only forstrings that unambiguously denote a value of the target type, in the same
spirit as
string → intandstring → float, and they slot directly intothe ordering principles: tried after the numeric parses, before the universal
pathfallback. Cases the spec already pinned down are otherwise unchanged,and previously unspecified ones are now defined. The wiki's
Expression Language page gets matching wording.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.