From aaaa1973365acf5d05212b4b2256ebb5ef4b7e73 Mon Sep 17 00:00:00 2001 From: Mark <399551+mwiebe@users.noreply.github.com> Date: Wed, 12 Aug 2026 14:27:02 -0700 Subject: [PATCH] fix: Restate RFC 0005 coercion as satisfaction then conversion MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This change started on the implementation side: we identified that the coercion code had become too complicated to understand and review properly, and evaluated how we might improve it. The improvement was to restructure coercion as two explicitly ordered steps — first check whether the result already satisfies the target, then convert it if not. Working through that split surfaced adjustments we wanted to make in the specification itself, restated here. Because EXPR isn't yet widely deployed in production, we believe this is still a good time to make a change like this. The coercion rules applied the scalar rule only "when the target types have a single scalar type (without counting `nulltype` or `list[T]`)" and the list rule only when there was a single list type, prescribing no coercion at all for a target with two or more candidates of the same shape. That leaves reachable targets undefined: a target built from several candidate signatures can carry two scalar candidates (`zfill`, `int`, `float`, and `bool` each have a `float | int | string` parameter position), and implementations coerce there rather than reporting an ambiguity. The RFC also listed `range_expr` → `string` and `range_expr` → `list[int]` as rules whose conditions both hold for a `list[int] | string` target, with no stated winner. Restate the section in the two steps an implementation actually performs: - Satisfaction. If the result's type already satisfies the target it is used unchanged. Spell out the relation, including that a union target needs one member satisfied and that `list[T]` is covariant in `T`, so `list[int]` satisfies `list[any]` and `list[int | string]`. Note it is directional and therefore not the symmetric matching used to bind type variables — using one for the other accepts a `list[T1]` target by binding `T1` and discarding the binding — and that a result's type is never itself a union, since union constraints on unresolved values are decomposed first. - Conversion. Otherwise convert toward one of the target's destinations, a union contributing each member, first success winning. This replaces the single-candidate conditions and makes a union accept at least what each member accepts on its own. Destinations are ordered non-list before list, and within each group by a per-result-type preference table set by two principles: a value prefers to stay within its own kind, so a number remains a number before it becomes text, and a conversion that can fail is attempted before one that always succeeds, since a universal fallback attempted first would make every destination after it unreachable. So `int` prefers `float` over `string`; `float` prefers `int` (exact wholes) over `string`; `string` prefers `int`, then `float`, then the selective `bool` and `range_expr` parses, then `path`, which every string trivially satisfies; and a list source orders list destinations by its element type's preference, recursively. This makes the choice fully deterministic — `5` against `float | string` is `5.0`, `"5"` against `int | float` is `5` — where a first-draft of this rewrite had left same-shape order unspecified, letting the same template produce different jobs on different conforming implementations. The non-list-first level resolves the `range_expr` overlap: against `list[int] | string` the result is the canonical string `"1-5"`, whose cost does not depend on the range size. Add `string` → `bool` (the same case-insensitive spellings as RFC 0006's explicit `bool()` conversion) and `string` → `range_expr` to the conversion list. Both are non-destructive parses that succeed only for strings that unambiguously denote a value of the target type, in the same spirit as `string` → `int` and `string` → `float`, and they slot directly into the ordering principles — after the numeric parses, before the universal `path` fallback — so `"true"` against a `bool | path` target is `Bool(true)`. Since satisfaction runs first, the conversions no longer need their "when the target types do not include ..." conditions; those were restating the first step. State that `nulltype` is never a destination, so a `string` whose text is `"null"` does not become `null`, and that the type-variable rule holds at any nesting depth: an implementation must reject a `list` destination whose element type mentions an unbound type variable rather than binding the variable and discarding the binding. Also sharpen the unresolved-value narrowing: against a union target the constraint narrows to the union of every destination with a type-level rule, rather than betting on any one of them, because the type level cannot see the payload that decides which destination wins. The narrowed constraint thus always satisfies the target and always describes the concrete result — an `unresolved[float]` narrows to `unresolved[int | string]` against `int | string`, covering both the 3.0 payload that lands on `int` and the 3.5 payload that falls through to `string`. For a non-union target exactly one destination exists, so the constraint is exactly the type evaluation will produce. Matching user-facing language in the wiki's Expression Language page. The openjd-rs implementation matches this text, with every stated example pinned by a test. Signed-off-by: Mark <399551+mwiebe@users.noreply.github.com> --- rfcs/0005-expression-language.md | 152 +++++++++++++++++++++++----- wiki/2026-02-Expression-Language.md | 66 ++++++++---- 2 files changed, 171 insertions(+), 47 deletions(-) diff --git a/rfcs/0005-expression-language.md b/rfcs/0005-expression-language.md index da9e4a9..495d962 100644 --- a/rfcs/0005-expression-language.md +++ b/rfcs/0005-expression-language.md @@ -755,34 +755,109 @@ The `T` type for a `list[T]` must satisfy: ### Implicit Type Coercion Implicit non-destructive type coercion is performed where the intent is obvious. The caller -provides a set of target types it expects, and we use this to affect coercion decisions. -When the expression result does not directly match a target type, the following implicit -conversions are attempted: - -- `int` → `float` when the target types do not include `int` -- `path` → `string` when the target types do not include `path` -- `range_expr` → `string` when the target types do not include `range_expr` (produces canonical form like `"1-5"`) -- `range_expr` → `list[int]` when the target types include `list[int]` but not `range_expr`. - This is the only list type a `range_expr` implicitly coerces to: a `list[T]` target with any - other element type (e.g. `list[float]` or `list[string]`) is an error. (A `list[any]` target - is also accepted, since a `list[int]` value already satisfies it.) Implicit rules do not - chain, so the materialized `list[int]` is not further widened element-wise toward the target. +provides the target type it expects, which need not be a single concrete type: it may be +`any`, a union such as `string?` or `string? | list[string]`, or a list whose element type +is either of those. Coercion asks two questions about the expression result, in this order. + +**1. Satisfaction.** Does the result's type already satisfy the target? If so, the value is +used unchanged and no conversion is attempted. A type satisfies a target when: + +- the target is `any`, which every type satisfies; +- the target is a union and *any* one of its members is satisfied, so an `int` satisfies + `int | string` and a `null` satisfies `T?`; +- both are lists and the result's element type satisfies the target's element type, so a + `list[int]` satisfies `list[any]` and `list[int | string]`; +- or the two types are equal. + +A result's type is never itself a union: concrete values have a single runtime type, and +for the not-yet-known values checked during validation, a union constraint is decomposed +into its members before these steps apply (see +[Coercion of Unresolved Values](#coercion-of-unresolved-values)). + +Satisfaction is directional: `int` satisfies `any`, but `any` does not satisfy `int`. It is +therefore not the same relation as the symmetric matching used to bind type variables during +signature matching, and an implementation must not use one for the other — the symmetric +relation would accept a `list[T1]` target by binding `T1` and discarding the binding. + +Because satisfaction is checked first, a result whose type the target already admits is +never converted: an `int` against `int | string` stays an `int` rather than becoming a +string. Consequently, a conversion is only ever attempted toward a type the result does +not already have. + +**2. Conversion.** Otherwise the result is converted toward one of the target's +*destinations*. A destination is a single non-union type toward which one of the +conversions below can be attempted. A non-union target is its own only destination. A union +target contributes each of its members, because producing any one of them satisfies the +union. Destinations are attempted in order and the first that converts gives the result; a +destination that fails is not an error so long as a later one succeeds, and the coercion +fails only if none converts. + +Destinations are attempted in a fixed order with two levels. **Non-list destinations come +before list destinations**: converting to a scalar produces a single value whose cost does +not depend on the source, while converting to a list materializes its elements and may +exceed an implementation's size limits, so this ordering keeps the outcome independent of +how large the value is. Within each of the two groups, the order is determined by the +result's type, following two principles: a value prefers to stay within its own kind — a +number remains a number before it becomes text — and a conversion that can fail is +attempted before one that always succeeds, because a universal fallback attempted first +would make every destination after it unreachable. + +| Result type | Destinations, in order | Notes | +|-------------|------------------------|-------| +| `bool` | `string` | only conversion | +| `int` | `float`, then `string` | stays a number first; text is the universal fallback | +| `float` | `int`, then `string` | `int` succeeds only for exact whole values, so `3.0` against `int \| string` gives `3` while `3.5` gives `"3.5"` | +| `string` | `int`, then `float`, then `bool`, then `range_expr`, then `path` | every string that parses as `int` also parses as `float`, so `int` must come first; `bool` and `range_expr` are selective parses, tried after the numeric ones; every string is a valid `path`, so `path` comes last | +| `path` | `string` | only conversion | +| `range_expr` | `string`, then `list[int]` | the non-list-first rule; when a target offers both, the `list[int]` destination is unreachable, and a template that wants the list uses the explicit `list()` conversion (RFC 0006) | +| `list[S]` | list destinations in `S`'s order, applied to their element types | `list[float]` against `list[int] \| list[string]` attempts `list[int]` first | + +So `5` against `float | string` becomes `5.0`, and `"5"` against `int | float` becomes `5` +while `"5.0"` becomes `5.0` — the stricter parse is attempted first, and the string's own +lexical form routes it. An empty list (`list[nulltype]`) converts to every list type and +the result is the empty list regardless; the nominal element type it carries follows the +first list destination in the union's normalized member order. + +`nulltype` is never a destination, because no conversion produces `null`. A `null` result +reaches a `T?` target by satisfying it, and a `nulltype` member is otherwise ignored when +converting; in particular, a `string` whose text happens to be `"null"` does not become +`null`. Likewise a type variable, `noreturn`, `unresolved[T]`, or a `list` parameterized by +any of those contributes no destination, so a target composed only of such types cannot be +coerced to at all. + +Because destinations are considered one at a time, a union target accepts at least +everything each of its members accepts on its own. Adding an alternative to a target never +takes away a conversion that was previously accepted, though it can change which +destination is chosen, and with it the resulting value. + +The non-destructive conversions are: + +- `bool`/`int`/`float`/`path`/`range_expr` → `string` (a `range_expr` produces its canonical + form, like `"1-5"`) +- `string` → `path` +- `float`/`string` → `int` (error if the value cannot be represented exactly, e.g. `3.75`, + `""`, `"nothing"`, `"3.1"`) +- `int`/`string` → `float` (error if the string cannot be parsed, e.g. `""`, `"nothing"`) +- `string` → `bool`, accepting the same case-insensitive spellings as the explicit `bool()` + conversion (RFC 0006): `"true"`/`"yes"`/`"on"`/`"1"` become `true` and + `"false"`/`"no"`/`"off"`/`"0"` become `false` (error otherwise) +- `string` → `range_expr` (error if the string does not parse as a range expression, e.g. + `""`, `"1-"`) +- `range_expr` → `list[int]`. This is the only list type a `range_expr` implicitly coerces + to, so the destination is accepted exactly when a `list[int]` value would satisfy it — + `list[int]`, `list[any]`, and `list[int | string]` — and rejected for any other element + type, such as `list[float]` or `list[string]`. Implicit rules do not chain, so the + materialized `list[int]` is not further widened element-wise toward the destination. A template that wants the widened list can chain the explicit conversion `list(value: range_expr) -> list[int]` (RFC 0006) — e.g. `list(r)` in a `list[string]` context — where the `list[T]` → `list[U]` rule below then applies to the conversion's result. -- `list[T]` → `list[U]` when each element `T` can be coerced to `U` (e.g., `list[path]` → `list[string]`) +- `list[T]` → `list[U]` when each element `T` can be coerced to `U` (e.g., `list[path]` → `list[string]`). + This applies recursively for nested lists. - `list[nulltype]` → `list[T]` for any `T` (empty list literal is compatible with any list type) -- Any scalar value when the target types have a single scalar type (without counting `nulltype` or `list[T]` for any `T`). - The value is coerced non-destructively to that type using the non-destructive coercion rules below. - For example, in a format string context where the target type is `string?`, - an `int` result is coerced to `string`. - - `bool`/`int`/`float`/`path` → `string` - - `string` → `path` - - `float`/`string` → `int` (error if value cannot be represented exactly, e.g. `3.75`, `""`, `"nothing"`, `"3.1"`) - - `int`/`string` → `float` (error if string cannot be parsed, e.g. `""`, `"nothing"`) -- `[v1, v2, v3, ...]` any values when the target types have a single `list` type (without counting `list[nulltype]`). - Every value is coerced non-destructively to `T` where that type is `list[T]`. This applies recursively - for nested lists. The non-destructive coercions are the same as defined for scalar values above. + +For example, in a format string context where the target type is `string?`, an `int` result +is coerced to `string`: it does not satisfy `string?`, whose destinations are just `string` +once the `nulltype` member is set aside, and `int` → `string` is in the table above. A **type variable** target (`T`, `T1`, `T2`, `T3`) has no coercion rule, for concrete and unresolved values alike. Type variables are placeholders in generic function signatures, @@ -791,13 +866,36 @@ Target Type Propagation Rules in [Expression Evaluation](#expression-evaluation) one still unbound is always an error. Accepting a type-variable target for unresolved values would let validation pass an expression that can only fail once the value is known. +This holds at any nesting depth: a `list[T1]` target is no more usable than a `T1` target, +since a list cannot be produced without knowing its element type. An implementation must +reject a `list` destination whose element type mentions an unbound type variable, rather +than binding the variable and discarding the binding. + #### Coercion of Unresolved Values -Target-type coercion applies the same conversion table to `unresolved[T]` values at the type -level: the payload remains unresolved, but its constraint is narrowed to the coercion result. -For example, `unresolved[int]` against a `string` target becomes `unresolved[string]`, and a +Target-type coercion applies the same two steps to `unresolved[T]` values at the type level: +the payload remains unresolved, but its constraint is narrowed to the coercion result. For +example, `unresolved[int]` against a `string` target becomes `unresolved[string]`, and a `range_expr` → `list[int]` coercion of an unresolved value yields `unresolved[list[int]]`. +The narrowed constraint is the type the applicable step produces, which is not always the +target. Satisfaction leaves the value alone, so its constraint keeps the source type: an +`unresolved[list[int]]` against a `list[any]` target stays `unresolved[list[int]]` rather than +widening to `unresolved[list[any]]`, matching the concrete `list[int]` value that would be +returned unchanged. Conversion yields the type its own rule produces, so an +`unresolved[range_expr]` against `list[any]` becomes `unresolved[list[int]]`, because +materializing a range only ever produces a `list[int]`. + +The narrowed constraint always satisfies the target, and it always describes the concrete +result: the concrete result's type satisfies the narrowed constraint. The type level cannot +see the payload that decides which destination of a union target wins, so conversion narrows +to the union of every destination with a type-level rule, rather than betting on any one of +them — anything narrower would misdescribe some resolved value. Against an `int | string` +target an `unresolved[float]` narrows to `unresolved[int | string]`: a `3.0` payload takes +the `float` → `int` rule while a `3.5` payload fails it and falls through to `string`, and +both outcomes lie within the constraint. For a non-union target exactly one destination +exists, so the constraint is exactly the type evaluation will produce. + When the constraint is a union, coercion is existential: it succeeds if at least one member of the union can coerce to the target, and the possibilities that cannot are discarded. Coercing `unresolved[int | string]` to an `int` target yields `unresolved[int]`, and diff --git a/wiki/2026-02-Expression-Language.md b/wiki/2026-02-Expression-Language.md index b36019f..9d48bfe 100644 --- a/wiki/2026-02-Expression-Language.md +++ b/wiki/2026-02-Expression-Language.md @@ -849,27 +849,53 @@ representation of frame ranges. Use `list(Task.Param.Frame)` to convert to a lis #### 1.2.3. Implicit Type Coercion As a glue expression language intended for convenience, implicit non-destructive type -coercion is performed where the intent is obvious. The following implicit conversions are supported: - -- `int` → `float` when the target types do not include `int` -- `path` → `string` when the target types do not include `path` -- `range_expr` → `string` when the target types do not include `range_expr` (produces canonical form like `"1-5"`) -- `range_expr` → `list[int]` when the target types include `list[int]` but not `range_expr`. - This is the only list type a `range_expr` implicitly coerces to: a `list[T]` target with any - other element type (e.g. `list[float]` or `list[string]`) is an error. (A `list[any]` target - is also accepted, since a `list[int]` value already satisfies it.) Implicit rules do not - chain, so the materialized `list[int]` is not further widened element-wise. Use the explicit - conversion `list(r)` to get a `list[int]` value that the `list[T]` → `list[U]` rule then applies to. -- `list[T]` → `list[U]` when each element `T` can be coerced to `U` (e.g., `list[path]` → `list[string]`) +coercion is performed where the intent is obvious. + +Each field gives the expression a target type, which may be a single type, an optional type +like `string?`, or a choice like `string? | list[string]` for a list item. Coercion happens in +two steps: + +1. **If the result already fits the target, it is used as is.** A value fits when its type is + the target, or is one of the target's choices, or the target is `any`. A `list[int]` also + fits `list[any]` and `list[int | string]`, since its elements fit the element type. +2. **Otherwise the result is converted** toward one of the target's types, using the + conversions below. Where the target offers a choice, each is tried in turn and the first + that works is used. Non-list types are tried before list types, and within that, the + result's type sets the order: a number prefers to stay a number before becoming text, and + conversions that always succeed (anything → `string`, any `string` → `path`) are tried + last, since trying them first would make the others unreachable. So `5` against + `float | string` becomes `5.0`, `"5"` against `int | float` becomes `5` while `"5.0"` + becomes `5.0`, and `list[float]` against `list[int] | list[string]` tries `list[int]` + first. The full ordering table is in RFC 0005. + +Because of the first step, a value is never converted when the target already accepts it — +an `int` stays an `int` for an `int | string` target rather than becoming a string. + +A `nulltype` choice is never converted *into*: `null` reaches a `T?` target only by already +being `null`, so a string whose text happens to be `"null"` stays a string. Because non-list +types are tried first, a `range_expr` against `list[int] | string` becomes the string `"1-5"` +rather than the expanded list. + +The conversions are: + +- `bool`/`int`/`float`/`path`/`range_expr` → `string` (a `range_expr` produces its canonical + form, like `"1-5"`) +- `string` → `path` +- `float`/`string` → `int` (error if value cannot be represented exactly, e.g. `3.75`, `""`, `"3.1"`) +- `int`/`string` → `float` (error if string cannot be parsed, e.g. `""`, `"nothing"`) +- `string` → `bool`, accepting the same case-insensitive spellings as the explicit `bool()` + conversion: `"true"`/`"yes"`/`"on"`/`"1"` become `true` and `"false"`/`"no"`/`"off"`/`"0"` + become `false` (error otherwise) +- `string` → `range_expr` (error if the string is not a valid range expression, like `"1-10"`) +- `range_expr` → `list[int]`. This is the only list type a `range_expr` implicitly coerces to, + so it is accepted for a target a `list[int]` would fit — `list[int]`, `list[any]`, or + `list[int | string]` — and is an error for any other element type, such as `list[float]` or + `list[string]`. Implicit rules do not chain, so the materialized `list[int]` is not further + widened element-wise. Use the explicit conversion `list(r)` to get a `list[int]` value that + the `list[T]` → `list[U]` rule then applies to. +- `list[T]` → `list[U]` when each element `T` can be coerced to `U` (e.g., `list[path]` → `list[string]`). + This applies recursively for nested lists. - `list[nulltype]` → `list[T]` for any `T` (empty list literal is compatible with any list type) -- Any scalar value when the target types have a single scalar type. The value is coerced - non-destructively to that type: - - `bool`/`int`/`float`/`path` → `string` - - `string` → `path` - - `float`/`string` → `int` (error if value cannot be represented exactly, e.g. `3.75`, `""`, `"3.1"`) - - `int`/`string` → `float` (error if string cannot be parsed, e.g. `""`, `"nothing"`) -- `[v1, v2, ...]` any values when the target types have a single `list` type. Every value is - coerced non-destructively to `T` where that type is `list[T]`. This applies recursively for nested lists. #### 1.2.4. Method Call Coercion Restriction