Skip to content

SIMILAR TO: SQL planner rejects non-literal LargeUtf8 / Utf8View patterns #23732

Description

@adriangb

Is your feature request related to a problem or challenge?

Follow-up to #23704 (which fixed the SIMILAR TO "failed to downcast array" panic from #22886 by coercing operands in the analyzer).

The analyzer now coerces SIMILAR TO operands to a common string type, but the SQL planner still rejects LargeUtf8 and Utf8View patterns before the analyzer ever runs. In datafusion/sql/src/expr/mod.rs:

let pattern_type = pattern.get_type(schema)?;
if pattern_type != DataType::Utf8 && pattern_type != DataType::Null {
    return plan_err!("Invalid pattern in SIMILAR TO expression");
}

So a non-literal pattern of type LargeUtf8 or Utf8View fails at plan time:

-- errors: "Invalid pattern in SIMILAR TO expression"
SELECT s FROM test, patterns WHERE s SIMILAR TO arrow_cast(pat, 'Utf8View');

#23704's examples all keep the Utf8View/LargeUtf8 on the input side with a Utf8 pattern, so this path isn't exercised there. Now that the analyzer coerces both operands, this planner restriction is unnecessary.

Describe the solution you'd like

Relax the pattern-type check to also accept LargeUtf8 and Utf8View:

if !matches!(
    pattern_type,
    DataType::Utf8 | DataType::LargeUtf8 | DataType::Utf8View | DataType::Null
) {
    return plan_err!("Invalid pattern in SIMILAR TO expression");
}

plus a sqllogictest case with a non-literal Utf8View (and LargeUtf8) pattern.

Describe alternatives you've considered

An earlier alternative implementation of the same fix (#22887) already included this planner relaxation and the corresponding test, so the change is small and self-contained.

Additional context

cc @u70b3 (author of #23704) — flagging in case you want to fold this into the same area. Happy to send the PR otherwise.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions