Skip to content

[BUG] dedup on a dotted object-hierarchy field throws "String cannot be cast to java.util.Map" (Calcite path) #5685

Description

@ahkcs

Describe the bug

On the Calcite path (plugins.calcite.enabled=true), dedup on a dotted field that maps into an OpenSearch object hierarchy fails 100% of the time with a ClassCastException. The same field projected with | fields works, so the failure is specific to dedup (and any command that turns the field into a partition/group key).

To Reproduce

Index with an object hierarchy (e.g. OpenTelemetry logs):

PUT logs-otel/_mapping
{ "properties": { "resource": { "properties": { "attributes": { "properties":
  { "k8s": { "properties": { "namespace": { "properties": { "name": { "type": "keyword" }}}}}}}}}}

Query:

source=logs-otel
| dedup resource.attributes.k8s.namespace.name
| fields resource.attributes.k8s.namespace.name, severityText, body
| head 50

Result: java.sql.SQLException: exception while executing query: class java.lang.String cannot be cast to class java.util.Map.

Expected behavior

Dedup should deduplicate on the leaf value, exactly as | fields resource.attributes.k8s.namespace.name resolves it. No cast error.

Root cause

OpenSearchIndex.getFieldTypes() flattens the mapping and emits a column for every intermediate object node and leaf (OpenSearchDataType.traverseAndFlatten). Each object/STRUCT node becomes a Calcite MAP<VARCHAR,ANY> (OpenSearchTypeFactory ~L186-190). When the exact dotted leaf is not itself a distinct mapped column, QualifiedNameResolver matches the longest existing prefix (a MAP column) and wraps the remaining segments in an INTERNAL_ITEM/ITEM call — so the dedup key becomes a MAP-typed RexCall, not a scalar RexInputRef.

Two throw sites, both (Map<String,Object>) value on a scalar String:

  • Agg-pushdown decode: ObjectContent.map() L81 (dedup w/o keepempty is rewritten to composite-agg + top_hits by DedupPushdownRule), reached via OpenSearchExprValueFactory.parse STRUCT branch + OpenSearchResponse.handleAggregationResponse.
  • JDBC decode: OpenSearchTypeFactory.getExprValueByExprType STRUCT branch L347, via OpenSearchExecutionEngine.buildResultSet/processValue.

DedupPushdownRule's guards miss it: it only bails on bare-RexInputRef MAP/ARRAY, and the nested-path guard (Utils.resolveNestedPath) fires only for nested→ARRAY, never plain object→STRUCT. | fields avoids it by resolving the leaf scalar (ExprValueUtils.resolveRefPaths) without re-typing the parent object.

Plugins/paths affected

Calcite path only. V2/legacy is unaffected (no dedup pushdown; DedupeOperator resolves the leaf scalar in memory).

Proposed fix

  1. QualifiedNameResolver: don't let a dedup/partition key silently become a MAP-typed ITEM when a scalar leaf is intended.
  2. DedupPushdownRule: extend the guard to bail on MAP (STRUCT) / INTERNAL_ITEM keys, not just bare-RexInputRef MAP/ARRAY.
  3. Defense-in-depth: guard the two Map casts to degrade a STRUCT-vs-scalar mismatch gracefully.
  4. Add a CalcitePPLDedupIT case with an object hierarchy (+ CalciteNoPushdownIT). No current dedup test uses a dotted/object field.

Metadata

Metadata

Assignees

No one assigned

    Labels

    PPLPiped processing languagebugSomething isn't working

    Type

    No type

    Projects

    Status
    Not Started

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions