Skip to content

feat: Simplify min(DISTINCT x) to min(x) - #25126

Draft
mkleen wants to merge 1 commit into
apache:mainfrom
mkleen:min-max-distinct
Draft

feat: Simplify min(DISTINCT x) to min(x)#25126
mkleen wants to merge 1 commit into
apache:mainfrom
mkleen:min-max-distinct

Conversation

@mkleen

@mkleen mkleen commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

Simplify min(DISTINCT x) to min(x) in a grouped query makes the query need 1000x less memory:

query rule fires smallest limit that completes
SELECT g, min(DISTINCT x) FROM t GROUP BY g yes 192M, fails at 160M
SELECT g, min(x) FROM t GROUP BY g no 192K, fails at 160K

The aggregation functions min is duplicate insensitive, therefore we can do the following transformation:

  • min(DISTINCT x) is identical min(x)

This prevents that the optimizer rule SingleDistinctToGroupBy fires which leads to a more efficient plan:

EXPLAIN FORMAT indent SELECT g, min(DISTINCT x) FROM t GROUP BY g;

Before this change:

Projection: t.g, min(alias1) AS min(DISTINCT t.x)
  Aggregate: groupBy=[[t.g]], aggr=[[min(alias1)]]
    Aggregate: groupBy=[[t.g, t.x AS alias1]], aggr=[[]]
      TableScan: t projection=[g, x]

after:

Aggregate: groupBy=[[t.g]], aggr=[[min(t.x) AS min(DISTINCT t.x)]]
   TableScan: t projection=[g, x]                                                 

The second plan is much more memory efficient:

What changes are included in this PR?

  • Extend of min/max to simplify min(distinct x)/max(distint x) to min(x)/max(x)
  • Adoption of expression simplifier to only report a change if a simplication happend
  • Unalias the expression in SingleDistinctToGroupBy to make this optimization work in the rule
  • Tests

What is the testing strategy for this PR?

Existing tests pass; new slt added.

Are there any user-facing changes?

No.

@mkleen mkleen changed the title feat: Remove distinct from min/max aggregations feat: Simplify min(distinct x)/max(distinct x) to min(x)/max(x) Sep 9, 2026
@mkleen mkleen changed the title feat: Simplify min(distinct x)/max(distinct x) to min(x)/max(x) feat: Simplify min(distinct x) to min(x) Sep 9, 2026
@mkleen mkleen changed the title feat: Simplify min(distinct x) to min(x) feat: Simplify min(DISTINCT x) to min(x) Sep 9, 2026
@github-actions github-actions Bot added optimizer Optimizer rules sqllogictest SQL Logic Tests (.slt) functions Changes to functions implementation labels Sep 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

functions Changes to functions implementation optimizer Optimizer rules sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant