Skip to content

[fix](aggregate) Keep cast aggregates out of storage pushdown - #67737

Open
morrySnow wants to merge 1 commit into
apache:masterfrom
morrySnow:fix/reject-unsafe-cast-count-pushdown
Open

[fix](aggregate) Keep cast aggregates out of storage pushdown#67737
morrySnow wants to merge 1 commit into
apache:masterfrom
morrySnow:fix/reject-unsafe-cast-count-pushdown

Conversation

@morrySnow

Copy link
Copy Markdown
Contributor

Problem

Storage-layer aggregation could be selected for COUNT, MIN, and MAX
whose argument was a numeric CAST or TRY_CAST. The storage scan aggregates
the source column directly and cannot preserve the cast's value and null
semantics, so the optimization can change query results.

For example, with a non-null DOUBLE column containing 42, 2147483647,
2147483648, -2147483649, and 1e20, and with strict casts disabled:

SET enable_strict_cast = false;
SET enable_push_down_no_group_agg = true;
SELECT COUNT(CAST(d AS INT)), COUNT(TRY_CAST(d AS INT)) FROM count_t;

Only two casts are non-null, so both counts must be 2. Before this change,
the scan used pushAggOp=COUNT and returned 5. A similar narrowing cast from
BIGINT values -200, 0, 200 to TINYINT made cast-based MIN/MAX use
pushAggOp=MINMAX and return NULL instead of 0.

Root cause

The eligibility check treated a numeric cast of a slot as equivalent to the
slot itself. However, the storage aggregation descriptor carries the source
slot and operation, not an expression evaluator for the cast. Numeric casts
are not necessarily lossless: they can introduce nulls on overflow and can
change values relevant to extrema.

Fix

Restrict storage-layer aggregation arguments to raw slot references. Keep any
explicit CAST or TRY_CAST in the regular row-evaluation and aggregation
path, including casts exposed through a project alias. This conservative rule
preserves correctness for all scan types and all supported aggregate
operations while retaining pushdown for unmodified columns.

Tests

  • Added unit coverage for direct COUNT, MIN, and MAX arguments using
    CAST/TRY_CAST, plus a projected cast alias.
  • Ran PhysicalStorageLayerAggregateTest: 8 tests passed, 0 failed.
  • Rebuilt and deployed the FE in a local sandbox. Both counts return 2, all
    four cast-based extrema return 0, and the scan plans show
    pushAggOp=NONE.

Issue Number: None

Storage-layer aggregation reads source column values, so pushing COUNT, MIN,
or MAX through CAST/TRY_CAST can change nullability or extrema and return
incorrect results. Restrict storage aggregation arguments to raw slots and
keep explicit casts in row evaluation. Add direct and projected cast coverage.

Issue Number: None
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@morrySnow

Copy link
Copy Markdown
Contributor Author

run buildall

@morrySnow morrySnow changed the title [fix](nereids) Keep cast aggregates out of storage pushdown [fix](aggregate) Keep cast aggregates out of storage pushdown Sep 9, 2026
@hello-stephen

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 100.00% (3/3) 🎉
Increment coverage report
Complete coverage report

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants