[fix](aggregate) Preserve AVG accumulator width in distinct rewrite - #67740
Open
morrySnow wants to merge 1 commit into
Open
[fix](aggregate) Preserve AVG accumulator width in distinct rewrite#67740morrySnow wants to merge 1 commit into
morrySnow wants to merge 1 commit into
Conversation
Problem: The multi-distinct rewrite decomposed AVG(DISTINCT BIGINT) into SUM and COUNT, but SUM(BIGINT) can overflow before the result is converted to AVG's return type. Fix: Losslessly widen the shared distinct argument to LARGEINT before creating SUM and COUNT so the rewritten expression preserves AVG's accumulator width and the argument sharing required by multi-distinct aggregation. Tests: - add focused rewrite coverage for the widened shared argument - add a boundary-value regression for AVG(DISTINCT BIGINT) Issue Number: None
morrySnow
requested review from
924060929,
englefly and
starocean999
as code owners
September 9, 2026 11:12
Contributor
Author
|
run buildall |
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
TPC-H: Total hot run time: 16882 ms |
Contributor
TPC-DS: Total hot run time: 81687 ms |
Contributor
ClickBench: Total hot run time: 14.74 s |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When multiple DISTINCT aggregates trigger the AVG decomposition rewrite,
AVG(DISTINCT BIGINT)can return an incorrect value. For example, averaging9223372036854775807and9223372036854775806produced-1.5; a predicatesuch as
avg_value > 0could therefore discard a row that should match.Root cause
The rewrite decomposed
AVG(DISTINCT BIGINT)intoSUM(DISTINCT BIGINT) / COUNT(DISTINCT BIGINT). Native AVG uses a LARGEINTaccumulator for BIGINT input, while SUM keeps a BIGINT accumulator. The SUM
overflowed before the division result was converted to AVG's return type.
Reproduction
With the multi-distinct rewrite enabled, the result was
-1.5instead ofapproximately
9.223372036854776e18.Fix
Losslessly widen a BIGINT AVG argument to LARGEINT before constructing the
replacement SUM and COUNT. The same widened expression is reused by both
aggregates, preserving the shared DISTINCT argument required by the
multi-distinct rewrite while matching AVG's original accumulator width.
Tests
the generated COUNT shares the same widened argument.
with another DISTINCT aggregate and an outer positive-value filter.
-1.5/0 matching rows to9.223372036854776e18/1 matching row.Issue Number: None