branch-4.1: [fix](window) Respect frames when simplifying window functions #67706 - #67743
Open
github-actions[bot] wants to merge 1 commit into
Open
branch-4.1: [fix](window) Respect frames when simplifying window functions #67706#67743github-actions[bot] wants to merge 1 commit into
github-actions[bot] wants to merge 1 commit into
Conversation
## Problem The optimizer simplifies window functions when the partition keys are unique because each partition contains at most one row. It replaces functions such as `SUM(value)` with `value` and a non-null `COUNT` with `1`. That replacement produces incorrect results when the window frame excludes the current row. For a one-row partition, a preceding-only or following-only frame is empty, so `SUM` must return `NULL` and `COUNT` must return `0`. ## Root cause The simplification rule used partition cardinality alone and did not check whether the normalized frame actually contains the partition's only row. ## How to reproduce Create a merge-on-write unique-key table with one row and run `SUM` and `COUNT(*)` over a window partitioned by the unique key with `ROWS BETWEEN 1 PRECEDING AND 1 PRECEDING` (or the equivalent following-only frame). Before this change, the optimizer removed the window and returned the current value and `1`; the correct result is `NULL` and `0`. ## Fix Check the normalized frame boundaries before simplifying frame-dependent functions. `COUNT`, `SUM`, `MIN`, `MAX`, `AVG`, `FIRST_VALUE`, and `LAST_VALUE` are simplified only when the frame contains the current row. Ranking functions retain their existing simplification because their result does not depend on frame membership. ## Tests - Added a regression suite covering preceding-only and following-only frames that must retain `PhysicalWindow` and return `NULL`/`0`. - Added current-row and centered-frame cases that continue to eliminate `PhysicalWindow` and return the simplified values. - Ran the new regression suite in verification mode: 1 suite passed, 0 failed. - Built the FE successfully with all reactor modules passing and no checkstyle violations.
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
|
run buildall |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cherry-picked from #67706