docs: correct window function support status and limitations#4833
Merged
Conversation
Window execution runs natively and is enabled by default, but the compatibility guide and roadmap still described it as disabled by default and known to be incorrect, citing the now-closed epic apache#2721. Update the window docs to describe the current reality: which ranking, value, and aggregate window functions are accelerated, which frame units and bounds are handled, and the specific shapes that fall back to Spark. State the unsupported aggregate input types positively (the Spark types Comet does not accept) rather than by exclusion.
Member
Author
|
@comphead could you review this for accuracy? |
comphead
reviewed
Jul 6, 2026
comphead
approved these changes
Jul 6, 2026
comphead
left a comment
Contributor
There was a problem hiding this comment.
Thanks @andygrove
you mentioned #4766 but we also need to mentioned collect_list, collect_set is in progress
Co-authored-by: Oleks V <comphead@users.noreply.github.com>
Member
Author
Thanks. I added this. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
N/A
Rationale for this change
The documentation for window functions was out of date and contradicted itself. The compatibility guide (
compatibility/operators.md) and the contributor roadmap still stated that native window execution is "disabled by default and known to be incorrect" and "should not be used in production", citing the epic #2721. That epic, along with the other referenced issues (#2705, #2841, #4007), is now closed.In reality,
WindowExecruns natively andspark.comet.exec.window.enableddefaults totrue. A broad set of window functions is accelerated, and any shape Comet does not handle falls back to Spark rather than producing an incorrect result. Theexpressions.mdandoperators.mdpages already reflected this, so the guide and roadmap were the stale pages that disagreed with them.What changes are included in this PR?
compatibility/operators.mdto describe the current behavior: the ranking, value, and aggregate window functions that are accelerated, the supported frame units and bounds, and the specific cases that fall back to Spark.WindowExecnote inoperators.mdto drop the closed-issue reference and point to the window compatibility section.window_funcsprose inexpressions.mdto note that statistical aggregates run natively as plain aggregations but fall back as window functions, and to link to the compatibility guide.lag/leaddefault (Support non-Literal default value forLAG/LEAD#4268), decimal sliding-framesum/avg([Bug] SUM(decimal) over a sliding window frame returns wrapped out-of-range value on overflow instead of NULL #4729, closed by the intentional fallback). New issues filed for the previously untracked gaps: RANGE frame with an explicit date/decimal offset (Support RANGE window frames with an explicit offset on DATE or DECIMAL ORDER BY #4834),first_value/last_valueon a RANGE frame with a literal offset (Support FIRST_VALUE/LAST_VALUE on a RANGE window frame with a literal offset #4835),GROUPSframe units (Support GROUPS window frame units #4836), andWindowGroupLimitExec(Support WindowGroupLimitExec (window-based limit pushdown) #4837).How are these changes tested?
Documentation-only change. Formatting was verified with
prettier --check.