Skip to content

docs: correct window function support status and limitations#4833

Merged
andygrove merged 4 commits into
apache:mainfrom
andygrove:docs-window-functions
Jul 6, 2026
Merged

docs: correct window function support status and limitations#4833
andygrove merged 4 commits into
apache:mainfrom
andygrove:docs-window-functions

Conversation

@andygrove

@andygrove andygrove commented Jul 6, 2026

Copy link
Copy Markdown
Member

Which issue does this PR close?

N/A

Rationale for this change

The documentation for window functions was out of date and contradicted itself. The compatibility guide (compatibility/operators.md) and the contributor roadmap still stated that native window execution is "disabled by default and known to be incorrect" and "should not be used in production", citing the epic #2721. That epic, along with the other referenced issues (#2705, #2841, #4007), is now closed.

In reality, WindowExec runs natively and spark.comet.exec.window.enabled defaults to true. A broad set of window functions is accelerated, and any shape Comet does not handle falls back to Spark rather than producing an incorrect result. The expressions.md and operators.md pages already reflected this, so the guide and roadmap were the stale pages that disagreed with them.

What changes are included in this PR?

How are these changes tested?

Documentation-only change. Formatting was verified with prettier --check.

Window execution runs natively and is enabled by default, but the
compatibility guide and roadmap still described it as disabled by default
and known to be incorrect, citing the now-closed epic apache#2721. Update the
window docs to describe the current reality: which ranking, value, and
aggregate window functions are accelerated, which frame units and bounds
are handled, and the specific shapes that fall back to Spark. State the
unsupported aggregate input types positively (the Spark types Comet does
not accept) rather than by exclusion.
@andygrove andygrove marked this pull request as draft July 6, 2026 15:30
@andygrove andygrove marked this pull request as ready for review July 6, 2026 15:35
@andygrove

Copy link
Copy Markdown
Member Author

@comphead could you review this for accuracy?

Comment thread docs/source/contributor-guide/roadmap.md Outdated

@comphead comphead left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @andygrove
you mentioned #4766 but we also need to mentioned collect_list, collect_set is in progress

@andygrove

Copy link
Copy Markdown
Member Author

Thanks @andygrove you mentioned #4766 but we also need to mentioned collect_list, collect_set is in progress

Thanks. I added this.

@andygrove andygrove merged commit 15b5098 into apache:main Jul 6, 2026
15 checks passed
@andygrove andygrove deleted the docs-window-functions branch July 6, 2026 19:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants