Skip to content

refactor(query-engine): split enable_topk into limiting vs formatting flags#391

Merged
milindsrivastava1997 merged 3 commits into
mainfrom
390-refactor-query-engine-split-enable_topk-into-limiting-vs-formatting-flags
Jun 6, 2026
Merged

refactor(query-engine): split enable_topk into limiting vs formatting flags#391
milindsrivastava1997 merged 3 commits into
mainfrom
390-refactor-query-engine-split-enable_topk-into-limiting-vs-formatting-flags

Conversation

@akanksha-akkihal

Copy link
Copy Markdown
Collaborator

Split enable_topk into limiting vs formatting flags

Background

Previously, enable_topk was responsible for two unrelated behaviors:

  • Limiting → using the sketch heap to select top-k results (sort + truncate)
  • Formatting → prepending the metric name (__name__) in PromQL-style output

This coupling worked for PromQL (which requires both behaviors), but it breaks SQL semantics.

SQL only needs top-k selection, not PromQL-style label formatting. With a single flag, it was impossible to express:

“Use heap-driven top-k selection without applying PromQL metric name formatting.”


Problem

The original design conflated two independent concerns:

  • How results are selected (ranking / top-k computation)
  • How results are represented (PromQL vs SQL output shape)

This caused SQL queries to unintentionally inherit PromQL formatting.


Solution: Split into two flags

We introduce two independent flags:

1. enable_topk_limiting

Controls heap-based top-k selection:

  • Uses sketch heap to identify top-k elements
  • Applies sorting + truncation logic

2. enable_topk_formatting

Controls PromQL-specific output formatting:

  • Prepends metric name (__name__)
  • Produces PromQL instant-vector style output

Resulting behavior

Query Type enable_topk_limiting enable_topk_formatting Behavior
PromQL true true Top-k + metric prefix formatting
SQL top-k true false Top-k only (clean rows)
SQL/Elastic (non-top-k) false false Standard execution

Subtle detail

Sorting remains enabled for Statistic::Topk regardless of formatting.

  • Ensures correct descending order before truncation
  • Prevents PromQL formatting logic from influencing SQL result structure

TL;DR

This change decouples:

  • How we select top-k rows (ranking logic)
    from
  • How results are formatted (PromQL vs SQL representation)

This separation aligns the execution model with the fundamental difference between SQL and PromQL semantics.

@milindsrivastava1997 milindsrivastava1997 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you check if there are tests that exercise PromQL's topk behavior? both in terms of the semantics, as well as the extra output label that is inserted into the result? If those tests are not there, could you please add them (use main as a source for adding them), and then test them against this branch?
Apart from that, looks good

@akanksha-akkihal

Copy link
Copy Markdown
Collaborator Author

Currently, there are no tests that exercise PromQL's topk behavior. Shall I add the test to the same PR?

@milindsrivastava1997 milindsrivastava1997 self-requested a review June 6, 2026 00:29
@milindsrivastava1997 milindsrivastava1997 merged commit ad99858 into main Jun 6, 2026
8 checks passed
@milindsrivastava1997 milindsrivastava1997 deleted the 390-refactor-query-engine-split-enable_topk-into-limiting-vs-formatting-flags branch June 6, 2026 00:31
akanksha-akkihal added a commit that referenced this pull request Jun 6, 2026
… flags (#391)

* split enable_topk into limiting vs formatting flags

* add PromQL topk pipeline and Prometheus wire format tests

* formatting
akanksha-akkihal added a commit that referenced this pull request Jun 6, 2026
… flags (#391)

* split enable_topk into limiting vs formatting flags

* add PromQL topk pipeline and Prometheus wire format tests

* formatting
milindsrivastava1997 pushed a commit that referenced this pull request Jun 6, 2026
* Support for SQL Top-K

* Fomatting

* refactor(query-engine): split enable_topk into limiting vs formatting flags (#391)

* split enable_topk into limiting vs formatting flags

* add PromQL topk pipeline and Prometheus wire format tests

* formatting

* Complete pipeline for SQL topK queries
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

refactor(query-engine): Split enable_topk into limiting vs formatting flags

2 participants