Skip to content

Push down JSONB existence predicates - #325

Open
k-bx wants to merge 3 commits into
ClickHouse:mainfrom
k-bx:agent/jsonb-exists-pushdown
Open

Push down JSONB existence predicates#325
k-bx wants to merge 3 commits into
ClickHouse:mainfrom
k-bx:agent/jsonb-exists-pushdown

Conversation

@k-bx

@k-bx k-bx commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Push PostgreSQL jsonb ? text predicates down for native ClickHouse JSON columns.
  • Preserve PostgreSQL top-level object-key and array-string semantics.
  • Support compatibility views that parse JSON documents stored in ClickHouse String columns, while leaving arbitrary JSON expressions local.
  • Handle nullable String-backed documents without producing unsupported nullable nested array types in ClickHouse.

Validation

  • make -j2
  • Focused PostgreSQL 18 regression test for native JSON, nullable String-backed JSON, combined predicates, aggregate pushdown, and local fallback
  • Tested on a real-world production canary system with PostgreSQL 18 and ClickHouse 26.3: EXPLAIN (FORMAT JSON, VERBOSE) confirmed that JSONHas, nullable-document handling, and count were all executed remotely; a month-range aggregate completed successfully, and a pushed single-row predicate matched a deliberately local PostgreSQL evaluation on the same replicated row

@k-bx
k-bx marked this pull request as ready for review July 24, 2026 07:29

@theory theory left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, looks handy, but the tests are insufficient. Can you improve them?

Comment thread test/sql/jsonb_exists.sql

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test does not properly validate execution on the ClickHouse side. Please create the corresponding tables and data in ClickHouse and execute the queries with and without EXPLAIN (VERBOSE COSTS OFF to show both the CH SQL generated and its successful execution.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in 9fa88ff.

The regression now creates the native JSON and Nullable(String)-backed tables in ClickHouse, inserts representative object, array, scalar, and NULL data, and runs each predicate both with EXPLAIN (VERBOSE, COSTS OFF) and as an executing query. The expected output includes the full generated ClickHouse SQL and returned rows for native JSON, String-backed compatibility views, combined predicates, and the intentionally local fallback.

Validation:

  • make -j2
  • Full focused regression against PostgreSQL 18 / ClickHouse 26.3 on an isolated Kharkiv test database
  • The same test update was applied to agent/v0.3.2-analytics-backports at 1869dcf

The PR branch was also rebased onto current main. GitHub Actions are currently awaiting maintainer approval.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@theory I want to add that this has been used in (relatively-heavy) production instances at my org for quite some time, ever since PR was opened

@theory theory self-assigned this Jul 29, 2026
@theory theory added pushdown Improvements to query pushdown operators Improve operator pushdown labels Jul 29, 2026
@k-bx
k-bx force-pushed the agent/jsonb-exists-pushdown branch from 15ce62a to 9fa88ff Compare July 31, 2026 16:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

operators Improve operator pushdown pushdown Improvements to query pushdown

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants