Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,10 +61,15 @@ All notable changes to this project will be documented in this file. It uses the
* Binary driver now supports inserting into `Array(Nullable(T))` columns ([#316]).
* Added pushdown for the three-argument forms of `ltrim`, `rtrim`, and
`btrim` ([#307]).
* Added support for binary driver to `clickhouse_raw_query()` ([#309]).
* Added `clickhouse_query(server, sql)`, a set-returning function that runs a
query against a configured foreign server and returns its rows typed by the
caller's column definition list ([#309]).
* Added `clickhouse_perform(server, sql)`, a procedure that runs a statement
against a configured foreign server and discards any result, for statements
such as DDL returning no columns ([#329]).
* Deprecated `clickhouse_raw_query()`, which will be removed in next release.
Use `clickhouse_query(server, sql)` or `CALL clickhouse_perform(server, sql)`,
which use a foreign server rather than a connection string ([#329]).
* Added pushdown support for partial aggregates under partitionwise
aggregation, so a query over a partitioned table mixing local and foreign
partitions computes the foreign partition's aggregate on ClickHouse
Expand Down Expand Up @@ -170,6 +175,8 @@ All notable changes to this project will be documented in this file. It uses the
"ClickHouse/pg_clickhouse#319 Fix foreign scan RTE selection"
[#326]: https://github.com/ClickHouse/pg_clickhouse/pull/326
"ClickHouse/pg_clickhouse#326 pg-clickhouse-c"
[#329]: https://github.com/ClickHouse/pg_clickhouse/pull/329
"ClickHouse/pg_clickhouse#329 Deprecate clickhouse_raw_query, add clickhouse_perform"
[#330]: https://github.com/ClickHouse/pg_clickhouse/pull/330
"ClickHouse/pg_clickhouse#330 preserve typmod in binary driver"

Expand Down
3 changes: 1 addition & 2 deletions doc/offload-partition.sql
Original file line number Diff line number Diff line change
Expand Up @@ -245,8 +245,7 @@ BEGIN

ddl := pg_catalog.format('CREATE TABLE %s (%s) ENGINE = %s ORDER BY %s',
ch_table, coldefs, engine, order_by);
-- DDL yields no rows; column list is a formality clickhouse_query requires
PERFORM * FROM clickhouse_query(server, ddl) AS (ddl_result text);
CALL clickhouse_perform(server, ddl);
RETURN ddl;
END;
$create$;
46 changes: 39 additions & 7 deletions doc/pg_clickhouse.md
Original file line number Diff line number Diff line change
Expand Up @@ -963,7 +963,7 @@ to [BYTEA]. Example:

```sql
-- Create clickHouse table with String columns.
SELECT clickhouse_raw_query($$
CALL clickhouse_perform('ch_srv', $$
CREATE TABLE bytes (
c1 Int8, c2 String, c3 String
) ENGINE = MergeTree ORDER BY (c1);
Expand Down Expand Up @@ -1080,6 +1080,13 @@ These functions provide the interface to query a ClickHouse database.

#### `clickhouse_raw_query`

> [!WARNING]
> Deprecated: `clickhouse_raw_query()` emits a deprecation warning and will be
> removed in the next release. Use [`clickhouse_query`](#clickhouse_query) to
> read rows and [`clickhouse_perform`](#clickhouse_perform) to run statements
> that return none. Both reuse a configured server's driver, credentials,
> database, and connection cache instead of an ad-hoc connection string.

```sql
SELECT clickhouse_raw_query(
'CREATE TABLE t1 (x String) ENGINE = Memory',
Expand Down Expand Up @@ -1167,23 +1174,46 @@ SELECT * FROM clickhouse_query(

Execute a query against an already-configured foreign server and return its
rows as a relation, mapping each ClickHouse result column to the PostgreSQL type
named in the column definition list. Unlike [`clickhouse_raw_query`](#clickhouse_raw_query)
it reuses the server's `driver`, credentials, database, and the connection cache
rather than an ad-hoc connection string.
named in the column definition list. It reuses the server's `driver`,
credentials, database, and the connection cache.

The first argument is the name of a server created with [CREATE SERVER]. A
column definition list (`AS name(col type, ...)`) is required: PostgreSQL needs
the result shape before fetching rows, and it must match the columns the query
returns. Values are converted from ClickHouse to the declared types the same way
a foreign table column would be.
a foreign table column would be. Statements that return no columns, such as DDL,
have nothing to declare; run them with
[`clickhouse_perform`](#clickhouse_perform) instead.

As with `clickhouse_raw_query`, no role has `EXECUTE` access by default;
`GRANT` to a role to allow it to use the function.
No role has `EXECUTE` access by default; `GRANT` to a role to allow it to use
the function.

```sql
GRANT EXECUTE ON FUNCTION clickhouse_query(text, text) TO ch_admin;
```

#### `clickhouse_perform`

```sql
CALL clickhouse_perform(
'server',
'CREATE TABLE remote_table (id Int32) ENGINE = MergeTree ORDER BY id'
);
```

Execute a statement against an already-configured foreign server and discard
any result. Use it for statements that return no columns, such as DDL, where
[`clickhouse_query`](#clickhouse_query) has no result shape to declare. It
resolves the server the same way `clickhouse_query` does, reusing its `driver`,
credentials, database, and the connection cache.

As a procedure it must be invoked with [CALL], not `SELECT`, and it reports no
rows. No role has `EXECUTE` access by default:

```sql
GRANT EXECUTE ON PROCEDURE clickhouse_perform(text, text) TO ch_admin;
```

### Pushdown Functions

`pg_clickhouse` pushes down a subset of the PostgreSQL builtin functions used
Expand Down Expand Up @@ -1669,6 +1699,8 @@ Copyright (c) 2025-2026, ClickHouse.
"PostgreSQL Docs: DROP EXTENSION"
[CREATE SERVER]: https://www.postgresql.org/docs/current/sql-createserver.html
"PostgreSQL Docs: CREATE SERVER"
[CALL]: https://www.postgresql.org/docs/current/sql-call.html
"PostgreSQL Docs: CALL"
[ALTER SERVER]: https://www.postgresql.org/docs/current/sql-alterserver.html
"PostgreSQL Docs: ALTER SERVER"
[DROP SERVER]: https://www.postgresql.org/docs/current/sql-dropserver.html
Expand Down
6 changes: 3 additions & 3 deletions doc/tutorial.md
Original file line number Diff line number Diff line change
Expand Up @@ -477,12 +477,12 @@ Here's an excerpt from the CSV file you're using in table format. The
| 4 | Manhattan | Alphabet City | Yellow Zone |
| 5 | Staten Island | Arden Heights | Boro Zone |

1. Still in Postgres, use the `clickhouse_raw_query` function to create a
1. Still in Postgres, use the `clickhouse_perform` procedure to create a
ClickHouse [dictionary] named `taxi_zone_dictionary` and populate the
dictionary from the CSV file in S3:

```sql
SELECT clickhouse_raw_query($$
CALL clickhouse_perform('taxi_srv', $$
CREATE DICTIONARY taxi.taxi_zone_dictionary (
LocationID Int64 DEFAULT 0,
Borough String,
Expand All @@ -493,7 +493,7 @@ Here's an excerpt from the CSV file you're using in table format. The
SOURCE(HTTP(URL 'https://datasets-documentation.s3.eu-west-3.amazonaws.com/nyc-taxi/taxi_zone_lookup.csv' FORMAT 'CSVWithNames'))
LIFETIME(MIN 0 MAX 0)
LAYOUT(HASHED_ARRAY())
$$, 'host=localhost dbname=taxi');
$$);
```

> [!NOTE]
Expand Down
10 changes: 9 additions & 1 deletion sql/pg_clickhouse--0.3--0.4.sql
Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,13 @@ RETURNS SETOF record
AS 'MODULE_PATHNAME'
LANGUAGE C STRICT;

-- As with clickhouse_raw_query(), don't let PUBLIC run arbitrary remote queries.
-- Don't let PUBLIC run arbitrary remote queries.
REVOKE EXECUTE ON FUNCTION clickhouse_query(text, text) FROM PUBLIC;

-- Execute statement against foreign server, discarding any result, e.g.
-- CALL clickhouse_perform('srv', 'CREATE TABLE t (a Int32) ENGINE = Memory');
CREATE PROCEDURE clickhouse_perform(TEXT, TEXT)
AS 'MODULE_PATHNAME'
LANGUAGE C;

REVOKE EXECUTE ON PROCEDURE clickhouse_perform(text, text) FROM PUBLIC;
14 changes: 13 additions & 1 deletion sql/pg_clickhouse.sql
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,9 @@ RETURNS fdw_handler
AS 'MODULE_PATHNAME'
LANGUAGE C STRICT;

-- Deprecated, removed in the next release: use clickhouse_query() or
-- clickhouse_perform(), which reuse a configured server rather than an ad-hoc
-- connection string. Emits a deprecation warning when called.
CREATE FUNCTION clickhouse_raw_query(TEXT, TEXT DEFAULT 'host=localhost port=8123')
RETURNS TEXT
AS 'MODULE_PATHNAME'
Expand All @@ -36,9 +39,18 @@ RETURNS SETOF record
AS 'MODULE_PATHNAME'
LANGUAGE C STRICT;

-- As with clickhouse_raw_query(), don't let PUBLIC run arbitrary remote queries.
-- Make sure PUBLIC can't run arbitrary remote queries; roles must be granted
-- explicit access.
REVOKE EXECUTE ON FUNCTION clickhouse_query(text, text) FROM PUBLIC;

-- Execute statement against foreign server, discarding any result, e.g.
-- CALL clickhouse_perform('srv', 'CREATE TABLE t (a Int32) ENGINE = Memory');
CREATE PROCEDURE clickhouse_perform(TEXT, TEXT)
AS 'MODULE_PATHNAME'
LANGUAGE C;

REVOKE EXECUTE ON PROCEDURE clickhouse_perform(text, text) FROM PUBLIC;

CREATE FUNCTION clickhouse_fdw_validator(text[], oid)
RETURNS VOID
AS 'MODULE_PATHNAME'
Expand Down
52 changes: 51 additions & 1 deletion src/fdw.c
Original file line number Diff line number Diff line change
Expand Up @@ -184,6 +184,7 @@ typedef struct {
PG_FUNCTION_INFO_V1(clickhouse_fdw_handler);
PG_FUNCTION_INFO_V1(clickhouse_raw_query);
PG_FUNCTION_INFO_V1(clickhouse_query);
PG_FUNCTION_INFO_V1(clickhouse_perform);
PG_FUNCTION_INFO_V1(clickhouse_op_push_fail);
PG_FUNCTION_INFO_V1(clickhouse_push_fail);
PG_FUNCTION_INFO_V1(clickhouse_noop);
Expand Down Expand Up @@ -373,6 +374,16 @@ clickhouse_raw_query(PG_FUNCTION_ARGS) {
ch_connection conn;
text* res;

ereport(
WARNING,
errcode(ERRCODE_WARNING_DEPRECATED_FEATURE),
errmsg("pg_clickhouse: clickhouse_raw_query() is deprecated"),
errhint(
"Use clickhouse_query() or clickhouse_perform(); "
"clickhouse_raw_query() will be removed in the next release."
)
);

if (strcmp(details->driver, "http") == 0) {
conn = chfdw_http_connect(details);
} else if (strcmp(details->driver, "binary") == 0) {
Expand Down Expand Up @@ -430,7 +441,7 @@ clickhouse_server_version(PG_FUNCTION_ARGS) {
* Execute a query against a configured foreign server and return its rows,
* mapping ClickHouse values to the Postgres types named in the caller's column
* definition list. Reuses the server's driver, credentials and connection
* cache, unlike clickhouse_raw_query which takes an ad-hoc connection string.
* cache.
*/
Datum
clickhouse_query(PG_FUNCTION_ARGS) {
Expand Down Expand Up @@ -540,6 +551,45 @@ clickhouse_query(PG_FUNCTION_ARGS) {
return (Datum)0;
}

/*
* Execute a statement against a configured foreign server and discard any
* result. Procedure counterpart of clickhouse_query() for statements such as
* DDL, which return no columns to describe in a column definition list.
*/
Datum
clickhouse_perform(PG_FUNCTION_ARGS) {
char* server_name;
char* sql;
ForeignServer* server;
UserMapping* user;
ch_scan_connection sconn;
ch_cursor* cursor;

/* procedures cannot be declared STRICT, so reject nulls here */
if (PG_ARGISNULL(0) || PG_ARGISNULL(1)) {
ereport(
ERROR,
errcode(ERRCODE_NULL_VALUE_NOT_ALLOWED),
errmsg("pg_clickhouse: clickhouse_perform() arguments must not be null")
);
}

server_name = text_to_cstring(PG_GETARG_TEXT_P(0));
sql = text_to_cstring(PG_GETARG_TEXT_P(1));
server = GetForeignServerByName(server_name, false);
user = GetUserMapping(GetUserId(), server->serverid);
sconn = chfdw_get_scan_connection(user);

ch_query query = new_query(sql, 0, NULL, NULL, NULL);

/* deleting the cursor context drains the connection so it stays reusable */
cursor = sconn.gate.methods->simple_query(sconn.gate.conn, &query);
MemoryContextDelete(cursor->memcxt);
chfdw_release_scan_connection(user, sconn);

PG_RETURN_VOID();
}

/* calculate difference */
double
time_diff(struct timeval* prior, struct timeval* latter) {
Expand Down
6 changes: 2 additions & 4 deletions src/pglink.c
Original file line number Diff line number Diff line change
Expand Up @@ -980,11 +980,9 @@ binary_simple_query(void* conn, const ch_query* query) {

/*
* Validate declared shape before any per-column access. Empty attr_nums
* keeps the zero-attribute NULL sentinel handled at fetch time. Ignore
* columns_count == 0 (DDL) to support callers passing a placeholder
* column list since clickhouse_query() requires one syntactically.
* keeps the zero-attribute NULL sentinel handled at fetch time.
*/
if (query->tupdesc && query->attr_nums && cursor->columns_count > 0 &&
if (query->tupdesc && query->attr_nums &&
(size_t)list_length(query->attr_nums) != cursor->columns_count) {
ereport(
ERROR,
Expand Down
30 changes: 0 additions & 30 deletions test/expected/aggregates.out
Original file line number Diff line number Diff line change
@@ -1,34 +1,4 @@
\unset ECHO
clickhouse_raw_query
----------------------

(1 row)

clickhouse_raw_query
----------------------

(1 row)

clickhouse_raw_query
----------------------

(1 row)

clickhouse_raw_query
----------------------

(1 row)

clickhouse_raw_query
----------------------

(1 row)

clickhouse_raw_query
----------------------

(1 row)

Foreign table "agg_bin.agg_numbers"
Column | Type | Collation | Nullable | Default | FDW options
--------+----------+-----------+----------+---------+-------------
Expand Down
30 changes: 0 additions & 30 deletions test/expected/aggregates_1.out
Original file line number Diff line number Diff line change
@@ -1,34 +1,4 @@
\unset ECHO
clickhouse_raw_query
----------------------

(1 row)

clickhouse_raw_query
----------------------

(1 row)

clickhouse_raw_query
----------------------

(1 row)

clickhouse_raw_query
----------------------

(1 row)

clickhouse_raw_query
----------------------

(1 row)

clickhouse_raw_query
----------------------

(1 row)

Foreign table "agg_bin.agg_numbers"
Column | Type | Collation | Nullable | Default | FDW options
--------+----------+-----------+----------+---------+-------------
Expand Down
30 changes: 0 additions & 30 deletions test/expected/aggregates_2.out
Original file line number Diff line number Diff line change
@@ -1,34 +1,4 @@
\unset ECHO
clickhouse_raw_query
----------------------

(1 row)

clickhouse_raw_query
----------------------

(1 row)

clickhouse_raw_query
----------------------

(1 row)

clickhouse_raw_query
----------------------

(1 row)

clickhouse_raw_query
----------------------

(1 row)

clickhouse_raw_query
----------------------

(1 row)

Foreign table "agg_bin.agg_numbers"
Column | Type | Collation | Nullable | Default | FDW options
--------+----------+-----------+----------+---------+-------------
Expand Down
Loading
Loading