diff --git a/.cursor/skills/write-doc-examples/SKILL.md b/.cursor/skills/write-doc-examples/SKILL.md new file mode 100644 index 000000000..3ea7e125d --- /dev/null +++ b/.cursor/skills/write-doc-examples/SKILL.md @@ -0,0 +1,306 @@ +--- +name: write-doc-examples +description: Write, rewrite, and format documentation code snippets into reusable, production-ready methods with necessary library imports and clean linting. Use when writing, updating, or formatting Java code examples in integration guides and documentation (such as docs/integration-client.md or docs/integration-jdbc.md). +--- + +# Write Documentation Code Examples for Production Readiness + +This skill guides writing, rewriting, and formatting code snippets in documentation so they can be directly reused or integrated into production Java applications while maintaining stylistic consistency across the document. + +## Core Principles + +### 1. Preserve Document Context and Style Consistency +Always align with patterns and conventions established earlier in the same document: +- **Consistent Builder / Factory Pattern**: If early sections establish building a client via a method returning `Client.Builder` (e.g., `public Client.Builder createBaseClient()`), subsequent configuration options (auth schemes, TLS, proxy, timeouts) must follow the same pattern rather than reverting to creating full `Client` instances from scratch. +- **Incremental Context**: Sub-sections and options should build consistently on preceding examples (e.g., `public Client createAnalyticsDBClient(Client.Builder baseClient)`). +- **Naming Conventions**: Maintain consistent method, variable, and parameter names (`createBaseClient`, `client`, `schema`, `settings`, `events`) across all snippets in the document. + +### 2. Wrap Code with Methods +Never present bare, loose statements floating outside a method. Encapsulate every snippet into a realistic, reusable method: +- Use factory methods or builder helpers for client instantiation (e.g., `public Client.Builder createBaseClient()`, `public Client createAnalyticsDBClient(Client.Builder baseClient)`). +- Use action-specific methods for queries, inserts, and updates (e.g., `public List readEvents(Client client, TableSchema schema)`, `public void writeEvents(Client client, List events)`). +- Use clear parameter lists (`Client client`, `TableSchema schema`, `InsertSettings settings`, etc.) and meaningful return types instead of writing top-level procedural scripts. + +### 3. Include Only Relevant Library Imports +At the top of the code block, list only the imports that belong to the library: +- **Include**: Classes and interfaces from `com.clickhouse.client.api.*`, `com.clickhouse.data.*`, `com.clickhouse.jdbc.*`, etc. +- **Exclude**: Common standard JDK classes (e.g., `java.util.List`, `java.util.Map`, `java.io.InputStream`, `java.util.concurrent.TimeUnit`) unless needed to avoid ambiguity. +- Keep the import list compact and directly relevant to the snippet. + +### 4. Allow Partial Code (Omit Obvious Definitions) +Keep examples focused on library usage: +- **Omit obvious custom classes**: Auxiliary classes, configuration containers, or DTOs (e.g., `AppConfiguration`) do not need full definitions. +- **Include definitions only when structurally important**: Provide the class definition only when its internal fields or annotations are essential to demonstrating the library feature (e.g., showing how POJO fields map to ClickHouse column types). + +### 5. Ensure Code is Linted and Production-Grade +- **Resource Management**: Always use `try-with-resources` for closable resources such as `QueryResponse`, `InsertResponse`, and streams. +- **Compatibility**: Ensure code is valid Java 8+ and follows repository patterns. +- **Error & Edge-Case Handling**: Guard against empty inputs, handle or propagate checked exceptions properly, and include comments at extension points (e.g., `// add db specific configuration`). +- **Formatting**: Maintain consistent indentation (4 spaces), balanced braces, and valid Java syntax. + +--- + +## Transformation Workflow + +When updating documentation examples: + +1. **Scan Prior Context in the Document**: Check how earlier sections structure their examples (e.g., whether client configuration uses `Client.Builder` factory methods). +2. **Identify the Intent**: Determine whether the example demonstrates configuration, querying, inserting, streaming, or error handling. +3. **Encapsulate in a Reusable Method Matching the Document Style**: + - For client creation/options: Return `Client.Builder` or take `Client.Builder baseClient` if established earlier in the guide. + - For operations: Accept `Client` (and any required schemas or options) as parameters. + - For callbacks/streaming: Pass inputs and manage the response lifecycle properly. +4. **Collect Library Imports**: Add all `com.clickhouse.*` imports required by the snippet at the top. +5. **Prune Unnecessary Boilerplate**: Strip out trivial DTO class definitions, keeping only structural POJO models where column mapping is highlighted. +6. **Lint and Format**: Check method signatures, variable types, semicolons, and `try-with-resources` blocks. + +--- + +## Transformation Examples + +### Example 1: Client Configuration & Instantiation + +**Before (Loose snippet):** +```java +Client client = new Client.Builder() + .addEndpoint("http://localhost:8123") + .setUsername("default") + .setPassword("secret") + .setDefaultDatabase("analytics") + .build(); +``` + +**After (Reusable production methods with library imports):** +```java +import com.clickhouse.client.api.Client; + +public Client.Builder createBaseClient() { + return new Client.Builder() + .addEndpoint("http://localhost:8123") + .setUsername("default") + .setPassword("secret") + // set common configuration + ; +} + +public Client createAnalyticsDBClient(Client.Builder baseClient) { + return baseClient + .setDefaultDatabase("analytics") + // add db specific configuration + .build(); +} +``` + +--- + +### Example 2: Following Established Document Style in Configuration Variants + +When earlier sections establish `createBaseClient()` returning `Client.Builder`, all variant auth/network options maintain that same style: + +**Option B (Bearer Auth):** +```java +import com.clickhouse.client.api.Client; + +public Client.Builder createBaseClient() { + return new Client.Builder() + .addEndpoint("http://localhost:8123") + .useBearerTokenAuth("my_access_token"); +} +``` + +**Option C (Mutual TLS):** +```java +import com.clickhouse.client.api.Client; +import com.clickhouse.client.api.enums.SSLMode; + +public Client.Builder createBaseClient() { + return new Client.Builder() + .addEndpoint("https://localhost:8443") + .useSSLAuthentication(true) + .setClientCertificate("/path/to/client.crt") + .setClientKey("/path/to/client.key") + .setRootCertificate("/path/to/ca.crt") + .setSSLMode(SSLMode.STRICT); +} +``` + +--- + +### Example 3: Runtime Operations with External Configuration + +**Before (Loose statement):** +```java +client.updateUserAndPassword("new_user", "new_password"); +``` + +**After (Wrapped method; obvious custom config class omitted):** +```java +void updateClientCredentials(Client client, AppConfiguration appConf) { + client.updateUserAndPassword(appConf.db_username, appConf.db_password); +} +``` + +--- + +### Example 4: POJO Mapping (Registration, Read, Write) + +**Before (Script-like sequence):** +```java +TableSchema schema = client.getTableSchema("events"); +client.register(Event.class, schema); + +List events = client.queryAll("SELECT * FROM events", Event.class, schema); +client.insert("events", events).get(); +``` + +**After (Structured into definition, registration, read, and write methods):** + +1. Structural POJO definition (included because field structure matters for column mapping): +```java +public static class Event { + public long id; + public String name; + public long timestamp; +} +``` + +2. Registration helper: +```java +import com.clickhouse.client.api.Client; +import com.clickhouse.client.api.metadata.TableSchema; + +void registerPojoMappings(Client client, Map, String> pojoTables) { + for (Map.Entry, String> entry : pojoTables.entrySet()) { + TableSchema schema = client.getTableSchema(entry.getValue()); + client.register(entry.getKey(), schema); + } +} +``` + +3. Read method: +```java +import com.clickhouse.client.api.Client; +import com.clickhouse.client.api.metadata.TableSchema; + +public List readEvents(Client client, TableSchema schema) { + return client.queryAll( + "SELECT id, name, timestamp FROM events", + Event.class, + schema); +} +``` + +4. Write method with response cleanup: +```java +import com.clickhouse.client.api.Client; +import com.clickhouse.client.api.insert.InsertResponse; + +public void writeEvents(Client client, List events) throws Exception { + if (events.isEmpty()) { + return; + } + + try (InsertResponse response = client.insert("events", events).get()) { + // handle response metrics or confirmation + } +} +``` + +--- + +### Example 5: Streaming Query with Binary Format Reader + +**Before (Procedural script):** +```java +QuerySettings settings = new QuerySettings() + .setFormat(ClickHouseFormat.RowBinaryWithNamesAndTypes); + +QueryResponse response = client.query("SELECT * FROM events", settings).get(); +ClickHouseBinaryFormatReader reader = client.newBinaryFormatReader(response); +while (reader.hasNext()) { + reader.next(); + long id = reader.getLong("id"); +} +``` + +**After (Encapsulated method with try-with-resources and library imports):** +```java +import com.clickhouse.client.api.Client; +import com.clickhouse.client.api.data_formats.ClickHouseBinaryFormatReader; +import com.clickhouse.client.api.query.QueryResponse; +import com.clickhouse.client.api.query.QuerySettings; +import com.clickhouse.data.ClickHouseFormat; + +public void streamEvents(Client client) throws Exception { + QuerySettings settings = new QuerySettings() + .setFormat(ClickHouseFormat.RowBinaryWithNamesAndTypes); + + try (QueryResponse response = client.query("SELECT * FROM events", settings) + .get(30, TimeUnit.SECONDS)) { + + ClickHouseBinaryFormatReader reader = client.newBinaryFormatReader(response); + while (reader.hasNext()) { + reader.next(); + long id = reader.getLong("id"); + String name = reader.getString("name"); + // process row data + } + } +} +``` + +--- + +### Example 6: Streaming Insert with Callback Writer + +**Before (Loose insert callback):** +```java +TableSchema schema = client.getTableSchema("events"); +ClickHouseFormat format = ClickHouseFormat.RowBinary; + +client.insert("events", out -> { + RowBinaryFormatWriter writer = new RowBinaryFormatWriter(out, schema, format); + for (Event event : events) { + writer.setValue("id", event.getId()); + writer.commitRow(); + } +}, format, new InsertSettings()).get(); +``` + +**After (Encapsulated write method with proper response closure):** +```java +import com.clickhouse.client.api.Client; +import com.clickhouse.client.api.data_formats.RowBinaryFormatWriter; +import com.clickhouse.client.api.insert.InsertResponse; +import com.clickhouse.client.api.insert.InsertSettings; +import com.clickhouse.client.api.metadata.TableSchema; +import com.clickhouse.data.ClickHouseFormat; + +public void writeEventsStream(Client client, TableSchema schema, List events) throws Exception { + ClickHouseFormat format = ClickHouseFormat.RowBinary; + + try (InsertResponse response = client.insert("events", out -> { + RowBinaryFormatWriter writer = new RowBinaryFormatWriter(out, schema, format); + for (Event event : events) { + writer.setValue("id", event.id); + writer.setValue("name", event.name); + writer.commitRow(); + } + }, format, new InsertSettings()).get()) { + // handle response metrics + } +} +``` + +--- + +## Validation Checklist + +Before finalizing any rewritten documentation example: +- [ ] Does the example follow the structural and naming style established in earlier sections of the document (e.g. `Client.Builder` return type)? +- [ ] Is every code snippet wrapped in a meaningful method (or a builder helper)? +- [ ] Are all library imports (`com.clickhouse.*`) present and accurate? +- [ ] Are redundant JDK imports omitted unless strictly helpful? +- [ ] Are trivial custom classes omitted and only essential structures (e.g. POJO schema mappings) defined? +- [ ] Are closable resources (`QueryResponse`, `InsertResponse`, etc.) properly handled with `try-with-resources`? +- [ ] Is the code syntactically valid and lint-clean? diff --git a/docs/integration-client.md b/docs/integration-client.md new file mode 100644 index 000000000..05df55e08 --- /dev/null +++ b/docs/integration-client.md @@ -0,0 +1,1021 @@ +# ClickHouse Java Client Integration Guide + +This guide is a **step-by-step, end-to-end integration path** for the **Java Client V2** (`client-v2`). It is written to be used as context for building an application or a downstream integration spec: each step states the decisions you must make, how to configure them, and the common pitfalls to avoid. It is self-contained — you can work through it from the empty project to a running read/write path without other prerequisites. + +> **Configuration philosophy.** This guide names only the properties relevant to each step. It does not repeat the exhaustive property list — that lives in [`ClientConfigProperties`](../client-v2/src/main/java/com/clickhouse/client/api/ClientConfigProperties.java) and the official docs. Configuration splits into two groups: +> - **Init configuration** — set once when the client is built: endpoint, connection pool size, async mode, authentication. Covered in Steps 1–4. +> - **Operation configuration** — set per request or as client defaults: formats, buffer sizes, timeouts, retries, dedup tokens. Covered in Steps 5–7. + +## Artifacts + +The client is published to Maven Central as **`com.clickhouse:client-v2`**. Browse versions and copy a ready-made dependency snippet for any build system (Maven, Gradle, sbt, Ivy, ...) from the [Maven Central page](https://central.sonatype.com/artifact/com.clickhouse/client-v2). + +Two distributions are published under the same artifact: + +- **Standard artifact** (default, no classifier) — the client together with its dependencies declared as ordinary transitive Maven dependencies. Recommended for managed builds in which the application controls the dependency tree. +- **Shaded artifact** (`all` classifier) — a single self-contained archive that bundles and **relocates** most third-party dependencies (Apache HttpClient, LZ4, RoaringBitmap, ASM, and others) under `com.clickhouse.shaded.*`. Recommended when transitive dependencies cannot be managed, such as self-contained deployment archives or standalone tooling. + +**Dependency conflicts to anticipate.** The standard artifact introduces libraries that the application may already depend on at different versions — notably **Guava**, **Apache HttpClient 5**, and **commons-compress**. If the application declares incompatible versions, `NoSuchMethodError` or `LinkageError` may occur at runtime. Two approaches resolve this: + +- Use the shaded artifact so that these dependencies are relocated and cannot collide. Note that `slf4j` and `micrometer` are intentionally left unshaded, so logging and metrics continue to bind to the application's own implementations. +- Alternatively, use the standard artifact and reconcile versions explicitly through dependency management or ``. + +--- + +## Integration path at a glance + +Work through these steps in order. Each one is a decision point; the "Common Pitfalls" notes describe the consequences of skipping it. + +| # | Step | Core decision | +|---|-----------|---------------| +| 1 | [Instantiation](#step-1--instantiation) | Client lifecycle, pool sizing, and workload identification | +| 2 | [Authentication](#step-2--authentication) | Which auth mechanism and how to configure it | +| 3 | [Transport & connectivity](#step-3--transport--connectivity-tls-proxies-timeouts) | TLS/mTLS, proxies, timeouts, health checks | +| 4 | [Connections Configuration](#step-4--connections-configuration) | Pool sizing, async vs sync, sessions | +| 5 | [Data formats, readers & writers](#step-5--data-formats-readers--writers) | Which wire format and reader/writer to use | +| 6 | [Read operations & tuning](#step-6--read-operations--tuning) | Streaming vs materializing; heavy-read tuning; read errors | +| 7 | [Write operations & tuning](#step-7--write-operations--tuning) | Insert pattern; heavy-ingest tuning; idempotency; write errors | +| 8 | [Metadata & schema discovery](#step-8--metadata--schema-discovery) | How to obtain schemas without JDBC metadata | +| 9 | [Miscellaneous features](#step-9--miscellaneous-features) | Sessions and other optional capabilities | + +--- + +## Step 1 — Instantiation + +**Goal:** decide how many `Client` instances exist, how long they live, and how the internal connection pool is sized. + +### What a `Client` is + +The [`Client`](https://javadoc.io/doc/com.clickhouse/client-v2/latest/com/clickhouse/client/api/Client.html) is the single entry point for all operations. It owns: + +- An HTTP connection pool (Apache HttpClient) +- Endpoint configuration and retry policy +- A table schema cache +- The POJO serialization/deserialization registry +- Optional client-wide session and settings defaults + +```java +import com.clickhouse.client.api.Client; + +public Client.Builder createBaseClient() { + return new Client.Builder() + .addEndpoint("http://localhost:8123") + .setUsername("default") + .setPassword("secret") + .setClientName("order-service/1.2.0") + // set common configuration + ; +} + +public Client createAnalyticsDBClient(Client.Builder baseClient) { + return baseClient + .setDefaultDatabase("analytics") + // add db specific configuration + .build(); +} +``` + +### Instantiation Strategy + +- **Share a single instance:** A single, shared `Client` instance suits most use cases. It is thread-safe and designed to be reused across your application. +- **Long-lived lifecycle:** Build the client once at startup and close it once at shutdown. Creating a new client per request or operation is an anti-pattern because initialization takes time to set up internal structures (like the connection pool and schema cache), which adds latency to your requests. +- **Serverless functions:** For serverless environments (like AWS Lambda), initialize the client outside the function handler so it can be reused across invocations. +- **Warm-up (optional):** Calling `client.ping()` at startup can help initialize the connectivity part and verify the endpoint before serving live traffic, though it is not strictly required. It may also require to wakeup cloud instance. +- **Caching:** The application is responsible for holding the reference to the `Client` instance (e.g., via dependency injection or a singleton). The library does not provide a global static cache. + +### Workload identification & client name + +In production environments, a single ClickHouse cluster often handles diverse workloads from multiple services: real-time user-facing APIs, event ingestion pipelines (e.g., Kafka consumers, CDC streams), scheduled ETL batch jobs, BI reporting dashboards (e.g., Superset, Tableau, Grafana), and ad-hoc analytical queries. When queries fail, time out, or cause memory spikes (`MEMORY_LIMIT_EXCEEDED`), identifying the originating application or workload is essential for fast troubleshooting, root-cause analysis, and resource attribution. + +#### Setting client name + +Use a structured format such as `/` or `:/` (for example, `order-service/1.2.0` or `etl-worker:cdc/2.0.1`). + +**1. Client Builder** (static setup at client creation): + +```java +import com.clickhouse.client.api.Client; + +public Client.Builder createBaseClient() { + return new Client.Builder() + .addEndpoint("http://localhost:8123") + .setUsername("default") + .setPassword("secret") + .setClientName("order-service/1.2.0"); +} +``` + +**2. Dynamic Runtime Update** (updating identity on an existing client): + +```java +import com.clickhouse.client.api.Client; + +public void updateClientWorkload(Client client, String workloadOrTenant) { + // Dynamically update workload identifier at runtime + client.updateClientName("order-service:" + workloadOrTenant + "/1.2.0"); +} +``` + +#### How it is observed on the server (`User-Agent` header) + +The Java Client communicates over HTTP and passes the client name as the leading segment of the HTTP `User-Agent` header. The client automatically appends the client library version, operating system, JVM version, and underlying HTTP client details: + +```text +order-service/1.2.0 clickhouse-java-v2/0.9.6 (Linux; jvm:17.0.2) Apache-HttpClient/5.4.4 +``` + +> **CRITICAL SERVER OBSERVATION NOTE:** +> In ClickHouse's `system.query_log` and `system.processes`, HTTP requests record this information in the **`http_user_agent`** column. The `client_name` column in `system.query_log` is populated **only** for native TCP protocol connections. Always query `http_user_agent` when troubleshooting Java Client applications. + +#### Finding workloads in `system.query_log` + +Use the following queries on ClickHouse to troubleshoot and monitor application workloads: + +**Find recent queries and execution metrics for a specific application:** +```sql +SELECT + event_time, + query_id, + query_duration_ms, + memory_usage, + read_rows, + read_bytes, + result_rows, + http_user_agent, + query +FROM system.query_log +WHERE type = 'QueryFinish' + AND http_user_agent LIKE '%order-service%' + AND event_time >= now() - INTERVAL 1 HOUR +ORDER BY event_time DESC +LIMIT 100; +``` + +**Find failed queries and exceptions for a workload:** +```sql +SELECT + event_time, + query_id, + exception_code, + exception, + http_user_agent, + query +FROM system.query_log +WHERE type = 'ExceptionWhileProcessing' + AND http_user_agent LIKE '%order-service%' + AND event_time >= now() - INTERVAL 24 HOUR +ORDER BY event_time DESC +LIMIT 50; +``` + +**Aggregate workload resource consumption across all applications:** +```sql +SELECT + extract(http_user_agent, '^([^ ]+)') AS workload, + count() AS query_count, + round(avg(query_duration_ms), 2) AS avg_duration_ms, + round(quantile(0.95)(query_duration_ms), 2) AS p95_duration_ms, + round(max(query_duration_ms), 2) AS max_duration_ms, + formatReadableSize(sum(memory_usage)) AS total_memory, + formatReadableQuantity(sum(read_rows)) AS total_read_rows, + countIf(type = 'ExceptionWhileProcessing') AS error_count +FROM system.query_log +WHERE event_time >= now() - INTERVAL 24 HOUR + AND type IN ('QueryFinish', 'ExceptionWhileProcessing') +GROUP BY workload +ORDER BY query_count DESC; +``` + +**Inspect active running queries (`system.processes`):** +```sql +SELECT + query_id, + elapsed, + memory_usage, + http_user_agent, + query +FROM system.processes +WHERE http_user_agent LIKE '%order-service%'; +``` + +**Query across a cluster:** +```sql +SELECT + hostName() AS host, + event_time, + query_id, + query_duration_ms, + memory_usage, + http_user_agent, + query +FROM clusterAllReplicas('default', system.query_log) +WHERE type = 'QueryFinish' + AND http_user_agent LIKE '%order-service%' + AND event_time >= now() - INTERVAL 1 HOUR +ORDER BY event_time DESC +LIMIT 100; +``` + + +## Step 2 — Authentication + +**Goal:** configure the authentication mechanism the ClickHouse deployment requires. The mechanism is dictated by the server and any fronting infrastructure, not chosen freely; the task is to identify it and configure it correctly. + +> **CONSTRAINT:** Configure exactly one mechanism. Different method cannot be mixed to avoid configuration errors. + +### Option A — Basic (username + password) + +The standard mechanism. Default installs ship a `default` user with no password. + +```java +import com.clickhouse.client.api.Client; + +public Client.Builder createBaseClient() { + return new Client.Builder() + .addEndpoint("http://localhost:8123") + .setUsername("default") + .setPassword("secret") + ; +} +``` + +Rotate credentials at runtime (thread-safe, non-blocking, applies to newly started requests): + +```java +void updateClientCredentials(AppConfiguration appConf) { + client.updateUserAndPassword(appConf.db_username, appConf.db_password); +} +``` + +**Note**: realtime credentials update would work well with runtime configuration update but would not work for multi-tenant setup. Multi tenant application should organize exclusive access to client +while handling tenant operation to avoid cross-talk problem. Separate client instance per tenant must be used when each tenant has own database. + +### Option B — Token / bearer + +For the standard `Authorization: Bearer ` scheme use `useBearerTokenAuth(...)` (the `Bearer ` prefix is added for you): + +```java +import com.clickhouse.client.api.Client; + +public Client.Builder createBaseClient() { + return new Client.Builder() + .addEndpoint("http://localhost:8123") + .useBearerTokenAuth("my_access_token"); +} + +``` + +For a non-`Bearer` scheme, use `setAccessToken(...)` — the value is sent verbatim, so include the scheme yourself. Runtime updates: `updateBearerToken(...)` (adds prefix) and `updateAccessToken(...)` (verbatim). + +**Note**: realtime credentials update would work well with runtime configuration update but would not work for multi-tenant setup. Multi tenant application should organize exclusive access to client +while handling tenant operation to avoid cross-talk problem. Separate client instance per tenant must be used when each tenant has own database. + +### Option C — Mutual TLS (client certificate) + +```java +import com.clickhouse.client.api.Client; +import com.clickhouse.client.api.enums.SSLMode; + +public Client.Builder createBaseClient() { + return new Client.Builder() + .addEndpoint("https://localhost:8443") + .useSSLAuthentication(true) + .setClientCertificate("/path/to/client.crt") + .setClientKey("/path/to/client.key") + .setRootCertificate("/path/to/ca.crt") // if the server cert is self-signed + .setSSLMode(SSLMode.STRICT) // STRICT (default), VERIFY_CA, TRUST, or DISABLED + ; +} + +``` + +Alternatively configure a trust store with `setSSLTrustStore(...)`, `setSSLTrustStorePassword(...)`, `setSSLTrustStoreType(...)`. + +### Option D — Custom HTTP headers (proxies / gateways) + +For OAuth gateways or custom handler configurations, inject arbitrary headers: + +```java +import com.clickhouse.client.api.Client; + +public Client.Builder createBaseClient() { + return new Client.Builder() + .addEndpoint("http://localhost:8123") + .httpHeader("X-API-Key", "my_custom_api_key"); +} + +``` + +### Option E — Proxy credentials + +If you connect to ClickHouse through an HTTP proxy that requires authentication, you can provide proxy credentials. This is configured alongside the proxy connection details and operates independently of the database authentication mechanism you chose above: + +```java +import com.clickhouse.client.api.Client; +import com.clickhouse.client.api.enums.ProxyType; + +public Client.Builder createBaseClient() { + return new Client.Builder() + .addEndpoint("http://localhost:8123") + // Database credentials (e.g. Option A) + .setUsername("default") + .setPassword("secret") + // Proxy configuration and credentials + .addProxy(ProxyType.HTTP, "proxy.example.com", 8080) + .setProxyCredentials("proxy_user", "proxy_password"); +} +``` + +### Client Identity & Default Database + +Regardless of the authentication mechanism, you can also configure the client's identity and default database: + +```java +import com.clickhouse.client.api.Client; + +public Client createAnalyticsClient(Client.Builder baseClient) { + return baseClient + .setClientName("my-analytics-app/1.0") + .setDefaultDatabase("analytics") // Default database for queries + .build(); +} +``` + +> **Note on Client Name:** How the client name surfaces in `system.query_log` depends on the protocol used. For HTTP connections (used by Java Client V2), it appears in the `http_user_agent` column. The `client_name` column in `system.query_log` is populated only for native TCP connections. See [Workload identification & client name](#workload-identification--client-name) for full details and query log troubleshooting queries. + +### Identifying the required mechanism + +The mechanism follows from how the server and any fronting infrastructure are configured: + +| Deployment | Required mechanism | +|------------|--------------------| +| Native ClickHouse users with passwords | Basic — username + password (Option A) | +| Bearer/token gateway or ClickHouse Cloud token | Token / bearer (Option B) | +| Certificate-based (zero-trust) access | Mutual TLS (Option C) | +| API gateway / Identity Aware Proxy in front of ClickHouse | Custom HTTP headers (Option D) | +| HTTP forward proxy requiring authentication | Proxy credentials (Option E) | + +Runtime rotation via `updateUserAndPassword` / `updateBearerToken` updates the credentials of the **already-selected** mechanism; it throws `ClientMisconfigurationException` rather than switching to a different mechanism. + +--- + +## Step 3 — Transport & connectivity (TLS, proxies, timeouts) + +**Goal:** Configure the client to match the application's networking and infrastructure environment. Because the application determines these requirements, you must understand the physical and logical topology between the client and the server. The client relies on this initialization configuration to establish secure connectivity (TLS, proxies) and gracefully handle network conditions. + +### TLS / mTLS / proxies + +| Scenario | Builder methods / properties | +|----------|------------------------------| +| HTTPS with public CA | `addEndpoint("https://host:8443")` | +| Self-signed server cert | `setRootCertificate("/path/to/ca.crt")` | +| mTLS (client certificate) | `useSSLAuthentication(true)`, `setClientCertificate(...)`, `setClientKey(...)` | +| Trust store (JKS/PKCS12) | `setSSLTrustStore(...)`, `setSSLTrustStorePassword(...)` | +| HTTP proxy | `setProxy(ProxyType.HTTP, host, port)`, `setProxyCredentials(user, password)` | + +See [SSLExamples](../examples/client-v2/src/main/java/com/clickhouse/examples/client_v2/SSLExamples.java) for a runnable walkthrough and [authentication.md](authentication.md) for full details. + +### Init configuration — timeouts + +Timeouts are critical parameters that directly impact application stability under load and over long distances: + +- **Connection timeout** (`.setConnectTimeout()`): The TCP connect timeout. Setting this value too low can cause failures when the application and server are in different geographical regions. Additionally, connection timeouts are closely tied to the connection pool: if the application issues concurrent requests that exceed the available pool size, it may manifest as a connection timeout because no free connections are present. +- **Socket timeout** (`.setSocketTimeout()`): The timeout for underlying socket read/write operations. While it applies strictly to socket activity, it is vital because it dictates how long the client will wait for long-running queries to return data. If your workload involves heavy analytical queries, you may need a very long socket timeout. However, the trade-off of a long socket timeout is the increased risk of encountering stale or silently dropped connections. +- **TCP keepalive**: Can be enabled to mitigate stale connections, though the host operating system's settings may ultimately override it. System-level TCP keepalive defaults are often several hours; configuring a shorter keepalive period makes sense for long-running operations. Keep in mind that executing extremely long operations over the public internet remains inherently risky. + +> **Note on runtime configuration:** You can optionally override the default network timeout on a per-operation basis using `QuerySettings.setNetworkTimeout(long timeout, ChronoUnit unit)`. This allows you to set stricter boundaries on specific queries without altering the client-wide defaults. + +### Ping - check connectivity + +Checking connectivity is a critical operation in the application lifecycle. The `client.ping()` method provides a lightweight way to verify that the ClickHouse server is reachable and responsive. + +```java +import com.clickhouse.client.api.Client; + +public boolean checkConnectivity(Client client) { + if (!client.ping()) { + // trigger recovery logic, mark service unhealthy, or fail fast + return false; + } + return true; +} +``` + +Key use cases include: +- **Application health checks**: Wire this method to your application's liveness or readiness probes (e.g., in Kubernetes or behind a load balancer) to automatically route traffic away from the application if the database connection is lost. +- **Handling recovery**: Use it in circuit breakers, reconnection logic, or recovery loops to verify that connectivity has been restored before resuming bulk operations or restarting failed data pipelines. + +### Common Pitfalls + +> **Timeouts too aggressive** for heavy analytical queries cause spurious failures — align `socket_timeout` with expected query duration or use per-operation network timeouts. +> +> **Proxy credentials omitted** on authenticated proxies produce opaque connection failures. +--- + +## Step 4 — Connections Configuration + +In the Java Client a "connection" is an **HTTP connection borrowed from the internal pool**, not a long-lived database session. Each operation borrows a connection, sends a request, streams the response, and returns the connection to the pool. + +### Connection limit (`max_open_connections`) + +The pool size depends on your workload — specifically on its **concurrency**, not on how much data it moves. What matters is **how many operations run at the same time**, not the number of rows or bytes any single operation transfers. A pool of 20 connections serves at most 20 simultaneous operations regardless of whether each returns one row or a million. This is the single setting you actually tune. The table below will help to estimate rough number. Having slightly bigger number than actualy needed is not a problem because unused connections will be garbage collected. It is recommended to perform a load testing with one application instance to detect if estimated number works. +Connection limit may acts as a backpreasure for incomming requests if they get blocked by DB access. When request backlog grows it may also slowdowns whole application so it is very important to find a balance between concurrent operations and their execution time. Be aware that in most applications allocated memory is freed only at the end of request. + + +| Workload | Concurrent requests | Suggested `max_open_connections` | What happens | +|----------|---------------------|----------------------------------|--------------| +| **Short-lived reads, low concurrency** | a few per second | **10–20** | Enough for the traffic; under bursts a few requests briefly **wait for a connection** to be returned to the pool, which is acceptable. | +| **Short-lived reads, higher concurrency** | tens+ per second | **20–100** | More parallel operations need more connections. Connections are **reused many times**, so `connection_reuse_strategy` and `connection_ttl` start to matter. | +| **Long-running operations** (large reads/writes, streaming) | any | **no fixed rule** | A connection is held for the whole operation and rarely reused, so pool sizing matters less. Size to the number of concurrent long operations and focus on **data-transfer tuning** (Steps 6–7) instead. | + +The reasoning is only valid for **short-lived read operations**, where connections cycle back to the pool quickly enough to be shared. For long operations the bottleneck is data transfer, not connection availability. + +### Connection pool + +These are the only pool-related properties you normally touch; the rest have safe defaults. + +| Property (Builder method) | Purpose | Default | +|---------------------------|---------|---------| +| `max_open_connections` (`.setMaxConnections()`) | Pool size — set from the parallelism guidance above | 10 | +| `connection_pool_enabled` (`.enableConnectionPool()`) | Enable/disable pooling (keep enabled) | true | +| `connection_ttl` (`.setConnectionTTL()`) | Max lifetime of a pooled connection | — | +| `connection_reuse_strategy` (`.setConnectionReuseStrategy()`) | FIFO or LIFO reuse | — | + +**`connection_ttl` against ClickHouse Cloud.** Keep it **relatively small** when the endpoint is a Cloud (or otherwise load-balanced) deployment. A short TTL forces connections to be retired and re-established frequently, so new connections keep going through the load balancer, which lets it **redistribute traffic across nodes** instead of pinning long-lived connections to whichever node they first landed on. + +**`connection_pool_enabled` is not recommended to change.** Leave pooling **enabled** (the default). The only reason to disable it is a workload with extreme concurrency where the internal Apache HttpClient connection pool itself becomes a contention point — under very high parallel request rates the pool's own bookkeeping can serialize threads. Disabling pooling avoids that bottleneck at the cost of a fresh connection per operation (higher latency, more sockets), so treat it as a last resort after measuring, not a default. + +### Common Pitfalls + +> **CONSTRAINT:** Do not create a `Client` per request. It destroys pool warm-up and adds latency on every call — the single most common mistake. +> +> **Pool too small** (`max_open_connections`) throttles concurrency; size it to peak concurrent operations, not average. +> +> **CONSTRAINT:** Always `close()` the client at shutdown to avoid leaking the pool and its threads. +--- + +## Step 5 — Data formats, readers & writers + +All formats are defined in [`ClickHouseFormat`](../clickhouse-data/src/main/java/com/clickhouse/data/ClickHouseFormat.java); each declares whether it supports input, output, binary encoding, headers, and row layout. + +The client provides support for the `RowBinary` and `Native` format families through built-in readers and writers (see the list below). Users can also implement their own readers and writers, as the client provides direct access to the underlying `InputStream` and `OutputStream`. + +We recommend using libraries like [Jackson](https://github.com/FasterXML/jackson) for processing text formats such as `JSON` or `CSV`. Any library capable of reading from a standard Java `InputStream` will work seamlessly. + +**Readers** + +Select with `QuerySettings.setFormat(format)` and consume the response stream: + +| Reader class | Formats | +|--------------|---------| +| [`NativeFormatReader`](../client-v2/src/main/java/com/clickhouse/client/api/data_formats/NativeFormatReader.java) | `Native` | +| [`RowBinaryFormatReader`](../client-v2/src/main/java/com/clickhouse/client/api/data_formats/RowBinaryFormatReader.java) | `RowBinary` variants | + +**Writers** + +| Writer class | Use | +|--------------|-----| +| [`RowBinaryFormatWriter`](../client-v2/src/main/java/com/clickhouse/client/api/data_formats/RowBinaryFormatWriter.java) | Row-oriented binary insert | + + +**Choosing a format** + +There is no silver bullet when choosing a data format—the best choice depends entirely on your data's origin, destination, and processing constraints. Client implements support of some ClickHouse formats. +Text format like JSON will work with client but require using 3rd-party library to parse stream. + +Only `RowBinaryWithDefaults` is supported for write operations from the list of ClickHouse own formats. + +Consider these trade-offs: +- **Binary formats (e.g., `RowBinary`, `Native`):** Highly compact and CPU-efficient. Best for general-purpose, high-throughput data transfer where you are mapping Java objects directly to ClickHouse rows. +- **Text formats (e.g., `JSONEachRow`, `CSV`):** Ideal for zero-copy passthrough. If your application receives newline-delimited JSON from an upstream service, passing `JSONEachRow` directly to ClickHouse avoids the CPU overhead of parsing JSON into Java objects only to serialize them back into a binary format. +- **Columnar formats (e.g., `Parquet`, `Arrow`):** Excellent for bulk data exports or interoperability with other analytical systems. + +Always pick the format that minimizes unnecessary transcoding in your application layer. + +## Step 6 — Read operations & tuning + +**Goal:** read results efficiently and configure the operation-level settings for heavy analytical reads. + +### Query methods overview + +The `Client` interface offers multiple ways to read data, allowing you to balance memory constraints with data mapping requirements: + +1. **`queryAll(...)`**: Materializes the entire result set into memory as a `List` or a list of POJOs. This is the simplest approach but should **only be used for small datasets**. For large result sets, it will consume significant heap space and risk an `OutOfMemoryError`. +2. **`queryRecords(...)`**: Returns an iterable `Records` object. This is ideal for iterating over rows sequentially as `GenericRecord` objects without loading the entire dataset into memory. +3. **`query(...)`**: Returns a `QueryResponse` object, offering the lowest-level control. Use this to initialize a `ClickHouseBinaryFormatReader` for high-performance streaming or to access the raw `InputStream` directly. + +### POJO mapping + +Mapping rows directly to Plain Old Java Objects (POJOs) is highly efficient when your class structure closely mirrors the table schema. However, before reading or writing POJOs, you **must** register the class and its corresponding schema with the client. This one-time registration compiles the necessary serializers and deserializers. + +Keep the POJO definition independent of the client logic: + +```java +public static class Event { + public long id; + public String name; + public long timestamp; +} +``` + +Register the POJO once during application initialization and retain the schema for subsequent reads: + +```java +import com.clickhouse.client.api.Client; +import com.clickhouse.client.api.metadata.TableSchema; + + +// pojoTables - mapping between POJO class and table +void registerPojoMappings(Client client, Map, String> pojoTables) { + for (Map.Entry, String> entry : pojoTables.entrySet()) { + TableSchema schema = client.getTableSchema(entry.getValue()); + client.register(entry.getKey(), schema); + } +} +``` + +Read rows into typed objects by passing the registered schema: + +```java +import com.clickhouse.client.api.Client; +import com.clickhouse.client.api.metadata.TableSchema; + +// schema that used in previous step so it make sense to build a cache for this mapping +public List readEvents(Client client, TableSchema schema) { + return client.queryAll( + "SELECT id, name, timestamp FROM events", + Event.class, + schema); +} +``` + +### Streaming with Readers + +Readers (such as `ClickHouseBinaryFormatReader`) enable true data streaming. Because only a small buffer is kept in memory, processed rows become immediately eligible for garbage collection as new data is fetched from the network. + +*Note:* To maintain high throughput, your application's processing loop must be fast enough to keep up with the incoming stream; otherwise, network backpressure will slow down the transfer. + +```java +import com.clickhouse.client.api.Client; +import com.clickhouse.client.api.data_formats.ClickHouseBinaryFormatReader; +import com.clickhouse.client.api.query.QueryResponse; +import com.clickhouse.client.api.query.QuerySettings; +import com.clickhouse.data.ClickHouseFormat; + +public void streamEvents(Client client) throws Exception { + QuerySettings settings = new QuerySettings() + .setFormat(ClickHouseFormat.RowBinaryWithNamesAndTypes); + + try (QueryResponse response = client.query("SELECT * FROM events", settings) + .get(30, TimeUnit.SECONDS)) { + + ClickHouseBinaryFormatReader reader = client.newBinaryFormatReader(response); + while (reader.hasNext()) { + reader.next(); + long id = reader.getLong("id"); + String name = reader.getString("name"); + // process row data + } + } +} +``` + +> **Reader schema source.** The one-argument `client.newBinaryFormatReader(response)` only works when the format carries its own schema (e.g. `RowBinaryWithNamesAndTypes`, `Native`). For a plain `RowBinary` stream, which has no embedded column names/types, use the two-argument overload `client.newBinaryFormatReader(response, schema)` and pass a `TableSchema`, or the reader cannot decode the rows. + +### Direct InputStream access + +The `QueryResponse` object also provides direct access to the underlying input stream via `response.getInputStream()`. This is incredibly useful when you want to read text formats using third-party libraries (like Jackson for JSON or OpenCSV for CSV). + +Because of this direct stream access, it is also possible to stream columnar formats directly into memory structures like **Apache Arrow**. (See our [examples directory](../examples/src/main/java/com/clickhouse/examples) for reference implementations). + +### Parameterized queries + +Parameterized queries use ClickHouse placeholder syntax `{name:Type}`: + +```java +import com.clickhouse.client.api.Client; +import com.clickhouse.client.api.query.QueryResponse; +import com.clickhouse.client.api.query.QuerySettings; + +public void queryEventsWithParam(Client client, long minId, QuerySettings settings) throws Exception { + Map params = Collections.singletonMap("min_id", minId); + try (QueryResponse response = client.query( + "SELECT * FROM events WHERE id > {min_id:UInt64}", params, settings).get()) { + // process query response + } +} +``` + +**Key classes:** + +| Class | Role | +|-------|------| +| [`QueryResponse`](../client-v2/src/main/java/com/clickhouse/client/api/query/QueryResponse.java) | Streaming HTTP response; must be closed | +| [`QuerySettings`](../client-v2/src/main/java/com/clickhouse/client/api/query/QuerySettings.java) | Per-query configuration | +| [`Records`](../client-v2/src/main/java/com/clickhouse/client/api/query/Records.java) | Lazy record iterator | +| [`GenericRecord`](../client-v2/src/main/java/com/clickhouse/client/api/query/GenericRecord.java) | Column access by name or index | +| [`OperationMetrics`](../client-v2/src/main/java/com/clickhouse/client/api/metrics/OperationMetrics.java) | Server timing and row counts via `QueryResponse.getMetrics()` | + +### Operation configuration — tuning heavy reads + +The most useful **client settings** for reads are configured on `QuerySettings`: + +| Setting | Property / method | Notes | +|---------|-------------------|-------| +| Output format | `QuerySettings.setFormat(...)` | Prefer binary formats for high throughput | +| Read buffer size | `QuerySettings.setReadBufferSize(n)` | Minimum 8192 bytes; raise for large streams | +| Network timeout | `QuerySettings.setNetworkTimeout(long, ChronoUnit)` | Per-operation socket timeout override; returns `void` (not chainable) | + +> **Server settings**: +> Server settings control query execution on the ClickHouse server itself. Pass them via `QuerySettings.serverSetting(key, value)`. Examples include `serverSetting("max_result_rows", "10000")` or `serverSetting("max_threads", "4")`. (For limiting execution time, there is a dedicated helper: `QuerySettings.setMaxExecutionTime(seconds)`). + +Useful correlation helpers: `settings.setQueryId(...)` and `settings.logComment(...)` surface in `system.query_log` alongside `http_user_agent` (set via `setClientName`). + +### Errors & how to handle them + +See the [Error model](#error-model) for the exception hierarchy and how to unwrap `ExecutionException`. Reads have **no side effects**, so a failed read is always safe to re-run from the start — the decisions are about *whether* it is worth retrying: + +| Failure | Surfaces as | Handling | +|---------|-------------|----------| +| Bad SQL, unknown column, placeholder type mismatch, missing table | `ServerException` (e.g. code `60` table-not-found) | **Not retryable** — a retry produces the same error. Inspect `getCode()` and fix the query. | +| Server aborted an excessively heavy query | `ServerException` code `159` (`TIMEOUT_EXCEEDED`) | The query exceeded `max_execution_time`. Raise the limit or optimize the query; do not retry unconditionally. | +| Transport connect/read timeout | `DataTransferException` / timeout | Often transient. A read is idempotent, so re-running the whole query is safe. | +| Connection dropped **mid-stream** (after you began iterating) | `DataTransferException` while reading | You cannot resume from the middle — some rows were already consumed. Close the `QueryResponse` and re-run the entire query. Make consumers tolerant of re-reading from the start. | +--- + +## Step 7 — Write operations & tuning + +**Goal:** choose an insert pattern, tune it for bulk ingest, and make retries idempotent. + +### Insert methods overview + +The `Client` interface offers multiple ways to insert data, allowing you to choose the right balance of convenience and performance: + +1. **`insert(..., List pojos, ...)`**: The simplest approach for typed data. You pass a `List` of POJOs directly to the client. +2. **`insert(..., InputStream data, ...)`**: The best approach for bulk ingest or pre-serialized data. You pass an `InputStream` directly to the client, avoiding intermediate memory allocations. +3. **`insert(..., DataStreamWriter writer, ...)`**: The best approach for generating binary data on the fly. The client provides an `OutputStream` via a callback, allowing you to write rows directly to the network. + +### POJO mapping + +Just like with read operations, inserting POJOs requires you to register the class and schema first. + +POJO (DTO) to write: +```java +public static class Event { + public long id; + public String name; + public long timestamp; +} +``` + +Register the POJO once during application initialization and retain the schema for subsequent reads: +```java +import com.clickhouse.client.api.Client; +import com.clickhouse.client.api.metadata.TableSchema; + + +// pojoTables - mapping between POJO class and table +void registerPojoMappings(Client client, Map, String> pojoTables) { + for (Map.Entry, String> entry : pojoTables.entrySet()) { + TableSchema schema = client.getTableSchema(entry.getValue()); + client.register(entry.getKey(), schema); + } +} +``` + +Write typed objects after registration. The insert uses the schema cached when `registerEventMapping` was called: + +```java +import com.clickhouse.client.api.Client; +import com.clickhouse.client.api.insert.InsertResponse; + +public void writeEvents(Client client, List events) throws Exception { + if (events.isEmpty()) { + return; + } + + try (InsertResponse response = client.insert("events", events).get()) { + // handle response metrics or confirmation + } +} +``` + +### Direct InputStream access + +If you already have serialized data (e.g., a file containing JSON or CSV, or data arriving from another network stream), you can pipe it directly into ClickHouse. This is highly efficient because it avoids parsing the data into Java objects. + +```java +import com.clickhouse.client.api.Client; +import com.clickhouse.client.api.insert.InsertResponse; +import com.clickhouse.client.api.insert.InsertSettings; +import com.clickhouse.data.ClickHouseFormat; + +public void insertJsonStream(Client client, InputStream dataStream) throws Exception { + InsertSettings settings = new InsertSettings().compressClientRequest(true); + try (InsertResponse response = client.insert("events", dataStream, ClickHouseFormat.JSONEachRow, settings).get()) { + // handle response metrics or confirmation + } +} +``` + +### Callback writer + +For maximum performance when generating data programmatically, use the callback-based `insert`. The client opens the stream, invokes your `DataStreamWriter`, and closes the stream for you. + +This approach is highly efficient because it writes directly to the network socket. By avoiding intermediate in-memory buffers (like building a massive `List` or a large `byte[]`), you eliminate extra memory allocations and garbage collection overhead—which can otherwise cause serious performance degradation on large datasets. + +```java +import com.clickhouse.client.api.Client; +import com.clickhouse.client.api.data_formats.RowBinaryFormatWriter; +import com.clickhouse.client.api.insert.InsertResponse; +import com.clickhouse.client.api.insert.InsertSettings; +import com.clickhouse.client.api.metadata.TableSchema; +import com.clickhouse.data.ClickHouseFormat; + +public void writeEventsStream(Client client, TableSchema schema, List events) throws Exception { + ClickHouseFormat format = ClickHouseFormat.RowBinary; + + try (InsertResponse response = client.insert("events", out -> { + RowBinaryFormatWriter writer = new RowBinaryFormatWriter(out, schema, format); + for (Event event : events) { + writer.setValue("id", event.id); + writer.setValue("name", event.name); + writer.commitRow(); + } + }, format, new InsertSettings()).get()) { + // handle response metrics + } +} +``` + +Populate each row with `setValue(column, value)` (by name or 1-based index) and finish it with `commitRow()`. Do not close the stream yourself — the client manages the lifecycle. + +**Key classes:** + +| Class | Role | +|-------|------| +| [`InsertResponse`](../client-v2/src/main/java/com/clickhouse/client/api/insert/InsertResponse.java) | Insert result; must be closed | +| [`InsertSettings`](../client-v2/src/main/java/com/clickhouse/client/api/insert/InsertSettings.java) | Per-insert configuration | +| [`DataStreamWriter`](../client-v2/src/main/java/com/clickhouse/client/api/DataStreamWriter.java) | Callback interface for writer-based inserts | + +### Operation configuration — tuning heavy writes + +The most useful **client settings** for writes are configured on `InsertSettings`: + +| Setting | Property / method | Notes | +|---------|-------------------|-------| +| Client request compression | `InsertSettings.compressClientRequest(true)` | LZ4-compress the insert body before sending | +| HTTP compression | `InsertSettings.useHttpCompression(true)` | Sets Content-Encoding HTTP header | +| Pre-compressed data | `InsertSettings.appCompressedData(true, "gzip")` | Informs the client you are providing already-compressed data | +| Copy buffer size | `InsertSettings.setInputStreamCopyBufferSize(n)` | Stream-to-stream copy buffer size | + +> **Server settings**: +> Server settings control how ClickHouse processes the ingested data. Pass them via `InsertSettings.serverSetting(key, value)`. For example, `serverSetting("async_insert", "1")` enables server-side buffering, and `serverSetting("wait_for_async_insert", "1")` determines whether the client waits for the flush. + +### Idempotency — deduplication token + +On a `MergeTree`-family table with deduplication enabled, the **`insert_deduplication_token`** lets the server skip duplicate blocks so retries do not create duplicate rows. +Please read [official documentation](https://clickhouse.com/docs/guides/developer/deduplicating-inserts-on-retries) and more about [deduplication strategies](https://clickhouse.com/docs/guides/developer/deduplication). + +```java +import com.clickhouse.client.api.Client; +import com.clickhouse.client.api.insert.InsertResponse; +import com.clickhouse.client.api.insert.InsertSettings; +import com.clickhouse.data.ClickHouseFormat; + +public void insertWithDeduplication(Client client, InputStream dataStream, String deduplicationToken) throws Exception { + InsertSettings settings = new InsertSettings() + .setDeduplicationToken(deduplicationToken); + + try (InsertResponse response = client.insert("events", dataStream, ClickHouseFormat.JSONEachRow, settings).get()) { + // handle response metrics + } +} +``` + +- Assign a **stable** token per logical batch (file name, Kafka offset, job ID). +- Use it for retry-safe pipelines and at-least-once sources (Kafka, SQS, file reprocessing). +- Requires a `MergeTree` engine with deduplication configured. See [`InsertTests.testInsertSettingsDeduplicationToken`](../client-v2/src/test/java/com/clickhouse/client/insert/InsertTests.java). + +### Errors & how to handle them + +See the [Error model](#error-model) for the exception hierarchy and how to unwrap `ExecutionException`. Writes have **side effects**, so error handling is fundamentally harder than for reads: + +| Failure | Surfaces as | Handling | +|---------|-------------|----------| +| Type mismatch, missing/extra column, quota exceeded, read-only table | `ServerException` | Mostly **not retryable** (schema errors). Fix the payload or schema. Check `isRetryable()` before retrying. | +| Transient server condition (too-many-parts `252`, memory limit `241`, network `210`, ...) | `ServerException` with `isRetryable() == true` | The built-in `retry` policy already re-sends. See the caveat below before relying on it. | +| Timeout/drop **after** the server received the body | `ServerException` code `319` (`UNKNOWN_STATUS_OF_INSERT`) or transport error | **Ambiguous** — you cannot tell whether the insert committed. Treat as "maybe written" and rely on a deduplication token so a safe re-insert cannot double-write. | + +**Retrying an insert re-sends the whole payload — this is the key difference from reads.** On a retry the client calls `DataStreamWriter.onRetry()` and then re-invokes `onOutput(...)`, so your data source must be **replayable**: + +- A one-shot `InputStream`, a drained queue, or a consumed iterator **cannot be re-read** — the retry either fails or sends truncated data. Buffer the batch in memory, hand the client a rewindable source, or implement `onRetry()` to reset your stream. +- Even a successful retry can **duplicate rows** unless you set a stable `insert_deduplication_token` (see [Idempotency](#idempotency--deduplication-token) above) on a MergeTree-family table. +- If retries are not safe for your source, disable them (`retry=0`) and handle re-submission at the application level with a dedup token. +--- + +## Step 8 — Metadata & schema discovery + +**Goal:** obtain table and query schemas using the client API. + +### Table schema from a table name + +```java +import com.clickhouse.client.api.Client; +import com.clickhouse.client.api.metadata.TableSchema; +import com.clickhouse.data.ClickHouseColumn; + +public void inspectTableSchema(Client client, String tableName) { + TableSchema schema = client.getTableSchema(tableName); + // Alternatively, specify database explicitly: + // TableSchema schema = client.getTableSchema(tableName, "analytics"); + + for (ClickHouseColumn column : schema.getColumns()) { + System.out.println(column.getColumnName() + " : " + column.getDataType()); + } +} +``` + +Internally runs `DESCRIBE TABLE` and parses via [`TableSchemaParser`](../client-v2/src/main/java/com/clickhouse/client/api/internal/TableSchemaParser.java). + +### Schema from a query + +```java +import com.clickhouse.client.api.Client; +import com.clickhouse.client.api.metadata.TableSchema; + +public TableSchema getQuerySchema(Client client) { + return client.getTableSchemaFromQuery( + "SELECT id, name, created_at FROM events WHERE id > 0"); +} +``` + +### POJO registration + +```java +import com.clickhouse.client.api.Client; +import com.clickhouse.client.api.insert.InsertResponse; +import com.clickhouse.client.api.metadata.TableSchema; + +public void registerAndProcessEvents(Client client) throws Exception { + TableSchema schema = client.getTableSchema("events"); + client.register(Event.class, schema); + + // The typed queryAll takes the target class AND the schema to bind against. + List events = client.queryAll("SELECT * FROM events", Event.class, schema); + + try (InsertResponse response = client.insert("events", events).get()) { + // handle response metrics or confirmation + } +} +``` + +Field-to-column matching is controlled by [`ColumnToMethodMatchingStrategy`](../client-v2/src/main/java/com/clickhouse/client/api/metadata/ColumnToMethodMatchingStrategy.java) (default: camelCase field → snake_case column). You can also register a known schema directly with `client.registerTableSchema("events", schema)`. + +### Tools summary + +| Tool | Method | Use case | +|------|--------|----------| +| Table schema | `getTableSchema(table)` | Inserts, POJO binding, writers | +| Query schema | `getTableSchemaFromQuery(sql)` | Dynamic queries, unknown result shape | +| POJO registry | `register(Class, schema)` | Typed query/insert | +| Column metadata | `TableSchema.getColumnByName(name)` | Type-aware read/write | +| Server info | `client.loadServerInfo()` (returns `void`) | Refreshes cached server info; then read `getServerVersion()`, `getServerTimeZone()`, `getUser()` | + +--- + +## Step 9 — Miscellaneous features + +### Runtime credentials & Access Tokens + +ClickHouse supports authentication via access tokens (e.g., JWTs) instead of traditional username/password credentials. This is common in cloud deployments or when using an authentication proxy. + +You can configure token authentication when building the client: + +```java +import com.clickhouse.client.api.Client; + +public Client.Builder createBearerAuthClient() { + return new Client.Builder() + .addEndpoint("https://my-cluster:8443") + // Use either setAccessToken (raw token) OR useBearerTokenAuth (prepends "Bearer ") + .useBearerTokenAuth("my_jwt_token"); +} +``` + +> **Constraint:** You cannot mix authentication mechanisms. A client must be built with either a username/password OR a token, not both. + +**Realtime credential updates** + +If your application uses short-lived credentials (like expiring JWTs or rotated passwords), you can update the credentials on an existing `Client` instance without recreating it. All subsequent requests will use the new credentials: + +```java +import com.clickhouse.client.api.Client; + +public void updateBearerToken(Client client, String newJwtToken) { + // If the client was built with a bearer token: + client.updateBearerToken(newJwtToken); +} + +public void updateAccessToken(Client client, String newRawToken) { + // If the client was built with a raw access token: + client.updateAccessToken(newRawToken); +} + +public void updateBasicCredentials(Client client, String user, String newPassword) { + // If the client was built with a username/password: + client.updateUserAndPassword(user, newPassword); +} +``` + +> **Note:** The authentication *method* is fixed at construction time. If you built the client with a password, you cannot switch to a token at runtime (and vice versa). Attempting to do so will throw a `ClientMisconfigurationException`. + +**Impersonation and Roles** + +Updating the username and password at runtime is particularly useful for **impersonation** (e.g., a proxy or BI tool switching to a specific user's context). When using this pattern, the initial user passed to the `Client.Builder` should have limited permissions (e.g., only enough to bootstrap or `CREATE USER`/`GRANT`). + +Alternatively, you can manage access using **roles** instead of switching users. Roles can be set globally on the client or overridden per-operation: + +```java +import com.clickhouse.client.api.Client; +import com.clickhouse.client.api.query.QueryResponse; +import com.clickhouse.client.api.query.QuerySettings; + +public void configureClientRoles(Client client, List roles) { + // Set roles globally for all subsequent operations on this client + client.setDBRoles(roles); +} + +public void queryWithCustomRoles(Client client, List roles) throws Exception { + // Or set roles per-operation + QuerySettings settings = new QuerySettings().setDBRoles(roles); + try (QueryResponse response = client.query("SELECT * FROM sensitive_data", settings).get()) { + // process response + } +} +``` + +### Sessions + +A [`Session`](../client-v2/src/main/java/com/clickhouse/client/api/Session.java) carries ClickHouse HTTP session state (`session_id`, `session_check`, `session_timeout`, `session_timezone`). This is completely optional and only needed if your application relies on ClickHouse session context. + +- **Client-wide** — `Client.Builder.use(session)` applies to every operation. +- **Operation-wide** — `QuerySettings.use(session)` / `InsertSettings.use(session)` overrides the session per request. + +> **Note on sessions behind a load balancer:** Using sessions behind a load balancer requires **server affinity** (sticky sessions). A session is pinned to a specific ClickHouse node; if a session-bound request is routed to a different node, it will fail. See the [Sessions example](../examples/client-v2/src/main/java/com/clickhouse/examples/client_v2/Sessions.java). + +--- + +## Error model + +This is the shared exception reference used by the read ([Step 6](#step-6--read-operations--tuning)) and write ([Step 7](#step-7--write-operations--tuning)) error sections. All exceptions extend [`ClickHouseException`](../client-v2/src/main/java/com/clickhouse/client/api/ClickHouseException.java) (an unchecked `RuntimeException`). Because operations return `CompletableFuture`, a failed operation surfaces its cause wrapped in `java.util.concurrent.ExecutionException` when you call `.get()`; unwrap it with `getCause()`. + +| Exception | Meaning | Typical cause | +|-----------|---------|---------------| +| [`ServerException`](../client-v2/src/main/java/com/clickhouse/client/api/ServerException.java) | ClickHouse rejected the request | Bad SQL, type mismatch, missing table; inspect `getCode()` for the CH error code, `isRetryable()`, and `getQueryId()` | +| [`ClientException`](../client-v2/src/main/java/com/clickhouse/client/api/ClientException.java) | Client-side failure | Serialization, reader/writer, or usage error | +| [`ClientMisconfigurationException`](../client-v2/src/main/java/com/clickhouse/client/api/ClientMisconfigurationException.java) | Invalid configuration | Mixed auth mechanisms; runtime mechanism switch | +| [`ConnectionInitiationException`](../client-v2/src/main/java/com/clickhouse/client/api/ConnectionInitiationException.java) | Could not establish a connection | Wrong endpoint, TLS handshake, proxy failure | +| [`DataTransferException`](../client-v2/src/main/java/com/clickhouse/client/api/DataTransferException.java) | Failure while streaming the request/response body | Dropped connection mid-transfer | + +```java +import com.clickhouse.client.api.Client; +import com.clickhouse.client.api.ServerException; +import com.clickhouse.client.api.query.QueryResponse; + +public void executeQueryWithErrorHandling(Client client, String sql) throws Exception { + try (QueryResponse response = client.query(sql).get()) { + // process query response + } catch (ExecutionException e) { + Throwable cause = e.getCause(); + if (cause instanceof ServerException) { + ServerException se = (ServerException) cause; + // ServerException.TABLE_NOT_FOUND == 60 + // Handle or log ClickHouse specific error: + // se.getCode(), se.getQueryId(), se.isRetryable(), se.getMessage() + } + throw e; + } +} +``` + +> **On built-in retries.** `ServerException.isRetryable()` marks transient server codes (timeouts, network, memory-limit, too-many-parts, ...) and the client's own `retry` policy already re-sends those. Do not add a second retry loop on top without accounting for it. Retries behave very differently for reads vs writes — see the per-operation "Errors & how to handle them" sections for the details (in particular, an insert retry must re-send the whole payload). + +## References + +**External resources:** + +| Resource | Link | +|----------|------| +| Official docs | [clickhouse.com/docs/integrations/java](https://clickhouse.com/docs/integrations/java) | +| Javadoc | [javadoc.io/doc/com.clickhouse/client-v2](https://javadoc.io/doc/com.clickhouse/client-v2) | +| Artifact (versions + build snippets) | [central.sonatype.com/artifact/com.clickhouse/client-v2](https://central.sonatype.com/artifact/com.clickhouse/client-v2) | +| Runnable examples | [examples/client-v2](../examples/client-v2) | +| Full property reference | [`ClientConfigProperties`](../client-v2/src/main/java/com/clickhouse/client/api/ClientConfigProperties.java) and [ClickHouse server settings](https://clickhouse.com/docs/operations/settings/settings) | + +**Related documents in this repository:** + +- [integration-index.md](integration-index.md) — choosing JDBC vs Client +- [integration-jdbc.md](integration-jdbc.md) — JDBC integration path +- [authentication.md](authentication.md) — full authentication and TLS reference (referenced from Steps 2–3) +- [features.md](features.md) — compatibility contract (referenced from Step 5) diff --git a/docs/integration-index.md b/docs/integration-index.md new file mode 100644 index 000000000..d8a158067 --- /dev/null +++ b/docs/integration-index.md @@ -0,0 +1,108 @@ +# ClickHouse Java Integration + +## Summary + +This document is the starting point for integrating ClickHouse into a Java application. It explains when to use the **Java Client** (`com.clickhouse:client-v2`) versus the **JDBC Driver** (`com.clickhouse:clickhouse-jdbc`), and points you to the detailed integration guides for each path. + +| Document | Audience | Link | +|----------|----------|------| +| This guide | Anyone evaluating options | this document | +| Java Client path | New applications, high-throughput pipelines, custom data processing | [integration-client.md](integration-client.md) | +| JDBC path | Existing JDBC-based stacks, BI tools, ORMs | [integration-jdbc.md](integration-jdbc.md) | + +Reference information should be fetched from official documentation for [Java Client](https://clickhouse.com/docs/integrations/language-clients/java/client) or [JDBC Driver](https://clickhouse.com/docs/integrations/language-clients/java/jdbc). + +This set of documents can be used to one's code over time. Keep checking your implementation at least per release as we going to add more information and provide migration guidance. + +--- + +## Overview of Libraries + +The `clickhouse-java` repository ships two modern integration layers built on the same HTTP transport: + +| Component | Maven artifact | Role | +|-----------|----------------|------| +| **Java Client** | `com.clickhouse:client-v2` | Native API for queries, inserts, commands, and streaming data | +| **JDBC Driver** | `com.clickhouse:clickhouse-jdbc` | JDBC 4.2 driver that wraps the Java Client internally | + +**A note on names.** The current JDBC driver is often called "JDBC V2". This refers to the `jdbc-v2` source module in this repository, which is the modern rewrite of the driver on top of `client-v2`. You do not depend on `jdbc-v2` directly: the published artifact you add to your build is `com.clickhouse:clickhouse-jdbc`, which bundles the `jdbc-v2` implementation. In short, `clickhouse-jdbc` **is** the JDBC V2 driver. + +Both components communicate with ClickHouse over HTTP(S). The JDBC driver is not a separate protocol stack — every JDBC connection is backed by a `Client` instance internally. + +**Official documentation:** + +- Java Client: [clickhouse.com/docs/integrations/java](https://clickhouse.com/docs/integrations/java) +- JDBC Driver: [clickhouse.com/docs/integrations/language-clients/java/jdbc](https://clickhouse.com/docs/integrations/language-clients/java/jdbc) +- Repository README: [README.md](../README.md) +- Feature contract (for reviewers and advanced users): [features.md](features.md) + +**Examples in this repository:** + +- Client: [examples/client-v2](../examples/client-v2) +- JDBC: [examples/jdbc](../examples/jdbc) +- Spring demo: [examples/demo-service](../examples/demo-service) + +--- + +## Choosing Between the Java Client and JDBC + +When choosing between the Java Client and JDBC Driver, start by considering how ClickHouse differs from typical OLTP databases. ClickHouse is a columnar, analytical database—designed for high-performance analytics, massive scans, and parallel data processing across large datasets, not for transactional (OLTP) workloads like MySQL or PostgreSQL. If you use JDBC simply because it's familiar or widely supported, you may miss out on ClickHouse's true strengths, such as efficient streaming, custom data formats, and bulk operations. JDBC is built around row-oriented, transaction-first APIs, which can be limiting for analytical use cases and may not align with ClickHouse's architecture or optimal access patterns. + +### When to use the Java Client (recommended for new work) + +Choose the Java Client when you: + +- Build a new ingestion or analytics pipeline and control the application code +- Need maximum read/write throughput +- Want to work with ClickHouse **native or binary formats** (`Native`, `RowBinary`, `Parquet`, `JSONEachRow`, ...) +- Need typed POJO serialization/deserialization +- Process data in most effecient way. Client support different binary formats and reads data without additional conversion. +- Require fine-grained control over compression, server settings, sessions, and operation-level configuration + +The Java Client exposes ClickHouse capabilities directly, with no JDBC abstraction between your code and the wire format. For metadata it offers `Client.getTableSchema(String, String)` as an equivalent to `java.sql.DatabaseMetaData`. + +### When to use the JDBC Driver + +Choose the JDBC Driver (`com.clickhouse:clickhouse-jdbc`) when you: + +- Must plug into an existing JDBC ecosystem — an ORM, a JDBC connection pool, a BI tool, or a Spark/Flink JDBC source. These tools speak `java.sql.*` and cannot call the Java Client directly. +- Want a single, standard API across several databases and accept trading some ClickHouse-specific power for that uniformity. +- Mainly need unified access to database metadata (`DatabaseMetaData`) and straightforward row-by-row data preview rather than high-throughput streaming. +- Can accept that ClickHouse-specific types still need handling in your own code. The driver maps types such as `JSON`, `Geometry`, or `Tuple` to Java objects, but your application must interpret them — for example, casting the result of `ResultSet.getObject("coords")` to the expected type, or parsing a `JSON` column that comes back as a `String`. + +--- + +## Side-by-Side Comparison + +| Concern | Java Client | JDBC Driver | +|---------|-------------|-------------| +| API style | Native async/streaming API | Standard JDBC interfaces | +| Read model | Streaming formats, `Records`, POJOs, binary readers | `ResultSet` (row-by-row) | +| Write model | Stream insert, POJO insert, format writers | `INSERT` SQL, batched `PreparedStatement` | +| Formats | RowBinary & Native built-in + custom reader | RowBinary | +| ClickHouse-specific types | Binary readers, POJO serialization/deserialization, generic records | JDBC type mapping + `getObject` overrides | +| Tooling compatibility | Requires application code | Works with JDBC tools and ORMs | +| Underlying transport | HTTP(S) via Apache HttpClient | Same — wraps `client-v2` | +| Configuration | `Client.Builder`, `ClientConfigProperties` | JDBC URL + `Properties`, passthrough to client | +| Best for | Pipelines, services, custom analytics | Existing JDBC stacks, JDBC-only integrations | +| Performance | Client gives access to output/input stream making it possible to use wide veriaty of performant formats. | JDBC reads/writes data via own API that may become performance bottle-neck in some cases. | + +--- + +## Limitations of the JDBC Driver Path + +If you choose the JDBC path, keep the following constraints in mind: + +- **Row-oriented by specification.** The public JDBC API always presents data as rows (`ResultSet.next()`), even though the driver can move native/binary formats internally. Column-oriented or parallel block processing is not expressible through JDBC. +- **No direct access to data streams.** JDBC cannot hand a raw ClickHouse output stream to a columnar consumer (for example, a tool that reads Parquet or JSON natively). You end up writing and maintaining glue code instead. +- **Fewer supported formats.** The driver exposes fewer ClickHouse data formats than the Java Client. +- **Slower feature adoption.** Each new ClickHouse capability must fit the fixed JDBC contract, so features tend to arrive later and sometimes only as workarounds. The Java Client can expose a new feature as a simple helper method. + +--- + +## Next Steps + +| Your choice | Continue with | +|-------------|---------------| +| Java Client | [integration-client.md](integration-client.md) | +| JDBC Driver | [integration-jdbc.md](integration-jdbc.md) | diff --git a/docs/integration-jdbc.md b/docs/integration-jdbc.md new file mode 100644 index 000000000..4d35fa885 --- /dev/null +++ b/docs/integration-jdbc.md @@ -0,0 +1,783 @@ +# ClickHouse JDBC Integration Guide + +This guide is a **step-by-step integration path** for the **JDBC Driver V2** (`jdbc-v2`, published as `com.clickhouse:clickhouse-jdbc`). It is written to be used as context for building an application or a downstream integration spec: each step states the decisions you must make, how to configure them, and the common pitfalls to avoid. + +**Prerequisites:** Read [integration-common.md](integration-common.md) to understand the JDBC trade-offs before committing to this path. + + +> **Architecture in one line.** Every JDBC `Connection` wraps a `client-v2` [`Client`](../client-v2/src/main/java/com/clickhouse/client/api/Client.java) internally. JDBC is not a separate protocol stack — it is a `java.sql.*` façade over the Java Client. +> +> ``` +> Application → ConnectionImpl → Client → HTTP pool → ClickHouse +> ``` + +> **Configuration philosophy.** This guide names only the properties relevant to each step. The exhaustive lists live in [`DriverProperties`](../jdbc-v2/src/main/java/com/clickhouse/jdbc/DriverProperties.java), [`ClientConfigProperties`](../client-v2/src/main/java/com/clickhouse/client/api/ClientConfigProperties.java), and the official docs. Configuration splits into two groups: +> - **Init configuration** — set once via the JDBC URL or `Properties`: endpoint, connection pool size, authentication, TLS. Covered in Steps 1–3. +> - **Operation configuration** — set per statement or as connection defaults: fetch size, timeouts, batch behavior, dedup tokens. Covered in Steps 4–6. +> +> Property routing (see [`DriverProperties`](../jdbc-v2/src/main/java/com/clickhouse/jdbc/DriverProperties.java)): if a property is a JDBC-specific driver property it is handled by the driver; **all other properties are forwarded to `ClientConfigProperties`**. + +## Artifacts + +The driver is published to Maven Central as **`com.clickhouse:clickhouse-jdbc`**. + +Two distributions are published under the same artifact: + +- **Standard artifact** (default, no classifier) — the driver together with its dependencies declared as ordinary transitive Maven dependencies. Recommended for managed builds in which the application controls the dependency tree. +- **Shaded artifact** (`all` classifier) — a single self-contained archive that bundles and **relocates** most third-party dependencies. Recommended when transitive dependencies cannot be managed. + +--- + +## Integration path at a glance + +Work through these steps in order. The "Common Pitfalls" notes tell you what breaks if you skip one. + +| # | Milestone | Core decision | +|---|-----------|---------------| +| 1 | [Instantiation strategy](#step-1--instantiation-strategy) | Connection lifecycle, pooling, and workload identification | +| 2 | [Authentication](#step-2--authentication) | Which auth mechanism and how to configure it via URL/Properties | +| 3 | [Transport & connectivity](#step-3--transport--connectivity-tls-proxies-timeouts) | TLS/mTLS, proxies, timeouts, health checks | +| 4 | [Formats under the hood](#step-4--formats-under-the-hood) | What the driver does internally; when JDBC is not enough | +| 5 | [Read operations & tuning](#step-5--read-operations--tuning) | `ResultSet` streaming; type mapping; heavy-read tuning | +| 6 | [Write operations & tuning](#step-6--write-operations--tuning) | Batch vs RowBinary beta; heavy-ingest tuning; idempotency | +| 7 | [Metadata & schema discovery](#step-7--metadata--schema-discovery) | `DatabaseMetaData`, `ResultSetMetaData`, type mapping | + +--- + +## Step 1 — Instantiation strategy + +**Goal:** decide the lifecycle of a JDBC `Connection` and how you pool connections. + +### What the JDBC objects are + +| Object | Class | Role | +|--------|-------|------| +| `Connection` | [`ConnectionImpl`](../jdbc-v2/src/main/java/com/clickhouse/jdbc/ConnectionImpl.java) | Wraps one `Client`; manages config and delegates to HTTP | +| `Statement` | [`StatementImpl`](../jdbc-v2/src/main/java/com/clickhouse/jdbc/StatementImpl.java) | Execute raw SQL strings | +| `PreparedStatement` | [`PreparedStatementImpl`](../jdbc-v2/src/main/java/com/clickhouse/jdbc/PreparedStatementImpl.java) | Parameterized SQL with `?` placeholders | +| Writer statement | [`WriterStatementImpl`](../jdbc-v2/src/main/java/com/clickhouse/jdbc/WriterStatementImpl.java) | Streaming RowBinary insert | +| `ResultSet` | [`ResultSetImpl`](../jdbc-v2/src/main/java/com/clickhouse/jdbc/ResultSetImpl.java) | Row-by-row streaming of query results | +| `Driver` | [`Driver`](../jdbc-v2/src/main/java/com/clickhouse/jdbc/Driver.java) | Registers `jdbc:clickhouse:` and `jdbc:ch:` URLs | + +### Decisions + +| Question | Recommended answer | +|----------|--------------------| +| How many connections? | **One per concurrent thread of work**, obtained from a pool. | +| Short- or long-lived? | Let a **connection pool** (HikariCP, DBCP, container-managed) manage lifetime; borrow and return. | +| Thread-safe? | **No** — `Connection` is *not* thread-safe. **CONSTRAINT:** Never share one across threads. | +| Own pool needed? | **Yes** — **CONSTRAINT:** Use a standard JDBC connection pool. Each `Connection` still owns an HTTP pool via its internal `Client`. | + +### JDBC URL format + +``` +jdbc:clickhouse://[host][:port][/[path/]database][?param=value&...] +jdbc:clickhouse:https://host:8443/mydb?ssl=true +jdbc:ch://localhost:8123/default +``` + +```java +public Connection createConnection() throws SQLException { + Properties props = new Properties(); + props.setProperty("user", "default"); + props.setProperty("password", "secret"); + return DriverManager.getConnection( + "jdbc:clickhouse://localhost:8123/default", props); +} +``` + +Prefer `Properties` over embedding credentials in the URL. URL/Properties parsing is handled by [`JdbcConfiguration`](../jdbc-v2/src/main/java/com/clickhouse/jdbc/internal/JdbcConfiguration.java); programmatic setup is available via [`DataSourceImpl`](../jdbc-v2/src/main/java/com/clickhouse/jdbc/DataSourceImpl.java). + +### Init configuration — pool sizing and threading + +Because each `Connection` wraps a `Client` with its own HTTP pool, the client-level `max_open_connections` (default 10) forwards through. Two layers of pooling exist: + +- **JDBC connection pool** (your responsibility) — controls how many `Connection` objects (and thus `Client` instances) exist. +- **HTTP pool per connection** — `max_open_connections` forwarded to `ClientConfigProperties`. + +### Workload identification & client name + +In production environments, a single ClickHouse cluster is often shared across diverse workloads: user-facing web services, streaming ingestion pipelines, ETL batch jobs, BI reporting dashboards (e.g., Superset, Tableau, Grafana), and ad-hoc analytics. When queries fail, time out, or consume excessive memory (`MEMORY_LIMIT_EXCEEDED`), identifying the originating application or workload is essential for fast troubleshooting, root-cause analysis, and resource attribution. + +#### Setting client name + +Use a structured format such as `/` or `:/` (for example, `order-service/1.2.0` or `etl-worker:cdc/2.0.1`). + +There are three ways to configure client identification in JDBC: + +**1. Connection Properties or JDBC URL** (static setup for the connection or pool): + +```java +import com.clickhouse.client.api.ClientConfigProperties; + +public Connection createIdentifiedConnection() throws SQLException { + Properties props = new Properties(); + props.setProperty("user", "default"); + props.setProperty("password", "secret"); + // Identify application workload in system.query_log (http_user_agent) + props.setProperty(ClientConfigProperties.CLIENT_NAME.getKey(), "order-service/1.2.0"); + // Alternatively pass as string key: + // props.setProperty("client_name", "order-service/1.2.0"); + + return DriverManager.getConnection( + "jdbc:clickhouse://localhost:8123/default", props); +} +``` + +Or directly via JDBC URL query parameters: + +``` +jdbc:clickhouse://localhost:8123/default?client_name=order-service/1.2.0 +``` + +**2. Standard JDBC `setClientInfo`** (dynamic per-connection or per-task setup): + +When sharing pooled connections across different worker threads or tasks, set the application name dynamically on the borrowed connection before executing work: + +```java +import com.clickhouse.jdbc.ClientInfoProperties; + +public void executeWorkloadTask(Connection conn, String taskName) throws SQLException { + // Dynamically tag the connection before executing queries + conn.setClientInfo(ClientInfoProperties.APPLICATION_NAME.getKey(), "order-service:" + taskName + "/1.2.0"); + // Standard JDBC property name "ApplicationName" is also supported: + // conn.setClientInfo("ApplicationName", "order-service:" + taskName + "/1.2.0"); + + try (Statement stmt = conn.createStatement(); + ResultSet rs = stmt.executeQuery("SELECT count() FROM orders")) { + // process query results + } +} +``` + +#### How it is observed on the server (`User-Agent` header) + +The JDBC driver communicates over HTTP and passes the client name as the leading segment of the HTTP `User-Agent` header. The driver automatically appends the JDBC driver version, detected frameworks (e.g. HikariCP), client version, operating system, and JVM version: + +```text +order-service/1.2.0 ClickHouse-JDBC/0.9.8 clickhouse-java-v2/0.9.8 (Linux; jvm:17.0.2) Apache-HttpClient/5.4.4 +``` + +> **CRITICAL SERVER OBSERVATION NOTE:** +> In ClickHouse's `system.query_log` and `system.processes`, HTTP requests record this information in the **`http_user_agent`** column. The `client_name` column in `system.query_log` is populated **only** for native TCP protocol connections. Always query `http_user_agent` when troubleshooting JDBC applications. + +#### Finding workloads in `system.query_log` + +Use the following queries on ClickHouse to troubleshoot and monitor application workloads: + +**Find recent queries and execution metrics for a specific application:** +```sql +SELECT + event_time, + query_id, + query_duration_ms, + memory_usage, + read_rows, + read_bytes, + result_rows, + http_user_agent, + query +FROM system.query_log +WHERE type = 'QueryFinish' + AND http_user_agent LIKE '%order-service%' + AND event_time >= now() - INTERVAL 1 HOUR +ORDER BY event_time DESC +LIMIT 100; +``` + +**Find failed queries and exceptions for a workload:** +```sql +SELECT + event_time, + query_id, + exception_code, + exception, + http_user_agent, + query +FROM system.query_log +WHERE type = 'ExceptionWhileProcessing' + AND http_user_agent LIKE '%order-service%' + AND event_time >= now() - INTERVAL 24 HOUR +ORDER BY event_time DESC +LIMIT 50; +``` + +**Aggregate workload resource consumption across all applications:** +```sql +SELECT + extract(http_user_agent, '^([^ ]+)') AS workload, + count() AS query_count, + round(avg(query_duration_ms), 2) AS avg_duration_ms, + round(quantile(0.95)(query_duration_ms), 2) AS p95_duration_ms, + round(max(query_duration_ms), 2) AS max_duration_ms, + formatReadableSize(sum(memory_usage)) AS total_memory, + formatReadableQuantity(sum(read_rows)) AS total_read_rows, + countIf(type = 'ExceptionWhileProcessing') AS error_count +FROM system.query_log +WHERE event_time >= now() - INTERVAL 24 HOUR + AND type IN ('QueryFinish', 'ExceptionWhileProcessing') +GROUP BY workload +ORDER BY query_count DESC; +``` + +**Inspect active running queries (`system.processes`):** +```sql +SELECT + query_id, + elapsed, + memory_usage, + http_user_agent, + query +FROM system.processes +WHERE http_user_agent LIKE '%order-service%'; +``` + +**Query across a cluster:** +```sql +SELECT + hostName() AS host, + event_time, + query_id, + query_duration_ms, + memory_usage, + http_user_agent, + query +FROM clusterAllReplicas('default', system.query_log) +WHERE type = 'QueryFinish' + AND http_user_agent LIKE '%order-service%' + AND event_time >= now() - INTERVAL 1 HOUR +ORDER BY event_time DESC +LIMIT 100; +``` + +### Common Pitfalls + + +- **CONSTRAINT:** Never share a `Connection` across threads. It causes data races — it is not thread-safe. +- **No connection pool** means a new `Client` + HTTP pool warm-up on every `getConnection()` — high latency. +- **Over-sized JDBC pool × per-connection HTTP pool** can multiply into far more server connections than expected. Size both deliberately. +- **`Connection.close()`** closes the underlying `Client` and its HTTP pool — expected when returning to a pool. + + +--- + +## Step 2 — Authentication + +**Goal:** choose exactly one primary authentication mechanism and pass it through the JDBC URL or `Properties`. The driver forwards these to the underlying client, whose `CredentialsManager` rejects mixed mechanisms. + +> This section is intentionally self-contained (it mirrors the [Java Client guide](integration-client.md#step-2--authentication) but with **JDBC URL / `Properties` configuration**). For the full reference, see [authentication.md](authentication.md). + +### Option A — Basic (username + password) + +```java +public Connection createBasicAuthConnection() throws SQLException { + Properties props = new Properties(); + props.setProperty("user", "default"); + props.setProperty("password", "secret"); + return DriverManager.getConnection( + "jdbc:clickhouse://localhost:8123/default", props); +} +``` + +### Option B — Token / bearer + +Pass the token as a client property; it forwards to the underlying client's token auth: + +```java +public Connection createTokenAuthConnection() throws SQLException { + Properties props = new Properties(); + props.setProperty("access_token", "my_access_token"); + // or: props.setProperty("bearer_token", "my_access_token"); + return DriverManager.getConnection( + "jdbc:clickhouse://localhost:8123/default", props); +} +``` + +### Option C — Mutual TLS (client certificate) + +```java +public Connection createMtlsConnection() throws SQLException { + Properties props = new Properties(); + props.setProperty("ssl", "true"); + props.setProperty("ssl_authentication", "true"); + props.setProperty("sslcert", "/path/to/client.crt"); + props.setProperty("ssl_key", "/path/to/client.key"); + props.setProperty("sslrootcert", "/path/to/ca.crt"); // if server cert is self-signed + return DriverManager.getConnection( + "jdbc:clickhouse://localhost:8443/default", props); +} +``` + +A trust store may be used instead: `trust_store`, `key_store_password`, `key_store_type`. + +### Option D — Custom headers (proxies / gateways) + +For OAuth gateways, custom authentication proxies, or API gateway configurations, inject arbitrary HTTP headers via connection properties using `DriverProperties.httpHeader(...)` (or the `http_header_` property prefix): + +```java +import com.clickhouse.jdbc.DriverProperties; + +public Connection createCustomHeadersConnection() throws SQLException { + Properties props = new Properties(); + props.setProperty("user", "default"); + props.setProperty("password", "secret"); + props.setProperty(DriverProperties.httpHeader("X-API-Key"), "my_custom_api_key"); + // or directly: props.setProperty("http_header_X-API-Key", "my_custom_api_key"); + return DriverManager.getConnection( + "jdbc:clickhouse://localhost:8123/default", props); +} +``` + +### Decisions + +| Question | Guidance | +|----------|----------| +| Which mechanism? | Password for most deployments; `access_token`/`bearer_token` for gateway-fronted or cloud setups; mTLS for certificate-based zero-trust; custom headers for API gateways. | +| Credentials in URL or Properties? | **Properties** — keeps secrets out of URLs and logs. | +| Behind an auth proxy? | Prefer token auth (Option B) or custom headers via `http_header_` (Option D). | + +### Common Pitfalls + + +- **CONSTRAINT:** Do not mix auth mechanisms (password *and* token). It throws a misconfiguration error when the connection is created. +- **mTLS requires HTTPS** (`ssl=true` + port 8443) and a valid cert/key pair; a missing root CA for self-signed servers fails the handshake. +- **Credentials embedded in the JDBC URL** leak into logs and connection-pool config dumps — use `Properties`. + + +--- + +## Step 3 — Transport & connectivity (TLS, proxies, timeouts) + +**Goal:** make the driver reach the server reliably. The driver delegates TLS and proxy handling to the underlying client; configure via URL/Properties. + +### TLS / mTLS / proxies + +| Scenario | Property / URL parameter | +|----------|--------------------------| +| Enable HTTPS | `ssl=true` + port 8443 | +| Self-signed server cert | `sslrootcert=/path/to/ca.crt` | +| Client certificate (mTLS) | `sslcert`, `ssl_key`, `ssl_authentication=true` | +| Trust store | `trust_store`, `key_store_password`, `key_store_type` | +| HTTP proxy | `proxy_type=http`, `proxy_host`, `proxy_port`, `proxy_user`, `proxy_password` | + +See [examples/jdbc SSLExamples](../examples/jdbc/src/main/java/com/clickhouse/examples/jdbc/SSLExamples.java) and [authentication.md](authentication.md). + +### Init configuration — server vs client settings + +Both forward through URL/Properties: + +```java +public Connection createConfiguredConnection() throws SQLException { + Properties props = new Properties(); + props.setProperty("user", "default"); + props.setProperty("max_execution_time", "60"); // server setting + props.setProperty("max_open_connections", "20"); // client setting + return DriverManager.getConnection( + "jdbc:clickhouse://localhost:8123/default", props); +} +``` + +Per-statement server settings: use `Statement.setQueryTimeout(...)` or an SQL `SETTINGS` clause. + +### Health check + +```java +public boolean checkConnectionHealth(Connection conn, int timeoutSeconds) throws SQLException { + if (!conn.isValid(timeoutSeconds)) { + // trigger recovery logic, mark service unhealthy, or fail fast + return false; + } + return true; +} +``` + +### Common Pitfalls + + +- **Timeouts too aggressive** for heavy analytical queries cause spurious failures — align `max_execution_time` / `setQueryTimeout` with expected duration. +- **Proxy credentials omitted** on authenticated proxies produce opaque connection failures. + + +--- + +## Step 4 — Formats under the hood + +**Goal:** understand that JDBC hides format selection, so you can decide up front whether JDBC's fixed contract is sufficient. + +JDBC does **not** expose format selection. The driver picks formats internally by operation type: + +| Operation | Internal format | Notes | +|-----------|-----------------|-------| +| Query (`executeQuery`) | Binary row format from server | Converted to JDBC `ResultSet` rows | +| Simple INSERT via `Statement` | SQL text | `INSERT INTO t VALUES (...)` | +| `PreparedStatement` INSERT | SQL text or RowBinary | RowBinary when `beta.row_binary_for_simple_insert=true` | +| Writer statement INSERT | RowBinary | Streaming binary writer | +| Batch INSERT | Multi-row SQL rewrite or RowBinary | Depends on statement shape | + +### When JDBC's format contract is not enough + +| Goal | JDBC approach | Better alternative | +|------|---------------|--------------------| +| Simple CRUD / reporting | Standard JDBC — sufficient | — | +| Bulk ingest (millions of rows) | Batch `PreparedStatement` + RowBinary beta | Java Client stream insert | +| Complex type handling | `getObject()` with type map | Java Client POJO/binary readers | +| Export to a file format | Not supported via JDBC | Java Client with format selection | +| BI tool integration | JDBC is the right choice | — | + +### Hybrid usage: dropping down to the Java Client + +If you need maximum ingest throughput, specific binary formats, or POJO serialization, but your application is fundamentally built on JDBC, you can extract the underlying `Client` from the `Connection`. + +```java +import com.clickhouse.client.api.Client; +import com.clickhouse.client.api.insert.InsertResponse; +import com.clickhouse.client.api.insert.InsertSettings; +import com.clickhouse.data.ClickHouseFormat; +import com.clickhouse.jdbc.ConnectionImpl; + +public void insertStreamViaHybridClient(Connection conn, InputStream dataStream) throws Exception { + // Unwrap the JDBC connection to get the native Java Client + Client client = conn.unwrap(ConnectionImpl.class).getClient(); + + // Use native Java Client streaming capabilities while sharing the underlying HTTP connection pool + InsertSettings settings = new InsertSettings().compressClientRequest(true); + try (InsertResponse response = client.insert("events", dataStream, ClickHouseFormat.JSONEachRow, settings).get()) { + // handle response metrics or confirmation + } +} +``` + +This hybrid approach allows you to use standard JDBC for simple CRUD and metadata, while using the native Java Client for bulk ingest or custom data processing. + +### Common Pitfalls + + +- **No format selection API** — you cannot request `Native`, `Parquet`, or `JSONEachRow` through standard JDBC. +- **Row-oriented output only** — no column-oriented or parallel block consumption. +- **Type mapping layer** may lose precision or structure for complex types. +- **Text INSERT overhead** — default SQL-based inserts are slower than binary streaming. Use the [Java Client](integration-client.md) for maximum throughput. + + +--- + +## Step 5 — Read operations & tuning + +**Goal:** read `ResultSet` rows correctly (especially ClickHouse-specific types) and tune heavy reads. + +### General interface + +```java +public void readEvents(Connection conn) throws SQLException { + try (Statement stmt = conn.createStatement(); + ResultSet rs = stmt.executeQuery( + "SELECT id, name, created_at FROM events LIMIT 1000")) { + + while (rs.next()) { + long id = rs.getLong("id"); + String name = rs.getString("name"); + Timestamp created = rs.getTimestamp("created_at"); + // process row data + } + } +} +``` + +**Key classes:** + +| Class | Role | +|-------|------| +| [`StatementImpl`](../jdbc-v2/src/main/java/com/clickhouse/jdbc/StatementImpl.java) | Execute queries, manage timeouts and cancellation | +| [`ResultSetImpl`](../jdbc-v2/src/main/java/com/clickhouse/jdbc/ResultSetImpl.java) | Row-by-row result streaming | +| [`PreparedStatementImpl`](../jdbc-v2/src/main/java/com/clickhouse/jdbc/PreparedStatementImpl.java) | Parameterized queries | + +### Reading ClickHouse-specific types + +```java +public void readSpecialTypes(ResultSet rs) throws SQLException { + String uuid = rs.getString("uuid_col"); + BigDecimal decimal = rs.getBigDecimal("decimal_col"); + LocalDateTime timestamp = rs.getObject("ts_col", LocalDateTime.class); // java.time + + Map> typeMap = Collections.singletonMap("UInt64", BigInteger.class); + BigInteger bigNum = (BigInteger) rs.getObject("big_num", typeMap); // custom mapping + + Array array = rs.getArray("tags"); // Array + Struct tuple = (Struct) rs.getObject("point"); // Tuple +} +``` + +### Operation configuration — tuning heavy reads + +| Setting | Property / method | Notes | +|---------|-------------------|-------| +| Response compression | `compress=true` (default) | Server-side LZ4 | +| Max execution time | `max_execution_time` | Server-side query timeout | +| Max result rows | `jdbc_use_max_result_rows=true` | Enforce server `max_result_rows` | +| Query timeout | `Statement.setQueryTimeout(seconds)` | JDBC-level timeout | +| Result-set auto-close | `jdbc_resultset_auto_close=true` (default) | Close previous result on new query | +| Fetch size | `Statement.setFetchSize(n)` | Streaming batch-size hint | + +**Tip:** enforce row limits in SQL (`LIMIT n`). With `jdbc_use_max_result_rows` disabled, the driver stops reading at the limit but the server may still send remaining data. + +### Best practices + + +- **Always use `LIMIT`** for exploratory queries. +- **Set a query timeout** to prevent hung queries. +- **Use try-with-resources** for `Connection`, `Statement`, `ResultSet`. +- **Prefer `PreparedStatement`** for repeated / parameterized queries. +- **Map large integers**: `jdbc_type_mappings=UInt64=java.math.BigInteger`. +- **Use `getObject(column, Class)`** for `java.time` types instead of legacy getters. + + +### Common Pitfalls + + +- **`getInt()` on `UInt64`/`Int128`** overflows — use `getBigDecimal`/`BigInteger` or a custom mapping. +- **Complex types** (`Array`, `Tuple`, `Map`, `Nested`, `Variant`, `Dynamic`, geometry) require `getObject()`/`getArray()`, not primitive getters. +- **Scrollable/updatable result sets are unsupported** — forward-only, read-only. +- **No server-side cursors** — the server streams the full result set. +- **Some frameworks materialize all rows** even though the driver streams — watch memory. + + +--- + +## Step 6 — Write operations & tuning + +**Goal:** choose an insert path, tune batching, and make retries idempotent. ClickHouse has **no transactions** — every statement auto-commits immediately. + +### Insert paths + +**Simple INSERT via `Statement`:** + +```java +public void insertDirect(Connection conn) throws SQLException { + try (Statement stmt = conn.createStatement()) { + stmt.executeUpdate("INSERT INTO events (id, name) VALUES (1, 'click'), (2, 'house')"); + } +} +``` + +**Batched INSERT via `PreparedStatement`:** + +```java +public void insertEventsBatch(Connection conn, List events) throws SQLException { + if (events.isEmpty()) { + return; + } + + try (PreparedStatement ps = conn.prepareStatement( + "INSERT INTO events (id, name, created_at) VALUES (?, ?, ?)")) { + for (Event event : events) { + ps.setLong(1, event.getId()); + ps.setString(2, event.getName()); + ps.setObject(3, event.getCreatedAt()); + ps.addBatch(); + } + ps.executeBatch(); + } +} +``` + +**RowBinary streaming insert (beta):** enable `beta.row_binary_for_simple_insert=true` so simple `INSERT INTO t VALUES (?, ?, ?)` statements serialize as RowBinary via [`WriterStatementImpl`](../jdbc-v2/src/main/java/com/clickhouse/jdbc/WriterStatementImpl.java) instead of SQL text. + +```java +public Connection createRowBinaryInsertConnection() throws SQLException { + Properties props = new Properties(); + props.setProperty("user", "default"); + props.setProperty("password", "secret"); + // switch using row binary writer for inserts + props.setProperty("beta.row_binary_for_simple_insert", "true"); + return DriverManager.getConnection( + "jdbc:clickhouse://localhost:8123/default", props); +} +``` + +### Operation configuration — tuning heavy writes + +| Setting | Property / method | Notes | +|---------|-------------------|-------| +| Batch inserts | `PreparedStatement.addBatch()` / `executeBatch()` | Multi-row rewrite for eligible INSERTs | +| RowBinary writer | `beta.row_binary_for_simple_insert=true` | Binary path for simple `VALUES (?, ?, ?)` | +| Client compression | `decompress=true` | LZ4-compress insert payload | +| HTTP compression | `client.use_http_compression=true` | Content-Encoding on the HTTP layer | +| Async insert | `async_insert=1` (server setting) | Server-side insert buffering | + +### Idempotency — deduplication token + +JDBC does not expose `insert_deduplication_token` as a first-class API. Three ways to use it: + + +**SQL `SETTINGS` clause** (per statement): + +```java +public void insertWithPerStatementDedup(Connection conn, String token) throws SQLException { + try (Statement stmt = conn.createStatement()) { + stmt.executeUpdate( + "INSERT INTO events SETTINGS insert_deduplication_token = '" + token + "' VALUES (1, 'a')"); + } +} +``` + +**2. Switch to the Java Client** for per-insert token control via `InsertSettings.setDeduplicationToken(...)`. + +See [integration-client.md — deduplication token](integration-client.md#idempotency--deduplication-token) for semantics and requirements. + +### Best practices + + +- **Batch inserts** — hundreds to thousands of rows per batch. +- **Enable the RowBinary beta** for simple inserts — significantly faster than SQL text rendering. +- **Use `PreparedStatement`** so the driver escapes values correctly. +- **Set deduplication tokens** on retry-prone pipelines. +- **Tune batch size** by row width; watch `system.query_log` for insert performance. + + +### Common Pitfalls + + +- **One row per `executeUpdate()`** — HTTP overhead dominates. +- **No transactional rollback** — a failed batch may leave partial data depending on the engine. +- **Batching complex INSERT shapes** (`INSERT SELECT`, multi-table) is unsupported — use `Statement`. +- **Retrying without a dedup token** on MergeTree can create duplicates. +- **Maximum ingest throughput** is not JDBC's strength — the [Java Client](integration-client.md) stream insert is faster. + + +--- + +## Step 7 — Metadata & schema discovery + +**Goal:** use standard JDBC metadata interfaces, backed by ClickHouse system tables, and understand type mapping. + +### DatabaseMetaData + +```java +public TableMetadata buildMetadataModel(Connection conn, String database, String table) throws SQLException { + DatabaseMetaData meta = conn.getMetaData(); + TableMetadata.Builder tableMetadataBuilder = new TableMetadata.Builder() + .setDatabase(database) + .setTableName(table); + + try (ResultSet tables = meta.getTables(null, database, table, new String[]{"TABLE"})) { + if (tables.next()) { + tableMetadataBuilder.setTableType(tables.getString("TABLE_TYPE")); + } + } + + try (ResultSet columns = meta.getColumns(null, database, table, "%")) { + while (columns.next()) { + String name = columns.getString("COLUMN_NAME"); + int jdbcType = columns.getInt("DATA_TYPE"); + String chType = columns.getString("TYPE_NAME"); + tableMetadataBuilder.addColumn(name, jdbcType, chType); + } + } + + return tableMetadataBuilder.build(); +} +``` + +Implemented by [`DatabaseMetaDataImpl`](../jdbc-v2/src/main/java/com/clickhouse/jdbc/metadata/DatabaseMetaDataImpl.java). Supports catalogs, schemas, tables/views/materialized views, column metadata (JDBC codes + native type names), primary keys, and index info (sorting keys), with table types mapped from ClickHouse engines. + +### ResultSetMetaData & ParameterMetaData + +```java +public void inspectResultSetMetadata(ResultSet rs) throws SQLException { + ResultSetMetaData rsMeta = rs.getMetaData(); + for (int i = 1; i <= rsMeta.getColumnCount(); i++) { + String colName = rsMeta.getColumnName(i); + String chType = rsMeta.getColumnTypeName(i); // exact ClickHouse type, e.g. Nullable(UInt64) + int jdbcType = rsMeta.getColumnType(i); // mapped JDBC type code + // process metadata + } +} + +public int getParameterCount(PreparedStatement ps) throws SQLException { + return ps.getParameterMetaData().getParameterCount(); +} +``` + +### Type mapping + +The driver maps ClickHouse types to JDBC types (e.g. `UInt64` → `NUMERIC`, `Tuple` → `STRUCT`); see [type_mapping.md](../type_mapping.md). Override defaults: + + +```java +public Connection createConnection() throws SQLException { + Properties props = new Properties(); + // .. base configuration + props.setProperty("jdbc_type_mappings", "UInt64=java.math.BigInteger,Int128=java.math.BigInteger"); + + return DriverManager.getConnection( + "jdbc:clickhouse://localhost:8123/default", props); +} +``` + +### Tools summary + +| Tool | Interface | Use case | +|------|-----------|----------| +| Table discovery | `DatabaseMetaData.getTables()` | List tables and views | +| Column discovery | `DatabaseMetaData.getColumns()` | Schema inspection, ORM tooling | +| Query result schema | `ResultSetMetaData` | Dynamic query handling | +| Native type name | `getColumnTypeName()` | Exact ClickHouse type | +| JDBC type code | `getColumnType()` | Standard JDBC interop | +| Custom type map | `jdbc_type_mappings` / `setTypeMap()` | Override default mappings | +| Client-side schema | `ConnectionImpl.getClient().getTableSchema()` | Advanced: reach the underlying client API | + +For schema-driven POJO binding and binary format writers, use the [Java Client integration guide](integration-client.md). + +### Common Pitfalls + + +- **JDBC metadata may not reflect every ClickHouse type nuance** — use `getColumnTypeName()` for the native string. +- **Default type mappings** may not match your expectations for large integers or complex types — override with `jdbc_type_mappings`. + + +--- + +## JDBC-specific features + +| Feature | How to use | +|---------|------------| +| Application name | Set via `client_name` property, URL parameter, or `Connection.setClientInfo("ApplicationName", "my-app")` — see [Workload identification & client name](#workload-identification--client-name) | +| Schema / database | `Connection.setSchema("analytics")` or the URL path | +| Query cancellation | `Statement.cancel()` → `KILL QUERY` (optionally `ON CLUSTER` via `jdbc_cluster_name`) | +| JDBC escape syntax | `{ts '...'}`, `{d '...'}`, `{fn ...}` — translated before execution | +| Default query settings | `default_query_settings` property | +| Role management | `SET ROLE` statements (roles remembered by default) | + +Key JDBC-specific properties (see [`DriverProperties`](../jdbc-v2/src/main/java/com/clickhouse/jdbc/DriverProperties.java)): + +| Property | Default | Purpose | +|----------|---------|---------| +| `ssl` | `false` | Enable HTTPS | +| `jdbc_ignore_unsupported_values` | `false` | Silently ignore unsupported JDBC calls | +| `jdbc_resultset_auto_close` | `true` | Auto-close result set on new query | +| `jdbc_use_max_result_rows` | `false` | Enforce server `max_result_rows` | +| `beta.row_binary_for_simple_insert` | `false` | RowBinary writer for simple INSERT | +| `jdbc_sql_parser` | `JAVACC` | SQL parser backend | +| `jdbc_cluster_name` | — | Cluster for `KILL QUERY ON CLUSTER` | +| `jdbc_type_mappings` | — | Custom ClickHouse → Java type overrides | +| `default_query_settings` | — | Default settings for all queries | + + +## References + +**External resources:** + +| Resource | Link | +|----------|------| +| Official docs | [clickhouse.com/docs/integrations/language-clients/java/jdbc](https://clickhouse.com/docs/integrations/language-clients/java/jdbc) | +| Javadoc | [javadoc.io/doc/com.clickhouse/clickhouse-jdbc](https://javadoc.io/doc/com.clickhouse/clickhouse-jdbc) | +| Maven artifact | `com.clickhouse:clickhouse-jdbc` (use the `all` classifier for bundled dependencies) | +| Examples | [examples/jdbc](../examples/jdbc) | +| Full property reference | [`DriverProperties`](../jdbc-v2/src/main/java/com/clickhouse/jdbc/DriverProperties.java), [`ClientConfigProperties`](../client-v2/src/main/java/com/clickhouse/client/api/ClientConfigProperties.java), and [ClickHouse server settings](https://clickhouse.com/docs/operations/settings/settings) | + +**Related documents in this repository:** + +- [integration-common.md](integration-common.md) — choosing JDBC vs Client +- [integration-client.md](integration-client.md) — Java Client integration path +- [authentication.md](authentication.md) — full authentication and TLS reference +- [features.md](features.md) — compatibility contract +- [type_mapping.md](../type_mapping.md) — JDBC type mapping recommendations \ No newline at end of file