Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions NEXT_CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,14 @@
## [Unreleased]

### Added
- Added support for creating mutable JDBC `Clob` values and binding them to prepared statements.

### Updated
- `DatabaseMetaData.getColumns(...)` with a `null` catalog now issues a single `SHOW COLUMNS IN ALL CATALOGS` statement (consistent with `getSchemas`/`getTables`) instead of enumerating every catalog and issuing a per-catalog `SHOW COLUMNS`. Older DBR versions that do not support the syntax transparently fall back to the previous enumerate-and-fan-out behavior.

### Fixed
- Fixed ASCII- and character-stream parameter binding to read unbounded streams to EOF and bounded streams without consuming data past the declared length.

- Fixed connections failing when the same parameter is provided in both the JDBC URL and the connection properties, with the JDBC URL taking precedence.
- Fixed `IdleConnectionEvictor` thread leak in long-running applications. Driver-side resources (HTTP client, background threads) are now always released when `Connection.close()` is called, even if statement cleanup or server-side session termination fails.

Expand Down
19 changes: 10 additions & 9 deletions docs/JDBC_METHOD_INVENTORY.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,7 +124,7 @@
| `setCharacterStream(int, Reader, int)` | YES | - | No | OCCASIONAL | Reads character stream |
| `setRef(int, Ref)` | THROWS_NOT_SUPPORTED | DatabricksSQLFeatureNotSupportedException | No | RARE | REF type not supported |
| `setBlob(int, Blob)` | THROWS_NOT_SUPPORTED | DatabricksSQLFeatureNotSupportedException | No | RARE | BLOB type not supported |
| `setClob(int, Clob)` | THROWS_NOT_SUPPORTED | DatabricksSQLFeatureNotSupportedException | No | RARE | CLOB type not supported |
| `setClob(int, Clob)` | YES | - | No | RARE | Binds CLOB contents as STRING |
| `setArray(int, Array)` | THROWS_NOT_SUPPORTED | DatabricksSQLFeatureNotSupportedException | No | RARE | Array parameter not supported |
| `getMetaData()` | YES | - | No | OCCASIONAL | Returns ResultSetMetaData |
| `setDate(int, Date, Calendar)` | YES | - | No | OCCASIONAL | Sets Date with Calendar |
Expand All @@ -137,7 +137,7 @@
| `setNString(int, String)` | THROWS_NOT_SUPPORTED | DatabricksSQLFeatureNotSupportedException | No | RARE | NString not supported |
| `setNCharacterStream(int, Reader, long)` | THROWS_NOT_SUPPORTED | DatabricksSQLFeatureNotSupportedException | No | RARE | NCharacterStream not supported |
| `setNClob(int, NClob)` | THROWS_NOT_SUPPORTED | DatabricksSQLFeatureNotSupportedException | No | RARE | NClob not supported |
| `setClob(int, Reader, long)` | THROWS_NOT_SUPPORTED | DatabricksSQLFeatureNotSupportedException | No | RARE | Clob reader not supported |
| `setClob(int, Reader, long)` | YES | - | No | RARE | Binds reader contents as STRING |
| `setBlob(int, InputStream, long)` | THROWS_NOT_SUPPORTED | DatabricksSQLFeatureNotSupportedException | No | RARE | Blob stream not supported |
| `setNClob(int, Reader, long)` | THROWS_NOT_SUPPORTED | DatabricksSQLFeatureNotSupportedException | No | RARE | NClob reader not supported |
| `setSQLXML(int, SQLXML)` | THROWS_NOT_SUPPORTED | DatabricksSQLFeatureNotSupportedException | No | RARE | SQLXML not supported |
Expand All @@ -149,7 +149,7 @@
| `setBinaryStream(int, InputStream)` | THROWS_NOT_SUPPORTED | DatabricksSQLFeatureNotSupportedException | No | RARE | Binary stream not supported |
| `setCharacterStream(int, Reader)` | YES | - | No | OCCASIONAL | Character stream no length |
| `setNCharacterStream(int, Reader)` | THROWS_NOT_SUPPORTED | DatabricksSQLFeatureNotSupportedException | No | RARE | NCharacterStream not supported |
| `setClob(int, Reader)` | THROWS_NOT_SUPPORTED | DatabricksSQLFeatureNotSupportedException | No | RARE | Clob reader not supported |
| `setClob(int, Reader)` | YES | - | No | RARE | Binds reader contents as STRING |
| `setBlob(int, InputStream)` | THROWS_NOT_SUPPORTED | DatabricksSQLFeatureNotSupportedException | No | RARE | Blob stream not supported |
| `setNClob(int, Reader)` | THROWS_NOT_SUPPORTED | DatabricksSQLFeatureNotSupportedException | No | RARE | NClob reader not supported |
| `setObject(int, Object, SQLType, int)` | YES | - | No | OCCASIONAL | JDBC 4.2 - SQLType variant |
Expand Down Expand Up @@ -514,7 +514,7 @@ OUT/INOUT parameters, named parameters, and return-value syntax (`{? = call ...}
| `prepareStatement(String, int)` | PARTIAL | DatabricksSQLFeatureNotSupportedException | No | OCCASIONAL | Only NO_GENERATED_KEYS |
| `prepareStatement(String, int[])` | THROWS_NOT_SUPPORTED | DatabricksSQLFeatureNotSupportedException | No | RARE | Generated keys not supported |
| `prepareStatement(String, String[])` | THROWS_NOT_SUPPORTED | DatabricksSQLFeatureNotSupportedException | No | RARE | Generated keys not supported |
| `createClob()` | THROWS_NOT_IMPLEMENTED | DatabricksSQLFeatureNotImplementedException | No | RARE | CLOB not supported |
| `createClob()` | YES | - | No | RARE | Returns a mutable in-memory CLOB |
| `createBlob()` | THROWS_NOT_IMPLEMENTED | DatabricksSQLFeatureNotImplementedException | No | RARE | BLOB not supported |
| `createNClob()` | THROWS_NOT_IMPLEMENTED | DatabricksSQLFeatureNotImplementedException | No | RARE | NCLOB not supported |
| `createSQLXML()` | THROWS_NOT_IMPLEMENTED | DatabricksSQLFeatureNotImplementedException | No | RARE | SQLXML not supported |
Expand Down Expand Up @@ -776,11 +776,11 @@ OUT/INOUT parameters, named parameters, and return-value syntax (`{? = call ...}
| Interface | Total Methods | Fully Implemented | Partially Implemented | Throws NOT_SUPPORTED | Throws NOT_IMPLEMENTED | Not Implemented |
|-----------|---------------|-------------------|----------------------|---------------------|------------------------|-----------------|
| Statement | 54 | 37 (69%) | 5 (9%) | 12 (22%) | 0 | 0 |
| PreparedStatement | 70 | 28 (40%) | 1 (1%) | 24 (34%) | 17 (24%) | 0 |
| PreparedStatement | 70 | 31 (44%) | 1 (1%) | 21 (30%) | 17 (24%) | 0 |
| CallableStatement | 100+ | 0 (0%) | 0 | 0 | 0 | 100+ (100%) |
| ResultSet | 200+ | 70 (35%) | 2 (1%) | 130+ (65%) | 0 | 0 |
| ResultSetMetaData | 23 | 23 (100%) | 0 | 0 | 0 | 0 |
| Connection | 60 | 26 (43%) | 12 (20%) | 8 (13%) | 14 (23%) | 0 |
| Connection | 60 | 27 (45%) | 12 (20%) | 8 (13%) | 13 (22%) | 0 |
| DatabaseMetaData | 180+ | 180+ (100%) | 0 | 0 | 0 | 0 |
| ParameterMetaData | 11 | 3 (27%) | 8 (73%) | 0 | 0 | 0 |

Expand All @@ -794,8 +794,8 @@ OUT/INOUT parameters, named parameters, and return-value syntax (`{? = call ...}
- `Statement` - 69% fully implemented, mostly core functionality

3. **Partially Implemented:**
- `Connection` - 43% fully implemented, 20% partially (many throw exceptions for advanced features)
- `PreparedStatement` - 40% fully implemented, many LOB/stream methods not supported
- `Connection` - 45% fully implemented, 20% partially (many throw exceptions for advanced features)
- `PreparedStatement` - 44% fully implemented; CLOB binding is supported, while BLOB/NCLOB binding is not
- `ResultSet` - 35% fully implemented, most update methods throw NOT_SUPPORTED

4. **Not Implemented:**
Expand All @@ -819,11 +819,12 @@ OUT/INOUT parameters, named parameters, and return-value syntax (`{? = call ...}
- Large update counts (supported)
- Streaming (ASCII/Character streams supported, binary not supported)
- Calendar-based date/time operations (supported)
- CLOB creation and PreparedStatement binding (mapped to STRING)

**NOT SUPPORTED:**
- CallableStatement (stored procedures)
- Updatable ResultSets (all update methods throw exceptions)
- LOB types (BLOB, CLOB, NCLOB)
- BLOB/NCLOB operations and CLOB retrieval/update
- Advanced types (REF, SQLXML, RowId, URL)
- Named cursors
- Bidirectional ResultSet navigation
Expand Down
49 changes: 25 additions & 24 deletions docs/JDBC_SPEC_COVERAGE_ANALYSIS.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,19 +32,19 @@ This document provides a comprehensive analysis of JDBC specification compliance

### Key Findings

**Overall Integration Test Coverage: 13.4%** (when considering only applicable methods)
**Overall Integration Test Coverage: 13.3%** (when considering only applicable methods)

**Comprehensive JDBC Inventory:**
- **Total JDBC Methods Analyzed:** 588+ methods across 8 core interfaces (ALL methods including rare, deprecated, vendor-specific)
- **Methods Applicable to Databricks:** 328 methods (excludes CallableStatement, updatable ResultSet, LOBs)
- **Methods Applicable to Databricks:** 332 methods (excludes CallableStatement, updatable ResultSet, and unsupported LOB operations)
- **Methods with Integration Tests:** 44 methods
- **Applicable Coverage:** 44/328 = **13.4%**
- **Applicable Coverage:** 44/332 = **13.3%**

**Implementation Status:**
- **Fully Implemented:** 367+ methods (62% of total)
- **Fully Implemented:** 371+ methods (63% of total)
- **Partially Implemented:** 28 methods
- **Throws NOT_SUPPORTED:** 174+ methods (intentional - feature not applicable)
- **Not Implemented:** 124+ methods (CallableStatement, etc.)
- **Throws NOT_SUPPORTED:** 171+ methods (intentional - feature not applicable)
- **Not Implemented:** 130+ methods (CallableStatement, etc.)

**Core Methods Analysis (Original 156 high-priority methods):**
- **Methods with Integration Tests:** 44 methods (28%)
Expand Down Expand Up @@ -1570,14 +1570,14 @@ This appendix provides a comprehensive inventory of ALL JDBC methods across all
| Interface | Total Methods | Fully Implemented | Partially Implemented | Throws NOT_SUPPORTED | Not Implemented | Implementation % |
|-----------|--------------|-------------------|----------------------|---------------------|----------------|-----------------|
| **Statement** | 54 | 37 | 5 | 12 | 0 | **69%** |
| **PreparedStatement** | 60 | 28 | 1 | 24 | 10 | **47%** |
| **PreparedStatement** | 70 | 31 | 1 | 21 | 17 | **44%** |
| **CallableStatement** | 100+ | 0 | 0 | 0 | 100+ | **0%** |
| **ResultSet** | 200+ | 70 | 2 | 130+ | 0 | **35%** |
| **ResultSetMetaData** | 23 | 23 | 0 | 0 | 0 | **100%** |
| **Connection** | 60 | 26 | 12 | 8 | 14 | **43%** |
| **Connection** | 60 | 27 | 12 | 8 | 13 | **45%** |
| **DatabaseMetaData** | 180+ | 180+ | 0 | 0 | 0 | **100%** |
| **ParameterMetaData** | 11 | 3 | 8 | 0 | 0 | **27%** |
| **TOTAL** | **588+** | **367+** | **28** | **174+** | **124+** | **62%** |
| **TOTAL** | **588+** | **371+** | **28** | **171+** | **130+** | **63%** |

#### D.2 Key Findings from Complete Inventory

Expand Down Expand Up @@ -1637,13 +1637,14 @@ This appendix provides a comprehensive inventory of ALL JDBC methods across all
- **Impact:** Cannot modify data through ResultSet, must use UPDATE statements
- **Test Strategy:** Verify exceptions are thrown correctly

3. **LOB Types** (0%)
3. **LOB Types** (Partial)
- **BLOB operations:** getBlob(), setBlob(), updateBlob() - Not supported
- **CLOB operations:** getClob(), setClob(), updateClob() - Not supported
- **CLOB operations:** createClob() and PreparedStatement setClob() are supported by mapping contents to STRING; ResultSet getClob() and updateClob() are not supported
- **NCLOB operations:** getNClob(), setNClob(), updateNClob() - Not supported
- **Reason:** Databricks does not support LOB types natively
- **Workaround:** Use String for CLOB-like data, byte[] for BLOB-like data
- **Test Strategy:** Verify NOT_SUPPORTED exceptions are thrown
- **Limitation:** CLOB creation and binding materialize character contents in JVM memory; there is no server-side LOB locator or streaming parameter transport
- **Test Strategy:** Verify supported CLOB creation/binding behavior and NOT_SUPPORTED exceptions for the remaining LOB operations

4. **Bidirectional ResultSet Navigation** (0%)
- **Methods:** previous(), first(), last(), absolute(), relative(), beforeFirst(), afterLast()
Expand Down Expand Up @@ -1676,7 +1677,7 @@ This appendix provides a comprehensive inventory of ALL JDBC methods across all
**RARE Methods (0-40% implementation):** ~150 methods
- Advanced scrolling
- Updatable ResultSets
- LOB operations
- Unsupported LOB retrieval/update and BLOB/NCLOB operations
- Named cursors
- **Test Priority:** LOW - Verify NOT_SUPPORTED exceptions only

Expand All @@ -1687,7 +1688,7 @@ This appendix provides a comprehensive inventory of ALL JDBC methods across all

#### D.4 Testing Recommendations by Implementation Status

**For Fully Implemented Methods (367+ methods):**
**For Fully Implemented Methods (371+ methods):**
- ✅ **Add integration tests for COMMON usage (Priority: HIGH)**
- ✅ **Add integration tests for OCCASIONAL usage (Priority: MEDIUM)**
- ⚠️ **Optional integration tests for RARE usage (Priority: LOW)**
Expand All @@ -1698,13 +1699,13 @@ This appendix provides a comprehensive inventory of ALL JDBC methods across all
- ✅ **Verify warnings/exceptions for unsupported features**
- ✅ **Document limitations clearly**

**For Methods Throwing NOT_SUPPORTED (174+ methods):**
**For Methods Throwing NOT_SUPPORTED (171+ methods):**
- ✅ **Verify exception type is correct** (SQLFeatureNotSupportedException)
- ✅ **Verify exception message is meaningful**
- ❌ **No need to test functionality** (not implemented)
- ✅ **Document in driver documentation**

**For Not Implemented Methods (124+ methods):**
**For Not Implemented Methods (130+ methods):**
- ❌ **No tests needed** (feature not applicable)
- ✅ **Document limitation in driver documentation**
- ✅ **Consider adding to FAQ/Known Limitations section**
Expand All @@ -1714,17 +1715,17 @@ This appendix provides a comprehensive inventory of ALL JDBC methods across all
**Revised Coverage Analysis:**

When we exclude methods that are intentionally not supported (not applicable to Databricks):
- **Excluded:** CallableStatement (100+ methods), Updatable ResultSet (130+ methods), LOBs (30+ methods)
- **Excluded Total:** ~260 methods not applicable to Databricks
- **Excluded:** CallableStatement (100+ methods), Updatable ResultSet (130+ methods), and unsupported LOB operations (approximately 26 methods)
- **Excluded Total:** ~256 methods not applicable to Databricks

**Adjusted Coverage:**
- **Applicable Methods:** 588 - 260 = 328 methods
- **Applicable Methods:** 588 - 256 = 332 methods
- **Tested Methods:** 44 integration tests cover ~44 methods
- **Applicable Coverage:** 44/328 = **13.4%** (vs. 28% when including non-applicable methods)
- **Implemented & Applicable:** 367 - 130 (updateXXX) - 100 (CallableStatement) = 137 methods
- **Implementation Coverage:** 367/328 = **112%** (includes partial implementations)
- **Applicable Coverage:** 44/332 = **13.3%** (vs. 28% when including non-applicable methods)
- **Implemented & Applicable:** 371 - 130 (updateXXX) - 100 (CallableStatement) = 141 methods
- **Implementation Coverage:** 371/332 = **112%** (includes partial implementations)

**Key Insight:** We have good *implementation* coverage (62% overall, 100% for applicable features), but poor *integration test* coverage (13.4% of applicable methods).
**Key Insight:** We have good *implementation* coverage (63% overall, 100% for applicable features), but poor *integration test* coverage (13.3% of applicable methods).

#### D.6 Detailed Method Inventory

Expand Down Expand Up @@ -1771,7 +1772,7 @@ Based on the complete inventory, integration tests should focus on:
**Tier 3 - NOT_SUPPORTED Verification (Low Priority):**
- ⚠️ Verify exceptions for updateable ResultSet methods
- ⚠️ Verify exceptions for scrollable ResultSet methods
- ⚠️ Verify exceptions for LOB operations
- ⚠️ Verify exceptions for unsupported LOB retrieval/update and BLOB/NCLOB operations
- ⚠️ Verify exceptions for generated keys
- **Estimated:** 10-15 integration tests needed

Expand Down
Loading
Loading