Skip to content

TCP Q3/Q4: an idle read deadline, and telling the server when a result is abandoned - #591

Open
alex-clickhouse wants to merge 1 commit into
tcp/epic-q1-exceptionsfrom
tcp/epic-q3-timeouts
Open

TCP Q3/Q4: an idle read deadline, and telling the server when a result is abandoned#591
alex-clickhouse wants to merge 1 commit into
tcp/epic-q1-exceptionsfrom
tcp/epic-q3-timeouts

Conversation

@alex-clickhouse

@alex-clickhouse alex-clickhouse commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator

Stacked on #590 (tcp/epic-q1-exceptions). Completes Q3 (read timeouts) and Q4 (cancellation).

ReadTimeout now limits each transport read after the handshake. It defaults to 300 seconds; zero disables it. Each successful read allows the next read a full timeout, and time spent processing a yielded block does not count. Expiry throws TimeoutException and discards the connection. Connection establishment remains governed by DialTimeout.

Incomplete responses trigger a best-effort Cancel packet before the connection closes, including caller cancellation, read timeout, and early disposal of result enumeration. Cancel uses an independent two-second flush timeout. It is suppressed during insert row writes, where a partial Data packet could make the server interpret Cancel as block data.

Timeout completion race

A timer can cancel its token after a transport read succeeds but before the read's continuation disarms the timer. Reusing that cancelled token would make the next read fail immediately.

The transport buffer now obtains a token for each read. CancellationTokenSource.TryReset() disarms and reuses the source only when no timeout callback can remain; otherwise the source is disposed and replaced. Parsing, decompression adapters, and writes receive the caller's token. Normal reads reuse the source without adding per-read allocations.

The new deterministic test holds a successful read's continuation until its timer fires, then verifies the next read succeeds, a later stalled read still times out, and caller cancellation still reaches a replacement source. Restoring the old disarming logic makes the regression test fail on the next read. Existing integration tests exercise cancellation on the server, compression, handshake timeouts, and pooled connection reuse. Comments and XML documentation have also been simplified.

Validation

  • Full TCP suite on .NET 9 against ClickHouse 26.7.3.19: 3,531 passed.
  • Focused protocol and live cancellation tests on .NET 8 and .NET 10: 157 passed on each framework.
  • Final deterministic regression cases rerun after test hardening: 2 passed.
  • After restacking, 159 selected protocol and live cancellation tests passed on the stack tip. All 95 descendant commits retained identical stable patch IDs and commit messages.
  • Coverage: IdleReadDeadline.cs and ReadBuffer.cs 100%; ClickHouseTcpConnection.cs 94.9%. Independent coverage review found no important gaps.
  • Independent review, Codex CLI review, and Claude CLI review completed. No correctness findings remain.

Performance

BenchmarkDotNet 0.15.8, .NET 9.0.10, Release, 10 measured iterations. Compares the PR's previous implementation with this fix over synchronous MemoryStream transport reads, using a cancellable caller token and 256 reads per operation. Values below are per transport read.

Read size Before After
64 bytes 114.20 ± 21.35 ns 98.70 ± 10.68 ns
16 KiB 633.35 ± 103.02 ns 678.67 ± 223.17 ns

Errors are BenchmarkDotNet's 99.9% confidence intervals. The intervals overlap; these timings do not establish a regression. Allocations are unchanged (reported as 1 B/read after amortizing operation setup). This isolates buffer/deadline overhead and is not a network throughput benchmark. The temporary benchmark is not committed.

No per-PR changelog fragment, following the approved policy for this unreleased experimental TCP stack.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds native TCP idle-read timeouts and best-effort server cancellation when responses are abandoned.

Changes:

  • Enforces ReadTimeout per transport read, with zero disabling it.
  • Sends Cancel before terminating incomplete queries/inserts.
  • Adds timeout, cancellation, pooling, and idle-semantics coverage.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
ClickHouse.Driver.Tcp/Protocol/ReadBuffer.cs Arms deadlines around transport reads.
ClickHouse.Driver.Tcp/Protocol/IdleReadDeadline.cs Implements reusable idle deadlines.
ClickHouse.Driver.Tcp/Protocol/ClickHouseTcpConnection.cs Integrates deadlines and cancellation packets.
ClickHouse.Driver.Tcp/Client/IConnectionFactory.cs Passes configured read timeout.
ClickHouse.Driver.Tcp/Client/ClickHouseTcpConnectionStringBuilder.cs Documents zero-timeout behavior.
ClickHouse.Driver.Tcp/Client/ClickHouseTcpClientOptions.cs Documents and validates timeout semantics.
ClickHouse.Driver.Tcp.Tests/Utilities/ScriptedDuplexStream.cs Simulates delayed transport reads.
ClickHouse.Driver.Tcp.Tests/Protocol/ClickHouseTcpConnectionQueryTests.cs Tests query timeout and cancellation behavior.
ClickHouse.Driver.Tcp.Tests/Protocol/ClickHouseTcpConnectionInsertTests.cs Tests insert cancellation boundaries.
ClickHouse.Driver.Tcp.Tests/Integration/ClickHouseTcpCancellationIntegrationTests.cs Verifies server-side cancellation and pooling.
ClickHouse.Driver.Tcp.Tests/Client/ClickHouseTcpClientOptionsTests.cs Covers updated timeout validation.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread ClickHouse.Driver.Tcp/Protocol/ClickHouseTcpConnection.cs Outdated
Comment thread ClickHouse.Driver.Tcp/Protocol/IdleReadDeadline.cs Outdated
Comment thread ClickHouse.Driver.Tcp/Client/ClickHouseTcpClientOptions.cs Outdated
Comment thread ClickHouse.Driver.Tcp/Protocol/ClickHouseTcpConnection.cs Outdated
Comment thread ClickHouse.Driver.Tcp/Protocol/ClickHouseTcpConnection.cs Outdated
@codecov

codecov Bot commented Aug 26, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@alex-clickhouse
alex-clickhouse force-pushed the tcp/epic-q3-timeouts branch 2 times, most recently from 1535b9f to 88971a0 Compare September 1, 2026 08:56
@alex-clickhouse
alex-clickhouse force-pushed the tcp/epic-q3-timeouts branch 2 times, most recently from 4bd8906 to 78dfbbf Compare September 3, 2026 09:25
@alex-clickhouse
alex-clickhouse force-pushed the tcp/epic-q3-timeouts branch 2 times, most recently from e8a2481 to f276df8 Compare September 4, 2026 09:01
@alex-clickhouse
alex-clickhouse force-pushed the tcp/epic-q3-timeouts branch 2 times, most recently from 3eb6336 to d15f8ac Compare September 8, 2026 10:11
@alex-clickhouse
alex-clickhouse marked this pull request as ready for review September 8, 2026 12:27

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 26e60d4. Configure here.

Comment thread ClickHouse.Driver.Tcp/Protocol/IdleReadDeadline.cs
@alex-clickhouse
alex-clickhouse force-pushed the tcp/epic-q3-timeouts branch 2 times, most recently from fa47d22 to f920c00 Compare September 9, 2026 09:39
…s abandoned

ReadTimeout was parsed, stored and read by nothing. It now bounds every read of
an operation, armed immediately before each read from the transport and disarmed
as soon as that read returns, so it measures silence rather than duration: a
result that streams for an hour never trips it, and neither does a consumer that
holds a block longer than the deadline. TimeSpan.Zero disables it, as it does for
the pool's limits.

Giving up on a result now sends the Cancel packet before closing the connection,
so the server stops rather than finishing a query nobody reads. That covers
cancellation, a read that gave up, and a consumer that breaks out of the
enumeration. The insert row phase is excluded: a block is part-written there, so
an appended Cancel would be read as more block bytes.

Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants