Skip to content

Fix gossip client timeout: TimeSpan .Milliseconds vs .TotalMilliseconds#1972

Merged
tiagonapoli merged 1 commit into
mainfrom
tiagonapoli/fix-gossip-client-timeout
Jul 23, 2026
Merged

Fix gossip client timeout: TimeSpan .Milliseconds vs .TotalMilliseconds#1972
tiagonapoli merged 1 commit into
mainfrom
tiagonapoli/fix-gossip-client-timeout

Conversation

@tiagonapoli

Copy link
Copy Markdown
Collaborator

Summary

Fixes a TimeSpan component-vs-total misuse that silently disables the gossip client's operation timeout, plus related occurrences of the same defect pattern.

Bug 1 (behavioral) — gossip client timeout never fires

GarnetServerNode built the gossip GarnetClient with:

timeoutMilliseconds: opts.ClusterTimeout <= 0 ? 0 : TimeSpan.FromSeconds(opts.ClusterTimeout).Milliseconds

TimeSpan.Milliseconds returns the sub-second component (0–999), not the total duration. For any whole-second ClusterTimeout (the type is int seconds, default 60), this evaluates to 0, which disables GarnetClient's TimeoutChecker. Result: gossip operations effectively never time out on the per-operation client timeout.

Fix: compute the value with TotalMilliseconds, extracted into a testable helper:

internal static int GetClientTimeoutMilliseconds(int clusterTimeoutSeconds)
    => clusterTimeoutSeconds <= 0 ? 0 : (int)TimeSpan.FromSeconds(clusterTimeoutSeconds).TotalMilliseconds;

Same-pattern cleanups

  • GarnetServerOptions.LoggingFrequency was TimeSpan.FromSeconds(5).Seconds — correct only because 5 < 60, but a latent trap. Simplified to the plain int-seconds literal 5 (value unchanged).
  • RespAofTests (RMW + Upsert recover tests) asserted recoveredValueTtl.Value.Milliseconds < 8500. Since the milliseconds component is always 0–999, this assertion was a silent no-op. Corrected to TotalMilliseconds with the real ceiling of 10_000 (Key2 is set with a 10s expiry), so it now actually validates 0 < ttl ≤ 10s.

Tests

  • New GarnetServerNodeTimeoutTests (7 cases) directly covers GetClientTimeoutMilliseconds, including the regression case 60 → 60000 (returns 0 under the old code).
  • Verified by reverting the fix: 4/7 timeout tests fail on revert; both RespAofTests continue to pass with the strengthened assertions.

Scope

Isolated fix branch off main — contains only these four files. No functional changes beyond the timeout correctness fix and the same-pattern cleanups.

@tiagonapoli
tiagonapoli force-pushed the tiagonapoli/fix-gossip-client-timeout branch 3 times, most recently from 2c72b0a to eb1cec2 Compare July 23, 2026 05:38
@tiagonapoli
tiagonapoli marked this pull request as ready for review July 23, 2026 05:41
Copilot AI review requested due to automatic review settings July 23, 2026 05:41

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes a TimeSpan component-vs-total misuse that caused the cluster gossip client operation timeout to be computed as 0 for whole-second values (disabling GarnetClient’s timeout checker). Also cleans up related same-pattern issues and adds regression coverage.

Changes:

  • Add GarnetServerNode.GetClientTimeoutMilliseconds(int) and use it when constructing the gossip GarnetClient.
  • Replace LoggingFrequency’s TimeSpan.FromSeconds(5).Seconds with the equivalent literal 5.
  • Strengthen AOF recovery TTL assertions to use TotalMilliseconds and add a new regression test fixture for timeout computation.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File Description
test/standalone/Garnet.test.scripting/RespAofTests.cs Fix TTL assertions to use total duration rather than component milliseconds.
test/cluster/Garnet.test.cluster/GarnetServerNodeTimeoutTests.cs Add regression tests for gossip client timeout computation.
libs/server/Servers/GarnetServerOptions.cs Simplify LoggingFrequency initialization to a literal seconds value.
libs/cluster/Server/Gossip/GarnetServerNode.cs Fix gossip client timeout computation by using total milliseconds via a helper.

Comment thread libs/cluster/Server/Gossip/GarnetServerNode.cs Outdated
Comment thread test/cluster/Garnet.test.cluster/GarnetServerNodeTimeoutTests.cs
GarnetServerNode computed the gossip GarnetClient's per-operation timeout
as `TimeSpan.FromSeconds(opts.ClusterTimeout).Milliseconds`, which returns
the sub-second component (0-999), not the total. For any whole-second
ClusterTimeout (default 60s) this yields 0, silently disabling the
client's TimeoutChecker so gossip operations never time out.

- GarnetServerNode: use TotalMilliseconds via a testable
  GetClientTimeoutMilliseconds helper.
- Add GarnetServerNodeTimeoutTests (7 cases) covering the regression.
- GarnetServerOptions: LoggingFrequency used TimeSpan.FromSeconds(5).Seconds
  (correct only because 5<60); simplify to the int-seconds literal.
- RespAofTests: TTL assertions used .Milliseconds (component, always <8500),
  a silent no-op; use .TotalMilliseconds with the correct 10s ceiling.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: de1c4e90-5293-4056-8783-1779a318e774
@tiagonapoli
tiagonapoli force-pushed the tiagonapoli/fix-gossip-client-timeout branch from eb1cec2 to 037b8e9 Compare July 23, 2026 05:49
@tiagonapoli
tiagonapoli merged commit 15a90b7 into main Jul 23, 2026
316 of 317 checks passed
@tiagonapoli
tiagonapoli deleted the tiagonapoli/fix-gossip-client-timeout branch July 23, 2026 15:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants