Skip to content

valkey - test: Run cluster tests on their own Valkey cluster - #2180

Merged
jaredwray merged 1 commit into
mainfrom
claude/wizardly-lamport-lyi7hy
Oct 1, 2026
Merged

jaredwray merged 1 commit into
mainfrom
claude/wizardly-lamport-lyi7hy

Conversation

@jaredwray

Copy link
Copy Markdown
Owner

Please check if the PR fulfills these requirements

  • Followed the Contributing and Code of Conduct guidelines.
  • Tests for the changes have been added (for bug fixes/features) with 100% code coverage. No source changes: this fixes the test setup, and @keyv/valkey's coverage is the same as on main.

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)
Test infrastructure fix (a CI race).

Problem

pnpm test:ci runs every package's tests at the same time. @keyv/redis and @keyv/valkey both ran their cluster tests against the one Redis cluster on ports 7001-7003. In each of the seven CI job logs I checked, the two cluster test files finished within a second of each other. Before each of its 23 tests, the Redis suite clears the whole cluster (clear() with noNamespaceAffectsAll, which runs FLUSHDB on every master). Keys the Valkey tests had just written disappeared mid-test.

On #2179 this failed the node-26 job. should track keys with useSets without CROSSSLOT errors failed all four attempts at expect(await store.delete(keys[0])).toBe(true): the key set one line earlier was already gone. The same run retried three more Valkey cluster tests (deleteMany, clear, iterator) that then passed. The per-test { retry: 3 } was hiding most of the hits.

Local reproduction: a three-node cluster on 7001-7003, with @keyv/redis's own clear() running in a loop (what its beforeEach does) while the Valkey cluster tests run with retries off. 3 of 5 runs failed.

Changes

  • scripts/docker-compose-valkey-cluster.yaml (new):
    • a three-node Valkey 9.1.0 cluster on 7101-7103, with bus ports 17101-17103;
    • uses the same image and digest as the standalone keyv_valkey service;
    • built like the Redis cluster: host networking, plus an init container that runs valkey-cli --cluster create.
  • scripts/test-services-start.sh and test-services-stop.sh: start and stop the new cluster with the other services, on x86 and ARM. Every workflow that needs services (tests, codecov, bun-test, release) goes through pnpm test:services:start.
  • storage/valkey/test/cluster.test.ts:
    • connect to 7101-7103, so the Valkey adapter's cluster code is now tested against Valkey instead of Redis;
    • drop the per-test { retry: 3 } that was hiding the race. The package-wide retry: 2 in vitest.config.ts stays.
  • AGENTS.md and CONTRIBUTING.md:
    • host networking is needed for both clusters;
    • packages must not share a cluster, and a new adapter that needs one gets its own.

No public API or adapter behavior changes, so the migration guide and skill are unchanged.

For local runs: after pulling this, run pnpm test:services:start again to start the new cluster. docker compose up -d adds it without touching the running services.

Verification

  • Local race after the change: Redis clear() looping on 7001-7003 while the Valkey tests ran on 7101-7103. 0 of 10 runs failed, with 1,050 flushes during the runs.
    • There's no Docker here, so the second local cluster was redis-server 7.0 standing in for Valkey. CI runs the real Valkey image.
  • pnpm test:ci in storage/valkey: 158/158 pass, lint is clean, and coverage matches main's CI.
  • docker compose config: parses the merged files for both the x86 and ARM service sets.
  • Branch run before opening this PR: dispatched tests run green on node 22, 24 and 26.
    • All four Valkey cluster containers started in every job.
    • The Valkey cluster tests passed 11/11 with no retries.
    • The run's only retry was an unrelated Postgres test.

🤖 Generated with Claude Code

https://claude.ai/code/session_01X5LJCvt3pkR7FtfyAdzg5x


Generated by Claude Code

`pnpm test:ci` runs every package's tests at once, and @keyv/redis and
@keyv/valkey both ran their cluster tests against the Redis cluster on
7001-7003. The Redis tests clear that cluster with FLUSHDB before each
test, so Valkey test keys vanished mid-test. On #2179 this failed the
node-26 job; retries had already masked three other hits in that run.

Add a three-node Valkey 9.1.0 cluster on 7101-7103, started and
stopped with the other test services, and point the Valkey cluster
tests at it. They now also run against Valkey rather than Redis. Drop
the per-test retries that were hiding the race.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X5LJCvt3pkR7FtfyAdzg5x
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 30, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-30T10:21:53.403236Z bb93eda PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@codecov

codecov Bot commented Sep 30, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (12415c5) to head (bb93eda).

Additional details and impacted files
@@            Coverage Diff            @@
##              main     #2180   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           56        56           
  Lines         5790      5790           
  Branches       997       989    -8     
=========================================
  Hits          5790      5790           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@jaredwray
jaredwray merged commit 53d0658 into main Oct 1, 2026
19 checks passed
@jaredwray
jaredwray deleted the claude/wizardly-lamport-lyi7hy branch October 1, 2026 05:14
jaredwray added a commit that referenced this pull request Oct 3, 2026
Seven tests in six adapters set a value that expires in 100 ms, then read
it back expecting it to still be there. When set() and get() together
take longer than 100 ms under CI load, the read finds the value expired.
The postgres copy needed a retry in the #2180 branch run.

- postgres, mysql, sqlite and cloudflare-kv decide expiry with
  Date.now(), so these tests now freeze Date and move it past the TTL
  instead of sleeping. They can't race the clock, and run 200 ms faster.
- redis and valkey expire keys on the server's clock, so those tests get
  a 1 s window instead of 100 ms.

The postgres and mysql copies also turn off Keyv's own expiry check,
as the sqlite copy already did. With it on, Keyv deleted the expired
value itself, so the tests passed even when the adapter left the
expires column empty, the bug they are meant to catch.


Claude-Session: https://claude.ai/code/session_01X5LJCvt3pkR7FtfyAdzg5x

Co-authored-by: Claude <noreply@anthropic.com>
jaredwray pushed a commit that referenced this pull request Oct 3, 2026
Nine storage packages retried every failed test twice. That hid real
problems: the valkey cluster tests were losing keys to @keyv/redis's
FLUSHDB (#2180) and several tests raced a 100 ms TTL (#2182), and both
only surfaced when a test failed three times in a row. With those fixed,
a test that needs a retry to pass should fail CI so it gets fixed.

Remove `retry: 2` from the vitest configs of cloudflare-kv, dynamo,
etcd, memcache, mongo, mysql, postgres, redis and valkey. The Cloudflare
KV live config keeps its retries: it calls the real Cloudflare API over
the internet.

AGENTS.md now says not to add retries to get CI green.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X5LJCvt3pkR7FtfyAdzg5x
jaredwray added a commit that referenced this pull request Oct 3, 2026
…2184)

Nine storage packages retried every failed test twice. That hid real
problems: the valkey cluster tests were losing keys to @keyv/redis's
FLUSHDB (#2180) and several tests raced a 100 ms TTL (#2182), and both
only surfaced when a test failed three times in a row. With those fixed,
a test that needs a retry to pass should fail CI so it gets fixed.

Remove `retry: 2` from the vitest configs of cloudflare-kv, dynamo,
etcd, memcache, mongo, mysql, postgres, redis and valkey. The Cloudflare
KV live config keeps its retries: it calls the real Cloudflare API over
the internet.

AGENTS.md now says not to add retries to get CI green.


Claude-Session: https://claude.ai/code/session_01X5LJCvt3pkR7FtfyAdzg5x

Co-authored-by: Claude <noreply@anthropic.com>
@jaredwray jaredwray mentioned this pull request Oct 4, 2026
8 of 10 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants