Skip to content

*: update kvproto to apiv3#11024

Open
disksing wants to merge 13 commits into
masterfrom
apiv3-update-proto
Open

*: update kvproto to apiv3#11024
disksing wants to merge 13 commits into
masterfrom
apiv3-update-proto

Conversation

@disksing

@disksing disksing commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: close #11025

The kvproto APIV3 branch changes several keyspace-related protobuf fields to oneofs. PD needs to consume the latest definitions and adapt the affected requests, responses, and test fixtures.

What is changed and how does it work?

  • Update kvproto to fd32127adca58438f6b05c869951182bbdd6041c.
  • Adapt TSO headers, keyspace metadata, GC safe point requests, and related mocks/tests to the generated oneof APIs.
  • Embed the generated unimplemented keyspace server where required.

Check List

Tests

  • Unit test
    • DASHBOARD=0 make pd-server-basic
    • make gotest GOTEST_ARGS='./pkg/gc ./pkg/keyspace ./pkg/storage ./pkg/mcs/resourcemanager/server ./server/...'
    • make -C client gotest GOTEST_ARGS="./clients/tso -run 'TestTSOStreamSuite|TestTSODispatcherTestSuite' -count=1"
    • make -C client gotest GOTEST_ARGS='./servicediscovery -count=1'

Code changes

  • Has the configuration change
  • Has HTTP APIs changed
  • Has persistent data change

Side effects

  • Possible performance regression
  • Increased code complexity
  • Breaking backward compatibility: updates generated protobuf APIs to the APIV3 definitions

Release note

None.

Summary by CodeRabbit

  • Bug Fixes
    • Improved compatibility with updated keyspace/TSO/gRPC protobuf interfaces by aligning keyspace ID encoding and accessors (including pagination tokens, GC safe points/states, and TSO request forwarding).
    • Ensured keyspace IDs are preserved consistently when serializing/deserializing metadata and scoping information.
    • Added the standard gRPC “unimplemented” stub for the keyspace service.
  • Tests
    • Updated test fixtures and assertions to use protobuf getter/wrapper forms for keyspace identifiers.
  • Chores
    • Updated the kvproto dependency to a newer revision.

Signed-off-by: disksing <i@disksing.com>
@ti-chi-bot ti-chi-bot Bot added do-not-merge/needs-linked-issue release-note-none Denotes a PR that doesn't merit a release note. dco-signoff: yes Indicates the PR's author has signed the dco. labels Jul 20, 2026
@ti-chi-bot

ti-chi-bot Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign okjiang for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot Bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Jul 20, 2026
@coderabbitai

coderabbitai Bot commented Jul 20, 2026

Copy link
Copy Markdown

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

The PR updates kvproto and migrates keyspace, GC, TSO, and resource manager code from direct protobuf fields to generated oneof wrappers and getter accessors. Related tests, API serialization, dependency files, and keyspace gRPC compatibility are updated.

Changes

kvproto oneof migration

Layer / File(s) Summary
Protobuf dependency and API boundaries
client/go.mod, go.mod, tests/integrations/go.mod, tools/go.mod, client/http/types.go, client/keyspace_client.go, server/..., tools/pd-ctl/...
Updates kvproto versions and adapts keyspace metadata, requests, accessors, API serialization, server compatibility, and client validation.
Keyspace metadata and storage migration
pkg/keyspace/..., pkg/storage/keyspace_test.go, pkg/gc/..., tests/server/gc/...
Uses ID oneof wrappers and getter accessors throughout keyspace creation, assignment, persistence, GC state handling, and tests.
GC keyspace scope conversion
client/gc_client.go, server/gc_service.go, tests/integrations/client/gc_test.go
Updates GC scope encoding and keyspace ID extraction for the revised protobuf fields.
TSO keyspace header conversion
client/clients/tso/stream.go, client/servicediscovery/..., server/forward.go, pkg/utils/tsoutil/..., tests/integrations/mcs/tso/...
Encodes TSO keyspace IDs through the request-header oneof wrapper across stream, discovery, forwarding, and factory paths.
Resource manager keyspace ID conversion
client/resource_manager_client.go, client/resource_group/..., pkg/mcs/resourcemanager/..., tests/integrations/mcs/resourcemanager/...
Uses nested keyspace ID wrappers and generated accessors in resource manager requests, responses, persistence, and tests.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Possibly related PRs

  • tikv/pd#10597 — Both modify keyspace creation codepaths in pkg/keyspace/keyspace.go.
  • tikv/pd#10911 — Both modify LoadKeyspaceByID protobuf request and response handling.

Suggested reviewers: jmpotato, lhy1024, rleungx, okjiang, amoebaprotozoa

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 12.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title is concise and accurately reflects the kvproto APIV3 update.
Description check ✅ Passed The description includes the required issue reference, change summary, checklist, side effects, and release note.
Linked Issues check ✅ Passed The diff updates kvproto and migrates the listed TSO, keyspace, GC safe point, HTTP, and pd-ctl call sites to oneof APIs.
Out of Scope Changes check ✅ Passed The changes are focused on the APIV3 protobuf migration and related tests, with no clear unrelated additions.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch apiv3-update-proto

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
server/apiv2/handlers/keyspace.go (1)

300-300: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Prefer strconv.FormatUint over casting uint32 to int.

While casting to int is safe on 64-bit architectures, it is generally more idiomatic to use unsigned formatting for unsigned integers rather than casting them to signed variants.

♻️ Proposed refactor
-		resp.NextPageToken = strconv.Itoa(int(scanned[len(scanned)-1].GetId()))
+		resp.NextPageToken = strconv.FormatUint(uint64(scanned[len(scanned)-1].GetId()), 10)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@server/apiv2/handlers/keyspace.go` at line 300, Update the NextPageToken
assignment in the keyspace handler to format scanned[len(scanned)-1].GetId()
with strconv.FormatUint, converting it only to the required unsigned integer
type and removing the uint32-to-int cast.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@pkg/keyspace/keyspace.go`:
- Around line 553-561: Move the manager.keyspaceNameLookup.Store call out of the
RunInTxn callback and perform it only after the enclosing transaction returns
successfully. Keep the storage update inside the transaction, and ensure the
cache is not mutated when the transaction fails or commit encounters a CAS
mismatch.

---

Nitpick comments:
In `@server/apiv2/handlers/keyspace.go`:
- Line 300: Update the NextPageToken assignment in the keyspace handler to
format scanned[len(scanned)-1].GetId() with strconv.FormatUint, converting it
only to the required unsigned integer type and removing the uint32-to-int cast.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 2146e9a5-68fc-4eef-aebb-6a5b35f47316

📥 Commits

Reviewing files that changed from the base of the PR and between 0b37df9 and 895c3ca.

⛔ Files ignored due to path filters (4)
  • client/go.sum is excluded by !**/*.sum
  • go.sum is excluded by !**/*.sum
  • tests/integrations/go.sum is excluded by !**/*.sum
  • tools/go.sum is excluded by !**/*.sum
📒 Files selected for processing (21)
  • client/clients/tso/stream.go
  • client/gc_client.go
  • client/go.mod
  • client/http/types.go
  • client/keyspace_client.go
  • client/keyspace_client_test.go
  • client/servicediscovery/tso_service_discovery.go
  • go.mod
  • pkg/gc/gc_state_manager.go
  • pkg/gc/gc_state_manager_test.go
  • pkg/keyspace/keyspace.go
  • pkg/keyspace/keyspace_test.go
  • pkg/mcs/resourcemanager/server/manager_test.go
  • pkg/storage/keyspace_test.go
  • pkg/utils/tsoutil/tso_proto_factory.go
  • server/apiv2/handlers/keyspace.go
  • server/forward.go
  • server/gc_service.go
  • server/keyspace_service.go
  • tests/integrations/go.mod
  • tools/go.mod

Comment thread pkg/keyspace/keyspace.go
Signed-off-by: disksing <i@disksing.com>
Signed-off-by: disksing <i@disksing.com>
Signed-off-by: disksing <i@disksing.com>
@ti-chi-bot ti-chi-bot Bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jul 20, 2026
Signed-off-by: disksing <i@disksing.com>
@codecov

codecov Bot commented Jul 20, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 90.00000% with 9 lines in your changes missing coverage. Please review.
✅ Project coverage is 79.21%. Comparing base (0b37df9) to head (9975cac).
⚠️ Report is 3 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #11024      +/-   ##
==========================================
- Coverage   79.23%   79.21%   -0.03%     
==========================================
  Files         541      540       -1     
  Lines       76035    76020      -15     
==========================================
- Hits        60245    60217      -28     
- Misses      11545    11564      +19     
+ Partials     4245     4239       -6     
Flag Coverage Δ
unittests 79.21% <90.00%> (-0.03%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@disksing

Copy link
Copy Markdown
Contributor Author

/test pull-unit-test-next-gen-3

Signed-off-by: disksing <i@disksing.com>
@ti-chi-bot ti-chi-bot Bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Jul 22, 2026
disksing added 5 commits July 24, 2026 14:03
Signed-off-by: disksing <i@disksing.com>
Signed-off-by: disksing <i@disksing.com>
Signed-off-by: disksing <i@disksing.com>
Signed-off-by: disksing <i@disksing.com>
Signed-off-by: disksing <i@disksing.com>
@disksing

Copy link
Copy Markdown
Contributor Author

/test pull-unit-test-next-gen-3

@disksing
disksing force-pushed the apiv3-update-proto branch from 9ca1e48 to a12bf1f Compare July 24, 2026 07:27
@disksing

Copy link
Copy Markdown
Contributor Author

/test pull-unit-test-next-gen-1

@disksing

Copy link
Copy Markdown
Contributor Author

/test pull-unit-test-next-gen-3

@disksing
disksing force-pushed the apiv3-update-proto branch 2 times, most recently from e3981f2 to 8748a5a Compare July 24, 2026 07:59
Signed-off-by: disksing <i@disksing.com>
@disksing
disksing force-pushed the apiv3-update-proto branch from 8748a5a to 0b59efc Compare July 24, 2026 08:05
@disksing

Copy link
Copy Markdown
Contributor Author

/test pull-unit-test-next-gen-2 pull-unit-test-next-gen-3

@disksing

Copy link
Copy Markdown
Contributor Author

/test pull-unit-test-next-gen-2

Signed-off-by: disksing <i@disksing.com>
@disksing

Copy link
Copy Markdown
Contributor Author

/test pull-integration-realcluster-test

1 similar comment
@disksing

Copy link
Copy Markdown
Contributor Author

/test pull-integration-realcluster-test

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dco-signoff: yes Indicates the PR's author has signed the dco. release-note-none Denotes a PR that doesn't merit a release note. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Update kvproto dependency to APIV3 definitions

1 participant