executor, store/copr: batch full-sampling Analyze requests

[0xTars]

What problem does this PR solve?

Issue Number: ref #70059

Problem Summary:

V2 full-sampling ANALYZE normally sends one coprocessor request per TiKV
Region. Large physical tables can therefore create many small requests.
Same-store batching can reduce that RPC count, but the batching policy must use
the request's actual resolved Regions, preserve the configured Analyze
physical-scan budget, and keep retry and non-Analyze paths unchanged.

This replaces #69686 because its source branch cannot be updated. Issue #70059
tracks the cross-repository batching contract and landing gates.

What changed and how does it work?

• Mark V2 full-sampling Analyze requests as able to consume merged child
  responses. The existing generic DAG path is unchanged.

• Resolve all KeyRanges partitions for one Analyze kv.Request exactly once,
  deduplicate their Region IDs, select one request-local width, and pack each
  partition independently. Batches never cross stores or partition-builder
  boundaries, and descending response order is preserved.

• Let N be the number of distinct Regions covered by one Analyze
  kv.Request (one nonpartitioned table or one current physical partition),
  and let C=max(resolved Analyze scan concurrency,1). The selected maximum
  same-store Region width is:

  if N <= 2*C:
      B = 1
  else:
      B = min(ceil(N/C), max { b | b*(b+1) <= C })
  

  The implementation uses overflow-safe integer comparisons. B=1 means
  singleton requests. The triangular cap guarantees
  floor(C/B) >= B+1 for a batched width.

• Full-sampling Analyze does not use tidb_store_batch_size; generic DAG
  batching still does.

• Derive iterator concurrency from actual built task weights
  (1 + child count) with a dynamic weight histogram. The admitted heaviest
  prefix never exceeds C, so the scalar worker count is safe for every
  scheduler order and for widths greater than five.

• Apply the request-local policy only during the initial build. Region-error
  and lock retries rebuild singleton tasks. Fallback counters count each
  original failed child once even when one child splits into several retry
  tasks.

• Keep unsigned integer-PK full-sampling Analyze in one request across
  MaxInt64. The sampler reconstructs order itself, so this avoids two
  independent scan budgets without changing histogram bounds or correlation.

The runtime evidence is deliberately limited:

• N=385,C=64 selected B=7 and emitted the expected 56 of 56 table-Region
  requests with zero retries.
• One N=449,C=64 resource comparison emitted 66 requests instead of 449
  (-85.3%). Its single paired wall-clock delta was +0.65%, which is not a
  performance conclusion. Statistics and checksums were identical; both arms
  stayed under the predefined resource gates and showed no backoff, OOM, swap,
  message-size, or fatal signal.
• Existing N=257,B=5 twelve-pair evidence was performance-neutral. The
  N=449,B=7 result is one pair only. This PR does not claim universal speedup
  or memory savings.
• Multi-round B=3 and B=7, more than four stores, and a real-cluster
  N<=2C gate remain unmeasured.

The temporary dependency revisions are validation pins:

• kvproto: a29e794b680d (current compatible base plus the merged-child
  response contract)
• client-go: 949253a114a6 (current compatible base plus nested child resource
  accounting)

This Draft cannot land until the corresponding upstream protocol and
client-go changes land and all consumers are repinned to mutually compatible
upstream revisions. These contributor refs are not a dependency exit.

Check List

Tests

• Unit test
  • ./tools/check/failpoint-go-test.sh pkg/executor -run '^TestAnalyzeBuildsSingleBatchableRequest$' -count=1
  • ./tools/check/failpoint-go-test.sh pkg/store/copr -run '^(TestBuildTasksWithoutBuckets|TestAnalyzeStoreBatchPolicy|TestAnalyzeStoreBatchWidthUsesRequestRegionCount|TestAnalyzeStoreBatchWidthMatchesSmallIntegerReference|TestHandleBatchCopResponseMergedAndUnansweredTasks|TestHandleBatchCopResponseUpdatesChildBucketsOnVersionNotMatch)$' -count=1
  • ./tools/check/failpoint-go-test.sh pkg/store/copr/copr_test -run '^(TestAnalyzeBatchUsesRequestWideDistinctRegionCount|TestBuildCopIteratorWithBatchStoreCopr|TestBuildCopIteratorWithRunawayChecker)$' -count=1
  • make fmt
  • make lint
  • go mod tidy (zero diff)
  • go mod verify
  • make bazel_prepare (two consecutive zero-diff runs)
  • git diff --check 93f713cf551f..HEAD
• Integration test
• Manual test
  • A pinned 1-PD/4-TiKV/1-TiDB fixture produced the bounded runtime evidence
    listed above; statistics and checksums matched.
• No need to test

  • I checked and no code files have been changed.

Side effects

• Performance regression: Consumes more CPU
• Performance regression: Consumes more Memory
• Breaking backward compatibility

Documentation

• Affects user behaviors
• Contains syntax changes
• Contains variable changes
• Contains experimental features
• Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

Reduce the number of TiKV requests used by V2 full-sampling ANALYZE for large
physical tables while preserving the configured Analyze scan-concurrency
budget.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 9910e5b6-3119-4e02-9753-2caecc418320

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[ti-chi-bot]

Hi @0xTars. Thanks for your PR.

I'm waiting for a pingcap member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[pingcap-cla-assistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.

---

Murphy seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign cfzjywxk, terry1purcell for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS
• pkg/distsql/OWNERS
• pkg/executor/OWNERS
• pkg/sessionctx/variable/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment