Skip to content

util/async: avoid blocking Append on Exec cancellation - #2034

Open
zyguan wants to merge 1 commit into
tikv:masterfrom
zyguan:fix/issue-2033
Open

util/async: avoid blocking Append on Exec cancellation#2034
zyguan wants to merge 1 commit into
tikv:masterfrom
zyguan:fix/issue-2033

Conversation

@zyguan

@zyguan zyguan commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Fix #2033

RunLoop.Append could block indefinitely when racing with cancellation of a waiting RunLoop.Exec.

Append changes the state from StateWaiting to StateIdle under the mutex, then sends a notification through the
unbuffered ready channel after releasing the mutex. If Exec selected ctx.Done() and returned before receiving that
notification, the Append caller was left blocked. When invoked from an async batch response callback, this could stop
the batch receive loop from retiring the request and processing later responses on the same stream.

Changes:

  • Make the cancellation path distinguish between a run loop that is still waiting and one for which an Append has
    already committed to sending a notification.
  • Consume the committed notification before returning from Exec, ensuring the notifying Append can complete.
  • Document the notification handshake and its single-Exec safety invariant.
  • Add regression coverage for the cancellation race, queued-task preservation, and subsequent Exec behavior.
  • Make the concurrent Exec test deterministic by synchronizing on the first executor entering StateRunning.

Summary by CodeRabbit

  • Bug Fixes

    • Improved run loop behavior when execution is cancelled while another task is being added.
    • Prevented potential hangs during concurrent task scheduling.
    • Ensured the run loop remains usable for subsequent executions after cancellation.
  • Tests

    • Added coverage for cancellation and concurrent task execution scenarios.

fix tikv#2033

Signed-off-by: zyguan <zhongyangguan@gmail.com>
@ti-chi-bot ti-chi-bot Bot added dco-signoff: yes Indicates the PR's author has signed the dco. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jul 28, 2026
@coderabbitai

coderabbitai Bot commented Jul 28, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 8973088b-08dd-41f9-b51e-6e04feff8f92

📥 Commits

Reviewing files that changed from the base of the PR and between 61ecd7c and 392411f.

📒 Files selected for processing (2)
  • util/async/runloop.go
  • util/async/runloop_test.go

📝 Walkthrough

Walkthrough

RunLoop.Exec now handles cancellation races with Append by consuming pending notifications when necessary. Tests add deterministic coverage for cancellation unblocking and concurrent execution outcomes.

Changes

RunLoop cancellation

Layer / File(s) Summary
Cancellation notification handling
util/async/runloop.go
Documents the state/ready handshake and updates Exec to consume a pending notification when cancellation races with Append.
Concurrency regression coverage
util/async/runloop_test.go
Adds cancellation coverage ensuring Append unblocks and subsequent execution succeeds, and replaces timing-based concurrent execution checks with channel-coordinated assertions.

Estimated code review effort: 3 (Moderate) | ~20 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the RunLoop.Append/Exec cancellation fix.
Linked Issues check ✅ Passed The changes address #2033 by preventing Exec cancellation from strand­ing Append and add regression coverage for the race.
Out of Scope Changes check ✅ Passed All changes stay within util/async and directly support the cancellation-race fix and tests.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@zyguan

zyguan commented Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

@D3Hunter @cfzjywxk @lcwangchao PTAL

@ti-chi-bot ti-chi-bot Bot added the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label Jul 28, 2026
@ti-chi-bot

ti-chi-bot Bot commented Jul 28, 2026

Copy link
Copy Markdown

[LGTM Timeline notifier]

Timeline:

  • 2026-07-28 09:23:29.37267314 +0000 UTC m=+1915195.408768196: ☑️ agreed by cfzjywxk.

@ti-chi-bot

ti-chi-bot Bot commented Jul 28, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cfzjywxk

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot Bot added the approved label Jul 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved dco-signoff: yes Indicates the PR's author has signed the dco. needs-1-more-lgtm Indicates a PR needs 1 more LGTM. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

util/async: cancellation race can block RunLoop.Append and batchRecvLoop indefinitely

2 participants