Skip to content

kafka(ticdc): ddl sink close the underline sink if send ddl or checkpoint failed and refactor the kafka ddl sink#12112

Merged
ti-chi-bot[bot] merged 1 commit into
pingcap:masterfrom
3AceShowHand:close-ddl-sink
Apr 2, 2025
Merged

kafka(ticdc): ddl sink close the underline sink if send ddl or checkpoint failed and refactor the kafka ddl sink#12112
ti-chi-bot[bot] merged 1 commit into
pingcap:masterfrom
3AceShowHand:close-ddl-sink

Conversation

@3AceShowHand

@3AceShowHand 3AceShowHand commented Mar 26, 2025

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: close #12666, close #12096

What is changed and how it works?

  • owner/ddl_sink should close if meet any error, and rebuilt by the retry.
  • kafka factory implementations should wrap error immediately, and caller should just return error, no need to trace the error.
  • kafka ddl sink use encoder directly, instead of encoderBuilder
  • syncProducer no need to close in a goroutine, since it only deliver one message at a time, won't be blocked if the downstream kafka cluster unavailable.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Questions

Will it cause performance regression or break compatibility?
Do you need to update user documentation, design documentation or monitoring documentation?

Release note

`None`

@ti-chi-bot ti-chi-bot Bot added do-not-merge/needs-triage-completed release-note-none Denotes a PR that doesn't merit a release note. labels Mar 26, 2025
@3AceShowHand 3AceShowHand requested a review from hicqu March 26, 2025 07:35
@ti-chi-bot ti-chi-bot Bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Mar 26, 2025
@ti-chi-bot ti-chi-bot Bot added approved needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Mar 26, 2025
Comment thread pkg/sink/kafka/factory.go
@3AceShowHand 3AceShowHand requested review from hicqu and wk989898 April 2, 2025 06:25
@ti-chi-bot

ti-chi-bot Bot commented Apr 2, 2025

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hongyunyan, wk989898

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:
  • OWNERS [hongyunyan,wk989898]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot Bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Apr 2, 2025
@ti-chi-bot

ti-chi-bot Bot commented Apr 2, 2025

Copy link
Copy Markdown
Contributor

[LGTM Timeline notifier]

Timeline:

  • 2025-03-26 07:45:34.95632763 +0000 UTC m=+1032828.640563726: ☑️ agreed by wk989898.
  • 2025-04-02 06:55:06.502853958 +0000 UTC m=+1634600.187090054: ☑️ agreed by hongyunyan.

@3AceShowHand

Copy link
Copy Markdown
Contributor Author

/retest

@3AceShowHand

Copy link
Copy Markdown
Contributor Author

/cherry-pick release-8.5

@ti-chi-bot

Copy link
Copy Markdown
Member

@3AceShowHand: new pull request created to branch release-8.5: #12677.
But this PR has conflicts, please resolve them!

Details

In response to this:

/cherry-pick release-8.5

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot

Copy link
Copy Markdown
Member

In response to a cherrypick label: new pull request created to branch release-8.1: #12679.
But this PR has conflicts, please resolve them!

@ti-chi-bot

Copy link
Copy Markdown
Member

In response to a cherrypick label: new pull request created to branch release-7.5: #12680.
But this PR has conflicts, please resolve them!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved lgtm needs-cherry-pick-release-7.5 Should cherry pick this PR to release-7.5 branch. needs-cherry-pick-release-8.1 Should cherry pick this PR to release-8.1 branch. release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

5 participants