pulsar: support debezium protocol#5054
Conversation
Extend IsPulsarSupportedProtocols to include ProtocolDebezium so that Pulsar changefeeds can use the debezium message format. The debezium codec is already implemented and shared with the Kafka sink via the common codec builder, so no additional encoding logic is needed.
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
Hi @HGHNice. Thanks for your PR. I'm waiting for a pingcap member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
Welcome @HGHNice! |
📝 WalkthroughWalkthroughThis PR extends Pulsar sink protocol support from ChangesPulsar Debezium Protocol Support
🎯 1 (Trivial) | ⏱️ ~3 minutes
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Code Review
This pull request adds support for the Debezium protocol to the Pulsar sink by updating the protocol validation logic and error messages. Feedback includes a recommendation to replace a hardcoded boolean with a dynamic protocol check in the event router, a request for additional test coverage in existing protocol parsing and string conversion tests, and a minor grammatical correction in a user-facing error message.
- Make isAvro parameter dynamic in NewEventRouter call - Add debezium and simple cases to protocol parsing and string tests
What problem does this PR solve?
Issue Number: close #5056
The Pulsar sink currently only supports
canal-json. Users who consumeTiCDC events via Pulsar with Debezium-compatible consumers (e.g. Flink CDC)
have no way to use the standard Debezium message format.
What is changed and how it works?
IsPulsarSupportedProtocols()inpkg/config/sink_protocol.goto include
ProtocolDebezium.downstreamadapter/sink/pulsar/helper.goto reflect the expanded protocol list.
sink via the common codec builder, so no additional encoding logic is needed.
TestIsPulsarSupportedProtocols.Check List
Tests
Questions
Will it cause performance regression or break compatibility?
No. Existing
canal-jsonbehavior is unchanged.Do you need to update user documentation, design documentation or monitoring documentation?
The Pulsar sink docs should note that
debeziumis now a validprotocolvalue.Release note