dm: keep S3 external ID in import-into source dir#12681
Conversation
|
Adding the "do-not-merge/release-note-label-needed" label because no release-note block was detected, please follow our release note process to remove it. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
|
There was a problem hiding this comment.
Code Review
This pull request removes the logic that strips the S3 external ID from the source directory when using the BackendImportInto backend in the DM loader. The helper function stripS3ExternalIDFromSourceDir has been removed, and the corresponding unit tests have been updated to verify that the S3 external ID is preserved. No review comments were provided, and there is no additional feedback to offer.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
|
@fgksgf: The following test failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
Summary
Mydumper.SourceDirunchanged for DM import-into tasks, including S3external-id.Why
DM import-into uses the same S3 location across dumpling, Lightning local metadata/file reads, and the final TiDB
IMPORT INTOSQL. All of these paths need the keyspace role parameters to remain available.Live dev validation on cluster
10492432391076049734showed dumpling completed after the dataflow-service fix, but Lightning failed in the Load phase before submittingIMPORT INTObecauseMydumper.SourceDirhad already been stripped and the local import SDK could not assume the keyspace role.TiDB
IMPORT INTOnow allows an explicit S3external-id, so the correct fix is to keep the original S3 URI end to end instead of stripping it in Tiflow.How
MakeGlobalConfigto always setlightningCfg.Mydumper.SourceDir = cfg.Dir.LoadModeImportIntostill maps toBackendImportInto.Testing
gofmton changed files.go test; this PR was prepared from GitHub contents without a full Tiflow checkout in the current workspace.Risks and Reviewer Focus
external-idinIMPORT INTOresource parameters.IMPORT INTO.