[fix](fe) Prevent stale recycle candidates from erasing new generations - #67750
Open
wenzhenghu wants to merge 1 commit into
Open
[fix](fe) Prevent stale recycle candidates from erasing new generations#67750wenzhenghu wants to merge 1 commit into
wenzhenghu wants to merge 1 commit into
Conversation
### What problem does this PR solve? Issue Number: close apache#67303 Related PR: None Problem Summary: CatalogRecycleBin collected only expired IDs under a read lock. If an object was recovered and recycled with the same ID before the erase worker reacquired the write lock, the stale candidate could erase the newly recycled generation. Snapshot the recycle info and timestamp, then validate identity, timestamp, and expiration under the write lock before any erase callback, removal, or journal entry. ### Release note Fix CatalogRecycleBin to preserve newly recycled databases, tables, and partitions when an erase cycle holds a stale expired candidate. ### Check List (For Author) - Test: Unit Test - Added deterministic FE unit tests for database, table, and partition recover/recycle races. - `./run-fe-ut.sh --run org.apache.doris.catalog.CatalogRecycleBinTest` - `mvn -pl fe-core checkstyle:check -DskipTests` - Behavior changed: Yes. Stale expired candidates are skipped when the same ID has been recycled as a new generation. - Does this need documentation: No
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
Author
|
run buildall |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number: close #67303
Related PR: HYDCP#103
Problem Summary:
CatalogRecycleBincollects expired IDs under a read lock and erases them later under per-item write locks. During that gap,RECOVER -> DROPcan recycle a new object generation with the same ID, allowing the stale candidate to erase the new generation and write an incorrect erase journal.This change snapshots the recycle-info object and recycle timestamp with each candidate. Under the write lock, it verifies object identity, timestamp, and current expiration before any erase callback, removal, or journal entry. Database, table, and partition paths are all covered.
Static review confirmed that the per-item lock scope remains unchanged and that the change does not alter persisted metadata formats, public APIs, configuration, or upgrade behavior.
Release note
Fix
CatalogRecycleBinto preserve newly recycled databases, tables, and partitions when an erase cycle holds a stale expired candidate.Check List (For Author)
Test
./run-fe-ut.sh --run org.apache.doris.catalog.CatalogRecycleBinTest(33 tests passed)./run-fe-ut.sh --run org.apache.doris.catalog.CatalogRecycleBinTest#testExpiredDatabaseSnapshotDoesNotEraseNewGeneration+testExpiredTableSnapshotDoesNotEraseNewGeneration+testExpiredPartitionSnapshotDoesNotEraseNewGeneration(3 tests passed)mvn -pl fe-core checkstyle:check -DskipTestsBehavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)