FDN-4939: Fix NOT IN → NOT EXISTS in ScheduleMigrateVersionsProcessor#972
Merged
Merged
Conversation
NOT IN with a subquery was causing ~99% CPU on apibuilderdb20211217. The subquery cast type_id::uuid per row, defeating the index, and the NOT IN semantics materialise the full subquery before filtering. NOT EXISTS with a correlated predicate (type_id = v.guid::text) lets Postgres use the existing unique index on tasks(type_id, type) for a targeted seek per outer row instead of a full subquery scan. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ScheduleMigrateVersionsProcessorwas running a query withNOT IN (SELECT type_id::uuid FROM tasks WHERE type = ...)that caused ~99% CPU onapibuilderdb20211217. Two issues combined to cause the full-table scan:type_id::uuidapplied a cast to an indexed column on every subquery row, defeating the index and forcing a full scan oftasks.NOT INmaterialises the entire subquery before the anti-join, compounding the cost.NOT EXISTSand flipped the cast tov.guid::text, so Postgres can seek the existingunique(type_id, type)index directly — one targeted lookup per outer row instead of a full scan.Dependencies
None. The existing
unique(type_id, type)index ontasksalready covers the new predicate; no schema migration needed.Validation
apibuilderdb20211217drops after deployScheduleMigrateVersionsProcessorcontinues to enqueueMigrateVersiontasks correctlyEng-Review Summary