perf: speed up RepositoryAdmin changelist search#1417
Open
thomasrockhu-codecov wants to merge 1 commit into
Open
perf: speed up RepositoryAdmin changelist search#1417thomasrockhu-codecov wants to merge 1 commit into
thomasrockhu-codecov wants to merge 1 commit into
Conversation
The admin search ORed `search_fields` across the repos<->owners join. Because
`author__username` lives on a different table, Postgres cannot use per-table
indexes for that OR and fell back to a full parallel seq-scan + hash join
(~4.4s p50 locally over 24.6M repos; ~73s p95 in production).
Rewrite get_search_results to resolve usernames to ownerids first and build a
single-table OR on repos, so each branch uses its own index (name trigram,
service_id, ownerid, pk) via a BitmapOr. Substring name search now requires a
3-char minimum since pg_trgm can only use repos_name_trgm_idx for >=3-char
terms. Also add list_select_related=("author",) to drop the per-row author
N+1 on the unsearched changelist.
Local EXPLAIN ANALYZE: 4425ms -> 34ms for a name search.
Co-authored-by: Cursor <cursoragent@cursor.com>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1417 +/- ##
==========================================
- Coverage 91.89% 91.88% -0.01%
==========================================
Files 1325 1325
Lines 50868 50875 +7
Branches 1626 1626
==========================================
+ Hits 46744 46749 +5
- Misses 3818 3820 +2
Partials 306 306
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. |
Codecov Report❌ Patch coverage is
❌ Your patch check has failed because the patch coverage (87.50%) is below the target coverage (90.00%). You can increase the patch coverage or adjust the target coverage. 📢 Thoughts on this report? Let us know! 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The Django admin
Repositorychangelist search is the #2 most expensive admin page in Sentry (/core/repository/, p95 ~73s). Its search ORedsearch_fieldsacross therepos⟷ownersjoin; becauseauthor__usernameis on a different table, Postgres can't use per-table indexes for that OR and falls back to a full parallel seq-scan + hash join.This PR rewrites
RepositoryAdmin.get_search_resultsto keep the filter single-table:owners, then filterreposwith a single-table OR so each branch uses its own index (repos_name_trgm_idxtrigram,service_id,ownerid, pk) via aBitmapOr.pg_trgmcan only use the GIN trigram index for ≥3-char terms).list_select_related = ("author",)to remove the per-row author N+1 on the unsearched changelist.No schema/migration change — the existing
repos_name_trgm_idxis reused.Measured impact
Local
EXPLAIN (ANALYZE)against ~24.6M seeded repos:repo-500)Behavior change
Substring name search now requires ≥3 characters; shorter terms still match
service_id/ username / repoid exactly.Test plan
core/tests/test_admin.py::get_search_resultstests passEXPLAIN ANALYZEconfirms BitmapOr / no seq scan for name, username, service_id, repoid searches/core/repository/p95 after deployMade with Cursor