Skip to content

perf: speed up RepositoryAdmin changelist search#1417

Open
thomasrockhu-codecov wants to merge 1 commit into
mainfrom
th/repository-admin-search-perf
Open

perf: speed up RepositoryAdmin changelist search#1417
thomasrockhu-codecov wants to merge 1 commit into
mainfrom
th/repository-admin-search-perf

Conversation

@thomasrockhu-codecov

Copy link
Copy Markdown
Contributor

Summary

The Django admin Repository changelist search is the #2 most expensive admin page in Sentry (/core/repository/, p95 ~73s). Its search ORed search_fields across the reposowners join; because author__username is on a different table, Postgres can't use per-table indexes for that OR and falls back to a full parallel seq-scan + hash join.

This PR rewrites RepositoryAdmin.get_search_results to keep the filter single-table:

  • Resolve author username → ownerids with one indexed query on owners, then filter repos with a single-table OR so each branch uses its own index (repos_name_trgm_idx trigram, service_id, ownerid, pk) via a BitmapOr.
  • Require a 3-char minimum for substring name search (pg_trgm can only use the GIN trigram index for ≥3-char terms).
  • Add list_select_related = ("author",) to remove the per-row author N+1 on the unsearched changelist.

No schema/migration change — the existing repos_name_trgm_idx is reused.

Measured impact

Local EXPLAIN (ANALYZE) against ~24.6M seeded repos:

Query Before After
name search (repo-500) 4,425 ms (seq scan + hash join) 34 ms (BitmapOr)
username search full join scan 1.9 ms
2-char term full join scan 0.03 ms (name skipped, index scan)

Behavior change

Substring name search now requires ≥3 characters; shorter terms still match service_id / username / repoid exactly.

Test plan

  • Existing core/tests/test_admin.py::get_search_results tests pass
  • EXPLAIN ANALYZE confirms BitmapOr / no seq scan for name, username, service_id, repoid searches
  • Verify results parity in staging admin (name substring, username, service_id, repoid)
  • Watch Sentry /core/repository/ p95 after deploy

Made with Cursor

The admin search ORed `search_fields` across the repos<->owners join. Because
`author__username` lives on a different table, Postgres cannot use per-table
indexes for that OR and fell back to a full parallel seq-scan + hash join
(~4.4s p50 locally over 24.6M repos; ~73s p95 in production).

Rewrite get_search_results to resolve usernames to ownerids first and build a
single-table OR on repos, so each branch uses its own index (name trigram,
service_id, ownerid, pk) via a BitmapOr. Substring name search now requires a
3-char minimum since pg_trgm can only use repos_name_trgm_idx for >=3-char
terms. Also add list_select_related=("author",) to drop the per-row author
N+1 on the unsearched changelist.

Local EXPLAIN ANALYZE: 4425ms -> 34ms for a name search.

Co-authored-by: Cursor <cursoragent@cursor.com>
@codecov

codecov Bot commented Jul 4, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 87.50000% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.88%. Comparing base (701e991) to head (27427d6).
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
apps/codecov-api/core/admin.py 87.50% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1417      +/-   ##
==========================================
- Coverage   91.89%   91.88%   -0.01%     
==========================================
  Files        1325     1325              
  Lines       50868    50875       +7     
  Branches     1626     1626              
==========================================
+ Hits        46744    46749       +5     
- Misses       3818     3820       +2     
  Partials      306      306              
Flag Coverage Δ
apiunit 94.93% <87.50%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

@codecov-notifications

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 87.50000% with 2 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
apps/codecov-api/core/admin.py 87.50% 2 Missing ⚠️

❌ Your patch check has failed because the patch coverage (87.50%) is below the target coverage (90.00%). You can increase the patch coverage or adjust the target coverage.

📢 Thoughts on this report? Let us know!

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant