Skip to content

fix(worker): Resolve N+1 query in ProcessFlakesTask#1420

Open
sentry[bot] wants to merge 1 commit into
mainfrom
seer/fix/process-flakes-n-plus-1-En0Eax
Open

fix(worker): Resolve N+1 query in ProcessFlakesTask#1420
sentry[bot] wants to merge 1 commit into
mainfrom
seer/fix/process-flakes-n-plus-1-En0Eax

Conversation

@sentry

@sentry sentry Bot commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

This PR addresses an N+1 query issue identified in the app.tasks.flakes.ProcessFlakesTask, specifically within the ta_process_flakes.py module.

Problem:
Previously, process_flakes_for_commit iterated through each upload and called get_testruns() for every single upload, leading to an N+1 query pattern. Additionally, handle_pass performed individual flake.save() operations when a flake reached its expiration criteria, causing further redundant database writes.

Solution:

  1. Batched Testrun Fetching: The get_testruns() function was replaced with get_testruns_for_uploads(). This new function fetches all relevant Testrun objects for all uploads in a single batched query, significantly reducing database round-trips.
  2. Optimized process_single_upload: The process_single_upload function was refactored to accept a list of pre-fetched Testrun objects, eliminating the need for individual database queries within the loop.
  3. Consolidated Bulk Updates: Testrun.objects.bulk_update is now called once after all uploads have been processed, rather than once per upload.
  4. Deferred Flake Saving: Individual flake.save() calls in handle_pass were removed. Expiring flakes are now marked for update and handled by the existing Flake.objects.bulk_create (with update_conflicts=True) at the end of process_flakes_for_commit, ensuring all flake updates are batched.

These changes drastically reduce the number of database queries, improving the performance and efficiency of the ProcessFlakesTask.

Legal Boilerplate

Look, I get it. The entity doing business as "Codecov" is owned by Harness, Inc. In 2026 Harness acquired Codecov and as a result Harness is going to need some rights from me in order to utilize my contributions in this PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Harness can use, modify, copy, and redistribute my contributions, under Harness's choice of terms.

Fixes WORKER-YYR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants