Improve test suite performance via module-scoped fixtures and batched… - #666
Open
ehelms wants to merge 1 commit into
Open
Improve test suite performance via module-scoped fixtures and batched…#666ehelms wants to merge 1 commit into
ehelms wants to merge 1 commit into
Conversation
2 tasks
ehelms
commented
Jul 15, 2026
| PULP_API_SOCKET = '/run/httpd.pulp-api.sock' | ||
| PULP_CONTENT_SOCKET = '/run/httpd.pulp-content.sock' | ||
|
|
||
| # Run both check --deploy and settings inspection in a single Django startup. |
Member
Author
There was a problem hiding this comment.
There's got to be a better / cleaner way to do this, consider this just a "hack" for now.
ehelms
force-pushed
the
improve-test-fixture-scoping
branch
from
July 16, 2026 14:44
dcef7e0 to
e936b6f
Compare
… calls Promote all remote resource fixtures (organization, product, repositories, lifecycle_environment, content_view, activation_key) from function scope to module scope so each is created once per test module rather than once per test. Batch two pulp-manager Django startup costs into a single podman exec call. Mark inherently-slow tests (backup, target lifecycle) with pytest.mark.slow so they can be excluded during development with -m "not slow". Co-Authored-By: Claude <noreply@anthropic.com>
ehelms
force-pushed
the
improve-test-fixture-scoping
branch
from
July 24, 2026 18:31
e936b6f to
5b2ea19
Compare
ehelms
marked this pull request as ready for review
July 24, 2026 18:31
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why are you introducing these changes? (Problem description, related links)
Test runs were spending a large fraction of their wall-clock time recreating
the same remote resources for every test. The ten Foreman/Katello resource
fixtures (organizations, products, repositories, lifecycle environments, content
views, activation keys) were all function-scoped by default, so each individual
test paid the full create-over-API cost. Additionally, two pulp-manager tests
each spawned a separate Django process inside the container, doubling that
startup cost, and inherently-slow tests (backup, target lifecycle) were mixed
with fast ones making it impossible to exclude them during development.
Fixture call reduction (measured with
--fixture-durations):yum_repositoryclient_environmentorganizationproduct¹ Before, each test launched its own
pulpcore-manager shellprocess (45.0s + 42.5s in test call time); after, both tests share one module-scoped fixture.Test phase overhead (pytest
--durationsgrand totals):Skippable slow tests (with
-m "not slow"):backup_test.pytarget_lifecycle_test.pyWhat are the changes introduced in this pull request?
tests/conftest.py— Promote all remote resource fixtures toscope="module"so resources are created once per test module, not once pertest. Expand
content_viewteardown to handle version cleanup (previouslyleft to test bodies, which broke with shared fixtures). Simplify
client_environmentteardown to delegate version cleanup tocontent_view.Change
wait_for_metadata_generateto filter withstate != stoppedinsteadof fetching all historical tasks. Register the
slowmarker inpytest_configure.tests/pulp_test.py— Add_PULP_MANAGER_SCRIPTconstant and module-scopedpulp_manager_infofixture that runs one Django startup viapulpcore-manager shell(base64-encoded to avoid shell quoting issues).test_pulp_manager_checkandtest_pulp_import_export_settingsnow consumethe shared result instead of each launching their own container process.
tests/feature/katello/api_test.py— Remove per-test content view versioncleanup loop from
test_foreman_content_view; thecontent_viewfixtureteardown now handles this.
tests/backup_test.py— Addpytestmark = pytest.mark.slow.tests/target_lifecycle_test.py— Addpytestmark = pytest.mark.slow.How to test this pull request
Steps to reproduce:
per test
content_viewteardown cleans up published versions withouterror when the
katellofeature is enabledChecklist