Skip to content

test: run each integration test class against a fresh Connect container - #486

Open
cgraham-rs wants to merge 1 commit into
mainfrom
cgraham-rs/issue-460-test-data-leak
Open

test: run each integration test class against a fresh Connect container#486
cgraham-rs wants to merge 1 commit into
mainfrom
cgraham-rs/issue-460-test-data-leak

Conversation

@cgraham-rs

Copy link
Copy Markdown
Collaborator

Fixes #460

  • Provisions a fresh, disposable Connect container per test class via a class-scoped autouse fixture wrapping with-connect so state can never leak between test classes
  • Cleans up the default Connect OAuth integration so related tests have a deterministic baseline and to prevent cross-test pollution. This is done via a scoped conftest.py fixture under oauth/
  • Container lifecycle is now owned by pytest instead of the Makefile
  • Updates CI to write the license file directly and drop the composite with-connect action
  • Documents the new integration test workflow in integration/README.md

Provisions a fresh, disposable Connect container per test class via a
class-scoped autouse fixture wrapping with-connect, so state can never
leak between test classes (#460). Container
lifecycle (including cleanup on incomplete with-connect output) is now
owned by pytest instead of the Makefile.

Also dedupes the OAuth default-integration cleanup into a scoped
conftest.py under oauth/, updates CI to write the license file directly
and drop the composite with-connect action, and documents the new
integration test workflow in integration/README.md.
@cgraham-rs
cgraham-rs requested a review from tdstein as a code owner July 28, 2026 15:08
@github-actions

Copy link
Copy Markdown

☂️ Python Coverage

current status: ✅

Overall Coverage

Lines Covered Coverage Threshold Status
2509 2359 94% 0% 🟢

New Files

No new covered files...

Modified Files

No covered modified files...

updated for commit: 10fbc1b by action🐍

@cgraham-rs

Copy link
Copy Markdown
Collaborator Author

@tdstein Beyond looking at the PR changes here I also hit a lint error in CI that was not evident locally. It looks like CI pulled a different version of ruff which expanded the ruleset. Things to discuss or create issues for.

  • Because uv.lock is gitignored, CI resolves dev dependencies fresh on every run while local environments stay pinned to whenever the lock was first created and the two will drift over time. I was running an older ruff which passes lint, CI pulled a newer version which is failing lint.
  • There's no current make target or CONTRIBUTING instruction to help users update local dev dependencies.
  • The latest ruff appears to include a variety of new rules which are throwing a significant amount of new lint errors across the entire repo. This may be related to using extend-select

@tdstein tdstein left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is so great. Thank you so much for putting in the effort to make this happen. What are your thoughts on utilizing this pattern more for doing end-to-end integration testing as part of the Connect CI workflow. That's always been part of the mission here, but it hasn't materialized.

Also, I know there is a bunch of logic in the integration tests that does manually cleanups to make things work between tests. I can delete all of that now, right?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gah! This is so cool. Sometime I just absolutely love Python!

Comment on lines +84 to +86
os.environ["CONNECT_SERVER"] = creds["CONNECT_SERVER"]
os.environ["CONNECT_API_KEY"] = creds["CONNECT_API_KEY"]
yield

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Beautiful! ✨

@tdstein

tdstein commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

@tdstein Beyond looking at the PR changes here I also hit a lint error in CI that was not evident locally. It looks like CI pulled a different version of ruff which expanded the ruleset. Things to discuss or create issues for.

  • Because uv.lock is gitignored, CI resolves dev dependencies fresh on every run while local environments stay pinned to whenever the lock was first created and the two will drift over time. I was running an older ruff which passes lint, CI pulled a newer version which is failing lint.
  • There's no current make target or CONTRIBUTING instruction to help users update local dev dependencies.
  • The latest ruff appears to include a variety of new rules which are throwing a significant amount of new lint errors across the entire repo. This may be related to using extend-select

Yeah, that keeps causing headaches. Is the fix to simply commit uv.lock? I don't recall why we ignored it to begin with.

@cgraham-rs

Copy link
Copy Markdown
Collaborator Author

@tdstein Beyond looking at the PR changes here I also hit a lint error in CI that was not evident locally. It looks like CI pulled a different version of ruff which expanded the ruleset. Things to discuss or create issues for.

  • Because uv.lock is gitignored, CI resolves dev dependencies fresh on every run while local environments stay pinned to whenever the lock was first created and the two will drift over time. I was running an older ruff which passes lint, CI pulled a newer version which is failing lint.
  • There's no current make target or CONTRIBUTING instruction to help users update local dev dependencies.
  • The latest ruff appears to include a variety of new rules which are throwing a significant amount of new lint errors across the entire repo. This may be related to using extend-select

Yeah, that keeps causing headaches. Is the fix to simply commit uv.lock? I don't recall why we ignored it to begin with.

I think that is the correct fix. But the fact it's been gitignored for so long gives me pause as there must have been a reason to do that. But that reason was not clear. I created two new issues so we can track and work on those separately.

@cgraham-rs

Copy link
Copy Markdown
Collaborator Author

This is so great. Thank you so much for putting in the effort to make this happen. What are your thoughts on utilizing this pattern more for doing end-to-end integration testing as part of the Connect CI workflow. That's always been part of the mission here, but it hasn't materialized.

I'm not sure what you mean.

Also, I know there is a bunch of logic in the integration tests that does manually cleanups to make things work between tests. I can delete all of that now, right?

The fresh Connect change is scoped to test classes, not individual tests. The tests within the same class still share one container, so one test's leftover state can still affect a sibling test in that same class. So we will still have to manage that, like we did with the oauth tests in this PR.

@cgraham-rs

Copy link
Copy Markdown
Collaborator Author

IMO we should merge this even though there are lint failures. Those are external to the changes here and are tracked separately.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Integration tests fail due to stale state on shared Connect instance

2 participants