Skip to content

test(openai): drain completion raw_response streams to fix cassette leak#19019

Merged
Kyle-Verhoog merged 2 commits into
mainfrom
fix/openai-raw-response-stream-cassette-leak
Jul 21, 2026
Merged

test(openai): drain completion raw_response streams to fix cassette leak#19019
Kyle-Verhoog merged 2 commits into
mainfrom
fix/openai-raw-response-stream-cassette-leak

Conversation

@Kyle-Verhoog

Copy link
Copy Markdown
Member

What

The same VCR cassette-leak pattern fixed in #18778 for the chat_completion raw-response streaming tests also exists two tests earlier for the plain completion variants:

test_completion_raw_response_stream and test_acompletion_raw_response_stream open completion_streamed.yaml with stream=True via with_raw_response.create(...) but never consume the stream, so the VCR connection is never released. Under pytest-randomly orderings the cassette bleeds into later tests, whose unrelated requests fail to match and raise CannotOverwriteExistingCassetteException — the same failure mode reported as "~63 llmobs rate-limit failures" in the recent dd-trace-py CI incident (mislabeled as external API 429s by automated triage; it's actually this cassette leak).

Fix

Drain the raw stream inside the cassette block (resp.parse() / async for), same approach as #18778.

Testing

Ran the affected tests directly and the full test_openai_v1.py file under the exact pytest-randomly seed (3338286823) that originally triggered the leak in #18778 — all pass, 0 failures.

Claude session: 46d356c1-1ac1-4826-b78b-ea87f4220dda
Resume: claude --resume 46d356c1-1ac1-4826-b78b-ea87f4220dda

test_completion_raw_response_stream and test_acompletion_raw_response_stream
open completion_streamed.yaml with stream=True but never consume the
response, leaving the VCR cassette connection unreleased. Under
pytest-randomly orderings this bleeds into later tests and raises
CannotOverwriteExistingCassetteException, matching the same leak pattern
fixed for the chat_completion variants in #18778.

Claude session: 46d356c1-1ac1-4826-b78b-ea87f4220dda
Resume: claude --resume 46d356c1-1ac1-4826-b78b-ea87f4220dda

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@Kyle-Verhoog
Kyle-Verhoog requested a review from a team as a code owner July 13, 2026 17:45
@Kyle-Verhoog Kyle-Verhoog added the changelog/no-changelog A changelog entry is not required for this PR. label Jul 13, 2026
@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Jul 13, 2026

Copy link
Copy Markdown

Circular import analysis

⚠️ Existing circular imports

There are 37 circular imports that already exist on the base branch and have not been changed by this PR.

Show existing cycles (showing 5 of 37 shortest)
ddtrace.internal.datastreams -> ddtrace.internal.datastreams.kafka -> ddtrace.internal.datastreams
ddtrace.internal.core -> ddtrace._trace.span -> ddtrace.internal.core
ddtrace.internal.datastreams -> ddtrace.internal.datastreams.kombu -> ddtrace.internal.datastreams
ddtrace.internal.datastreams -> ddtrace.internal.datastreams.aiokafka -> ddtrace.internal.datastreams
ddtrace.internal.datastreams -> ddtrace.internal.datastreams.botocore -> ddtrace.internal.datastreams

To see all cycles, download the cycles-base.json and cycles-pr.json artifacts from this CI job and run:

uv run --script scripts/import-analysis/cycles.py compare cycles-base.json cycles-pr.json

@cit-pr-commenter-54b7da

Copy link
Copy Markdown

Codeowners resolved as

tests/contrib/openai/test_openai_v1.py                                  @DataDog/ml-observability

@Kyle-Verhoog
Kyle-Verhoog enabled auto-merge (squash) July 21, 2026 13:48
@Kyle-Verhoog
Kyle-Verhoog disabled auto-merge July 21, 2026 13:51
@Kyle-Verhoog
Kyle-Verhoog merged commit 0b091c9 into main Jul 21, 2026
332 checks passed
@Kyle-Verhoog
Kyle-Verhoog deleted the fix/openai-raw-response-stream-cassette-leak branch July 21, 2026 13:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/no-changelog A changelog entry is not required for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants