Skip to content

fix(bigquery): pass timeout to the lookup after an exists_ok conflict - #18767

Open
abhishekacharya200 wants to merge 1 commit into
googleapis:mainfrom
abhishekacharya200:fix/bigquery-exists-ok-timeout
Open

abhishekacharya200 wants to merge 1 commit into
googleapis:mainfrom
abhishekacharya200:fix/bigquery-exists-ok-timeout

Conversation

@abhishekacharya200

Copy link
Copy Markdown
Contributor

Fixes #18766

create_dataset, create_table and create_routine with exists_ok=True fall back to get_dataset, get_table or get_routine when the insert returns 409. That fallback passed retry but not timeout. BigQuery's DEFAULT_TIMEOUT is None, so the lookup ran with no timeout at all, even when the caller set one, and a slow or stuck GET could block the call far past the timeout it asked for.

This passes the caller's timeout to the three lookups as well. When no timeout is given, the lookups get None exactly as before, so default behavior does not change. Each request still gets its own timeout and retry keeps working the same way, which matches the docstring: the number of seconds to wait for the HTTP transport before using retry.

Added test_create_dataset_alreadyexists_w_exists_ok_true_w_timeout, test_create_table_alreadyexists_w_exists_ok_true_w_timeout and test_create_routine_w_conflict_exists_ok_w_timeout, which check that both the POST and the GET get the caller's timeout. All three fail on main (the GET goes out with timeout=None) and pass with this change. The full tests/unit suite passes locally (2408 passed), and black==23.7.0 and flake8 report nothing.

I also ran the real client against a small local server that answers 409 to every insert and waits 10 seconds before answering every GET, with timeout=2. A minimal version of the script and server is in #18766. I used retry=None so each run shows a single request, because DEFAULT_RETRY treats a read timeout as retryable and keeps trying until its deadline. The last case uses the default retry with a 5 second deadline instead.

Call main (57366ee) This PR (55dc9a2)
create_dataset(exists_ok=True, timeout=2, retry=None) returned after 10.1s ReadTimeout after 2.0s
create_table(exists_ok=True, timeout=2, retry=None) returned after 10.0s ReadTimeout after 2.0s
create_routine(exists_ok=True, timeout=2, retry=None) returned after 10.0s ReadTimeout after 2.0s
create_table(exists_ok=True, timeout=2, retry=DEFAULT_RETRY.with_deadline(5)) returned after 10.0s RetryError after 4.3s

create_dataset, create_table and create_routine with exists_ok=True
fall back to get_dataset, get_table or get_routine when the insert
returns 409. The fallback passed retry but not timeout, and BigQuery's
default timeout is None, so that GET could wait forever even when the
caller set a timeout. Forward the caller's timeout to the lookup too.
@abhishekacharya200
abhishekacharya200 requested review from a team as code owners October 19, 2026 16:53
@abhishekacharya200
abhishekacharya200 requested review from sindhuvy and removed request for a team October 19, 2026 16:53

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request ensures that the caller's timeout parameter is preserved and passed to the subsequent get_* calls when handling resource conflicts (where exists_ok=True and a Conflict exception is raised) in the create_dataset, create_routine, and create_table methods of the BigQuery client. Additionally, corresponding unit tests have been added to verify this behavior for datasets, routines, and tables. There are no review comments, so I have no feedback to provide.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BigQuery: create_dataset, create_table and create_routine with exists_ok=True ignore timeout on the lookup after a 409

1 participant