Repository navigation
fix(bigquery): pass timeout to the lookup after an exists_ok conflict - #18767
Open
abhishekacharya200 wants to merge 1 commit into
Open
abhishekacharya200 wants to merge 1 commit into
abhishekacharya200 wants to merge 1 commit into
Conversation
create_dataset, create_table and create_routine with exists_ok=True fall back to get_dataset, get_table or get_routine when the insert returns 409. The fallback passed retry but not timeout, and BigQuery's default timeout is None, so that GET could wait forever even when the caller set a timeout. Forward the caller's timeout to the lookup too.
abhishekacharya200
requested review from
sindhuvy
and removed request for
a team
October 19, 2026 16:53
Contributor
There was a problem hiding this comment.
Code Review
This pull request ensures that the caller's timeout parameter is preserved and passed to the subsequent get_* calls when handling resource conflicts (where exists_ok=True and a Conflict exception is raised) in the create_dataset, create_routine, and create_table methods of the BigQuery client. Additionally, corresponding unit tests have been added to verify this behavior for datasets, routines, and tables. There are no review comments, so I have no feedback to provide.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #18766
create_dataset,create_tableandcreate_routinewithexists_ok=Truefall back toget_dataset,get_tableorget_routinewhen the insert returns 409. That fallback passedretrybut nottimeout. BigQuery'sDEFAULT_TIMEOUTisNone, so the lookup ran with no timeout at all, even when the caller set one, and a slow or stuck GET could block the call far past the timeout it asked for.This passes the caller's
timeoutto the three lookups as well. When no timeout is given, the lookups getNoneexactly as before, so default behavior does not change. Each request still gets its own timeout andretrykeeps working the same way, which matches the docstring: the number of seconds to wait for the HTTP transport before usingretry.Added
test_create_dataset_alreadyexists_w_exists_ok_true_w_timeout,test_create_table_alreadyexists_w_exists_ok_true_w_timeoutandtest_create_routine_w_conflict_exists_ok_w_timeout, which check that both the POST and the GET get the caller's timeout. All three fail onmain(the GET goes out withtimeout=None) and pass with this change. The fulltests/unitsuite passes locally (2408 passed), andblack==23.7.0andflake8report nothing.I also ran the real client against a small local server that answers 409 to every insert and waits 10 seconds before answering every GET, with
timeout=2. A minimal version of the script and server is in #18766. I usedretry=Noneso each run shows a single request, becauseDEFAULT_RETRYtreats a read timeout as retryable and keeps trying until its deadline. The last case uses the default retry with a 5 second deadline instead.main(57366ee)create_dataset(exists_ok=True, timeout=2, retry=None)ReadTimeoutafter 2.0screate_table(exists_ok=True, timeout=2, retry=None)ReadTimeoutafter 2.0screate_routine(exists_ok=True, timeout=2, retry=None)ReadTimeoutafter 2.0screate_table(exists_ok=True, timeout=2, retry=DEFAULT_RETRY.with_deadline(5))RetryErrorafter 4.3s