impl(bigquery): job level retry for queries - #6318
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces a RetryContext to handle job-level retries and re-issuances for BigQuery queries, refactoring the query execution path to delegate to this context and updating Query::until_done to retry on transient failures. Feedback on the changes highlights a critical compilation error in until_done where self is partially moved inside a loop, and suggests avoiding destructuring self at the loop's start. Additionally, it is recommended to pass self by value in reissue to avoid an unnecessary clone of the context.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #6318 +/- ##
==========================================
+ Coverage 96.26% 96.27% +0.01%
==========================================
Files 282 282
Lines 73304 73508 +204
==========================================
+ Hits 70563 70772 +209
+ Misses 2741 2736 -5 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces a RetryContext to manage job-level retries and re-issuances for BigQuery queries, refactoring PostQueryExecutor and InsertJobExecutor to return raw responses and updating Query::until_done to handle retries in a loop. The review feedback focuses on minor Rust idiomatic improvements and optimizations, such as avoiding an unnecessary double reference (&&str) when passing project_id, removing an unnecessary .to_string() allocation on set_project_id, and using pattern matching instead of .take() on the owned retry_context option.
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces a RetryContext to manage job-level retries and re-issuances for BigQuery queries, refactoring PostQueryExecutor and InsertJobExecutor to return raw responses and updating Query::until_done to handle retryable errors. Feedback on the changes suggests deriving Debug for the public RunQuery struct to adhere to the repository style guide, and addresses an off-by-one error in RetryableJobErrors that allows an extra retry attempt beyond the configured limit.
| pub(crate) job_service: Arc<JobService>, | ||
| pub(crate) request: RunQueryRequest, | ||
| pub(crate) project_id: Option<String>, | ||
| pub(crate) job_retry_policy: Arc<dyn JobRetryPolicy>, |
There was a problem hiding this comment.
According to the repository style guide, all public types should implement both Debug and Clone to ensure interoperability. Since RunQuery is a public struct, please derive Debug for it (e.g., by changing #[derive(Clone)] to #[derive(Clone, Debug)] on line 31).
References
- Public types should implement both
DebugandCloneto ensure interoperability. (link) - Public types, such as request builders, should implement both
DebugandCloneto ensure interoperability.
There was a problem hiding this comment.
will do that in a separated PR. Need to figure out how to add #[derive(Debug)] on the generated types
| } | ||
|
|
||
| #[derive(Clone, Debug)] | ||
| #[allow(dead_code)] |
There was a problem hiding this comment.
The attempt_limit in RetryableJobErrors actually allows attempt_limit + 1 total attempts due to an off-by-one check in its on_error implementation (which checks state.attempt_count >= self.attempt_limit). Since attempt_count starts at 0 for the first failure, an attempt_limit of 3 will allow attempts 0, 1, 2, and 3 (total of 4 attempts). Consider changing the check to state.attempt_count + 1 >= self.attempt_limit to strictly adhere to the attempt limit while keeping attempt_count 0-indexed for backoff calculations.
There was a problem hiding this comment.
will fix in a separated PR
When a query job fails with a retryable BigQuery error (for example
backendErrororrateLimitExceeded), the client now automatically creates a new job attempt with backoff instead of failing immediately. Move all logic for query execution away from builder layer (RunQuery) to the execution layer.Towards #5844 #6218