Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: update documentation and examples on configuring retries #808

Merged
merged 2 commits into from
Jun 3, 2022
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
19 changes: 17 additions & 2 deletions docs/storage/retry_timeout.rst
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,8 @@ for each method, base on its semantics:
the same "generation", the library uses its
:data:`~google.cloud.storage.retry.DEFAULT_RETRY_IF_GENERATION_SPECIFIED`
policy, which retries API requests which returns a "transient" error,
but only if the original request includes an ``ifGenerationMatch`` header.
but only if the original request includes a ``generation`` or
``ifGenerationMatch`` header.

- For API requests which are idempotent only if the bucket or blob has
the same "metageneration", the library uses its
Expand All @@ -99,6 +100,20 @@ explicit policy in your code.

bucket = client.get_bucket(BUCKET_NAME, retry=None)

- You can modify the default retry behavior and create a copy of :data:`~google.cloud.storage.retry.DEFAULT_RETRY`
by calling it with a ``with_XXX`` method. E.g.:

.. code-block:: python

from google.cloud.storage.retry import DEFAULT_RETRY

# Customize retry with a deadline of 500 seconds (default=120 seconds).
modified_retry = DEFAULT_RETRY.with_deadline(500.0)
# Customize retry with an initial wait time of 1.5 (default=1.0).
# Customize retry with a wait time multiplier per iteration of 1.2 (default=2.0).
# Customize retry with a maximum wait time of 45.0 (default=60.0).
modified_retry = modified_retry.with_delay(initial=1.5, multiplier=1.2, maximum=45.0)

- You can pass an instance of :class:`google.api_core.retry.Retry` to enable
retries; the passed object will define retriable response codes and errors,
as well as configuring backoff and retry interval options. E.g.:
Expand Down Expand Up @@ -140,5 +155,5 @@ explicit policy in your code.

my_retry_policy = Retry(predicate=is_retryable)
my_cond_policy = ConditionalRetryPolicy(
my_retry_policy, conditional_predicate=is_etag_in_data)
my_retry_policy, conditional_predicate=is_etag_in_data, ["query_params"])
bucket = client.get_bucket(BUCKET_NAME, retry=my_cond_policy)
3 changes: 2 additions & 1 deletion google/cloud/storage/retry.py
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,8 @@ class ConditionalRetryPolicy(object):
:type required_kwargs: list(str)
:param required_kwargs:
A list of keyword argument keys that will be extracted from the API call
and passed into the ``conditional predicate`` in order.
and passed into the ``conditional predicate`` in order. For example,
``["query_params"]`` is commmonly used for preconditions in query_params.
"""

def __init__(self, retry_policy, conditional_predicate, required_kwargs):
Expand Down