Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: jd/tenacity
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: 8.3.0
Choose a base ref
...
head repository: jd/tenacity
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: 8.4.0
Choose a head ref
  • 5 commits
  • 13 files changed
  • 7 contributors

Commits on Jun 12, 2024

  1. Add async strategies (#451)

    * Add async strategies
    
    * Fix init typing
    
    * Reuse is_coroutine_callable
    
    * Keep only async predicate overrides and DRY implementations
    
    * Ensure async and/or versions called when necessary
    
    * Run ruff format
    
    * Copy over strategies as async
    
    * Add release note
    hasier authored Jun 12, 2024
    1
    Copy the full SHA
    21137e7 View commit details
  2. chore(deps): bump the github-actions group across 1 directory with 2 …

    …updates (#466)
    
    Bumps the github-actions group with 2 updates in the / directory: [actions/checkout](https://github.com/actions/checkout) and [actions/setup-python](https://github.com/actions/setup-python).
    
    
    Updates `actions/checkout` from 4.1.1 to 4.1.6
    - [Release notes](https://github.com/actions/checkout/releases)
    - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
    - [Commits](actions/checkout@v4.1.1...v4.1.6)
    
    Updates `actions/setup-python` from 5.0.0 to 5.1.0
    - [Release notes](https://github.com/actions/setup-python/releases)
    - [Commits](actions/setup-python@v5.0.0...v5.1.0)
    
    ---
    updated-dependencies:
    - dependency-name: actions/checkout
      dependency-type: direct:production
      update-type: version-update:semver-patch
      dependency-group: github-actions
    - dependency-name: actions/setup-python
      dependency-type: direct:production
      update-type: version-update:semver-minor
      dependency-group: github-actions
    ...
    
    Signed-off-by: dependabot[bot] <support@github.com>
    Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
    dependabot[bot] and mergify[bot] authored Jun 12, 2024

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    5b00c15 View commit details

Commits on Jun 13, 2024

  1. Update index.rst: Remove * (#465)

    Co-authored-by: Julien Danjou <julien@danjou.info>
    TheRealBecks and jd authored Jun 13, 2024

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    952189b View commit details

Commits on Jun 17, 2024

  1. Support Trio out-of-the-box, take 2 (#463)

    * Support Trio out-of-the-box
    
    This PR makes `@retry` just work when running under Trio.
    
    * Add a no-trio test environment
    
    * Switch to only testing trio in one environment
    
    * bump releasenote so it is later in history->reno puts it in the correct place in the changelog
    
    * fix mypy & pep8 checks
    
    * Update doc/source/index.rst
    
    fix example
    
    Co-authored-by: Julien Danjou <julien@danjou.info>
    
    * Update tests/test_tornado.py
    
    * Update tests/test_tornado.py
    
    * make _portably_async_sleep a sync function that returns an async function
    
    ---------
    
    Co-authored-by: Nathaniel J. Smith <njs@pobox.com>
    Co-authored-by: Julien Danjou <julien@danjou.info>
    3 people authored Jun 17, 2024
    Copy the full SHA
    ade0567 View commit details
  2. ci: add support for trio in Mergify automerge (#470)

    Change-Id: Idc6ec012cceae67ceb11914763350b34addcce5e
    jd authored Jun 17, 2024
    Copy the full SHA
    702014b View commit details
6 changes: 3 additions & 3 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
@@ -27,19 +27,19 @@ jobs:
- python: "3.11"
tox: py311
- python: "3.12"
tox: py312
tox: py312,py312-trio
- python: "3.12"
tox: pep8
- python: "3.11"
tox: mypy
steps:
- name: Checkout 🛎️
uses: actions/checkout@v4.1.1
uses: actions/checkout@v4.1.6
with:
fetch-depth: 0

- name: Setup Python 🔧
uses: actions/setup-python@v5.0.0
uses: actions/setup-python@v5.1.0
with:
python-version: ${{ matrix.python }}
allow-prereleases: true
4 changes: 2 additions & 2 deletions .github/workflows/deploy.yaml
Original file line number Diff line number Diff line change
@@ -11,12 +11,12 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout 🛎️
uses: actions/checkout@v4.1.1
uses: actions/checkout@v4.1.6
with:
fetch-depth: 0

- name: Setup Python 🔧
uses: actions/setup-python@v5.0.0
uses: actions/setup-python@v5.1.0
with:
python-version: 3.11

2 changes: 1 addition & 1 deletion .mergify.yml
Original file line number Diff line number Diff line change
@@ -14,7 +14,7 @@ queue_rules:
- "check-success=test (3.9, py39)"
- "check-success=test (3.10, py310)"
- "check-success=test (3.11, py311)"
- "check-success=test (3.12, py312)"
- "check-success=test (3.12, py312,py312-trio)"
- "check-success=test (3.12, pep8)"

pull_request_rules:
20 changes: 13 additions & 7 deletions doc/source/index.rst
Original file line number Diff line number Diff line change
@@ -79,7 +79,7 @@ Examples
Basic Retry
~~~~~~~~~~~

.. testsetup:: *
.. testsetup::

import logging
#
@@ -568,28 +568,34 @@ in retry strategies like ``retry_if_result``. This can be done accessing the
Async and retry
~~~~~~~~~~~~~~~

Finally, ``retry`` works also on asyncio and Tornado (>= 4.5) coroutines.
Finally, ``retry`` works also on asyncio, Trio, and Tornado (>= 4.5) coroutines.
Sleeps are done asynchronously too.

.. code-block:: python
@retry
async def my_async_function(loop):
async def my_asyncio_function(loop):
await loop.getaddrinfo('8.8.8.8', 53)
.. code-block:: python
@retry
async def my_async_trio_function():
await trio.socket.getaddrinfo('8.8.8.8', 53)
.. code-block:: python
@retry
@tornado.gen.coroutine
def my_async_function(http_client, url):
def my_async_tornado_function(http_client, url):
yield http_client.fetch(url)
You can even use alternative event loops such as `curio` or `Trio` by passing the correct sleep function:
You can even use alternative event loops such as `curio` by passing the correct sleep function:

.. code-block:: python
@retry(sleep=trio.sleep)
async def my_async_function(loop):
@retry(sleep=curio.sleep)
async def my_async_curio_function():
await asks.get('https://example.org')
Contribute
5 changes: 5 additions & 0 deletions releasenotes/notes/add-async-actions-b249c527d99723bb.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
features:
- |
Added the ability to use async functions for retries. This way, you can now use
asyncio coroutines for retry strategy predicates.
6 changes: 6 additions & 0 deletions releasenotes/notes/trio-support-retry-22bd544800cd1f36.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
features:
- |
If you're using `Trio <https://trio.readthedocs.io>`__, then
``@retry`` now works automatically. It's no longer necessary to
pass ``sleep=trio.sleep``.
28 changes: 19 additions & 9 deletions tenacity/__init__.py
Original file line number Diff line number Diff line change
@@ -24,7 +24,8 @@
import warnings
from abc import ABC, abstractmethod
from concurrent import futures
from inspect import iscoroutinefunction

from . import _utils

# Import all built-in retry strategies for easier usage.
from .retry import retry_base # noqa
@@ -87,6 +88,7 @@
if t.TYPE_CHECKING:
import types

from . import asyncio as tasyncio
from .retry import RetryBaseT
from .stop import StopBaseT
from .wait import WaitBaseT
@@ -593,16 +595,24 @@ def retry(func: WrappedFn) -> WrappedFn: ...

@t.overload
def retry(
sleep: t.Callable[[t.Union[int, float]], t.Optional[t.Awaitable[None]]] = sleep,
sleep: t.Callable[[t.Union[int, float]], t.Union[None, t.Awaitable[None]]] = sleep,
stop: "StopBaseT" = stop_never,
wait: "WaitBaseT" = wait_none(),
retry: "RetryBaseT" = retry_if_exception_type(),
before: t.Callable[["RetryCallState"], None] = before_nothing,
after: t.Callable[["RetryCallState"], None] = after_nothing,
before_sleep: t.Optional[t.Callable[["RetryCallState"], None]] = None,
retry: "t.Union[RetryBaseT, tasyncio.retry.RetryBaseT]" = retry_if_exception_type(),
before: t.Callable[
["RetryCallState"], t.Union[None, t.Awaitable[None]]
] = before_nothing,
after: t.Callable[
["RetryCallState"], t.Union[None, t.Awaitable[None]]
] = after_nothing,
before_sleep: t.Optional[
t.Callable[["RetryCallState"], t.Union[None, t.Awaitable[None]]]
] = None,
reraise: bool = False,
retry_error_cls: t.Type["RetryError"] = RetryError,
retry_error_callback: t.Optional[t.Callable[["RetryCallState"], t.Any]] = None,
retry_error_callback: t.Optional[
t.Callable[["RetryCallState"], t.Union[t.Any, t.Awaitable[t.Any]]]
] = None,
) -> t.Callable[[WrappedFn], WrappedFn]: ...


@@ -624,7 +634,7 @@ def wrap(f: WrappedFn) -> WrappedFn:
f"this will probably hang indefinitely (did you mean retry={f.__class__.__name__}(...)?)"
)
r: "BaseRetrying"
if iscoroutinefunction(f):
if _utils.is_coroutine_callable(f):
r = AsyncRetrying(*dargs, **dkw)
elif (
tornado
@@ -640,7 +650,7 @@ def wrap(f: WrappedFn) -> WrappedFn:
return wrap


from tenacity._asyncio import AsyncRetrying # noqa:E402,I100
from tenacity.asyncio import AsyncRetrying # noqa:E402,I100

if tornado:
from tenacity.tornadoweb import TornadoRetrying
12 changes: 12 additions & 0 deletions tenacity/_utils.py
Original file line number Diff line number Diff line change
@@ -87,3 +87,15 @@ def is_coroutine_callable(call: typing.Callable[..., typing.Any]) -> bool:
partial_call = isinstance(call, functools.partial) and call.func
dunder_call = partial_call or getattr(call, "__call__", None)
return inspect.iscoroutinefunction(dunder_call)


def wrap_to_async_func(
call: typing.Callable[..., typing.Any],
) -> typing.Callable[..., typing.Awaitable[typing.Any]]:
if is_coroutine_callable(call):
return call

async def inner(*args: typing.Any, **kwargs: typing.Any) -> typing.Any:
return call(*args, **kwargs)

return inner
101 changes: 77 additions & 24 deletions tenacity/_asyncio.py → tenacity/asyncio/__init__.py
Original file line number Diff line number Diff line change
@@ -19,34 +19,87 @@
import sys
import typing as t

import tenacity
from tenacity import AttemptManager
from tenacity import BaseRetrying
from tenacity import DoAttempt
from tenacity import DoSleep
from tenacity import RetryCallState
from tenacity import RetryError
from tenacity import after_nothing
from tenacity import before_nothing
from tenacity import _utils

# Import all built-in retry strategies for easier usage.
from .retry import RetryBaseT
from .retry import retry_all # noqa
from .retry import retry_any # noqa
from .retry import retry_if_exception # noqa
from .retry import retry_if_result # noqa
from ..retry import RetryBaseT as SyncRetryBaseT

if t.TYPE_CHECKING:
from tenacity.stop import StopBaseT
from tenacity.wait import WaitBaseT

WrappedFnReturnT = t.TypeVar("WrappedFnReturnT")
WrappedFn = t.TypeVar("WrappedFn", bound=t.Callable[..., t.Awaitable[t.Any]])


def asyncio_sleep(duration: float) -> t.Awaitable[None]:
def _portable_async_sleep(seconds: float) -> t.Awaitable[None]:
# If trio is already imported, then importing it is cheap.
# If trio isn't already imported, then it's definitely not running, so we
# can skip further checks.
if "trio" in sys.modules:
# If trio is available, then sniffio is too
import trio
import sniffio

if sniffio.current_async_library() == "trio":
return trio.sleep(seconds)
# Otherwise, assume asyncio
# Lazy import asyncio as it's expensive (responsible for 25-50% of total import overhead).
import asyncio

return asyncio.sleep(duration)
return asyncio.sleep(seconds)


class AsyncRetrying(BaseRetrying):
sleep: t.Callable[[float], t.Awaitable[t.Any]]

def __init__(
self,
sleep: t.Callable[[float], t.Awaitable[t.Any]] = asyncio_sleep,
**kwargs: t.Any,
sleep: t.Callable[
[t.Union[int, float]], t.Union[None, t.Awaitable[None]]
] = _portable_async_sleep,
stop: "StopBaseT" = tenacity.stop.stop_never,
wait: "WaitBaseT" = tenacity.wait.wait_none(),
retry: "t.Union[SyncRetryBaseT, RetryBaseT]" = tenacity.retry_if_exception_type(),
before: t.Callable[
["RetryCallState"], t.Union[None, t.Awaitable[None]]
] = before_nothing,
after: t.Callable[
["RetryCallState"], t.Union[None, t.Awaitable[None]]
] = after_nothing,
before_sleep: t.Optional[
t.Callable[["RetryCallState"], t.Union[None, t.Awaitable[None]]]
] = None,
reraise: bool = False,
retry_error_cls: t.Type["RetryError"] = RetryError,
retry_error_callback: t.Optional[
t.Callable[["RetryCallState"], t.Union[t.Any, t.Awaitable[t.Any]]]
] = None,
) -> None:
super().__init__(**kwargs)
self.sleep = sleep
super().__init__(
sleep=sleep, # type: ignore[arg-type]
stop=stop,
wait=wait,
retry=retry, # type: ignore[arg-type]
before=before, # type: ignore[arg-type]
after=after, # type: ignore[arg-type]
before_sleep=before_sleep, # type: ignore[arg-type]
reraise=reraise,
retry_error_cls=retry_error_cls,
retry_error_callback=retry_error_callback,
)

async def __call__( # type: ignore[override]
self, fn: WrappedFn, *args: t.Any, **kwargs: t.Any
@@ -65,39 +118,29 @@ async def __call__( # type: ignore[override]
retry_state.set_result(result)
elif isinstance(do, DoSleep):
retry_state.prepare_for_next_attempt()
await self.sleep(do)
await self.sleep(do) # type: ignore[misc]
else:
return do # type: ignore[no-any-return]

@classmethod
def _wrap_action_func(cls, fn: t.Callable[..., t.Any]) -> t.Callable[..., t.Any]:
if _utils.is_coroutine_callable(fn):
return fn

async def inner(*args: t.Any, **kwargs: t.Any) -> t.Any:
return fn(*args, **kwargs)

return inner

def _add_action_func(self, fn: t.Callable[..., t.Any]) -> None:
self.iter_state.actions.append(self._wrap_action_func(fn))
self.iter_state.actions.append(_utils.wrap_to_async_func(fn))

async def _run_retry(self, retry_state: "RetryCallState") -> None: # type: ignore[override]
self.iter_state.retry_run_result = await self._wrap_action_func(self.retry)(
self.iter_state.retry_run_result = await _utils.wrap_to_async_func(self.retry)(
retry_state
)

async def _run_wait(self, retry_state: "RetryCallState") -> None: # type: ignore[override]
if self.wait:
sleep = await self._wrap_action_func(self.wait)(retry_state)
sleep = await _utils.wrap_to_async_func(self.wait)(retry_state)
else:
sleep = 0.0

retry_state.upcoming_sleep = sleep

async def _run_stop(self, retry_state: "RetryCallState") -> None: # type: ignore[override]
self.statistics["delay_since_first_attempt"] = retry_state.seconds_since_start
self.iter_state.stop_run_result = await self._wrap_action_func(self.stop)(
self.iter_state.stop_run_result = await _utils.wrap_to_async_func(self.stop)(
retry_state
)

@@ -127,7 +170,7 @@ async def __anext__(self) -> AttemptManager:
return AttemptManager(retry_state=self._retry_state)
elif isinstance(do, DoSleep):
self._retry_state.prepare_for_next_attempt()
await self.sleep(do)
await self.sleep(do) # type: ignore[misc]
else:
raise StopAsyncIteration

@@ -146,3 +189,13 @@ async def async_wrapped(*args: t.Any, **kwargs: t.Any) -> t.Any:
async_wrapped.retry_with = fn.retry_with # type: ignore[attr-defined]

return async_wrapped # type: ignore[return-value]


__all__ = [
"retry_all",
"retry_any",
"retry_if_exception",
"retry_if_result",
"WrappedFn",
"AsyncRetrying",
]
Loading