Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bail should prevent any further calls to the retrier function #69

Open
axnsan12 opened this issue Mar 5, 2020 · 14 comments
Open

bail should prevent any further calls to the retrier function #69

axnsan12 opened this issue Mar 5, 2020 · 14 comments

Comments

@axnsan12
Copy link

axnsan12 commented Mar 5, 2020

From the example:

// Packages
const retry = require('async-retry')
const fetch = require('node-fetch')
 
await retry(async bail => {
  // if anything throws, we retry
  const res = await fetch('https://google.com')
 
  if (403 === res.status) {
    // don't retry upon 403
    bail(new Error('Unauthorized'))
    // return  <---- don't immediately return here
    throw new Error('Throw after bail'); 
  }
 
  const data = await res.text()
  return data.substr(0, 500)
}, {
  retries: 5
})

Calling bail will immediately reject the promise returned by retry, but if the promise returned by the retirer function is then itself rejected, attempts will continue running in the background until e.g. the max retry count is reached.

This can be surprising and result in obscure bugs.

@Abramovick
Copy link

I'm facing a similar issue

@jamesholcomb
Copy link

I might be running into this as well....would it occur if calling bail() inside a try/catch block within the retrier?

@axnsan12
Copy link
Author

I might be running into this as well....would it occur if calling bail() inside a try/catch block within the retrier?

Yes, if the call to bail() is followed by code that might throw before returning.

@Abramovick
Copy link

Abramovick commented Mar 31, 2020

I've done this, and it seems to be working.
Hopefully I'm not adding too much code, but if you're lazy to read it then, just skip to the bottom function where the comment that says "key part for this example" is.

The synopsis of all this is: the try/catch block worked for me.

import retry from 'async-retry';
import merge frorm 'lodash/merge';

const retries = 3;
const defaultOptions = {
  headers: {
    Accept: 'application/json',
    'Content-Type': 'application/json',
  },
  method: 'GET',
};
const isJson = response => {
  const contentType = response.headers.get('Content-Type');
  return !!(contentType && contentType.indexOf('application/json') !== -1);
};

async function request(
  url,
  options = {},
) {
  const allOptions = merge({}, defaultOptions, options);
  const response = await fetch(url, allOptions);

  const json = isJson(response) ? await response.json() : null;

  if (response.ok) {
    return json;
  }

  throw json;
}


/* ------ key part for this example -------  */
const response = await retry(
  async bail => {
    try {
      return await request('https://google.com');

      /* 
       * If the request function definition above is confusing you, just do a normal fetch like this 
       * return await fetch('https://google.com');
       */
    } catch (e) {
      if (e.status >= 400 || e.status < 500) return bail(e);

      throw e;
    }
  },
  {
    retries,
  },
);

@axnsan12
Copy link
Author

axnsan12 commented Mar 31, 2020

Yes, of course that if you do return bail() you avoid this issue.

The actual issue is when you don't do that. More specifically, after calling bail() it should not matter what the outcome of the returned promise is, it should stop retrying even if you do bail() + throw.

@xenjke
Copy link

xenjke commented Feb 25, 2021

Were just hit by that as well. Surprising number of unexpected background requests in runtime, while all unit tests were green :) can be caught while unit testing with something like:

// `retryer` being a wrapper that retries `request`, similar to the one above in the comments

it("rejects with passed function error and doesn't make more calls than needed", async () => {
            request.rejects(new Error("custom error"));
            assert.isRejected(retryer(), "custom error"); // actual call to the retrier, pass, rejected
            sinon.assert.callCount(request, 1)) // pass, just one call so far

            // that was added to catch this issue
            return await new Promise(resolve => {
                // wait to see if there any retries in the background
                // even if the call was rejected
                setTimeout(() => {
                    resolve(sinon.assert.callCount(request, 1)); // will fail, as more requests will be done
                }, 500);
            });
        });

@przemyslaw-wlodek
Copy link

przemyslaw-wlodek commented Apr 9, 2021

I've faced the same issue and for me it feels like an unexpected behaviour.
Unfortunately it was not easy to catch until I've seen application logs that appeared far after the promise execution (bail).

I think that retries should be simply stopped after bail was called.

Pattern provided in README example breaks TypeScript typings:

  if (403 === res.status) {
    // don't retry upon 403
    bail(new Error('Unauthorized'))
    return
  }

which leads to an awful type-casting e.g.

  if (403 === res.status) {
    // don't retry upon 403
    bail(new Error('Unauthorized'))
    return (null as unknown) as PromiseReturnType;
  }

@cdimitroulas
Copy link

I agree with @przemyslaw-wlodek - the Typescript typings are not great.

It would be great if bail had the type of (e: Error) => never. This would allow the return type of the retry function to not have a potential null or undefined.

@rayzhangcl
Copy link

just found this issue before pushing my code to prod. any plan to get this fixed? it's been 2 years since the issue was reported.

@albertodiazdorado
Copy link

Bumping this, we would love to have this thing fixed

@glebsts
Copy link

glebsts commented Nov 18, 2022

+1 :(

@agalatan
Copy link

agalatan commented Nov 27, 2022

I think this wrapper should do it. Once bailed, it won't retry.
I can open a PR, if I get first a green-light from the maintainer.

const retry = require('async-retry');

const _retry = (cb, ...opts) => {
  let bailedOut;

  return retry(
    (bail, ...rest) =>
      new Promise((resolve, reject) => {
        cb((error) => {
          bailedOut = true;
          bail(error);
        }, ...rest).then(
          (result) => {
              if (!bailedOut) {
                resolve(result);
              }
          },
          (error) => {
              if (!bailedOut) {
                reject(error);
              }
          }
        );
      }),
    ...opts
  );
};

@agalatan
Copy link

agalatan commented Nov 28, 2022

Is this actually still an issue? I wrote these Jest tests that pass, which suggest it works just fine:

// @flow strict
import retry from 'async-retry';

describe('async-retry', () => {
  const bailError = new Error('BAIL');
  const rejectError = new Error('BOOM');

  test('retries N times', async () => {
    const cb = jest.fn().mockImplementation(
      async () => {
        throw rejectError;
      }
    );

    await expect(
      retry(cb, { retries: 3, factor: 1, minTimeout: 0 })
    ).rejects.toBe(rejectError);

    expect(cb).toHaveBeenCalledTimes(4);
  });

  test('does not keep retrying once bail out is called', async () => {
    const cb = jest.fn().mockImplementation(async (bail, count) => {
      console.log('START');

      if (count === 2) {
        bail(bailError);
        console.log('BAIL');
      }
      throw rejectError;
    });

    await expect(
      retry(cb, { retries: 5, factor: 1, minTimeout: 0 })
    ).rejects.toBe(bailError);

    expect(cb).toHaveBeenCalledTimes(2);
  });
});

@albertodiazdorado
Copy link

@agalatan any chance of you opening a PR? I really would love to have this thing fixed :D

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

10 participants