Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat!: Remove request fallback, Update Docs, Update Types #107

Merged
merged 8 commits into from
Oct 11, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
5 changes: 3 additions & 2 deletions index.d.ts
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
declare module 'retry-request' {
// eslint-disable-next-line node/no-unpublished-import
import * as request from 'request';
import * as teenyRequest from 'teeny-request';

namespace retryRequest {
defaults = retryRequest.Options;
function getNextRetryDelay(retryNumber: number): void;
interface Options {
objectMode?: boolean;
request?: typeof request;
request: typeof request | typeof teenyRequest;
retries?: number;
noResponseRetries?: number;
currentRetryAttempt?: number;
Expand Down
23 changes: 16 additions & 7 deletions index.js
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,10 @@ const DEFAULTS = {
};

function retryRequest(requestOpts, opts, callback) {
if (typeof requestOpts === 'string') {
requestOpts = {url: requestOpts};
}

const streamMode = typeof arguments[arguments.length - 1] !== 'function';

if (typeof opts === 'function') {
Expand All @@ -71,12 +75,7 @@ function retryRequest(requestOpts, opts, callback) {
opts = extend({}, DEFAULTS, opts);

if (typeof opts.request === 'undefined') {
try {
// eslint-disable-next-line node/no-unpublished-require
opts.request = require('request');
} catch (e) {
throw new Error('A request library must be provided to retry-request.');
}
throw new Error('A request library must be provided to retry-request.');
}

let currentRetryAttempt = opts.currentRetryAttempt;
Expand Down Expand Up @@ -131,9 +130,17 @@ function retryRequest(requestOpts, opts, callback) {
}

function makeRequest() {
let finishHandled = false;
currentRetryAttempt++;
debug(`Current retry attempt: ${currentRetryAttempt}`);

function handleFinish(args = []) {
if (!finishHandled) {
finishHandled = true;
retryStream.emit('complete', ...args);
}
}

if (streamMode) {
streamResponseHandled = false;

Expand Down Expand Up @@ -164,7 +171,8 @@ function retryRequest(requestOpts, opts, callback) {
streamResponseHandled = true;
onResponse(null, resp, body);
})
.on('complete', retryStream.emit.bind(retryStream, 'complete'));
.on('complete', (...params) => handleFinish(params))
.on('finish', (...params) => handleFinish(params));

requestStream.pipe(delayStream);
} else {
Expand Down Expand Up @@ -270,4 +278,5 @@ function getNextRetryDelay(config) {
);
}

module.exports.defaults = DEFAULTS;
module.exports.getNextRetryDelay = getNextRetryDelay;
10 changes: 5 additions & 5 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
"docs-test": "linkinator docs",
"fix": "gts fix",
"lint": "gts check",
"test": "mocha --timeout 0",
"test": "mocha --timeout 30000",
"system-test": ""
},
"files": [
Expand All @@ -30,20 +30,20 @@
"node": ">=14"
},
"dependencies": {
"@types/request": "^2.48.8",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this still needed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In case someone still wants to use requests (or similar custom interface) we still support its types in index.d.ts.

"debug": "^4.1.1",
"extend": "^3.0.2"
"extend": "^3.0.2",
"teeny-request": "^9.0.0"
},
"devDependencies": {
"@types/request": "^2.48.8",
"async": "^3.0.1",
"gts": "^5.0.0",
"jsdoc": "^4.0.0",
"jsdoc-fresh": "^3.0.0",
"jsdoc-region-tag": "^3.0.0",
"linkinator": "^4.0.0",
"lodash.range": "^3.2.0",
"mocha": "^9.2.2",
"request": "^2.87.0",
"mocha": "^10.2.0",
"typescript": "^5.1.6"
}
}
50 changes: 25 additions & 25 deletions readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,31 +3,30 @@
|Retry a [request][request] with built-in [exponential backoff](https://developers.google.com/analytics/devguides/reporting/core/v3/coreErrors#backoff).

```sh
$ npm install --save request
$ npm install --save teeny-request
$ npm install --save retry-request
```

```js
var request = require('retry-request', {
request: require('request')
request: require('teeny-request'),
});
```

It should work the same as `request` in both callback mode and stream mode.
It should work the same as `request` and `teeny-request` in both callback mode and stream mode.

Note: This module only works when used as a readable stream, i.e. POST requests aren't supported ([#3](https://github.com/stephenplusplus/retry-request/issues/3)).
Note: This module only works when used as a readable stream, i.e. POST requests aren't supported ([#3](https://github.com/stephenplusplus/retry-request/issues/3)).

## Do I need to install `request`?

Yes! You must independently install `request` and provide it to this library:
Yes! You must independently install `teeny-request` OR `request` (_deprecated_) and provide it to this library:

```js
var request = require('retry-request', {
request: require('request')
request: require('teeny-request'),
});
```

*The code will actually look for the `request` module automatically to save you this step. But, being explicit like in the example is also welcome.*

#### Callback

`urlThatReturns503` will be requested 3 total times before giving up and executing the callback.
Expand All @@ -49,17 +48,20 @@ request(urlThatReturns503)

## Can I monitor what retry-request is doing internally?

Yes! This project uses [debug](https://www.npmjs.com/package/debug) to provide the current retry attempt, each response status, and the delay computed until the next retry attempt is made. To enable the debug mode, set the environment variable `DEBUG` to *retry-request*.
Yes! This project uses [debug](https://www.npmjs.com/package/debug) to provide the current retry attempt, each response status, and the delay computed until the next retry attempt is made. To enable the debug mode, set the environment variable `DEBUG` to _retry-request_.

(Thanks for the implementation, @yihaozhadan!)

## request(requestOptions, [opts], [cb])

### requestOptions

Passed directly to `request`. See the list of options supported: https://github.com/request/request/#requestoptions-callback.
Passed directly to `request` or `teeny-request`. See the list of options supported:

### opts *(optional)*
- https://github.com/request/request/#requestoptions-callback
- https://github.com/googleapis/teeny-request#teenyrequestoptions-callback

### opts _(optional)_

#### `opts.noResponseRetries`

Expand All @@ -71,7 +73,7 @@ The number of times to retry after a response fails to come through, such as a D

```js
var opts = {
noResponseRetries: 0
noResponseRetries: 0,
};

request(url, opts, function (err, resp, body) {
Expand All @@ -96,7 +98,7 @@ Default: `2`

```js
var opts = {
retries: 4
retries: 4,
};

request(urlThatReturns503, opts, function (err, resp, body) {
Expand All @@ -113,7 +115,7 @@ Default: `0`

```js
var opts = {
currentRetryAttempt: 1
currentRetryAttempt: 1,
};

request(urlThatReturns503, opts, function (err, resp, body) {
Expand All @@ -131,7 +133,7 @@ Default: Returns `true` if [http.incomingMessage](https://nodejs.org/api/http.ht
var opts = {
shouldRetryFn: function (incomingHttpMessage) {
return incomingHttpMessage.statusMessage !== 'OK';
}
},
};

request(urlThatReturnsNonOKStatusMessage, opts, function (err, resp, body) {
Expand All @@ -146,21 +148,19 @@ request(urlThatReturnsNonOKStatusMessage, opts, function (err, resp, body) {

Type: `Function`

Default: `try { require('request') }`

If we cannot locate `request`, we will throw an error advising you to provide it explicitly.
If we not provided we will throw an error advising you to provide it.

*NOTE: If you override the request function, and it returns a stream in object mode, be sure to set `opts.objectMode` to `true`.*
_NOTE: If you override the request function, and it returns a stream in object mode, be sure to set `opts.objectMode` to `true`._

```js
var originalRequest = require('request').defaults({
var originalRequest = require('teeny-request').defaults({
pool: {
maxSockets: Infinity
}
maxSockets: Infinity,
},
});

var opts = {
request: originalRequest
request: originalRequest,
};

request(urlThatReturns503, opts, function (err, resp, body) {
Expand Down Expand Up @@ -190,9 +190,9 @@ Type: `Number`

Default: `600`

The length of time to keep retrying in seconds. The last sleep period will be shortened as necessary, so that the last retry runs at deadline (and not considerably beyond it). The total time starting from when the initial request is sent, after which an error will be returned, regardless of the retrying attempts made meanwhile.
The length of time to keep retrying in seconds. The last sleep period will be shortened as necessary, so that the last retry runs at deadline (and not considerably beyond it). The total time starting from when the initial request is sent, after which an error will be returned, regardless of the retrying attempts made meanwhile.

### cb *(optional)*
### cb _(optional)_

Passed directly to `request`. See the callback section: https://github.com/request/request/#requestoptions-callback.

Expand Down
7 changes: 5 additions & 2 deletions test.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,15 @@ const async = require('async');
const range = require('lodash.range');
const {describe, it, beforeEach} = require('mocha');
const {PassThrough} = require('stream');
const {teenyRequest} = require('teeny-request');

const retryRequest = require('./index.js');

retryRequest.defaults.request = teenyRequest.defaults();

describe('retry-request', () => {
const URI_404 = 'http://yahoo.com/theblahstore';
const URI_200 = 'http://yahoo.com/';
const URI_404 = 'http://google.com/theblahstore';
const URI_200 = 'http://google.com/';
const URI_NON_EXISTENT = 'http://theblahstore';

describe('streams', () => {
Expand Down