diff --git a/README.md b/README.md index b10b4e48a..23f0d67c7 100644 --- a/README.md +++ b/README.md @@ -1,15 +1,18 @@ -# Request — Simplified HTTP client + +# Request - Simplified HTTP client + [![npm package](https://nodei.co/npm/request.png?downloads=true&downloadRank=true&stars=true)](https://nodei.co/npm/request/) -[![Build status](https://img.shields.io/travis/request/request.svg?style=flat)](https://travis-ci.org/request/request) -[![Coverage](https://img.shields.io/coveralls/request/request.svg?style=flat)](https://coveralls.io/r/request/request) -[![Gitter](https://img.shields.io/badge/gitter-join_chat-blue.svg?style=flat)](https://gitter.im/request/request?utm_source=badge) +[![Build status](https://img.shields.io/travis/request/request.svg?style=flat-square)](https://travis-ci.org/request/request) +[![Coverage](https://img.shields.io/coveralls/request/request.svg?style=flat-square)](https://coveralls.io/r/request/request) +[![Gitter](https://img.shields.io/badge/gitter-join_chat-blue.svg?style=flat-square)](https://gitter.im/request/request?utm_source=badge) + ## Super simple to use Request is designed to be the simplest way possible to make http calls. It supports HTTPS and follows redirects by default. -```javascript +```js var request = require('request'); request('http://www.google.com', function (error, response, body) { if (!error && response.statusCode == 200) { @@ -18,29 +21,51 @@ request('http://www.google.com', function (error, response, body) { }) ``` + +## Table of contents + +- [Streaming](#streaming) +- [Forms](#forms) +- [HTTP Authentication](#http-authentication) +- [Custom HTTP Headers](#custom-http-headers) +- [OAuth Signing](#oauth-signing) +- [Proxies](#proxies) +- [Unix Domain Sockets](#unix-domain-sockets) +- [TLS/SSL Protocol](#tlsssl-protocol) +- [**All Available Options**](#requestoptions-callback) + +Request also offers [convenience methods](#convenience-methods) like +`request.defaults` and `request.post`, and there are +lots of [usage examples](#examples) and several +[debugging techniques](#debugging). + + +--- + + ## Streaming You can stream any response to a file stream. -```javascript +```js request('http://google.com/doodle.png').pipe(fs.createWriteStream('doodle.png')) ``` You can also stream a file to a PUT or POST request. This method will also check the file extension against a mapping of file extensions to content-types (in this case `application/json`) and use the proper `content-type` in the PUT request (if the headers don’t already provide one). -```javascript +```js fs.createReadStream('file.json').pipe(request.put('http://mysite.com/obj.json')) ``` Request can also `pipe` to itself. When doing so, `content-type` and `content-length` are preserved in the PUT headers. -```javascript +```js request.get('http://google.com/img.png').pipe(request.put('http://mysite.com/img.png')) ``` Request emits a "response" event when a response is received. The `response` argument will be an instance of [http.IncomingMessage](http://nodejs.org/api/http.html#http_http_incomingmessage). -```javascript +```js request .get('http://google.com/img.png') .on('response', function(response) { @@ -52,7 +77,7 @@ request To easily handle errors when streaming requests, listen to the `error` event before piping: -```javascript +```js request .get('http://mysite.com/doodle.png') .on('error', function(err) { @@ -63,7 +88,7 @@ request Now let’s get fancy. -```javascript +```js http.createServer(function (req, resp) { if (req.url === '/doodle.png') { if (req.method === 'PUT') { @@ -77,7 +102,7 @@ http.createServer(function (req, resp) { You can also `pipe()` from `http.ServerRequest` instances, as well as to `http.ServerResponse` instances. The HTTP method, headers, and entity-body data will be sent. Which means that, if you don't really care about security, you can do: -```javascript +```js http.createServer(function (req, resp) { if (req.url === '/doodle.png') { var x = request('http://mysite.com/doodle.png') @@ -89,13 +114,13 @@ http.createServer(function (req, resp) { And since `pipe()` returns the destination stream in ≥ Node 0.5.x you can do one line proxying. :) -```javascript +```js req.pipe(request('http://mysite.com/doodle.png')).pipe(resp) ``` Also, none of this new functionality conflicts with requests previous features, it just expands them. -```javascript +```js var r = request.defaults({'proxy':'http://localproxy.com'}) http.createServer(function (req, resp) { @@ -107,139 +132,22 @@ http.createServer(function (req, resp) { You can still use intermediate proxies, the requests will still follow HTTP forwards, etc. -## Proxies - -If you specify a `proxy` option, then the request (and any subsequent -redirects) will be sent via a connection to the proxy server. - -If your endpoint is an `https` url, and you are using a proxy, then -request will send a `CONNECT` request to the proxy server *first*, and -then use the supplied connection to connect to the endpoint. - -That is, first it will make a request like: - -``` -HTTP/1.1 CONNECT endpoint-server.com:80 -Host: proxy-server.com -User-Agent: whatever user agent you specify -``` - -and then the proxy server make a TCP connection to `endpoint-server` -on port `80`, and return a response that looks like: - -``` -HTTP/1.1 200 OK -``` - -At this point, the connection is left open, and the client is -communicating directly with the `endpoint-server.com` machine. - -See [the wikipedia page on HTTP Tunneling](http://en.wikipedia.org/wiki/HTTP_tunnel) -for more information. - -By default, when proxying `http` traffic, request will simply make a -standard proxied `http` request. This is done by making the `url` -section of the initial line of the request a fully qualified url to -the endpoint. - -For example, it will make a single request that looks like: - -``` -HTTP/1.1 GET http://endpoint-server.com/some-url -Host: proxy-server.com -Other-Headers: all go here - -request body or whatever -``` - -Because a pure "http over http" tunnel offers no additional security -or other features, it is generally simpler to go with a -straightforward HTTP proxy in this case. However, if you would like -to force a tunneling proxy, you may set the `tunnel` option to `true`. - -You can also make a standard proxied `http` request by explicitly setting -`tunnel : false`, but **note that this will allow the proxy to see the traffic -to/from the destination server**. - -If you are using a tunneling proxy, you may set the -`proxyHeaderWhiteList` to share certain headers with the proxy. - -You can also set the `proxyHeaderExclusiveList` to share certain -headers only with the proxy and not with destination host. - -By default, this set is: - -``` -accept -accept-charset -accept-encoding -accept-language -accept-ranges -cache-control -content-encoding -content-language -content-length -content-location -content-md5 -content-range -content-type -connection -date -expect -max-forwards -pragma -proxy-authorization -referer -te -transfer-encoding -user-agent -via -``` - -Note that, when using a tunneling proxy, the `proxy-authorization` -header and any headers from custom `proxyHeaderExclusiveList` are -*never* sent to the endpoint server, but only to the proxy server. +[back to top](#table-of-contents) -### Controlling proxy behaviour using environment variables -The following environment variables are respected by `request`: - - * `HTTP_PROXY` / `http_proxy` - * `HTTPS_PROXY` / `https_proxy` - * `NO_PROXY` / `no_proxy` - -When `HTTP_PROXY` / `http_proxy` are set, they will be used to proxy non-SSL requests that do not have an explicit `proxy` configuration option present. Similarly, `HTTPS_PROXY` / `https_proxy` will be respected for SSL requests that do not have an explicit `proxy` configuration option. It is valid to define a proxy in one of the environment variables, but then override it for a specific request, using the `proxy` configuration option. Furthermore, the `proxy` configuration option can be explicitly set to false / null to opt out of proxying altogether for that request. - -`request` is also aware of the `NO_PROXY`/`no_proxy` environment variables. These variables provide a granular way to opt out of proxying, on a per-host basis. It should contain a comma separated list of hosts to opt out of proxying. It is also possible to opt of proxying when a particular destination port is used. Finally, the variable may be set to `*` to opt out of the implicit proxy configuration of the other environment variables. - -Here's some examples of valid `no_proxy` values: - - * `google.com` - don't proxy HTTP/HTTPS requests to Google. - * `google.com:443` - don't proxy HTTPS requests to Google, but *do* proxy HTTP requests to Google. - * `google.com:443, yahoo.com:80` - don't proxy HTTPS requests to Google, and don't proxy HTTP requests to Yahoo! - * `*` - ignore `https_proxy`/`http_proxy` environment variables altogether. - -## UNIX Socket - -`request` supports making requests to [UNIX Domain Sockets](http://en.wikipedia.org/wiki/Unix_domain_socket). To make one, use the following URL scheme: - -```javascript -/* Pattern */ 'http://unix:SOCKET:PATH' -/* Example */ request.get('http://unix:/absolute/path/to/unix.socket:/request/path') -``` - -Note: The `SOCKET` path is assumed to be absolute to the root of the host file system. +--- ## Forms `request` supports `application/x-www-form-urlencoded` and `multipart/form-data` form uploads. For `multipart/related` refer to the `multipart` API. + #### application/x-www-form-urlencoded (URL-Encoded Forms) URL-encoded forms are simple. -```javascript +```js request.post('http://service.com/upload', {form:{key:'value'}}) // or request.post('http://service.com/upload').form({key:'value'}) @@ -247,12 +155,13 @@ request.post('http://service.com/upload').form({key:'value'}) request.post({url:'http://service.com/upload', form: {key:'value'}}, function(err,httpResponse,body){ /* ... */ }) ``` + #### multipart/form-data (Multipart Form Uploads) For `multipart/form-data` we use the [form-data](https://github.com/felixge/node-form-data) library by [@felixge](https://github.com/felixge). For the most cases, you can pass your upload form data via the `formData` option. -```javascript +```js var formData = { // Pass a simple key-value pair my_field: 'my_value', @@ -286,7 +195,7 @@ request.post({url:'http://service.com/upload', formData: formData}, function opt For advanced cases, you can access the form-data object itself via `r.form()`. This can be modified until the request is fired on the next cycle of the event-loop. (Note that this calling `form()` will clear the currently set form data for that request.) -```javascript +```js // NOTE: Advanced use-case, for normal use see 'formData' usage above var r = request.post('http://service.com/upload', function optionalCallback(err, httpResponse, body) { // ... @@ -297,11 +206,12 @@ form.append('custom_file', fs.createReadStream(__dirname + '/unicycle.jpg'), {fi ``` See the [form-data README](https://github.com/felixge/node-form-data) for more information & examples. + #### multipart/related Some variations in different HTTP implementations require a newline/CRLF before, after, or both before and after the boundary of a `multipart/related` request (using the multipart option). This has been observed in the .NET WebAPI version 4.0. You can turn on a boundary preambleCRLF or postamble by passing them as `true` to your request options. -```javascript +```js request({ method: 'PUT', preambleCRLF: true, @@ -335,10 +245,15 @@ Some variations in different HTTP implementations require a newline/CRLF before, }) ``` +[back to top](#table-of-contents) + + +--- + ## HTTP Authentication -```javascript +```js request.get('http://some.server.com/').auth('username', 'password', false); // or request.get('http://some.server.com/', { @@ -378,7 +293,7 @@ Note that you can also specify basic authentication using the URL itself, as detailed in [RFC 1738](http://www.ietf.org/rfc/rfc1738.txt). Simply pass the `user:password` before the host with an `@` sign: -```javascript +```js var username = 'username', password = 'password', url = 'http://' + username + ':' + password + '@some.server.com'; @@ -398,13 +313,53 @@ available. The value may be either a `String` or a `Function` returning a used in conjuction with `defaults` to allow a single function to supply the last known token at the time of sending a request, or to compute one on the fly. +[back to top](#table-of-contents) + + +--- + + +## Custom HTTP Headers + +HTTP Headers, such as `User-Agent`, can be set in the `options` object. +In the example below, we call the github API to find out the number +of stars and forks for the request repository. This requires a +custom `User-Agent` header as well as https. + +```js +var request = require('request'); + +var options = { + url: 'https://api.github.com/repos/request/request', + headers: { + 'User-Agent': 'request' + } +}; + +function callback(error, response, body) { + if (!error && response.statusCode == 200) { + var info = JSON.parse(body); + console.log(info.stargazers_count + " Stars"); + console.log(info.forks_count + " Forks"); + } +} + +request(options, callback); +``` + +[back to top](#table-of-contents) + + +--- + + ## OAuth Signing [OAuth version 1.0](https://tools.ietf.org/html/rfc5849) is supported. The default signing algorithm is [HMAC-SHA1](https://tools.ietf.org/html/rfc5849#section-3.4.2): -```javascript +```js // OAuth1.0 - 3-legged server side flow (Twitter example) // step 1 var qs = require('querystring') @@ -478,34 +433,148 @@ section of the oauth1 spec: options object. * `transport_method` defaults to `'header'` -## Custom HTTP Headers +[back to top](#table-of-contents) -HTTP Headers, such as `User-Agent`, can be set in the `options` object. -In the example below, we call the github API to find out the number -of stars and forks for the request repository. This requires a -custom `User-Agent` header as well as https. -```javascript -var request = require('request'); +--- -var options = { - url: 'https://api.github.com/repos/request/request', - headers: { - 'User-Agent': 'request' - } -}; -function callback(error, response, body) { - if (!error && response.statusCode == 200) { - var info = JSON.parse(body); - console.log(info.stargazers_count + " Stars"); - console.log(info.forks_count + " Forks"); - } -} +## Proxies -request(options, callback); +If you specify a `proxy` option, then the request (and any subsequent +redirects) will be sent via a connection to the proxy server. + +If your endpoint is an `https` url, and you are using a proxy, then +request will send a `CONNECT` request to the proxy server *first*, and +then use the supplied connection to connect to the endpoint. + +That is, first it will make a request like: + +``` +HTTP/1.1 CONNECT endpoint-server.com:80 +Host: proxy-server.com +User-Agent: whatever user agent you specify +``` + +and then the proxy server make a TCP connection to `endpoint-server` +on port `80`, and return a response that looks like: + +``` +HTTP/1.1 200 OK +``` + +At this point, the connection is left open, and the client is +communicating directly with the `endpoint-server.com` machine. + +See [the wikipedia page on HTTP Tunneling](http://en.wikipedia.org/wiki/HTTP_tunnel) +for more information. + +By default, when proxying `http` traffic, request will simply make a +standard proxied `http` request. This is done by making the `url` +section of the initial line of the request a fully qualified url to +the endpoint. + +For example, it will make a single request that looks like: + +``` +HTTP/1.1 GET http://endpoint-server.com/some-url +Host: proxy-server.com +Other-Headers: all go here + +request body or whatever +``` + +Because a pure "http over http" tunnel offers no additional security +or other features, it is generally simpler to go with a +straightforward HTTP proxy in this case. However, if you would like +to force a tunneling proxy, you may set the `tunnel` option to `true`. + +You can also make a standard proxied `http` request by explicitly setting +`tunnel : false`, but **note that this will allow the proxy to see the traffic +to/from the destination server**. + +If you are using a tunneling proxy, you may set the +`proxyHeaderWhiteList` to share certain headers with the proxy. + +You can also set the `proxyHeaderExclusiveList` to share certain +headers only with the proxy and not with destination host. + +By default, this set is: + +``` +accept +accept-charset +accept-encoding +accept-language +accept-ranges +cache-control +content-encoding +content-language +content-length +content-location +content-md5 +content-range +content-type +connection +date +expect +max-forwards +pragma +proxy-authorization +referer +te +transfer-encoding +user-agent +via +``` + +Note that, when using a tunneling proxy, the `proxy-authorization` +header and any headers from custom `proxyHeaderExclusiveList` are +*never* sent to the endpoint server, but only to the proxy server. + + +### Controlling proxy behaviour using environment variables + +The following environment variables are respected by `request`: + + * `HTTP_PROXY` / `http_proxy` + * `HTTPS_PROXY` / `https_proxy` + * `NO_PROXY` / `no_proxy` + +When `HTTP_PROXY` / `http_proxy` are set, they will be used to proxy non-SSL requests that do not have an explicit `proxy` configuration option present. Similarly, `HTTPS_PROXY` / `https_proxy` will be respected for SSL requests that do not have an explicit `proxy` configuration option. It is valid to define a proxy in one of the environment variables, but then override it for a specific request, using the `proxy` configuration option. Furthermore, the `proxy` configuration option can be explicitly set to false / null to opt out of proxying altogether for that request. + +`request` is also aware of the `NO_PROXY`/`no_proxy` environment variables. These variables provide a granular way to opt out of proxying, on a per-host basis. It should contain a comma separated list of hosts to opt out of proxying. It is also possible to opt of proxying when a particular destination port is used. Finally, the variable may be set to `*` to opt out of the implicit proxy configuration of the other environment variables. + +Here's some examples of valid `no_proxy` values: + + * `google.com` - don't proxy HTTP/HTTPS requests to Google. + * `google.com:443` - don't proxy HTTPS requests to Google, but *do* proxy HTTP requests to Google. + * `google.com:443, yahoo.com:80` - don't proxy HTTPS requests to Google, and don't proxy HTTP requests to Yahoo! + * `*` - ignore `https_proxy`/`http_proxy` environment variables altogether. + +[back to top](#table-of-contents) + + +--- + + +## UNIX Domain Sockets + +`request` supports making requests to [UNIX Domain Sockets](http://en.wikipedia.org/wiki/Unix_domain_socket). To make one, use the following URL scheme: + +```js +/* Pattern */ 'http://unix:SOCKET:PATH' +/* Example */ request.get('http://unix:/absolute/path/to/unix.socket:/request/path') ``` +Note: The `SOCKET` path is assumed to be absolute to the root of the host file system. + +[back to top](#table-of-contents) + + +--- + + ## TLS/SSL Protocol TLS/SSL Protocol options, such as `cert`, `key` and `passphrase`, can be @@ -513,7 +582,7 @@ set in the `agentOptions` property of the `options` object. In the example below, we call an API requires client side SSL certificate (in PEM format) with passphrase protected private key (in PEM format) and disable the SSLv3 protocol: -```javascript +```js var fs = require('fs') , path = require('path') , certFile = path.resolve(__dirname, 'ssl/client.crt') @@ -537,7 +606,7 @@ request.get(options); It is able to force using SSLv3 only by specifying `secureProtocol`: -```javascript +```js request.get({ url: 'https://api.some-server.com/', agentOptions: { @@ -550,7 +619,7 @@ It is possible to accept other certificates than those signed by generally allow This can be useful, for example, when using self-signed certificates. To allow a different certificate, you can specify the signing CA by adding the contents of the CA's certificate file to the `agentOptions`: -```javascript +```js request.get({ url: 'https://api.some-server.com/', agentOptions: { @@ -559,76 +628,103 @@ request.get({ }); ``` +[back to top](#table-of-contents) + + +--- + + ## request(options, callback) The first argument can be either a `url` or an `options` object. The only required option is `uri`; all others are optional. -* `uri` || `url` - fully qualified uri or a parsed url object from `url.parse()` -* `baseUrl` - fully qualified uri string used as the base url. Most useful with `request.defaults`, for example when you want to do many requests to the same domain. If `baseUrl` is `https://example.com/api/`, then requesting `/end/point?test=true` will fetch `https://example.com/api/end/point?test=true`. When `baseUrl` is given, `uri` must also be a string. -* `qs` - object containing querystring values to be appended to the `uri` -* `qsParseOptions` - object containing options to pass to the [qs.parse](https://github.com/hapijs/qs#parsing-objects) method or [querystring.parse](https://nodejs.org/docs/v0.12.0/api/querystring.html#querystring_querystring_parse_str_sep_eq_options) method -* `qsStringifyOptions` - object containing options to pass to the [qs.stringify](https://github.com/hapijs/qs#stringifying) method or to the [querystring.stringify](https://nodejs.org/docs/v0.12.0/api/querystring.html#querystring_querystring_stringify_obj_sep_eq_options) method. For example, to change the way arrays are converted to query strings pass the `arrayFormat` option with one of `indices|brackets|repeat` -* `useQuerystring` - If true, use `querystring` to stringify and parse +- `uri` || `url` - fully qualified uri or a parsed url object from `url.parse()` +- `baseUrl` - fully qualified uri string used as the base url. Most useful with `request.defaults`, for example when you want to do many requests to the same domain. If `baseUrl` is `https://example.com/api/`, then requesting `/end/point?test=true` will fetch `https://example.com/api/end/point?test=true`. When `baseUrl` is given, `uri` must also be a string. +- `method` - http method (default: `"GET"`) +- `headers` - http headers (default: `{}`) + +--- + +- `qs` - object containing querystring values to be appended to the `uri` +- `qsParseOptions` - object containing options to pass to the [qs.parse](https://github.com/hapijs/qs#parsing-objects) method or [querystring.parse](https://nodejs.org/docs/v0.12.0/api/querystring.html#querystring_querystring_parse_str_sep_eq_options) method +- `qsStringifyOptions` - object containing options to pass to the [qs.stringify](https://github.com/hapijs/qs#stringifying) method or to the [querystring.stringify](https://nodejs.org/docs/v0.12.0/api/querystring.html#querystring_querystring_stringify_obj_sep_eq_options) method. For example, to change the way arrays are converted to query strings pass the `arrayFormat` option with one of `indices|brackets|repeat` +- `useQuerystring` - If true, use `querystring` to stringify and parse querystrings, otherwise use `qs` (default: `false`). Set this option to `true` if you need arrays to be serialized as `foo=bar&foo=baz` instead of the default `foo[0]=bar&foo[1]=baz`. -* `method` - http method (default: `"GET"`) -* `headers` - http headers (default: `{}`) -* `body` - entity body for PATCH, POST and PUT requests. Must be a `Buffer` or `String`, unless `json` is `true`. If `json` is `true`, then `body` must be a JSON-serializable object. -* `form` - when passed an object or a querystring, this sets `body` to a querystring representation of value, and adds `Content-type: application/x-www-form-urlencoded` header. When passed no options, a `FormData` instance is returned (and is piped to request). See "Forms" section above. -* `formData` - Data to pass for a `multipart/form-data` request. See + +--- + +- `body` - entity body for PATCH, POST and PUT requests. Must be a `Buffer` or `String`, unless `json` is `true`. If `json` is `true`, then `body` must be a JSON-serializable object. +- `form` - when passed an object or a querystring, this sets `body` to a querystring representation of value, and adds `Content-type: application/x-www-form-urlencoded` header. When passed no options, a `FormData` instance is returned (and is piped to request). See "Forms" section above. +- `formData` - Data to pass for a `multipart/form-data` request. See [Forms](#forms) section above. -* `multipart` - array of objects which contain their own headers and `body` +- `multipart` - array of objects which contain their own headers and `body` attributes. Sends a `multipart/related` request. See [Forms](#forms) section above. - * Alternatively you can pass in an object `{chunked: false, data: []}` where + - Alternatively you can pass in an object `{chunked: false, data: []}` where `chunked` is used to specify whether the request is sent in [chunked transfer encoding](https://en.wikipedia.org/wiki/Chunked_transfer_encoding) In non-chunked requests, data items with body streams are not allowed. -* `auth` - A hash containing values `user` || `username`, `pass` || `password`, and `sendImmediately` (optional). See documentation above. -* `json` - sets `body` but to JSON representation of value and adds `Content-type: application/json` header. Additionally, parses the response body as JSON. -* `jsonReviver` - a [reviver function](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/parse) that will be passed to `JSON.parse()` when parsing a JSON response body. -* `preambleCRLF` - append a newline/CRLF before the boundary of your `multipart/form-data` request. -* `postambleCRLF` - append a newline/CRLF at the end of the boundary of your `multipart/form-data` request. -* `followRedirect` - follow HTTP 3xx responses as redirects (default: `true`). This property can also be implemented as function which gets `response` object as a single argument and should return `true` if redirects should continue or `false` otherwise. -* `followAllRedirects` - follow non-GET HTTP 3xx responses as redirects (default: `false`) -* `maxRedirects` - the maximum number of redirects to follow (default: `10`) -* `encoding` - Encoding to be used on `setEncoding` of response data. If `null`, the `body` is returned as a `Buffer`. Anything else **(including the default value of `undefined`)** will be passed as the [encoding](http://nodejs.org/api/buffer.html#buffer_buffer) parameter to `toString()` (meaning this is effectively `utf8` by default). -* `pool` - An object describing which agents to use for the request. If this option is omitted the request will use the global agent (as long as [your options allow for it](request.js#L747)). Otherwise, request will search the pool for your custom agent. If no custom agent is found, a new agent will be created and added to the pool. - * A `maxSockets` property can also be provided on the `pool` object to set the max number of sockets for all agents created (ex: `pool: {maxSockets: Infinity}`). - * Note that if you are sending multiple requests in a loop and creating +- `preambleCRLF` - append a newline/CRLF before the boundary of your `multipart/form-data` request. +- `postambleCRLF` - append a newline/CRLF at the end of the boundary of your `multipart/form-data` request. +- `json` - sets `body` but to JSON representation of value and adds `Content-type: application/json` header. Additionally, parses the response body as JSON. +- `jsonReviver` - a [reviver function](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/parse) that will be passed to `JSON.parse()` when parsing a JSON response body. + +--- + +- `auth` - A hash containing values `user` || `username`, `pass` || `password`, and `sendImmediately` (optional). See documentation above. +- `oauth` - Options for OAuth HMAC-SHA1 signing. See documentation above. +- `hawk` - Options for [Hawk signing](https://github.com/hueniverse/hawk). The `credentials` key must contain the necessary signing info, [see hawk docs for details](https://github.com/hueniverse/hawk#usage-example). +- `aws` - `object` containing AWS signing information. Should have the properties `key`, `secret`. Also requires the property `bucket`, unless you’re specifying your `bucket` as part of the path, or the request doesn’t use a bucket (i.e. GET Services) +- `httpSignature` - Options for the [HTTP Signature Scheme](https://github.com/joyent/node-http-signature/blob/master/http_signing.md) using [Joyent's library](https://github.com/joyent/node-http-signature). The `keyId` and `key` properties must be specified. See the docs for other options. + +--- + +- `followRedirect` - follow HTTP 3xx responses as redirects (default: `true`). This property can also be implemented as function which gets `response` object as a single argument and should return `true` if redirects should continue or `false` otherwise. +- `followAllRedirects` - follow non-GET HTTP 3xx responses as redirects (default: `false`) +- `maxRedirects` - the maximum number of redirects to follow (default: `10`) + +--- + +- `encoding` - Encoding to be used on `setEncoding` of response data. If `null`, the `body` is returned as a `Buffer`. Anything else **(including the default value of `undefined`)** will be passed as the [encoding](http://nodejs.org/api/buffer.html#buffer_buffer) parameter to `toString()` (meaning this is effectively `utf8` by default). +- `gzip` - If `true`, add an `Accept-Encoding` header to request compressed content encodings from the server (if not already present) and decode supported content encodings in the response. **Note:** Automatic decoding of the response content is performed on the body data returned through `request` (both through the `request` stream and passed to the callback function) but is not performed on the `response` stream (available from the `response` event) which is the unmodified `http.IncomingMessage` object which may contain compressed data. See example below. +- `jar` - If `true` and `tough-cookie` is installed, remember cookies for future use (or define your custom cookie jar; see examples section) + +--- + +- `pool` - An object describing which agents to use for the request. If this option is omitted the request will use the global agent (as long as [your options allow for it](request.js#L747)). Otherwise, request will search the pool for your custom agent. If no custom agent is found, a new agent will be created and added to the pool. + - A `maxSockets` property can also be provided on the `pool` object to set the max number of sockets for all agents created (ex: `pool: {maxSockets: Infinity}`). + - Note that if you are sending multiple requests in a loop and creating multiple new `pool` objects, `maxSockets` will not work as intended. To work around this, either use [`request.defaults`](#requestdefaultsoptions) with your pool options or create the pool object with the `maxSockets` property outside of the loop. -* `timeout` - Integer containing the number of milliseconds to wait for a +- `timeout` - Integer containing the number of milliseconds to wait for a request to respond before aborting the request. Note that if the underlying TCP connection cannot be established, the OS-wide TCP connection timeout will overrule the `timeout` option ([the default in Linux is around 20 seconds](http://www.sekuda.com/overriding_the_default_linux_kernel_20_second_tcp_socket_connect_timeout)). -* `proxy` - An HTTP proxy to be used. Supports proxy Auth with Basic Auth, identical to support for the `url` parameter (by embedding the auth info in the `uri`) -* `oauth` - Options for OAuth HMAC-SHA1 signing. See documentation above. -* `hawk` - Options for [Hawk signing](https://github.com/hueniverse/hawk). The `credentials` key must contain the necessary signing info, [see hawk docs for details](https://github.com/hueniverse/hawk#usage-example). -* `strictSSL` - If `true`, requires SSL certificates be valid. **Note:** to use your own certificate authority, you need to specify an agent that was created with that CA as an option. -* `agentOptions` - Object containing user agent options. See documentation above. **Note:** [see tls API doc for TLS/SSL options](http://nodejs.org/api/tls.html#tls_tls_connect_options_callback). -* `time` - If `true`, the request-response cycle (including all redirects) is timed at millisecond resolution, and the result provided on the response's `elapsedTime` property. -* `jar` - If `true` and `tough-cookie` is installed, remember cookies for future use (or define your custom cookie jar; see examples section) -* `aws` - `object` containing AWS signing information. Should have the properties `key`, `secret`. Also requires the property `bucket`, unless you’re specifying your `bucket` as part of the path, or the request doesn’t use a bucket (i.e. GET Services) -* `httpSignature` - Options for the [HTTP Signature Scheme](https://github.com/joyent/node-http-signature/blob/master/http_signing.md) using [Joyent's library](https://github.com/joyent/node-http-signature). The `keyId` and `key` properties must be specified. See the docs for other options. -* `localAddress` - Local interface to bind for network connections. -* `gzip` - If `true`, add an `Accept-Encoding` header to request compressed content encodings from the server (if not already present) and decode supported content encodings in the response. **Note:** Automatic decoding of the response content is performed on the body data returned through `request` (both through the `request` stream and passed to the callback function) but is not performed on the `response` stream (available from the `response` event) which is the unmodified `http.IncomingMessage` object which may contain compressed data. See example below. -* `tunnel` - controls the behavior of +- `localAddress` - Local interface to bind for network connections. +- `proxy` - An HTTP proxy to be used. Supports proxy Auth with Basic Auth, identical to support for the `url` parameter (by embedding the auth info in the `uri`) +- `strictSSL` - If `true`, requires SSL certificates be valid. **Note:** to use your own certificate authority, you need to specify an agent that was created with that CA as an option. +- `agentOptions` - Object containing user agent options. See documentation above. **Note:** [see tls API doc for TLS/SSL options](http://nodejs.org/api/tls.html#tls_tls_connect_options_callback). +- `tunnel` - controls the behavior of [HTTP `CONNECT` tunneling](https://en.wikipedia.org/wiki/HTTP_tunnel#HTTP_CONNECT_tunneling) as follows: - * `undefined` (default) - `true` if the destination is `https` or a previous + - `undefined` (default) - `true` if the destination is `https` or a previous request in the redirect chain used a tunneling proxy, `false` otherwise - * `true` - always tunnel to the destination by making a `CONNECT` request to + - `true` - always tunnel to the destination by making a `CONNECT` request to the proxy - * `false` - request the destination as a `GET` request. -* `proxyHeaderWhiteList` - A whitelist of headers to send to a + - `false` - request the destination as a `GET` request. +- `proxyHeaderWhiteList` - A whitelist of headers to send to a tunneling proxy. -* `proxyHeaderExclusiveList` - A whitelist of headers to send +- `proxyHeaderExclusiveList` - A whitelist of headers to send exclusively to a tunneling proxy and not to destination. -* `removeRefererHeader` - removes the referer header when a redirect happens (default: `false`). +- `removeRefererHeader` - removes the referer header when a redirect happens (default: `false`). + +--- + +- `time` - If `true`, the request-response cycle (including all redirects) is timed at millisecond resolution, and the result provided on the response's `elapsedTime` property. The callback argument gets 3 arguments: @@ -637,10 +733,17 @@ The callback argument gets 3 arguments: 2. An [`http.IncomingMessage`](http://nodejs.org/api/http.html#http_http_incomingmessage) object 3. The third is the `response` body (`String` or `Buffer`, or JSON object if the `json` option is supplied) +[back to top](#table-of-contents) + + +--- + + ## Convenience methods There are also shorthand methods for different HTTP METHODs and some other conveniences. + ### request.defaults(options) This method **returns a wrapper** around the normal request API that defaults @@ -653,7 +756,7 @@ instead, it **returns a wrapper** that has your default settings applied to it. `request.defaults` to add/override defaults that were previously defaulted. For example: -```javascript +```js //requests using baseRequest() will set the 'x-token' header var baseRequest = request.defaults({ headers: {x-token: 'my-token'} @@ -670,7 +773,7 @@ var specialRequest = baseRequest.defaults({ Same as `request()`, but defaults to `method: "PUT"`. -```javascript +```js request.put(url) ``` @@ -678,7 +781,7 @@ request.put(url) Same as `request()`, but defaults to `method: "PATCH"`. -```javascript +```js request.patch(url) ``` @@ -686,7 +789,7 @@ request.patch(url) Same as `request()`, but defaults to `method: "POST"`. -```javascript +```js request.post(url) ``` @@ -694,7 +797,7 @@ request.post(url) Same as `request()`, but defaults to `method: "HEAD"`. -```javascript +```js request.head(url) ``` @@ -702,7 +805,7 @@ request.head(url) Same as `request()`, but defaults to `method: "DELETE"`. -```javascript +```js request.del(url) ``` @@ -710,28 +813,52 @@ request.del(url) Same as `request()` (for uniformity). -```javascript +```js request.get(url) ``` ### request.cookie Function that creates a new cookie. -```javascript +```js request.cookie('key1=value1') ``` ### request.jar() Function that creates a new cookie jar. -```javascript +```js request.jar() ``` +[back to top](#table-of-contents) + + +--- + + +## Debugging + +There are at least three ways to debug the operation of `request`: + +1. Launch the node process like `NODE_DEBUG=request node script.js` + (`lib,request,otherlib` works too). + +2. Set `require('request').debug = true` at any time (this does the same thing + as #1). + +3. Use the [request-debug module](https://github.com/nylen/request-debug) to + view request and response headers and bodies. + +[back to top](#table-of-contents) + + +--- + ## Examples: -```javascript +```js var request = require('request') , rand = Math.floor(Math.random()*100000000).toString() ; @@ -762,7 +889,7 @@ that the body data passed through `request` is automatically decompressed while the response object is unmodified and will contain compressed data if the server sent a compressed response. -```javascript +```js var request = require('request') request( { method: 'GET' @@ -789,7 +916,7 @@ the server sent a compressed response. Cookies are disabled by default (else, they would be used in subsequent requests). To enable cookies, set `jar` to `true` (either in `defaults` or `options`) and install `tough-cookie`. -```javascript +```js var request = request.defaults({jar: true}) request('http://www.google.com', function () { request('http://images.google.com') @@ -798,7 +925,7 @@ request('http://www.google.com', function () { To use a custom cookie jar (instead of `request`’s global cookie jar), set `jar` to an instance of `request.jar()` (either in `defaults` or `options`) -```javascript +```js var j = request.jar() var request = request.defaults({jar:j}) request('http://www.google.com', function () { @@ -808,7 +935,7 @@ request('http://www.google.com', function () { OR -```javascript +```js var j = request.jar(); var cookie = request.cookie('key1=value1'); var url = 'http://www.google.com'; @@ -823,7 +950,7 @@ To use a custom cookie store (such as a which supports saving to and restoring from JSON files), pass it as a parameter to `request.jar()`: -```javascript +```js var FileCookieStore = require('tough-cookie-filestore'); // NOTE - currently the 'cookies.json' file must already exist! var j = request.jar(new FileCookieStore('cookies.json')); @@ -841,7 +968,7 @@ for details. To inspect your cookie jar after a request: -```javascript +```js var j = request.jar() request({url: 'http://www.google.com', jar: j}, function () { var cookie_string = j.getCookieString(uri); // "key1=value1; key2=value2; ..." @@ -850,15 +977,4 @@ request({url: 'http://www.google.com', jar: j}, function () { }) ``` -## Debugging - -There are at least three ways to debug the operation of `request`: - -1. Launch the node process like `NODE_DEBUG=request node script.js` - (`lib,request,otherlib` works too). - -2. Set `require('request').debug = true` at any time (this does the same thing - as #1). - -3. Use the [request-debug module](https://github.com/nylen/request-debug) to - view request and response headers and bodies. +[back to top](#table-of-contents)