Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

url: expose urlToOptions utility #35960

Closed
wants to merge 1 commit into from
Closed

url: expose urlToOptions utility #35960

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
46 changes: 46 additions & 0 deletions doc/api/url.md
View file
Open in desktop
Expand Up @@ -1029,6 +1029,50 @@ new URL('/some/path%.c', 'file:'); // Incorrect: file:///some/path%.c
pathToFileURL('/some/path%.c'); // Correct: file:///some/path%25.c (POSIX)
```

### `url.urlToHttpOptions(url)`
<!-- YAML
added: REPLACEME
-->

* `url` {URL} The [WHATWG URL][] object to convert to an options object.
* Returns: {Object} Options object
jasnell marked this conversation as resolved.
Show resolved Hide resolved
* `protocol` {string} Protocol to use.
* `hostname` {string} A domain name or IP address of the server to issue the
request to.
* `hash` {string} The fragment portion of the URL.
* `search` {string} The serialized query portion of the URL.
* `pathname` {string} The path portion of the URL.
* `path` {string} Request path. Should include query string if any.
E.G. `'/index.html?page=12'`. An exception is thrown when the request path
contains illegal characters. Currently, only spaces are rejected but that
may change in the future.
* `href` {string} The serialized URL.
* `port` {number} Port of remote server.
* `auth` {string} Basic authentication i.e. `'user:password'` to compute an
Authorization header.

This utility function converts a URL object into an ordinary options object as
expected by the [`http.request()`][] and [`https.request()`][] APIs.

```js
const { urlToHttpOptions } = require('url');
const myURL = new URL('https://a:b@測試?abc#foo');
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for intruding here - but just out of curiosity: what's urlToHttpOptions doing that you wouldn't get out of a URL already, exactly? Looks almost 1:1 to me.

> new URL('https://a:b@測試?abc#foo')
URL {
  href: 'https://a:b@xn--g6w251d/?abc#foo',
  origin: 'https://xn--g6w251d',
  protocol: 'https:',
  username: 'a',
  password: 'b',
  host: 'xn--g6w251d',
  hostname: 'xn--g6w251d',
  port: '',
  pathname: '/',
  search: '?abc',
  searchParams: URLSearchParams { 'abc' => '' },
  hash: '#foo'
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

URL does not have an auth option, port is a string (it should be a number or undefined), it doesn't have a path property. These options are used by the http module.

Copy link

@nfantone nfantone Mar 18, 2021 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the clarification 👍🏼. Exactly - that's why I was saying "almost 1:1". I mean,

function urlToHttpOptions(url) {
  return {
    ...url,
    auth: `${url.username}:${url.password}`.
    port: url.port === '' ? undefined : Number(url.port),
    path: `${url.pathname ?? ''}${url.search ?? ''}`
  }
}

Not 100%, but that's pretty much it, right?

EDIT
...ok, just checked the implementation myself and I think I can answer my own question.


console.log(urlToHttpOptions(myUrl));
/**
{
protocol: 'https:',
hostname: 'xn--g6w251d',
hash: '#foo',
search: '?abc',
pathname: '/',
path: '/?abc',
href: 'https://a:b@xn--g6w251d/?abc#foo',
auth: 'a:b'
}
*/
```

## Legacy URL API
<!-- YAML
deprecated: v11.0.0
Expand Down Expand Up @@ -1388,6 +1432,8 @@ console.log(myURL.origin);
[`TypeError`]: errors.md#errors_class_typeerror
[`URLSearchParams`]: #url_class_urlsearchparams
[`array.toString()`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/toString
[`http.request()`]: http.md#http_http_request_options_callback
[`https.request()`]: https.md#https_https_request_options_callback
[`new URL()`]: #url_new_url_input_base
[`querystring`]: querystring.md
[`require('url').format()`]: #url_url_format_url_options
Expand Down
6 changes: 3 additions & 3 deletions lib/_http_client.js
View file
Open in desktop
Expand Up @@ -57,7 +57,7 @@ const { OutgoingMessage } = require('_http_outgoing');
const Agent = require('_http_agent');
const { Buffer } = require('buffer');
const { defaultTriggerAsyncIdScope } = require('internal/async_hooks');
const { URL, urlToOptions, searchParamsSymbol } = require('internal/url');
const { URL, urlToHttpOptions, searchParamsSymbol } = require('internal/url');
const { kOutHeaders, kNeedDrain } = require('internal/http');
const { connResetException, codes } = require('internal/errors');
const {
Expand Down Expand Up @@ -104,7 +104,7 @@ function ClientRequest(input, options, cb) {
if (typeof input === 'string') {
const urlStr = input;
try {
input = urlToOptions(new URL(urlStr));
input = urlToHttpOptions(new URL(urlStr));
} catch (err) {
input = url.parse(urlStr);
if (!input.hostname) {
Expand All @@ -121,7 +121,7 @@ function ClientRequest(input, options, cb) {
} else if (input && input[searchParamsSymbol] &&
input[searchParamsSymbol][searchParamsSymbol]) {
// url.URL instance
input = urlToOptions(input);
input = urlToHttpOptions(input);
} else {
cb = options;
options = input;
Expand Down
6 changes: 3 additions & 3 deletions lib/https.js
View file
Open in desktop
Expand Up @@ -48,7 +48,7 @@ const { ClientRequest } = require('_http_client');
let debug = require('internal/util/debuglog').debuglog('https', (fn) => {
debug = fn;
});
const { URL, urlToOptions, searchParamsSymbol } = require('internal/url');
const { URL, urlToHttpOptions, searchParamsSymbol } = require('internal/url');
const { IncomingMessage, ServerResponse } = require('http');
const { kIncomingMessage } = require('_http_common');

Expand Down Expand Up @@ -303,7 +303,7 @@ function request(...args) {
if (typeof args[0] === 'string') {
const urlStr = ArrayPrototypeShift(args);
try {
options = urlToOptions(new URL(urlStr));
options = urlToHttpOptions(new URL(urlStr));
} catch (err) {
options = url.parse(urlStr);
if (!options.hostname) {
Expand All @@ -320,7 +320,7 @@ function request(...args) {
} else if (args[0] && args[0][searchParamsSymbol] &&
args[0][searchParamsSymbol][searchParamsSymbol]) {
// url.URL instance
options = urlToOptions(ArrayPrototypeShift(args));
options = urlToHttpOptions(ArrayPrototypeShift(args));
}

if (args[0] && typeof args[0] !== 'function') {
Expand Down
4 changes: 2 additions & 2 deletions lib/internal/url.js
View file
Open in desktop
Expand Up @@ -1295,7 +1295,7 @@ function domainToUnicode(domain) {
// Utility function that converts a URL object into an ordinary
// options object as expected by the http.request and https.request
// APIs.
function urlToOptions(url) {
function urlToHttpOptions(url) {
const options = {
protocol: url.protocol,
hostname: typeof url.hostname === 'string' &&
Expand Down Expand Up @@ -1494,7 +1494,7 @@ module.exports = {
URLSearchParams,
domainToASCII,
domainToUnicode,
urlToOptions,
urlToHttpOptions,
formatSymbol: kFormat,
searchParamsSymbol: searchParams,
encodeStr
Expand Down
6 changes: 4 additions & 2 deletions lib/url.js
View file
Open in desktop
Expand Up @@ -47,9 +47,10 @@ const {
URLSearchParams,
domainToASCII,
domainToUnicode,
fileURLToPath,
formatSymbol,
pathToFileURL,
fileURLToPath
urlToHttpOptions,
} = require('internal/url');

// Original url.parse() API
Expand Down Expand Up @@ -987,5 +988,6 @@ module.exports = {

// Utilities
pathToFileURL,
fileURLToPath
fileURLToPath,
urlToHttpOptions,
};
37 changes: 37 additions & 0 deletions test/parallel/test-url-urltooptions.js
View file
Open in desktop
@@ -0,0 +1,37 @@
'use strict';
require('../common');
const assert = require('assert');
const { urlToHttpOptions } = require('url');

// Test urlToHttpOptions
const urlObj = new URL('http://user:pass@foo.bar.com:21/aaa/zzz?l=24#test');
const opts = urlToHttpOptions(urlObj);
assert.strictEqual(opts instanceof URL, false);
assert.strictEqual(opts.protocol, 'http:');
assert.strictEqual(opts.auth, 'user:pass');
assert.strictEqual(opts.hostname, 'foo.bar.com');
assert.strictEqual(opts.port, 21);
assert.strictEqual(opts.path, '/aaa/zzz?l=24');
assert.strictEqual(opts.pathname, '/aaa/zzz');
assert.strictEqual(opts.search, '?l=24');
assert.strictEqual(opts.hash, '#test');

const { hostname } = urlToHttpOptions(new URL('http://[::1]:21'));
assert.strictEqual(hostname, '::1');

// If a WHATWG URL object is copied, it is possible that the resulting copy
// contains the Symbols that Node uses for brand checking, but not the data
// properties, which are getters. Verify that urlToHttpOptions() can handle
// such a case.
const copiedUrlObj = { ...urlObj };
const copiedOpts = urlToHttpOptions(copiedUrlObj);
assert.strictEqual(copiedOpts instanceof URL, false);
assert.strictEqual(copiedOpts.protocol, undefined);
assert.strictEqual(copiedOpts.auth, undefined);
assert.strictEqual(copiedOpts.hostname, undefined);
assert.strictEqual(copiedOpts.port, NaN);
assert.strictEqual(copiedOpts.path, '');
assert.strictEqual(copiedOpts.pathname, undefined);
assert.strictEqual(copiedOpts.search, undefined);
assert.strictEqual(copiedOpts.hash, undefined);
assert.strictEqual(copiedOpts.href, undefined);
36 changes: 0 additions & 36 deletions test/parallel/test-whatwg-url-custom-properties.js
View file
Open in desktop
Expand Up @@ -6,7 +6,6 @@
require('../common');
const URL = require('url').URL;
const assert = require('assert');
const urlToOptions = require('internal/url').urlToOptions;

const url = new URL('http://user:pass@foo.bar.com:21/aaa/zzz?l=24#test');
const oldParams = url.searchParams; // For test of [SameObject]
Expand Down Expand Up @@ -131,41 +130,6 @@ assert.strictEqual(url.toString(),
assert.strictEqual((delete url.searchParams), true);
assert.strictEqual(url.searchParams, oldParams);

// Test urlToOptions
{
const urlObj = new URL('http://user:pass@foo.bar.com:21/aaa/zzz?l=24#test');
const opts = urlToOptions(urlObj);
assert.strictEqual(opts instanceof URL, false);
assert.strictEqual(opts.protocol, 'http:');
assert.strictEqual(opts.auth, 'user:pass');
assert.strictEqual(opts.hostname, 'foo.bar.com');
assert.strictEqual(opts.port, 21);
assert.strictEqual(opts.path, '/aaa/zzz?l=24');
assert.strictEqual(opts.pathname, '/aaa/zzz');
assert.strictEqual(opts.search, '?l=24');
assert.strictEqual(opts.hash, '#test');

const { hostname } = urlToOptions(new URL('http://[::1]:21'));
assert.strictEqual(hostname, '::1');

// If a WHATWG URL object is copied, it is possible that the resulting copy
// contains the Symbols that Node uses for brand checking, but not the data
// properties, which are getters. Verify that urlToOptions() can handle such
// a case.
const copiedUrlObj = { ...urlObj };
const copiedOpts = urlToOptions(copiedUrlObj);
assert.strictEqual(copiedOpts instanceof URL, false);
assert.strictEqual(copiedOpts.protocol, undefined);
assert.strictEqual(copiedOpts.auth, undefined);
assert.strictEqual(copiedOpts.hostname, undefined);
assert.strictEqual(copiedOpts.port, NaN);
assert.strictEqual(copiedOpts.path, '');
assert.strictEqual(copiedOpts.pathname, undefined);
assert.strictEqual(copiedOpts.search, undefined);
assert.strictEqual(copiedOpts.hash, undefined);
assert.strictEqual(copiedOpts.href, undefined);
}

// Test special origins
[
{ expected: 'https://whatwg.org',
Expand Down