Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

deps: update undici to 5.10.0 #44319

Merged
merged 2 commits into from Aug 24, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion deps/undici/src/docs/api/Client.md
Expand Up @@ -18,7 +18,7 @@ Returns: `Client`
### Parameter: `ClientOptions`

* **bodyTimeout** `number | null` (optional) - Default: `30e3` - The timeout after which a request will time out, in milliseconds. Monitors time between receiving body data. Use `0` to disable it entirely. Defaults to 30 seconds.
* **headersTimeout** `number | null` (optional) - Default: `30e3` - The amount of time the parser will wait to receive the complete HTTP headers. Defaults to 30 seconds.
* **headersTimeout** `number | null` (optional) - Default: `30e3` - The amount of time the parser will wait to receive the complete HTTP headers while not sending the request. Defaults to 30 seconds.
* **keepAliveMaxTimeout** `number | null` (optional) - Default: `600e3` - The maximum allowed `keepAliveTimeout` when overridden by *keep-alive* hints from the server. Defaults to 10 minutes.
* **keepAliveTimeout** `number | null` (optional) - Default: `4e3` - The timeout after which a socket without active requests will time out. Monitors time between activity on a connected socket. This value may be overridden by *keep-alive* hints from the server. See [MDN: HTTP - Headers - Keep-Alive directives](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Keep-Alive#directives) for more details. Defaults to 4 seconds.
* **keepAliveTimeoutThreshold** `number | null` (optional) - Default: `1e3` - A number subtracted from server *keep-alive* hints when overriding `keepAliveTimeout` to account for timing inaccuracies caused by e.g. transport latency. Defaults to 1 second.
Expand Down
2 changes: 1 addition & 1 deletion deps/undici/src/docs/api/Dispatcher.md
Expand Up @@ -199,7 +199,7 @@ Returns: `Boolean` - `false` if dispatcher is busy and further dispatch calls wo
* **blocking** `boolean` (optional) - Default: `false` - Whether the response is expected to take a long time and would end up blocking the pipeline. When this is set to `true` further pipelining will be avoided on the same connection until headers have been received.
* **upgrade** `string | null` (optional) - Default: `null` - Upgrade the request. Should be used to specify the kind of upgrade i.e. `'Websocket'`.
* **bodyTimeout** `number | null` (optional) - The timeout after which a request will time out, in milliseconds. Monitors time between receiving body data. Use `0` to disable it entirely. Defaults to 30 seconds.
* **headersTimeout** `number | null` (optional) - The amount of time the parser will wait to receive the complete HTTP headers. Defaults to 30 seconds.
* **headersTimeout** `number | null` (optional) - The amount of time the parser will wait to receive the complete HTTP headers while not sending the request. Defaults to 30 seconds.
* **throwOnError** `boolean` (optional) - Default: `false` - Whether Undici should throw an error upon receiving a 4xx or 5xx response from the server.

#### Parameter: `DispatchHandler`
Expand Down
2 changes: 1 addition & 1 deletion deps/undici/src/lib/api/readable.js
Expand Up @@ -93,7 +93,7 @@ module.exports = class BodyReadable extends Readable {
}

push (chunk) {
if (this[kConsume] && chunk !== null) {
if (this[kConsume] && chunk !== null && this.readableLength === 0) {
consumePush(this[kConsume], chunk)
return this[kReading] ? super.push(chunk) : true
}
Expand Down
17 changes: 15 additions & 2 deletions deps/undici/src/lib/client.js
Expand Up @@ -889,8 +889,10 @@ function onParserTimeout (parser) {

/* istanbul ignore else */
if (timeoutType === TIMEOUT_HEADERS) {
assert(!parser.paused, 'cannot be paused while waiting for headers')
util.destroy(socket, new HeadersTimeoutError())
if (!socket[kWriting] || socket.writableNeedDrain || client[kRunning] > 1) {
assert(!parser.paused, 'cannot be paused while waiting for headers')
util.destroy(socket, new HeadersTimeoutError())
}
} else if (timeoutType === TIMEOUT_BODY) {
if (!parser.paused) {
util.destroy(socket, new BodyTimeoutError())
Expand Down Expand Up @@ -1641,7 +1643,18 @@ class AsyncWriter {
this.bytesWritten += len

const ret = socket.write(chunk)

request.onBodySent(chunk)

if (!ret) {
if (socket[kParser].timeout && socket[kParser].timeoutType === TIMEOUT_HEADERS) {
// istanbul ignore else: only for jest
if (socket[kParser].timeout.refresh) {
socket[kParser].timeout.refresh()
}
}
}

return ret
}

Expand Down
6 changes: 5 additions & 1 deletion deps/undici/src/lib/core/util.js
Expand Up @@ -244,7 +244,11 @@ function parseHeaders (headers, obj = {}) {
const key = headers[i].toString().toLowerCase()
let val = obj[key]
if (!val) {
obj[key] = headers[i + 1].toString()
if (Array.isArray(headers[i + 1])) {
obj[key] = headers[i + 1]
} else {
obj[key] = headers[i + 1].toString()
}
} else {
if (!Array.isArray(val)) {
val = [val]
Expand Down
14 changes: 7 additions & 7 deletions deps/undici/src/lib/fetch/body.js
Expand Up @@ -57,16 +57,16 @@ function extractBody (object, keepalive = false) {

// Set Content-Type to `application/x-www-form-urlencoded;charset=UTF-8`.
contentType = 'application/x-www-form-urlencoded;charset=UTF-8'
} else if (isArrayBuffer(object) || ArrayBuffer.isView(object)) {
// BufferSource
} else if (isArrayBuffer(object)) {
// BufferSource/ArrayBuffer

if (object instanceof DataView) {
// TODO: Blob doesn't seem to work with DataView?
object = object.buffer
}
// Set source to a copy of the bytes held by object.
source = new Uint8Array(object.slice())
} else if (ArrayBuffer.isView(object)) {
// BufferSource/ArrayBufferView

// Set source to a copy of the bytes held by object.
source = new Uint8Array(object)
source = new Uint8Array(object.buffer.slice(object.byteOffset, object.byteOffset + object.byteLength))
} else if (util.isFormDataLike(object)) {
const boundary = '----formdata-undici-' + Math.random()
const prefix = `--${boundary}\r\nContent-Disposition: form-data`
Expand Down
4 changes: 3 additions & 1 deletion deps/undici/src/lib/fetch/file.js
Expand Up @@ -278,7 +278,9 @@ function processBlobParts (parts, options) {
if (!element.buffer) { // ArrayBuffer
bytes.push(new Uint8Array(element))
} else {
bytes.push(element.buffer)
bytes.push(
new Uint8Array(element.buffer, element.byteOffset, element.byteLength)
)
}
} else if (isBlobLike(element)) {
// 3. If element is a Blob, append the bytes it represents
Expand Down
19 changes: 6 additions & 13 deletions deps/undici/src/lib/fetch/index.js
Expand Up @@ -13,7 +13,7 @@ const { Headers } = require('./headers')
const { Request, makeRequest } = require('./request')
const zlib = require('zlib')
const {
matchRequestIntegrity,
bytesMatch,
makePolicyContainer,
clonePolicyContainer,
requestBadPort,
Expand All @@ -34,7 +34,8 @@ const {
sameOrigin,
isCancelled,
isAborted,
isErrorLike
isErrorLike,
fullyReadBody
} = require('./util')
const { kState, kHeaders, kGuard, kRealm } = require('./symbols')
const assert = require('assert')
Expand Down Expand Up @@ -724,7 +725,7 @@ async function mainFetch (fetchParams, recursive = false) {
const processBody = (bytes) => {
// 1. If bytes do not match request’s integrity metadata,
// then run processBodyError and abort these steps. [SRI]
if (!matchRequestIntegrity(request, bytes)) {
if (!bytesMatch(bytes, request.integrity)) {
processBodyError('integrity mismatch')
return
}
Expand All @@ -738,11 +739,7 @@ async function mainFetch (fetchParams, recursive = false) {
}

// 4. Fully read response’s body given processBody and processBodyError.
try {
processBody(await response.arrayBuffer())
} catch (err) {
processBodyError(err)
}
await fullyReadBody(response.body, processBody, processBodyError)
} else {
// 21. Otherwise, run fetch finale given fetchParams and response.
fetchFinale(fetchParams, response)
Expand Down Expand Up @@ -974,11 +971,7 @@ async function fetchFinale (fetchParams, response) {
} else {
// 4. Otherwise, fully read response’s body given processBody, processBodyError,
// and fetchParams’s task destination.
try {
processBody(await response.body.stream.arrayBuffer())
} catch (err) {
processBodyError(err)
}
await fullyReadBody(response.body, processBody, processBodyError)
}
}
}
Expand Down
6 changes: 5 additions & 1 deletion deps/undici/src/lib/fetch/request.js
Expand Up @@ -4,6 +4,7 @@

const { extractBody, mixinBody, cloneBody } = require('./body')
const { Headers, fill: fillHeaders, HeadersList } = require('./headers')
const { FinalizationRegistry } = require('../compat/dispatcher-weakref')()
const util = require('../core/util')
const {
isValidHTTPToken,
Expand Down Expand Up @@ -914,7 +915,10 @@ webidl.converters.RequestInit = webidl.dictionaryConverter([
{
key: 'signal',
converter: webidl.nullableConverter(
webidl.converters.AbortSignal
(signal) => webidl.converters.AbortSignal(
signal,
{ strict: false }
)
)
},
{
Expand Down
171 changes: 168 additions & 3 deletions deps/undici/src/lib/fetch/util.js
Expand Up @@ -4,9 +4,20 @@ const { redirectStatus } = require('./constants')
const { performance } = require('perf_hooks')
const { isBlobLike, toUSVString, ReadableStreamFrom } = require('../core/util')
const assert = require('assert')
const { isUint8Array } = require('util/types')

let File

// https://nodejs.org/api/crypto.html#determining-if-crypto-support-is-unavailable
/** @type {import('crypto')|undefined} */
let crypto

try {
crypto = require('crypto')
} catch {

}

// https://fetch.spec.whatwg.org/#block-bad-port
const badPorts = [
'1', '7', '9', '11', '13', '15', '17', '19', '20', '21', '22', '23', '25', '37', '42', '43', '53', '69', '77', '79',
Expand Down Expand Up @@ -339,10 +350,116 @@ function determineRequestsReferrer (request) {
return 'no-referrer'
}

function matchRequestIntegrity (request, bytes) {
/**
* @see https://w3c.github.io/webappsec-subresource-integrity/#does-response-match-metadatalist
* @param {Uint8Array} bytes
* @param {string} metadataList
*/
function bytesMatch (bytes, metadataList) {
// If node is not built with OpenSSL support, we cannot check
// a request's integrity, so allow it by default (the spec will
// allow requests if an invalid hash is given, as precedence).
/* istanbul ignore if: only if node is built with --without-ssl */
if (crypto === undefined) {
return true
}

// 1. Let parsedMetadata be the result of parsing metadataList.
const parsedMetadata = parseMetadata(metadataList)

// 2. If parsedMetadata is no metadata, return true.
if (parsedMetadata === 'no metadata') {
return true
}

// 3. If parsedMetadata is the empty set, return true.
if (parsedMetadata.length === 0) {
return true
}

// 4. Let metadata be the result of getting the strongest
// metadata from parsedMetadata.
// Note: this will only work for SHA- algorithms and it's lazy *at best*.
const metadata = parsedMetadata.sort((c, d) => d.algo.localeCompare(c.algo))

// 5. For each item in metadata:
for (const item of metadata) {
// 1. Let algorithm be the alg component of item.
const algorithm = item.algo

// 2. Let expectedValue be the val component of item.
const expectedValue = item.hash

// 3. Let actualValue be the result of applying algorithm to bytes.
// Note: "applying algorithm to bytes" converts the result to base64
const actualValue = crypto.createHash(algorithm).update(bytes).digest('base64')

// 4. If actualValue is a case-sensitive match for expectedValue,
// return true.
if (actualValue === expectedValue) {
return true
}
}

// 6. Return false.
return false
}

// https://w3c.github.io/webappsec-subresource-integrity/#grammardef-hash-with-options
// hash-algo is defined in Content Security Policy 2 Section 4.2
// base64-value is similary defined there
// VCHAR is defined https://www.rfc-editor.org/rfc/rfc5234#appendix-B.1
const parseHashWithOptions = /((?<algo>sha256|sha384|sha512)-(?<hash>[A-z0-9+/]{1}.*={1,2}))( +[\x21-\x7e]?)?/i

/**
* @see https://w3c.github.io/webappsec-subresource-integrity/#parse-metadata
* @param {string} metadata
*/
function parseMetadata (metadata) {
// 1. Let result be the empty set.
/** @type {{ algo: string, hash: string }[]} */
const result = []

// 2. Let empty be equal to true.
let empty = true

const supportedHashes = crypto.getHashes()

// 3. For each token returned by splitting metadata on spaces:
for (const token of metadata.split(' ')) {
// 1. Set empty to false.
empty = false

// 2. Parse token as a hash-with-options.
const parsedToken = parseHashWithOptions.exec(token)

// 3. If token does not parse, continue to the next token.
if (parsedToken === null || parsedToken.groups === undefined) {
// Note: Chromium blocks the request at this point, but Firefox
// gives a warning that an invalid integrity was given. The
// correct behavior is to ignore these, and subsequently not
// check the integrity of the resource.
continue
}

// 4. Let algorithm be the hash-algo component of token.
const algorithm = parsedToken.groups.algo

// 5. If algorithm is a hash function recognized by the user
// agent, add the parsed token to result.
if (supportedHashes.includes(algorithm.toLowerCase())) {
result.push(parsedToken.groups)
}
}

// 4. Return no metadata if empty is true, otherwise return result.
if (empty === true) {
return 'no metadata'
}

return result
}

// https://w3c.github.io/webappsec-upgrade-insecure-requests/#upgrade-request
function tryUpgradeRequestToAPotentiallyTrustworthyURL (request) {
// TODO
Expand Down Expand Up @@ -438,6 +555,53 @@ function makeIterator (iterator, name) {
return Object.setPrototypeOf({}, i)
}

/**
* @see https://fetch.spec.whatwg.org/#body-fully-read
*/
async function fullyReadBody (body, processBody, processBodyError) {
// 1. If taskDestination is null, then set taskDestination to
// the result of starting a new parallel queue.

// 2. Let promise be the result of fully reading body as promise
// given body.
try {
/** @type {Uint8Array[]} */
const chunks = []
let length = 0

const reader = body.stream.getReader()

while (true) {
const { done, value } = await reader.read()

if (done === true) {
break
}

// read-loop chunk steps
assert(isUint8Array(value))

chunks.push(value)
length += value.byteLength
}

// 3. Let fulfilledSteps given a byte sequence bytes be to queue
// a fetch task to run processBody given bytes, with
// taskDestination.
const fulfilledSteps = (bytes) => queueMicrotask(() => {
processBody(bytes)
})

fulfilledSteps(Buffer.concat(chunks, length))
} catch (err) {
// 4. Let rejectedSteps be to queue a fetch task to run
// processBodyError, with taskDestination.
queueMicrotask(() => processBodyError(err))
}

// 5. React to promise with fulfilledSteps and rejectedSteps.
}

/**
* Fetch supports node >= 16.8.0, but Object.hasOwn was added in v16.9.0.
*/
Expand All @@ -451,7 +615,6 @@ module.exports = {
toUSVString,
tryUpgradeRequestToAPotentiallyTrustworthyURL,
coarsenedSharedCurrentTime,
matchRequestIntegrity,
determineRequestsReferrer,
makePolicyContainer,
clonePolicyContainer,
Expand All @@ -477,5 +640,7 @@ module.exports = {
isValidHeaderName,
isValidHeaderValue,
hasOwn,
isErrorLike
isErrorLike,
fullyReadBody,
bytesMatch
}