Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: implement websockets #1795

Merged
merged 82 commits into from Dec 16, 2022
Merged
Show file tree
Hide file tree
Changes from 79 commits
Commits
Show all changes
82 commits
Select commit Hold shift + click to select a range
bb688ac
initial handshake
KhafraDev Nov 28, 2022
e6e2686
minor fixes
KhafraDev Nov 29, 2022
9aea277
feat: working initial handshake!
KhafraDev Dec 1, 2022
f1ae69a
feat(ws): initial WebSocket class implementation
KhafraDev Dec 1, 2022
bec82fd
fix: allow http: and ws: urls
KhafraDev Dec 1, 2022
7e6725e
fix(ws): use websocket spec
KhafraDev Dec 1, 2022
5030683
fix(ws): use websocket spec
KhafraDev Dec 1, 2022
a928c58
feat: implement url getter
KhafraDev Dec 1, 2022
ecff3d7
feat: implement some of `WebSocket.close` and ready state
KhafraDev Dec 1, 2022
6ec2701
fix: body is null for websockets & pass socket to response
KhafraDev Dec 1, 2022
39c7b43
fix: store the fetch controller & response on ws
KhafraDev Dec 1, 2022
fb84cb8
fix: remove invalid tests
KhafraDev Dec 1, 2022
85a4046
feat: implement readyState getter
KhafraDev Dec 2, 2022
190cabe
feat: implement `protocol` and `extensions` getters
KhafraDev Dec 2, 2022
454a5c3
feat: implement event listeners
KhafraDev Dec 2, 2022
5ae04ee
feat: implement binaryType attribute
KhafraDev Dec 2, 2022
c9ff83a
fix: add argument length checks
KhafraDev Dec 2, 2022
484e732
feat: basic unfragmented message parsing
KhafraDev Dec 2, 2022
8619653
fix: always remove previous listener
KhafraDev Dec 3, 2022
c70bd8b
feat: add in idlharness WPT
KhafraDev Dec 3, 2022
34758e6
implement sending a message for WS and add a websocketFrame class
jodevsa Dec 4, 2022
6a6b1b3
Merge pull request #30 from jodevsa/ws
KhafraDev Dec 4, 2022
a87e53b
feat: allow sending ArrayBuffer/Views & Blob
KhafraDev Dec 4, 2022
d983115
fix: remove duplicate `upgrade` and `connection` headers
KhafraDev Dec 4, 2022
feee358
feat: add in WebSocket.close() and handle closing frames
KhafraDev Dec 4, 2022
5e61a84
refactor WebsocketFrame and support receiving frames in multiple chunks
jodevsa Dec 4, 2022
0e07468
fixes
jodevsa Dec 4, 2022
e288ea5
Merge pull request #31 from jodevsa/ws
KhafraDev Dec 5, 2022
cbad426
move WebsocketFrame to its own file
KhafraDev Dec 5, 2022
d2462a1
feat: export WebSocket & add types
KhafraDev Dec 5, 2022
1340dd4
fix: tsd
KhafraDev Dec 5, 2022
8d98382
feat(wpt): use WebSocketServer & run test
KhafraDev Dec 5, 2022
3cae2ec
fix: properly set/read close code & close reason
KhafraDev Dec 5, 2022
73c8389
fix: flakiness in websocket test runner
KhafraDev Dec 6, 2022
8b7bc44
fix: receive message with arraybuffer binary type
KhafraDev Dec 6, 2022
eb09dc6
feat: split WebsocketFrame into 2 classes (sent & received)
KhafraDev Dec 6, 2022
174c030
fix: parse fragmented frames more efficiently & close frame
KhafraDev Dec 6, 2022
7ee907b
fix: add types for MessageEvent and CloseEvent
KhafraDev Dec 6, 2022
6985bc3
fix: subprotocol validation & add wpts
KhafraDev Dec 6, 2022
eae7f76
fix: protocol validation & protocol webidl & add wpts
KhafraDev Dec 7, 2022
682d880
fix: correct bufferedAmount calc. & message event w/ blob
KhafraDev Dec 7, 2022
052a804
fix: don't truncate typedarrays
KhafraDev Dec 7, 2022
5f2fdab
feat: add remaining wpts
KhafraDev Dec 7, 2022
6caaa47
fix: allow sending payloads > 65k bytes
KhafraDev Dec 7, 2022
2d5b9ae
fix: mask data > 125 bytes properly
KhafraDev Dec 7, 2022
0880cef
revert changes to core
KhafraDev Dec 7, 2022
02805e8
fix: decrement bufferedAmount after write
KhafraDev Dec 8, 2022
58114ab
fix: handle ping and pong frames
KhafraDev Dec 8, 2022
a099471
fix: simplify receiving frame logic
KhafraDev Dec 8, 2022
346bdb8
fix: disable extensions & validate frames
KhafraDev Dec 9, 2022
5113fb2
fix: send close frame upon receiving
KhafraDev Dec 9, 2022
dee913c
lint
KhafraDev Dec 9, 2022
b3c4314
fix: validate status code & utf-8
KhafraDev Dec 9, 2022
7741277
fix: add hooks
KhafraDev Dec 9, 2022
b1339fb
fix: check if frame is unfragmented correctly
KhafraDev Dec 9, 2022
61c921c
fix: send ping app data in pong frames
KhafraDev Dec 10, 2022
c7249d3
export websocket on node >= 18 & add diagnostic_channels
KhafraDev Dec 10, 2022
aadf25a
mark test as flaky
KhafraDev Dec 10, 2022
30b1e23
fix: couple bug fixes
KhafraDev Dec 10, 2022
6a7134b
fix: fragmented frame end detection
KhafraDev Dec 10, 2022
b6411a6
fix: use TextDecoder for utf-8 validation
KhafraDev Dec 11, 2022
4fbfbf4
fix: handle incomplete chunks
KhafraDev Dec 11, 2022
e43fa5b
revert: handle incomplete chunks
KhafraDev Dec 11, 2022
553a0f2
mark WebSockets as experimental
KhafraDev Dec 11, 2022
e0289f7
fix: sending 65k bytes is still flaky on linux
KhafraDev Dec 11, 2022
dcc3801
fix: apply suggestions
KhafraDev Dec 11, 2022
797acc3
fix: apply some suggestions
KhafraDev Dec 12, 2022
dfc57ab
add basic docs
KhafraDev Dec 12, 2022
a0094a4
feat: use streaming parser for frames
KhafraDev Dec 13, 2022
883d9d9
feat: validate some frames & remove WebsocketFrame class
KhafraDev Dec 13, 2022
71848d9
fix: parse close frame & move failWebsocketConnection
KhafraDev Dec 14, 2022
b468029
fix: read close reason and read entire close body
KhafraDev Dec 14, 2022
8839538
fix: echo close frame if one hasn't been sent
KhafraDev Dec 14, 2022
78b77a8
fix: emit message event on message receive
KhafraDev Dec 14, 2022
6cfc42e
fix: minor fixes
KhafraDev Dec 14, 2022
7f1b5d4
fix: ci
KhafraDev Dec 14, 2022
24ae13b
fix: set was clean exit after server receives close frame
KhafraDev Dec 14, 2022
5086ce8
fix: check if received close frame for clean close
KhafraDev Dec 14, 2022
6441dc3
fix: set sent close after writing frame
KhafraDev Dec 14, 2022
476a192
feat: implement error messages
KhafraDev Dec 15, 2022
28c7ff6
fix: add error event handler to socket
KhafraDev Dec 15, 2022
73128fd
fix: address reviews
KhafraDev Dec 15, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
55 changes: 55 additions & 0 deletions docs/api/DiagnosticsChannel.md
Expand Up @@ -135,3 +135,58 @@ diagnosticsChannel.channel('undici:client:connectError').subscribe(({ error, soc
// connector is a function that creates the socket
console.log(`Connect failed with ${error.message}`)
})
```

## `undici:websocket:ping`

This message is published after the client receives a ping from the server.

```js
import diagnosticsChannel from 'diagnostics_channel'

diagnosticsChannel.channel('undici:websocket:ping').subscribe(({ frame }) => {
// The frame received from the server
console.log(frame)
})
```

## `undici:websocket:pong`

This message is published after the client receives a pong from the server.

```js
import diagnosticsChannel from 'diagnostics_channel'

diagnosticsChannel.channel('undici:websocket:pong').subscribe(({ frame }) => {
// The frame received from the server
console.log(frame)
})
```

## `undici:websocket:open`

This message is published after the client has successfully connected to a server.

```js
import diagnosticsChannel from 'diagnostics_channel'

diagnosticsChannel.channel('undici:websocket:open').subscribe(({ address, protocol, extensions }) => {
console.log(address) // address, family, and port
console.log(protocol) // negotiated subprotocols
console.log(extensions) // negotiated extensions
})
```

## `undici:websocket:close`

This message is published after the connection has closed.

```js
import diagnosticsChannel from 'diagnostics_channel'

diagnosticsChannel.channel('undici:websocket:close').subscribe(({ websocket, code, reason }) => {
console.log(websocket) // the WebSocket object
console.log(code) // the closing status code
console.log(reason) // the closing reason
})
```
20 changes: 20 additions & 0 deletions docs/api/WebSocket.md
@@ -0,0 +1,20 @@
# Class: WebSocket

> ⚠️ Warning: the WebSocket API is experimental and has known bugs.

Extends: [`EventTarget`](https://developer.mozilla.org/en-US/docs/Web/API/EventTarget)

The WebSocket object provides a way to manage a WebSocket connection to a server, allowing bidirectional communication. The API follows the [WebSocket spec](https://developer.mozilla.org/en-US/docs/Web/API/WebSocket).

## `new WebSocket(url[, protocol])`

Arguments:

* **url** `URL | string` - The url's protocol *must* be `ws` or `wss`.
* **protocol** `string | string[]` (optional) - Subprotocol(s) to request the server use.

## Read More

- [MDN - WebSocket](https://developer.mozilla.org/en-US/docs/Web/API/WebSocket)
- [The WebSocket Specification](https://www.rfc-editor.org/rfc/rfc6455)
- [The WHATWG WebSocket Specification](https://websockets.spec.whatwg.org/)
1 change: 1 addition & 0 deletions docsify/sidebar.md
Expand Up @@ -16,6 +16,7 @@
* [MockErrors](/docs/api/MockErrors.md "Undici API - MockErrors")
* [API Lifecycle](/docs/api/api-lifecycle.md "Undici API - Lifecycle")
* [Diagnostics Channel Support](/docs/api/DiagnosticsChannel.md "Diagnostics Channel Support")
* [WebSocket](/docs/api/WebSocket.md "Undici API - WebSocket")
* Best Practices
* [Proxy](/docs/best-practices/proxy.md "Connecting through a proxy")
* [Client Certificate](/docs/best-practices/client-certificate.md "Connect using a client certificate")
Expand Down
1 change: 1 addition & 0 deletions index.d.ts
Expand Up @@ -21,6 +21,7 @@ export * from './types/file'
export * from './types/filereader'
export * from './types/formdata'
export * from './types/diagnostics-channel'
export * from './types/websocket'
export { Interceptable } from './types/mock-interceptor'

export { Dispatcher, BalancedPool, Pool, Client, buildConnector, errors, Agent, request, stream, pipeline, connect, upgrade, setGlobalDispatcher, getGlobalDispatcher, setGlobalOrigin, getGlobalOrigin, MockClient, MockPool, MockAgent, mockErrors, ProxyAgent, RedirectHandler, DecoratorHandler }
Expand Down
6 changes: 6 additions & 0 deletions index.js
Expand Up @@ -119,6 +119,12 @@ if (nodeMajor > 16 || (nodeMajor === 16 && nodeMinor >= 8)) {
module.exports.getGlobalOrigin = getGlobalOrigin
}

if (nodeMajor >= 18) {
const { WebSocket } = require('./lib/websocket/websocket')

module.exports.WebSocket = WebSocket
}

module.exports.request = makeDispatcher(api.request)
module.exports.stream = makeDispatcher(api.stream)
module.exports.pipeline = makeDispatcher(api.pipeline)
Expand Down
46 changes: 40 additions & 6 deletions lib/fetch/index.js
Expand Up @@ -58,6 +58,7 @@ const { dataURLProcessor, serializeAMimeType } = require('./dataURL')
const { TransformStream } = require('stream/web')
const { getGlobalDispatcher } = require('../../index')
const { webidl } = require('./webidl')
const { STATUS_CODES } = require('http')

/** @type {import('buffer').resolveObjectURL} */
let resolveObjectURL
Expand Down Expand Up @@ -1743,12 +1744,17 @@ async function httpNetworkFetch (
}

try {
const { body, status, statusText, headersList } = await dispatch({ body: requestBody })
// socket is only provided for websockets
const { body, status, statusText, headersList, socket } = await dispatch({ body: requestBody })

const iterator = body[Symbol.asyncIterator]()
fetchParams.controller.next = () => iterator.next()
if (socket) {
response = makeResponse({ status, statusText, headersList, socket })
} else {
const iterator = body[Symbol.asyncIterator]()
fetchParams.controller.next = () => iterator.next()

response = makeResponse({ status, statusText, headersList })
response = makeResponse({ status, statusText, headersList })
}
} catch (err) {
// 10. If aborted, then:
if (err.name === 'AbortError') {
Expand Down Expand Up @@ -1932,7 +1938,10 @@ async function httpNetworkFetch (

async function dispatch ({ body }) {
const url = requestCurrentURL(request)
return new Promise((resolve, reject) => fetchParams.controller.dispatcher.dispatch(
/** @type {import('../..').Agent} */
const agent = fetchParams.controller.dispatcher

return new Promise((resolve, reject) => agent.dispatch(
{
path: url.pathname + url.search,
origin: url.origin,
Expand All @@ -1941,7 +1950,8 @@ async function httpNetworkFetch (
headers: [...request.headersList].flat(),
maxRedirections: 0,
bodyTimeout: 300_000,
headersTimeout: 300_000
headersTimeout: 300_000,
upgrade: request.mode === 'websocket' ? 'websocket' : undefined
},
{
body: null,
Expand Down Expand Up @@ -2060,6 +2070,30 @@ async function httpNetworkFetch (
fetchParams.controller.terminate(error)

reject(error)
},

onUpgrade (status, headersList, socket) {
if (status !== 101) {
return
}

const headers = new Headers()

for (let n = 0; n < headersList.length; n += 2) {
const key = headersList[n + 0].toString('latin1')
const val = headersList[n + 1].toString('latin1')
KhafraDev marked this conversation as resolved.
Show resolved Hide resolved

headers.append(key, val)
}

resolve({
status,
statusText: STATUS_CODES[status],
headersList: headers[kHeadersList],
socket
})

return true
}
}
))
Expand Down
4 changes: 2 additions & 2 deletions lib/fetch/webidl.js
Expand Up @@ -473,9 +473,9 @@ webidl.converters['unsigned long long'] = function (V) {
}

// https://webidl.spec.whatwg.org/#es-unsigned-short
webidl.converters['unsigned short'] = function (V) {
webidl.converters['unsigned short'] = function (V, opts) {
// 1. Let x be ? ConvertToInt(V, 16, "unsigned").
const x = webidl.util.ConvertToInt(V, 16, 'unsigned')
const x = webidl.util.ConvertToInt(V, 16, 'unsigned', opts)

// 2. Return the IDL unsigned short value that represents
// the same numeric value as x.
Expand Down