refactor: use v8 serialization for ipc (#20214)

* refactor: use v8 serialization for ipc

* cloning process.env doesn't work

* serialize host objects by enumerating key/values

* new serialization can handle NaN, Infinity, and undefined correctly

* can't allocate v8 objects during GC

* backport microtasks fix

* fix compile

* fix node_stream_loader reentrancy

* update subframe spec to expect undefined instead of null

* write undefined instead of crashing when serializing host objects

* fix webview spec

* fix download spec

* buffers are transformed into uint8arrays

* can't serialize promises

* fix chrome.i18n.getMessage

* fix devtools tests

* fix zoom test

* fix debug build

* fix lint

* update ipcRenderer tests

* fix printToPDF test

* update patch

* remove accidentally re-added remote-side spec

* wip

* don't attempt to serialize host objects

* jump through different hoops to set options.webContents sometimes

* whoops

* fix lint

* clean up error-handling logic

* fix memory leak

* fix lint

* convert host objects using old base::Value serialization

* fix lint more

* fall back to base::Value-based serialization

* remove commented-out code

* add docs to breaking-changes.md

* Update breaking-changes.md

* update ipcRenderer and WebContents docs

* lint

* use named values for format tag

* save a memcpy for ~30% speedup

* get rid of calls to ShallowClone

* extra debugging for paranoia

* d'oh, use the correct named tags

* apparently msstl doesn't like this DCHECK

* funny story about that DCHECK

* disable remote-related functions when enable_remote_module = false

* nits

* use EnableIf to disable remote methods in mojom

* fix include

* review comments

docs/api/breaking-changes.md

@@ -6,6 +6,59 @@ Breaking changes will be documented here, and deprecation warnings added to JS c
 
 The `FIXME` string is used in code comments to denote things that should be fixed for future releases. See https://github.com/electron/electron/search?q=fixme
 
+## Planned Breaking API Changes (8.0)
+
+### Values sent over IPC are now serialized with Structured Clone Algorithm
+
+The algorithm used to serialize objects sent over IPC (through
+`ipcRenderer.send`, `ipcRenderer.sendSync`, `WebContents.send` and related
+methods) has been switched from a custom algorithm to V8's built-in [Structured
+Clone Algorithm][SCA], the same algorithm used to serialize messages for
+`postMessage`. This brings about a 2x performance improvement for large
+messages, but also brings some breaking changes in behavior.
+
+- Sending Functions, Promises, WeakMaps, WeakSets, or objects containing any
+  such values, over IPC will now throw an exception, instead of silently
+  converting the functions to `undefined`.
+```js
+// Previously:
+ipcRenderer.send('channel', { value: 3, someFunction: () => {} })
+// => results in { value: 3 } arriving in the main process
+
+// From Electron 8:
+ipcRenderer.send('channel', { value: 3, someFunction: () => {} })
+// => throws Error("() => {} could not be cloned.")
+```
+- `NaN`, `Infinity` and `-Infinity` will now be correctly serialized, instead
+  of being converted to `null`.
+- Objects containing cyclic references will now be correctly serialized,
+  instead of being converted to `null`.
+- `Set`, `Map`, `Error` and `RegExp` values will be correctly serialized,
+  instead of being converted to `{}`.
+- `BigInt` values will be correctly serialized, instead of being converted to
+  `null`.
+- Sparse arrays will be serialized as such, instead of being converted to dense
+  arrays with `null`s.
+- `Date` objects will be transferred as `Date` objects, instead of being
+  converted to their ISO string representation.
+- Typed Arrays (such as `Uint8Array`, `Uint16Array`, `Uint32Array` and so on)
+  will be transferred as such, instead of being converted to Node.js `Buffer`.
+- Node.js `Buffer` objects will be transferred as `Uint8Array`s. You can
+  convert a `Uint8Array` back to a Node.js `Buffer` by wrapping the underlying
+  `ArrayBuffer`:
+```js
+Buffer.from(value.buffer, value.byteOffset, value.byteLength)
+```
+
+Sending any objects that aren't native JS types, such as DOM objects (e.g.
+`Element`, `Location`, `DOMMatrix`), Node.js objects (e.g. `process.env`,
+`Stream`), or Electron objects (e.g. `WebContents`, `BrowserWindow`,
+`WebFrame`) is deprecated. In Electron 8, these objects will be serialized as
+before with a DeprecationWarning message, but starting in Electron 9, sending
+these kinds of objects will throw a 'could not be cloned' error.
+
+[SCA]: https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Structured_clone_algorithm
+
 ## Planned Breaking API Changes (7.0)
 
 ### Node Headers URL

docs/api/ipc-renderer.md

@@ -55,9 +55,15 @@ Removes all listeners, or those of the specified `channel`.
 * `channel` String
 * `...args` any[]
 
-Send a message to the main process asynchronously via `channel`, you can also
-send arbitrary arguments. Arguments will be serialized as JSON internally and
-hence no functions or prototype chain will be included.
+Send an asynchronous message to the main process via `channel`, along with
+arguments. Arguments will be serialized with the [Structured Clone
+Algorithm][SCA], just like [`postMessage`][], so prototype chains will not be
+included. Sending Functions, Promises, Symbols, WeakMaps, or WeakSets will
+throw an exception.
+
+> **NOTE**: Sending non-standard JavaScript types such as DOM objects or
+> special Electron objects is deprecated, and will begin throwing an exception
+> starting with Electron 9.
 
 The main process handles it by listening for `channel` with the
 [`ipcMain`](ipc-main.md) module.
@@ -69,9 +75,15 @@ The main process handles it by listening for `channel` with the
 
 Returns `Promise<any>` - Resolves with the response from the main process.
 
-Send a message to the main process asynchronously via `channel` and expect an
-asynchronous result. Arguments will be serialized as JSON internally and
-hence no functions or prototype chain will be included.
+Send a message to the main process via `channel` and expect a result
+asynchronously. Arguments will be serialized with the [Structured Clone
+Algorithm][SCA], just like [`postMessage`][], so prototype chains will not be
+included. Sending Functions, Promises, Symbols, WeakMaps, or WeakSets will
+throw an exception.
+
+> **NOTE**: Sending non-standard JavaScript types such as DOM objects or
+> special Electron objects is deprecated, and will begin throwing an exception
+> starting with Electron 9.
 
 The main process should listen for `channel` with
 [`ipcMain.handle()`](ipc-main.md#ipcmainhandlechannel-listener).
@@ -97,15 +109,23 @@ ipcMain.handle('some-name', async (event, someArgument) => {
 
 Returns `any` - The value sent back by the [`ipcMain`](ipc-main.md) handler.
 
-Send a message to the main process synchronously via `channel`, you can also
-send arbitrary arguments. Arguments will be serialized in JSON internally and
-hence no functions or prototype chain will be included.
+Send a message to the main process via `channel` and expect a result
+synchronously. Arguments will be serialized with the [Structured Clone
+Algorithm][SCA], just like [`postMessage`][], so prototype chains will not be
+included. Sending Functions, Promises, Symbols, WeakMaps, or WeakSets will
+throw an exception.
+
+> **NOTE**: Sending non-standard JavaScript types such as DOM objects or
+> special Electron objects is deprecated, and will begin throwing an exception
+> starting with Electron 9.
 
 The main process handles it by listening for `channel` with [`ipcMain`](ipc-main.md) module,
 and replies by setting `event.returnValue`.
 
-**Note:** Sending a synchronous message will block the whole renderer process,
-unless you know what you are doing you should never use it.
+> :warning: **WARNING**: Sending a synchronous message will block the whole
+> renderer process until the reply is received, so use this method only as a
+> last resort. It's much better to use the asynchronous version,
+> [`invoke()`](ipc-renderer.md#ipcrendererinvokechannel-args).
 
 ### `ipcRenderer.sendTo(webContentsId, channel, ...args)`
 
@@ -129,3 +149,5 @@ The documentation for the `event` object passed to the `callback` can be found
 in the [`ipc-renderer-event`](structures/ipc-renderer-event.md) structure docs.
 
 [event-emitter]: https://nodejs.org/api/events.html#events_class_eventemitter
+[SCA]: https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Structured_clone_algorithm
+[`postMessage`]: https://developer.mozilla.org/en-US/docs/Web/API/Window/postMessage

docs/api/web-contents.md

@@ -1479,9 +1479,15 @@ Opens the developer tools for the service worker context.
 * `channel` String
 * `...args` any[]
 
-Send an asynchronous message to renderer process via `channel`, you can also
-send arbitrary arguments. Arguments will be serialized in JSON internally and
-hence no functions or prototype chain will be included.
+Send an asynchronous message to the renderer process via `channel`, along with
+arguments. Arguments will be serialized with the [Structured Clone
+Algorithm][SCA], just like [`postMessage`][], so prototype chains will not be
+included. Sending Functions, Promises, Symbols, WeakMaps, or WeakSets will
+throw an exception.
+
+> **NOTE**: Sending non-standard JavaScript types such as DOM objects or
+> special Electron objects is deprecated, and will begin throwing an exception
+> starting with Electron 9.
 
 The renderer process can handle the message by listening to `channel` with the
 [`ipcRenderer`](ipc-renderer.md) module.
@@ -1522,8 +1528,14 @@ app.on('ready', () => {
 * `...args` any[]
 
 Send an asynchronous message to a specific frame in a renderer process via
-`channel`. Arguments will be serialized
-as JSON internally and as such no functions or prototype chains will be included.
+`channel`, along with arguments. Arguments will be serialized with the
+[Structured Clone Algorithm][SCA], just like [`postMessage`][], so prototype
+chains will not be included. Sending Functions, Promises, Symbols, WeakMaps, or
+WeakSets will throw an exception.
+
+> **NOTE**: Sending non-standard JavaScript types such as DOM objects or
+> special Electron objects is deprecated, and will begin throwing an exception
+> starting with Electron 9.
 
 The renderer process can handle the message by listening to `channel` with the
 [`ipcRenderer`](ipc-renderer.md) module.
@@ -1785,3 +1797,5 @@ A [`Debugger`](debugger.md) instance for this webContents.
 
 [keyboardevent]: https://developer.mozilla.org/en-US/docs/Web/API/KeyboardEvent
 [event-emitter]: https://nodejs.org/api/events.html#events_class_eventemitter
+[SCA]: https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Structured_clone_algorithm
+[`postMessage`]: https://developer.mozilla.org/en-US/docs/Web/API/Window/postMessage

filenames.gni

@@ -483,6 +483,7 @@ filenames = {
     "shell/common/gin_converters/net_converter.cc",
     "shell/common/gin_converters/net_converter.h",
     "shell/common/gin_converters/std_converter.h",
+    "shell/common/gin_converters/blink_converter_gin_adapter.h",
     "shell/common/gin_converters/value_converter_gin_adapter.h",
     "shell/common/gin_helper/callback.cc",
     "shell/common/gin_helper/callback.h",

lib/browser/chrome-extension.js

@@ -240,7 +240,7 @@ const getMessagesPath = (extensionId) => {
 
 ipcMainUtils.handleSync('CHROME_GET_MESSAGES', async function (event, extensionId) {
   const messagesPath = getMessagesPath(extensionId)
-  return fs.promises.readFile(messagesPath)
+  return fs.promises.readFile(messagesPath, 'utf8')
 })
 
 const validStorageTypes = new Set(['sync', 'local'])

lib/browser/guest-view-manager.js

@@ -23,7 +23,6 @@ const supportedWebViewEvents = [
   'devtools-opened',
   'devtools-closed',
   'devtools-focused',
-  'new-window',
   'will-navigate',
   'did-start-navigation',
   'did-navigate',
@@ -48,6 +47,13 @@ const supportedWebViewEvents = [
 const guestInstances = {}
 const embedderElementsMap = {}
 
+function sanitizeOptionsForGuest (options) {
+  const ret = { ...options }
+  // WebContents values can't be sent over IPC.
+  delete ret.webContents
+  return ret
+}
+
 // Create a new guest instance.
 const createGuest = function (embedder, params) {
   if (webViewManager == null) {
@@ -114,6 +120,12 @@ const createGuest = function (embedder, params) {
     fn(event)
   }
 
+  guest.on('new-window', function (event, url, frameName, disposition, options, additionalFeatures, referrer) {
+    sendToEmbedder('ELECTRON_GUEST_VIEW_INTERNAL_DISPATCH_EVENT', 'new-window', url,
+      frameName, disposition, sanitizeOptionsForGuest(options),
+      additionalFeatures, referrer)
+  })
+
   // Dispatch guest's IPC messages to embedder.
   guest.on('ipc-message-host', function (_, channel, args) {
     sendToEmbedder('ELECTRON_GUEST_VIEW_INTERNAL_IPC_MESSAGE', channel, ...args)

lib/browser/guest-window-manager.js

@@ -250,8 +250,7 @@ ipcMainInternal.on('ELECTRON_GUEST_WINDOW_MANAGER_WINDOW_OPEN', (event, url, fra
 
 // Routed window.open messages with fully parsed options
 ipcMainInternal.on('ELECTRON_GUEST_WINDOW_MANAGER_INTERNAL_WINDOW_OPEN', function (event, url, referrer,
-  frameName, disposition, options,
-  additionalFeatures, postData) {
+  frameName, disposition, options, additionalFeatures, postData) {
   options = mergeBrowserWindowOptions(event.sender, options)
   event.sender.emit('new-window', event, url, frameName, disposition, options, additionalFeatures, referrer)
   const { newGuest } = event

lib/browser/rpc-server.js

@@ -121,7 +121,7 @@ ipcMainUtils.handleSync('ELECTRON_BROWSER_SANDBOX_LOAD', async function (event)
     process: {
       arch: process.arch,
       platform: process.platform,
-      env: process.env,
+      env: { ...process.env },
       version: process.version,
       versions: process.versions,
       execPath: process.helperExecPath

patches/chromium/.patches

@@ -78,4 +78,5 @@ expose_setuseragent_on_networkcontext.patch
 feat_add_set_theme_source_to_allow_apps_to.patch
 revert_cleanup_remove_menu_subtitles_sublabels.patch
 ui_views_fix_jumbo_build.patch
+export_fetchapi_mojo_traits_to_fix_component_build.patch
 fix_windows_build.patch

patches/chromium/export_fetchapi_mojo_traits_to_fix_component_build.patch

@@ -0,0 +1,36 @@
+From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
+From: Jeremy Apthorp <jeremya@chromium.org>
+Date: Fri, 20 Sep 2019 16:44:18 -0400
+Subject: export FetchAPI mojo traits to fix component build
+
+Without these, we get link errors in the component build when using the
+blink::CloneableMessage mojo traits.
+
+diff --git a/third_party/blink/public/common/fetch/fetch_api_request_body_mojom_traits.h b/third_party/blink/public/common/fetch/fetch_api_request_body_mojom_traits.h
+index 1ddfc2108f9a0104247ea2559b597552cd20a342..f9a10b5b428e29a8824b87616f19292b38e38024 100644
+--- a/third_party/blink/public/common/fetch/fetch_api_request_body_mojom_traits.h
++++ b/third_party/blink/public/common/fetch/fetch_api_request_body_mojom_traits.h
+@@ -11,12 +11,13 @@
+ #include "mojo/public/cpp/bindings/pending_remote.h"
+ #include "services/network/public/cpp/resource_request_body.h"
+ #include "services/network/public/mojom/url_loader.mojom-forward.h"
++#include "third_party/blink/public/common/common_export.h"
+ #include "third_party/blink/public/mojom/fetch/fetch_api_request.mojom-forward.h"
+
+ namespace mojo {
+
+ template <>
+-struct StructTraits<blink::mojom::FetchAPIRequestBodyDataView,
++struct BLINK_COMMON_EXPORT StructTraits<blink::mojom::FetchAPIRequestBodyDataView,
+                     scoped_refptr<network::ResourceRequestBody>> {
+   static bool IsNull(const scoped_refptr<network::ResourceRequestBody>& r) {
+     return !r;
+@@ -46,7 +47,7 @@ struct StructTraits<blink::mojom::FetchAPIRequestBodyDataView,
+ };
+
+ template <>
+-struct StructTraits<blink::mojom::FetchAPIDataElementDataView,
++struct BLINK_COMMON_EXPORT StructTraits<blink::mojom::FetchAPIDataElementDataView,
+                     network::DataElement> {
+   static const network::mojom::DataElementType& type(
+       const network::DataElement& element) {

patches/chromium/gin_with_namespace.patch

@@ -12,7 +12,7 @@ native_mate, and we should remove this patch once native_mate is erased
 from Electron.
 
 diff --git a/gin/arguments.h b/gin/arguments.h
-index eaded13e2991..03e1495566d1 100644
+index eaded13e29919793494dfe2f7f85fad7dcb125cf..03e1495566d1ab561dcd67517053173911288cea 100644
 --- a/gin/arguments.h
 +++ b/gin/arguments.h
 @@ -28,14 +28,14 @@ class GIN_EXPORT Arguments {
@@ -60,7 +60,7 @@ index eaded13e2991..03e1495566d1 100644
      (is_for_property_ ? info_for_property_->GetReturnValue()
                        : info_for_function_->GetReturnValue())
 diff --git a/gin/converter.h b/gin/converter.h
-index 27b4d0acd016..b19209a8534a 100644
+index 27b4d0acd016df378e4cb44ccda1a433244fe2c6..b19209a8534a497373c5a2f861b26502e96144c9 100644
 --- a/gin/converter.h
 +++ b/gin/converter.h
 @@ -250,7 +250,7 @@ std::enable_if_t<ToV8ReturnsMaybe<T>::value, bool> TryConvertToV8(