MessagePort, MessageChannel and StructuredClone implementation

[alexandercerutti]

Hi there! I was in need of testing my implementation for a system based on postMessages and MessageChannels but noticed there wasn't an implementation inside jsdom.

I started looking around in issues and pull requests and found these issues and failed attempts:

• Support transfer on postMessage #3287
• Add support for ports in postmessage #3284
• add support for ports #3507

So based on the comments in the discussions above, I decided to fork jsdom and attempt to implement both of them.

Of course, this is an untested and maybe quite incomplete implementation, but I tried to made it to follow the standard.

So, I'm opening this draft PR to let you know I'm working on it and because, as this is my first PR here, I'd like to receive an early feedback and some guidance about the next steps, because I had a few difficulties with testing and so on.

I hope this attempt is appreciated and I'm really looking forward into collaborating with the community to bring this into production.

Thank you!

Closes #3363
Closes #3287

[alexandercerutti]

Hey @domenic, sorry for tagging you but I don't know who else to tag. Two weeks have passed. Did you perhaps have a chance to check for my work?

Thank you

[domenic]

I think the next step is seeing if you pass the relevant tests.

Since I notice you're implementing the structured clone algorithm as part of this, perhaps the simpler starting point would be to implement and pass the tests for self.structuredClone(). Those can be found in https://github.com/web-platform-tests/wpt/tree/master/html/webappapis/structured-clone . https://github.com/jsdom/jsdom/blob/master/Contributing.md#web-platform-feature-tests has information about how to get those tests running in jsdom.

[alexandercerutti]

@domenic thank you for your reply.

So, perhaps would it be better to move structured cloning functions to a different file and folder, outside of MessagePort and also implement the structuredClone API?

Should I honor the URL path? Like, if the docs url is https://html.spec.whatwg.org/#dom-structuredclone, the folder should be living/structuredclone, isn't it?

[domenic]

This is impressive work; thanks! To be clear though, I was suggesting one single PR to land structured clone, and then later on top of that build MessagePort/MessageChannel. I'll review just the structured clone parts for now.

However, the biggest work here is getting the tests to pass. As you can see from CI, there's still a good amount of issues with the algorithm.

Good luck!

[domenic]

It's best not to copy the spec, since it gets out of date over time. Instead just provide a single-line comment linking to the relevant spec, so people can compare the spec and the code side-by-side.

[alexandercerutti]

I copied it almost everywhere ahah.
I think we might keep them until the last moment. I mean, I do not expect specs to change in short time. Wdyt?

[alexandercerutti]

Thank you! It is fine to me. I mean, structuredClone is the biggest part now. I don't think it is needed to get back and open another pr for these two. I mean, they are... "entangled" (like MessagePorts 😄).
I'll review you comments soon. Thanks again

[alexandercerutti]

Hey @domenic, I hit a wall that I honestly don't really know how to bypass.

Among the tests I found this one:

https://github.com/web-platform-tests/wpt/blob/fb1192db4cdcd515e5f8c4e478f3e2f908ce2d8d/html/webappapis/structured-clone/structured-clone-battery-of-tests.js#L77-L98

What follows is given that I had to convert serialization memorys to be a Map (not yet committed) because saving objects in memory would have given a series of [object Object] and, therefore, some wrong serialization and deserialization results. Also, without a Map we are unable to detect circular references.

By converting the memory to be a Map, I'm able to keep a series of data and pointers to objects themselves.

I found that both Map and Objects treat -0 and +0 to be always +0. Therefore, by quering an array, if there's 0 and then a -0 (lines 87 and 88), -0 lookup in the memory gets resolved to be a 0 and therefore the structure saved and the numberData received becomes wrong.

I see that both numbers can be checked by using Object.is(0, -0) but in a structure like this, having an explicit check on 0 feels to be a tailored thing and not very good to me.

Also, converting -0 to be a string is not a viable way.

Do you, perhaps. have any possible solution to this?
Is this perhaps a case where we have explicitly to get out of the standard?

Thank you.

[alexandercerutti]

Okay, nvm. I found that using .toLocaleString(), -0 gets preserved. So now I can proceed into solving the other issues.

[alexandercerutti]

Hey there @domenic, I just hit another wall. This one is thicker this time.

These are the tests that are now failing and that I don't know how to solve them:

Open the list

Failed in "Blob basic":
	assert_true: instanceof Blob expected true got false

Failed in "Blob unpaired high surrogate (invalid utf-8)":
	assert_true: instanceof Blob expected true got false

Failed in "Blob unpaired low surrogate (invalid utf-8)":
	assert_true: instanceof Blob expected true got false

Failed in "Blob paired surrogates (invalid utf-8)":
	assert_true: instanceof Blob expected true got false

Failed in "Blob empty":
	assert_true: instanceof Blob expected true got false

Failed in "Blob NUL":
	assert_true: instanceof Blob expected true got false

Failed in "Array Blob object, Blob basic":
	assert_true: instanceof Blob expected true got false

Failed in "Array Blob object, Blob unpaired high surrogate (invalid utf-8)":
	assert_true: instanceof Blob expected true got false

Failed in "Array Blob object, Blob unpaired low surrogate (invalid utf-8)":
	assert_true: instanceof Blob expected true got false

Failed in "Array Blob object, Blob paired surrogates (invalid utf-8)":
	assert_true: instanceof Blob expected true got false

Failed in "Array Blob object, Blob empty":
	assert_true: instanceof Blob expected true got false

Failed in "Array Blob object, Blob NUL":
	assert_true: instanceof Blob expected true got false

Failed in "Array Blob object, two Blobs":
	assert_true: instanceof Blob expected true got false

Failed in "Object Blob object, Blob basic":
	assert_true: instanceof Blob expected true got false

Failed in "Object Blob object, Blob unpaired high surrogate (invalid utf-8)":
	assert_true: instanceof Blob expected true got false

Failed in "Object Blob object, Blob unpaired low surrogate (invalid utf-8)":
	assert_true: instanceof Blob expected true got false

Failed in "Object Blob object, Blob paired surrogates (invalid utf-8)":
	assert_true: instanceof Blob expected true got false

Failed in "Object Blob object, Blob empty":
	assert_true: instanceof Blob expected true got false

Failed in "Object Blob object, Blob NUL":
	assert_true: instanceof Blob expected true got false

Failed in "File basic":
	assert_true: instanceof File expected true got false

Failed in "FileList empty":
	assert_true: instanceof FileList expected true got false

Failed in "Array FileList object, FileList empty":
	assert_true: instanceof FileList expected true got false

Failed in "Object FileList object, FileList empty":
	assert_true: instanceof FileList expected true got false

Failed in "ImageData 1x1 transparent black":
	promise_test: Unhandled rejection with value: object "TypeError: Cannot read properties of null (reading 'createImageData')"

Failed in "ImageData 1x1 non-transparent non-black":
	promise_test: Unhandled rejection with value: object "TypeError: Cannot read properties of null (reading 'createImageData')"

Failed in "Array ImageData object, ImageData 1x1 transparent black":
	promise_test: Unhandled rejection with value: object "TypeError: Cannot read properties of null (reading 'createImageData')"

Failed in "Array ImageData object, ImageData 1x1 non-transparent non-black":
	promise_test: Unhandled rejection with value: object "TypeError: Cannot read properties of null (reading 'createImageData')"

Failed in "Object ImageData object, ImageData 1x1 transparent black":
	promise_test: Unhandled rejection with value: object "TypeError: Cannot read properties of null (reading 'createImageData')"

Failed in "Object ImageData object, ImageData 1x1 non-transparent non-black":
	promise_test: Unhandled rejection with value: object "TypeError: Cannot read properties of null (reading 'createImageData')"

Failed in "ImageBitmap 1x1 transparent black":
	promise_test: Unhandled rejection with value: object "ReferenceError: createImageBitmap is not defined"

Failed in "ImageBitmap 1x1 non-transparent non-black":
	promise_test: Unhandled rejection with value: object "TypeError: Cannot read properties of null (reading 'getImageData')"

Failed in "Array ImageBitmap object, ImageBitmap 1x1 transparent black":
	promise_test: Unhandled rejection with value: object "ReferenceError: createImageBitmap is not defined"

Failed in "Array ImageBitmap object, ImageBitmap 1x1 transparent non-black":
	promise_test: Unhandled rejection with value: object "TypeError: Cannot read properties of null (reading 'getImageData')"

Failed in "Object ImageBitmap object, ImageBitmap 1x1 transparent black":
	promise_test: Unhandled rejection with value: object "ReferenceError: createImageBitmap is not defined"

Failed in "Object ImageBitmap object, ImageBitmap 1x1 transparent non-black":
	promise_test: Unhandled rejection with value: object "TypeError: Cannot read properties of null (reading 'getImageData')"

Failed in "Serializing a non-serializable platform object fails":
	promise_test: Unhandled rejection with value: object "ReferenceError: Response is not defined"

Failed in "An object whose interface is deleted from the global must still deserialize":
	assert_true: expected true got false

Failed in "A subclass instance will deserialize as its closest serializable superclass":
	assert_equals: expected object "[object File]" but got object "[object Object]"

Failed in "Resizable ArrayBuffer":
	assert_true: expected true got undefined

Failed in "Growable SharedArrayBuffer":
	assert_true: expected true got undefined

Failed in "Length-tracking TypedArray":
	assert_true: expected true got undefined

Failed in "Length-tracking DataView":
	assert_true: expected true got undefined

Failed in "Serializing OOB TypedArray throws":
	promise_test: Unhandled rejection with value: object "TypeError: ab.resize is not a function"

Failed in "Serializing OOB DataView throws":
	promise_test: Unhandled rejection with value: object "TypeError: ab.resize is not a function"

Failed in "ArrayBuffer":
	assert_equals: expected 0 but got 1

Failed in "MessagePort":
	promise_test: Unhandled rejection with value: object "DataCloneError: The object can not be cloned."

Failed in "A detached ArrayBuffer cannot be transferred":
	assert_unreached: Should have rejected: undefined Reached unreachable code

Failed in "A detached platform object cannot be transferred":
	promise_test: Unhandled rejection with value: object "DataCloneError: The object can not be cloned."

Failed in "Transferring a non-transferable platform object fails":
	assert_unreached: Should have rejected: undefined Reached unreachable code

Failed in "An object whose interface is deleted from the global object must still be received":
	promise_test: Unhandled rejection with value: object "DataCloneError: The object can not be cloned."

Failed in "A subclass instance will be received as its closest transferable superclass":
	promise_test: Unhandled rejection with value: object "ReferenceError: ReadableStream is not defined"

Failed in "Resizable ArrayBuffer is transferable":
	assert_equals: expected 0 but got 16

Failed in "Length-tracking TypedArray is transferable":
	assert_equals: expected 0 but got 16

Failed in "Length-tracking DataView is transferable":
	promise_test: Unhandled rejection with value: object "TypeError: First argument to DataView constructor must be an ArrayBuffer"

Failed in "Transferring OOB TypedArray throws":
	promise_test: Unhandled rejection with value: object "TypeError: ab.resize is not a function"

Failed in "Transferring OOB DataView throws":
	promise_test: Unhandled rejection with value: object "TypeError: ab.resize is not a function"

As you might notice the biggest topics here are:

• Growable / Resizable ArrayBuffer
• Platform Objects (like Blobs, File, FileList)
• Detaching buffers

---

Growable / Resizable ArrayBuffers are not supported in Node yet. I don't know how could this be implemented in JSDOM if so and I don't know if I can go on somehow. So these kinds of tests will always fail until a Node version will support these features.

---

I honestly don't know how to verify if something is a platform object. The specification is quite vague and refers to IDL, which means that every single native object should probably get defined in the code in a completely arbitrary way, but I don't know the current situation of JSDOM nor a full list of Serializable and non-serializable objects (is there an index somewhere?). I don't know if all the native classes are implemented inside JSDOM and if so how to interact with them.

---

The final biggest point is about detaching buffers.
I don't know if you ever had the chance to reason about it. I saw that in WPT and in JSDOM, some code attempts to detach buffers by using window.postMessage('', '*', [buffer]); which should be completely useless because the current postMessage function in JSDOM doesn't perform any real detaching.

So, how should buffers get detached in your opinion? I didn't find a correct way, other than postMessage to do that in JS.

---

Other than the points above, there are some undefined methods that I didn't dig into to understand what was the issue (like ReferenceError: createImageBitmap is not defined).

Also, there's a specific point where I literally said "WTF?".

An object whose interface is deleted from the global object must still be received

I honestly cannot figure out how a deleted reference from the global object can get retrieved from a function like "structureDeserialize", which receives the targetRealm, without receiving a specific reference to that object.

So, I cannot go on with this PR without finding out how to fix these three points.

My current idea is that we'll get forced to ignore some tests.

Let me know,
thank you!

[alexandercerutti]

Hey @domenic, sorry again for tagging you, but I'd like to bring this on, but I'm locked right now.

Thank you!

[domenic]

Hey, I'm sorry, but the point of needing PRs that pass the tests is that I don't have time to do feature development. That includes helping others with their code for feature development. Maybe you can find a collaborator, but unfortunately I don't have the time to do more than review and merge PRs that already meet the project requirements of passing the test suite. Getting it to pass needs to be done by you!

[alexandercerutti]

@domenic I see your point. First of all, thank you for your reply.
I wouldn't have tagged you if not in need. I was just asking a point of view from you (or someone else that already have experience with maintaining jsdom).

I'll try to figure out a way to fix those things or ignore the tests.

[blipper]

https://nodejs.org/en/blog/announcements/v20-release-announce supports resizable?