Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check out await-sync - replace child process with WebWorkers #36

Open
jimmywarting opened this issue Apr 18, 2023 · 4 comments
Open

Check out await-sync - replace child process with WebWorkers #36

jimmywarting opened this issue Apr 18, 2023 · 4 comments

Comments

@jimmywarting
Copy link

jimmywarting commented Apr 18, 2023

I just created this async-to-sync package today that i named await-sync

it utilize web workers instead of spawning sync processes and data is transfered synchronous over SharedArrayBuffer so less code needs to be copied over.

Therefore it's also more compatible with other enviorments like Deno, Bun, and also Web Workers.

it's as simple as just doing:

import { createWorker } from 'to-sync'

const awaitSync = createWorker()

const fetchSync = awaitSync(async function (...args) {
  const res = await fetch(...args)
  const ab = await res.arrayBuffer()
  return new Uint8Array(ab)
})

const uint8 = fetchSync(url)
const text = new TextDecoder().decode(uint8)
const json = JSON.parse(text)
const blob = new Blob([uint8])

if you would use this instead then your package would have the possibility of being able to run in Deno, Bun.js and also Web Workers. but why web workers when you sync xhr...? that's a good question...

@larsgw
Copy link
Owner

larsgw commented Apr 18, 2023

I think someone else might have mentioned web workers before. I haven't used Deno or Bun but I'm not against supporting that. I think the main doubt I had was the same as what you mention:

but why web workers when you sync xhr...? that's a good question...

But there's actually a clear answer to that: sync xhr has a bunch of limitations, including not allowing binary responses and (at least in the past) some cryptic, undocumented CORS restrictions.

@jimmywarting
Copy link
Author

jimmywarting commented Apr 18, 2023

including not allowing binary responses

Oh, that brings me back in the memory lane...
Remembered trying out to fetch some pdf or some kind of binary file and had problem with that.

And now i remembered that i also wanted to try and read Blob's in a sync mannar also in the main thread in browsers too.

i even have a old gist for that.

const blob = new Blob(['123'])
const xhr = new XMLHttpRequest()
const url = URL.createObjectURL(blob)

// required if you need to read binary data:
xhr.overrideMimeType('text/plain; charset=x-user-defined') 
xhr.open('GET', url, false)
xhr.send()

const uint8 = Uint8Array.from(xhr.response, c => c.charCodeAt(0))
const text = new TextDecoder().decode(uint8)
const arrayBuffer = uint8.buffer
const json = JSON.parse(text)

the trick is to override mime type and use a charset that isn't recognised. then xhr won't try to be smart and change the response, so you will actually get back the raw bytes.
but then you can't use responseType but you will be able create those later more manually

@jimmywarting
Copy link
Author

ironically MDN says it's bad to use sync

This restriction is designed in part to help ensure that synchronous operations aren't used for large transactions

And yet they have a tutorial on how to get around it here: Receiving binary data in older browsers

overriding the MIME type, force the browser to treat it as plain text, using a user-defined character set. This tells the browser not to parse it, and to let the bytes pass through unprocessed.

@jimmywarting jimmywarting changed the title Check out to-sync - replace child process with WebWorkers Check out awaitSync - replace child process with WebWorkers Apr 19, 2023
@jimmywarting jimmywarting changed the title Check out awaitSync - replace child process with WebWorkers Check out await-sync - replace child process with WebWorkers Apr 19, 2023
larsgw added a commit that referenced this issue Jun 11, 2023
@larsgw
Copy link
Owner

larsgw commented Jun 11, 2023

I've implemented the trick for binary data from MDN that you mentioned, it's released in v0.4.4.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants