Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using tar stream in the browser #157

Open
vritant24 opened this issue Jun 22, 2023 · 8 comments
Open

Using tar stream in the browser #157

vritant24 opened this issue Jun 22, 2023 · 8 comments

Comments

@vritant24
Copy link

I'm trying to use this library in a browser environment, but I'm running into some issues I'd like help with.

const stream: ReadableStream<Uint8Array> = <get stream> 
await stream.pipeTo(extractor);

this doesn't work since pipeTo expects a WritableStream. If I change the line to:
await stream.pipeTo(extractor as unknown as WritableStream<Uint8Array>);

This fails with the following error:
TypeError: The "transform.writable" property must be an instance of WritableStream. Received an instance of Extract

Is there a recommended way of extracting a tar from a browser ReadableStream?

@mafintosh
Copy link
Owner

don't know much about browser streams, but if you share some docs on them, mb we can make a wrapper

@vritant24
Copy link
Author

https://developer.mozilla.org/en-US/docs/Web/API/Streams_API/Concepts

MDN has the docs for readable streams. unfortunately I'm a little less versed on these as well, but happy to help implement them @mafintosh

@mafintosh
Copy link
Owner

We just need a streamx -> browser stream general module. prop not too hard :)

@gchartier
Copy link

Any update on this? I have the exact same need / problem

@luiscastro193
Copy link

luiscastro193 commented Sep 14, 2023

+1 on this

@luiscastro193
Copy link

In the meantime, I recommend using https://www.npmjs.com/package/readable-web-to-node-stream and https://www.npmjs.com/package/readable-stream-node-to-web

@maxpatiiuk
Copy link

readable-web-to-node-stream appears to no longer be maintained, and fails to build in Webpack for me. I was able to use @smessie/readable-web-to-node-stream instead

Example code

Install dependencies:

npm i tar-stream @smessie/readable-web-to-node-stream
npm i -D @types/tar-stream

Content script:

const worker = new Worker('worker.js');

// User picks a file - we convert the file to a stream, and give it to a web worker:
const fileStream = file.stream();
worker.postMessage({ fileStream }, [fileStream]);

Web worker:

import { ReadableWebToNodeStream } from '@smessie/readable-web-to-node-stream';
import type { Readable } from 'node:stream';
import tar from 'tar-stream';

self.addEventListener('message', async (event) => {
  const fileStream: ReadableStream<Uint8Array> = event.data.fileStream;
  const decompressionStream = new DecompressionStream('gzip');
  const decompressedStream = fileStream.pipeThrough(decompressionStream);
  const extract = tar.extract();
  const nodeStream = new ReadableWebToNodeStream(decompressedStream) as unknown as Readable;
  nodeStream.pipe(extract);
  for await (const entry of extract) {
    console.log(entry.header.type, entry.header.name)
    entry.resume();
  }
});

@piranna
Copy link
Collaborator

piranna commented Feb 11, 2024

There's Native support for WebStreams un Node.js, we should use them instead of third-party modules.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants