Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve streams #150

Merged
merged 7 commits into from
Jun 17, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
51 changes: 33 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,14 +27,14 @@ If you want to pack/unpack directories on the file system check out [tar-fs](htt
To create a pack stream use `tar.pack()` and call `pack.entry(header, [callback])` to add tar entries.

``` js
var tar = require('tar-stream')
var pack = tar.pack() // pack is a stream
const tar = require('tar-stream')
const pack = tar.pack() // pack is a stream

// add a file called my-test.txt with the content "Hello World!"
pack.entry({ name: 'my-test.txt' }, 'Hello World!')

// add a file called my-stream-test.txt from a stream
var entry = pack.entry({ name: 'my-stream-test.txt', size: 11 }, function(err) {
const entry = pack.entry({ name: 'my-stream-test.txt', size: 11 }, function(err) {
// the stream was added
// no more entries
pack.finalize()
Expand All @@ -54,21 +54,21 @@ pack.pipe(process.stdout)
To extract a stream use `tar.extract()` and listen for `extract.on('entry', (header, stream, next) )`

``` js
var extract = tar.extract()
const extract = tar.extract()

extract.on('entry', function(header, stream, next) {
extract.on('entry', function (header, stream, next) {
// header is the tar header
// stream is the content body (might be an empty stream)
// call next when you are done with this entry

stream.on('end', function() {
stream.on('end', function () {
next() // ready for next entry
})

stream.resume() // just auto drain the stream
})

extract.on('finish', function() {
extract.on('finish', function () {
// all entries read
})

Expand All @@ -77,6 +77,21 @@ pack.pipe(extract)

The tar archive is streamed sequentially, meaning you **must** drain each entry's stream as you get them or else the main extract stream will receive backpressure and stop reading.

## Extracting as an async iterator

The extraction stream in addition to being a writable stream is also an async iterator

``` js
const extract = tar.extract()

someStream.pipe(extract)

for await (const entry of extract) {
entry.header // the tar header
entry.resume() // the entry is the stream also
}
```

## Headers

The header object using in `entry` should contain the following properties.
Expand Down Expand Up @@ -106,18 +121,18 @@ Most of these values can be found by stat'ing a file.
Using tar-stream it is easy to rewrite paths / change modes etc in an existing tarball.

``` js
var extract = tar.extract()
var pack = tar.pack()
var path = require('path')
const extract = tar.extract()
const pack = tar.pack()
const path = require('path')

extract.on('entry', function(header, stream, callback) {
extract.on('entry', function (header, stream, callback) {
// let's prefix all names with 'tmp'
header.name = path.join('tmp', header.name)
// write the new entry to the pack stream
stream.pipe(pack.entry(header, callback))
})

extract.on('finish', function() {
extract.on('finish', function () {
// all entries done - lets finalize it
pack.finalize()
})
Expand All @@ -133,15 +148,15 @@ pack.pipe(newTarballStream)


``` js
var fs = require('fs')
var tar = require('tar-stream')
const fs = require('fs')
const tar = require('tar-stream')

var pack = tar.pack() // pack is a stream
var path = 'YourTarBall.tar'
var yourTarball = fs.createWriteStream(path)
const pack = tar.pack() // pack is a stream
const path = 'YourTarBall.tar'
const yourTarball = fs.createWriteStream(path)

// add a file called YourFile.txt with the content "Hello World!"
pack.entry({name: 'YourFile.txt'}, 'Hello World!', function (err) {
pack.entry({ name: 'YourFile.txt' }, 'Hello World!', function (err) {
if (err) throw err
pack.finalize()
})
Expand Down
12 changes: 12 additions & 0 deletions constants.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
try {
module.exports = require('fs').constants
} catch {
module.exports = { // just for envs without fs
S_IFMT: 61440,
S_IFDIR: 16384,
S_IFCHR: 8192,
S_IFBLK: 24576,
S_IFIFO: 4096,
S_IFLNK: 40960
}
}