Specify target of write operations

[rixo]

Hi! Thanks for this very useful collection of modules :)

I'm using unionfs to run some fs intensive tests. I have basically a layer shared among all tests (real fs), and a layer for test-specific input files, and I'd like to have a third layer to collect all the test write operations output. Or at least, make sure that any write operations only affects the test-specific layer, not the shared one.

Unfortunately, it seems that write operations will target the first fs where the parent directory exists:

import { Volume } from 'memfs'
import { Union } from 'unionfs'

const shared = Volume.fromJSON({
  '/app/foo': 'foo',
})

const specific = new Volume()

const fs = new Union()
  .use(shared)
  .use(specific)

fs.writeFileSync('/app/bar', 'bar0', 'utf8')

console.log(
  shared.existsSync('/app/bar'), // expected: false, actual: true
  specific.existsSync('/app/bar'), // espected: true, actual: false
)

specific.mkdirSync('/app')

fs.writeFileSync('/app/bar', 'bar1', 'utf8')

console.log(
  shared.existsSync('/app/bar'), // expected: false, actual: true
  specific.existsSync('/app/bar'), // expected: true, actual: true
)

One way to achieve this would be to have a way to specify the fs to use for write operations, and automatically creates directories in the write fs when they exists in previous fs volumes. Maybe something like this:

const fs = new Union()
  .use(shared)
  .use(specific)
  .writeTo(specific)

// considering above fixtures
fs.writeFileSync('/app/baz', ...) // OK
fs.writeFileSync('/other/foo', ...) // ENOENT

I think this also relates to #181, where a behaviour similar to what I'm describing for directories would be expected when appending to files.

[G-Rath]

I believe what you're wanting is a readonly mode - I've got plans to implement one when I have the bandwidth in the next month or so.

Unionfs will not create new folders for you automatically but if a fs is marked as read-only all methods that modify would instead throw (and thus cause unionfs to move onto the next fs or throw if it's the final fs).

I plan to expand the api of use:

const fs = new Union()
  .use(base, true)
  .use(vol)

fs.writeFileSync('/app/myfile'); // ENOENT
fs.mkdirSync('/app'); // create the folder in the vol fs
fs.writeFileSync('/app/myfile'); // OK

This should be pretty easy to implement, as it's just a matter of holding an object instead of just the fss:

class Unionfs {
  private ffs: Array<{
    fs: FileSystem;
    isReadonly: boolean;
  }> = []; // approx. type
}

[rixo]

Readonly mode is a welcome addition because it can give total serenity that I won't pollute my shared volume; especially appreciable when it is the real fs.

It might also cover some of my use case because my test targets (bundlers like Rollup & Webpack) will probably create the parent directories recursively when they don't exist.

However, I still don't know how I can elegantly handle the situation where some of the parent directory hierarchy already exists in the parents fs, but not the "output" fs (which might start completely empty typically). Because, for example, the whole union might report that /path/to/app already exist, so the bundler will only try to mkdir /path/to/app/dist, and fail because /path/to/app exists only on the readonly fs...

I can probably mitigate on a case by case basis, by manually creating well known output directories for each targets, but I was hoping to be able to write a more generic tool. For example, I'd like to handle the case of plugins that also write to other locations, without having to know about every such plugin and where they might output their intermediary results...

Ideally I'd like to intercept any incoming write operation on the union, which would provide an opportunity for me to mirror existing directories from the parents on the last volume of the union, before letting the write operation complete with the same behaviour it would have on the real fs.

Is there a central method I can override to achieve this, or something to this effect? Or do you have any advice on how I could implement this behaviour?

[G-Rath]

Readonly mode is a welcome addition because it can give total serenity that I won't pollute my shared volume; especially appreciable when it is the real fs.

That's exactly why I want it too 😂

Is there a central method I can override to achieve this, or something to this effect? Or do you have any advice on how I could implement this behaviour?

It sounds like you might want to use spyfs.

Ultimately, unionfs shouldn't create folders for you as that's a responsibility that belongs to the file system itself.

That's also a way you could work around this: replace fs with a Proxy that passes everything through to your unionfs, but that calls fs.mkdir with recursive: true on every path before hand should do the trick :)

Something like this should do:

new Proxy({}, {
  get(_, property: string): (...args: unknown[]) => unknown {
    return (...args: unknown[]) => {
      unionfs.mkdir(args[0], { recursive: true });

      return unionfs[property].call(args);
    }
  }
}

(That's off the top of my head, so it's probably somewhat wrong, but thats the basic shape you want)

[elmpp]

Hi @G-Rath, @streamich would be grateful if you cast a beady eye over my PR