Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide virtual input file system to .build() #690

Open
ggoodman opened this issue Jan 18, 2021 · 11 comments · May be fixed by #2968
Open

Provide virtual input file system to .build() #690

ggoodman opened this issue Jan 18, 2021 · 11 comments · May be fixed by #2968

Comments

@ggoodman
Copy link

Description

Allow an initial partial filesystem seed to be provided as an input to a .build() operation.

Use-case

In order to increase the common capabilities between the WASM-based and binary-based implementations, it would be helpful to have some measure of control over the filesystem. I thought that one potential avenue for this that would avoid expensive and complicated coordination between esbuild and the controlling system might be to allow an initial FS snapshot to be seeded.

For my use-case, this would allow me to fully embrace esbuild-wasm and have platform and runtime environment portability.

@evanw
Copy link
Owner

evanw commented Jan 22, 2021

This is an interesting idea, and is something that I assume would be far faster than a previous proposal you had somewhere for a file system API to essentially intercept readFile and readdir IIRC. I'll think more about this proposal.

@JounQin
Copy link

JounQin commented Jun 29, 2021

I'm trying to develop an HTML as entry plugin, and I need the virtual file system too, especially for esbuild.serve.

It calls esbuild.build in the onLoad hook of plugin at https://github.com/rx-ts/esbuilder/blob/main/packages/html/src/loader.ts#L93.

When using esbuild.build, there is a physical file system, so that the generated files by esbuild.build in plugin will also be available in the top level HTML files.

However, for esbuild.serve, no virtual file system is provided, and I can't inject the generated files into the service of esbuild.serve. So I have to hack with data:text/javascript;base64,. But I can imagine this will not work when using dynamic import because of multi files.

@aletorrado
Copy link

It would be nice to have an output virtual filesystem as well because we do some runtime/dynamic js bundling (esbuild's speed is an incredible fit for that).

@loynoir
Copy link

loynoir commented Aug 1, 2021

gulp use vinyl as virtual fs
https://gulpjs.com/docs/en/api/vinyl/

@PierBover
Copy link

There are many use cases for virtual entries. I've used this feature in Webpack and Rollup many times for creating hydration scripts, dynamic entry points, Svelte components, etc.

It would be great if there was a way to solve this with esbuild!

@jimafisk
Copy link

Virtual entrypoints would help me avoid expensive read/writes to the filesystem for Svelte SSR components, which I'm only using to render to HTML and then discard.

@evanw
Copy link
Owner

evanw commented Mar 14, 2023

Virtual entrypoints would help me avoid expensive read/writes to the filesystem for Svelte SSR components, which I'm only using to render to HTML and then discard.

I've found writing to the file system to actually be more efficient than passing data through esbuild's IPC pipe, especially for large amounts of data. This is why esbuild's transform API deliberately writes the input string to the file system anyway. If your code base is large enough that the performance cost of running esbuild is noticeable, a virtual file system API would probably make things slower instead of faster.

@jakebailey
Copy link

For my use-case, this would allow me to fully embrace esbuild-wasm and have platform and runtime environment portability.

FYI, Go 1.21 will feature GOOS=wasip1 GOARCH=wasm which esbuild "just works" under and so should work in WASI runtimes (+/- runtime / Go bugs; the target is new). I already have it working in the browser.

The target still has some issues being worked through for various runtimes, so hopefully by 1.21 it's stable.

@callumlocke
Copy link

Just to clarify, is the feature being requested here conceptually similar to @rollup/plugin-virtual or have I misunderstood?

@ggoodman
Copy link
Author

I would say that @evanw has proven that a full virtual FS is achievable through https://esbuild.github.io/try/.

A precedent for this was produced in https://github.com/esbuild/esbuild.github.io/blob/main/src/try/fs.ts but what I'd love to see is a bit more of an ergonomic approach / DX. I'd also love to see this sort of thing to:

  • Allow plugins to provide non-runtime files that influence the build like package.json and tsconfig.json.

@jakebailey
Copy link

That's not really indicitive of the effort needed to do this. The FS API used to achieve that playground page relies on the implementation detail that Go sticks its FS abstractions onto the globals when executed in the browser (https://github.com/golang/go/blob/master/misc/wasm/wasm_exec.js#L16), not that there's been any change internally within esbuild to allow a VFS anywhere other than the browser using this trick. The same goes for my WASI version; it's modifying the host, not esbuild.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

9 participants