[RFC] Parcel 2: Better Electron support

[devongovett]

Parcel 1 has an electron target, but it is not super well baked. We should improve on it for Parcel 2.

• Support separate electron and electron-renderer environments. electron is the main process (node), and electron-renderer is the browser environment. They have slightly different characteristics.
• Support external builtin modules for both electron and electron-renderer like we do for node. This means we should ignore require calls to builtin electron modules.
• Automatically detect calls to the BrowserWindow constructor, and loadFile method, and add the referenced file as a dependency (similar to web workers and service workers). This should also start a new environment with the electron-renderer context automatically.
• Disable environment variable substitution and fs inlining in both electron environments since those are available natively.
• Disable bundling node_modules for electron like we do for node? (not sure about this one)
• Better support for native node modules (separate issue)
• Improved HMR. Ideally support both main and renderer process.

Related: #1738, #2042, #840.

Please let me know what else would be useful for electron development with Parcel! 😄

[devongovett]

@kitze on twitter:

replacing this with 1 line: https://github.com/kitze/JSUI/blob/master/package.json#L164-L167

[atilacamurca]

Documentation and options to generate the release, that is exe, app, deb, rpm, etc.

[timfish]

Once you move away from the most basic examples, there are more than just those two environments to cater for.

You have:

• Main process - node
• Renderer process - Chrome + node
• Renderer process (with nodeIntegration: false) - Chrome
• Renderer preload script - Chrome + node
• Renderer preload script (sandbox: true) - Chrome + limited node env. with no require
• There is also a nodeIntegrationInWorker which defaults to false but can give you full node env in web workers 🤦‍♂️

If you're loading remote content or want to protect from other arbitrary code execution, it's recommended to disable nodeIntegration in the renderer and expose only the required Electron/Node API's via a preload script. Here is an example of this. In Electron v5 nodeIntegration will default false to push devs towards making better security decisions.

With sandbox mode enabled, the preload environment is even more restrictive and API's have to be exposed via IPC to the main process. The key distinction in this mode from a bundler point of view, is that the entire preload script must be in a single file, so no bundle splitting.

Most production apps will also end up with some native modules to contend with and then bundling node-modules gets very tricky. The majority of commonly used native modules use the bindings or node-pre-gyp modules to dynamically locate the binary at run time.

With Webpack I've managed to get it to bundle main/renderer/preload entry points, including all node_modules + native binaries using awesome-node-loader. To do this, I had to use NormalModuleReplacementPlugin to replace bindings module with fixed paths. This is obviously rather manual and nasty:

webpack.config.js

new webpack.NormalModuleReplacementPlugin(/^bindings$/, require.resolve('./bindings')),

bindings.js

module.exports = function(str) {
  if (str === 'detection.node') {
    const result = require('../node_modules/usb-detection/build/Release/detection.node');
    result.path = '../node_modules/usb-detection/build/Release/detection.node';
    return result;
  }
};

[malept]

👋 Electron Forge developer here. I'm very interested in this because I'd like to have integrated support for Parcel 2, like we currently do (in beta) for Webpack.

• Disable bundling node_modules for electron like we do for node? (not sure about this one)

This is probably for the best, given that there will likely be native modules in there.

[...] options to generate the release, that be exe, app, deb, rpm, etc.

I think this is outside the scope of Parcel. You're better off using a tool like Electron Forge.

[devongovett]

@timfish thanks so much, that's really great info! 👍

Sounds like we will need a few more environments, or perhaps options for them to enable/disable node integration etc. We could support detecting the preload script via static analysis as well I think.

On native modules, we are looking to support them for both Node and Electron by doing static analysis of bindings and node-pre-gyp calls, similar to how ncc does it. See #2493.

[devongovett]

@malept re generating the release, what do you think about a Parcel plugin using electron-forge or electron-packager under the hood to do this? Parcel is primarily a runner of other tools, be that babel, typescript, postcss, even Rust. It does all of this via plugins, so a Parcel plugin to run electron-forge would make sense I think.

Parcel 2 has the concept of "targets", which allows compiling your app for different environments (e.g. browser, node, etc.). One of those targets could be electron binaries of various types. Then all you need in your package.json:

{
  "electron:osx": "./dist/app.app",
  "electron:exe": "./dist/app.exe",
  "engines": {
    "electron": ">=x.x.x"
  }
}

I'm probably oversimplifying this, but lmk what you think!

[devongovett]

I should add that the advantage of inverting the model to have Parcel run electron-forge rather than the other way around is that you could take advantage of the other plugins in the Parcel ecosystem to run things like compilers, resolvers, optimizers, etc., utilize the Parcel cache, compile your app for multiple targets at once (electron + web + react native), and more.

[malept]

I can see the usefulness of having an Electron Forge plugin for Parcel. However, in order for the development mode workflow of Electron Forge to work (e.g., electron-forge start), there would need to be a Parcel plugin for Forge so features like HMR can work correctly in that mode. I don't (currently) see a reason why both kinds of plugins can't exist at the same time, in the same (Electron app) project.

Since Electron Forge encompasses the entire Electron app development workflow, it also provides a publish function. I'd imagine that feature is out of scope for Parcel.

[devongovett]

Good points! I think there are valid reasons for both workflows to exist.

[anaisbetts]

@devongovett A great "getting started" Parcel+Electron integration would be a new start-app command, that would bring down Electron, run Parcel on the app, then run the app in Electron with HMR set up - this would be super easy for Parcel to do, and would Get People Excited about what they could do as a desktop app

[felixrieseberg]

Disable bundling node_modules for electron like we do for node? (not sure about this one)

I get the motivation behind that, but bundling node modules is a big performance win for Electron apps, in particular on Windows. In short, file system access and script parsing is a lot slower for many small files than for one big one, so I’d love to see Parcel making bundling easier for Electron apps. That said, it’s true that native node addons are pretty popular with Electron devs and getting this right might be tricky.

re generating the release, what do you think about a Parcel plugin using electron-forge or electron-packager under the hood to do this?

I’m not sure that’s a great idea - Forge needs to run plenty of fairly big operations and I’m afraid that inverting the relationship would constrict Electron developers. Web bundling is just one of many steps to a binary (think signing of binaries, releasing to update servers, integrating with app stores) and I’d be concerned that Electron is simply not a big enough share of Parcel users. In other words - Parcel doesn’t quite work for Electron out of the box and I’d need some convincing to trust it with building my apps.

[timfish]

@devongovett thanks. I didn't know ncc already attempts to solve the bindings/node-pre-gyp static analysis.

It's also possible to set preload scripts per session. You can no doubt pick these out and automatically bundle them too, but it would be tricky to work out which renderers these will be loaded into and therefore which preload environment to target.

[frankshaka]

Disable bundling node_modules for electron like we do for node? (not sure about this one)

We adopt Two package.json Structure in our Electron projects.

In the outer ./package.json we put development dependencies and declare non-native production modules that should be bundled (to improve loading+parsing time) with entry points.

In the inner ./app/package.json we simply put dependencies of native modules that don’t need bundling. This package info file will be bundled inside the target app so that during runtime Electron can locate these modules in node_modules folder.

Currently we have to write complex Webpack config files (with lots of explanatory comments) to accomplish this behavior in build process. It will be highly appreciated if Parcel can support this, too.

[MarshallOfSound]

Hey @devongovett Thanks for opening this issue and moving forward on this. Just wanted to chime in here as an Electron and Electron Forge maintainer that I agree with @malept and @felixrieseberg on their points here. Mostly around:

• Parcel should definitely still bundle node_modules for Electron apps, it is a significant start-up performance hurdle in many apps even if they don't know it
• In an ideal world, Electron Forge would call parcel (as a plugin similar to how webpack is set up) rather than the other way around

Weighing in quite heavily on what @frankshaka just outlined above. This is strongly IMO but based heavily in experience in this area.

No one should be using the two package.json structure

It doesn't solve any problems that haven't been solved with module-tree tracing for rebuilding native modules (what electron-rebuild does) and in fact makes it incredibly complicated / difficult to use any standard tooling within your app (actually pointed out in the comment above)

Currently we have to write complex Webpack config files (with lots of explanatory comments) to accomplish this behavior in build process

TLDR is please don't go down the route of two package.json structure. It was an incredibly hacky / bad solution to a problem that has since been solved in much better ways.

[frankshaka]

@MarshallOfSound Maybe my information is a little outdated (a few months?), but I’m strongly curious about what are the better ways you mentioned to solve problems we are facing like having the same native module (e.g. fsevents) running simultaneously in Node and Electron during development, which should be built against different targets. Aren’t they stored in the same node_modules folder with the same path?

If there IS a better way, please do tell me and I would love to adopt it in our projects and replace two package.json structure.

[MarshallOfSound]

If there IS a better way, please do tell me and I would love to adopt it in our projects and replace two package.json structure.

@frankshaka I've only run across the situation you describe once before (and in fact it was with fsevents). The easiest solution, if you're using yarn as your package manager, is to force yarn to resolve fsevents to two different versions using dependency resolution. Then tools like electron-rebuild will only rebuild one of them (the one your app uses) and leave the one you use for dev alone.

Although also a "hacky" solution, it beens that standard tooling like electron-rebuild, electron-forge and webpack and friends don't get confused by multiple package.jsons

[frankshaka]

@MarshallOfSound Thanks for the reply, and it has opened my mind. However, we cannot adopt your solution immediately because our projects relies heavily on npm (and I’m sure that someone may remind me that no one should be using npm now). Thanks anyway.

I know this is not the right place to argue the political correctness of the Two package.json strcture (TPS for short), but since Parcel is interested in better support of Electron, let me explain how Parcel can benefit from such structure and why it’s more intuitive.

1. TPS helps developers define a clear boundary between what should be bundled and what should not.

   Besides native modules, we have some other modules that we want to keep outside of the bundle and load them from disk (long stories if you wonder).

   With TPS, we declare those packages in the inner package.json, and externalize them all at once in webpack config. So when you look at the inner package.json, it declares exactly the packages that exist in the node_modules folder in the final Electron app package. It’s clear and easier to maintain.

   Without TPS, Parcel has to either a) guess what to exclude, or b) let developers specify what to exclude somewhere else. The former may cause incorrectness (even native modules are sometimes hard to detect), while the latter complicates things. Until How do i mark some modules to external? #144 is resolved, there will not be external in Parcel.

2. The inner package.json can act as a file template for the production one.

   No matter you like it or not, you have to have a new package.json file generated inside the dist app package, to provide important information to Electron runtime, e.g. package name, version, entry script, dependencies, as well as some custom metadata developers may want to inject.

   With TPS, all you have to do is to put all these declarations inside the inner package.json, and let the tooling copy it or slightly modify it. Doesn’t It sound pretty natural to have a dist/app/package.json built from src/app/package.json?

   Without TPS, Parcel or Electron Forge may have to either a) guess what to gather, or b) let developers specify what to inject as extra metadata. Cool, another build option!

IMHO, all the complexity of implementing TPS is just because it’s not a standard and thus no tools like to support it. It’s not a standard because most of Electron apps are simply wrappers of their existing web apps! They may even share the same repository with the web app projects. If I were them I definitely not need another package.json. But if you’ve ever dived into the development of a serious desktop-only app, you will find there’re often barely few choices you can make. The result is that we found the TPS is the most intuitive way to self-express the project’s structure while allowing the developers to have some control at the build time.

PLEASE, consider some support for such TPS usage, and if you don’t want to make it the default behavior in Parcel, at least provide an option to support it, and that would greatly reduce your work as well as ours and make Electron projects more self-expressive.

[MarshallOfSound]

@frankshaka I'm not going to have this discussion here to avoid rail-roading this issue. I've made my opinion clear and that opinion is based on years of experience building Electron apps, Electron tooling, and Electron itself. If you want some help/pointers moving off of TPJS I'm more than happy to hear the problems you are solving through it and point you in the right direction for better alternatives. Just ask away in the #atomio Electron slack channel and I'll chime in.

I would ask the Parcel team though (@devongovett and folks) to not introduce complexity by supporting TPJS, it is not a recommended technique of either Electron or any of the major Electron toolchains (forge, rebuild, builder, etc.)

[pumano]

Looks like parcel can't working when I try to build for renderer and main processes (main is nodejs typescript app + createWindow, renderer is typescript frontend). I have typescript on client and server, but currently I have requirement to "pack" it into electron. I know it fully works with webpack. Reason why I choose parcel few months ago - you don't need to configure something. But in that case, looks like it is not working and I currently going to webpack.

If devs from parcel implement proper bundling for electron renderer and main, it will be good, and I can migrate to parcel back.

[andreiglingeanu]

This is what I've came with too. I'll fallback to something simpler until Parcel gets a native support for render and main bundling.

Thanks for all the work that goes into this, it is really appreciated!

[lesmoutonssauvages]

Hi @devongovett , First thanks for your work on parcel 2.
It's the bundler i was waiting for. Im using parcel 1 and really like working whith it in an electron project.
Just tested the alpha3, it builds my sources so fast! Congrats!
Also just a questions about something im waiting for, have you plan to manage multiples BrowserWindows with their own "chunks" ?

Thanks again

[millimoose]

Vaguely necro comment, but since this is open and I’ve been eyeing Parcel as a Webpack alternative for an Electron project facing issues cited here:

Disabling bundling node_modules might be the safe and simple solution, but I’d be disappointed if that ended up being settled on; in my current scenario doing so bloated my final build by about 50% compared to bundling; even assuming most of the rest is the fixed cost imposed by Electron (i.e. my actual app’s code minifying next to nothing) that’s already not negligible overhead. It’s rude to have clients download completely unused code and obnoxious to have to wait for it to be copied around for every single build.

Right now my webpack + electron-builder configuration might be a weird pile of that made me lose sanity points while putting them together; but crucially it does work despite the mess that having two different binding loaders, two versions of one of them with a breaking change in naming convention, and whatever the hell gRPC does. (Which, for instance, didn’t work with ncc.)

My point being: bundling Node/Electron Main, with node_modules, and with native extensions may be off the beaten path, but it’s not intractable; my hunch is the Electron community could benefit from charting these waters and figuring out best practices that one could point native library authors towards to make their projects compatible, and it seems to me Parcel may be the sort of project for which “implementing an opinionated-ish approach that Just Works” is appropriate. (Because, say, maybe awesome-node-loader would have helped me save some of the headaches mentioned; but due to the atomized nature of the JS ecosystem, I just had no idea it exists until now.)

[Inateno]

Hello, just to put my 2 cents here.

I love Parcel and didn't wanted to rollback our full build stack just for making "native versions", so for now I use hacky ways to solve my issues.

About bundling "node-modules" or not, actually it does, and it specially tries to wrap all of them.
Here is a screenshot of the error I'm actually facing with a "standard build command"

The best/easiest I think, would be to tell to parcel to just not try to go over a specific entry point we have in the HTML file (to split native from non-native stuff, that's how we do so far), no renaming no parsing, nothing.

My fix for now is to just give to parcel the JS file we want to be build, and the others files are moved with a DIY script (just cp things).
Also loading that JS file in electron while using the "development" mode brake of course the parcel hot reload, but it's fine for me just do a ctrl+r do the work so far.

Can't wait to use those new tools, would make our life easier for sure. Keep up the good work, every day is a better day since I use Parcel :)

[beautyfree]

Any news?

[aminya]

Any news?

In parcel@nightly, #5049 is fixed that added a bunch of features related to JSRuntime to the Electron environment (like using dynamic imports). If there is anything specific, a separate issue with reproducing steps or repository will allow the contributors to fix things faster.

[pie6k]

Is there any status update for this?

[malept]

(Note: I know about as much as everyone else who isn't a Parcel maintainer, that is to say, nothing.)

I'm generally going to assume the answer is "no" until one of the Parcel maintainers posts in this issue otherwise. It's still on the roadmap (#7345) but it's not one of the immediate items.

Probably the best thing you can do at this point is 👍 the issue summary so the Parcel maintainers can know how much of the community wants this feature. Adding "any news", "status update", or "+1" comments, on the other hand, is not particularly constructive.

---

That being said, I just want to reiterate to the Parcel maintainers that once y'all have the bandwidth to take on this feature (and the related Node.js one), please feel free to reach out to the Electron maintainers. I, at least, would be more than happy to advise on the best way to have a bundler handle native modules in a first party manner.

[ghost]

How is it going?

[briandk]

I'm also curious how the Parcel/Electron integration is going

[pie6k]

It's going bad. I was not able to figure it out and used vite instead

[TechQuery]

I build an Electron scaffold with Parcel 2: idea2app/Electron-Parcel-PNPM.tsx#1