ES6 modules don't work in 2.0.0-beta.1

[jarek-foksa]

• Electron version: 2.0.0-beta.1
• Operating system: macOS 10.13.3

<script src="script.js" type="module"></script> should execute the code in script.js file. Instead, the following error is thrown:

Failed to load module script: The server responded with a non-JavaScript MIME type of "". Strict MIME type checking is enforced for module scripts per HTML spec.

NW.js used to be affected by exactly the same issue: nwjs/nw.js#6106

[codebytere]

this is a chromium 61 issue; i'll see if there's a patch i can cherry-pick into libcc

[deepak1556]

Loading es6 modules over null origin (file:// scheme) is not supported by standard https://html.spec.whatwg.org/multipage/webappapis.html#fetch-a-single-module-script, we would like to follow similar behavior. At the moment we are setting CORS header for file:// to support service workers in electron, but this will be removed in the upcoming version. Closing this as expected behavior. Thanks!

[jarek-foksa]

@deepak1556 Do you mean that Electron will not support ES6 modules unless they are loaded from server?

[MarshallOfSound]

@jarek-foksa No, you can just use a custom protocol to provide your files (you should probably do this anyway, the file:// protocol doesn't follow the same rules as other protocols)

[SMotaal]

@MarshallOfSound can you elaborate a little on this please... A link on setting up a custom protocol in such a way that it would work would be really helpful.

[SMotaal]

Just to confirm the following (crude) test case works:

Assuming the conventional node requires like const {URL} = require('url')… per electron guides.

function createProtocol(name = 'app', {
  prefix = `${name}:///`, basepath = __dirname
} = {}) {
  protocol.registerStandardSchemes([name], { secure: true });
  app.on('ready', () => {
    protocol.registerBufferProtocol(name, (request, callback) => {
      const url = new URL(request.url.replace(/^.*?:[/]*/, prefix));
      const { pathname, hash, searchParams } = url;
      const filename = path.resolve(basepath, pathname.slice(1));
      const extension = path.extname(pathname);
      const scope = { request, url, pathname, filename, extension };

      try {
        const content = `${readFile(filename)}` || '';
        const data = scope.data = Buffer.from(content);
        const mimeType = scope.mimeType = mimeTypeFor(extension);
        callback({ mimeType, data });
      } catch (exception) {
        scope.exception = exception;
        console.error(exception, scope);
      }
    }, (error) => {
      if (error) console.error('Failed to register protocol')
    });
  });

  const mimeTypeFor = (extension) =>
    mimeTypeFor[`${/^.\w+$/.test(extension) && extension || ''}`.toLowerCase()];

  mimeTypeFor[''] =
    mimeTypeFor['.js'] =
    mimeTypeFor['.ts'] =
    mimeTypeFor['.mjs'] = 'text/javascript',
    mimeTypeFor['.html'] =
    mimeTypeFor['.htm'] = 'text/html',
    mimeTypeFor['.json'] = 'application/json',
    mimeTypeFor['.css'] = 'text/css',
    mimeTypeFor['.svg'] = 'application/svg+xml';
}

[jarek-foksa]

Could this protocol thing be somehow abstracted away by Electron? Most developers (including me) don't even know what custom protocols are and I would expect a core language feature such as modules to work out of the box.

[SMotaal]

@jarek-foksa After a bit of testing, I think @deepak1556's view on keeping with standards re file: is absolutely right. Though this does complicate things a little for prototyping, the idea that you can load content from a site inside a BrowserView means that blessing the file: protocol simply opens up a new potential for vulnerabilities.

That said, I think for non-web BrowserViews with node integration, it can be a lot more useful to consider NodeJS's new Loader.

Here is a quick example with a preload script:

// preload.js
const { NativeModule, mainModule, mainModule: { filename } } = process;

{ /* Loader */

  const module = mainModule;

  const base = module.base =
    `file://${filename.replace(/[^/]*([#?].*)?$/, '')}`;

  const Loader = NativeModule.require('internal/loader/Loader');

  const AsyncModule = 'async module'; // [Symbol.for('async module')];

  class NodeLoader extends Loader {
    async import(specifier, parentURL = this.base) {
      const job = await this.getModuleJob(specifier, parentURL);
      const module = await (job[AsyncModule] || (job[AsyncModule] = job.run()));
      return module.namespace();
    }
  }

  const loader = module.loader = new NodeLoader(base);

  module.import = (specifier, referrer) =>
    loader.import(specifier, referrer);
}

And to use it in your page:

<script>
  { /* Load a "valid" Web Component */
    const { log, error } = console;
    const specifier = './tests/static/custom-element.mjs';
    (async () => {
      try {
        const a = await module.import(specifier); // .catch(error);
        const b = await module.import(specifier);
        log('import(%o) => [#1 %O] === [#2 %O] %O', specifier, a, b, a === b);
      } catch (exception) {
        error(exception);
      }
    })();
  }
</script>

Node's loader is a far more flexible option for desktop applications in my opinion. It has a learning curve, and it is technically still experimental, but the things it can do and how fast it can load (and even transpile on-demand) stuff puts it in a whole new category of loaders (if you are patient enough to give it a shot and figure it out). Just keep in mind that "by default" it expects modules to be awkwardly .mjs because someone apparently invented something called CommonJS modules with .js 😉 before JavaScript so that extension was taken.

Future:

Once Electron hits Chromium 63 and NodeJS 10, it will even be possible to bind the loader to the BrowserView's dynamic import and literally defer all import(…) statements (but not static imports) to the loader.

[jarek-foksa]

@SMotaal Thanks for providing all the examples. In your first example (with protocols), how do you load the app page? After registering the app: protocol I was initially trying to load the page as follows, but the ES6 modules inside the page would still fail to load:

appWindow.loadURL(`file://${__dirname}/index.html`);

After that I switched to the code below, but it would just load a blank/stub page instead of index.html:

appWindow.loadURL(`app://index.html`);

[SMotaal]

@jarek-foksa I worked on a gist, let's continue this discussion there:
https://gist.github.com/smotaal/f1e6dbb5c0420bfd585874bd29f11c43

[jarrodek]

I am trying to run electron app with Polymer 3 modules. I have difficulties getting it right.

With given gist I can make import modules from the HTML page but imported modules can't resolve their dependencies.

This is an example setup (from Polymer documentation):

(demo/demo.html and main is in demo/main.js)

<script src="../node_modules/@webcomponents/webcomponentsjs/webcomponents-loader.js"></script>
<script type="module">
  // case 1
  import '../node_modules/@polymer/paper-styles/typography.js';
  // case 2
  import 'app:@polymer/paper-styles/shadow.js';
</script>

In case 1 it throws the error with mime type check.

In case 2 the create protocol function fails to load the module.
It says the request.url is app://polymer/paper-styles/typography.js which is already not true because it is missing @ of Polymer scope. Eventually it resolves the pathname to ./polymer/paper-styles/typography.js.
I never work with custom protocols so I am not sure what to do next. Could you advise?

[dravenk]

I tried running polymer 3 on electron 3, but failed. I saw the release log. Electron 3 using chrome 66, which is much higher than 61. Why is this problem?

[SMotaal]

@jarrodek sorry for not seeing this before was it resolved?

Unfortunately I have pulled back on my polymer experiments a bit, but in a more general sense, can you guys try troubleshooting (in your current setup).

First the try following try in console, if in doubt, try them in a script tag (not module).

• import( … ).then(console.log, console.warn) for each of:

  • the module that you imported
  • the resolved path to one or two of that module’s own dependencies

• also do the same with fetch( … then … etc.

• If it works at least in script, add type=module

• If it works, save that to a separate file right next to the page (like import-test.js) and add src=./import-test.js to the now empty script tag

• try both with and without type=model.

• finally, rewrite those to static imports import * as x from ' … ';

• let me know

[jarrodek]

@SMotaal It's not about polymer itself but how web components works. Polymer just adds some sugar on top of the spec.

My problem is about making it work when imported module has more imports. With the custom protocol I cannot properly resolve those dependencies. Or at least I am not sure how to do it properly.

I was working on a demo app some time ago. My main app still uses HTML imports that doesn't use modules imports. However in the beginning of the next year I would probably switch to module imports. When I do this I would like to know how to do it properly.

I may try to do this demo app again this weekend. If I won't be able to then I will try in about 2 weeks as I am going to vacations next week.

Thanks

[SMotaal]

@jarrodek I don't think it is worth the effort, honestly, given the state modules are in for the next couple of months I think it is best to focus on fixing the odd issue in something working well otherwise. I know for a fact that Electron awaits Node awaits TC39 (relevant proposals this week).

If you have a way to demo this quickly, and only if it helps you and not simply pull you into more experimental dilemmas, I can try (always find those interesting for some reason). But, please, don't waste your time, we are almost across the threshold — after ~4 years of eagerly motivated efforts wasted due to implementation delays maybe for good reasons.

[SMotaal]

A trick to resolve modules from relative specifiers:

HTML

<script>
  { // This block scope simply prevents making resolve a global constant

    // Classic script tags can use window.location as the referrer
    const resolve = (specifier, referrer = location)  => `${new URL(specifier, referrer)}`;

    // Resolves the parent directory of this page
    console.log(resolve('.')); 
    // Resolves "./index.js" relative to this page
    console.log(resolve('./index.js')); 
    // Resolves "./modules/module-a/index.js" relative to this page
    console.log(resolve('./modules/module-a/index.js')); 
    // Resolves "./modules/module-a/submodule-b/index.js" relative to this page
    console.log(resolve('./submodule-b/index.js', resolve('./modules/module-a/index.js')));

  }
</script>

<script type=module>

  // Module script tags can use import.meta.url (if in doubt you can just use window.location)
  const resolve = (specifier, referrer = import.meta.url || location)  => `${new URL(specifier, referrer)}`;

  // Resolves the parent directory of this page
  console.log(resolve('.')); 
  // Resolves "./index.js" relative to this page
  console.log(resolve('./index.js')); 
  // Resolves "./modules/module-a/index.js" relative to this page
  console.log(resolve('./modules/module-a/index.js')); 
  // Resolves "./modules/module-a/submodule-b/index.js" relative to this page
  console.log(resolve('./submodule-b/index.js', resolve('./modules/module-a/index.js')));

</script>

ES Modules Only

<script type=module src="./path/to/the/module/that/resolves/another.js"></script>

// Module script tags can use import.meta.url (never use window.location here)
const resolve = (specifier, referrer = import.meta.url)  => `${new URL(specifier, referrer)}`;

// Resolves the parent directory of this module
console.log(resolve('.')); 
// Resolves "./index.js" relative to this module
console.log(resolve('./index.js')); 
// Resolves "./modules/module-a/index.js" relative to this module
console.log(resolve('./modules/module-a/index.js')); 
// Resolves "./modules/module-a/submodule-b/index.js" relative to this module
console.log(resolve('./submodule-b/index.js', resolve('./modules/module-a/index.js')));

export {resolve}

Trick to help debug how modules would resolve

<script type=module>

  import { resolve } from './path/to/the/module/that/resolves/another.js';

  // Resolves the parent directory of the module NOT this page
  console.log(resolve('.'));

  // So now we can check where that module resolves module-a from exactly
  console.log(resolve('../../../../../../modules/module-a/index.js'));

</script>

[jarrodek]

Thanks for the example @SMotaal. I will experiment with it.

[dsanders11]

I don't think it is worth the effort, honestly, given the state modules are in for the next couple of months...But, please, don't waste your time, we are almost across the threshold — after ~4 years of eagerly motivated efforts wasted due to implementation delays maybe for good reasons.

@SMotaal, could you elaborate on that or provide a link where I could read about it? It sounds like you're saying it'll soon (relatively?) be easier to use ES6 modules in Electron?

[SMotaal]

@dsanders11 from my own perspective as a user, I know for a fact that ES module support in Electron overlaps with support in Chromium and support in Node.

For over a year, support through Chromium (ie script type module) was available. But personally, I have not used it in a while, and when I did, it was clearly browser-centric (may have changed).

For the Node.js part, which only offers an experimental implementation via flags. This part is not meant for production, comes with frequent breaking changes, and is expected to be replaced with a separate implementation all together once the modules team designs a new module system based on gained insights from the experimental one.

A while back I tried to use experimental-modules in Electron, and I ran into some compatibility issues which were premature to address in either Node.js or Electron. Fixing it was not hard, however, it became clear to me that experimental patching across projects is far more complicated, and instead it makes sense to factor such findings into the design of the new module system instead.

Obviously there is a lot going on there, which takes place at nodejs/modules. Just a heads up, it can be very hard to track discussion threads there, as a lot of discussions involve a lot of context, brainstorming and so on. The actual design is now in Phase 2, and there is a lot more work that needs to take place to bring it to a level that compares with and goes beyond the features in experimental before they land in Node.js.

Until then, it is hard to predict how this work will integrate in Electron, which I am sure is something that Electron folks are keeping track of and are best able to elaborate on.

[stefaneidelloth]

Related issue: #13402

[jarrodek]

I have finally figured out what is happening with Polymer elements. The thing is that all paper-* and iron-* elements have node like paths when using imports (starts with @polymer/ for example) instead of using relative paths. When trying to load any of Google's components into Electron app it fails giving the error message I described above. In the development process you should use library like polyserve to transform the components to final and working form and build the application the Polymer way before publishing it.
It is inconvenient but I guess the Polymer team had good reason to do this. I am upgrading my set of components and after upgrading just a few I run into problem with dependency management. I ended up doing the same thing as the Polymer team did (I think).

So to work with some libraries based on web components (especially Google's) you have to register a protocol for the components and use tools to transform the output to correct syntax (Polymer CLI, polyserve, own solution) or import files directly from node_modules but using polyserve at the same time. I had to do similar thing recently for another (not Electron) app: https://github.com/advanced-rest-client/api-components-apps/blob/master/ci-status-app/app.js#L40
In first step I run polymer serve command to run the UI server and then a proxy module when running the application. This way when requesting the UI in the application the request is finally proxied to Polymer transformed components.

[Lonniebiz]

This is disappointing. Electron should abstract away all of this custom protocol complexity and make ES Modules something that just works.

[jarrodek]

@Lonniebiz It's more than that. I do agree that ES modules should return proper content type headers so it can be executed without any additional setup. However authors tend to not to comply with ES spec and use node like paths when importing other modules. It is because they assume a some kind pre-processing in development. In electron you have to take care of it yourself and I am not sure if that's the role of Electron to abstract this.

[Lonniebiz]

@jarrodek If Chromium needs a server to serve ES modules, instead of the direct filesystem, then Electron should be a server (as far as Chromium is concerned).

[royvelich]

I have finally figured out what is happening with Polymer elements. The thing is that all paper-* and iron-* elements have node like paths when using imports (starts with @polymer/ for example) instead of using relative paths. When trying to load any of Google's components into Electron app it fails giving the error message I described above. In the development process you should use library like polyserve to transform the components to final and working form and build the application the Polymer way before publishing it.
It is inconvenient but I guess the Polymer team had good reason to do this. I am upgrading my set of components and after upgrading just a few I run into problem with dependency management. I ended up doing the same thing as the Polymer team did (I think).

So to work with some libraries based on web components (especially Google's) you have to register a protocol for the components and use tools to transform the output to correct syntax (Polymer CLI, polyserve, own solution) or import files directly from node_modules but using polyserve at the same time. I had to do similar thing recently for another (not Electron) app: https://github.com/advanced-rest-client/api-components-apps/blob/master/ci-status-app/app.js#L40
In first step I run polymer serve command to run the UI server and then a proxy module when running the application. This way when requesting the UI in the application the request is finally proxied to Polymer transformed components.

I just use pika-web in order to transform paths - a great solution for electron apps that use web components. it saves the hassle of performing a build step every time you run your app. However, the protocol hack is still needed.

https://spectrum.chat/pika/pika-web/pika-web-and-web-components~c2647922-d8d7-457d-993f-0dd38198a749

[johannesjo]

I agree with what @Lonniebiz says. There are so many unresolved issues related to this (service worker, es6 modules, security), that it would make things just much easier. It wouldn't even need to be the default behavior. Providing a standard convenient way for new projects to do so, would be enough.

[samwhaleIV]

My workaround is to host the web content using express then load it through a localhost URL in the Electron app. Expose require() (or just simply the node module you need) to the globalThis object in a normal script. Load your root type="module" script and access any node integration from before through globalThis, with the support of ES6 modules.

The solution has glaring security issues, but given as its an application for my own, personal use, I'm not too worried about it. I just needed a web app with file system integration and don't want to have to deal with sending post requests to the server with large amounts of file data.

<script>globalThis.require = require;</script>

<script type="module">

import something from "../somewhere-else.js";
const fs = require("fs");

</script>