PoC integration between browser and AMD modules #1533

izaera · 2021-10-04T11:57:27Z

This is a quick and dirty test to show how it is possible to interop browser and AMD modules.

I initially started it to decide if we should get rid of bundler 3, or keep it for further refinement, but given that this simple PoC can give rise to new ideas, I wanted to share it as a draft.

I explain in the comments what each thing does.

Note that I'm not proposing this as the final solution, but showing that we may be nearer to using browser modules than we think.

We would still need interoperation the other way round (use browser modules from AMD ones) but in case we decided to support that, it would be a matter of injecting the browser modules into the AMD loader (i.e: it would need a new AMD loader). In any case, another legitimate option would be not supporting it, I think (the same way we don't provide any too to loadAUI from AMD).

One last thing to note is that, in theory, we wouldn't need to rewrite any line of JS in portal to switch from AMD to browser modules, given that we are already using ES module syntax.

izaera · 2021-10-04T11:58:44Z

modules/apps/frontend-taglib/frontend-taglib-clay-sample-web/package.json

@@ -2,7 +2,6 @@
 	"name": "frontend-taglib-clay-sample-web",
 	"private": true,
 	"scripts": {
-		"build": "liferay-npm-scripts build",


I'm turning off any build here because I don't want babel or the bundler to rewrite anything.

izaera · 2021-10-04T12:00:12Z

...d-taglib/frontend-taglib-clay-sample-web/src/main/resources/META-INF/resources/_amd/react.js

+	);
+});
+
+export default await promise;


This is a special module that would be created by the build system automagically. It provides a bridge from browser modules to AMD ones, i.e, lets index.js import react as if it was local to the project, but then it is diverted to the AMD loader transparently.

The export await syntax is pure gold because it lets interchange of sync and async imports/exports without any need to change the consumer code.

browser modules to AMD ones

Any idea what the cost would be to migrate our AMD modules into browser modules? Or is this just a silly thought?

In theory we would be able without touching anything in the source code. We would need a tool and some configuration files to specify where to find things.

However, we would need to do a PoC for that too, because the devil is in the details, and it's not the first time (in this context) we think something is easy, then we discover it is much more complex or even impossible (see what happened with webpack federation 😞 ).

izaera · 2021-10-04T12:01:58Z

...ontend-taglib/frontend-taglib-clay-sample-web/src/main/resources/META-INF/resources/index.js

@@ -0,0 +1,5 @@
+import holi from './js/holi.js';


Import local module holi.js synchronously

izaera · 2021-10-04T12:03:40Z

...ontend-taglib/frontend-taglib-clay-sample-web/src/main/resources/META-INF/resources/index.js

@@ -0,0 +1,5 @@
+import holi from './js/holi.js';
+import react from './_amd/react.js';


Import third party module react synchronously.

Note that this is a trick for the PoC, but in the original source code we would have something like import react from 'react'; and then a configuration to tell the bundler to remap that react to something (in this case an AMD module, in other case it could be a local browser module inside node_modules or in any other OSGi bundle).

It looks interesting :)

Mixing amd and esm modules will not cause a race condition?

Does the browser do these imports sequentially?

If not, I think the amd loader can be slower(I'm guessing, yes) and the browser can load assets in esm faster and a dependency of an esm can be the currently-loading amd loader

There's no race condition because we would use export await in the AMD-browser bridge module.

Does the browser do these imports sequentially?

I haven't checked it, but I guess it doesn't. It can simply create the dep graph, fetch the files, and visit the graph in parallel (respecting deps, of course) instantiating and resolving modules. Much in the same way we do in AMD loader.

However, I haven't done any performance test yet. And I'm not 100% sure if deploying npm packages without bundling will be performant, because I don't know if the browser is going to download the modules one by one or in parallel (using HTTP 2 or something like that). This is something we need to explore before committing to any design.

izaera · 2021-10-04T12:04:27Z

...tend-taglib/frontend-taglib-clay-sample-web/src/main/resources/META-INF/resources/js/holi.js

@@ -0,0 +1,3 @@
+export default function holi() {


This would be a regular browser module.

Note that this is put inside the JAR verbatim: we don't need to build anything.

izaera · 2021-10-04T12:06:18Z

...ontend-taglib/frontend-taglib-clay-sample-web/src/main/resources/META-INF/resources/view.jsp

This is how you bootstrap a browser module.

It is the equivalent of <aui:script require="<%= npmResolvedPackageName + ...%>">

In the future, we could make a new tag to avoid using <aui:script> to be able to make our own magic before printing the <script> tag to the HTML, but it would be straightforward, most probably.

to make our own magic before printing

By magic you mean pointing to a valid resolved path(with version of the module and a path to /o/) like the generated NPMResolver ones?

By magic I meant "anything we want to do with it, now or in the future". So I'm proposing to have our own taglib just for the sake of having an extra layer between the JSP and the actual <script> tag.

I don't know if in this browser modules solution we would need to have version numbers or not 🤔 In this case, the canonical names of modules contain the Web-ContextPath of the OSGi bundle containing them, and that has to be unique per OSGi bundle version, so maybe we can switch from npm versions to OSGi locations to designate which npm modules we want to use. It's something that we should investigate.

Most probably, we would want to do things like:

<new:script-tag import="/index.js"/> to import local (contained in the OSGi bundle contanining the JSP's web context) modules.

<new:script-tag import="react" context="frontend-js-react-web" /> to import a module contained in a different web context (if we wanted to allow for it).

Or, altenatively, we could have some way to configure mappings from logical package names to web context + name which are then applied when building. So, doing something like:

<new:script-tag import="react" /> in your code, then having a file where it says that react has to be loaded from frontend-js-react-web

Nothing is yet decided, but those are the possibilities I have currently in mind...

This is very interesting @izaera, I also think that having this tag would probably make things easier. Can't wait to see the final result

Can't wait to see the final result

Well, you will have too, because this is only a brainstorming, nothing that we have decided or planned yet 😅

kresimir-coko · 2021-10-04T12:46:21Z

It looks simple enough @izaera

bryceosterhaus

Overall this is pretty neat! I expected much more of a headache when interacting between the two but this doesn't seem so bad.

jbalsas · 2021-10-06T09:12:22Z

Hey @izaera, I think we should proceed a bit further to see if this actually plays out.

Import maps

I think this should be pretty straightforward. Import maps are shipped by default on Chrome and can be polyfilled using es-module-shims, so that should give us enough support to give them a try. From there, we'd want to check:

Can we easily generate an aggregated import map with all of our modules? (I'd say this is the easiest thing, akin to what we do with the config.js files for AUI loader config)
How smart can we be about the information we put out there?
How big would the import map become?
Will this pose any kind of security concern?
Could we define both main and secondary imports? Could we for example do import { navigate } from 'frontend-js-web/navigate so we don't have to load all of frontend-js-web via the main module if we just want navigate? How would we define what each module exports so we don't open up the map to /anything?

Additionally, there's quite a big movement in the Rails community about this, so I'd suggest you check some of these for context and inspiration:

React

React is a key part of our ecosystem now, so we would need to figure out:

How would we actually import some of our key external dependencies like react? There's es-react out there and the React team is also talking about how to Formalize top-level ES exports
, so it would be good to check.
How would we support JSX? It's one thing to import React from 'react' and a completely different one to use JSX which would indeed need a transpilation step. Could it produce es modules?

I'm sure some other questions will arise, but these are the first that come to mind, so I'd love to see some experimentation in this area. I can work on that if you're busy with something else as well, so just let me know how can I help!

izaera · 2021-10-06T10:37:40Z

Interesting links to read related to this PR

(I will maintain a list of interesting articles I find, related to this feature, in this comment)

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Modules (crash course on browser modules 😅 )
https://v8.dev/features/modules#performance (a recommendation to keep bundling even when using browser modules; caveat: from 2018, situation may have changed)
https://github.com/tipalti/webpack-import-map-plugin (a webpack plugin to generate import maps)

izaera · 2021-12-15T08:05:44Z

I'm going to close this, since all information and feedback gathered here has already been used in LPS-143013.

Special thanks to @jbalsas for the very useful links regarding import maps 😘

izaera · 2021-12-15T08:06:25Z

PoC integration between browser and AMD modules

f54d83d

izaera marked this pull request as draft October 4, 2021 11:57

Repository owner deleted a comment from liferay-continuous-integration Oct 4, 2021

izaera added the s-devexp Pull requests to be reviewed by Dev Experience squad label Oct 4, 2021

liferay-continuous-integration added the ci:test:sf - pending label Oct 4, 2021

izaera commented Oct 4, 2021

View reviewed changes

liferay-continuous-integration added ci:test:relevant - pending ci:test:stable - pending ci:test:sf - failure and removed ci:test:sf - pending labels Oct 4, 2021

izaera commented Oct 4, 2021

View reviewed changes

Repository owner deleted a comment from liferay-continuous-integration Oct 4, 2021

izaera mentioned this pull request Oct 4, 2021

Proposal for stabilization plan liferay/liferay-frontend-projects#668

Closed

6 tasks

bryceosterhaus reviewed Oct 4, 2021

View reviewed changes

izaera mentioned this pull request Oct 7, 2021

feat: Liferay Dev Server liferay/liferay-frontend-projects#635

Merged

izaera mentioned this pull request Nov 8, 2021

Next steps for JS Toolkit and target platforms liferay/liferay-frontend-projects#740

Closed

izaera closed this Dec 15, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

PoC integration between browser and AMD modules #1533

PoC integration between browser and AMD modules #1533

izaera commented Oct 4, 2021

izaera Oct 4, 2021

izaera Oct 4, 2021

izaera Oct 4, 2021

bryceosterhaus Oct 4, 2021

izaera Oct 5, 2021 •

edited

izaera Oct 4, 2021

izaera Oct 4, 2021

diegonvs Oct 4, 2021

izaera Oct 5, 2021

izaera Oct 5, 2021

izaera Oct 4, 2021

izaera Oct 4, 2021

diegonvs Oct 4, 2021

izaera Oct 5, 2021 •

edited

julien Oct 5, 2021 •

edited

izaera Oct 6, 2021

julien Oct 6, 2021

kresimir-coko commented Oct 4, 2021

bryceosterhaus left a comment

jbalsas commented Oct 6, 2021

izaera commented Oct 6, 2021 •

edited

izaera commented Dec 15, 2021

izaera commented Dec 15, 2021

		@@ -0,0 +1,5 @@
		import holi from './js/holi.js';
		import react from './_amd/react.js';

PoC integration between browser and AMD modules #1533

PoC integration between browser and AMD modules #1533

Conversation

izaera commented Oct 4, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

izaera Oct 5, 2021 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

izaera Oct 5, 2021 • edited

Choose a reason for hiding this comment

julien Oct 5, 2021 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kresimir-coko commented Oct 4, 2021

bryceosterhaus left a comment

Choose a reason for hiding this comment

jbalsas commented Oct 6, 2021

Import maps

React

izaera commented Oct 6, 2021 • edited

Interesting links to read related to this PR

izaera commented Dec 15, 2021

izaera commented Dec 15, 2021

izaera Oct 5, 2021 •

edited

izaera Oct 5, 2021 •

edited

julien Oct 5, 2021 •

edited

izaera commented Oct 6, 2021 •

edited