Transition from Watchify to Webpack

[Anoninnyc]

Please describe the issue (What happens? What do you expect?)

We introduced Watchify in order to cut down the built time from ~30 seconds to ~4 seconds. This made the iterations much quicker, which was a benefit to developers. But we had to remove the watchify module because since introduction of watchify module, ~ every other time a change is saved, its omitted in hot-reloaded version of app.

Steps to reproduce the problem (1, 2, 3...), including links

1.Make a change. hit save. wait for page to reload.
2. repeat
(notice when you're changes are and are not reflected on screen)

[Anoninnyc]

@DaleMcGrew Where did you see that this was a mac specific issue?

[DaleMcGrew]

Useful posts I found (I did not do extensive digging):
https://github.com/substack/watchify#rebuilds-on-os-x-never-trigger
browserify/watchify#224

I saw a couple of posts that recommended passing in the ", { poll: true }" option.

[Anoninnyc]

@DaleMcGrew Thanks! I'll check it out.

[Anoninnyc]

@DaleMcGrew I think I fixed it- pls see Pr

[Anoninnyc]

@DaleMcGrew Can we close this? I think integration of my last PR fixed this, no?

[DaleMcGrew]

Hello @Anoninnyc, the PR you submitted to fix this did not work. I would recommend we pass this to a developer using a Mac so it can be tested locally. Thank you for your work so far on this!

[Anoninnyc]

@DaleMcGrew Do you think this X-machine compatibility issue could be solved by Docker? Just putting it out there (never used Docker myself, and maybe it would be overkill if only using to solve this one issue).

[DaleMcGrew]

Hi @Anoninnyc, you are welcome to try Docker. We discovered that the watchify/browserify problem we saw on Mac also existed for our Linux developers.

[pertrai1]

@DaleMcGrew Long time no talk in this project for myself. I am wondering why there was the introduction of watchify? Would help to put into context the exact use case of the need for it in this project.

[DaleMcGrew]

Great to hear from you Rob @pertrai1 ! I just updated the description with more details: "We introduced Watchify in order to cut down the built time from ~30 seconds to ~4 seconds. This made the iterations much quicker, which was a benefit to developers. But we had to remove the watchify module because since introduction of watchify module, ~ every other time a change is saved, its omitted in hot-reloaded version of app."

[pertrai1]

@DaleMcGrew might be time to change over to Webpack.

[DaleMcGrew]

Hello @bharathdn and @kuwood, do you have time to collaborate on this upgrade this week?

[n0f3]

are you guys switching over the entire build to webpack?

[DaleMcGrew]

Closed until we can return to this.

[user512]

We started investigating this issue again to speed up the build process and we looked into Webpack and Parcel.
We think that either option would be an improvement.

Here is the comparison:
Both improve build speed and offer intelligent configuration and remove the custom wiring that needs to be done with Gulp.
Webpack is more robust because it's more mature. However Parcel is better designed to be easier to configure.

We'd like to recommend Parcel because of the ease of use with the caveat that it's less mature and therefore may not support certain unique customization.

@DaleMcGrew Does the team have a preference of Webpack or Parcel? We can start working on this issue.

[DaleMcGrew]

Thank you @user512! My biggest concern about Parcel is that it doesn't seem to end up with a single bundle file (which our deployment is set up around). If I am mistaken, and Parcel can bundle into a single file, that would be interesting. Otherwise my preference would be Webpack 4.

[SailingSteve]

Wow Parcel sounds amazing. It has 29,000 stars and 43,000 downloads a week on npm. (vs 6M/week for webpack)

I used Webpack a few years ago and it was both great and tragic, but a few years of improvement could mean a world of difference.

Gulp is slow and kind of dumb, few of those 30 seconds add any value.

I'd lean toward trying Parcel - Could it be added in parallel with our existing gulp setup, so we could use both for a while?

[utsab]

That's an interesting idea @SailingSteve to try to run both gulp and parcel in parallel. We can investigate that.

Based on this article, it looks the code splitting feature where Parcel splits the output into multiple bundles can be triggered by using the import() command as follows:

import('./path/to/module').then(function(page)

When Parcel sees the promise structure it knows that it's possible to split up the bundle into multiple files which can be loaded at different times. If my understanding is correct, as long as we do not use the important().then() promise structure, then Parcel will not split the output, and it will end up as a single bundle.

@DaleMcGrew do you think this would satisfy your concern around the bundle splitting?

[SailingSteve]

Parcel is pretty easy to get going for a new project, but the startup fails when it gets to our scss files. It seems that node-sass doesn't handle concurrent threads well (deep in its 'c' implementation), and Parcel gets much of its boost by using multiple cores simultaneously. The Parcel team recommends going with 'dart-sass'..

our sass implementation would need to be reworked. Dart-sass has you import the sass files into the pages where they are used, which is some work, but hopefully that would avoid our current issue where all sass styles are global.

[DaleMcGrew]

Hello @utsab and @SailingSteve would you be able to work together to get to a new version of the Webpack bundle generator? Steve would like to use a new PDF display package that requires webpack.

[utsab]

Hi @DaleMcGrew, I will follow up with @SailingSteve. Making the new webpack bundle generator work with Cordova has been a challenge so far. I have been meaning to reach out to Steve in any case to get his insight on this.

[DaleMcGrew]

Hi @utsab, thank you for the update. You might want to create a WebApp pull request (merged with all of the most recent changes) that @SailingSteve can review prior to your conversation.

[utsab]

@SailingSteve I believe the the new webpack work is ready to be tested in Cordova. Would you be up for testing it on your side and giving us your feedback?

We successfully launched the app in Cordova (using the webpack generated bundle.js) following the Xcode ios instructions, and the app correctly ran in ios simulator.

Here are the instructions you can follow for running the app locally in Xcode:

1. Clone my fork
2. Check out the webpack branch
3. Install the new dependencies

npm install

4. clear out any old build files (generated by the old build process)

rm -rf build

5. Create the production build files using the new build process

npm run webpack-build

This should generate a new bundle.js inside the build directory.

7. To test in cordova, I assume you have already set up the Cordova repository based on the instructions. You must rewire the symlink for bundle.js:

cd WeVoteCordova
cd www
ln -s /Users/your-username/MyProjects/WebApp/build/bundle.js bundle.js

8. Delete any old symlinks

cd WeVoteCordova/platforms/ios/www
rm -rf css
rm -rf fonts
rm -rf img
rm -rf javascript

Note that you should not attempt to recreate these symlinks, because the corresponding directories in WebApp no longer exist.

9. Run as usual in the Xcode simulator

10. If you'd like to also run the app in the browser,

cd WebApp
npm run webpack-start

[DaleMcGrew]

It is my pleasure to mark this Issue as fully resolved and running on our live production servers. Many thanks to @user512, @utsab and @SailingSteve for making this transition happen. This new Webpack build system significantly increases development speed. Thank you!