Update to webpack v5

[cw789]

Revision

• Including v1.x version annotation within package.json
• Again "source-map" instead of "eval-cheap-source-map" for now

Background "source-map":
Terser plugin doesn't work with "eval-cheap-module-source-map".

Suggestion for additional PR (if it will improve build time):

const webpack = require('webpack');
...
plugins: [
  new webpack.EvalSourceMapDevToolPlugin({
     exclude: ['./vendor/**/*.js']
   })
]
devtool: false

---

Initial

Including update to webpack 5.
Some things are probably opinionated.

• Obvious "fixed" versions within package.json (real version is anyway in package-lock.json)
• No license field in package.json as not defined it must be MIT ...

Edit:

• Replaced optimize-css-assets-webpack-plugin with css-minimizer-webpack-plugin (Thx @optikfluffel)
• source-map instead of eval-cheap-source-map

Has open Issue (warning in CLI, but seem to work anyway): webpack/webpack-cli#1918
Concerns discussion: https://elixirforum.com/t/upgrade-to-webpack-5/35010

Resolve #4020

[optikfluffel]

optimize-css-assets-webpack-plugin isn't working with webpack 5 and has to be replaced:

⚠️ For webpack v5 or above please use css-minimizer-webpack-plugin instead.

Also: is there a reason you removed the parallel: true option from the TerserPlugin config?

[cw789]

optimize-css-assets-webpack-plugin isn't working with webpack 5 and has to be replaced:

⚠️ For webpack v5 or above please use css-minimizer-webpack-plugin instead.

Good catch - wasn't aware of that.

Also: is there a reason you removed the parallel: true option from the TerserPlugin config?

Not sure anymore - isn't it default to true?

[optikfluffel]

@cw789 you're right, parallel is true by default now

[jeregrine]

I have set this up locally and can confirm it woks. And I upgraded a legacy project. @chrismccord @josevalim

[josevalim]

Thank you @cw789! I removed the stats stuff because this is something we discussed in the past and we should probably have a separate discussion before moving forward. This looks good to be merged to me. By the time Phoenix v1.6 is out, webpack will probably be more stable and some plugins will have caught up.

[cw789]

Thanks everyone for reviewing.

I'm totally fine by removing the stats stuff within this PR. Maybe it's not bad to show people what's going on.
Even if I'm more focused in what is Phoenix doing (webpack should just do it's minimal necessary things and work).

By the way I also would like the same version syntax 1.x within the mix.exs - deps.
It makes it more obvious which range of versions you will get.
But that of course need to be also part of a different discussion.

[navinpeiris]

Live reloading seems to be broken after these changes. Might be my something in my config, but thought I'll let you know just in case.

[rockwood]

Live reloading seems to be broken after these changes. Might be my something in my config, but thought I'll let you know just in case.

This is an issue for for me as well. Live reload is triggering an event for every file copied into /priv/static. Previously, it would only trigger an event for the changed file. I have quite a few images and each time I change a js file, this is what I see in the logs:

[webpack-cli] Compilation starting...
[webpack-cli] Compilation finished
webpack output....
[debug] Live reload: priv/static/js/app.js
[debug] Live reload: priv/static/js/app.js
[debug] Live reload: priv/static/js/app.js
[debug] Live reload: priv/static/images/alert.svg
[debug] Live reload: priv/static/images/alert.svg
[debug] Live reload: priv/static/images/alert.svg
[debug] Live reload: priv/static/images/logo.png
[debug] Live reload: priv/static/images/logo.png
[debug] Live reload: priv/static/images/logo.png
...hundreds more

The browser is trying to refresh for each of those events.

Could this be an issue webpack's copy plugin? I assume it shouldn't be copying files that haven't changed.

[cw789]

Could this be an issue webpack's copy plugin? I assume it shouldn't be copying files that haven't changed.

Indeed this doesn't look like I would expect it to be too.
But I'm not aware of how the behavior regarding live-reload and CopyWebpackPlugin was before.

For now I have no solution.

[createdbypete]

Live reloading seems to be broken after these changes. Might be my something in my config, but thought I'll let you know just in case.

🤔 Interesting, I've not experienced this in my project so far but then I don't have a huge number of things in /static. Could this be reproduced in a repo we could take a look at?

[vortizhe]

I think webpack-cli 5.x isn't watching stdin any more so SIGINT signal do not close node process. The param --watch-options-stdin seems to do nothing.

After phoenix server closed, check ps aux for node or webpack processes.

I use a small watch.js file to monitor end and SIGINT signals.

#! /usr/bin/env node
const process = require('process');
const child_process = require('child_process');

const stdin = process.openStdin();
const child = child_process
  .spawn('node_modules/webpack/bin/webpack.js', 
    ['--config', 'webpack.dev.js'], 
    {detached: true}
  );

const endAll = () => {
  process.kill(-child.pid);
  process.exit(0);
};

child.stderr.on('data', (data) => console.error(data.toString('utf8')));
child.stdout.on('data', (data) => console.log(data.toString('utf8')));

stdin.on('end', endAll);
process.on('SIGINT', endAll);

[rockwood]

@vortizhe That's the issue I'm seeing. Thanks!

[createdbypete]

webpack/webpack-cli#885 seems related to this issue? This was reported way before v5 release though (May 2019) so I wonder what's changed in v5 to make this noticeable for you folks if they are related.

[cw789]

So I again invested bit of time in this.
I don't see anymore the problem of livereload beeing triggered for every static assets copied each time by the webpack copy plugin.

I only see livereload beeing triggered twice every time.
For compilation starting & finished.

[webpack-cli] watching files for updates...
[webpack-cli] Compilation starting...
[debug] Live reload: priv/static/js/app.js
[debug] Live reload: priv/static/js/app.js
[webpack-cli] Compilation finished

But I don't know if this behavior was the same before.
I also don't see this as blocking issue for now?

So from my side, this is ready to be merged.
Thanks all.

[cw789]

Hopefully my last changes:

• Missing --watch-options-stdin in umbrella installation dev config templates/phx_umbrella/apps/app_name_web/config/dev.exs
• Upgrade copy-webpack-plugin to ^7.x
• Upgrade topbar to ^1.x
• Update :watchers documentation in Endpoint with new webpack parameters

[cw789]

Are we going foreward with this (webpack v5)?
Or at least with #4146?

Background:
New Phoenix projects created with Node.js v15 seems to be in a "broken" state. Even when it's not Phoenixs fault (JS packages issues), it is probably frustrating for new devs (I also already felt apart this trouble).

If webpack means to much maintenance trouble it's probably the better way to slim down JS things (webpack?, Sass?, ...).
I know this is not the place for discussion. ;)

Snowpack seems promising:
https://www.richardtaylor.dev/articles/replacing-webpack-with-snowpack-in-a-phoenix-application

[josevalim]

@cw789 the plan is to go ahead with this but we need to balance between pushing to Webpack 5 too soon and that leading to frustration because the ecosystem did not catch up yet. I assume that, by the time we release a new Phoenix version, this will be safe, otherwise we will go with #4146. :)

If there is anything broken right now - not warnings but literally something does not work - then we should do the smallest PR possible to fix it and backport to the v1.5 branch. Otherwise these changes go to v1.6. :)

[chrismccord]

@cw789 as José said please let us know what issues you are having on the current setup. I just built a fresh 1.5 app and webpack has no problems.

[cw789]

Phoenix 1.5.7 wont work with Node.js v15.
≤ v14 seems to work...

@chrismccord Can you approve you've used Node.js ≤ v14?

I've not reproduced this issue, but seems so from reports in an issue & Elixir forum.

So the smallest PR is mentioning it will only work with Node.js ≤ v14 ...
Or by backporting the replacement of node-sass with sass and a minor upgrade of webpack v4.

[chrismccord]

Sidebar, I actually plan on removing node-sass so that is perhaps the fastest way to fix this for all users?

[chrismccord]

Note:

$ node -v
v15.2.1

[cw789]

Related threads:

• https://elixirforum.com/t/npm-installation-for-fresh-phoenixliveview-project-failing-in-windows-10/37170
• npm install fails on fresh project #4126

[chrismccord]

node-sass wouldn't be removed until 1.6, but the fact other folks might be having issues with webpack/node/et al is not new, so we need to tread carefully. I'm still seeing vast majority of folks have no issues on the current setup. Historically the js build tool ecosystem has been fragile, so moving to this PR is not guaranteed to be a panacea. We may cause as many issues as we solve for people, so we need to be sure that webpack v5 is golden across OS's, node versions, major packages, etc.

To be clear, we will be moving to webpack v5 as soon as makes sense, but I hope this gives insight into the slow roll to get there.

[cw789]

Thanks Chris. I understand the rollout tempo & totally agree.
I also know the difficulties, so I would recommend to keep the complexity within the dark side (JS ecosystem ;) always as low as possible...

& all that is not a critic. It's the cause why I came across to Elixir.
Thanks for all!

[cduruk]

Just tested this with node v15 and it does indeed work and fixes the current broken state of the new project. Thought definitely agree, I'm sure this would break something new for someone soon, so not too bullish either way (as new projects work fine with node v14)

[kivikakk]

I've been testing this for the last few days and it's been working okay, but I haven't gone too deep with it yet. (v14 less desirable for me as I'm on darwin/arm64 and afaik there's no native build for v14 there yet, only v15.)

[skovmand-wh]

It would be good to change the minimum required Node.js version in the docs, which currently state:

it's important to note that Phoenix requires version 5.0.0 or greater.

https://github.com/phoenixframework/phoenix/blob/master/guides/introduction/installation.md

[lessless]

Thanks a lot to everyone involved. I hope there are simpler solutions coming our way. Maybe snowpack.

Meanwhile, I had to set url: false for css-loader https://github.com/webpack-contrib/css-loader#url in order to fix image resolution

ERROR in ./css/app.scss
Module build failed (from ./node_modules/mini-css-extract-plugin/dist/loader.js):
ModuleBuildError: Module build failed (from ./node_modules/css-loader/dist/cjs.js):

in assets/css/app.scss

@import "./components/betting_table";

in assets/css/components/betting_table.scss

background-image: url("/images/main-illustration.svg");

and the image itself is located at assets/static/images/main-illustration.svg

my webpack config

const path = require('path');
const glob = require('glob');
const TerserPlugin = require("terser-webpack-plugin");
const MiniCssExtractPlugin = require('mini-css-extract-plugin');
const CssMinimizerPlugin = require('css-minimizer-webpack-plugin');
const CopyWebpackPlugin = require('copy-webpack-plugin');
const webpack = require('webpack');

module.exports = (env, options) => {
  const devMode = options.mode !== 'production';

  return {
    optimization: {
      minimize: devMode ? undefined : true,
      minimizer: [
        `...`,
        new TerserPlugin({  parallel: true }),
        new CssMinimizerPlugin(),
      ]
    },
    entry: {
      './js/app.js': glob.sync('./vendor/**/*.js').concat(['./js/app.js'])
    },
    output: {
      filename: 'app.js',
      path: path.resolve(__dirname, '../priv/static/js')
    },
    module: {
      rules: [
        {
          test: /\.js$/,
          exclude: /node_modules/,
          use: {
            loader: 'babel-loader'
          }
        },
        {
          test: /\.s?css$/,
          use: [
            {
              loader: MiniCssExtractPlugin.loader,
              options: {
                publicPath: '/',
              },
            },
            {
              loader: 'css-loader',
              options: {url: false},
            },
            'postcss-loader',
            'sass-loader',
          ]
        }
      ]
    },
    plugins: [
      new MiniCssExtractPlugin({ filename: '../css/app.css' }),
      new CopyWebpackPlugin({patterns: [{ from: 'static/', to: '../' }]}),
      new webpack.ProvidePlugin({
        $: "jQuery",
        jQuery: "jQuery",
        Modal: "exports-loader?Modal!bootstrap/js/dist/modal",
        Util: "exports-loader?Util!bootstrap/js/dist/util"
      }),
    ],
    devtool: devMode ? 'source-map' : undefined
  }
};

[aj-foster]

In case anyone is following along with these changes in their own project — and also as an FYI for the author and maintainers — I ran into an issue where the glob package was not available for use during a deploy (npm run deploy).

Original issue:

$ npm run deploy
> @ deploy /builds/pluralsight/experience/content/lab-tools/report-csp/report-csp/server/assets
> webpack --mode production
[webpack-cli] Failed to load '[redacted]/assets/webpack.config.js' config
[webpack-cli] Error: Cannot find module 'glob'

Note this only happened after I upgraded css-minimizer-webpack-plugin to 2.x; presumably glob was available as a sub-dependency of this package in 1.x. Either way, it might be good to include glob as a dev dependency in package.json since we use it in the webpack config.

At the end of the day, the changes I made are:

-    "css-minimizer-webpack-plugin": "^1.x",
+    "css-minimizer-webpack-plugin": "^2.x",
+    "glob": "^7.1.6",

Thank you all for your contributions and thoughtfulness.

[acconrad]

Hey so I did my own webpack upgrade and I think you'll need a few other rules to ensure the most compatibility:

        {
          test: /\.svg$/i,
          type: 'asset/inline'
        },
        {
          test: /\.(png|jpg|jpeg|gif)$/i,
          type: 'asset/resource',
        },
        {
          test: /\.(woff|woff2|eot|ttf|otf)$/i,
          type: 'asset/resource',
        },

This comes directly from the Webpack 5 tutorial to ensure fonts and images are loaded successfully as well

[cw789]

I also noticed once that this is needed if you want to process assets by webpack.
But then I didn't touch any of these anymore.
It was not clear for me which file types people really need to have in here.

[acconrad]

makes sense - i think it's pretty safe to say that in general css, js, images, and fonts are all fair game

[acconrad]

I also think you will get compilation errors if you don't specify target: 'nodeX.Y', because the default option for target is web which will usually cause things like http and url to blow up upon upgrading. Since these assets are built in a node-like environment targeting Node over web will ensure Webpack successfully compiles.

[cw789]

That's a thing I for myself would need to explore more in detail. I didn't get the consequences of this now.
For Phoenix deploys the target in my opinion should be web - not node.
But you might be right.

[acconrad]

I thought the same too but when I deployed with web I was getting errors around url and http and you'd have to set resolutions that resolve to false for both. Since this is a node environment for how we build our assets if you change it to node everything will resolve and pass.

[acconrad]

I did my own Webpack 5 upgrade and the only way to get all compilation errors to disappear was to implement a few additional changes on top of yours

[cw789]

Thanks @acconrad for your review.

To all:
I stopped ongoing maintenance of this PR past 3 months ago.
Of course in the meanwhile Node & Webpack are moving.
So this is no up-to-date reference anymore for using webpack v5 with Phoenix.

When someone tell me for sure this will get relevance for Phoenix again, of course I could have a fresh look.

[acconrad]

@cw789 I'm happy to take this over! You can either close this out and I can resume it in my own branch or I can just take this over from this open PR? @josevalim @chrismccord any recommendations? I have a version of this working perfectly with no compiler errors or warnings from webpack.

[Yattol]

Just tested this with node v15 and it does indeed work and fixes the current broken state of the new project. Thought definitely agree, I'm sure this would break something new for someone soon, so not too bullish either way (as new projects work fine with node v14)

It breaks also with node 16, which is the default version if you install via homebrew, which is quite common for OS X users.
(#4126 (comment))

I think it should at least be mentioned in the getting started guide, it's quite frustrating to follow a list of instructions and see they don't work, it doesn't give a good 'first impression' to newcomers.

[acconrad]

@cw789 if you close this PR I'll open one with the corrected points

[guidotripaldi]

I think webpack-cli 5.x isn't watching stdin any more so SIGINT signal do not close node process. The param --watch-options-stdin seems to do nothing.

with webpack 5 we have to give both --watch and --watch-options-stdin params to its CLI to have it working as expected (i.e. watching for changes and for SIGINT):

# config/dev.exs

use Mix.Config

config :myapp_web, MyAppWeb.Endpoint,
  http: [port: 4000],
  debug_errors: true,
  code_reloader: true,
  check_origin: false,
  watchers: [
    node: [
      "node_modules/webpack/bin/webpack.js",
      "--mode",
      "development",
      "--watch",
      "--watch-options-stdin",
     cd: Path.expand("../assets", __DIR__)
    ]
  ]

...

[cw789]

Dear maintainers. I'm in favour in closing this PR - please feel free.
PR #4337 is probably the more appropriate way foreward.

PS: This PR is anyway outdated now. In need of a revisit.
By the way, thanks for removing SASS already.

[acconrad]

@cw789 I've also created a new version at #4310