Migrate to webpack, scss, and hugo

[nickbabcock]

I've grown disillusioned with the parcel bundler for a Rust WebAssembly purposes:

• Bizarre, baroque, and hard to reproduce errors
• Incorrect cache busting causing stale files breaking the web app (hence why I had to resort to a poor man's asset pipeline in this project with a combo of sha + sed)
• And a whole host of other problems revolving around rust.
• That specifying typescript compilation target means nothing without a browserlist.

Webpack is a natural replacement for parcel, except I have a big gripe with webpack: it's configuration. I'm sure if you polled developers how they feel about webpack configuration it'd be one word: magic. I find that they are write once and cross fingers it's perfect for eternity. I'm not ok with trading one type of magic (parcel's one size fits all) for another (webpack's unintelligible config).

There are a ton of webpack intermediaries that claim a zero or simplified config: nwb, neutrino, poi but they all fell short and left a sour taste in my mouth. I spent way more time trying to simplify a webpack setup through these tools than if I had just written it myself.

This project is quite simple:

• Plain Preact project written in typescript (no importing css, images, json, etc)
• A web worker that runs expensive synchronous webassembly functions off the UI thread
• Separate CSS file that is preprocessed to include 3rd party css files on npm
• Plain html that expects links to fingerprinted assets

It dawned on me that no magic is needed:

• We'll invoke wasm-pack build ourselves
• Then we'll invoke the typescript compiler npx tsc ourselves
• Then we'll invoke webpack ourselves npx webpack --mode production ourselves
• Then we'll invoke a css preprocessor ourselves
• Then we'll generate a site that includes all these assets fingerprinted

Examining the steps in closer detail:

• wasm-pack build generates a wasm bundle with typescript definitions that can be directly imported into our project.
• The typescript compiler chews through the typescript files in src and outputs modern ES2017+ js (as we're relying on modern features already (ie: webassembly)) into the dist directory
• Webpack takes the dist directory, does the one thing it does best -- bundling -- and outputs fingerprinted files to the static/js directory.

There are only two webpack plugins needed to accomplish this:

• worker-plugin so that webpack correctly splits the worker script into a fingerprinted file that also can correctly import a fingerprinted web assembly module
• assets-webpack-plugin which will output a file that contains a map of original js filenames to the fingerprinted ones for a downstream process.

That's it for webpack. This all can be accomplished with the shortest webpack config I've ever seen:

const path = require("path");
const WorkerPlugin = require("worker-plugin");
const AssetsPlugin = require("assets-webpack-plugin");

module.exports = {
  entry: "./dist/index.js",
  plugins: [
    new WorkerPlugin(),
    new AssetsPlugin({filename: "data/webpack.json"}),
  ],
  output: {
    filename: "[name].[contenthash].js",
    path: path.join(__dirname, "static", "js"),
    publicPath: "/js/",
  }
};

Now for the most opinionated part -- we'll be using the static site generator, hugo (specifically hugo extended), from here on out:

• Hugo (extended) has a built in Sass preprocessor, so this PR drops postcss and config for a built in preprocessor.
• Hugo can fingerprint assets unlike zola
• Hugo reads in the file generated by assets-webpack-plugin (data/webpack.json) so it can insert the fingerprinted js links

aside: the reason why we need webpack to fingerprint the js (instead of having hugo do it) is that webpack also spits out fingerprinted web worker .js and .wasm files are loaded asynchronously

In essence we'll be using a heavily simplified victor-hugo setup.

Yes it seems like overkill to introduce a static site generator for this project, but I believe it is the right approach. There is no magic glue between any of the components. Nothing really to go wrong.

I see people complain about the complexity of hugo and all the ceremony it takes to set it up but I only had to move around some files and add a single config.toml file. Brace yourself, it's long:

disableKinds = ["taxonomy", "taxonomyTerm", "RSS", "sitemap"]

Even this isn't required, but for a simple project I don't want all those extra files generated. Everything comes together with npm run build, which executes

wasm-pack build crate && tsc && mkdir -p data && webpack --mode production && hugo

Webpack Config Additions

The posted webpack config is quite short, but are there other some additions that one may want to consider that hopefully won't break the "magic" barrier.

Sourcemaps

Sometimes one needs to step through JS. Without souremaps it can be difficult to reason about how the code is behaving, so we can add devtool: "source-map" to help set breakpoints and the like:

const path = require("path");
const WorkerPlugin = require("worker-plugin");
const AssetsPlugin = require("assets-webpack-plugin");

module.exports = {
  entry: "./dist/index.js",
  devtool: "source-map",
  plugins: [
    new WorkerPlugin(),
    new AssetsPlugin({filename: "data/webpack.json"}),
  ],
  output: {
    filename: "[name].[contenthash].js",
    path: path.join(__dirname, "static", "js"),
    publicPath: "/js/",
  }
};

A one line change. Everything is still reasonable. Couple things to note:

• The source maps are distributed into production. I have no problem with this and neither should you.
• The source maps will be of the javascript that was compiled from typescript. Since typescript generates modern javascript that is quite close to the original typescript, I have had no problems with this

Adding sourcemaps is a well worth addition.

Ts-loader

One can consolidate the typescript then webpack invocations into the webpack config if they want to add a dash of complexity to their webpack config and add the ts-loader dependency. Personally, I don't see the need for this consolidation as the "simplest setup" is still far too long to be a replacement for a single tsc command. Here is what our config would look like if we added in ts-loader

const path = require("path");
const WorkerPlugin = require("worker-plugin");
const AssetsPlugin = require("assets-webpack-plugin");

module.exports = {
  entry: "./src/index.tsx",
  devtool: "source-map",
  module: {
    rules: [
      {
        test: /\.tsx?$/,
        loader: "ts-loader",
      },
    ],
  },
  resolve: {
    extensions: [".ts", ".tsx", ".js"],
  },
  plugins: [
    new WorkerPlugin(),
    new AssetsPlugin({ filename: "data/webpack.json" }),
  ],
  output: {
    filename: "[name].[contenthash].js",
    path: path.join(__dirname, "static", "js"),
    publicPath: "/js/",
  },
};

The benefits of ts-loader:

• Sourcemaps now contain the original typescript code

If there was like a new TsLoaderPlugin() then, it would be worth considering.

Dependency Analysis

While we have more dependencies specified in our package.json there are 3000 lines less in the lockfile.

Hugo is now a required dependency, but it's widely available as a single executable.

Categorizing our dev dependencies:

  "devDependencies": {
    "@types/jest": "^25.1.4",           // unit test
    "assets-webpack-plugin": "^3.9.12", // bundler
    "cypress": "^4.2.0",                // integration test 
    "cypress-file-upload": "^4.0.4",    // integration test
    "jest": "^25.1.0",                  // unit test
    "start-server-and-test": "^1.10.11",// integration test
    "ts-jest": "^25.2.1",               // unit test
    "typescript": "^3.8.3",             // bundler
    "webpack": "^4.42.1",               // bundler
    "webpack-cli": "^3.3.11",           // bundler
    "worker-plugin": "^4.0.2"           // bundler
  },

The majority of them relate to testing (once jest can run the typescript compiled modules directly, jest will need to be the only unit testing dependency left). I'm satisfied though, I feel like I've whittled this list to down to a reasonable level. Any more would require too many hoops to jump through.