sass-loader performance

[mmase]

We have a fairly expansive suite of sass files (around 600 files total). When running our suite directly with libsass, behind node-sass, it takes less than 5 seconds. Yet, sass-loader through webpack takes over 15 seconds.

I am wondering if others have run into this or if we can debug this to find where the performance lag lies.

[khankuan]

I experienced great improvements after switching to node-sass. But doing that would go in opposite direction with component-based styling. @mmase do you have lots of variables and mixins that you import with @import? Also, are you importing .sass files multiple times, i.e, every component imports its own sass file?

[wildeyes]

"Also, are you importing .sass files multiple times, i.e, every component imports its own sass file?" would doing this improve or worsen the compiliation speed?

[jhnns]

Related to: #307

[khankuan]

@wildeyes i think so. I have 2 sass files, variables.sass and componentA.sass. The variables file is used by multiple components. Incremental rebuild speed for 700 components took 8 secs on a variable.sass change while 2 secs for componentA.sass.

[jhnns]

Based on this discussion, I started profiling an example project yesterday and had some interesting insights. First of all, the sass-loader performance (49.2s) is completely unacceptable compared to fast-sass-loader (5.29s) or even bare node-sass (2.25s) from the command line. All three toolchains compile the same source code and produce similar output. We can expect some overhead coming from webpack since we need to initialize it, set up the loader pipeline, etc., but even the fast-sass-loader is unacceptable slow compared to bare node-sass in my opinion.

Overview

So let's take a look at the flame graphs:

First of all, as you can see, node-sass is spending most of the time outside of Node.js which is what we expected, since the hard work is done by Libsass.

Now, let's take a look at both sass-loaders. The execution can roughly be divided into five phases (indicated as black boxes in the graph) that I will discuss in detail later:

1. webpack initialization
2. first execution sass-loader
3. first execution css-loader
4. second execution sass-loader (this block is not visible in the fast-sass-loader graph, reasons below)
5. second execution css-loader

So, the first surprise here is that we have two separate compilations going on. This is due to the extract-text-webpack-plugin which initiates a child compilation. Tbh, I'm not sure why this is necessary at all. @sokra can give us more insights here.

The second surprise is, that the second compilation doesn't use the cache from the first compilation. In the sass-loader graph, you can clearly see that the second pass takes almost exactly the same amount of time. This could be improved a little bit because the sass-loader is currently not using webpack's fs cache, but that would only reduce the second sass-loader execution time (4.), which is not where most of the time is spent.

The most time, however, is spent—and that's the third surprise—inside the css-loader. You can clearly see that in both graphs. It takes up half of the total time in the fast-sass-loader setup, and even more in the sass-loader setup.

It might also be surprising that node-sass and webpack + fast-sass-loader take almost the same amount of time to compile. In the flame graphs, you can clearly see that node-sass finishes almost at the same time where the 2. block of the fast-sass-loader graph ends. However, we need to remember that the fast-sass-loader is deduping all files before compilation, so there's far less to compile.

---

Now, let's take a look at all the phases in detail.

Phase 1: Webpack initialization & loader execution start

No big surprise here: The initialization of both setups takes almost the same amount of time. Differences here are probably due to I/O or general OS flakyness because there is no actual difference between both loaders here.

You can clearly see that during the first 1000ms, most of the time is spent requiring modules. That's why webpack almost always takes at least 1s to compile—even with the simplest setup. Maybe we could win something here with lazy requiring ala lazy-req.

Phase 2: Import resolving & sass compilation (1. compilation)

Now that looks different!

First, let's look at the sass-loader: The sass-loader registers a custom importer and kicks off the compilation. Now, we're essentially waiting for Libsass to call back. You can clearly see that there is a lot of back-and-forth going on with recurring breaks (grey bars) where the process is just sitting and waiting for Libsass. This graph led me to the assumption that Libsass might be doing sequential I/O. Concurrent I/O would look different, I suppose. After all the resolving is done, Libsass is performing the actual compilation and we're just waiting for Libsass to call back. Nothing to optimize here.

Now, let's look at the fast-sass-loader: The fast-sass-loader preparses the source code with regexps for import statements and passes all imports to webpack's resolver. After the import paths have been resolved, the imported files are read from disk via node's fs module and then replaced with their import statements. Duplicated imports are simply omitted, only the first occurence gets included. This is way faster than the sass-loader, which shows that webpack's resolver is not the bottleneck. In fact, it's really fast and almost negligible in the graph.

The actual Sass compilation is also a lot faster since a lot of imports have been deduped. It's just that Libsass has less to parse.

Phase 3: css-loader (1. compilation)

This phase is the most interesting one because:

1. it takes up most of the time
2. there is nothing the sass-loader or the fast-sass-loader can do

The only difference is that the amount of data is very different. The fast-sass-loader produces a string with length 627067, the sass-loader with 6914854. This is 11 times longer. And this translates roughly to the amount of time. The css-loader takes 11 times longer processing the output from the sass-loader.

There are two things that I found surprising:

1. Why does the css-loader take so long? In this case, it is only required to translate the CSS into a CommonJS module while transforming all url() statements and remaining @import into require(). To me, this task seems a lot more trivial than the Sass compilation. The funny thing is: We're spending 9.6s just inside postcss-modules-local-by-default during the first compilation. In total, we're spending 16.29s there – and we didn't even ask for CSS modules... And that's just the first postcss plugin. The thread is completely busy for 13s in the first compilation while processing postcss plugins without any chance for other tasks. In total: 26s just inside postcss, hogging the CPU.

2. Why is postcss busy with generating source maps? We didn't ask for source maps... then I took a look at the css-loader: This call applies the map option irregardless of an inputMap. In fact, inputMap is undefined in this case, but we're still telling postcss to generate a map. As an experiment, I set map: false and was able to shave off 10s in total—just by setting one flag. It's still crazy that source maps take up 10s. Switching source maps on and off with node-sass makes a difference of 150ms.

Phase 4: sass-loader & css-loader (2. compilation)

The sass-loader performs exactly the same compilation again. No results from the first compilation are re-used.

The fast-sass-loader skips the second Sass compilation because it uses its own cache. That's a nice shortcut, but this should be fixed in webpack.

I don't understand why we need a second compilation. And if we need the second one, can't we just skip the first one? This seems redundant.

---

Conclusion

• We spend 1s just requiring webpack modules and loaders and initializing the loader pipeline. This could possibly be improved with a tool like lazy-req.
• The sass-loader could be improved by:
  • preparsing and resolving all dependencies. While this might seem like a good idea, there are also problems: 1. The source maps will be incorrect. 2. Errors won't be reported at the correct location. It all boils down to the fact that Libsass doesn't know the original location of the source anymore.
  • using the webpack cache when resolving the file. A custom importer is also allowed to pass the actual source to Libsass. This way, we could use webpack's internal fs caching layer.
  • deduping dependencies. This could be achieved by passing an empty string if the custom importer has resolved a resource for the second time. While this operation is not entirely safe, it will most likely do more good than bad in most cases. I'd say: if the specificity relies on the correct source order, there's a bigger problem.
• The css-loader could be improved by:
  • using a simpler pipeline for CSS files that are not CSS modules. This performance penalty is not acceptable.
  • only generating source maps when they are actually requested
  • "somehow" improving the postcss pipeline. Blocking the process for 13s is not acceptable, even for CSS modules. I don't know how we can improve that. Maybe postcss is applying the plugins in an unfortunate way?
• The extract-text-webpack-plugin could be improved by:
  • getting rid of the second compilation
  • or using cached loader results where possible

Here are both CPU profiles. Load them into your Chrome Developer Tools if you want to take a look for yourself.

cpuprofiles.zip

---

This was bothering me for a long time because I think we can do a lot better than that 😁

[jhnns]

As comparison, the sass-loader setup with css-loader@0.14 takes 10.1s and looks like this:

sass-loader-old-css-loader.cpuprofile.zip

[jhnns]

As @sokra pointed out: With the allChunks: true option...

    new ExtractTextPlugin({
      filename: '[name].css',
      allChunks: true
    })

the second compilation can be skipped:

thus reducing the built time to 23.9s

[feng-xiao]

@jhnns Hi jhnns, may I ask what tools you were using to generate those profiling screenshots? I need to troubleshoot webpack issues as well, I need a tool to help me pinpoint the problem.

[jhnns]

Run

node --inspect-brk ./node_modules/.bin/webpack

to start the debugging process. The process will halt at the first line and print a debugger URL. Copy the debugger URL into the Chrome Browser and the developers tools will initialize.

Then go to the JavaScript profiler tab and start profiling :)
After you've pressed stop, the flame graph will be generated.

If you don't want to copy debugger URLs around, you can also use the NIM chrome extension. It discovers debuggable node processes automatically.

[feng-xiao]

@jhnns thank you very much!!!

[jhnns]

Not directly related to this thread, but also interesting: Looks like the most recent v8 version (with their new optimizing compiler TurboFan) gives a startup performance boost of 160% (1150ms vs 720ms).

Using v8/node, phase 1 looks like this:

The overall build time decreased from 49.2s to 41.35s (~119% faster).

TurboFan doesn't bail out on try/catch which will probably improve the startup performance of all node applications because node is using try/catch to initialize the modules.

[sokra]

awesome 👍

[alexander-akait]

Postcss issue about source map: webpack-contrib/postcss-loader#195

[alexander-akait]

After disabling generation of source maps using my PR webpack-contrib/css-loader#478 i get about ~1.5x-2x speedup my build

[mikesherov]

hello friends. Just as an FYI, I'm looking into this. With any luck, we'll see if there is any low hanging fruit to go after besides for the allChunks and sourcemap: false optimizations.

[0x11-dev]

any new updates? @mikesherov

[mikesherov]

@levin-du, several of css-loaders dependencies are extremely slow but have been rewritten to be much much faster, but are semver major. css-loader is currently undergoing a rewrite along with the rest of the CSS module loading architecture, so I asked if a patch would be accepted. Due to the fact that it's a major bump, I was asked to wait for the rewrite to be completed instead.

[sentience]

That's a shame, @mikesherov, but thanks for the update!

[jhnns]

Maybe dart-sass #435 is faster. I need to try that out.

[xzyfer]

I quickly put together a proof of concept loader for first class node module imports. It'll probably need some API changes to work within the webpack context but it's worth a look.

https://www.npmjs.com/package/@node-sass/node-module-importer

[manniL]

Hey folks 👋
Are there any news/successes on the perf topic? ☺️

[alexander-akait]

@manniL no,

We already cache many things and need improve speed on node-sass/dart-sass side, anyway you can look our code and send a PR with code where you can improve speed. Close issue because no places for optimization on our side. Also you can try cache-loader, it is allow to caching styles which don't changed, webpack@5 will support caching out of box and it is speed your build. Thanks!

[Grawl]

years gone, my projects still get ~15s to re-compile on watch

[DimaGashko]

@Grawl try move imports from scss to js (as many as you can). In my case it speeded up my build by more than 5 times (10+ if turn off source maps).

[alexander-akait]

@DimaGashko can you create minimum reproducible test repo?

[Grawl]

@DimaGashko my mixins, variables and placeholder selectors will work as usual after this changes?

[DimaGashko]

@Grawl you can create file _disappearing.scss

@import "variables";

@import "mixins/mixinA";
@import "mixins/mixinB";

@import "somePlaceholderSelectors";

And import it in each entry style file.
Or by sass-loader automaticly:

loader: 'sass-loader',
options: {
       // Of cource, you have to use aliases
       data: '@import "~@/styles/disappearing" '; 
       . . .
}

[rodoabad]

Are we still favoring fast-sass-loader over sass-loader?

[simeyla]

@rodoabad did you ever get an answer on this (one year ago today) question?

[rodoabad]

@simeyla no I have not.

[DzmVasileusky]

Should we revive this topic?

2023 and my extra big project takes 9 mins, 24.091 secs of which SCSS compilation takes 8 mins, 21.86 secs

General output time took 9 mins, 24.091 secs
 SMP  ⏱  Plugins
SourceMapDevToolPlugin took 16.66 secs
LicenseWebpackPlugin took 5.67 secs
DevToolsIgnorePlugin took 0.733 secs
OccurrencesPlugin took 0.631 secs
CopyPlugin took 0.188 secs
CommonJsUsageWarnPlugin took 0.053 secs
NamedChunksPlugin took 0.043 secs
SuppressExtractedTextChunksWebpackPlugin took 0.037 secs
ProgressPlugin took 0.003 secs
AnyComponentStyleBudgetChecker took 0.001 secs
StylesWebpackPlugin took 0 secs
DedupeModuleResolvePlugin took 0 secs
 SMP  ⏱  Loaders
css-loader, and 
postcss-loader, and 
resolve-url-loader, and 
sass-loader took 8 mins, 21.86 secs
  module count = 558
@angular-devkit/build-angular, and 
@ngtools/webpack took 2 mins, 31.37 secs
  module count = 4519
mini-css-extract-plugin, and 
css-loader, and 
postcss-loader, and 
resolve-url-loader, and 
sass-loader took 2 mins, 12.51 secs
  module count = 3
@angular-devkit/build-angular, and 
source-map-loader took 1 min, 43.57 secs
  module count = 2174
modules with no loaders took 23.01 secs
  module count = 666
source-map-loader took 3.58 secs
  module count = 34
mini-css-extract-plugin, and 
css-loader, and 
postcss-loader took 2.94 secs
  module count = 2
css-loader, and 
postcss-loader took 0.92 secs
  module count = 7
✔ Browser application bundle generation complete.

[alexander-akait]

@DzmVasileusky If you can give me access to the project I can profile, I don't think we have a problem with sass-loader itself here

[DzmVasileusky]

@alexander-akait I will try to make a big enough project to play with, can't share our enterprise app's code.

BTW, removing all the SCSS from the app decreased the build time from 9min 24s to 3min 33s:
sass/css loaders are still working 1 min, 53.83 secs but this it probably because of pending or looking up since there is no SCSS in project anymore.
So it practically shows that sass / css loaders are taking ~ 5 min 51s which is ~ 62% of all the Angular / NX project build time.

General output time took 3 mins, 33.4 secs
 SMP  ⏱  Plugins
SourceMapDevToolPlugin took 11.19 secs
LicenseWebpackPlugin took 5.23 secs
DevToolsIgnorePlugin took 0.439 secs
OccurrencesPlugin took 0.432 secs
CopyPlugin took 0.17 secs
CommonJsUsageWarnPlugin took 0.039 secs
SuppressExtractedTextChunksWebpackPlugin took 0.032 secs
NamedChunksPlugin took 0.027 secs
AnyComponentStyleBudgetChecker took 0.013 secs
ProgressPlugin took 0.002 secs
DedupeModuleResolvePlugin took 0.001 secs
 SMP  ⏱  Loaders
css-loader, and 
postcss-loader, and 
resolve-url-loader, and 
sass-loader took 1 min, 53.83 secs
  module count = 27
@angular-devkit/build-angular, and 
@ngtools/webpack took 1 min, 50.15 secs
  module count = 4519
@angular-devkit/build-angular, and 
source-map-loader took 1 min, 44.91 secs
  module count = 2174
modules with no loaders took 41.081 secs
  module count = 666
source-map-loader took 6.081 secs
  module count = 34
✔ Browser application bundle generation complete.

[alexander-akait]

@DzmVasileusky

So it practically shows that sass / css loaders are taking ~ 5 min 51s which is ~ 62% of all the Angular / NX project build time.

Do you use sass, node-sass or sass-embedded?

Also let's try to check this - try to compile sass code without webpack, like you have one big file and check perf, thank you

[DzmVasileusky]

@alexander-akait I have found the biggest issue.
It was usage of @use '@angular/material' as mat; in many files.
I thought @use would be much faster than @import but not as fast to use it in hundreds of files.
In my case I used @angular/material in project mixins then imported these mixins almost in every file. And it is ~80ms vs ~2ms without angular material usage.

[alexander-akait]

@DzmVasileusky Great, yeah, most of problems with sass perfomance are not in sass-loader, we just wrapper with custom resolver, it has some overhead, but it is very very little and we cache mostly everything

[DiazAilan]

@alexander-akait I have found the biggest issue. It was usage of @use '@angular/material' as mat; in many files. I thought @use would be much faster than @import but not as fast to use it in hundreds of files. In my case I used @angular/material in project mixins then imported these mixins almost in every file. And it is ~80ms vs ~2ms without angular material usage.

But how we can use mat.get-color-from-palette without @useing @angular/material?
Right now I've 104 files that use this function and I dunno how can I speed up this without making a big refactor.
There is something I'm doing wrong with this kind of implementations?. E.g.:

@use '@angular/material' as mat;

$backgroundColor: mat.get-color-from-palette($neutral, 50);

[alexander-akait]

@DiazAilan You can ask @angular/material developers to split files and load only required

[DzmVasileusky]

@DiazAilan sorry to hear it. but you can do the following

_colors.scss

$neutral-palette: (
    000: #000,
    300: #111,
    500: #222
    contrast: (
        000: #fff,
        300: #fff,
        500: #fff
    )
);

theme.scss

@use '@angular/material' as mat;
@use 'colors';

$neutral: mat.define-palette(colors.$neutral-palette, 000, 300, 500);

$your-theme: mat.define-light-theme((
...use $neutral here if you want
));

any-component.scss

@use 'colors';

$backgroundColor: map.get(colors.$neutral-palette, 000);

[DiazAilan]

@DiazAilan sorry to hear it. but you can do the following

_colors.scss

$neutral-palette: (
    000: #000,
    300: #111,
    500: #222
    contrast: (
        000: #fff,
        300: #fff,
        500: #fff
    )
);

theme.scss

@use '@angular/material' as mat;
@use 'colors';

$neutral: mat.define-palette(colors.$neutral-palette, 000, 300, 500);

$your-theme: mat.define-light-theme((
...use $neutral here if you want
));

any-component.scss

@use 'colors';

$backgroundColor: map.get(colors.$neutral-palette, 000);

This works like a charm! Thanks!
Nevertheless, while doing so we loss the possibilty to use the -contrast prefix of an Angular/material theme. Any workarounds or idea for this?:

$primary: (
  500: #2382D2,
  contrast: (
    500: $light-text,
  )
);

.count {
    color: map.get($primary, '500-contrast'); // <---- this expression will not work =/
}

[DzmVasileusky]

@DiazAilan

@use 'sass:map';

@function map-deep-get($map, $keys...) {
    @each $key in $keys {
        $map: map.get($map, $key);
    }
    @return $map;
}

.some {
  color: map-deep-get($primary, contrast, 500);
}