Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add bechmarks for chalk 5 and yoctocolors #61

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

prantlf
Copy link

@prantlf prantlf commented Feb 25, 2024

Fixes #45.

Changes

The latest version of chalk does not support CJS, but it is possible to bundle it in the CJS format. It is worth testing the latest available version.

yoctocolors does not support CJS, but it is possible to bundle it in the CJS format. This library aspires to be the new tiniest one. It supports FORCE_COLOR and NO_COLOR too. It is worth comparing it to picocolors.

Split chalk to chalk4 and chalk5 using NPM module aliases. Also upgrade development dependencies. Keep the same major version of clean-publish not to break the legacy Node.js tests.

Use esbuild to create CJS bundles of chalk5 and yoctocolors. Add chalk5 and yoctocolors to every benchmark. Rename the former chalk tests to chalk4.

Results

The simple.js benchmark shows 5% decrease of performance in chalk 5 against chalk 4. yoctocolors beat picocolors by 284%.

❯ node benchmarks/simple.js
colorette
kleur
chalk5
chalk4
ansi
cliColor
picocolors
nanocolors
yoctocolors
  chalk5         26,287,893 ops/sec
  chalk4         25,221,564 ops/sec
  cli-color       1,412,525 ops/sec
  ansi-colors     6,463,786 ops/sec
  kleur          26,363,857 ops/sec
  kleur/colors   52,166,754 ops/sec
  colorette      50,574,214 ops/sec
  nanocolors     51,646,373 ops/sec
  yoctocolors   195,458,649 ops/sec
+ picocolors     50,840,902 ops/sec

The complex.js benchmark shows 3% decrease of performance in chalk 5 against chalk 4. yoctocolors beat picocolors by 61%.

❯ node benchmarks/complex.js
  chalk5          1,178,597 ops/sec
  chalk4          1,215,094 ops/sec
  cli-color         152,419 ops/sec
  ansi-colors       558,016 ops/sec
  kleur           1,251,816 ops/sec
  kleur/colors    1,523,289 ops/sec
  colorette       1,813,436 ops/sec
  nanocolors      1,682,924 ops/sec
  yoctocolors     4,424,045 ops/sec
+ picocolors      2,747,818 ops/sec

The loading.js benchmark shows 44% decrease of performance in chalk 5 against chalk 4. yoctocolors is 75% slower than picocolors. I tried replacing the color detection in yoctocolors with the code from picocolors and the loading time went down to 0.588 ms, 52% slower than picocolors.

❯ node benchmarks/loading.js
  chalk5         7.713 ms
  chalk4         5.372 ms
  cli-color     32.428 ms
  ansi-colors    1.420 ms
  kleur          2.345 ms
  kleur/colors   0.960 ms
  colorette      0.887 ms
  nanocolors     0.708 ms
  yoctocolors    0.679 ms
+ picocolors     0.388 ms

The size.js benchmark shows 57% size reduction for chalk 5 against chalk 4. yoctocolors is practically the same as picocolors. This benchmark isn't reliable, because the size of the actually used code may be much smaller, as the comparison of the single-file libraries yoctocolors and picocolors shows.

❯ node benchmarks/size.js
Data from packagephobia.com
  chalk@5.3.0  43 kB
  chalk@4.1.2 101 kB
  cli-color   796 kB
  ansi-colors  27 kB
  kleur        20 kB
  colorette    17 kB
  nanocolors   15 kB
  yoctocolors   7 kB
+ picocolors    7 kB

❯ minified-size picocolors.js
picocolors.js: 2.6 kB, 1.86 kB, 626 B, 528 B

❯ minified-size benchmarks/yoctocolors.js
benchmarks/yoctocolors.js: 2.92 kB, 1.82 kB, 789 B, 697 B```

@prantlf prantlf force-pushed the chalk5-yoctocolors-benchmarks branch 3 times, most recently from d37a6a6 to 1cd18ec Compare February 25, 2024 13:30
The latest version of `chalk` does not support CJS, but it is possible
to bundle it in the CJS format. It is worth testing the latest available
version.

`yoctocolors` does not support CJS, but it is possible to bundle it
in the CJS format. This library aspires to be the new tiniest one.
It supports `FORCE_COLOR` and `NO_COLOR` too. It is worth comparing
it to `picocolors`.

Split `chalk` to `chalk4` and `chalk5` using NPM module aliases.
Also upgrade development dependencies. Keep the same major version
of `clean-publish` not to break the legacy Node.js tests.

Use `esbuild` to create CJS bundles of `chalk5` and `yoctocolors`.
Add `chalk5` and `yoctocolors` to every benchmark. Rename the former
`chalk` tests to `chalk4`.

Correct the number formatting to allow one more decimal place.
@prantlf prantlf force-pushed the chalk5-yoctocolors-benchmarks branch from 1cd18ec to e50360d Compare February 25, 2024 13:48
Copy the `simple` and `complex` benchmarks and convert
the module-importing code.

Use the dynamic import to measure the loading time
of ESM modules. The `--esm` argument enables the ESM mode.
It does not support NPM package aliases needed for tests
of `chalk` 4 and `chalk` 5. It could be worked around
by an additional `package.json`, but it is probably
not worth the effort.
@alexeyraspopov
Copy link
Owner

Thank you for taking time to contribute!

I will need a bit more time to thoroughly review the changes, apologies for the delay.

The very least I can say is that the decision to drop Node.js v6 should not be in the scope of this PR. This is something better be done in a separate PR and dropping anything below v10.16 would allow us to simplify color detection by a lot. Most likely a breaking change, so we gotta check if the largest user would be okay with it moving forward cc @ai

Another thing I'd like to point out is that yoctocolors is not a serious package to keep alongside others. At the very least, it's not complete. See how the code snippets from complex benchmark work:

image

Let's remove yoctocolors from benchmarks.

@ai
Copy link
Contributor

ai commented Mar 21, 2024

dropping anything below v10.16 would allow us to simplify color detection by a lot. Most likely a breaking change, so we gotta check if the largest user would be okay with it moving forward cc @ai

Browserslist supports Node.js 6 and releasing new major is very painful for the project.

But I think I can just avoid requiring picocolors on old Node.js versions in Browserslist. Anyway it is too old version.

@sindresorhus
Copy link

sindresorhus commented May 18, 2024

Another thing I'd like to point out is that yoctocolors is not a serious package to keep alongside others. At the very least, it's not complete.

It is now: https://github.com/sindresorhus/yoctocolors/releases/tag/v2.0.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Update benchmarks for Chalk (and other packages)
4 participants