Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

benchmark: rewrite the startup performance benchmark #45746

Merged
merged 5 commits into from Dec 8, 2022

Commits on Dec 5, 2022

  1. benchmark: make benchmarks runnable in older versions of Node.js

    Also remove the require-cachable.js benchmarks because now all builtin
    modules are cacheable, it would be comparing oranges to apples when
    we try to compare the performance of loading all cacheable modules
    in different Node.js binaries since the set of modules are just
    different. Comparison of startup performance that involves loading
    of the long-standing, stable builtins is already covered by the
    require-builtins benchmark.
    joyeecheung committed Dec 5, 2022
    Copy the full SHA
    996719f View commit details
    Browse the repository at this point in the history
  2. benchmark: rewrite the startup performance benchmark

    The previous benchmark was inaccurate in several ways:
    
    - It previously measured "how many workers/child processes can finish
      loading in parallel in 1 second" which is prone to wrong
      interpretations. It's also not typical for users to spin up >20
      workers/child processes in parallel, which was what the benchmark
      usually ended up doing. The parallelism can also be greatly affected
      by system configurations.
    - The time spent on loading a Node.js binary for the first time varies
      greatly depending on system configurations, but it was inevitable
      for the benchmark to load Node.js multiple times to reduce the
      influence of fluctuations.
    
    This patch rewrites the benchmark so that:
    
    - It now measures "how long it takes to finish 30 workers/child processes
      in serial" which is generally more in line with how other benchmarks
      are written and with the figures one gets from doing something like
      `time node index.js` or `hyperfine 'node index.js'`
    - It now warms up the loading of the Node.js instance to reduce the
      statistical outliers, so that we get more meaningful figures when
      trying to compare the startup performance of different Node.js
      binaries.
    joyeecheung committed Dec 5, 2022
    Copy the full SHA
    bc05a5f View commit details
    Browse the repository at this point in the history
  3. Copy the full SHA
    4b1b762 View commit details
    Browse the repository at this point in the history
  4. Copy the full SHA
    9f82938 View commit details
    Browse the repository at this point in the history

Commits on Dec 6, 2022

  1. Copy the full SHA
    ee2cbe3 View commit details
    Browse the repository at this point in the history