Compare speed by speedometer

[onegreyonewhite]

Recently, I noticed that official Chrome seemed to render pages a little faster. I decided to take some measurements.

Thorium 122.0.6261.132 (from official repo)
Google Chrome 124.0.6367.60 (from official repo)

Ubuntu 22.04 LTS with all updates.

https://browserbench.org/Speedometer3.0/

Test Name	Thorium Result (ms)	Google Chrome Result (ms)
Geomean	107.58 ± 7.28 (6.8%)	92.94 ± 5.63 (6.1%)
TodoMVC-JavaScript-ES5	137.08 ± 12.17 (8.9%)	122.83 ± 11.67 (9.5%)
TodoMVC-JavaScript-ES6-Webpack-Complex-DOM	173.59 ± 6.93 (4.0%)	154.73 ± 5.44 (3.5%)
TodoMVC-WebComponents	43.97 ± 3.20 (7.3%)	40.18 ± 5.23 (13.0%)
TodoMVC-React-Complex-DOM	122.86 ± 15.09 (12.3%)	97.98 ± 9.18 (9.4%)
TodoMVC-React-Redux	117.57 ± 17.39 (14.8%)	113.73 ± 15.83 (13.9%)
TodoMVC-Backbone	82.31 ± 12.37 (15.0%)	66.95 ± 7.69 (11.5%)
TodoMVC-Angular-Complex-DOM	116.02 ± 17.08 (14.7%)	98.84 ± 11.36 (11.5%)
TodoMVC-Vue	74.98 ± 11.02 (14.7%)	69.62 ± 11.05 (15.9%)
TodoMVC-jQuery	466.08 ± 61.82 (13.3%)	372.57 ± 45.67 (12.3%)
TodoMVC-Preact-Complex-DOM	45.72 ± 6.78 (14.8%)	36.87 ± 4.93 (13.4%)
TodoMVC-Svelte-Complex-DOM	44.46 ± 4.13 (9.3%)	33.57 ± 4.74 (14.1%)
TodoMVC-Lit-Complex-DOM	48.27 ± 4.38 (9.1%)	41.07 ± 3.23 (7.9%)
NewsSite-Next	222.31 ± 13.26 (6.0%)	208.87 ± 18.39 (8.8%)
NewsSite-Nuxt	181.07 ± 8.22 (4.5%)	162.87 ± 8.90 (5.5%)
Editor-CodeMirror	52.35 ± 6.15 (11.8%)	44.56 ± 6.22 (14.0%)
Editor-TipTap	159.89 ± 12.99 (8.1%)	142.37 ± 7.97 (5.6%)
Charts-observable-plot	95.15 ± 7.80 (8.2%)	101.73 ± 13.68 (13.4%)
Charts-chartjs	174.07 ± 12.62 (7.2%)	121.15 ± 11.36 (9.4%)
React-Stockcharts-SVG	183.46 ± 18.92 (10.3%)	166.73 ± 10.08 (6.0%)
Perf-Dashboard	95.22 ± 5.49 (5.8%)	80.07 ± 3.25 (4.1%)

Somehow it doesn't look very good. Are these two major versions really packed with so many optimizations?

[ms178]

Could you tell us more details on the used system (e.g. CPU and which Thorium build has been used, the AVX or AVX2 build)?! Benching different versions against each other could be misleading as there are indeed newer features/optimizations in newer builds that might have a performance impact.

Needless to say that also having the same modifications/options enabled in both builds could make a difference.

[onegreyonewhite]

Could you tell us more details on the used system (e.g. CPU and which Thorium build has been used, the AVX or AVX2 build)?! Benching different versions against each other could be misleading as there are indeed newer features/optimizations in newer builds that might have a performance impact.

Needless to say that also having the same modifications/options enabled in both builds could make a difference.

As I write on top of the issue, I used deb packages from the official repo. No manual builds.

Processor Intel i5-10500H. GPU is Nvidia 3060M (laptop). Enabled additionally only GPU's based rendering. Without it, the result is almost the same.

[ms178]

As I write on top of the issue, I used deb packages from the official repo. No manual builds.

As there are several Thorium deb packages from the official repo under https://github.com/Alex313031/thorium/releases/tag/M122.0.6261.132 please specify which one you used. Your CPU should be compatible with the AVX2 build that is available under the given link and should provide the best performance for your system, please test that one if you haven't already.

To make a fair comparison, I'd suggest to benchmark Thorium AVX2 and Google Chrome on the same revision and with the same settings/config options.

[onegreyonewhite]

As I write on top of the issue, I used deb packages from the official repo. No manual builds.

As there are several Thorium deb packages from the official repo under https://github.com/Alex313031/thorium/releases/tag/M122.0.6261.132 please specify which one you used. Your CPU should be compatible with the AVX2 build that is available under the given link and should provide the best performance for your system, please test that one if you haven't already.

Official debian/ubuntu repo.

To make a fair comparison, I'd suggest to benchmark Thorium AVX2 and Google Chrome on the same revision and with the same settings/config options.

What's profit of that? Official debian repos provide different versions. Users don't hold a version of Chrome. Currently provided version is good point to comparison.

[ms178]

Official debian/ubuntu repo.

Then please test the AVX2 version that is provided under the release link I gave you as I cannot say which compiler flags Ubuntu uses to compile their provided Thorium version and if they honor the aggressive flags used by Thorium by default.

What's profit of that? Official debian repos provide different versions. Users don't hold a version of Chrome. Currently provided version is good point to comparison.

As you haven't noticed yourself yet, you are comparing an "unofficial" Ubuntu-provided build of Thorium which is most likely not using all the capabilities of your CPU properly against a newer revision of Chrome. That's not a fair comparison for these following two most important reasons:

1. The official Thorium build uses optimized compiler flags and offers many builds on a per-capability basis to squeeze out more performance of newer CPUs, chances are that Ubuntu uses less optimized compiler flags in their repo by default as they don't ship x86-64-v3 repos yet (which would enable AVX2), This means that you end up with a less performant version using Thorium from the Ubuntu repo. So please just use the official AVX2 Thorium build that is provided from this repo on Github and not the Ubuntu-provided build to show the best case scenario for Thorium on your CPU. I hope you realize that making generalized claims about the lack of Thorium's performance while not using the best official build for your CPU is just dishonest to the Thorium project.

2. Fair benchmarking is an art. As already stated, there might be performance work on the source code level in newer revisions, you'd want to rule out that variable in a fair comparison to assess the qualities of Thorium vs. vanilla Chrome. What you do is an apples to oranges comparison at the moment that has no meaningful value, I am afraid.

[onegreyonewhite]

Official debian/ubuntu repo.

Then please test the AVX2 version that is provided under the release link I gave you as I cannot say which compiler flags Ubuntu uses to compile their provided Thorium version and if they honor the aggressive flags used by Thorium by default.

What's profit of that? Official debian repos provide different versions. Users don't hold a version of Chrome. Currently provided version is good point to comparison.

As you haven't noticed yourself yet, you are comparing an "unofficial" Ubuntu-provided build of Thorium, which is most likely not using all the capabilities of your CPU properly against a newof Chrome.

OFFICIAL Thorium repository. I wrote "official". And official Google repository. That means we have two providers who give almost the same product. And I am trying to compare them.

sudo rm -fv /etc/apt/sources.list.d/thorium.list && \
sudo wget --no-hsts -P /etc/apt/sources.list.d/ \
http://dl.thorium.rocks/debian/dists/stable/thorium.list && \
sudo apt update

Ask @Alex313031, which flags he used in his repo.

[onegreyonewhite]

I test latest thorium release (123) with AVX2 support (got from latest gh release for the integrity of the experiment) and latest chrome stable release (124).

Each browser was run with clear user profile dir (run with --user-data-dir=/tmp/[browser]). That means disabled all flags and used native capabilities.

Test	Thorium (ms)	Chrome (ms)	Ratio (Thorium/Chrome)
Geomean	85.49 ± 2.30 (2.7%)	77.66 ± 1.66 (2.1%)	1.10
TodoMVC-JavaScript-ES5	105.10 ± 12.33 (11.7%)	87.57 ± 6.14 (7.0%)	1.20
TodoMVC-JavaScript-ES6-Webpack-Complex-DOM	144.20 ± 9.15 (6.3%)	119.06 ± 7.00 (5.9%)	1.21
TodoMVC-WebComponents	36.71 ± 1.91 (5.2%)	35.82 ± 1.49 (4.1%)	1.02
TodoMVC-React-Complex-DOM	93.32 ± 12.65 (13.6%)	80.75 ± 5.63 (7.0%)	1.16
TodoMVC-React-Redux	85.80 ± 3.27 (3.8%)	84.62 ± 3.72 (4.4%)	1.01
TodoMVC-Backbone	67.25 ± 4.81 (7.2%)	58.74 ± 2.79 (4.8%)	1.14
TodoMVC-Angular-Complex-DOM	79.02 ± 8.90 (11.3%)	74.42 ± 5.90 (7.9%)	1.06
TodoMVC-Vue	60.82 ± 4.50 (7.4%)	55.58 ± 5.89 (10.6%)	1.09
TodoMVC-jQuery	305.23 ± 14.59 (4.8%)	216.14 ± 21.59 (10.0%)	1.41
TodoMVC-Preact-Complex-DOM	39.19 ± 6.00 (15.3%)	32.94 ± 5.96 (18.1%)	1.19
TodoMVC-Svelte-Complex-DOM	33.75 ± 3.18 (9.4%)	28.23 ± 2.56 (9.1%)	1.20
TodoMVC-Lit-Complex-DOM	43.44 ± 6.77 (15.6%)	43.31 ± 6.79 (15.7%)	1.00
NewsSite-Next	186.57 ± 5.86 (3.1%)	170.46 ± 7.15 (4.2%)	1.09
NewsSite-Nuxt	159.65 ± 4.81 (3.0%)	145.22 ± 8.34 (5.7%)	1.10
Editor-CodeMirror	43.44 ± 5.90 (13.6%)	45.12 ± 9.46 (21.0%)	0.96
Editor-TipTap	147.55 ± 9.30 (6.3%)	125.66 ± 6.11 (4.9%)	1.17
Charts-observable-plot	97.28 ± 7.44 (7.7%)	90.22 ± 6.78 (7.5%)	1.08
Charts-chartjs	103.88 ± 12.47 (12.0%)	111.20 ± 14.39 (12.9%)	0.93
React-Stockcharts-SVG	143.95 ± 3.87 (2.7%)	135.70 ± 5.45 (4.0%)	1.06
Perf-Dashboard	70.06 ± 2.93 (4.2%)	72.39 ± 12.18 (16.8%)	0.97

[Alex313031]

@onegreyonewhite You should be comparing identical versions. Using M124 for Chrome is unfair.

For example, M123 got a nice speed boost from upstream over M122. So comparing the two would be unfair to the M122 build.

[onegreyonewhite]

@onegreyonewhite You should be comparing identical versions. Using M124 for Chrome is unfair.

For example, M123 got a nice speed boost from upstream over M122. So comparing the two would be unfair to the M122 build.

I don't agree with you, because users uses the latest available version. So now you're behind one version. Users should know current difference in performance.

Anyway, now I hold current chrome version and I'm going to wait 124 version of Thorium for the fairest comparing.

@Alex313031 you should know that I like thorium. I think you make really useful job. I highly respect you for your work. This is an impartial comparison.

[onegreyonewhite]

Hi, everyone!

As you can see from the results below, the official Debian packages for Google Chrome often show much better performance. In situations where Google extensions and services are running, you can occasionally see better results with Thorium for older technologies like ES5. However, anything built with more current JavaScript standards shows significant or slight performance degradation.

I would look into which rendering acceleration flags can be used. For example, enabling GPU rendering and zero-copy significantly speeds up Thorium compared to Chrome (the tests already include these flags, so the difference isn’t as dramatic without "powered anon"). It might be worth running individual tests to pinpoint where the slowdown occurs and see if it's possible to address it.

Additionally, it would be beneficial to focus on more current code porting from Chromium. Yes, this is the most challenging part, but it actually provides the greatest speed boost.

Thorium - 124.0.6367.218
Chrome - 124.0.6367.207 (older but same)

Test	Thorium (ms)	Chrome (ms)	Ratio (Thorium/Chrome)
Geomean	90.43 ± 3.53 (3.9%)	84.55 ± 2.53 (3.0%)	1.07
TodoMVC-JavaScript-ES5	107.47 ± 5.54 (5.2%)	118.81 ± 13.46 (11.3%)	0.90
TodoMVC-JavaScript-ES6-Webpack-Complex-DOM	149.62 ± 4.87 (3.3%)	143.59 ± 9.05 (6.3%)	1.04
TodoMVC-WebComponents	37.60 ± 2.99 (8.0%)	35.29 ± 5.59 (15.8%)	1.07
TodoMVC-React-Complex-DOM	98.33 ± 20.18 (20.5%)	86.43 ± 6.81 (7.9%)	1.14
TodoMVC-React-Redux	88.27 ± 6.68 (7.6%)	95.53 ± 17.08 (17.9%)	0.92
TodoMVC-Backbone	59.78 ± 2.30 (3.9%)	64.86 ± 6.14 (9.5%)	0.92
TodoMVC-Angular-Complex-DOM	94.68 ± 16.17 (17.1%)	75.27 ± 5.11 (6.8%)	1.26
TodoMVC-Vue	58.96 ± 6.95 (11.8%)	51.35 ± 5.86 (11.4%)	1.15
TodoMVC-jQuery	362.98 ± 23.36 (6.4%)	329.75 ± 21.95 (6.7%)	1.10
TodoMVC-Preact-Complex-DOM	37.39 ± 5.27 (14.1%)	30.38 ± 2.77 (9.1%)	1.23
TodoMVC-Svelte-Complex-DOM	37.91 ± 3.65 (9.6%)	33.23 ± 4.10 (12.3%)	1.14
TodoMVC-Lit-Complex-DOM	42.44 ± 2.82 (6.6%)	39.37 ± 1.23 (3.1%)	1.08
NewsSite-Next	214.35 ± 10.65 (5.0%)	195.58 ± 7.58 (3.9%)	1.10
NewsSite-Nuxt	174.61 ± 8.29 (4.7%)	158.13 ± 5.30 (3.3%)	1.10
Editor-CodeMirror	44.34 ± 2.54 (5.7%)	43.72 ± 7.86 (18.0%)	1.01
Editor-TipTap	143.13 ± 15.03 (10.5%)	137.13 ± 9.98 (7.3%)	1.04
Charts-observable-plot	96.70 ± 7.93 (8.2%)	101.59 ± 7.42 (7.3%)	0.95
Charts-chartjs	119.93 ± 13.88 (11.6%)	109.07 ± 6.42 (5.9%)	1.10
React-Stockcharts-SVG	177.51 ± 13.75 (7.7%)	155.21 ± 11.39 (7.3%)	1.14
Perf-Dashboard	80.86 ± 3.46 (4.3%)	77.05 ± 4.97 (6.4%)	1.05

Actual versions without access to google account and without any extensions (I call it "powered anon") and same versions:

Thorium - 124.0.6367.218
Chrome - 124.0.6367.207 (older but same)

Test	Thorium (ms)	Chrome (ms)	Ratio (Thorium/Chrome)
Geomean	76.24 ± 3.20 (4.2%)	66.73 ± 1.84 (2.8%)	1.14
TodoMVC-JavaScript-ES5	89.20 ± 8.31 (9.3%)	80.54 ± 7.20 (8.9%)	1.11
TodoMVC-JavaScript-ES6-Webpack-Complex-DOM	114.04 ± 7.98 (7.0%)	102.40 ± 9.87 (9.6%)	1.11
TodoMVC-WebComponents	30.80 ± 1.42 (4.6%)	27.99 ± 0.75 (2.7%)	1.10
TodoMVC-React-Complex-DOM	79.85 ± 6.04 (7.6%)	69.09 ± 4.35 (6.3%)	1.16
TodoMVC-React-Redux	82.57 ± 6.25 (7.6%)	69.95 ± 1.97 (2.8%)	1.18
TodoMVC-Backbone	52.89 ± 2.05 (3.9%)	46.43 ± 1.58 (3.4%)	1.14
TodoMVC-Angular-Complex-DOM	74.23 ± 4.19 (5.6%)	65.90 ± 5.45 (8.3%)	1.13
TodoMVC-Vue	48.43 ± 4.02 (8.3%)	41.94 ± 2.90 (6.9%)	1.15
TodoMVC-jQuery	280.94 ± 41.42 (14.7%)	188.39 ± 11.79 (6.3%)	1.49
TodoMVC-Preact-Complex-DOM	30.64 ± 5.29 (17.3%)	24.66 ± 1.22 (5.0%)	1.24
TodoMVC-Svelte-Complex-DOM	28.88 ± 4.90 (17.0%)	24.38 ± 4.56 (18.7%)	1.18
TodoMVC-Lit-Complex-DOM	38.43 ± 2.89 (7.5%)	36.33 ± 5.60 (15.4%)	1.06
NewsSite-Next	189.43 ± 11.96 (6.3%)	158.49 ± 4.55 (2.9%)	1.20
NewsSite-Nuxt	146.84 ± 3.99 (2.7%)	141.83 ± 7.40 (5.2%)	1.04
Editor-CodeMirror	42.74 ± 6.45 (15.1%)	35.92 ± 5.95 (16.6%)	1.19
Editor-TipTap	149.36 ± 47.73 (32.0%)	115.46 ± 5.59 (4.8%)	1.29
Charts-observable-plot	79.13 ± 6.22 (7.9%)	73.37 ± 5.41 (7.4%)	1.08
Charts-chartjs	99.85 ± 10.65 (10.7%)	98.75 ± 8.19 (8.3%)	1.01
React-Stockcharts-SVG	148.92 ± 8.98 (6.0%)	133.58 ± 7.49 (5.6%)	1.11
Perf-Dashboard	65.68 ± 1.83 (2.8%)	64.20 ± 4.30 (6.7%)	1.02

Current available stable versions on ubuntu with google services and enabled extensions:

Thorium - 124.0.6367.218
Chrome - 125.0.6422.60 (newer)

Test	Thorium (ms)	Chrome (ms)	Ratio (Thorium/Chrome)
Geomean	90.43 ± 3.53 (3.9%)	80.76 ± 1.91 (2.4%)	1.12
TodoMVC-JavaScript-ES5	107.47 ± 5.54 (5.2%)	113.55 ± 21.31 (18.8%)	0.95
TodoMVC-JavaScript-ES6-Webpack-Complex-DOM	149.62 ± 4.87 (3.3%)	141.97 ± 6.70 (4.7%)	1.05
TodoMVC-WebComponents	37.60 ± 2.99 (8.0%)	32.39 ± 3.82 (11.8%)	1.16
TodoMVC-React-Complex-DOM	98.33 ± 20.18 (20.5%)	79.90 ± 5.95 (7.5%)	1.23
TodoMVC-React-Redux	88.27 ± 6.68 (7.6%)	88.11 ± 9.18 (10.4%)	1.00
TodoMVC-Backbone	59.78 ± 2.30 (3.9%)	59.95 ± 4.62 (7.7%)	1.00
TodoMVC-Angular-Complex-DOM	94.68 ± 16.17 (17.1%)	75.22 ± 5.24 (7.0%)	1.26
TodoMVC-Vue	58.96 ± 6.95 (11.8%)	55.28 ± 7.18 (13.0%)	1.07
TodoMVC-jQuery	362.98 ± 23.36 (6.4%)	332.88 ± 16.17 (4.9%)	1.09
TodoMVC-Preact-Complex-DOM	37.39 ± 5.27 (14.1%)	27.93 ± 4.02 (14.4%)	1.34
TodoMVC-Svelte-Complex-DOM	37.91 ± 3.65 (9.6%)	27.76 ± 1.54 (5.6%)	1.37
TodoMVC-Lit-Complex-DOM	42.44 ± 2.82 (6.6%)	37.73 ± 1.99 (5.3%)	1.13
NewsSite-Next	214.35 ± 10.65 (5.0%)	186.28 ± 8.21 (4.4%)	1.15
NewsSite-Nuxt	174.61 ± 8.29 (4.7%)	155.83 ± 5.08 (3.3%)	1.12
Editor-CodeMirror	44.34 ± 2.54 (5.7%)	51.95 ± 9.18 (17.7%)	0.85
Editor-TipTap	143.13 ± 15.03 (10.5%)	127.89 ± 4.43 (3.5%)	1.12
Charts-observable-plot	96.70 ± 7.93 (8.2%)	91.96 ± 6.53 (7.1%)	1.05
Charts-chartjs	119.93 ± 13.88 (11.6%)	95.35 ± 4.89 (5.1%)	1.26
React-Stockcharts-SVG	177.51 ± 13.75 (7.7%)	152.92 ± 5.74 (3.8%)	1.16
Perf-Dashboard	80.86 ± 3.46 (4.3%)	68.80 ± 2.10 (3.1%)	1.18

Actual versions without access to google account and without any extensions (I call it "powered anon"):

Thorium - 124.0.6367.218
Chrome - 125.0.6422.60 (newer)

Test	Thorium (ms)	Chrome (ms)	Ratio (Thorium/Chrome)
Geomean	76.24 ± 3.20 (4.2%)	62.67 ± 2.41 (3.9%)	1.22
TodoMVC-JavaScript-ES5	89.20 ± 8.31 (9.3%)	69.97 ± 8.14 (11.6%)	1.28
TodoMVC-JavaScript-ES6-Webpack-Complex-DOM	114.04 ± 7.98 (7.0%)	93.34 ± 5.82 (6.2%)	1.22
TodoMVC-WebComponents	30.80 ± 1.42 (4.6%)	27.25 ± 0.91 (3.3%)	1.13
TodoMVC-React-Complex-DOM	79.85 ± 6.04 (7.6%)	67.29 ± 6.00 (8.9%)	1.19
TodoMVC-React-Redux	82.57 ± 6.25 (7.6%)	67.64 ± 4.38 (6.5%)	1.22
TodoMVC-Backbone	52.89 ± 2.05 (3.9%)	45.09 ± 1.80 (4.0%)	1.17
TodoMVC-Angular-Complex-DOM	74.23 ± 4.19 (5.6%)	63.20 ± 6.25 (9.9%)	1.17
TodoMVC-Vue	48.43 ± 4.02 (8.3%)	38.25 ± 1.56 (4.1%)	1.27
TodoMVC-jQuery	280.94 ± 41.42 (14.7%)	193.06 ± 9.12 (4.7%)	1.45
TodoMVC-Preact-Complex-DOM	30.64 ± 5.29 (17.3%)	28.54 ± 5.61 (19.7%)	1.07
TodoMVC-Svelte-Complex-DOM	28.88 ± 4.90 (17.0%)	20.18 ± 1.97 (9.7%)	1.43
TodoMVC-Lit-Complex-DOM	38.43 ± 2.89 (7.5%)	32.15 ± 0.99 (3.1%)	1.20
NewsSite-Next	189.43 ± 11.96 (6.3%)	158.27 ± 5.49 (3.5%)	1.20
NewsSite-Nuxt	146.84 ± 3.99 (2.7%)	139.56 ± 5.78 (4.1%)	1.05
Editor-CodeMirror	42.74 ± 6.45 (15.1%)	33.51 ± 6.70 (20.0%)	1.28
Editor-TipTap	149.36 ± 47.73 (32.0%)	108.66 ± 7.41 (6.8%)	1.37
Charts-observable-plot	79.13 ± 6.22 (7.9%)	68.72 ± 5.19 (7.6%)	1.15
Charts-chartjs	99.85 ± 10.65 (10.7%)	79.14 ± 12.34 (15.6%)	1.26
React-Stockcharts-SVG	148.92 ± 8.98 (6.0%)	126.21 ± 5.16 (4.1%)	1.18
Perf-Dashboard	65.68 ± 1.83 (2.8%)	56.08 ± 2.01 (3.6%)	1.17