Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improved performance benchmark #168

Open
wants to merge 17 commits into
base: main
Choose a base branch
from

Conversation

rafaelsc
Copy link

  • Add many new benchmarks
    • with many calls, with many classes
    • Singleton, Transient, Scoped, Mixed, Complex with simple and deep dependences
  • Add all benchmark results to 'docs/benchmark'
  • Update Readme benchmark results.

Some new benchmarks show that MEDI is still faster than jab in some cases.

@pakrym
Copy link
Owner

pakrym commented Jan 21, 2024

Hi @rafaelsc, thank you for your work. I'm a bit swamped by work so it might take me a bit to get through these benchmarks.


| Method | Mean | Error | StdDev | Ratio | RatioSD |
|-------- |-------------:|--------------:|------------:|----------:|---------:|
| **Jab** | **279.6 ns** | **154.26 ns** | **8.46 ns** | **1.00** | **0.00** |
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is quite interesting. Do you have an idea why complex resolution is so much slower in Jab?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to find it and didn't understand why yet.
Some could be explained as the lock {}, but I removed it and didn't see a big improvement. See #169

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I managed to beat the MEDI performance for this case.

  • Removing locks and using LazyInitializer.EnsureInitialized
  • Removing the TryAddDisposable() - (this could be done in compilation time instead of runtime)
  • Replace all this.GetService<T>() for ((IServiceProvider<T>)this).GetService()
Method NumbersOfCalls NumbersOfClasses Mean Error StdDev Ratio RatioSD Gen0 Gen1 Allocated Alloc Ratio
Jab 1 1 270.81 ns 63.19 ns 3.463 ns 1.00 0.00 0.0563 - 472 B 1.00
Improved_Jab 1 1 92.74 ns 65.84 ns 3.609 ns 0.34 0.01 0.0736 - 616 B 1.31
MEDI 1 1 157.47 ns 37.13 ns 2.035 ns 0.58 0.01 0.0870 0.0002 728 B 1.54
Jab 1 2 543.88 ns 229.26 ns 12.566 ns 1.00 0.00 0.1059 - 888 B 1.00
Improved_Jab 1 2 255.73 ns 213.70 ns 11.714 ns 0.47 0.03 0.1402 - 1176 B 1.32
MEDI 1 2 276.63 ns 129.49 ns 7.098 ns 0.51 0.02 0.1364 0.0005 1144 B 1.29
Jab 1 3 814.39 ns 65.88 ns 3.611 ns 1.00 0.00 0.1554 - 1304 B 1.00
Improved_Jab 1 3 389.66 ns 204.35 ns 11.201 ns 0.48 0.01 0.2074 0.0010 1736 B 1.33
MEDI 1 3 397.50 ns 134.14 ns 7.353 ns 0.49 0.01 0.1864 0.0010 1560 B 1.20
Jab 10 1 2,811.38 ns 127.55 ns 6.992 ns 1.00 0.00 0.5608 - 4720 B 1.00
Improved_Jab 10 1 932.41 ns 132.33 ns 7.253 ns 0.33 0.00 0.7362 - 6160 B 1.31
MEDI 10 1 1,517.62 ns 697.80 ns 38.249 ns 0.54 0.01 0.8698 0.0019 7280 B 1.54
Jab 10 2 5,394.83 ns 1,473.69 ns 80.778 ns 1.00 0.00 1.0605 - 8880 B 1.00
Improved_Jab 10 2 2,563.65 ns 542.04 ns 29.711 ns 0.48 0.01 1.4038 - 11760 B 1.32
MEDI 10 2 2,820.40 ns 1,988.28 ns 108.984 ns 0.52 0.03 1.3657 0.0038 11440 B 1.29
Jab 10 3 8,515.73 ns 1,780.40 ns 97.590 ns 1.00 0.00 1.5564 - 13040 B 1.00
Improved_Jab 10 3 3,791.29 ns 348.95 ns 19.127 ns 0.45 0.01 2.0752 0.0076 17360 B 1.33
MEDI 10 3 4,014.64 ns 1,245.71 ns 68.282 ns 0.47 0.00 1.8616 0.0076 15600 B 1.20
Jab 100 1 28,236.83 ns 12,613.72 ns 691.401 ns 1.00 0.00 5.6152 - 47200 B 1.00
Improved_Jab 100 1 10,499.51 ns 15,496.89 ns 849.437 ns 0.37 0.02 7.3547 - 61600 B 1.31
MEDI 100 1 16,673.16 ns 3,621.32 ns 198.497 ns 0.59 0.02 8.6975 0.0153 72800 B 1.54
Jab 100 2 56,434.78 ns 4,705.42 ns 257.920 ns 1.00 0.00 10.5591 - 88800 B 1.00
Improved_Jab 100 2 25,188.64 ns 6,705.33 ns 367.542 ns 0.45 0.00 14.0381 0.0305 117600 B 1.32
MEDI 100 2 27,463.55 ns 15,492.67 ns 849.206 ns 0.49 0.02 13.6719 0.0305 114400 B 1.29
Jab 100 3 84,041.73 ns 35,261.40 ns 1,932.796 ns 1.00 0.00 15.5029 - 130400 B 1.00
Improved_Jab 100 3 38,046.35 ns 11,730.31 ns 642.978 ns 0.45 0.02 20.7520 0.0610 173600 B 1.33
MEDI 100 3 42,855.80 ns 46,857.75 ns 2,568.432 ns 0.51 0.04 18.6157 0.0610 156000 B 1.20

| **Jab** | **10** | **100** | **2,100.85 ns** | **2,662.20 ns** | **145.924 ns** | **1.00** | **0.00** | **1.0223** | **-** | **8560 B** | **1.00** |
| MEDI | 10 | 100 | 855.43 ns | 178.70 ns | 9.795 ns | 0.41 | 0.02 | 0.4015 | - | 3360 B | 0.39 |
| | | | | | | | | | | | |
| **Jab** | **100** | **1** | **19,231.59 ns** | **4,371.35 ns** | **239.608 ns** | **1.00** | **0.00** | **10.2234** | **0.0305** | **85600 B** | **1.00** |
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quite surprised Jab allocates so much more..

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants