Regressions in System.Numerics.Tests.Perf_Vector3 #86033

performanceautofiler · 2023-05-09T19:19:13Z

The Vector3 / Matrix regressions are "real" [see note below]. The rest are noise.

performanceautofiler · 2023-05-09T19:19:15Z

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	46693f249bdab5c0d4d008df1cea67d51a38d144
Compare	754d9ecdafb293c6532fc2f00cd701e6b2c485a8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Numerics.Tests.Perf_Vector3

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
LerpBenchmark - Duration of single invocation	0.02 ns	5.87 ns	339.21	0.23	True
MultiplyOperatorBenchmark - Duration of single invocation	0.07 ns	5.88 ns	83.92	0.17	True
ClampBenchmark - Duration of single invocation	0.35 ns	5.86 ns	16.82	0.09	True
UnitYBenchmark - Duration of single invocation	0.01 ns	5.90 ns	488.13	0.17	True
MaxBenchmark - Duration of single invocation	0.35 ns	5.90 ns	16.98	0.10	True
SquareRootBenchmark - Duration of single invocation	0.07 ns	5.84 ns	84.37	0.16	False
CreateFromScalarXYZBenchmark - Duration of single invocation	0.07 ns	5.88 ns	84.08	0.16	True
TransformByMatrix4x4Benchmark - Duration of single invocation	0.52 ns	6.14 ns	11.74	0.06	True
AddOperatorBenchmark - Duration of single invocation	0.07 ns	5.88 ns	85.46	0.10	True
DivideByScalarBenchmark - Duration of single invocation	0.07 ns	5.88 ns	84.49	0.13	True
OneBenchmark - Duration of single invocation	0.01 ns	5.90 ns	400.49	0.16	True
MultiplyByScalarBenchmark - Duration of single invocation	0.07 ns	5.88 ns	84.36	0.18	True
MultiplyFunctionBenchmark - Duration of single invocation	0.07 ns	5.88 ns	84.44	0.25	True
NegateOperatorBenchmark - Duration of single invocation	0.02 ns	5.80 ns	323.83	0.15	True
CrossBenchmark - Duration of single invocation	0.07 ns	5.88 ns	84.53	0.06	True
AbsBenchmark - Duration of single invocation	0.07 ns	5.85 ns	84.02	0.20	True
SubtractOperatorBenchmark - Duration of single invocation	0.07 ns	5.88 ns	85.80	0.14	True
CreateFromVector2WithScalarBenchmark - Duration of single invocation	0.07 ns	5.88 ns	84.49	0.16	True
CreateFromScalar - Duration of single invocation	0.07 ns	5.88 ns	82.56	0.16	True
TransformByQuaternionBenchmark - Duration of single invocation	0.07 ns	5.88 ns	84.79	0.10	True
ZeroBenchmark - Duration of single invocation	0.07 ns	5.82 ns	82.18	0.11	True
NormalizeBenchmark - Duration of single invocation	1.00 ns	6.89 ns	6.88	0.02	False
ReflectBenchmark - Duration of single invocation	1.10 ns	7.05 ns	6.43	0.26	False
DivideByScalarOperatorBenchmark - Duration of single invocation	0.07 ns	5.88 ns	84.54	0.12	True
MinBenchmark - Duration of single invocation	0.35 ns	5.89 ns	16.94	0.09	True
NegateBenchmark - Duration of single invocation	0.02 ns	5.79 ns	331.93	0.14	True
TransformNormalByMatrix4x4Benchmark - Duration of single invocation	0.37 ns	6.11 ns	16.71	0.06	True
SubtractFunctionBenchmark - Duration of single invocation	0.07 ns	5.88 ns	84.71	0.08	True
DivideByVector3Benchmark - Duration of single invocation	0.07 ns	5.88 ns	83.60	0.16	True
MultiplyByScalarOperatorBenchmark - Duration of single invocation	0.07 ns	5.77 ns	82.97	0.19	True
DivideByVector3OperatorBenchmark - Duration of single invocation	0.07 ns	5.83 ns	84.04	0.20	True
UnitZBenchmark - Duration of single invocation	0.01 ns	5.89 ns	397.36	0.18	True
UnitXBenchmark - Duration of single invocation	0.02 ns	5.90 ns	363.44	0.20	True
AddFunctionBenchmark - Duration of single invocation	0.07 ns	5.88 ns	84.45	0.14	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector3*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_Vector3.LerpBenchmark

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.869150004819592 > 0.018312349235285918.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -944.4342039761011 (T) = (0 -5.862976307226863) / Math.Sqrt((0.0006858823953708886 / (32)) + (0.00024707359093606126 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -120.27546826008222 = (0.04834428917358103 - 5.862976307226863) / 0.04834428917358103 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.MultiplyOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.876990464532814 > 0.07304269083065675.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -1251.9276453796665 (T) = (0 -5.87763613629927) / Math.Sqrt((0.0006833943713813057 / (32)) + (5.953613343853747E-06 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -151.1359607286263 = (0.038634101419213765 - 5.87763613629927) / 0.038634101419213765 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.ClampBenchmark

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.864890321803821 > 0.3656433094622453.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -226.68943503486528 (T) = (0 -5.865325941857435) / Math.Sqrt((0.019921172508435874 / (32)) + (7.90342730684673E-05 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -30.63873369489182 = (0.18538434560686642 - 5.865325941857435) / 0.18538434560686642 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.UnitYBenchmark

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.898701401582163 > 0.015998103161833643.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -987.6748927522145 (T) = (0 -5.890843414561283) / Math.Sqrt((0.0007526820564431713 / (32)) + (0.00017228055686346888 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -124.01265401373828 = (0.0471219770593296 - 5.890843414561283) / 0.0471219770593296 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.MaxBenchmark

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.901074407607987 > 0.36545949326040555.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -195.20890731975067 (T) = (0 -5.89274180641211) / Math.Sqrt((0.02716493320912055 / (32)) + (0.00024006374156668284 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -37.82434435959474 = (0.15177955748158886 - 5.89274180641211) / 0.15177955748158886 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.SquareRootBenchmark

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.839573689110131 > 0.0728942768410056.
IsChangePoint: Marked as a change because one of 3/5/2023 11:54:18 PM, 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -1026.5104539963963 (T) = (0 -5.834547999972641) / Math.Sqrt((0.0006835914770895175 / (32)) + (0.0001577547285622658 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -149.77404265819274 = (0.038697297605793185 - 5.834547999972641) / 0.038697297605793185 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.CreateFromScalarXYZBenchmark

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.8787069538787895 > 0.07305520225811438.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -973.5440344975609 (T) = (0 -5.874360450776458) / Math.Sqrt((0.0006873437011912633 / (32)) + (0.00021687081455885526 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -152.91873617101453 = (0.03816533709222788 - 5.874360450776458) / 0.03816533709222788 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.TransformByMatrix4x4Benchmark

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.140247699428579 > 0.5483169863415611.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -609.7837253013909 (T) = (0 -6.10920643518911) / Math.Sqrt((7.669961482110583E-05 / (32)) + (0.001218624621029532 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -10.47301320251901 = (0.5324849128429291 - 6.10920643518911) / 0.5324849128429291 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.AddOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.878641171917319 > 0.0735640217583538.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -1036.0226120731172 (T) = (0 -5.873448826543983) / Math.Sqrt((0.0006830450015523125 / (32)) + (0.00015558988936146458 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -150.70044571644254 = (0.03871741311507149 - 5.873448826543983) / 0.03871741311507149 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.DivideByScalarBenchmark

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.880242463258705 > 0.07322628120595122.
IsChangePoint: Marked as a change because one of 2/21/2023 5:46:41 AM, 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -867.6197463694291 (T) = (0 -5.872587524740816) / Math.Sqrt((0.0006783883425966324 / (32)) + (0.00036019548140429816 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -150.82247278380072 = (0.03868062097172886 - 5.872587524740816) / 0.03868062097172886 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.OneBenchmark

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.896606206073583 > 0.016072529047701376.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -896.2109073975471 (T) = (0 -5.888186471534173) / Math.Sqrt((0.000731731708079712 / (30)) + (0.0002711167610334674 / (15))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (15) - 2, .025) and -121.72802240980248 = (0.04797752262211857 - 5.888186471534173) / 0.04797752262211857 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.MultiplyByScalarBenchmark

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.877680953306362 > 0.07311508284878573.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -1246.8595007445642 (T) = (0 -5.878660004229877) / Math.Sqrt((0.0006979234018854897 / (32)) + (1.8930343071024386E-06 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -150.34696081611276 = (0.038842273227887775 - 5.878660004229877) / 0.038842273227887775 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.MultiplyFunctionBenchmark

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.877067994952507 > 0.07307238289666912.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -899.7041376911515 (T) = (0 -5.8700203542510385) / Math.Sqrt((0.0007133192357873346 / (30)) + (0.0002736996046417852 / (15))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (15) - 2, .025) and -155.08004149072048 = (0.037609038914818795 - 5.8700203542510385) / 0.037609038914818795 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.NegateOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.796522834894209 > 0.018175359281172778.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -523.119652657894 (T) = (0 -5.8237278939478845) / Math.Sqrt((0.0006878122180734999 / (32)) + (0.0015058163280997224 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -119.12608206692087 = (0.04848012849285763 - 5.8237278939478845) / 0.04848012849285763 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.CrossBenchmark

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.883040439446438 > 0.07302769042102565.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -1056.835045179588 (T) = (0 -5.873407882066947) / Math.Sqrt((0.0006733996594590206 / (32)) + (0.00014157388169669808 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -151.28599264039482 = (0.03856827394451371 - 5.873407882066947) / 0.03856827394451371 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.AbsBenchmark

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.851707315170469 > 0.07321606556881202.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -995.6916035720841 (T) = (0 -5.872857523815003) / Math.Sqrt((0.0006848510967514247 / (32)) + (0.0001939542903277394 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -150.516982107443 = (0.03876039135765303 - 5.872857523815003) / 0.03876039135765303 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.SubtractOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.8769214897303375 > 0.0727634713983128.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -930.1587605536108 (T) = (0 -5.870055541618295) / Math.Sqrt((0.0006768151094873978 / (32)) + (0.0002722953202488746 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -150.84627351784758 = (0.03865788343451409 - 5.870055541618295) / 0.03865788343451409 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.CreateFromVector2WithScalarBenchmark

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.878910149458185 > 0.07317725363082354.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -1265.2596439718056 (T) = (0 -5.87744526340715) / Math.Sqrt((0.000674831897790708 / (32)) + (3.1067701376873296E-06 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -151.14786242886615 = (0.038629824761126946 - 5.87744526340715) / 0.038629824761126946 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.CreateFromScalar

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.878912575009822 > 0.07306261400073712.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -979.0746569082388 (T) = (0 -5.875152023177565) / Math.Sqrt((0.0006874726214003212 / (32)) + (0.00021085237224542953 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -152.28746885267586 = (0.038327673273959244 - 5.875152023177565) / 0.038327673273959244 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.TransformByQuaternionBenchmark

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.877413803909016 > 0.07290047603622628.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -865.7705702154833 (T) = (0 -5.868975216295775) / Math.Sqrt((0.0006706199206588235 / (32)) + (0.00034148749886644253 / (14))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (14) - 2, .025) and -150.4869731787359 = (0.03874244163140754 - 5.868975216295775) / 0.03874244163140754 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.ZeroBenchmark

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.817672423281147 > 0.07323143315874059.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -975.1353784020473 (T) = (0 -5.844992430122703) / Math.Sqrt((0.0006737250328464663 / (31)) + (0.00020603209122448118 / (15))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (15) - 2, .025) and -154.7501897536511 = (0.03752799556371445 - 5.844992430122703) / 0.03752799556371445 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.NormalizeBenchmark

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.89473821772923 > 1.0513962925425318.
IsChangePoint: Marked as a change because one of 3/5/2023 11:54:18 PM, 4/10/2023 1:33:42 PM, 4/12/2023 8:39:35 PM, 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -611.956954603982 (T) = (0 -6.946335182730175) / Math.Sqrt((0.0002667775394577814 / (32)) + (0.0012821761122139498 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -5.816633570925491 = (1.019027223695563 - 6.946335182730175) / 1.019027223695563 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.ReflectBenchmark

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.0519147164968246 > 1.151442657313142.
IsChangePoint: Marked as a change because one of 2/24/2023 9:01:57 AM, 3/8/2023 9:36:36 PM, 4/10/2023 1:33:42 PM, 4/26/2023 6:27:29 AM, 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -784.3400732519124 (T) = (0 -7.046962638571843) / Math.Sqrt((7.698617531725888E-05 / (31)) + (0.0008291701394989137 / (15))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (15) - 2, .025) and -5.489489111550428 = (1.0859040700182678 - 7.046962638571843) / 1.0859040700182678 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.DivideByScalarOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.87792465612683 > 0.07305427294814419.
IsChangePoint: Marked as a change because one of 2/21/2023 5:46:41 AM, 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -1120.4169713406748 (T) = (0 -5.875870235565016) / Math.Sqrt((0.0006862366902874316 / (32)) + (8.547049092547514E-05 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -151.13122430706363 = (0.03862369649839328 - 5.875870235565016) / 0.03862369649839328 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.MinBenchmark

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.893232521868853 > 0.3652867985603465.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -194.60349276351528 (T) = (0 -5.889165724437448) / Math.Sqrt((0.02715932223999465 / (32)) + (0.00030757887509781656 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -37.81740989139164 = (0.15171454615119648 - 5.889165724437448) / 0.15171454615119648 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.NegateBenchmark

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.793889756356246 > 0.018280053995381816.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -521.5638609914137 (T) = (0 -5.833666989488944) / Math.Sqrt((0.0006755209130848179 / (32)) + (0.0015290089740657743 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -120.00441164571247 = (0.04821036613581723 - 5.833666989488944) / 0.04821036613581723 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.TransformNormalByMatrix4x4Benchmark

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.11338556268992 > 0.38412432195518864.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -502.9125740431132 (T) = (0 -6.077904026731327) / Math.Sqrt((0.0012273680281705117 / (32)) + (0.0013410535208638063 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -14.447294977846763 = (0.3934607344164629 - 6.077904026731327) / 0.3934607344164629 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.SubtractFunctionBenchmark

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.883300020161512 > 0.07293692564982082.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -1143.2956875963375 (T) = (0 -5.876289150424712) / Math.Sqrt((0.000675569419684778 / (32)) + (7.440166826014467E-05 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -151.33652171233743 = (0.03857439492757437 - 5.876289150424712) / 0.03857439492757437 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.DivideByVector3Benchmark

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.875222587841023 > 0.07314769830295316.
IsChangePoint: Marked as a change because one of 2/21/2023 5:46:41 AM, 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -925.3712134234198 (T) = (0 -5.870034178295591) / Math.Sqrt((0.0006759642727527337 / (32)) + (0.00027879526104741586 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -150.64255782290678 = (0.03870967532182366 - 5.870034178295591) / 0.03870967532182366 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.MultiplyByScalarOperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.771932770963405 > 0.07318150462131054.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -655.6350521058314 (T) = (0 -5.867401935177729) / Math.Sqrt((0.0006878411989643787 / (32)) + (0.0008631600556282106 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -151.20091351277716 = (0.038550372660445066 - 5.867401935177729) / 0.038550372660445066 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.DivideByVector3OperatorBenchmark

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.830584181022009 > 0.07292126398945532.
IsChangePoint: Marked as a change because one of 2/21/2023 5:46:41 AM, 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -857.5116601607913 (T) = (0 -5.867530985674923) / Math.Sqrt((0.0006672305649292404 / (32)) + (0.00038031403458718075 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -150.83232031124976 = (0.03864480878410299 - 5.867530985674923) / 0.03864480878410299 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.UnitZBenchmark

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.892539746256579 > 0.01611261687194399.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -907.1339470948907 (T) = (0 -5.88738231926318) / Math.Sqrt((0.0007752817584031429 / (32)) + (0.0002583506922033907 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -124.1669249521977 = (0.04703624636869221 - 5.88738231926318) / 0.04703624636869221 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.UnitXBenchmark

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.8953487499894015 > 0.015594001411952363.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -1088.3488405914 (T) = (0 -5.893003071069317) / Math.Sqrt((0.0007323916448830455 / (32)) + (8.939435639805005E-05 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -122.90590710495998 = (0.047560307726712235 - 5.893003071069317) / 0.047560307726712235 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.AddFunctionBenchmark

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.878560465852833 > 0.07292669406212862.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -1254.8711620471543 (T) = (0 -5.8781288471227775) / Math.Sqrt((0.0006711043076802783 / (32)) + (1.0239821938633918E-05 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -151.1273154031193 = (0.03863953578321181 - 5.8781288471227775) / 0.03863953578321181 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-05-09T19:19:17Z

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	46693f249bdab5c0d4d008df1cea67d51a38d144
Compare	754d9ecdafb293c6532fc2f00cd701e6b2c485a8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Numerics.Tests.Perf_Matrix4x4

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
TranslationBenchmark - Duration of single invocation	2.38 ns	8.98 ns	3.77	0.00	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix4x4*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_Matrix4x4.TranslationBenchmark

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.981126298239065 > 2.503691519614902.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -4018.8639683067727 (T) = (0 -8.980880931361924) / Math.Sqrt((7.172580314311726E-05 / (32)) + (6.657471522791547E-06 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -2.7494382359752287 = (2.39526040066266 - 8.980880931361924) / 2.39526040066266 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	46693f249bdab5c0d4d008df1cea67d51a38d144
Compare	754d9ecdafb293c6532fc2f00cd701e6b2c485a8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.IterateFor<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ImmutableList - Duration of single invocation	5.87 μs	6.45 μs	1.10	0.04	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateFor&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateFor<Int32>.ImmutableList(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.452952242014742 > 5.9340199824583655.
IsChangePoint: Marked as a change because one of 2/23/2023 8:54:44 PM, 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 3/25/2023 12:18:49 AM, 4/19/2023 8:38:27 PM, 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -11.429915201718831 (T) = (0 -6396.431264303924) / Math.Sqrt((74052.00254322296 / (32)) + (5404.462046299951 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.10181944263165987 = (5805.335263485872 - 6396.431264303924) / 5805.335263485872 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	46693f249bdab5c0d4d008df1cea67d51a38d144
Compare	754d9ecdafb293c6532fc2f00cd701e6b2c485a8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
IsMatch - Duration of single invocation	43.47 ns	51.96 ns	1.20	0.07	False
IsMatch - Duration of single invocation	115.39 ns	130.59 ns	1.13	0.09	False
IsMatch - Duration of single invocation	39.60 ns	44.18 ns	1.12	0.06	False
IsMatch - Duration of single invocation	43.94 ns	52.27 ns	1.19	0.06	False
IsMatch - Duration of single invocation	112.76 ns	129.63 ns	1.15	0.12	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 12, Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 51.960027787898234 > 45.721373567418865.
IsChangePoint: Marked as a change because one of 2/20/2023 10:31:48 AM, 3/6/2023 11:14:52 AM, 3/17/2023 10:06:19 AM, 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -32.979630663910754 (T) = (0 -53.6807735065115) / Math.Sqrt((0.6965863550437492 / (32)) + (0.9179655864042894 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.21500901871832315 = (44.18137863959047 - 53.6807735065115) / 44.18137863959047 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 9, Options: None)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 130.5941208152741 > 121.55071955883.
IsChangePoint: Marked as a change because one of 2/21/2023 12:38:54 AM, 5/1/2023 3:42:23 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -17.3185697481287 (T) = (0 -129.29309525506574) / Math.Sqrt((2.538266623019449 / (13)) + (12.815172740087934 / (34))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (34) - 2, .025) and -0.11274409890482563 = (116.19301812727414 - 129.29309525506574) / 116.19301812727414 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 11, Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 44.18283685057445 > 41.65189774556145.
IsChangePoint: Marked as a change because one of 2/23/2023 8:54:44 PM, 3/6/2023 6:03:21 AM, 3/13/2023 12:04:12 PM, 4/8/2023 3:09:32 AM, 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -19.16832143183523 (T) = (0 -45.69479915803193) / Math.Sqrt((0.7636333858344254 / (32)) + (0.8636327939726038 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.13599043717924172 = (40.22463364348022 - 45.69479915803193) / 40.22463364348022 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 13, Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 52.27357293875085 > 45.6496814934083.
IsChangePoint: Marked as a change because one of 2/20/2023 10:31:48 AM, 3/6/2023 11:14:52 AM, 3/17/2023 8:17:18 AM, 4/23/2023 10:00:36 AM, 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -29.440294705423472 (T) = (0 -53.65170904564683) / Math.Sqrt((0.9582174250014176 / (32)) + (1.1050950706555827 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.21452696460327544 = (44.17498384910057 - 53.65170904564683) / 44.17498384910057 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 10, Options: None)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 129.63348189548915 > 118.01910768980073.
IsChangePoint: Marked as a change because one of 2/17/2023 9:53:52 PM, 2/20/2023 10:31:48 AM, 5/2/2023 9:04:55 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -7.414950897817189 (T) = (0 -127.92177978014954) / Math.Sqrt((33.471143810580024 / (19)) + (18.802123162019747 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.09940427783740677 = (116.35554123163796 - 127.92177978014954) / 116.35554123163796 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	46693f249bdab5c0d4d008df1cea67d51a38d144
Compare	754d9ecdafb293c6532fc2f00cd701e6b2c485a8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
Count - Duration of single invocation	28.88 μs	46.90 μs	1.62	0.04	False
Count - Duration of single invocation	28.45 ms	31.70 ms	1.11	0.20	False
Count - Duration of single invocation	29.55 ms	34.22 ms	1.16	0.17	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?m)^Sherlock Holmes|Sherlock Holmes$", Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 46.895924401197604 > 30.34608956412505.
IsChangePoint: Marked as a change because one of 2/16/2023 9:36:08 AM, 3/14/2023 8:42:09 PM, 3/20/2023 8:31:51 AM, 4/10/2023 1:33:42 PM, 4/25/2023 3:46:39 AM, 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -580.8560921975018 (T) = (0 -46875.840147391704) / Math.Sqrt((3902.472911183518 / (32)) + (12567.821171738133 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.6230976069758367 = (28880.481337614066 - 46875.840147391704) / 28880.481337614066 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\p{Ll}", Options: None)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 31.69744 > 29.96469984375.
IsChangePoint: Marked as a change because one of 2/24/2023 8:06:06 PM, 3/6/2023 11:14:52 AM, 3/28/2023 3:45:48 AM, 4/7/2023 7:53:47 AM, 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -20.80080263020417 (T) = (0 -32104501.622807022) / Math.Sqrt((27605546185.24313 / (32)) + (444559867252.49725 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.12758955045743078 = (28471797.747489896 - 32104501.622807022) / 28471797.747489896 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\p{L}", Options: None)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 34.2217725 > 30.97733034375.
IsChangePoint: Marked as a change because one of 3/14/2023 5:06:59 PM, 3/17/2023 9:25:32 PM, 4/18/2023 7:42:00 PM, 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -9.358233346038569 (T) = (0 -33744325.865079366) / Math.Sqrt((2716179884931.591 / (32)) + (976714266826 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.12033106082262301 = (30119959.220180858 - 33744325.865079366) / 30119959.220180858 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	46693f249bdab5c0d4d008df1cea67d51a38d144
Compare	754d9ecdafb293c6532fc2f00cd701e6b2c485a8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in Span.IndexerBench

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Ref - Duration of single invocation	576.84 ns	629.94 ns	1.09	0.04	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Span.IndexerBench*'

Payloads

Baseline
Compare

Histogram

Span.IndexerBench.Ref(length: 1024)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 629.9357309658976 > 605.8813569646004.
IsChangePoint: Marked as a change because one of 2/24/2023 9:01:57 AM, 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 4/9/2023 11:04:55 PM, 4/19/2023 8:38:27 PM, 4/25/2023 3:46:39 AM, 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -13.917713499274333 (T) = (0 -629.517133621924) / Math.Sqrt((325.0680366962239 / (32)) + (4.766773242178038 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.07707382103203109 = (584.4698119379905 - 629.517133621924) / 584.4698119379905 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	46693f249bdab5c0d4d008df1cea67d51a38d144
Compare	754d9ecdafb293c6532fc2f00cd701e6b2c485a8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.IterateFor<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ImmutableArray - Duration of single invocation	150.33 ns	291.67 ns	1.94	0.09	False
ImmutableSortedSet - Duration of single invocation	5.93 μs	6.45 μs	1.09	0.04	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateFor&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateFor<String>.ImmutableArray(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 291.6730825078758 > 158.020392353817.
IsChangePoint: Marked as a change because one of 3/17/2023 9:25:32 PM, 3/20/2023 8:31:51 AM, 4/9/2023 11:04:55 PM, 4/19/2023 8:38:27 PM, 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -13.042496684528572 (T) = (0 -291.4536473622062) / Math.Sqrt((2674.2148167017685 / (32)) + (0.020101835356277437 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.6923024321462486 = (172.2231451222182 - 291.4536473622062) / 172.2231451222182 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.IterateFor<String>.ImmutableSortedSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.452979633275906 > 6.148482490811257.
IsChangePoint: Marked as a change because one of 2/24/2023 9:01:57 AM, 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 3/25/2023 12:18:49 AM, 4/19/2023 8:38:27 PM, 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -9.334363076154773 (T) = (0 -6503.321610398701) / Math.Sqrt((67822.87966099926 / (32)) + (5802.273637545374 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.07741930817621136 = (6036.017324960623 - 6503.321610398701) / 6036.017324960623 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	46693f249bdab5c0d4d008df1cea67d51a38d144
Compare	754d9ecdafb293c6532fc2f00cd701e6b2c485a8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.Span<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
StartsWith - Duration of single invocation	16.56 ns	21.46 ns	1.30	0.12	False
SequenceCompareTo - Duration of single invocation	29.18 ns	33.04 ns	1.13	0.01	False
IndexOfAnyFiveValues - Duration of single invocation	6.11 ns	7.71 ns	1.26	0.22	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Int32>.StartsWith(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.462592828339265 > 17.427186980538025.
IsChangePoint: Marked as a change because one of 2/24/2023 9:01:57 AM, 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 3/29/2023 12:02:44 PM, 4/9/2023 11:04:55 PM, 4/12/2023 8:39:35 PM, 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -11.60380505879388 (T) = (0 -20.93874008405969) / Math.Sqrt((3.508780415476016 / (32)) + (0.17785343576978144 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.23942500206542008 = (16.893914556481157 - 20.93874008405969) / 16.893914556481157 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.SequenceCompareTo(Size: 33)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 33.03512752007616 > 31.226824523340174.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -15.698582304689209 (T) = (0 -32.304988796558206) / Math.Sqrt((0.6902904705965442 / (32)) + (0.0498534634031049 / (14))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (14) - 2, .025) and -0.08346906256042858 = (29.816254024102744 - 32.304988796558206) / 29.816254024102744 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.IndexOfAnyFiveValues(Size: 4)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.713768216325434 > 6.458015042958341.
IsChangePoint: Marked as a change because one of 3/24/2023 2:55:56 PM, 4/9/2023 11:04:55 PM, 4/17/2023 6:40:18 PM, 4/19/2023 8:38:27 PM, 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -12.7658197771596 (T) = (0 -7.88570933342024) / Math.Sqrt((0.2921716111212383 / (32)) + (0.06968921647708845 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.23458177535466077 = (6.387352778761777 - 7.88570933342024) / 6.387352778761777 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	46693f249bdab5c0d4d008df1cea67d51a38d144
Compare	754d9ecdafb293c6532fc2f00cd701e6b2c485a8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Xml.Linq.Perf_XElement

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetElementWithNamespace - Duration of single invocation	63.46 ns	73.14 ns	1.15	0.10	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Xml.Linq.Perf_XElement*'

Payloads

Baseline
Compare

Histogram

System.Xml.Linq.Perf_XElement.GetElementWithNamespace

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 73.13933888980952 > 66.52807665630598.
IsChangePoint: Marked as a change because one of 3/17/2023 10:06:19 AM, 4/25/2023 3:46:39 AM, 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -7.617897309777486 (T) = (0 -73.57708870528762) / Math.Sqrt((22.039026320255832 / (32)) + (2.257513321586362 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.10478745169737104 = (66.59841093618995 - 73.57708870528762) / 66.59841093618995 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	46693f249bdab5c0d4d008df1cea67d51a38d144
Compare	754d9ecdafb293c6532fc2f00cd701e6b2c485a8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Net.Tests.Perf_WebUtility

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Decode_NoDecodingRequired - Duration of single invocation	64.41 ns	74.91 ns	1.16	0.18	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Net.Tests.Perf_WebUtility*'

Payloads

Baseline
Compare

Histogram

System.Net.Tests.Perf_WebUtility.Decode_NoDecodingRequired

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 74.90579874808483 > 67.19353926327624.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -17.694406700601256 (T) = (0 -75.32175369741172) / Math.Sqrt((6.208067782259942 / (32)) + (0.5052891258085961 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.12624644041608252 = (66.87857203755931 - 75.32175369741172) / 66.87857203755931 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-05-09T19:19:18Z

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	46693f249bdab5c0d4d008df1cea67d51a38d144
Compare	754d9ecdafb293c6532fc2f00cd701e6b2c485a8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Version

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToStringL - Duration of single invocation	50.47 ns	55.98 ns	1.11	0.05	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Version*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Version.ToStringL

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 55.98434886366171 > 53.0016507219182.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 4/10/2023 1:33:42 PM, 4/25/2023 6:46:56 PM, 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -16.59808163872543 (T) = (0 -62.27429033997593) / Math.Sqrt((0.7274296885552028 / (32)) + (6.987785757588002 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.22895812790404715 = (50.67242644481545 - 62.27429033997593) / 50.67242644481545 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	46693f249bdab5c0d4d008df1cea67d51a38d144
Compare	754d9ecdafb293c6532fc2f00cd701e6b2c485a8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
IP_IsMatch - Duration of single invocation	123.32 ns	134.57 ns	1.09	0.01	False
IP_IsNotMatch - Duration of single invocation	124.41 ns	134.21 ns	1.08	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsMatch(Options: IgnoreCase, Compiled)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 134.56729453264344 > 130.37772457887291.
IsChangePoint: Marked as a change because one of 3/25/2023 3:46:49 AM, 4/17/2023 7:48:00 AM, 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -54.97192211354177 (T) = (0 -136.41059189124945) / Math.Sqrt((0.3915378036513955 / (31)) + (0.6089594657687941 / (15))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (15) - 2, .025) and -0.10250411081595295 = (123.72796668331083 - 136.41059189124945) / 123.72796668331083 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsNotMatch(Options: IgnoreCase, Compiled)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 134.20628162136722 > 130.22719980666005.
IsChangePoint: Marked as a change because one of 3/25/2023 12:18:49 AM, 4/17/2023 6:40:18 PM, 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -39.49361310597608 (T) = (0 -136.7931895888444) / Math.Sqrt((0.4030849177862767 / (32)) + (1.492409509274024 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.10700283317640151 = (123.57076738126679 - 136.7931895888444) / 123.57076738126679 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	46693f249bdab5c0d4d008df1cea67d51a38d144
Compare	754d9ecdafb293c6532fc2f00cd701e6b2c485a8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Xml.Tests.Perf_XmlConvert

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
TimeSpan_ToString - Duration of single invocation	69.92 ns	76.01 ns	1.09	0.07	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Xml.Tests.Perf_XmlConvert*'

Payloads

Baseline
Compare

Histogram

System.Xml.Tests.Perf_XmlConvert.TimeSpan_ToString

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 76.00919205291297 > 73.33830727771489.
IsChangePoint: Marked as a change because one of 4/7/2023 3:57:10 PM, 4/9/2023 11:04:55 PM, 4/12/2023 8:39:35 PM, 4/17/2023 6:40:18 PM, 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -24.792786440649433 (T) = (0 -75.55618835174387) / Math.Sqrt((0.20932630109196346 / (32)) + (0.7052543250157105 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.08218050587435208 = (69.8184710790904 - 75.55618835174387) / 69.8184710790904 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	46693f249bdab5c0d4d008df1cea67d51a38d144
Compare	754d9ecdafb293c6532fc2f00cd701e6b2c485a8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.IterateForEach<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Queue - Duration of single invocation	1.59 μs	1.72 μs	1.08	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateForEach<Int32>.Queue(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.7231828652564527 > 1.67178249278314.
IsChangePoint: Marked as a change because one of 2/24/2023 9:01:57 AM, 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 4/9/2023 11:04:55 PM, 4/19/2023 8:38:27 PM, 4/25/2023 3:46:39 AM, 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -12.77093217726167 (T) = (0 -1724.0728180185465) / Math.Sqrt((2427.769593368939 / (32)) + (0.5973452159078103 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.06898952227692978 = (1612.806095934692 - 1724.0728180185465) / 1612.806095934692 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	46693f249bdab5c0d4d008df1cea67d51a38d144
Compare	754d9ecdafb293c6532fc2f00cd701e6b2c485a8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in Benchstone.MDBenchI.MDNDhrystone

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Test - Duration of single invocation	361.97 ms	467.74 ms	1.29	0.07	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.MDBenchI.MDNDhrystone*'

Payloads

Baseline
Compare

Histogram

Benchstone.MDBenchI.MDNDhrystone.Test

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 467.7399266666667 > 380.64877499999994.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -36.91436534384888 (T) = (0 -463563813.3894994) / Math.Sqrt((168286020530059.03 / (32)) + (48659649660400.055 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.3024293781342535 = (355922417.8846153 - 463563813.3894994) / 355922417.8846153 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	46693f249bdab5c0d4d008df1cea67d51a38d144
Compare	754d9ecdafb293c6532fc2f00cd701e6b2c485a8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Uri

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetComponents - Duration of single invocation	23.71 ns	27.70 ns	1.17	0.13	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Uri*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Uri.GetComponents

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 27.697003269957975 > 24.94527807102475.
IsChangePoint: Marked as a change because one of 2/24/2023 9:01:57 AM, 4/6/2023 1:55:21 PM, 4/24/2023 6:11:43 PM, 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -12.847726608657505 (T) = (0 -28.701708461907536) / Math.Sqrt((2.537117866641022 / (32)) + (0.38446583378231314 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.16957757010760147 = (24.540235034831397 - 28.701708461907536) / 24.540235034831397 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.19042
Queue	OwlWindows
Baseline	46693f249bdab5c0d4d008df1cea67d51a38d144
Compare	754d9ecdafb293c6532fc2f00cd701e6b2c485a8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.CtorFromCollection<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Dictionary - Duration of single invocation	3.73 μs	4.15 μs	1.11	0.04	False
FrozenDictionary - Duration of single invocation	3.86 μs	4.26 μs	1.10	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollection&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorFromCollection<String>.Dictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.154855256410255 > 3.929535967279191.
IsChangePoint: Marked as a change because one of 3/3/2023 11:43:56 AM, 3/25/2023 12:18:49 AM, 3/31/2023 8:05:26 PM, 4/6/2023 9:09:21 PM, 4/10/2023 1:33:42 PM, 4/26/2023 4:28:25 PM, 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -45.77825545631771 (T) = (0 -4158.137026042835) / Math.Sqrt((2242.230452137892 / (32)) + (6.491989530895976 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.1018562395317055 = (3773.756391133261 - 4158.137026042835) / 3773.756391133261 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<String>.FrozenDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.255343773592818 > 4.034568506901554.
IsChangePoint: Marked as a change because one of 3/25/2023 12:18:49 AM, 3/31/2023 8:05:26 PM, 4/6/2023 9:09:21 PM, 4/10/2023 1:33:42 PM, 4/26/2023 4:28:25 PM, 5/5/2023 3:18:41 PM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -44.338350546109375 (T) = (0 -4260.497579711765) / Math.Sqrt((2365.016571365763 / (32)) + (66.25937231447637 / (14))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (14) - 2, .025) and -0.10166961377568612 = (3867.3096965160157 - 4260.497579711765) / 3867.3096965160157 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

cincuranet · 2023-05-10T11:30:47Z

Majority of regressions comes from this commit range 82640bd...640df60. There's a PGO update in that range. But might still be worth looking at the regressions as the change is not small.

ghost · 2023-05-10T15:31:22Z

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

Issue Details

Author:	performanceautofiler[bot]
Assignees:	-
Labels:	`os-windows`, `arch-x64`, `area-CodeGen-coreclr`, `untriaged`, `runtime-coreclr`, `PGO`, `needs-area-label`
Milestone:	-

AndyAyersMS · 2023-05-12T19:18:06Z

Can repro (on intel x64)

Method	Job	Toolchain	Mean	Error	StdDev	Median	Min	Max	Ratio	RatioSD	Allocated	Alloc Ratio
LerpBenchmark	Job-LIESSR	\base-rel\corerun.exe	0.0398 ns	0.0235 ns	0.0208 ns	0.0329 ns	0.0172 ns	0.0894 ns	1.00	0.00	-	NA
LerpBenchmark	Job-QBMICR	\diff-rel\corerun.exe	3.1508 ns	0.0385 ns	0.0341 ns	3.1564 ns	3.0869 ns	3.2008 ns	98.88	45.14	-	NA

base is 0535b03
diff is 2ea9237

;; base

08.35%   3.25E+06    ?        Unknown
46.38%   1.804E+07   Tier-1   [MicroBenchmarks]Perf_Vector3.LerpBenchmark()
44.34%   1.725E+07   Tier-1   [98b026af-e432-4692-8dd9-fc7039dc0e9f]Runnable_0.WorkloadActionUnroll(int64)

;; diff

00.37%   4.2E+05     ?        Unknown
84.57%   9.631E+07   Tier-1   [69bc9bd6-89b0-481a-9254-3a85d754c5f2]Runnable_0.WorkloadActionUnroll(int64)
14.56%   1.658E+07   Tier-1   [MicroBenchmarks]Perf_Vector3.LerpBenchmark()

Oddly this looks like a lot more time spent in the BDN wrapper; the perf "payload" here takes about the same amount of time.

AndyAyersMS · 2023-05-12T21:11:24Z

BDN codegen for WorkloadActionUnroll changes between base and diff, and in diff it changes quite a bit
from the codegen for OverheadActionUnroll. Thus BDN does not properly account for the overhead and this
almost certainly explains the apparent regression.

;; base -- overhead

       call     [rax+18H]BenchmarkDotNet.Autogenerated.Runnable_0+OverheadDelegate:Invoke():float:this
       vmovss   dword ptr [rbp+48H], xmm0

;; base -- WorkloadActionNoUnroll

       call     [rax+18H]BenchmarkDotNet.Autogenerated.Runnable_0+WorkloadDelegate:Invoke():System.Numerics.Vector3:this
       vmovss   xmm0, dword ptr [rsp+110H]
       vmovss   dword ptr [rbp+48H], xmm0

;; diff -- overhead

       call     [rax+18H]BenchmarkDotNet.Autogenerated.Runnable_0+OverheadDelegate:Invoke():float:this
       vmovss   dword ptr [rbp+48H], xmm0

;; diff -- workoad

       call     [rax+18H]BenchmarkDotNet.Autogenerated.Runnable_0+WorkloadDelegate:Invoke():System.Numerics.Vector3:this
       vmovaps  xmm0, xmmword ptr [rsp+110H]
       vextractps dword ptr [rbp+48H], xmm0, 0

At the IR level (post inline):

;; base

***** BB02
STMT00003 ( ??? ... ??? )
               [000012] S-CXG------                         *  CALL      void   BenchmarkDotNet.Autogenerated.Runnable_0+WorkloadDelegate:Invoke():System.Numerics.Vector3:this
               [000011] ---XG------ this                    +--*  FIELD     ref    BenchmarkDotNet.Autogenerated.Runnable_0:workloadDelegate
               [000010] -----------                         |  \--*  LCL_VAR   ref    V00 this         
               [000017] ----------- retbuf                  \--*  LCL_ADDR  byref  V05 tmp2         [+0]

***** BB02
STMT00052 ( INL01 @ 0x000[E-] ... ??? ) <- INLRT @ ???
               [000225] -A-XGO-----                         *  ASG       float 
               [000224] V--XGO-N---                         +--*  FIELD     float  BenchmarkDotNet.Engines.Consumer:floatHolder
               [000015] -----------                         |  \--*  LCL_VAR   ref    V04 tmp1         
               [000019] -----------                         \--*  FIELD     float  System.Numerics.Vector3:X
               [000018] -----------                            \--*  LCL_ADDR  byref  V05 tmp2         [+0]

;; diff

***** BB02
STMT00003 ( ??? ... ??? )
               [000014] S-CXG------                         *  CALL      void   BenchmarkDotNet.Autogenerated.Runnable_0+WorkloadDelegate:Invoke():System.Numerics.Vector3:this
               [000013] n--XG------ this                    +--*  IND       ref   
               [000012] ---X-------                         |  \--*  FIELD_ADDR byref  BenchmarkDotNet.Autogenerated.Runnable_0:workloadDelegate
               [000011] -----------                         |     \--*  LCL_VAR   ref    V00 this         
               [000019] ----------- retbuf                  \--*  LCL_ADDR  byref  V05 tmp2         [+0]

***** BB02
STMT00052 ( INL01 @ 0x000[E-] ... ??? ) <- INLRT @ ???
               [000274] -A-XGO-----                         *  ASG       float 
               [000273] V--XGO-N---                         +--*  IND       float 
               [000272] ---X-------                         |  \--*  FIELD_ADDR byref  BenchmarkDotNet.Engines.Consumer:floatHolder
               [000017] -----------                         |     \--*  LCL_VAR   ref    V04 tmp1         
               [000022] n----------                         \--*  IND       float 
               [000021] -----------                            \--*  FIELD_ADDR byref  System.Numerics.Vector3:X
               [000020] -----------                               \--*  LCL_ADDR  byref  V05 tmp2         [+0]

AndyAyersMS · 2023-05-12T21:21:43Z

So this is not a PGO issue, but none of the implicated commits make sense either.

Any ideas @tannergooding @SingleAccretion ?

SingleAccretion · 2023-05-12T21:47:07Z

The IR diff is of course from #85568, and the assembly diff is I am guessing because local morph transforms the access into a GetElement node. Hard to say more without the dumps - perhaps this is some lowering hiccup due to the slightly changed IR shape?

AndyAyersMS · 2023-05-12T21:57:34Z

Diffs and such: https://gist.github.com/AndyAyersMS/ee7484f3b48ded260212e30c90e0ff53

SingleAccretion · 2023-05-12T22:30:45Z

So the cause here is that we don't promote the vectors anymore, probably due to #85561 (they no longer have handles). This is "good" in general, we want and expect most SIMDs to never get promoted.

Here though in lowering+codegen, we get a vextractps for our STOREIND(..., GetElement(x, 0)) instead of movss that the simpler promoted store generated.

tannergooding · 2023-05-12T22:48:20Z

That latter case is a simple codegen opt we can make. I had actually thought we were already doing it and optimizing extract vector, 0 to be the same as ToScalar

AndyAyersMS · 2023-05-12T22:56:03Z

Also I still need to figure out why BDN is generating different code for the overhead and action wrappers. That sort of divergence will only lead to headaches since we subtract one from the other (and if more widespread, might explain some of those mysterious "0 ns" results).

I guess if they were both inefficient, we might never have noticed this.

tannergooding · 2023-05-13T16:03:09Z

I've put up #86209 which should handle the extractps pessimization we were seeing.

AndyAyersMS · 2023-05-13T16:31:59Z

BDN is also measuring different things with "overhead" here which would be nice to fix. Opened dotnet/BenchmarkDotNet#2305

AndyAyersMS · 2023-05-17T14:47:43Z

Foxed by #86209

(etc)

performanceautofiler bot assigned AndyAyersMS May 9, 2023

performanceautofiler bot added arch-x64 os-windows PGO runtime-coreclr specific to the CoreCLR runtime untriaged New issue has not been triaged by the area owner labels May 9, 2023

cincuranet removed the untriaged New issue has not been triaged by the area owner label May 10, 2023

cincuranet changed the title ~~[Perf] Windows/x64: 61 Regressions on 5/5/2023 7:39:36 PM~~ System.Numerics.Tests.Perf_Vector3 May 10, 2023

cincuranet changed the title ~~System.Numerics.Tests.Perf_Vector3~~ Regressions in System.Numerics.Tests.Perf_Vector3 May 10, 2023

dotnet-issue-labeler bot added the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label May 10, 2023

cincuranet transferred this issue from dotnet/perf-autofiling-issues May 10, 2023

ghost added the untriaged New issue has not been triaged by the area owner label May 10, 2023

cincuranet unassigned AndyAyersMS May 10, 2023

jeffschwMSFT added the area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI label May 10, 2023

AndyAyersMS mentioned this issue May 10, 2023

Regressions in System.Numerics.Tests.Perf_Vector3 #85992

Closed

vcsjones removed the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label May 11, 2023

JulieLeeMSFT assigned AndyAyersMS May 11, 2023

JulieLeeMSFT added this to the 8.0.0 milestone May 11, 2023

ghost removed the untriaged New issue has not been triaged by the area owner label May 11, 2023

tannergooding mentioned this issue May 13, 2023

Improve the handling of ToScalar and GetElement(0) #86209

Merged

AndyAyersMS mentioned this issue May 13, 2023

overhead and workload invocation sequences diverge dotnet/BenchmarkDotNet#2305

Open

AndyAyersMS closed this as completed May 17, 2023

ghost locked as resolved and limited conversation to collaborators Jun 16, 2023

Regressions in System.Numerics.Tests.Perf_Vector3 #86033

Regressions in System.Numerics.Tests.Perf_Vector3 #86033

Comments

performanceautofiler bot commented May 9, 2023 • edited by AndyAyersMS

performanceautofiler bot commented May 9, 2023

Run Information

Regressions in System.Numerics.Tests.Perf_Vector3

Repro

Payloads

Payloads

Histogram

System.Numerics.Tests.Perf_Vector3.LerpBenchmark

Description of detection logic

JIT Disasms

System.Numerics.Tests.Perf_Vector3.MultiplyOperatorBenchmark

Description of detection logic

JIT Disasms

System.Numerics.Tests.Perf_Vector3.ClampBenchmark

Description of detection logic

JIT Disasms

System.Numerics.Tests.Perf_Vector3.UnitYBenchmark

Description of detection logic

JIT Disasms

System.Numerics.Tests.Perf_Vector3.MaxBenchmark

Description of detection logic

JIT Disasms

System.Numerics.Tests.Perf_Vector3.SquareRootBenchmark

Description of detection logic

JIT Disasms

System.Numerics.Tests.Perf_Vector3.CreateFromScalarXYZBenchmark

Description of detection logic

JIT Disasms

System.Numerics.Tests.Perf_Vector3.TransformByMatrix4x4Benchmark

Description of detection logic

JIT Disasms

System.Numerics.Tests.Perf_Vector3.AddOperatorBenchmark

Description of detection logic

JIT Disasms

System.Numerics.Tests.Perf_Vector3.DivideByScalarBenchmark

Description of detection logic

JIT Disasms

System.Numerics.Tests.Perf_Vector3.OneBenchmark

Description of detection logic

JIT Disasms

System.Numerics.Tests.Perf_Vector3.MultiplyByScalarBenchmark

Description of detection logic

JIT Disasms

System.Numerics.Tests.Perf_Vector3.MultiplyFunctionBenchmark

Description of detection logic

JIT Disasms

System.Numerics.Tests.Perf_Vector3.NegateOperatorBenchmark

Description of detection logic

JIT Disasms

System.Numerics.Tests.Perf_Vector3.CrossBenchmark

Description of detection logic

JIT Disasms

System.Numerics.Tests.Perf_Vector3.AbsBenchmark

Description of detection logic

JIT Disasms

System.Numerics.Tests.Perf_Vector3.SubtractOperatorBenchmark

Description of detection logic

JIT Disasms

System.Numerics.Tests.Perf_Vector3.CreateFromVector2WithScalarBenchmark

Description of detection logic

JIT Disasms

System.Numerics.Tests.Perf_Vector3.CreateFromScalar

Description of detection logic

JIT Disasms

System.Numerics.Tests.Perf_Vector3.TransformByQuaternionBenchmark

Description of detection logic

JIT Disasms

System.Numerics.Tests.Perf_Vector3.ZeroBenchmark

Description of detection logic

JIT Disasms

System.Numerics.Tests.Perf_Vector3.NormalizeBenchmark

Description of detection logic

JIT Disasms

System.Numerics.Tests.Perf_Vector3.ReflectBenchmark

Description of detection logic

JIT Disasms

performanceautofiler bot commented May 9, 2023 •

edited by AndyAyersMS