Fix for benchmark_all.sh output #2644
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Result looks like that. At least for my usage in the DELab, it works fine. System cx31 - click to expand
Commit Info and Build Time
hyriseBenchmarkTPCH - single-threaded, SF 10.0 Sum of avg. item runtimes: -5% || Geometric mean of throughput changes: +6%Configuration Overview - click to expand +Configuration Overview----+------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------+
| Parameter | /hyrise/rel_amd/benchmark_all_results/hyriseBenchmarkTPCH_84d3b1e17651f113943003fabf6735b7dac79baa_st.json | /hyrise/rel_amd/benchmark_all_results/hyriseBenchmarkTPCH_da72f86758477648ff8cee02d6098f96f66155c6_st.json |
+--------------------------+------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 84d3b1e17651f113943003fabf6735b7dac79baa-dirty | da72f86758477648ff8cee02d6098f96f66155c6-dirty |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| clustering | None | None |
| compiler | clang 17.0.2 | clang 17.0.2 |
| cores | 0 | 0 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-15 15:42:49 | 2024-05-15 21:53:22 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 60000000000 | 60000000000 |
| max_runs | 50 | 50 |
| scale_factor | 10.0 | 10.0 |
| time_unit | ns | ns |
| use_prepared_statements | False | False |
| using_scheduler | False | False |
| verify | False | False |
| warmup_duration | 1000000000 | 1000000000 |
+--------------------------+------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------+ +----------++----------+----------+--------++----------+----------+--------+----------------------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+----------++----------+----------+--------++----------+----------+--------+----------------------+
+| TPC-H 01 || 6246.86 | 5961.33 | -5% || 0.16 | 0.17 | +5% | 0.0044 |
+| TPC-H 02 || 49.82 | 46.83 | -6%˄ || 20.07 | 21.35 | +6%˄ | (run time too short) |
+| TPC-H 03 || 1830.24 | 1656.21 | -10% || 0.55 | 0.60 | +11% | 0.0000 |
| TPC-H 04 || 1891.68 | 1835.89 | -3% || 0.53 | 0.54 | +3% | 0.0000 |
| TPC-H 05 || 3354.55 | 3402.24 | +1% || 0.30 | 0.29 | -1% | 0.3726 |
+| TPC-H 06 || 241.07 | 198.78 | -18%˄ || 4.15 | 5.03 | +21%˄ | (run time too short) |
+| TPC-H 07 || 1187.05 | 959.95 | -19%˄ || 0.84 | 1.04 | +24%˄ | (run time too short) |
-| TPC-H 08 || 732.98 | 779.52 | +6%˄ || 1.36 | 1.28 | -6%˄ | (run time too short) |
+| TPC-H 09 || 7481.88 | 7126.25 | -5% || 0.13 | 0.14 | +5% | ˅ |
| TPC-H 10 || 1817.30 | 1756.88 | -3% || 0.55 | 0.57 | +3% | 0.0000 |
| TPC-H 11 || 69.37 | 70.38 | +1%˄ || 14.41 | 14.21 | -1%˄ | (run time too short) |
+| TPC-H 12 || 1850.62 | 1613.89 | -13% || 0.54 | 0.62 | +15% | 0.0000 |
| TPC-H 13 || 6619.09 | 6396.28 | -3% || 0.15 | 0.16 | +3% | 0.0000 |
+| TPC-H 14 || 726.92 | 616.25 | -15%˄ || 1.38 | 1.62 | +18%˄ | (run time too short) |
| TPC-H 15 || 266.07 | 271.15 | +2%˄ || 3.76 | 3.69 | -2%˄ | (run time too short) |
+| TPC-H 16 || 712.55 | 644.29 | -10%˄ || 1.40 | 1.55 | +11%˄ | (run time too short) |
| TPC-H 17 || 284.05 | 285.98 | +1%˄ || 3.52 | 3.50 | -1%˄ | (run time too short) |
| TPC-H 18 || 1608.89 | 1650.39 | +3% || 0.62 | 0.61 | -3% | 0.0176 |
+| TPC-H 19 || 293.98 | 276.27 | -6%˄ || 3.40 | 3.62 | +6%˄ | (run time too short) |
+| TPC-H 20 || 814.27 | 706.49 | -13%˄ || 1.23 | 1.42 | +15%˄ | (run time too short) |
| TPC-H 21 || 6114.15 | 5915.85 | -3% || 0.16 | 0.17 | +3% | 0.0007 |
| TPC-H 22 || 630.97 | 604.83 | -4%˄ || 1.58 | 1.65 | +4%˄ | (run time too short) |
+----------++----------+----------+--------++----------+----------+--------+----------------------+
+| Sum || 44824.36 | 42775.92 | -5% || | | | |
+| Geomean || | | || | | +6% | |
+----------++----------+----------+--------++----------+----------+--------+----------------------+
| Notes || ˄ Execution stopped due to max runs reached |
| || ˅ Insufficient number of runs for p-value calculation |
+----------++----------+----------+--------++----------+----------+--------+----------------------+hyriseBenchmarkTPCH - single-threaded, SF 0.01 Sum of avg. item runtimes: -13% || Geometric mean of throughput changes: +16%Configuration Overview - click to expand +Configuration Overview----+----------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------+
| Parameter | /hyrise/rel_amd/benchmark_all_results/hyriseBenchmarkTPCH_84d3b1e17651f113943003fabf6735b7dac79baa_st_s01.json | /hyrise/rel_amd/benchmark_all_results/hyriseBenchmarkTPCH_da72f86758477648ff8cee02d6098f96f66155c6_st_s01.json |
+--------------------------+----------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 84d3b1e17651f113943003fabf6735b7dac79baa-dirty | da72f86758477648ff8cee02d6098f96f66155c6-dirty |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| clustering | None | None |
| compiler | clang 17.0.2 | clang 17.0.2 |
| cores | 0 | 0 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-15 16:01:03 | 2024-05-15 22:11:03 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 60000000000 | 60000000000 |
| max_runs | 50 | 50 |
| scale_factor | 0.009999999776482582 | 0.009999999776482582 |
| time_unit | ns | ns |
| use_prepared_statements | False | False |
| using_scheduler | False | False |
| verify | False | False |
| warmup_duration | 1000000000 | 1000000000 |
+--------------------------+----------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------+ +----------++----------+---------+--------++----------+----------+--------+----------------------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+----------++----------+---------+--------++----------+----------+--------+----------------------+
+| TPC-H 01 || 10.70 | 9.88 | -8%˄ || 93.46 | 101.25 | +8%˄ | (run time too short) |
+| TPC-H 02 || 4.70 | 4.05 | -14%˄ || 212.70 | 246.69 | +16%˄ | (run time too short) |
+| TPC-H 03 || 0.73 | 0.58 | -21%˄ || 1364.64 | 1733.86 | +27%˄ | (run time too short) |
+| TPC-H 04 || 0.42 | 0.36 | -14%˄ || 2382.92 | 2766.42 | +16%˄ | (run time too short) |
+| TPC-H 05 || 1.04 | 0.90 | -13%˄ || 959.84 | 1108.92 | +16%˄ | (run time too short) |
+| TPC-H 06 || 0.23 | 0.20 | -16%˄ || 4246.54 | 5029.93 | +18%˄ | (run time too short) |
+| TPC-H 07 || 12.40 | 10.15 | -18%˄ || 80.67 | 98.48 | +22%˄ | (run time too short) |
+| TPC-H 08 || 15.86 | 13.80 | -13%˄ || 63.06 | 72.46 | +15%˄ | (run time too short) |
+| TPC-H 09 || 4.44 | 3.23 | -27%˄ || 225.29 | 309.47 | +37%˄ | (run time too short) |
+| TPC-H 10 || 0.71 | 0.63 | -12%˄ || 1396.45 | 1578.83 | +13%˄ | (run time too short) |
+| TPC-H 11 || 0.20 | 0.17 | -15%˄ || 5007.08 | 5906.60 | +18%˄ | (run time too short) |
+| TPC-H 12 || 0.73 | 0.59 | -20%˄ || 1363.04 | 1700.83 | +25%˄ | (run time too short) |
+| TPC-H 13 || 2.12 | 1.97 | -7%˄ || 472.12 | 507.95 | +8%˄ | (run time too short) |
+| TPC-H 14 || 0.33 | 0.27 | -18%˄ || 3014.29 | 3681.51 | +22%˄ | (run time too short) |
+| TPC-H 15 || 1.34 | 1.16 | -13%˄ || 744.73 | 857.81 | +15%˄ | (run time too short) |
+| TPC-H 16 || 2.45 | 2.18 | -11%˄ || 407.50 | 457.65 | +12%˄ | (run time too short) |
+| TPC-H 17 || 0.68 | 0.56 | -18%˄ || 1466.31 | 1785.85 | +22%˄ | (run time too short) |
| TPC-H 18 || 1.10 | 1.08 | -2%˄ || 907.06 | 928.82 | +2%˄ | (run time too short) |
+| TPC-H 19 || 6.43 | 5.92 | -8%˄ || 155.57 | 168.92 | +9%˄ | (run time too short) |
+| TPC-H 20 || 2.92 | 2.53 | -13%˄ || 342.45 | 394.63 | +15%˄ | (run time too short) |
+| TPC-H 21 || 1.11 | 0.95 | -14%˄ || 899.98 | 1050.60 | +17%˄ | (run time too short) |
+| TPC-H 22 || 1.29 | 1.15 | -11%˄ || 771.59 | 869.43 | +13%˄ | (run time too short) |
+----------++----------+---------+--------++----------+----------+--------+----------------------+
+| Sum || 71.93 | 62.30 | -13% || | | | |
+| Geomean || | | || | | +16% | |
+----------++----------+---------+--------++----------+----------+--------+----------------------+
| Notes || ˄ Execution stopped due to max runs reached |
+----------++----------+---------+--------++----------+----------+--------+----------------------+hyriseBenchmarkTPCH - multi-threaded, ordered, 1 client, 64 cores, SF 10.0 Sum of avg. item runtimes: -5% || Geometric mean of throughput changes: +6%Configuration Overview - click to expand +Configuration Overview---------+--------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------+
| Parameter | /hyrise/rel_amd/benchmark_all_results/hyriseBenchmarkTPCH_84d3b1e17651f113943003fabf6735b7dac79baa_mt_ordered.json | /hyrise/rel_amd/benchmark_all_results/hyriseBenchmarkTPCH_da72f86758477648ff8cee02d6098f96f66155c6_mt_ordered.json |
+-------------------------------+--------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 84d3b1e17651f113943003fabf6735b7dac79baa-dirty | da72f86758477648ff8cee02d6098f96f66155c6-dirty |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| clustering | None | None |
| compiler | clang 17.0.2 | clang 17.0.2 |
| cores | 64 | 64 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-15 16:01:29 | 2024-05-15 22:11:29 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 60000000000 | 60000000000 |
| max_runs | 50 | 50 |
| scale_factor | 10.0 | 10.0 |
| time_unit | ns | ns |
| use_prepared_statements | False | False |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [64] | [64] |
| verify | False | False |
| warmup_duration | 1000000000 | 1000000000 |
+-------------------------------+--------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------+ +----------++----------+----------+--------++----------+----------+--------+----------------------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+----------++----------+----------+--------++----------+----------+--------+----------------------+
+| TPC-H 01 || 5850.90 | 5463.11 | -7% || 0.17 | 0.18 | +7% | 0.0000 |
+| TPC-H 02 || 75.37 | 71.05 | -6%˄ || 13.25 | 14.05 | +6%˄ | (run time too short) |
+| TPC-H 03 || 898.94 | 817.28 | -9%˄ || 1.11 | 1.22 | +10%˄ | (run time too short) |
+| TPC-H 04 || 871.29 | 823.63 | -5%˄ || 1.15 | 1.21 | +6%˄ | (run time too short) |
+| TPC-H 05 || 692.38 | 660.20 | -5%˄ || 1.44 | 1.51 | +5%˄ | (run time too short) |
+| TPC-H 06 || 78.82 | 69.65 | -12%˄ || 12.67 | 14.34 | +13%˄ | (run time too short) |
+| TPC-H 07 || 357.02 | 320.53 | -10%˄ || 2.80 | 3.12 | +11%˄ | (run time too short) |
+| TPC-H 08 || 301.16 | 286.91 | -5%˄ || 3.32 | 3.48 | +5%˄ | (run time too short) |
| TPC-H 09 || 2844.46 | 2769.75 | -3% || 0.35 | 0.36 | +3% | 0.0002 |
| TPC-H 10 || 826.34 | 805.00 | -3%˄ || 1.21 | 1.24 | +3%˄ | (run time too short) |
| TPC-H 11 || 78.60 | 78.17 | -1%˄ || 12.71 | 12.77 | +1%˄ | (run time too short) |
+| TPC-H 12 || 756.97 | 641.49 | -15%˄ || 1.32 | 1.56 | +18%˄ | (run time too short) |
| TPC-H 13 || 3950.65 | 3875.89 | -2% || 0.25 | 0.26 | +2% | 0.0006 |
+| TPC-H 14 || 180.11 | 171.98 | -5%˄ || 5.55 | 5.81 | +5%˄ | (run time too short) |
| TPC-H 15 || 172.89 | 170.24 | -2%˄ || 5.78 | 5.87 | +2%˄ | (run time too short) |
+| TPC-H 16 || 1009.32 | 843.25 | -16%˄ || 0.99 | 1.19 | +20%˄ | (run time too short) |
+| TPC-H 17 || 105.87 | 100.72 | -5%˄ || 9.44 | 9.92 | +5%˄ | (run time too short) |
| TPC-H 18 || 2596.50 | 2568.85 | -1% || 0.39 | 0.39 | +1% | 0.4838 |
| TPC-H 19 || 134.57 | 138.31 | +3%˄ || 7.43 | 7.22 | -3%˄ | (run time too short) |
+| TPC-H 20 || 279.32 | 257.72 | -8%˄ || 3.58 | 3.88 | +8%˄ | (run time too short) |
| TPC-H 21 || 1437.59 | 1377.59 | -4% || 0.70 | 0.73 | +4% | 0.0004 |
| TPC-H 22 || 179.14 | 174.85 | -2%˄ || 5.58 | 5.72 | +2%˄ | (run time too short) |
+----------++----------+----------+--------++----------+----------+--------+----------------------+
+| Sum || 23678.21 | 22486.19 | -5% || | | | |
+| Geomean || | | || | | +6% | |
+----------++----------+----------+--------++----------+----------+--------+----------------------+
| Notes || ˄ Execution stopped due to max runs reached |
+----------++----------+----------+--------++----------+----------+--------+----------------------+hyriseBenchmarkTPCH - multi-threaded, shuffled, 64 clients, 64 cores, SF 10.0 Sum of avg. item runtimes: -4% || Geometric mean of throughput changes: +4%Configuration Overview - click to expand +Configuration Overview---------+------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------+
| Parameter | /hyrise/rel_amd/benchmark_all_results/hyriseBenchmarkTPCH_84d3b1e17651f113943003fabf6735b7dac79baa_mt.json | /hyrise/rel_amd/benchmark_all_results/hyriseBenchmarkTPCH_da72f86758477648ff8cee02d6098f96f66155c6_mt.json |
+-------------------------------+------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 84d3b1e17651f113943003fabf6735b7dac79baa-dirty | da72f86758477648ff8cee02d6098f96f66155c6-dirty |
| benchmark_mode | Shuffled | Shuffled |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 64 | 64 |
| clustering | None | None |
| compiler | clang 17.0.2 | clang 17.0.2 |
| cores | 64 | 64 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-15 16:15:54 | 2024-05-15 22:25:09 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 1200000000000 | 1200000000000 |
| max_runs | -1 | -1 |
| scale_factor | 10.0 | 10.0 |
| time_unit | ns | ns |
| use_prepared_statements | False | False |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [64] | [64] |
| verify | False | False |
| warmup_duration | 0 | 0 |
+-------------------------------+------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------+ +----------++----------+----------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+----------++----------+----------+--------++----------+----------+--------+---------+
+| TPC-H 01 || 6709.55 | 6271.26 | -7% || 1.00 | 1.04 | +4% | 0.0000 |
| TPC-H 02 || 531.03 | 515.26 | -3% || 1.00 | 1.04 | +4% | 0.8121 |
| TPC-H 03 || 2881.05 | 2936.91 | +2% || 1.00 | 1.04 | +4% | 0.6990 |
| TPC-H 04 || 2621.79 | 2549.54 | -3% || 1.00 | 1.04 | +4% | 0.6277 |
+| TPC-H 05 || 4305.07 | 4088.31 | -5% || 1.00 | 1.04 | +4% | 0.2337 |
-| TPC-H 06 || 764.79 | 910.55 | +19% || 1.00 | 1.04 | +4% | 0.0615 |
+| TPC-H 07 || 3798.17 | 3495.57 | -8% || 1.00 | 1.04 | +3% | 0.1454 |
+| TPC-H 08 || 2633.57 | 2415.37 | -8% || 1.00 | 1.04 | +4% | 0.1620 |
+| TPC-H 09 || 6803.76 | 6459.07 | -5% || 1.00 | 1.04 | +4% | 0.0702 |
| TPC-H 10 || 3591.65 | 3535.26 | -2% || 1.00 | 1.04 | +3% | 0.7431 |
| TPC-H 11 || 542.04 | 563.55 | +4% || 1.00 | 1.04 | +4% | 0.7722 |
+| TPC-H 12 || 2858.63 | 2698.73 | -6% || 1.00 | 1.04 | +4% | 0.2516 |
| TPC-H 13 || 5969.76 | 5754.15 | -4% || 1.00 | 1.04 | +4% | 0.0535 |
+| TPC-H 14 || 1200.08 | 1116.20 | -7% || 1.00 | 1.04 | +4% | 0.3476 |
| TPC-H 15 || 705.52 | 677.71 | -4% || 1.00 | 1.04 | +4% | 0.6575 |
+| TPC-H 16 || 2752.13 | 2380.68 | -13% || 1.00 | 1.04 | +4% | 0.0061 |
| TPC-H 17 || 844.58 | 881.72 | +4% || 1.00 | 1.04 | +4% | 0.6643 |
| TPC-H 18 || 3797.84 | 3829.16 | +1% || 1.00 | 1.04 | +4% | 0.8026 |
-| TPC-H 19 || 1179.21 | 1354.98 | +15% || 1.00 | 1.04 | +4% | 0.1547 |
+| TPC-H 20 || 1962.98 | 1834.46 | -7% || 1.00 | 1.04 | +4% | 0.2981 |
+| TPC-H 21 || 5782.09 | 5330.46 | -8% || 1.00 | 1.04 | +4% | 0.0297 |
-| TPC-H 22 || 1098.81 | 1167.74 | +6% || 1.01 | 1.04 | +3% | 0.4799 |
+----------++----------+----------+--------++----------+----------+--------+---------+
| Sum || 63334.09 | 60766.66 | -4% || | | | |
| Geomean || | | || | | +4% | |
+----------++----------+----------+--------++----------+----------+--------+---------+hyriseBenchmarkTPCDS - single-threaded Sum of avg. item runtimes: -4% || Geometric mean of throughput changes: +7%Configuration Overview - click to expand +Configuration Overview---+-------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+
| Parameter | /hyrise/rel_amd/benchmark_all_results/hyriseBenchmarkTPCDS_84d3b1e17651f113943003fabf6735b7dac79baa_st.json | /hyrise/rel_amd/benchmark_all_results/hyriseBenchmarkTPCDS_da72f86758477648ff8cee02d6098f96f66155c6_st.json |
+-------------------------+-------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 84d3b1e17651f113943003fabf6735b7dac79baa-dirty | da72f86758477648ff8cee02d6098f96f66155c6-dirty |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| compiler | clang 17.0.2 | clang 17.0.2 |
| cores | 0 | 0 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-15 16:38:00 | 2024-05-15 22:47:13 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 60000000000 | 60000000000 |
| max_runs | 50 | 50 |
| time_unit | ns | ns |
| using_scheduler | False | False |
| verify | False | False |
| warmup_duration | 1000000000 | 1000000000 |
+-------------------------+-------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+ +---------++----------+----------+--------++----------+----------+--------+----------------------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+---------++----------+----------+--------++----------+----------+--------+----------------------+
+| 01 || 304.30 | 291.12 | -4%˄ || 3.29 | 3.43 | +5%˄ | (run time too short) |
| 03 || 101.20 | 99.31 | -2%˄ || 9.88 | 10.07 | +2%˄ | (run time too short) |
+| 06 || 239.73 | 222.18 | -7%˄ || 4.17 | 4.50 | +8%˄ | (run time too short) |
+| 07 || 389.13 | 337.38 | -13%˄ || 2.57 | 2.96 | +15%˄ | (run time too short) |
| 09 || 1055.48 | 1064.63 | +1%˄ || 0.95 | 0.94 | -1%˄ | (run time too short) |
+| 10 || 166.94 | 153.89 | -8%˄ || 5.99 | 6.50 | +8%˄ | (run time too short) |
+| 13 || 693.04 | 618.73 | -11%˄ || 1.44 | 1.62 | +12%˄ | (run time too short) |
+| 15 || 136.29 | 125.07 | -8%˄ || 7.34 | 8.00 | +9%˄ | (run time too short) |
| 16 || 69.84 | 68.62 | -2%˄ || 14.32 | 14.57 | +2%˄ | (run time too short) |
| 17 || 430.58 | 412.06 | -4%˄ || 2.32 | 2.43 | +4%˄ | (run time too short) |
+| 19 || 140.79 | 122.15 | -13%˄ || 7.10 | 8.19 | +15%˄ | (run time too short) |
+| 25 || 213.90 | 202.96 | -5%˄ || 4.67 | 4.93 | +5%˄ | (run time too short) |
+| 26 || 160.64 | 139.58 | -13%˄ || 6.22 | 7.16 | +15%˄ | (run time too short) |
| 28 || 795.49 | 779.17 | -2%˄ || 1.26 | 1.28 | +2%˄ | (run time too short) |
| 29 || 593.90 | 581.05 | -2%˄ || 1.68 | 1.72 | +2%˄ | (run time too short) |
+| 31 || 1596.03 | 1481.72 | -7% || 0.63 | 0.67 | +8% | 0.0000 |
| 32 || 35.70 | 34.63 | -3%˄ || 28.01 | 28.87 | +3%˄ | (run time too short) |
+| 34 || 231.46 | 199.00 | -14%˄ || 4.32 | 5.02 | +16%˄ | (run time too short) |
+| 35 || 824.60 | 763.00 | -7%˄ || 1.21 | 1.31 | +8%˄ | (run time too short) |
+| 37 || 851.92 | 728.61 | -14%˄ || 1.17 | 1.37 | +17%˄ | (run time too short) |
| 39a || 1889.24 | 1883.40 | -0% || 0.53 | 0.53 | +0% | 0.2981 |
| 39b || 1871.62 | 1867.31 | -0% || 0.53 | 0.54 | +0% | 0.2766 |
+| 41 || 336.37 | 262.88 | -22%˄ || 2.97 | 3.80 | +28%˄ | (run time too short) |
+| 42 || 110.92 | 99.07 | -11%˄ || 9.01 | 10.09 | +12%˄ | (run time too short) |
+| 43 || 1081.82 | 969.90 | -10%˄ || 0.92 | 1.03 | +12%˄ | (run time too short) |
+| 45 || 177.97 | 154.26 | -13%˄ || 5.62 | 6.48 | +15%˄ | (run time too short) |
| 48 || 1125.78 | 1078.61 | -4%˄ || 0.89 | 0.93 | +4%˄ | (run time too short) |
+| 50 || 162.87 | 154.29 | -5%˄ || 6.14 | 6.48 | +6%˄ | (run time too short) |
+| 52 || 115.24 | 99.15 | -14%˄ || 8.68 | 10.09 | +16%˄ | (run time too short) |
+| 55 || 108.90 | 92.55 | -15%˄ || 9.18 | 10.80 | +18%˄ | (run time too short) |
+| 62 || 601.76 | 569.01 | -5%˄ || 1.66 | 1.76 | +6%˄ | (run time too short) |
| 65 || 2379.87 | 2360.62 | -1% || 0.42 | 0.42 | +1% | 0.0758 |
+| 69 || 183.16 | 171.18 | -7%˄ || 5.46 | 5.84 | +7%˄ | (run time too short) |
+| 73 || 108.98 | 96.97 | -11%˄ || 9.18 | 10.31 | +12%˄ | (run time too short) |
+| 79 || 632.48 | 575.56 | -9%˄ || 1.58 | 1.74 | +10%˄ | (run time too short) |
+| 81 || 208.39 | 189.48 | -9%˄ || 4.80 | 5.28 | +10%˄ | (run time too short) |
+| 82 || 923.28 | 824.19 | -11%˄ || 1.08 | 1.21 | +12%˄ | (run time too short) |
| 83 || 52.07 | 51.45 | -1%˄ || 19.20 | 19.44 | +1%˄ | (run time too short) |
-| 84 || 18.58 | 19.71 | +6%˄ || 53.79 | 50.73 | -6%˄ | (run time too short) |
+| 85 || 208.82 | 193.07 | -8%˄ || 4.79 | 5.18 | +8%˄ | (run time too short) |
+| 88 || 953.60 | 882.78 | -7%˄ || 1.05 | 1.13 | +8%˄ | (run time too short) |
| 91 || 20.90 | 20.32 | -3%˄ || 47.84 | 49.19 | +3%˄ | (run time too short) |
+| 92 || 50.44 | 46.06 | -9%˄ || 19.82 | 21.71 | +10%˄ | (run time too short) |
| 93 || 5225.37 | 5093.36 | -3% || 0.19 | 0.20 | +3% | 0.0000 |
+| 94 || 52.43 | 47.75 | -9%˄ || 19.07 | 20.94 | +10%˄ | (run time too short) |
| 95 || 4585.98 | 4777.85 | +4% || 0.22 | 0.21 | -4% | 0.0041 |
| 96 || 83.35 | 80.42 | -4%˄ || 12.00 | 12.43 | +4%˄ | (run time too short) |
+| 97 || 4706.37 | 4502.91 | -4% || 0.21 | 0.22 | +5% | 0.0000 |
+| 99 || 1112.49 | 1052.90 | -5%˄ || 0.90 | 0.95 | +6%˄ | (run time too short) |
+---------++----------+----------+--------++----------+----------+--------+----------------------+
| Sum || 38120.01 | 36641.86 | -4% || | | | |
+| Geomean || | | || | | +7% | |
+---------++----------+----------+--------++----------+----------+--------+----------------------+
| Notes || ˄ Execution stopped due to max runs reached |
+---------++----------+----------+--------++----------+----------+--------+----------------------+hyriseBenchmarkTPCDS - multi-threaded, ordered, 1 client, 64 cores Sum of avg. item runtimes: -4% || Geometric mean of throughput changes: +4%Configuration Overview - click to expand +Configuration Overview---------+---------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------+
| Parameter | /hyrise/rel_amd/benchmark_all_results/hyriseBenchmarkTPCDS_84d3b1e17651f113943003fabf6735b7dac79baa_mt_ordered.json | /hyrise/rel_amd/benchmark_all_results/hyriseBenchmarkTPCDS_da72f86758477648ff8cee02d6098f96f66155c6_mt_ordered.json |
+-------------------------------+---------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 84d3b1e17651f113943003fabf6735b7dac79baa-dirty | da72f86758477648ff8cee02d6098f96f66155c6-dirty |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| compiler | clang 17.0.2 | clang 17.0.2 |
| cores | 64 | 64 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-15 17:00:04 | 2024-05-15 23:08:12 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 60000000000 | 60000000000 |
| max_runs | 50 | 50 |
| time_unit | ns | ns |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [64] | [64] |
| verify | False | False |
| warmup_duration | 1000000000 | 1000000000 |
+-------------------------------+---------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------+ +---------++----------+----------+--------++----------+----------+--------+----------------------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+---------++----------+----------+--------++----------+----------+--------+----------------------+
| 01 || 306.10 | 308.12 | +1%˄ || 3.27 | 3.24 | -1%˄ | (run time too short) |
| 03 || 38.22 | 37.42 | -2%˄ || 26.12 | 26.67 | +2%˄ | (run time too short) |
+| 06 || 217.23 | 201.50 | -7%˄ || 4.60 | 4.96 | +8%˄ | (run time too short) |
| 07 || 250.88 | 241.36 | -4%˄ || 3.98 | 4.14 | +4%˄ | (run time too short) |
-| 09 || 125.72 | 132.44 | +5%˄ || 7.95 | 7.55 | -5%˄ | (run time too short) |
| 10 || 38.54 | 37.10 | -4%˄ || 25.91 | 26.92 | +4%˄ | (run time too short) |
| 13 || 334.48 | 329.82 | -1%˄ || 2.99 | 3.03 | +1%˄ | (run time too short) |
+| 15 || 83.11 | 75.64 | -9%˄ || 12.02 | 13.21 | +10%˄ | (run time too short) |
+| 16 || 39.22 | 37.31 | -5%˄ || 25.45 | 26.76 | +5%˄ | (run time too short) |
| 17 || 165.69 | 160.40 | -3%˄ || 6.03 | 6.23 | +3%˄ | (run time too short) |
+| 19 || 85.91 | 81.71 | -5%˄ || 11.63 | 12.23 | +5%˄ | (run time too short) |
+| 25 || 126.68 | 120.84 | -5%˄ || 7.89 | 8.27 | +5%˄ | (run time too short) |
+| 26 || 135.78 | 128.18 | -6%˄ || 7.36 | 7.80 | +6%˄ | (run time too short) |
| 28 || 182.03 | 178.55 | -2%˄ || 5.49 | 5.60 | +2%˄ | (run time too short) |
| 29 || 492.26 | 484.89 | -1%˄ || 2.03 | 2.06 | +2%˄ | (run time too short) |
+| 31 || 666.63 | 629.14 | -6%˄ || 1.50 | 1.59 | +6%˄ | (run time too short) |
| 32 || 33.01 | 32.67 | -1%˄ || 30.21 | 30.52 | +1%˄ | (run time too short) |
| 34 || 109.50 | 107.14 | -2%˄ || 9.12 | 9.32 | +2%˄ | (run time too short) |
+| 35 || 566.69 | 522.47 | -8%˄ || 1.76 | 1.91 | +8%˄ | (run time too short) |
+| 37 || 246.22 | 229.84 | -7%˄ || 4.06 | 4.35 | +7%˄ | (run time too short) |
+| 39a || 1157.68 | 1059.89 | -8%˄ || 0.86 | 0.94 | +9%˄ | (run time too short) |
| 39b || 1072.36 | 1081.35 | +1%˄ || 0.93 | 0.92 | -1%˄ | (run time too short) |
+| 41 || 624.16 | 555.56 | -11%˄ || 1.60 | 1.80 | +12%˄ | (run time too short) |
| 42 || 58.30 | 57.15 | -2%˄ || 17.13 | 17.48 | +2%˄ | (run time too short) |
| 43 || 647.65 | 628.76 | -3%˄ || 1.54 | 1.59 | +3%˄ | (run time too short) |
+| 45 || 66.95 | 62.06 | -7%˄ || 14.92 | 16.10 | +8%˄ | (run time too short) |
| 48 || 591.30 | 575.98 | -3%˄ || 1.69 | 1.74 | +3%˄ | (run time too short) |
+| 50 || 134.62 | 128.30 | -5%˄ || 7.42 | 7.79 | +5%˄ | (run time too short) |
| 52 || 59.72 | 59.27 | -1%˄ || 16.73 | 16.85 | +1%˄ | (run time too short) |
| 55 || 54.38 | 52.55 | -3%˄ || 18.37 | 19.01 | +3%˄ | (run time too short) |
+| 62 || 515.73 | 475.73 | -8%˄ || 1.94 | 2.10 | +8%˄ | (run time too short) |
| 65 || 2320.09 | 2301.20 | -1% || 0.43 | 0.43 | +1% | 0.2169 |
| 69 || 84.06 | 82.01 | -2%˄ || 11.88 | 12.18 | +3%˄ | (run time too short) |
| 73 || 64.16 | 63.17 | -2%˄ || 15.57 | 15.81 | +2%˄ | (run time too short) |
+| 79 || 769.18 | 696.56 | -9%˄ || 1.30 | 1.44 | +10%˄ | (run time too short) |
+| 81 || 235.62 | 216.48 | -8%˄ || 4.24 | 4.62 | +9%˄ | (run time too short) |
+| 82 || 281.60 | 258.37 | -8%˄ || 3.55 | 3.87 | +9%˄ | (run time too short) |
+| 83 || 50.07 | 46.50 | -7%˄ || 19.94 | 21.47 | +8%˄ | (run time too short) |
| 84 || 11.79 | 12.02 | +2%˄ || 84.49 | 82.60 | -2%˄ | (run time too short) |
| 85 || 78.07 | 75.00 | -4%˄ || 12.79 | 13.32 | +4%˄ | (run time too short) |
| 88 || 84.55 | 82.81 | -2%˄ || 11.82 | 12.07 | +2%˄ | (run time too short) |
+| 91 || 17.78 | 16.90 | -5%˄ || 56.04 | 58.91 | +5%˄ | (run time too short) |
| 92 || 50.72 | 50.74 | +0%˄ || 19.67 | 19.66 | -0%˄ | (run time too short) |
| 93 || 1060.70 | 1018.87 | -4%˄ || 0.94 | 0.98 | +4%˄ | (run time too short) |
+| 94 || 40.18 | 35.84 | -11%˄ || 24.84 | 27.86 | +12%˄ | (run time too short) |
| 95 || 1016.34 | 991.05 | -2%˄ || 0.98 | 1.01 | +3%˄ | (run time too short) |
| 96 || 56.05 | 55.91 | -0%˄ || 17.82 | 17.86 | +0%˄ | (run time too short) |
| 97 || 2104.53 | 2048.53 | -3% || 0.48 | 0.49 | +3% | 0.0015 |
+| 99 || 990.63 | 932.40 | -6%˄ || 1.01 | 1.07 | +6%˄ | (run time too short) |
+---------++----------+----------+--------++----------+----------+--------+----------------------+
| Sum || 18542.87 | 17797.47 | -4% || | | | |
| Geomean || | | || | | +4% | |
+---------++----------+----------+--------++----------+----------+--------+----------------------+
| Notes || ˄ Execution stopped due to max runs reached |
+---------++----------+----------+--------++----------+----------+--------+----------------------+hyriseBenchmarkTPCDS - multi-threaded, shuffled, 64 clients, 64 cores Sum of avg. item runtimes: -1% || Geometric mean of throughput changes: +1%Configuration Overview - click to expand +Configuration Overview---------+-------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+
| Parameter | /hyrise/rel_amd/benchmark_all_results/hyriseBenchmarkTPCDS_84d3b1e17651f113943003fabf6735b7dac79baa_mt.json | /hyrise/rel_amd/benchmark_all_results/hyriseBenchmarkTPCDS_da72f86758477648ff8cee02d6098f96f66155c6_mt.json |
+-------------------------------+-------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 84d3b1e17651f113943003fabf6735b7dac79baa-dirty | da72f86758477648ff8cee02d6098f96f66155c6-dirty |
| benchmark_mode | Shuffled | Shuffled |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 64 | 64 |
| compiler | clang 17.0.2 | clang 17.0.2 |
| cores | 64 | 64 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-15 17:15:54 | 2024-05-15 23:23:25 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 1200000000000 | 1200000000000 |
| max_runs | -1 | -1 |
| time_unit | ns | ns |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [64] | [64] |
| verify | False | False |
| warmup_duration | 0 | 0 |
+-------------------------------+-------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------+ +---------++----------+----------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+---------++----------+----------+--------++----------+----------+--------+---------+
+| 01 || 878.37 | 789.89 | -10% || 0.98 | 0.99 | +1% | 0.1076 |
-| 03 || 277.60 | 326.60 | +18% || 0.98 | 0.99 | +1% | 0.1661 |
| 06 || 951.69 | 915.58 | -4% || 0.98 | 0.99 | +1% | 0.5692 |
| 07 || 1040.51 | 1074.57 | +3% || 0.98 | 0.99 | +1% | 0.6365 |
| 09 || 874.50 | 885.39 | +1% || 0.98 | 0.99 | +1% | 0.8703 |
+| 10 || 942.71 | 827.90 | -12% || 0.98 | 0.99 | +1% | 0.1463 |
| 13 || 1883.83 | 1893.15 | +0% || 0.98 | 0.99 | +1% | 0.9229 |
+| 15 || 569.30 | 507.75 | -11% || 0.98 | 1.00 | +1% | 0.2070 |
+| 16 || 529.82 | 438.38 | -17% || 0.98 | 1.00 | +1% | 0.0586 |
+| 17 || 1508.07 | 1412.80 | -6% || 0.98 | 0.99 | +1% | 0.2534 |
+| 19 || 707.38 | 672.75 | -5% || 0.98 | 0.99 | +1% | 0.5446 |
-| 25 || 986.54 | 1209.17 | +23% || 0.98 | 0.99 | +1% | 0.0025 |
| 26 || 544.09 | 567.70 | +4% || 0.98 | 1.00 | +1% | 0.6246 |
| 28 || 1672.54 | 1699.62 | +2% || 0.98 | 0.99 | +1% | 0.7986 |
| 29 || 1934.54 | 1857.22 | -4% || 0.98 | 0.99 | +1% | 0.4111 |
| 31 || 2813.93 | 2695.24 | -4% || 0.98 | 0.99 | +1% | 0.3204 |
| 32 || 212.46 | 207.21 | -2% || 0.98 | 1.00 | +1% | 0.8745 |
+| 34 || 756.78 | 718.57 | -5% || 0.98 | 1.00 | +1% | 0.4909 |
| 35 || 2144.10 | 2218.00 | +3% || 0.98 | 0.99 | +1% | 0.4408 |
+| 37 || 966.92 | 912.62 | -6% || 0.98 | 0.99 | +1% | 0.4201 |
| 39a || 2959.74 | 2971.97 | +0% || 0.98 | 0.99 | +1% | 0.8961 |
| 39b || 2957.04 | 2827.99 | -4% || 0.98 | 0.99 | +1% | 0.1627 |
| 41 || 2216.03 | 2142.28 | -3% || 0.98 | 0.99 | +1% | 0.4678 |
| 42 || 509.38 | 505.82 | -1% || 0.98 | 1.00 | +1% | 0.9428 |
| 43 || 1596.09 | 1588.33 | -0% || 0.98 | 0.99 | +1% | 0.9208 |
-| 45 || 795.15 | 848.23 | +7% || 0.98 | 0.99 | +1% | 0.4273 |
| 48 || 2709.91 | 2588.57 | -4% || 0.98 | 0.99 | +1% | 0.2597 |
| 50 || 1118.47 | 1131.95 | +1% || 0.98 | 0.99 | +1% | 0.8588 |
+| 52 || 535.47 | 488.96 | -9% || 0.98 | 0.99 | +1% | 0.3055 |
+| 55 || 480.58 | 439.63 | -9% || 0.98 | 1.00 | +1% | 0.3781 |
| 62 || 1103.73 | 1106.05 | +0% || 0.98 | 0.99 | +1% | 0.9697 |
| 65 || 4364.70 | 4516.20 | +3% || 0.98 | 0.99 | +1% | 0.0544 |
| 69 || 1129.50 | 1115.30 | -1% || 0.98 | 0.99 | +1% | 0.8709 |
| 73 || 537.01 | 560.18 | +4% || 0.98 | 0.99 | +1% | 0.6240 |
-| 79 || 1537.16 | 1658.95 | +8% || 0.98 | 0.99 | +1% | 0.0808 |
| 81 || 753.07 | 735.36 | -2% || 0.98 | 1.00 | +1% | 0.7358 |
| 82 || 1099.19 | 1078.35 | -2% || 0.98 | 0.99 | +1% | 0.7618 |
+| 83 || 400.61 | 310.24 | -23% || 0.98 | 1.00 | +1% | 0.0639 |
+| 84 || 175.45 | 152.00 | -13% || 0.98 | 1.00 | +1% | 0.5346 |
+| 85 || 1017.15 | 904.50 | -11% || 0.98 | 0.99 | +1% | 0.1708 |
| 88 || 2753.95 | 2817.60 | +2% || 0.98 | 0.99 | +1% | 0.6308 |
-| 91 || 174.80 | 194.40 | +11% || 0.98 | 0.99 | +1% | 0.4475 |
+| 92 || 340.93 | 316.80 | -7% || 0.98 | 0.99 | +1% | 0.4910 |
+| 93 || 2328.85 | 2211.06 | -5% || 0.98 | 0.99 | +1% | 0.1371 |
-| 94 || 474.04 | 505.97 | +7% || 0.98 | 1.00 | +1% | 0.5400 |
-| 95 || 2273.05 | 2437.36 | +7% || 0.98 | 0.99 | +1% | 0.1002 |
+| 96 || 447.38 | 417.37 | -7% || 0.98 | 0.99 | +1% | 0.4838 |
| 97 || 4360.20 | 4401.57 | +1% || 0.98 | 0.99 | +1% | 0.6973 |
| 99 || 1395.76 | 1357.15 | -3% || 0.98 | 0.99 | +1% | 0.4865 |
+---------++----------+----------+--------++----------+----------+--------+---------+
| Sum || 64740.06 | 64160.29 | -1% || | | | |
| Geomean || | | || | | +1% | |
+---------++----------+----------+--------++----------+----------+--------+---------+hyriseBenchmarkTPCC - single-threaded Sum of avg. item runtimes: -11% || Geometric mean of throughput changes: +11%Configuration Overview - click to expand +Configuration Overview---+------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------+
| Parameter | /hyrise/rel_amd/benchmark_all_results/hyriseBenchmarkTPCC_84d3b1e17651f113943003fabf6735b7dac79baa_st.json | /hyrise/rel_amd/benchmark_all_results/hyriseBenchmarkTPCC_da72f86758477648ff8cee02d6098f96f66155c6_st.json |
+-------------------------+------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 84d3b1e17651f113943003fabf6735b7dac79baa-dirty | da72f86758477648ff8cee02d6098f96f66155c6-dirty |
| benchmark_mode | Shuffled | Shuffled |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| compiler | clang 17.0.2 | clang 17.0.2 |
| cores | 0 | 0 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-15 17:36:28 | 2024-05-15 23:43:54 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 60000000000 | 60000000000 |
| max_runs | -1 | -1 |
| scale_factor | 10 | 10 |
| time_unit | ns | ns |
| using_scheduler | False | False |
| verify | False | False |
| warmup_duration | 0 | 0 |
+-------------------------+------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------+ +--------------++----------+---------+--------++----------+----------+--------+----------------------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+--------------++----------+---------+--------++----------+----------+--------+----------------------+
+| Delivery || 63.22 | 55.69 | -12% || 1.55 | 1.55 | -0% | (run time too short) |
+| New-Order || 43.13 | 38.92 | -10% || 17.61 | 19.93 | +13% | 0.0000 |
+| Order-Status || 2.17 | 2.06 | -5% || 1.52 | 1.90 | +25% | (run time too short) |
+| Payment || 8.09 | 6.72 | -17% || 16.28 | 18.88 | +16% | 0.0000 |
+| Stock-Level || 3.62 | 3.41 | -6% || 1.77 | 1.83 | +4% | 0.0069 |
+--------------++----------+---------+--------++----------+----------+--------+----------------------+
+| Sum || 120.24 | 106.79 | -11% || | | | |
+| Geomean || | | || | | +11% | |
+--------------++----------+---------+--------++----------+----------+--------+----------------------+hyriseBenchmarkTPCC - multi-threaded, shuffled, 64 clients, 10 warehouses, 64 cores (high contention) Sum of avg. item runtimes: -3% || Geometric mean of throughput changes: +3%Configuration Overview - click to expand +Configuration Overview---------+---------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------+
| Parameter | /hyrise/rel_amd/benchmark_all_results/hyriseBenchmarkTPCC_84d3b1e17651f113943003fabf6735b7dac79baa_mt_highcont.json | /hyrise/rel_amd/benchmark_all_results/hyriseBenchmarkTPCC_da72f86758477648ff8cee02d6098f96f66155c6_mt_highcont.json |
+-------------------------------+---------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 84d3b1e17651f113943003fabf6735b7dac79baa-dirty | da72f86758477648ff8cee02d6098f96f66155c6-dirty |
| benchmark_mode | Shuffled | Shuffled |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 64 | 64 |
| compiler | clang 17.0.2 | clang 17.0.2 |
| cores | 64 | 64 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-15 17:37:36 | 2024-05-15 23:45:02 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 600000000000 | 600000000000 |
| max_runs | -1 | -1 |
| scale_factor | 10 | 10 |
| time_unit | ns | ns |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [64] | [64] |
| verify | False | False |
| warmup_duration | 0 | 0 |
+-------------------------------+---------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------+ +--------------++----------+---------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+--------------++----------+---------+--------++----------+----------+--------+---------+
| Delivery || 378.04 | 364.39 | -4% || 15.90 | 16.41 | +3% | 0.0000 |
| unsucc.: || 4.85 | 4.73 | -2% || 23.15 | 23.49 | +1% | |
| New-Order || 230.18 | 224.51 | -2% || 202.36 | 208.95 | +3% | 0.0000 |
| unsucc.: || 12.82 | 12.04 | -6% || 236.12 | 239.64 | +1% | |
+| Order-Status || 17.02 | 16.05 | -6% || 39.37 | 39.76 | +1% | 0.0003 |
+| Payment || 38.50 | 35.09 | -9% || 105.21 | 111.02 | +6% | 0.0000 |
| unsucc.: || 8.18 | 8.18 | +0% || 314.82 | 319.06 | +1% | |
| Stock-Level || 23.01 | 22.72 | -1% || 39.15 | 39.82 | +2% | 0.5373 |
+--------------++----------+---------+--------++----------+----------+--------+---------+
| Sum || 686.74 | 662.76 | -3% || | | | |
| Geomean || | | || | | +3% | |
+--------------++----------+---------+--------++----------+----------+--------+---------+hyriseBenchmarkTPCC - multi-threaded, shuffled, 10 clients, 10 warehouses, 64 cores (low contention) Sum of avg. item runtimes: -11% || Geometric mean of throughput changes: +13%Configuration Overview - click to expand +Configuration Overview---------+--------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------+
| Parameter | /hyrise/rel_amd/benchmark_all_results/hyriseBenchmarkTPCC_84d3b1e17651f113943003fabf6735b7dac79baa_mt_lowcont.json | /hyrise/rel_amd/benchmark_all_results/hyriseBenchmarkTPCC_da72f86758477648ff8cee02d6098f96f66155c6_mt_lowcont.json |
+-------------------------------+--------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 84d3b1e17651f113943003fabf6735b7dac79baa-dirty | da72f86758477648ff8cee02d6098f96f66155c6-dirty |
| benchmark_mode | Shuffled | Shuffled |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 10 | 10 |
| compiler | clang 17.0.2 | clang 17.0.2 |
| cores | 64 | 64 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-15 17:47:48 | 2024-05-15 23:55:13 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 600000000000 | 600000000000 |
| max_runs | -1 | -1 |
| scale_factor | 10 | 10 |
| time_unit | ns | ns |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [64] | [64] |
| verify | False | False |
| warmup_duration | 0 | 0 |
+-------------------------------+--------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------+ +--------------++----------+---------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+--------------++----------+---------+--------++----------+----------+--------+---------+
+| Delivery || 144.44 | 128.16 | -11% || 7.09 | 7.73 | +9% | 0.0000 |
| unsucc.: || 1.20 | 0.72 | -40% || 0.69 | 0.80 | +16% | |
+| New-Order || 97.20 | 87.29 | -10% || 76.69 | 86.33 | +13% | 0.0000 |
| unsucc.: || 6.16 | 6.24 | +1% || 7.89 | 8.88 | +12% | |
+| Order-Status || 5.87 | 5.43 | -7% || 7.64 | 8.70 | +14% | 0.0000 |
+| Payment || 20.10 | 16.67 | -17% || 65.54 | 75.33 | +15% | 0.0000 |
| unsucc.: || 2.27 | 2.07 | -9% || 15.31 | 16.59 | +8% | |
+| Stock-Level || 8.44 | 7.94 | -6% || 7.74 | 8.70 | +12% | 0.0000 |
+--------------++----------+---------+--------++----------+----------+--------+---------+
+| Sum || 276.06 | 245.49 | -11% || | | | |
+| Geomean || | | || | | +13% | |
+--------------++----------+---------+--------++----------+----------+--------+---------+hyriseBenchmarkJoinOrder - single-threaded Sum of avg. item runtimes: -7% || Geometric mean of throughput changes: +10%Configuration Overview - click to expand +Configuration Overview---+-----------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------+
| Parameter | /hyrise/rel_amd/benchmark_all_results/hyriseBenchmarkJoinOrder_84d3b1e17651f113943003fabf6735b7dac79baa_st.json | /hyrise/rel_amd/benchmark_all_results/hyriseBenchmarkJoinOrder_da72f86758477648ff8cee02d6098f96f66155c6_st.json |
+-------------------------+-----------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 84d3b1e17651f113943003fabf6735b7dac79baa-dirty | da72f86758477648ff8cee02d6098f96f66155c6-dirty |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| compiler | clang 17.0.2 | clang 17.0.2 |
| cores | 0 | 0 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-15 17:57:59 | 2024-05-16 00:05:23 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 60000000000 | 60000000000 |
| max_runs | 50 | 50 |
| time_unit | ns | ns |
| using_scheduler | False | False |
| verify | False | False |
| warmup_duration | 1000000000 | 1000000000 |
+-------------------------+-----------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------+ +---------++----------+----------+--------++----------+----------+--------+----------------------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+---------++----------+----------+--------++----------+----------+--------+----------------------+
+| 10a || 134.32 | 119.14 | -11%˄ || 7.44 | 8.39 | +13%˄ | (run time too short) |
+| 10b || 87.84 | 71.71 | -18%˄ || 11.38 | 13.94 | +22%˄ | (run time too short) |
+| 10c || 441.27 | 413.11 | -6%˄ || 2.27 | 2.42 | +7%˄ | (run time too short) |
+| 11a || 129.27 | 119.05 | -8%˄ || 7.74 | 8.40 | +9%˄ | (run time too short) |
+| 11b || 125.11 | 117.08 | -6%˄ || 7.99 | 8.54 | +7%˄ | (run time too short) |
| 11c || 28.70 | 27.72 | -3%˄ || 34.83 | 36.07 | +4%˄ | (run time too short) |
+| 11d || 31.46 | 30.10 | -4%˄ || 31.78 | 33.22 | +5%˄ | (run time too short) |
+| 12a || 40.74 | 36.64 | -10%˄ || 24.54 | 27.29 | +11%˄ | (run time too short) |
| 12b || 33.01 | 33.65 | +2%˄ || 30.29 | 29.72 | -2%˄ | (run time too short) |
+| 12c || 106.32 | 96.84 | -9%˄ || 9.41 | 10.33 | +10%˄ | (run time too short) |
+| 13a || 236.91 | 222.88 | -6%˄ || 4.22 | 4.49 | +6%˄ | (run time too short) |
| 13b || 284.16 | 275.28 | -3%˄ || 3.52 | 3.63 | +3%˄ | (run time too short) |
+| 13c || 226.91 | 215.22 | -5%˄ || 4.41 | 4.65 | +5%˄ | (run time too short) |
+| 13d || 613.27 | 561.13 | -9%˄ || 1.63 | 1.78 | +9%˄ | (run time too short) |
+| 14a || 144.16 | 132.32 | -8%˄ || 6.94 | 7.56 | +9%˄ | (run time too short) |
+| 14b || 143.06 | 128.25 | -10%˄ || 6.99 | 7.80 | +12%˄ | (run time too short) |
+| 14c || 276.59 | 227.23 | -18%˄ || 3.62 | 4.40 | +22%˄ | (run time too short) |
| 15a || 63.68 | 61.43 | -4%˄ || 15.70 | 16.28 | +4%˄ | (run time too short) |
+| 15b || 68.09 | 63.90 | -6%˄ || 14.69 | 15.65 | +7%˄ | (run time too short) |
+| 15c || 57.86 | 50.26 | -13%˄ || 17.28 | 19.89 | +15%˄ | (run time too short) |
+| 15d || 56.70 | 48.25 | -15%˄ || 17.64 | 20.72 | +17%˄ | (run time too short) |
| 16a || 1935.87 | 1864.22 | -4% || 0.52 | 0.54 | +4% | 0.0000 |
+| 16b || 3240.07 | 3026.52 | -7% || 0.31 | 0.33 | +7% | 0.0000 |
+| 16c || 2155.92 | 2047.32 | -5% || 0.46 | 0.49 | +5% | 0.0000 |
| 16d || 2103.23 | 2022.70 | -4% || 0.48 | 0.49 | +4% | 0.0000 |
+| 17a || 761.69 | 684.38 | -10%˄ || 1.31 | 1.46 | +11%˄ | (run time too short) |
+| 17b || 634.65 | 567.94 | -11%˄ || 1.58 | 1.76 | +12%˄ | (run time too short) |
+| 17c || 608.89 | 545.60 | -10%˄ || 1.64 | 1.83 | +12%˄ | (run time too short) |
+| 17d || 675.54 | 605.46 | -10%˄ || 1.48 | 1.65 | +12%˄ | (run time too short) |
+| 17e || 2187.31 | 2042.73 | -7% || 0.46 | 0.49 | +7% | 0.0000 |
+| 17f || 1260.86 | 1158.72 | -8%˄ || 0.79 | 0.86 | +9%˄ | (run time too short) |
+| 18a || 155.18 | 141.10 | -9%˄ || 6.44 | 7.09 | +10%˄ | (run time too short) |
+| 18b || 149.07 | 128.43 | -14%˄ || 6.71 | 7.79 | +16%˄ | (run time too short) |
+| 18c || 301.03 | 266.51 | -11%˄ || 3.32 | 3.75 | +13%˄ | (run time too short) |
+| 19a || 290.97 | 246.66 | -15%˄ || 3.44 | 4.05 | +18%˄ | (run time too short) |
+| 19b || 240.46 | 215.04 | -11%˄ || 4.16 | 4.65 | +12%˄ | (run time too short) |
+| 19c || 338.32 | 296.09 | -12%˄ || 2.96 | 3.38 | +14%˄ | (run time too short) |
+| 19d || 845.32 | 734.57 | -13%˄ || 1.18 | 1.36 | +15%˄ | (run time too short) |
| 1a || 13.90 | 13.60 | -2%˄ || 71.90 | 73.50 | +2%˄ | (run time too short) |
+| 1b || 22.20 | 18.95 | -15%˄ || 45.04 | 52.76 | +17%˄ | (run time too short) |
+| 1c || 21.43 | 18.28 | -15%˄ || 46.66 | 54.70 | +17%˄ | (run time too short) |
| 1d || 13.27 | 12.86 | -3%˄ || 75.32 | 77.70 | +3%˄ | (run time too short) |
| 20a || 767.54 | 742.82 | -3%˄ || 1.30 | 1.35 | +3%˄ | (run time too short) |
+| 20b || 808.35 | 770.14 | -5%˄ || 1.24 | 1.30 | +5%˄ | (run time too short) |
| 20c || 470.98 | 458.40 | -3%˄ || 2.12 | 2.18 | +3%˄ | (run time too short) |
+| 21a || 149.00 | 132.46 | -11%˄ || 6.71 | 7.55 | +12%˄ | (run time too short) |
+| 21b || 134.70 | 122.29 | -9%˄ || 7.42 | 8.18 | +10%˄ | (run time too short) |
+| 21c || 143.61 | 130.14 | -9%˄ || 6.96 | 7.68 | +10%˄ | (run time too short) |
+| 22a || 236.76 | 215.29 | -9%˄ || 4.22 | 4.64 | +10%˄ | (run time too short) |
+| 22b || 215.89 | 195.71 | -9%˄ || 4.63 | 5.11 | +10%˄ | (run time too short) |
+| 22c || 322.17 | 291.39 | -10%˄ || 3.10 | 3.43 | +11%˄ | (run time too short) |
+| 22d || 496.23 | 457.24 | -8%˄ || 2.02 | 2.19 | +9%˄ | (run time too short) |
+| 23a || 58.48 | 54.29 | -7%˄ || 17.10 | 18.42 | +8%˄ | (run time too short) |
+| 23b || 74.01 | 70.47 | -5%˄ || 13.51 | 14.19 | +5%˄ | (run time too short) |
+| 23c || 63.32 | 59.03 | -7%˄ || 15.79 | 16.94 | +7%˄ | (run time too short) |
+| 24a || 256.28 | 223.87 | -13%˄ || 3.90 | 4.47 | +14%˄ | (run time too short) |
+| 24b || 247.61 | 215.92 | -13%˄ || 4.04 | 4.63 | +15%˄ | (run time too short) |
+| 25a || 160.70 | 141.39 | -12%˄ || 6.22 | 7.07 | +14%˄ | (run time too short) |
+| 25b || 95.24 | 81.91 | -14%˄ || 10.50 | 12.21 | +16%˄ | (run time too short) |
+| 25c || 367.40 | 328.93 | -10%˄ || 2.72 | 3.04 | +12%˄ | (run time too short) |
| 26a || 265.84 | 262.28 | -1%˄ || 3.76 | 3.81 | +1%˄ | (run time too short) |
| 26b || 230.27 | 223.56 | -3%˄ || 4.34 | 4.47 | +3%˄ | (run time too short) |
| 26c || 445.71 | 433.64 | -3%˄ || 2.24 | 2.31 | +3%˄ | (run time too short) |
+| 27a || 138.32 | 124.11 | -10%˄ || 7.23 | 8.06 | +11%˄ | (run time too short) |
+| 27b || 122.58 | 109.36 | -11%˄ || 8.16 | 9.14 | +12%˄ | (run time too short) |
+| 27c || 143.69 | 129.68 | -10%˄ || 6.96 | 7.71 | +11%˄ | (run time too short) |
+| 28a || 264.31 | 245.97 | -7%˄ || 3.78 | 4.07 | +7%˄ | (run time too short) |
+| 28b || 99.63 | 88.83 | -11%˄ || 10.04 | 11.26 | +12%˄ | (run time too short) |
+| 28c || 242.22 | 220.47 | -9%˄ || 4.13 | 4.54 | +10%˄ | (run time too short) |
+| 29a || 204.90 | 177.48 | -13%˄ || 4.88 | 5.63 | +15%˄ | (run time too short) |
| 29b || 895.38 | 864.74 | -3%˄ || 1.12 | 1.16 | +4%˄ | (run time too short) |
+| 29c || 262.19 | 228.72 | -13%˄ || 3.81 | 4.37 | +15%˄ | (run time too short) |
+| 2a || 44.95 | 41.50 | -8%˄ || 22.24 | 24.09 | +8%˄ | (run time too short) |
+| 2b || 41.42 | 38.18 | -8%˄ || 24.14 | 26.19 | +9%˄ | (run time too short) |
+| 2c || 35.23 | 32.70 | -7%˄ || 28.38 | 30.57 | +8%˄ | (run time too short) |
+| 2d || 87.32 | 80.37 | -8%˄ || 11.45 | 12.44 | +9%˄ | (run time too short) |
+| 30a || 137.67 | 120.17 | -13%˄ || 7.26 | 8.32 | +15%˄ | (run time too short) |
+| 30b || 136.28 | 123.77 | -9%˄ || 7.34 | 8.08 | +10%˄ | (run time too short) |
+| 30c || 241.41 | 212.23 | -12%˄ || 4.14 | 4.71 | +14%˄ | (run time too short) |
+| 31a || 116.87 | 102.30 | -12%˄ || 8.56 | 9.77 | +14%˄ | (run time too short) |
+| 31b || 113.30 | 99.10 | -13%˄ || 8.83 | 10.09 | +14%˄ | (run time too short) |
+| 31c || 131.37 | 113.76 | -13%˄ || 7.61 | 8.79 | +15%˄ | (run time too short) |
| 32a || 23.40 | 22.63 | -3%˄ || 42.72 | 44.18 | +3%˄ | (run time too short) |
+| 32b || 54.51 | 50.28 | -8%˄ || 18.34 | 19.89 | +8%˄ | (run time too short) |
+| 33a || 47.23 | 40.82 | -14%˄ || 21.17 | 24.50 | +16%˄ | (run time too short) |
+| 33b || 29.43 | 26.45 | -10%˄ || 33.98 | 37.81 | +11%˄ | (run time too short) |
+| 33c || 60.62 | 53.46 | -12%˄ || 16.50 | 18.71 | +13%˄ | (run time too short) |
+| 3a || 107.24 | 89.75 | -16%˄ || 9.32 | 11.14 | +19%˄ | (run time too short) |
+| 3b || 24.44 | 21.01 | -14%˄ || 40.90 | 47.59 | +16%˄ | (run time too short) |
+| 3c || 301.17 | 276.14 | -8%˄ || 3.32 | 3.62 | +9%˄ | (run time too short) |
+| 4a || 198.36 | 184.70 | -7%˄ || 5.04 | 5.41 | +7%˄ | (run time too short) |
+| 4b || 22.27 | 19.69 | -12%˄ || 44.88 | 50.77 | +13%˄ | (run time too short) |
| 4c || 252.09 | 241.42 | -4%˄ || 3.97 | 4.14 | +4%˄ | (run time too short) |
+| 5a || 70.45 | 58.06 | -18%˄ || 14.19 | 17.22 | +21%˄ | (run time too short) |
+| 5b || 62.76 | 55.19 | -12%˄ || 15.93 | 18.12 | +14%˄ | (run time too short) |
+| 5c || 117.37 | 100.84 | -14%˄ || 8.52 | 9.92 | +16%˄ | (run time too short) |
| 6a || 223.88 | 214.36 | -4%˄ || 4.47 | 4.66 | +4%˄ | (run time too short) |
| 6b || 241.57 | 233.26 | -3%˄ || 4.14 | 4.29 | +4%˄ | (run time too short) |
| 6c || 209.82 | 202.91 | -3%˄ || 4.77 | 4.93 | +3%˄ | (run time too short) |
+| 6d || 564.22 | 527.66 | -6%˄ || 1.77 | 1.90 | +7%˄ | (run time too short) |
| 6e || 220.51 | 212.31 | -4%˄ || 4.53 | 4.71 | +4%˄ | (run time too short) |
+| 6f || 1046.72 | 974.69 | -7%˄ || 0.96 | 1.03 | +7%˄ | (run time too short) |
+| 7a || 98.52 | 91.68 | -7%˄ || 10.15 | 10.91 | +7%˄ | (run time too short) |
+| 7b || 89.59 | 84.40 | -6%˄ || 11.16 | 11.85 | +6%˄ | (run time too short) |
+| 7c || 728.40 | 665.21 | -9%˄ || 1.37 | 1.50 | +9%˄ | (run time too short) |
| 8a || 197.21 | 196.05 | -1%˄ || 5.07 | 5.10 | +1%˄ | (run time too short) |
| 8b || 190.43 | 183.64 | -4%˄ || 5.25 | 5.45 | +4%˄ | (run time too short) |
+| 8c || 2228.20 | 2080.32 | -7% || 0.45 | 0.48 | +7% | 0.0000 |
+| 8d || 367.39 | 345.99 | -6%˄ || 2.72 | 2.89 | +6%˄ | (run time too short) |
+| 9a || 347.26 | 319.76 | -8%˄ || 2.88 | 3.13 | +9%˄ | (run time too short) |
+| 9b || 222.84 | 211.16 | -5%˄ || 4.49 | 4.74 | +6%˄ | (run time too short) |
+| 9c || 372.08 | 327.91 | -12%˄ || 2.69 | 3.05 | +13%˄ | (run time too short) |
+| 9d || 586.02 | 506.81 | -14%˄ || 1.71 | 1.97 | +16%˄ | (run time too short) |
+---------++----------+----------+--------++----------+----------+--------+----------------------+
+| Sum || 40059.74 | 37092.04 | -7% || | | | |
+| Geomean || | | || | | +10% | |
+---------++----------+----------+--------++----------+----------+--------+----------------------+
| Notes || ˄ Execution stopped due to max runs reached |
+---------++----------+----------+--------++----------+----------+--------+----------------------+hyriseBenchmarkJoinOrder - multi-threaded, ordered, 1 client, 64 cores Sum of avg. item runtimes: -7% || Geometric mean of throughput changes: +6%Configuration Overview - click to expand +Configuration Overview---------+-------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------+
| Parameter | /hyrise/rel_amd/benchmark_all_results/hyriseBenchmarkJoinOrder_84d3b1e17651f113943003fabf6735b7dac79baa_mt_ordered.json | /hyrise/rel_amd/benchmark_all_results/hyriseBenchmarkJoinOrder_da72f86758477648ff8cee02d6098f96f66155c6_mt_ordered.json |
+-------------------------------+-------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 84d3b1e17651f113943003fabf6735b7dac79baa-dirty | da72f86758477648ff8cee02d6098f96f66155c6-dirty |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| compiler | clang 17.0.2 | clang 17.0.2 |
| cores | 64 | 64 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-15 18:28:37 | 2024-05-16 00:34:12 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 60000000000 | 60000000000 |
| max_runs | 50 | 50 |
| time_unit | ns | ns |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [64] | [64] |
| verify | False | False |
| warmup_duration | 1000000000 | 1000000000 |
+-------------------------------+-------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------+ +---------++----------+----------+--------++----------+----------+--------+----------------------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+---------++----------+----------+--------++----------+----------+--------+----------------------+
+| 10a || 42.63 | 39.93 | -6%˄ || 23.42 | 25.00 | +7%˄ | (run time too short) |
+| 10b || 19.49 | 18.65 | -4%˄ || 51.18 | 53.50 | +5%˄ | (run time too short) |
+| 10c || 143.95 | 128.14 | -11%˄ || 6.94 | 7.80 | +12%˄ | (run time too short) |
| 11a || 25.59 | 24.82 | -3%˄ || 39.00 | 40.20 | +3%˄ | (run time too short) |
+| 11b || 23.06 | 21.23 | -8%˄ || 43.25 | 46.99 | +9%˄ | (run time too short) |
+| 11c || 25.40 | 23.42 | -8%˄ || 39.24 | 42.56 | +8%˄ | (run time too short) |
+| 11d || 31.40 | 29.53 | -6%˄ || 31.70 | 33.64 | +6%˄ | (run time too short) |
+| 12a || 26.96 | 25.65 | -5%˄ || 36.99 | 38.88 | +5%˄ | (run time too short) |
+| 12b || 15.03 | 14.29 | -5%˄ || 66.25 | 69.73 | +5%˄ | (run time too short) |
+| 12c || 62.76 | 58.64 | -7%˄ || 15.91 | 17.03 | +7%˄ | (run time too short) |
+| 13a || 187.68 | 173.19 | -8%˄ || 5.32 | 5.77 | +8%˄ | (run time too short) |
| 13b || 112.78 | 116.34 | +3%˄ || 8.86 | 8.59 | -3%˄ | (run time too short) |
| 13c || 91.18 | 90.96 | -0%˄ || 10.96 | 10.98 | +0%˄ | (run time too short) |
+| 13d || 583.11 | 506.61 | -13%˄ || 1.71 | 1.97 | +15%˄ | (run time too short) |
+| 14a || 74.00 | 67.97 | -8%˄ || 13.50 | 14.70 | +9%˄ | (run time too short) |
+| 14b || 89.96 | 83.31 | -7%˄ || 11.10 | 11.99 | +8%˄ | (run time too short) |
+| 14c || 185.61 | 173.63 | -6%˄ || 5.38 | 5.76 | +7%˄ | (run time too short) |
| 15a || 34.53 | 34.00 | -2%˄ || 28.90 | 29.35 | +2%˄ | (run time too short) |
| 15b || 27.69 | 26.61 | -4%˄ || 36.02 | 37.48 | +4%˄ | (run time too short) |
+| 15c || 26.54 | 25.07 | -6%˄ || 37.57 | 39.78 | +6%˄ | (run time too short) |
+| 15d || 33.41 | 31.36 | -6%˄ || 29.87 | 31.80 | +6%˄ | (run time too short) |
| 16a || 974.67 | 948.48 | -3%˄ || 1.03 | 1.05 | +3%˄ | (run time too short) |
+| 16b || 2567.72 | 2413.94 | -6% || 0.39 | 0.41 | +6% | 0.0000 |
+| 16c || 1223.10 | 1170.26 | -4%˄ || 0.82 | 0.85 | +5%˄ | (run time too short) |
| 16d || 1177.14 | 1131.72 | -4%˄ || 0.85 | 0.88 | +4%˄ | (run time too short) |
+| 17a || 250.87 | 228.08 | -9%˄ || 3.98 | 4.38 | +10%˄ | (run time too short) |
+| 17b || 198.83 | 190.24 | -4%˄ || 5.03 | 5.25 | +5%˄ | (run time too short) |
| 17c || 162.94 | 158.06 | -3%˄ || 6.13 | 6.32 | +3%˄ | (run time too short) |
+| 17d || 147.95 | 135.40 | -8%˄ || 6.76 | 7.38 | +9%˄ | (run time too short) |
+| 17e || 1178.96 | 1064.08 | -10%˄ || 0.85 | 0.94 | +11%˄ | (run time too short) |
+| 17f || 481.34 | 434.14 | -10%˄ || 2.08 | 2.30 | +11%˄ | (run time too short) |
+| 18a || 62.50 | 59.73 | -4%˄ || 15.98 | 16.72 | +5%˄ | (run time too short) |
+| 18b || 63.67 | 59.97 | -6%˄ || 15.69 | 16.66 | +6%˄ | (run time too short) |
+| 18c || 113.29 | 106.37 | -6%˄ || 8.82 | 9.40 | +7%˄ | (run time too short) |
+| 19a || 117.62 | 108.78 | -8%˄ || 8.50 | 9.19 | +8%˄ | (run time too short) |
+| 19b || 76.00 | 70.61 | -7%˄ || 13.14 | 14.15 | +8%˄ | (run time too short) |
| 19c || 145.63 | 140.23 | -4%˄ || 6.86 | 7.13 | +4%˄ | (run time too short) |
+| 19d || 856.39 | 757.18 | -12%˄ || 1.17 | 1.32 | +13%˄ | (run time too short) |
| 1a || 7.83 | 7.58 | -3%˄ || 127.02 | 131.20 | +3%˄ | (run time too short) |
| 1b || 4.99 | 4.91 | -2%˄ || 198.82 | 202.14 | +2%˄ | (run time too short) |
| 1c || 4.51 | 4.41 | -2%˄ || 219.69 | 224.76 | +2%˄ | (run time too short) |
| 1d || 6.23 | 6.08 | -2%˄ || 159.35 | 163.16 | +2%˄ | (run time too short) |
| 20a || 92.71 | 91.41 | -1%˄ || 10.78 | 10.93 | +1%˄ | (run time too short) |
+| 20b || 79.96 | 76.35 | -5%˄ || 12.50 | 13.09 | +5%˄ | (run time too short) |
| 20c || 96.12 | 93.17 | -3%˄ || 10.40 | 10.73 | +3%˄ | (run time too short) |
+| 21a || 33.16 | 30.88 | -7%˄ || 30.10 | 32.32 | +7%˄ | (run time too short) |
+| 21b || 27.72 | 26.23 | -5%˄ || 36.01 | 38.04 | +6%˄ | (run time too short) |
+| 21c || 34.97 | 32.29 | -8%˄ || 28.54 | 30.91 | +8%˄ | (run time too short) |
+| 22a || 92.37 | 85.64 | -7%˄ || 10.82 | 11.67 | +8%˄ | (run time too short) |
+| 22b || 72.28 | 65.67 | -9%˄ || 13.82 | 15.21 | +10%˄ | (run time too short) |
+| 22c || 157.08 | 142.20 | -9%˄ || 6.36 | 7.03 | +10%˄ | (run time too short) |
+| 22d || 261.72 | 239.97 | -8%˄ || 3.82 | 4.17 | +9%˄ | (run time too short) |
+| 23a || 19.65 | 18.61 | -5%˄ || 50.76 | 53.59 | +6%˄ | (run time too short) |
| 23b || 31.21 | 30.42 | -3%˄ || 31.96 | 32.80 | +3%˄ | (run time too short) |
+| 23c || 22.40 | 21.15 | -6%˄ || 44.54 | 47.14 | +6%˄ | (run time too short) |
+| 24a || 67.17 | 63.96 | -5%˄ || 14.87 | 15.62 | +5%˄ | (run time too short) |
+| 24b || 73.81 | 68.35 | -7%˄ || 13.53 | 14.61 | +8%˄ | (run time too short) |
+| 25a || 72.07 | 68.43 | -5%˄ || 13.86 | 14.60 | +5%˄ | (run time too short) |
+| 25b || 41.16 | 39.33 | -4%˄ || 24.26 | 25.38 | +5%˄ | (run time too short) |
+| 25c || 220.58 | 207.73 | -6%˄ || 4.53 | 4.81 | +6%˄ | (run time too short) |
| 26a || 97.96 | 96.21 | -2%˄ || 10.20 | 10.39 | +2%˄ | (run time too short) |
| 26b || 70.60 | 70.23 | -1%˄ || 14.15 | 14.22 | +1%˄ | (run time too short) |
| 26c || 158.55 | 155.14 | -2%˄ || 6.30 | 6.44 | +2%˄ | (run time too short) |
+| 27a || 29.02 | 27.46 | -5%˄ || 34.39 | 36.35 | +6%˄ | (run time too short) |
+| 27b || 27.77 | 26.24 | -5%˄ || 35.94 | 38.02 | +6%˄ | (run time too short) |
+| 27c || 34.08 | 31.97 | -6%˄ || 29.29 | 31.22 | +7%˄ | (run time too short) |
+| 28a || 156.92 | 147.01 | -6%˄ || 6.37 | 6.80 | +7%˄ | (run time too short) |
+| 28b || 35.75 | 32.93 | -8%˄ || 27.93 | 30.31 | +9%˄ | (run time too short) |
+| 28c || 135.74 | 126.74 | -7%˄ || 7.36 | 7.88 | +7%˄ | (run time too short) |
-| 29a || 58.66 | 62.27 | +6%˄ || 17.02 | 16.04 | -6%˄ | (run time too short) |
+| 29b || 191.27 | 174.76 | -9%˄ || 5.23 | 5.72 | +9%˄ | (run time too short) |
+| 29c || 76.61 | 71.38 | -7%˄ || 13.04 | 14.00 | +7%˄ | (run time too short) |
+| 2a || 29.65 | 27.72 | -7%˄ || 33.63 | 36.01 | +7%˄ | (run time too short) |
+| 2b || 26.07 | 24.34 | -7%˄ || 38.30 | 40.98 | +7%˄ | (run time too short) |
| 2c || 17.32 | 16.69 | -4%˄ || 57.49 | 59.74 | +4%˄ | (run time too short) |
+| 2d || 69.89 | 64.81 | -7%˄ || 14.29 | 15.40 | +8%˄ | (run time too short) |
+| 30a || 62.13 | 58.65 | -6%˄ || 16.07 | 17.03 | +6%˄ | (run time too short) |
| 30b || 45.80 | 44.66 | -2%˄ || 21.80 | 22.36 | +3%˄ | (run time too short) |
| 30c || 144.33 | 138.63 | -4%˄ || 6.92 | 7.21 | +4%˄ | (run time too short) |
+| 31a || 48.36 | 46.05 | -5%˄ || 20.64 | 21.68 | +5%˄ | (run time too short) |
+| 31b || 42.41 | 37.07 | -13%˄ || 23.54 | 26.93 | +14%˄ | (run time too short) |
+| 31c || 53.12 | 49.78 | -6%˄ || 18.80 | 20.06 | +7%˄ | (run time too short) |
+| 32a || 8.22 | 7.78 | -5%˄ || 120.47 | 127.84 | +6%˄ | (run time too short) |
+| 32b || 39.84 | 37.86 | -5%˄ || 25.03 | 26.36 | +5%˄ | (run time too short) |
+| 33a || 12.97 | 12.32 | -5%˄ || 76.82 | 80.82 | +5%˄ | (run time too short) |
+| 33b || 10.30 | 9.67 | -6%˄ || 96.63 | 102.94 | +7%˄ | (run time too short) |
+| 33c || 22.09 | 20.38 | -8%˄ || 45.17 | 48.95 | +8%˄ | (run time too short) |
| 3a || 46.28 | 44.43 | -4%˄ || 21.58 | 22.47 | +4%˄ | (run time too short) |
| 3b || 7.91 | 7.61 | -4%˄ || 125.90 | 130.70 | +4%˄ | (run time too short) |
+| 3c || 91.67 | 87.27 | -5%˄ || 10.90 | 11.45 | +5%˄ | (run time too short) |
+| 4a || 83.69 | 78.06 | -7%˄ || 11.93 | 12.79 | +7%˄ | (run time too short) |
| 4b || 6.15 | 6.17 | +0%˄ || 161.62 | 161.09 | -0%˄ | (run time too short) |
+| 4c || 105.45 | 98.02 | -7%˄ || 9.47 | 10.19 | +8%˄ | (run time too short) |
+| 5a || 33.86 | 28.68 | -15%˄ || 29.48 | 34.80 | +18%˄ | (run time too short) |
+| 5b || 26.03 | 22.90 | -12%˄ || 38.35 | 43.58 | +14%˄ | (run time too short) |
+| 5c || 62.79 | 55.46 | -12%˄ || 15.91 | 18.01 | +13%˄ | (run time too short) |
| 6a || 30.17 | 29.98 | -1%˄ || 33.08 | 33.30 | +1%˄ | (run time too short) |
| 6b || 59.00 | 57.86 | -2%˄ || 16.93 | 17.26 | +2%˄ | (run time too short) |
| 6c || 29.12 | 27.89 | -4%˄ || 34.27 | 35.79 | +4%˄ | (run time too short) |
| 6d || 161.45 | 159.31 | -1%˄ || 6.19 | 6.27 | +1%˄ | (run time too short) |
| 6e || 30.52 | 30.86 | +1%˄ || 32.70 | 32.34 | -1%˄ | (run time too short) |
+| 6f || 751.19 | 679.46 | -10%˄ || 1.33 | 1.47 | +11%˄ | (run time too short) |
+| 7a || 31.17 | 28.28 | -9%˄ || 32.02 | 35.28 | +10%˄ | (run time too short) |
| 7b || 29.80 | 29.18 | -2%˄ || 33.49 | 34.19 | +2%˄ | (run time too short) |
| 7c || 361.30 | 348.87 | -3%˄ || 2.77 | 2.87 | +4%˄ | (run time too short) |
| 8a || 44.23 | 42.63 | -4%˄ || 22.58 | 23.42 | +4%˄ | (run time too short) |
| 8b || 39.99 | 39.11 | -2%˄ || 24.96 | 25.52 | +2%˄ | (run time too short) |
+| 8c || 1815.39 | 1690.47 | -7% || 0.55 | 0.59 | +7% | 0.0000 |
+| 8d || 321.69 | 287.22 | -11%˄ || 3.11 | 3.48 | +12%˄ | (run time too short) |
| 9a || 171.58 | 165.65 | -3%˄ || 5.82 | 6.03 | +4%˄ | (run time too short) |
| 9b || 94.93 | 93.50 | -2%˄ || 10.53 | 10.69 | +2%˄ | (run time too short) |
| 9c || 181.78 | 174.16 | -4%˄ || 5.50 | 5.74 | +4%˄ | (run time too short) |
+| 9d || 586.19 | 515.13 | -12%˄ || 1.71 | 1.94 | +14%˄ | (run time too short) |
+---------++----------+----------+--------++----------+----------+--------+----------------------+
+| Sum || 20089.86 | 18762.30 | -7% || | | | |
+| Geomean || | | || | | +6% | |
+---------++----------+----------+--------++----------+----------+--------+----------------------+
| Notes || ˄ Execution stopped due to max runs reached |
+---------++----------+----------+--------++----------+----------+--------+----------------------+hyriseBenchmarkJoinOrder - multi-threaded, shuffled, 64 clients, 64 cores Sum of avg. item runtimes: -3% || Geometric mean of throughput changes: +3%Configuration Overview - click to expand +Configuration Overview---------+-----------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------+
| Parameter | /hyrise/rel_amd/benchmark_all_results/hyriseBenchmarkJoinOrder_84d3b1e17651f113943003fabf6735b7dac79baa_mt.json | /hyrise/rel_amd/benchmark_all_results/hyriseBenchmarkJoinOrder_da72f86758477648ff8cee02d6098f96f66155c6_mt.json |
+-------------------------------+-----------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 84d3b1e17651f113943003fabf6735b7dac79baa-dirty | da72f86758477648ff8cee02d6098f96f66155c6-dirty |
| benchmark_mode | Shuffled | Shuffled |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 64 | 64 |
| compiler | clang 17.0.2 | clang 17.0.2 |
| cores | 64 | 64 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-15 18:47:03 | 2024-05-16 00:51:36 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 1200000000000 | 1200000000000 |
| max_runs | -1 | -1 |
| time_unit | ns | ns |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [64] | [64] |
| verify | False | False |
| warmup_duration | 0 | 0 |
+-------------------------------+-----------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------+ +---------++----------+----------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+---------++----------+----------+--------++----------+----------+--------+---------+
+| 10a || 230.52 | 193.05 | -16% || 1.13 | 1.17 | +3% | 0.0819 |
+| 10b || 167.10 | 147.71 | -12% || 1.13 | 1.17 | +3% | 0.2557 |
+| 10c || 668.29 | 605.01 | -9% || 1.13 | 1.17 | +3% | 0.0594 |
+| 11a || 138.42 | 122.48 | -12% || 1.13 | 1.17 | +3% | 0.3089 |
-| 11b || 115.75 | 123.23 | +6% || 1.13 | 1.17 | +3% | 0.6707 |
| 11c || 101.45 | 104.36 | +3% || 1.13 | 1.17 | +3% | 0.8320 |
-| 11d || 105.54 | 110.36 | +5% || 1.13 | 1.17 | +3% | 0.7168 |
-| 12a || 181.65 | 190.94 | +5% || 1.13 | 1.17 | +3% | 0.6582 |
| 12b || 99.87 | 96.29 | -4% || 1.13 | 1.17 | +3% | 0.8241 |
+| 12c || 408.13 | 367.16 | -10% || 1.13 | 1.17 | +3% | 0.1232 |
-| 13a || 492.71 | 521.62 | +6% || 1.13 | 1.17 | +3% | 0.2988 |
| 13b || 381.00 | 367.39 | -4% || 1.13 | 1.17 | +3% | 0.6362 |
+| 13c || 391.50 | 356.80 | -9% || 1.13 | 1.17 | +3% | 0.2495 |
+| 13d || 985.24 | 930.82 | -6% || 1.13 | 1.17 | +3% | 0.1146 |
+| 14a || 494.98 | 457.94 | -7% || 1.13 | 1.17 | +3% | 0.2991 |
| 14b || 461.57 | 449.27 | -3% || 1.13 | 1.17 | +3% | 0.6804 |
+| 14c || 622.77 | 581.49 | -7% || 1.13 | 1.17 | +3% | 0.1557 |
| 15a || 130.52 | 131.82 | +1% || 1.13 | 1.17 | +3% | 0.9334 |
+| 15b || 149.89 | 136.05 | -9% || 1.13 | 1.17 | +3% | 0.4343 |
-| 15c || 145.41 | 157.52 | +8% || 1.13 | 1.17 | +3% | 0.4732 |
| 15d || 179.13 | 175.16 | -2% || 1.13 | 1.17 | +3% | 0.8090 |
+| 16a || 1762.93 | 1668.21 | -5% || 1.13 | 1.17 | +3% | 0.0432 |
+| 16b || 3288.49 | 3083.94 | -6% || 1.12 | 1.16 | +3% | 0.0000 |
| 16c || 1970.33 | 1935.43 | -2% || 1.13 | 1.17 | +3% | 0.4636 |
| 16d || 1967.17 | 1913.45 | -3% || 1.13 | 1.16 | +3% | 0.2598 |
| 17a || 904.64 | 935.15 | +3% || 1.13 | 1.17 | +3% | 0.4287 |
| 17b || 721.54 | 690.88 | -4% || 1.13 | 1.17 | +3% | 0.3898 |
+| 17c || 648.01 | 605.45 | -7% || 1.13 | 1.17 | +3% | 0.1958 |
-| 17d || 611.91 | 647.76 | +6% || 1.13 | 1.17 | +4% | 0.2748 |
+| 17e || 1815.45 | 1722.98 | -5% || 1.13 | 1.16 | +3% | 0.0276 |
+| 17f || 1282.70 | 1189.25 | -7% || 1.13 | 1.17 | +3% | 0.0460 |
| 18a || 280.50 | 280.19 | -0% || 1.13 | 1.17 | +3% | 0.9886 |
-| 18b || 405.03 | 446.71 | +10% || 1.13 | 1.17 | +3% | 0.1891 |
+| 18c || 726.48 | 675.03 | -7% || 1.13 | 1.17 | +3% | 0.1934 |
+| 19a || 683.69 | 596.97 | -13% || 1.13 | 1.17 | +3% | 0.0244 |
| 19b || 473.63 | 454.26 | -4% || 1.13 | 1.17 | +3% | 0.5894 |
| 19c || 683.08 | 652.50 | -4% || 1.13 | 1.17 | +3% | 0.3837 |
| 19d || 1524.07 | 1478.57 | -3% || 1.13 | 1.16 | +3% | 0.3433 |
-| 1a || 37.26 | 50.38 | +35% || 1.13 | 1.17 | +3% | 0.1337 |
-| 1b || 59.55 | 65.02 | +9% || 1.13 | 1.17 | +3% | 0.6087 |
+| 1c || 72.21 | 61.14 | -15% || 1.13 | 1.17 | +3% | 0.3621 |
+| 1d || 32.50 | 29.07 | -11% || 1.13 | 1.17 | +3% | 0.4662 |
| 20a || 495.46 | 498.30 | +1% || 1.13 | 1.17 | +3% | 0.9276 |
+| 20b || 450.53 | 429.52 | -5% || 1.13 | 1.17 | +3% | 0.5045 |
| 20c || 439.82 | 443.15 | +1% || 1.13 | 1.17 | +3% | 0.9127 |
+| 21a || 212.03 | 201.23 | -5% || 1.13 | 1.17 | +3% | 0.6140 |
| 21b || 145.57 | 150.66 | +3% || 1.13 | 1.17 | +3% | 0.7532 |
| 21c || 235.55 | 234.03 | -1% || 1.13 | 1.17 | +3% | 0.9513 |
-| 22a || 522.89 | 587.06 | +12% || 1.13 | 1.17 | +3% | 0.0793 |
-| 22b || 426.44 | 450.14 | +6% || 1.13 | 1.16 | +3% | 0.3894 |
+| 22c || 778.05 | 676.76 | -13% || 1.13 | 1.17 | +3% | 0.0066 |
| 22d || 1022.33 | 988.73 | -3% || 1.13 | 1.17 | +3% | 0.4384 |
| 23a || 176.06 | 169.37 | -4% || 1.13 | 1.17 | +3% | 0.7092 |
-| 23b || 128.44 | 169.64 | +32% || 1.13 | 1.17 | +3% | 0.0099 |
-| 23c || 177.92 | 194.57 | +9% || 1.13 | 1.17 | +3% | 0.4339 |
| 24a || 441.96 | 449.89 | +2% || 1.13 | 1.17 | +3% | 0.8161 |
| 24b || 330.63 | 326.92 | -1% || 1.13 | 1.17 | +3% | 0.8952 |
+| 25a || 487.91 | 455.52 | -7% || 1.13 | 1.17 | +3% | 0.3415 |
| 25b || 238.65 | 231.42 | -3% || 1.13 | 1.17 | +3% | 0.7560 |
| 25c || 1019.91 | 1022.17 | +0% || 1.13 | 1.17 | +3% | 0.9606 |
| 26a || 464.64 | 452.31 | -3% || 1.13 | 1.17 | +3% | 0.6652 |
+| 26b || 307.47 | 268.46 | -13% || 1.13 | 1.17 | +3% | 0.0559 |
| 26c || 665.73 | 639.15 | -4% || 1.13 | 1.17 | +3% | 0.3780 |
+| 27a || 214.73 | 197.58 | -8% || 1.13 | 1.17 | +3% | 0.4448 |
+| 27b || 223.34 | 182.66 | -18% || 1.13 | 1.17 | +3% | 0.0784 |
+| 27c || 271.62 | 238.67 | -12% || 1.13 | 1.17 | +3% | 0.2535 |
| 28a || 695.45 | 686.93 | -1% || 1.13 | 1.17 | +3% | 0.8105 |
+| 28b || 281.70 | 259.33 | -8% || 1.13 | 1.17 | +3% | 0.3717 |
+| 28c || 724.25 | 660.12 | -9% || 1.13 | 1.17 | +3% | 0.0951 |
+| 29a || 335.52 | 319.26 | -5% || 1.13 | 1.17 | +3% | 0.5854 |
| 29b || 398.15 | 393.29 | -1% || 1.13 | 1.17 | +3% | 0.8721 |
+| 29c || 465.33 | 396.84 | -15% || 1.13 | 1.17 | +3% | 0.0378 |
+| 2a || 168.14 | 148.82 | -11% || 1.13 | 1.17 | +3% | 0.2615 |
-| 2b || 138.29 | 148.18 | +7% || 1.13 | 1.17 | +3% | 0.5094 |
+| 2c || 114.59 | 97.01 | -15% || 1.13 | 1.17 | +3% | 0.2598 |
+| 2d || 245.30 | 233.61 | -5% || 1.13 | 1.17 | +3% | 0.5187 |
| 30a || 403.01 | 398.58 | -1% || 1.13 | 1.17 | +3% | 0.8942 |
| 30b || 409.87 | 424.26 | +4% || 1.13 | 1.17 | +3% | 0.6719 |
| 30c || 729.92 | 742.61 | +2% || 1.13 | 1.17 | +3% | 0.7355 |
+| 31a || 346.28 | 329.78 | -5% || 1.13 | 1.17 | +3% | 0.6337 |
+| 31b || 295.32 | 281.84 | -5% || 1.13 | 1.17 | +3% | 0.6396 |
| 31c || 351.92 | 353.00 | +0% || 1.13 | 1.17 | +3% | 0.9687 |
| 32a || 58.19 | 57.35 | -1% || 1.13 | 1.17 | +3% | 0.9411 |
| 32b || 169.49 | 171.34 | +1% || 1.13 | 1.17 | +3% | 0.9031 |
| 33a || 147.00 | 152.08 | +3% || 1.13 | 1.17 | +3% | 0.8045 |
| 33b || 102.36 | 99.90 | -2% || 1.13 | 1.17 | +3% | 0.8605 |
-| 33c || 172.53 | 187.76 | +9% || 1.13 | 1.17 | +3% | 0.5055 |
-| 3a || 299.94 | 326.91 | +9% || 1.13 | 1.17 | +3% | 0.3393 |
+| 3b || 96.41 | 82.57 | -14% || 1.13 | 1.17 | +3% | 0.3701 |
| 3c || 495.82 | 479.38 | -3% || 1.13 | 1.17 | +3% | 0.6211 |
-| 4a || 286.23 | 310.54 | +8% || 1.13 | 1.17 | +3% | 0.3061 |
-| 4b || 58.47 | 68.90 | +18% || 1.13 | 1.17 | +3% | 0.4424 |
+| 4c || 377.83 | 334.26 | -12% || 1.13 | 1.17 | +3% | 0.1101 |
-| 5a || 244.48 | 277.38 | +13% || 1.13 | 1.17 | +3% | 0.2121 |
-| 5b || 162.59 | 178.23 | +10% || 1.13 | 1.17 | +3% | 0.3999 |
+| 5c || 383.28 | 339.36 | -11% || 1.13 | 1.17 | +3% | 0.1448 |
+| 6a || 163.07 | 138.28 | -15% || 1.13 | 1.17 | +3% | 0.1613 |
-| 6b || 195.49 | 205.34 | +5% || 1.13 | 1.17 | +3% | 0.5881 |
| 6c || 122.31 | 122.64 | +0% || 1.13 | 1.17 | +3% | 0.9780 |
+| 6d || 596.90 | 561.78 | -6% || 1.13 | 1.17 | +3% | 0.2759 |
+| 6e || 151.55 | 135.68 | -10% || 1.13 | 1.17 | +3% | 0.2801 |
| 6f || 1260.12 | 1219.14 | -3% || 1.13 | 1.17 | +3% | 0.2495 |
+| 7a || 187.82 | 170.75 | -9% || 1.13 | 1.17 | +3% | 0.4298 |
-| 7b || 178.20 | 189.89 | +7% || 1.13 | 1.17 | +3% | 0.5689 |
+| 7c || 1196.89 | 1142.99 | -5% || 1.13 | 1.17 | +3% | 0.2420 |
+| 8a || 268.40 | 232.60 | -13% || 1.13 | 1.17 | +3% | 0.1895 |
| 8b || 203.65 | 198.72 | -2% || 1.13 | 1.17 | +3% | 0.8209 |
| 8c || 2429.42 | 2446.06 | +1% || 1.13 | 1.16 | +3% | 0.7300 |
-| 8d || 700.92 | 739.12 | +5% || 1.13 | 1.17 | +3% | 0.2435 |
| 9a || 713.90 | 728.93 | +2% || 1.13 | 1.17 | +3% | 0.6803 |
| 9b || 493.53 | 484.93 | -2% || 1.13 | 1.17 | +3% | 0.7816 |
-| 9c || 695.62 | 755.63 | +9% || 1.13 | 1.17 | +3% | 0.1114 |
+| 9d || 1234.63 | 1152.69 | -7% || 1.13 | 1.17 | +3% | 0.0674 |
+---------++----------+----------+--------++----------+----------+--------+---------+
| Sum || 56430.37 | 54731.29 | -3% || | | | |
| Geomean || | | || | | +3% | |
+---------++----------+----------+--------++----------+----------+--------+---------+hyriseBenchmarkStarSchema - single-threaded Sum of avg. item runtimes: -8% || Geometric mean of throughput changes: +7%Configuration Overview - click to expand +Configuration Overview---+------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------+
| Parameter | /hyrise/rel_amd/benchmark_all_results/hyriseBenchmarkStarSchema_84d3b1e17651f113943003fabf6735b7dac79baa_st.json | /hyrise/rel_amd/benchmark_all_results/hyriseBenchmarkStarSchema_da72f86758477648ff8cee02d6098f96f66155c6_st.json |
+-------------------------+------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 84d3b1e17651f113943003fabf6735b7dac79baa-dirty | da72f86758477648ff8cee02d6098f96f66155c6-dirty |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| compiler | clang 17.0.2 | clang 17.0.2 |
| cores | 0 | 0 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-15 19:07:34 | 2024-05-16 01:12:05 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 60000000000 | 60000000000 |
| max_runs | 50 | 50 |
| scale_factor | 10.0 | 10.0 |
| time_unit | ns | ns |
| using_scheduler | False | False |
| verify | False | False |
| warmup_duration | 1000000000 | 1000000000 |
+-------------------------+------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------+ +---------++----------+----------+--------++----------+----------+--------+----------------------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+---------++----------+----------+--------++----------+----------+--------+----------------------+
| 1.1 || 476.76 | 468.60 | -2%˄ || 2.10 | 2.13 | +2%˄ | (run time too short) |
+| 1.2 || 245.29 | 230.14 | -6%˄ || 4.08 | 4.35 | +7%˄ | (run time too short) |
+| 1.3 || 233.96 | 223.56 | -4%˄ || 4.27 | 4.47 | +5%˄ | (run time too short) |
+| 2.1 || 836.73 | 767.22 | -8%˄ || 1.20 | 1.30 | +9%˄ | (run time too short) |
+| 2.2 || 442.68 | 421.57 | -5%˄ || 2.26 | 2.37 | +5%˄ | (run time too short) |
| 2.3 || 258.81 | 260.11 | +1%˄ || 3.86 | 3.84 | -0%˄ | (run time too short) |
+| 3.1 || 3210.00 | 2975.12 | -7% || 0.31 | 0.34 | +8% | 0.0000 |
+| 3.2 || 426.00 | 397.13 | -7%˄ || 2.35 | 2.52 | +7%˄ | (run time too short) |
+| 3.3 || 167.83 | 155.87 | -7%˄ || 5.96 | 6.42 | +8%˄ | (run time too short) |
| 3.4 || 154.44 | 148.13 | -4%˄ || 6.47 | 6.75 | +4%˄ | (run time too short) |
+| 4.1 || 3641.59 | 3302.49 | -9% || 0.27 | 0.30 | +10% | 0.0000 |
+| 4.2 || 1086.26 | 944.81 | -13%˄ || 0.92 | 1.06 | +15%˄ | (run time too short) |
+| 4.3 || 322.16 | 287.09 | -11%˄ || 3.10 | 3.48 | +12%˄ | (run time too short) |
+---------++----------+----------+--------++----------+----------+--------+----------------------+
+| Sum || 11502.50 | 10581.86 | -8% || | | | |
+| Geomean || | | || | | +7% | |
+---------++----------+----------+--------++----------+----------+--------+----------------------+
| Notes || ˄ Execution stopped due to max runs reached |
+---------++----------+----------+--------++----------+----------+--------+----------------------+hyriseBenchmarkStarSchema - multi-threaded, ordered, 1 client, 64 cores Sum of avg. item runtimes: -4% || Geometric mean of throughput changes: +3%Configuration Overview - click to expand +Configuration Overview---------+--------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------+
| Parameter | /hyrise/rel_amd/benchmark_all_results/hyriseBenchmarkStarSchema_84d3b1e17651f113943003fabf6735b7dac79baa_mt_ordered.json | /hyrise/rel_amd/benchmark_all_results/hyriseBenchmarkStarSchema_da72f86758477648ff8cee02d6098f96f66155c6_mt_ordered.json |
+-------------------------------+--------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 84d3b1e17651f113943003fabf6735b7dac79baa-dirty | da72f86758477648ff8cee02d6098f96f66155c6-dirty |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| compiler | clang 17.0.2 | clang 17.0.2 |
| cores | 64 | 64 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-15 19:14:13 | 2024-05-16 01:18:30 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 60000000000 | 60000000000 |
| max_runs | 50 | 50 |
| scale_factor | 10.0 | 10.0 |
| time_unit | ns | ns |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [64] | [64] |
| verify | False | False |
| warmup_duration | 1000000000 | 1000000000 |
+-------------------------------+--------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------+ +---------++----------+---------+--------++----------+----------+--------+----------------------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+---------++----------+---------+--------++----------+----------+--------+----------------------+
| 1.1 || 138.11 | 136.54 | -1%˄ || 7.24 | 7.32 | +1%˄ | (run time too short) |
| 1.2 || 95.15 | 94.53 | -1%˄ || 10.50 | 10.57 | +1%˄ | (run time too short) |
| 1.3 || 92.54 | 89.80 | -3%˄ || 10.79 | 11.12 | +3%˄ | (run time too short) |
| 2.1 || 423.49 | 414.05 | -2%˄ || 2.36 | 2.41 | +2%˄ | (run time too short) |
| 2.2 || 202.68 | 197.98 | -2%˄ || 4.93 | 5.05 | +2%˄ | (run time too short) |
| 2.3 || 74.25 | 72.57 | -2%˄ || 13.45 | 13.76 | +2%˄ | (run time too short) |
| 3.1 || 1263.42 | 1210.60 | -4%˄ || 0.79 | 0.83 | +4%˄ | 0.0000 |
| 3.2 || 192.33 | 188.51 | -2%˄ || 5.20 | 5.30 | +2%˄ | (run time too short) |
| 3.3 || 90.47 | 87.70 | -3%˄ || 11.04 | 11.39 | +3%˄ | (run time too short) |
+| 3.4 || 85.97 | 79.76 | -7%˄ || 11.62 | 12.53 | +8%˄ | (run time too short) |
+| 4.1 || 1115.93 | 1055.80 | -5%˄ || 0.90 | 0.95 | +6%˄ | (run time too short) |
+| 4.2 || 443.45 | 418.01 | -6%˄ || 2.25 | 2.39 | +6%˄ | (run time too short) |
| 4.3 || 124.71 | 119.64 | -4%˄ || 8.01 | 8.35 | +4%˄ | (run time too short) |
+---------++----------+---------+--------++----------+----------+--------+----------------------+
| Sum || 4342.51 | 4165.49 | -4% || | | | |
| Geomean || | | || | | +3% | |
+---------++----------+---------+--------++----------+----------+--------+----------------------+
| Notes || ˄ Execution stopped due to max runs reached |
+---------++----------+---------+--------++----------+----------+--------+----------------------+hyriseBenchmarkStarSchema - multi-threaded, shuffled, 64 clients, 64 cores Sum of avg. item runtimes: -3% || Geometric mean of throughput changes: +3%Configuration Overview - click to expand +Configuration Overview---------+------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------+
| Parameter | /hyrise/rel_amd/benchmark_all_results/hyriseBenchmarkStarSchema_84d3b1e17651f113943003fabf6735b7dac79baa_mt.json | /hyrise/rel_amd/benchmark_all_results/hyriseBenchmarkStarSchema_da72f86758477648ff8cee02d6098f96f66155c6_mt.json |
+-------------------------------+------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 84d3b1e17651f113943003fabf6735b7dac79baa-dirty | da72f86758477648ff8cee02d6098f96f66155c6-dirty |
| benchmark_mode | Shuffled | Shuffled |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 64 | 64 |
| compiler | clang 17.0.2 | clang 17.0.2 |
| cores | 64 | 64 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-15 19:18:34 | 2024-05-16 01:22:45 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 1200000000000 | 1200000000000 |
| max_runs | -1 | -1 |
| scale_factor | 10.0 | 10.0 |
| time_unit | ns | ns |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [64] | [64] |
| verify | False | False |
| warmup_duration | 0 | 0 |
+-------------------------------+------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------+ +---------++----------+----------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+---------++----------+----------+--------++----------+----------+--------+---------+
| 1.1 || 896.51 | 856.31 | -4% || 3.61 | 3.70 | +3% | 0.0725 |
| 1.2 || 476.40 | 476.42 | +0% || 3.61 | 3.70 | +3% | 0.9989 |
+| 1.3 || 449.13 | 416.72 | -7% || 3.61 | 3.70 | +3% | 0.0224 |
| 2.1 || 2175.73 | 2095.97 | -4% || 3.60 | 3.70 | +3% | 0.0273 |
+| 2.2 || 1167.79 | 1110.62 | -5% || 3.61 | 3.70 | +3% | 0.0317 |
| 2.3 || 377.76 | 374.89 | -1% || 3.61 | 3.70 | +3% | 0.8210 |
| 3.1 || 3427.34 | 3365.06 | -2% || 3.60 | 3.69 | +3% | 0.0925 |
+| 3.2 || 1073.55 | 1015.17 | -5% || 3.61 | 3.70 | +3% | 0.0166 |
| 3.3 || 524.46 | 521.78 | -1% || 3.61 | 3.70 | +3% | 0.8857 |
| 3.4 || 500.54 | 499.98 | -0% || 3.61 | 3.70 | +3% | 0.9749 |
| 4.1 || 3227.52 | 3228.61 | +0% || 3.60 | 3.69 | +3% | 0.9764 |
| 4.2 || 2626.57 | 2524.62 | -4% || 3.60 | 3.70 | +3% | 0.0104 |
| 4.3 || 792.64 | 774.32 | -2% || 3.61 | 3.70 | +3% | 0.4143 |
+---------++----------+----------+--------++----------+----------+--------+---------+
| Sum || 17715.93 | 17260.49 | -3% || | | | |
| Geomean || | | || | | +3% | |
+---------++----------+----------+--------++----------+----------+--------+---------+ |
I usually run |
|
Since you have been seeing different results than I had on nemea (at least some weeks ago that was the case) I would kindly ask to re-check the output if you have some results laying around. |
|
Nemea looks good to me (I think you had a different setup, so I am still interested in your output). System nemea - click to expand
Commit Info and Build Time
hyriseBenchmarkTPCH - single-threaded, SF 10.0 Sum of avg. item runtimes: -1% || Geometric mean of throughput changes: +1%Configuration Overview - click to expand +Configuration Overview----+-----------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------+
| Parameter | /home/Martin.Boissier/hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCH_43b228e1bc09ffbd5956577684cb99d0e060df1c_st.json | /home/Martin.Boissier/hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCH_556e8941a1168263ed4aedb7bf6bb4aaf25e8e28_st.json |
+--------------------------+-----------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 43b228e1bc09ffbd5956577684cb99d0e060df1c-dirty | 556e8941a1168263ed4aedb7bf6bb4aaf25e8e28-dirty |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| clustering | None | None |
| compiler | clang 15.0.7 | clang 15.0.7 |
| cores | 0 | 0 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-05 23:09:02 | 2024-05-06 02:48:04 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 60000000000 | 60000000000 |
| max_runs | 50 | 50 |
| scale_factor | 10.0 | 10.0 |
| time_unit | ns | ns |
| use_prepared_statements | False | False |
| using_scheduler | False | False |
| verify | False | False |
| warmup_duration | 1000000000 | 1000000000 |
+--------------------------+-----------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------+ +----------++----------+----------+--------++----------+----------+--------+----------------------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+----------++----------+----------+--------++----------+----------+--------+----------------------+
| TPC-H 01 || 6499.71 | 6421.73 | -1% || 0.15 | 0.16 | +1% | 0.5867 |
| TPC-H 02 || 49.64 | 48.93 | -1%˄ || 20.14 | 20.43 | +1%˄ | (run time too short) |
| TPC-H 03 || 1320.45 | 1332.59 | +1% || 0.76 | 0.75 | -1% | 0.4410 |
+| TPC-H 04 || 1399.07 | 1330.07 | -5% || 0.71 | 0.75 | +5% | 0.0006 |
| TPC-H 05 || 2217.48 | 2211.98 | -0% || 0.45 | 0.45 | +0% | 0.7011 |
| TPC-H 06 || 159.77 | 154.79 | -3%˄ || 6.26 | 6.46 | +3%˄ | (run time too short) |
| TPC-H 07 || 833.44 | 823.48 | -1%˄ || 1.20 | 1.21 | +1%˄ | (run time too short) |
| TPC-H 08 || 515.98 | 525.45 | +2%˄ || 1.94 | 1.90 | -2%˄ | (run time too short) |
| TPC-H 09 || 5097.73 | 5115.40 | +0% || 0.20 | 0.20 | -0% | 0.5663 |
+| TPC-H 10 || 1730.98 | 1277.14 | -26% || 0.58 | 0.78 | +36% | 0.0000 |
| TPC-H 11 || 64.99 | 62.97 | -3%˄ || 15.39 | 15.88 | +3%˄ | (run time too short) |
| TPC-H 12 || 927.89 | 932.86 | +1%˄ || 1.08 | 1.07 | -1%˄ | (run time too short) |
| TPC-H 13 || 5048.13 | 4918.39 | -3% || 0.20 | 0.20 | +3% | 0.0184 |
| TPC-H 14 || 414.49 | 413.34 | -0%˄ || 2.41 | 2.42 | +0%˄ | (run time too short) |
+| TPC-H 15 || 195.72 | 186.93 | -4%˄ || 5.11 | 5.35 | +5%˄ | (run time too short) |
| TPC-H 16 || 551.87 | 553.78 | +0%˄ || 1.81 | 1.81 | -0%˄ | (run time too short) |
| TPC-H 17 || 222.62 | 223.99 | +1%˄ || 4.49 | 4.46 | -1%˄ | (run time too short) |
-| TPC-H 18 || 1658.21 | 1858.42 | +12% || 0.60 | 0.54 | -11% | 0.0000 |
| TPC-H 19 || 217.53 | 213.99 | -2%˄ || 4.60 | 4.67 | +2%˄ | (run time too short) |
-| TPC-H 20 || 491.05 | 556.23 | +13%˄ || 2.04 | 1.80 | -12%˄ | (run time too short) |
-| TPC-H 21 || 3794.57 | 3968.33 | +5% || 0.26 | 0.25 | -4% | 0.0017 |
| TPC-H 22 || 440.06 | 434.99 | -1%˄ || 2.27 | 2.30 | +1%˄ | (run time too short) |
+----------++----------+----------+--------++----------+----------+--------+----------------------+
| Sum || 33851.37 | 33565.78 | -1% || | | | |
| Geomean || | | || | | +1% | |
+----------++----------+----------+--------++----------+----------+--------+----------------------+
| Notes || ˄ Execution stopped due to max runs reached |
+----------++----------+----------+--------++----------+----------+--------+----------------------+hyriseBenchmarkTPCH - single-threaded, SF 0.01 Sum of avg. item runtimes: +1% || Geometric mean of throughput changes: +1%Configuration Overview - click to expand +Configuration Overview----+---------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | /home/Martin.Boissier/hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCH_43b228e1bc09ffbd5956577684cb99d0e060df1c_st_s01.json | /home/Martin.Boissier/hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCH_556e8941a1168263ed4aedb7bf6bb4aaf25e8e28_st_s01.json |
+--------------------------+---------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 43b228e1bc09ffbd5956577684cb99d0e060df1c-dirty | 556e8941a1168263ed4aedb7bf6bb4aaf25e8e28-dirty |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| clustering | None | None |
| compiler | clang 15.0.7 | clang 15.0.7 |
| cores | 0 | 0 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-05 23:24:47 | 2024-05-06 03:03:59 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 60000000000 | 60000000000 |
| max_runs | 50 | 50 |
| scale_factor | 0.009999999776482582 | 0.009999999776482582 |
| time_unit | ns | ns |
| use_prepared_statements | False | False |
| using_scheduler | False | False |
| verify | False | False |
| warmup_duration | 1000000000 | 1000000000 |
+--------------------------+---------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------+ +----------++----------+---------+--------++----------+----------+--------+----------------------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+----------++----------+---------+--------++----------+----------+--------+----------------------+
| TPC-H 01 || 5.22 | 5.28 | +1%˄ || 191.40 | 189.29 | -1%˄ | (run time too short) |
+| TPC-H 02 || 4.32 | 3.85 | -11%˄ || 231.46 | 259.66 | +12%˄ | (run time too short) |
| TPC-H 03 || 0.56 | 0.58 | +3%˄ || 1779.37 | 1725.93 | -3%˄ | (run time too short) |
| TPC-H 04 || 0.43 | 0.44 | +3%˄ || 2346.09 | 2280.88 | -3%˄ | (run time too short) |
| TPC-H 05 || 1.01 | 1.05 | +4%˄ || 989.06 | 946.77 | -4%˄ | (run time too short) |
| TPC-H 06 || 0.18 | 0.18 | +0%˄ || 5594.45 | 5580.32 | -0%˄ | (run time too short) |
| TPC-H 07 || 11.47 | 11.52 | +0%˄ || 87.18 | 86.80 | -0%˄ | (run time too short) |
| TPC-H 08 || 16.12 | 16.53 | +3%˄ || 62.03 | 60.48 | -2%˄ | (run time too short) |
-| TPC-H 09 || 3.50 | 4.09 | +17%˄ || 285.21 | 244.37 | -14%˄ | (run time too short) |
+| TPC-H 10 || 0.78 | 0.66 | -15%˄ || 1285.22 | 1517.15 | +18%˄ | (run time too short) |
| TPC-H 11 || 0.20 | 0.20 | +1%˄ || 4905.55 | 4847.90 | -1%˄ | (run time too short) |
-| TPC-H 12 || 0.59 | 0.62 | +6%˄ || 1693.97 | 1599.13 | -6%˄ | (run time too short) |
| TPC-H 13 || 2.03 | 2.04 | +0%˄ || 491.24 | 489.77 | -0%˄ | (run time too short) |
| TPC-H 14 || 0.33 | 0.33 | +0%˄ || 3040.16 | 3031.53 | -0%˄ | (run time too short) |
| TPC-H 15 || 1.32 | 1.34 | +1%˄ || 756.22 | 745.95 | -1%˄ | (run time too short) |
| TPC-H 16 || 2.30 | 2.31 | +0%˄ || 434.11 | 433.39 | -0%˄ | (run time too short) |
+| TPC-H 17 || 0.76 | 0.63 | -18%˄ || 1306.16 | 1585.53 | +21%˄ | (run time too short) |
+| TPC-H 18 || 1.21 | 1.16 | -4%˄ || 824.78 | 863.38 | +5%˄ | (run time too short) |
| TPC-H 19 || 5.92 | 5.94 | +0%˄ || 168.85 | 168.36 | -0%˄ | (run time too short) |
| TPC-H 20 || 2.79 | 2.78 | -0%˄ || 358.74 | 359.44 | +0%˄ | (run time too short) |
| TPC-H 21 || 1.06 | 1.04 | -2%˄ || 937.86 | 955.74 | +2%˄ | (run time too short) |
| TPC-H 22 || 1.19 | 1.20 | +1%˄ || 839.58 | 834.28 | -1%˄ | (run time too short) |
+----------++----------+---------+--------++----------+----------+--------+----------------------+
| Sum || 63.29 | 63.76 | +1% || | | | |
| Geomean || | | || | | +1% | |
+----------++----------+---------+--------++----------+----------+--------+----------------------+
| Notes || ˄ Execution stopped due to max runs reached |
+----------++----------+---------+--------++----------+----------+--------+----------------------+hyriseBenchmarkTPCH - multi-threaded, ordered, 1 client, 28 cores, SF 10.0 Sum of avg. item runtimes: -6% || Geometric mean of throughput changes: +5%Configuration Overview - click to expand +Configuration Overview---------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | /home/Martin.Boissier/hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCH_43b228e1bc09ffbd5956577684cb99d0e060df1c_mt_ordered.json | /home/Martin.Boissier/hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCH_556e8941a1168263ed4aedb7bf6bb4aaf25e8e28_mt_ordered.json |
+-------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 43b228e1bc09ffbd5956577684cb99d0e060df1c-dirty | 556e8941a1168263ed4aedb7bf6bb4aaf25e8e28-dirty |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| clustering | None | None |
| compiler | clang 15.0.7 | clang 15.0.7 |
| cores | 28 | 28 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-05 23:25:12 | 2024-05-06 03:04:24 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 60000000000 | 60000000000 |
| max_runs | 50 | 50 |
| scale_factor | 10.0 | 10.0 |
| time_unit | ns | ns |
| use_prepared_statements | False | False |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [0, 0, 0, 28] | [0, 0, 0, 28] |
| verify | False | False |
| warmup_duration | 1000000000 | 1000000000 |
+-------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ +----------++----------+----------+--------++----------+----------+--------+----------------------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+----------++----------+----------+--------++----------+----------+--------+----------------------+
| TPC-H 01 || 4641.16 | 4598.50 | -1% || 0.22 | 0.22 | +1% | 0.1638 |
+| TPC-H 02 || 66.14 | 61.10 | -8%˄ || 15.08 | 16.32 | +8%˄ | (run time too short) |
| TPC-H 03 || 593.65 | 588.18 | -1%˄ || 1.68 | 1.70 | +1%˄ | (run time too short) |
| TPC-H 04 || 514.51 | 517.18 | +1%˄ || 1.94 | 1.93 | -1%˄ | (run time too short) |
| TPC-H 05 || 521.35 | 507.68 | -3%˄ || 1.92 | 1.97 | +3%˄ | (run time too short) |
| TPC-H 06 || 61.40 | 60.99 | -1%˄ || 16.24 | 16.35 | +1%˄ | (run time too short) |
| TPC-H 07 || 271.45 | 269.59 | -1%˄ || 3.68 | 3.71 | +1%˄ | (run time too short) |
| TPC-H 08 || 252.26 | 249.80 | -1%˄ || 3.96 | 4.00 | +1%˄ | (run time too short) |
| TPC-H 09 || 1681.85 | 1650.72 | -2% || 0.59 | 0.61 | +2% | 0.0146 |
+| TPC-H 10 || 996.83 | 538.83 | -46%˄ || 1.00 | 1.86 | +85%˄ | (run time too short) |
-| TPC-H 11 || 59.15 | 63.53 | +7%˄ || 16.86 | 15.71 | -7%˄ | (run time too short) |
| TPC-H 12 || 454.82 | 451.58 | -1%˄ || 2.20 | 2.21 | +1%˄ | (run time too short) |
| TPC-H 13 || 2294.99 | 2198.25 | -4% || 0.44 | 0.45 | +4% | 0.0000 |
| TPC-H 14 || 145.38 | 144.16 | -1%˄ || 6.87 | 6.93 | +1%˄ | (run time too short) |
| TPC-H 15 || 146.55 | 141.74 | -3%˄ || 6.82 | 7.05 | +3%˄ | (run time too short) |
| TPC-H 16 || 628.02 | 621.81 | -1%˄ || 1.59 | 1.61 | +1%˄ | (run time too short) |
| TPC-H 17 || 77.15 | 77.33 | +0%˄ || 12.94 | 12.90 | -0%˄ | (run time too short) |
+| TPC-H 18 || 2570.65 | 2168.44 | -16% || 0.39 | 0.46 | +19% | 0.0000 |
+| TPC-H 19 || 131.82 | 124.58 | -5%˄ || 7.58 | 8.02 | +6%˄ | (run time too short) |
| TPC-H 20 || 203.41 | 203.52 | +0%˄ || 4.91 | 4.91 | -0%˄ | (run time too short) |
| TPC-H 21 || 862.13 | 866.75 | +1%˄ || 1.16 | 1.15 | -1%˄ | (run time too short) |
| TPC-H 22 || 143.25 | 147.60 | +3%˄ || 6.97 | 6.77 | -3%˄ | (run time too short) |
+----------++----------+----------+--------++----------+----------+--------+----------------------+
+| Sum || 17317.92 | 16251.86 | -6% || | | | |
+| Geomean || | | || | | +5% | |
+----------++----------+----------+--------++----------+----------+--------+----------------------+
| Notes || ˄ Execution stopped due to max runs reached |
+----------++----------+----------+--------++----------+----------+--------+----------------------+hyriseBenchmarkTPCH - multi-threaded, shuffled, 28 clients, 28 cores, SF 10.0 Sum of avg. item runtimes: -1% || Geometric mean of throughput changes: +2%Configuration Overview - click to expand +Configuration Overview---------+-----------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------+
| Parameter | /home/Martin.Boissier/hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCH_43b228e1bc09ffbd5956577684cb99d0e060df1c_mt.json | /home/Martin.Boissier/hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCH_556e8941a1168263ed4aedb7bf6bb4aaf25e8e28_mt.json |
+-------------------------------+-----------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 43b228e1bc09ffbd5956577684cb99d0e060df1c-dirty | 556e8941a1168263ed4aedb7bf6bb4aaf25e8e28-dirty |
| benchmark_mode | Shuffled | Shuffled |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 28 | 28 |
| clustering | None | None |
| compiler | clang 15.0.7 | clang 15.0.7 |
| cores | 28 | 28 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-05 23:37:03 | 2024-05-06 03:15:50 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 1200000000000 | 1200000000000 |
| max_runs | -1 | -1 |
| scale_factor | 10.0 | 10.0 |
| time_unit | ns | ns |
| use_prepared_statements | False | False |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [0, 0, 0, 28] | [0, 0, 0, 28] |
| verify | False | False |
| warmup_duration | 0 | 0 |
+-------------------------------+-----------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------+ +----------++----------+----------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+----------++----------+----------+--------++----------+----------+--------+---------+
| TPC-H 01 || 6050.40 | 6013.76 | -1% || 0.58 | 0.58 | +2% | 0.6628 |
| TPC-H 02 || 348.73 | 357.92 | +3% || 0.58 | 0.59 | +2% | 0.8886 |
+| TPC-H 03 || 2329.40 | 2068.79 | -11% || 0.58 | 0.58 | +2% | 0.0704 |
| TPC-H 04 || 1920.64 | 1981.10 | +3% || 0.58 | 0.59 | +1% | 0.6729 |
| TPC-H 05 || 3164.51 | 3192.20 | +1% || 0.58 | 0.59 | +2% | 0.8936 |
+| TPC-H 06 || 651.69 | 614.51 | -6% || 0.58 | 0.59 | +2% | 0.6667 |
| TPC-H 07 || 2538.50 | 2612.20 | +3% || 0.58 | 0.58 | +1% | 0.6679 |
-| TPC-H 08 || 1660.99 | 1987.32 | +20% || 0.58 | 0.59 | +2% | 0.0272 |
| TPC-H 09 || 4629.78 | 4558.91 | -2% || 0.57 | 0.58 | +2% | 0.6865 |
+| TPC-H 10 || 3285.06 | 2485.86 | -24% || 0.58 | 0.58 | +2% | 0.0000 |
-| TPC-H 11 || 325.71 | 380.74 | +17% || 0.58 | 0.59 | +1% | 0.2826 |
+| TPC-H 12 || 2036.60 | 1925.33 | -5% || 0.58 | 0.59 | +1% | 0.3939 |
| TPC-H 13 || 4754.28 | 4894.79 | +3% || 0.58 | 0.58 | +1% | 0.3010 |
-| TPC-H 14 || 745.95 | 861.27 | +15% || 0.58 | 0.59 | +2% | 0.1479 |
| TPC-H 15 || 556.05 | 571.09 | +3% || 0.58 | 0.59 | +2% | 0.8124 |
| TPC-H 16 || 1672.99 | 1699.92 | +2% || 0.58 | 0.59 | +2% | 0.8083 |
| TPC-H 17 || 661.52 | 645.65 | -2% || 0.58 | 0.59 | +2% | 0.8509 |
+| TPC-H 18 || 3442.09 | 3249.03 | -6% || 0.58 | 0.58 | +2% | 0.0270 |
+| TPC-H 19 || 953.50 | 885.17 | -7% || 0.58 | 0.59 | +2% | 0.5098 |
| TPC-H 20 || 1512.70 | 1523.03 | +1% || 0.58 | 0.58 | +1% | 0.9369 |
| TPC-H 21 || 4091.52 | 4090.75 | -0% || 0.58 | 0.58 | +2% | 0.9969 |
| TPC-H 22 || 922.30 | 936.45 | +2% || 0.58 | 0.59 | +2% | 0.8981 |
+----------++----------+----------+--------++----------+----------+--------+---------+
| Sum || 48254.91 | 47535.79 | -1% || | | | |
| Geomean || | | || | | +2% | |
+----------++----------+----------+--------++----------+----------+--------+---------+hyriseBenchmarkTPCDS - single-threaded Sum of avg. item runtimes: +6% || Geometric mean of throughput changes: -1%Configuration Overview - click to expand +Configuration Overview---+------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | /home/Martin.Boissier/hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCDS_43b228e1bc09ffbd5956577684cb99d0e060df1c_st.json | /home/Martin.Boissier/hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCDS_556e8941a1168263ed4aedb7bf6bb4aaf25e8e28_st.json |
+-------------------------+------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 43b228e1bc09ffbd5956577684cb99d0e060df1c-dirty | 556e8941a1168263ed4aedb7bf6bb4aaf25e8e28-dirty |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| compiler | clang 15.0.7 | clang 15.0.7 |
| cores | 0 | 0 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-05 23:58:41 | 2024-05-06 03:37:30 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 60000000000 | 60000000000 |
| max_runs | 50 | 50 |
| time_unit | ns | ns |
| using_scheduler | False | False |
| verify | False | False |
| warmup_duration | 1000000000 | 1000000000 |
+-------------------------+------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ +---------++----------+----------+--------++----------+----------+--------+----------------------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+---------++----------+----------+--------++----------+----------+--------+----------------------+
| 01 || 235.54 | 245.37 | +4%˄ || 4.25 | 4.08 | -4%˄ | (run time too short) |
| 03 || 80.92 | 80.37 | -1%˄ || 12.36 | 12.44 | +1%˄ | (run time too short) |
| 06 || 198.51 | 198.06 | -0%˄ || 5.04 | 5.05 | +0%˄ | (run time too short) |
| 07 || 318.12 | 323.51 | +2%˄ || 3.14 | 3.09 | -2%˄ | (run time too short) |
| 09 || 683.46 | 681.75 | -0%˄ || 1.46 | 1.47 | +0%˄ | (run time too short) |
| 10 || 173.02 | 168.37 | -3%˄ || 5.78 | 5.94 | +3%˄ | (run time too short) |
| 13 || 531.99 | 516.72 | -3%˄ || 1.88 | 1.94 | +3%˄ | (run time too short) |
| 15 || 113.99 | 114.52 | +0%˄ || 8.77 | 8.73 | -0%˄ | (run time too short) |
| 16 || 82.76 | 82.17 | -1%˄ || 12.08 | 12.17 | +1%˄ | (run time too short) |
| 17 || 342.89 | 339.55 | -1%˄ || 2.92 | 2.94 | +1%˄ | (run time too short) |
| 19 || 120.23 | 120.09 | -0%˄ || 8.32 | 8.33 | +0%˄ | (run time too short) |
| 25 || 198.39 | 197.27 | -1%˄ || 5.04 | 5.07 | +1%˄ | (run time too short) |
| 26 || 146.14 | 143.65 | -2%˄ || 6.84 | 6.96 | +2%˄ | (run time too short) |
| 28 || 595.66 | 601.26 | +1%˄ || 1.68 | 1.66 | -1%˄ | (run time too short) |
| 29 || 492.14 | 502.30 | +2%˄ || 2.03 | 1.99 | -2%˄ | (run time too short) |
| 31 || 1350.73 | 1355.55 | +0% || 0.74 | 0.74 | -0% | 0.1508 |
| 32 || 41.39 | 42.07 | +2%˄ || 24.16 | 23.77 | -2%˄ | (run time too short) |
| 34 || 178.68 | 180.43 | +1%˄ || 5.60 | 5.54 | -1%˄ | (run time too short) |
| 35 || 639.57 | 639.00 | -0%˄ || 1.56 | 1.56 | +0%˄ | (run time too short) |
| 37 || 539.05 | 561.38 | +4%˄ || 1.86 | 1.78 | -4%˄ | (run time too short) |
| 39a || 1778.41 | 1728.31 | -3% || 0.56 | 0.58 | +3% | 0.0000 |
| 39b || 1782.31 | 1732.82 | -3% || 0.56 | 0.58 | +3% | 0.0000 |
-| 41 || 281.73 | 295.00 | +5%˄ || 3.55 | 3.39 | -4%˄ | (run time too short) |
| 42 || 101.87 | 103.55 | +2%˄ || 9.82 | 9.66 | -2%˄ | (run time too short) |
| 43 || 992.09 | 980.48 | -1%˄ || 1.01 | 1.02 | +1%˄ | (run time too short) |
| 45 || 120.23 | 120.30 | +0%˄ || 8.32 | 8.31 | -0%˄ | (run time too short) |
| 48 || 1043.64 | 1033.68 | -1%˄ || 0.96 | 0.97 | +1%˄ | (run time too short) |
| 50 || 136.86 | 136.12 | -1%˄ || 7.31 | 7.35 | +1%˄ | (run time too short) |
| 52 || 100.28 | 98.42 | -2%˄ || 9.97 | 10.16 | +2%˄ | (run time too short) |
| 55 || 96.78 | 95.52 | -1%˄ || 10.33 | 10.47 | +1%˄ | (run time too short) |
| 62 || 543.94 | 561.81 | +3%˄ || 1.84 | 1.78 | -3%˄ | (run time too short) |
-| 65 || 1651.14 | 1778.92 | +8% || 0.61 | 0.56 | -7% | 0.0000 |
| 69 || 174.91 | 171.91 | -2%˄ || 5.72 | 5.82 | +2%˄ | (run time too short) |
| 73 || 98.77 | 95.37 | -3%˄ || 10.12 | 10.48 | +4%˄ | (run time too short) |
| 79 || 489.85 | 488.46 | -0%˄ || 2.04 | 2.05 | +0%˄ | (run time too short) |
| 81 || 152.34 | 152.49 | +0%˄ || 6.56 | 6.56 | -0%˄ | (run time too short) |
| 82 || 628.15 | 619.75 | -1%˄ || 1.59 | 1.61 | +1%˄ | (run time too short) |
+| 83 || 37.70 | 35.89 | -5%˄ || 26.52 | 27.86 | +5%˄ | (run time too short) |
| 84 || 13.16 | 13.18 | +0%˄ || 75.99 | 75.89 | -0%˄ | (run time too short) |
| 85 || 157.67 | 157.80 | +0%˄ || 6.34 | 6.34 | -0%˄ | (run time too short) |
| 88 || 783.94 | 776.46 | -1%˄ || 1.28 | 1.29 | +1%˄ | (run time too short) |
| 91 || 14.80 | 14.42 | -3%˄ || 67.58 | 69.34 | +3%˄ | (run time too short) |
| 92 || 29.92 | 30.14 | +1%˄ || 33.42 | 33.18 | -1%˄ | (run time too short) |
| 93 || 3847.05 | 3790.97 | -1% || 0.26 | 0.26 | +1% | 0.0000 |
| 94 || 47.52 | 47.10 | -1%˄ || 21.04 | 21.23 | +1%˄ | (run time too short) |
| 95 || 4832.40 | 5043.25 | +4% || 0.21 | 0.20 | -4% | 0.1194 |
| 96 || 81.44 | 79.49 | -2%˄ || 12.28 | 12.58 | +2%˄ | (run time too short) |
-| 97 || 2759.20 | 4272.43 | +55% || 0.36 | 0.23 | -35% | 0.0000 |
| 99 || 1042.00 | 1067.61 | +2%˄ || 0.96 | 0.94 | -2%˄ | (run time too short) |
+---------++----------+----------+--------++----------+----------+--------+----------------------+
-| Sum || 30883.26 | 32615.05 | +6% || | | | |
| Geomean || | | || | | -1% | |
+---------++----------+----------+--------++----------+----------+--------+----------------------+
| Notes || ˄ Execution stopped due to max runs reached |
+---------++----------+----------+--------++----------+----------+--------+----------------------+hyriseBenchmarkTPCDS - multi-threaded, ordered, 1 client, 28 cores Sum of avg. item runtimes: -4% || Geometric mean of throughput changes: +0%Configuration Overview - click to expand +Configuration Overview---------+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | /home/Martin.Boissier/hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCDS_43b228e1bc09ffbd5956577684cb99d0e060df1c_mt_ordered.json | /home/Martin.Boissier/hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCDS_556e8941a1168263ed4aedb7bf6bb4aaf25e8e28_mt_ordered.json |
+-------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 43b228e1bc09ffbd5956577684cb99d0e060df1c-dirty | 556e8941a1168263ed4aedb7bf6bb4aaf25e8e28-dirty |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| compiler | clang 15.0.7 | clang 15.0.7 |
| cores | 28 | 28 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-06 00:18:01 | 2024-05-06 03:56:53 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 60000000000 | 60000000000 |
| max_runs | 50 | 50 |
| time_unit | ns | ns |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [0, 0, 0, 28] | [0, 0, 0, 28] |
| verify | False | False |
| warmup_duration | 1000000000 | 1000000000 |
+-------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+ +---------++----------+----------+--------++----------+----------+--------+----------------------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+---------++----------+----------+--------++----------+----------+--------+----------------------+
-| 01 || 210.37 | 228.83 | +9%˄ || 4.75 | 4.37 | -8%˄ | (run time too short) |
| 03 || 39.64 | 40.64 | +3%˄ || 25.13 | 24.50 | -3%˄ | (run time too short) |
| 06 || 194.94 | 194.38 | -0%˄ || 5.13 | 5.14 | +0%˄ | (run time too short) |
| 07 || 175.01 | 178.33 | +2%˄ || 5.71 | 5.60 | -2%˄ | (run time too short) |
| 09 || 100.57 | 101.06 | +0%˄ || 9.93 | 9.88 | -0%˄ | (run time too short) |
| 10 || 48.68 | 48.65 | -0%˄ || 20.50 | 20.51 | +0%˄ | (run time too short) |
| 13 || 258.25 | 257.97 | -0%˄ || 3.87 | 3.87 | +0%˄ | (run time too short) |
| 15 || 69.91 | 69.64 | -0%˄ || 14.27 | 14.32 | +0%˄ | (run time too short) |
| 16 || 44.34 | 44.88 | +1%˄ || 22.50 | 22.24 | -1%˄ | (run time too short) |
| 17 || 149.56 | 147.98 | -1%˄ || 6.68 | 6.75 | +1%˄ | (run time too short) |
| 19 || 71.26 | 71.36 | +0%˄ || 14.00 | 13.99 | -0%˄ | (run time too short) |
| 25 || 113.14 | 113.53 | +0%˄ || 8.83 | 8.80 | -0%˄ | (run time too short) |
| 26 || 93.27 | 94.30 | +1%˄ || 10.70 | 10.59 | -1%˄ | (run time too short) |
| 28 || 89.42 | 91.39 | +2%˄ || 11.17 | 10.93 | -2%˄ | (run time too short) |
| 29 || 400.67 | 398.43 | -1%˄ || 2.50 | 2.51 | +1%˄ | (run time too short) |
| 31 || 460.83 | 461.63 | +0%˄ || 2.17 | 2.17 | -0%˄ | (run time too short) |
| 32 || 38.50 | 37.23 | -3%˄ || 25.87 | 26.76 | +3%˄ | (run time too short) |
| 34 || 98.55 | 94.50 | -4%˄ || 10.13 | 10.57 | +4%˄ | (run time too short) |
| 35 || 375.20 | 370.40 | -1%˄ || 2.66 | 2.70 | +1%˄ | (run time too short) |
| 37 || 157.30 | 156.38 | -1%˄ || 6.35 | 6.39 | +1%˄ | (run time too short) |
+| 39a || 828.14 | 773.93 | -7%˄ || 1.21 | 1.29 | +7%˄ | (run time too short) |
+| 39b || 761.97 | 722.54 | -5%˄ || 1.31 | 1.38 | +5%˄ | (run time too short) |
-| 41 || 336.86 | 361.00 | +7%˄ || 2.97 | 2.77 | -7%˄ | (run time too short) |
| 42 || 57.25 | 57.08 | -0%˄ || 17.42 | 17.47 | +0%˄ | (run time too short) |
| 43 || 427.25 | 429.19 | +0%˄ || 2.34 | 2.33 | -0%˄ | (run time too short) |
| 45 || 56.34 | 56.48 | +0%˄ || 17.71 | 17.66 | -0%˄ | (run time too short) |
| 48 || 441.60 | 446.02 | +1%˄ || 2.26 | 2.24 | -1%˄ | (run time too short) |
| 50 || 105.66 | 105.37 | -0%˄ || 9.45 | 9.47 | +0%˄ | (run time too short) |
| 52 || 58.42 | 58.40 | -0%˄ || 17.07 | 17.08 | +0%˄ | (run time too short) |
| 55 || 53.11 | 54.27 | +2%˄ || 18.77 | 18.37 | -2%˄ | (run time too short) |
| 62 || 308.03 | 311.78 | +1%˄ || 3.24 | 3.21 | -1%˄ | (run time too short) |
+| 65 || 1722.86 | 1478.03 | -14% || 0.58 | 0.68 | +17% | 0.0000 |
| 69 || 77.38 | 77.24 | -0%˄ || 12.90 | 12.92 | +0%˄ | (run time too short) |
| 73 || 63.02 | 61.11 | -3%˄ || 15.84 | 16.33 | +3%˄ | (run time too short) |
| 79 || 435.71 | 430.70 | -1%˄ || 2.29 | 2.32 | +1%˄ | (run time too short) |
| 81 || 190.35 | 191.78 | +1%˄ || 5.25 | 5.21 | -1%˄ | (run time too short) |
| 82 || 188.07 | 191.03 | +2%˄ || 5.31 | 5.23 | -2%˄ | (run time too short) |
| 83 || 45.65 | 46.43 | +2%˄ || 21.83 | 21.47 | -2%˄ | (run time too short) |
| 84 || 16.10 | 16.65 | +3%˄ || 61.64 | 59.47 | -4%˄ | (run time too short) |
| 85 || 63.91 | 64.77 | +1%˄ || 15.61 | 15.40 | -1%˄ | (run time too short) |
| 88 || 79.54 | 82.91 | +4%˄ || 12.56 | 12.05 | -4%˄ | (run time too short) |
-| 91 || 19.44 | 20.49 | +5%˄ || 51.16 | 48.49 | -5%˄ | (run time too short) |
| 92 || 35.94 | 37.33 | +4%˄ || 27.73 | 26.68 | -4%˄ | (run time too short) |
| 93 || 631.43 | 623.35 | -1%˄ || 1.58 | 1.60 | +1%˄ | (run time too short) |
+| 94 || 43.18 | 41.32 | -4%˄ || 23.08 | 24.12 | +5%˄ | (run time too short) |
| 95 || 656.07 | 654.57 | -0%˄ || 1.52 | 1.53 | +0%˄ | (run time too short) |
| 96 || 54.52 | 53.25 | -2%˄ || 18.31 | 18.75 | +2%˄ | (run time too short) |
+| 97 || 1682.20 | 1502.70 | -11% || 0.59 | 0.67 | +12% | 0.0000 |
| 99 || 597.86 | 599.61 | +0%˄ || 1.67 | 1.67 | -0%˄ | (run time too short) |
+---------++----------+----------+--------++----------+----------+--------+----------------------+
| Sum || 13227.27 | 12750.85 | -4% || | | | |
| Geomean || | | || | | +0% | |
+---------++----------+----------+--------++----------+----------+--------+----------------------+
| Notes || ˄ Execution stopped due to max runs reached |
+---------++----------+----------+--------++----------+----------+--------+----------------------+hyriseBenchmarkTPCDS - multi-threaded, shuffled, 28 clients, 28 cores Sum of avg. item runtimes: +3% || Geometric mean of throughput changes: -3%Configuration Overview - click to expand +Configuration Overview---------+------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | /home/Martin.Boissier/hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCDS_43b228e1bc09ffbd5956577684cb99d0e060df1c_mt.json | /home/Martin.Boissier/hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCDS_556e8941a1168263ed4aedb7bf6bb4aaf25e8e28_mt.json |
+-------------------------------+------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 43b228e1bc09ffbd5956577684cb99d0e060df1c-dirty | 556e8941a1168263ed4aedb7bf6bb4aaf25e8e28-dirty |
| benchmark_mode | Shuffled | Shuffled |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 28 | 28 |
| compiler | clang 15.0.7 | clang 15.0.7 |
| cores | 28 | 28 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-06 00:29:57 | 2024-05-06 04:08:45 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 1200000000000 | 1200000000000 |
| max_runs | -1 | -1 |
| time_unit | ns | ns |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [0, 0, 0, 28] | [0, 0, 0, 28] |
| verify | False | False |
| warmup_duration | 0 | 0 |
+-------------------------------+------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ +---------++----------+----------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+---------++----------+----------+--------++----------+----------+--------+---------+
-| 01 || 584.81 | 648.45 | +11% || 0.58 | 0.57 | -3% | 0.2713 |
-| 03 || 225.94 | 296.51 | +31% || 0.59 | 0.57 | -3% | 0.0608 |
| 06 || 719.51 | 749.53 | +4% || 0.58 | 0.57 | -3% | 0.6372 |
-| 07 || 778.38 | 818.67 | +5% || 0.58 | 0.57 | -3% | 0.5185 |
| 09 || 645.69 | 634.89 | -2% || 0.59 | 0.57 | -3% | 0.8445 |
-| 10 || 590.07 | 663.18 | +12% || 0.58 | 0.57 | -3% | 0.2425 |
+| 13 || 1487.21 | 1344.15 | -10% || 0.58 | 0.57 | -3% | 0.0967 |
| 15 || 386.73 | 372.44 | -4% || 0.59 | 0.57 | -3% | 0.7347 |
+| 16 || 400.71 | 362.07 | -10% || 0.58 | 0.57 | -3% | 0.3637 |
-| 17 || 967.33 | 1058.45 | +9% || 0.58 | 0.57 | -3% | 0.2393 |
+| 19 || 608.50 | 532.14 | -13% || 0.58 | 0.57 | -3% | 0.1918 |
| 25 || 821.81 | 841.11 | +2% || 0.58 | 0.57 | -3% | 0.7746 |
| 26 || 426.18 | 407.00 | -4% || 0.58 | 0.57 | -3% | 0.6734 |
-| 28 || 1116.10 | 1197.30 | +7% || 0.58 | 0.57 | -3% | 0.3716 |
-| 29 || 1258.32 | 1363.73 | +8% || 0.58 | 0.57 | -3% | 0.2046 |
| 31 || 1998.58 | 2013.25 | +1% || 0.59 | 0.57 | -3% | 0.8886 |
+| 32 || 194.06 | 171.64 | -12% || 0.58 | 0.57 | -3% | 0.4719 |
+| 34 || 714.62 | 638.48 | -11% || 0.58 | 0.57 | -3% | 0.2217 |
| 35 || 1608.52 | 1591.21 | -1% || 0.59 | 0.57 | -3% | 0.8367 |
-| 37 || 691.13 | 768.33 | +11% || 0.58 | 0.57 | -3% | 0.1400 |
-| 39a || 2055.17 | 2187.26 | +6% || 0.58 | 0.57 | -3% | 0.1088 |
-| 39b || 2066.44 | 2175.64 | +5% || 0.58 | 0.57 | -3% | 0.1755 |
-| 41 || 1566.47 | 1895.97 | +21% || 0.58 | 0.57 | -3% | 0.0005 |
-| 42 || 350.64 | 405.44 | +16% || 0.59 | 0.57 | -3% | 0.1628 |
-| 43 || 1362.74 | 1459.53 | +7% || 0.58 | 0.57 | -3% | 0.1985 |
-| 45 || 586.50 | 621.30 | +6% || 0.58 | 0.57 | -3% | 0.5623 |
| 48 || 2011.96 | 2089.34 | +4% || 0.58 | 0.57 | -3% | 0.4649 |
+| 50 || 891.67 | 842.28 | -6% || 0.58 | 0.57 | -3% | 0.4887 |
+| 52 || 430.09 | 391.00 | -9% || 0.58 | 0.57 | -3% | 0.3795 |
-| 55 || 334.54 | 381.13 | +14% || 0.58 | 0.57 | -3% | 0.2423 |
-| 62 || 785.76 | 916.39 | +17% || 0.58 | 0.57 | -3% | 0.0059 |
| 65 || 3178.25 | 3093.83 | -3% || 0.58 | 0.57 | -3% | 0.0989 |
-| 69 || 714.92 | 898.08 | +26% || 0.58 | 0.57 | -3% | 0.0117 |
+| 73 || 482.27 | 459.42 | -5% || 0.58 | 0.57 | -3% | 0.6717 |
| 79 || 1228.11 | 1228.73 | +0% || 0.59 | 0.57 | -3% | 0.9921 |
-| 81 || 457.71 | 542.28 | +18% || 0.58 | 0.57 | -3% | 0.0610 |
| 82 || 848.17 | 850.48 | +0% || 0.58 | 0.57 | -3% | 0.9710 |
-| 83 || 255.71 | 305.87 | +20% || 0.59 | 0.57 | -3% | 0.1797 |
-| 84 || 96.28 | 120.01 | +25% || 0.59 | 0.57 | -3% | 0.2503 |
| 85 || 756.52 | 750.90 | -1% || 0.58 | 0.57 | -3% | 0.9438 |
| 88 || 1837.70 | 1877.88 | +2% || 0.58 | 0.57 | -3% | 0.7228 |
-| 91 || 180.07 | 190.37 | +6% || 0.58 | 0.57 | -3% | 0.7500 |
-| 92 || 238.62 | 251.33 | +5% || 0.58 | 0.57 | -3% | 0.7133 |
| 93 || 1665.36 | 1602.67 | -4% || 0.58 | 0.57 | -3% | 0.3806 |
-| 94 || 335.83 | 359.66 | +7% || 0.58 | 0.57 | -3% | 0.5945 |
| 95 || 1833.48 | 1807.25 | -1% || 0.58 | 0.57 | -3% | 0.7471 |
+| 96 || 421.38 | 303.49 | -28% || 0.59 | 0.57 | -3% | 0.0100 |
| 97 || 3370.84 | 3463.70 | +3% || 0.58 | 0.57 | -3% | 0.2895 |
| 99 || 1179.40 | 1197.26 | +2% || 0.58 | 0.57 | -3% | 0.7445 |
+---------++----------+----------+--------++----------+----------+--------+---------+
| Sum || 47746.79 | 49141.05 | +3% || | | | |
| Geomean || | | || | | -3% | |
+---------++----------+----------+--------++----------+----------+--------+---------+hyriseBenchmarkTPCC - single-threaded Sum of avg. item runtimes: -1% || Geometric mean of throughput changes: -0%Configuration Overview - click to expand +Configuration Overview---+-----------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------+
| Parameter | /home/Martin.Boissier/hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCC_43b228e1bc09ffbd5956577684cb99d0e060df1c_st.json | /home/Martin.Boissier/hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCC_556e8941a1168263ed4aedb7bf6bb4aaf25e8e28_st.json |
+-------------------------+-----------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 43b228e1bc09ffbd5956577684cb99d0e060df1c-dirty | 556e8941a1168263ed4aedb7bf6bb4aaf25e8e28-dirty |
| benchmark_mode | Shuffled | Shuffled |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| compiler | clang 15.0.7 | clang 15.0.7 |
| cores | 0 | 0 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-06 00:50:20 | 2024-05-06 04:29:07 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 60000000000 | 60000000000 |
| max_runs | -1 | -1 |
| scale_factor | 10 | 10 |
| time_unit | ns | ns |
| using_scheduler | False | False |
| verify | False | False |
| warmup_duration | 0 | 0 |
+-------------------------+-----------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------+ +--------------++----------+---------+--------++----------+----------+--------+----------------------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+--------------++----------+---------+--------++----------+----------+--------+----------------------+
+| Delivery || 57.42 | 57.11 | -1% || 1.78 | 2.10 | +18% | 0.2594 |
| New-Order || 42.14 | 41.34 | -2% || 17.91 | 18.13 | +1% | 0.1415 |
-| Order-Status || 1.90 | 1.90 | -0% || 1.70 | 1.50 | -12% | (run time too short) |
-| Payment || 7.27 | 7.25 | -0% || 18.35 | 16.78 | -9% | 0.6491 |
| Stock-Level || 3.46 | 3.40 | -2% || 1.53 | 1.58 | +3% | (run time too short) |
+--------------++----------+---------+--------++----------+----------+--------+----------------------+
| Sum || 112.19 | 110.99 | -1% || | | | |
| Geomean || | | || | | -0% | |
+--------------++----------+---------+--------++----------+----------+--------+----------------------+hyriseBenchmarkTPCC - multi-threaded, shuffled, 28 clients, 10 warehouses, 28 cores (high contention) Sum of avg. item runtimes: -2% || Geometric mean of throughput changes: +1%Configuration Overview - click to expand +Configuration Overview---------+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | /home/Martin.Boissier/hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCC_43b228e1bc09ffbd5956577684cb99d0e060df1c_mt_highcont.json | /home/Martin.Boissier/hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCC_556e8941a1168263ed4aedb7bf6bb4aaf25e8e28_mt_highcont.json |
+-------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 43b228e1bc09ffbd5956577684cb99d0e060df1c-dirty | 556e8941a1168263ed4aedb7bf6bb4aaf25e8e28-dirty |
| benchmark_mode | Shuffled | Shuffled |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 28 | 28 |
| compiler | clang 15.0.7 | clang 15.0.7 |
| cores | 28 | 28 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-06 00:51:27 | 2024-05-06 04:30:14 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 600000000000 | 600000000000 |
| max_runs | -1 | -1 |
| scale_factor | 10 | 10 |
| time_unit | ns | ns |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [0, 0, 0, 28] | [0, 0, 0, 28] |
| verify | False | False |
| warmup_duration | 0 | 0 |
+-------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+ +--------------++----------+---------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+--------------++----------+---------+--------++----------+----------+--------+---------+
| Delivery || 125.54 | 123.22 | -2% || 23.38 | 23.71 | +1% | 0.0002 |
| unsucc.: || 2.10 | 1.55 | -26% || 9.03 | 9.41 | +4% | |
| New-Order || 75.35 | 74.32 | -1% || 274.65 | 278.25 | +1% | 0.0000 |
| unsucc.: || 6.42 | 6.47 | +1% || 90.43 | 93.19 | +3% | |
| Order-Status || 6.19 | 6.07 | -2% || 32.45 | 33.01 | +2% | 0.3980 |
| Payment || 14.20 | 14.02 | -1% || 194.86 | 198.65 | +2% | 0.0015 |
| unsucc.: || 2.92 | 2.83 | -3% || 153.14 | 156.72 | +2% | |
| Stock-Level || 10.15 | 9.89 | -3% || 32.87 | 33.06 | +1% | 0.2191 |
+--------------++----------+---------+--------++----------+----------+--------+---------+
| Sum || 231.42 | 227.53 | -2% || | | | |
| Geomean || | | || | | +1% | |
+--------------++----------+---------+--------++----------+----------+--------+---------+hyriseBenchmarkTPCC - multi-threaded, shuffled, 10 clients, 10 warehouses, 28 cores (low contention) Sum of avg. item runtimes: -1% || Geometric mean of throughput changes: +1%Configuration Overview - click to expand +Configuration Overview---------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | /home/Martin.Boissier/hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCC_43b228e1bc09ffbd5956577684cb99d0e060df1c_mt_lowcont.json | /home/Martin.Boissier/hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCC_556e8941a1168263ed4aedb7bf6bb4aaf25e8e28_mt_lowcont.json |
+-------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 43b228e1bc09ffbd5956577684cb99d0e060df1c-dirty | 556e8941a1168263ed4aedb7bf6bb4aaf25e8e28-dirty |
| benchmark_mode | Shuffled | Shuffled |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 10 | 10 |
| compiler | clang 15.0.7 | clang 15.0.7 |
| cores | 28 | 28 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-06 01:01:36 | 2024-05-06 04:40:23 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 600000000000 | 600000000000 |
| max_runs | -1 | -1 |
| scale_factor | 10 | 10 |
| time_unit | ns | ns |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [0, 0, 0, 28] | [0, 0, 0, 28] |
| verify | False | False |
| warmup_duration | 0 | 0 |
+-------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ +--------------++----------+---------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+--------------++----------+---------+--------++----------+----------+--------+---------+
| Delivery || 91.47 | 90.85 | -1% || 9.62 | 9.58 | -0% | 0.0000 |
| unsucc.: || 0.98 | 0.88 | -10% || 0.86 | 0.89 | +3% | |
| New-Order || 65.99 | 64.47 | -2% || 107.65 | 109.68 | +2% | 0.0000 |
| unsucc.: || 3.97 | 4.46 | +12% || 10.68 | 10.86 | +2% | |
| Order-Status || 6.81 | 6.83 | +0% || 10.33 | 10.64 | +3% | 0.6690 |
| Payment || 19.81 | 19.77 | -0% || 87.55 | 89.59 | +2% | 0.2405 |
| unsucc.: || 2.29 | 2.27 | -1% || 25.10 | 25.52 | +2% | |
| Stock-Level || 9.64 | 9.43 | -2% || 10.57 | 10.48 | -1% | 0.0001 |
+--------------++----------+---------+--------++----------+----------+--------+---------+
| Sum || 193.72 | 191.35 | -1% || | | | |
| Geomean || | | || | | +1% | |
+--------------++----------+---------+--------++----------+----------+--------+---------+hyriseBenchmarkJoinOrder - single-threaded Sum of avg. item runtimes: -1% || Geometric mean of throughput changes: -0%Configuration Overview - click to expand +Configuration Overview---+----------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | /home/Martin.Boissier/hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkJoinOrder_43b228e1bc09ffbd5956577684cb99d0e060df1c_st.json | /home/Martin.Boissier/hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkJoinOrder_556e8941a1168263ed4aedb7bf6bb4aaf25e8e28_st.json |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 43b228e1bc09ffbd5956577684cb99d0e060df1c-dirty | 556e8941a1168263ed4aedb7bf6bb4aaf25e8e28-dirty |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| compiler | clang 15.0.7 | clang 15.0.7 |
| cores | 0 | 0 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-06 01:11:44 | 2024-05-06 04:50:31 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 60000000000 | 60000000000 |
| max_runs | 50 | 50 |
| time_unit | ns | ns |
| using_scheduler | False | False |
| verify | False | False |
| warmup_duration | 1000000000 | 1000000000 |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------+ +---------++----------+----------+--------++----------+----------+--------+----------------------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+---------++----------+----------+--------++----------+----------+--------+----------------------+
| 10a || 110.59 | 109.52 | -1%˄ || 9.04 | 9.13 | +1%˄ | (run time too short) |
+| 10b || 70.43 | 66.78 | -5%˄ || 14.20 | 14.97 | +5%˄ | (run time too short) |
| 10c || 336.68 | 340.38 | +1%˄ || 2.97 | 2.94 | -1%˄ | (run time too short) |
| 11a || 94.31 | 95.27 | +1%˄ || 10.60 | 10.50 | -1%˄ | (run time too short) |
| 11b || 90.58 | 90.70 | +0%˄ || 11.04 | 11.03 | -0%˄ | (run time too short) |
| 11c || 25.30 | 25.12 | -1%˄ || 39.52 | 39.81 | +1%˄ | (run time too short) |
| 11d || 27.36 | 27.17 | -1%˄ || 36.55 | 36.81 | +1%˄ | (run time too short) |
| 12a || 29.29 | 29.06 | -1%˄ || 34.13 | 34.41 | +1%˄ | (run time too short) |
| 12b || 40.23 | 39.98 | -1%˄ || 24.85 | 25.01 | +1%˄ | (run time too short) |
| 12c || 62.83 | 62.42 | -1%˄ || 15.91 | 16.02 | +1%˄ | (run time too short) |
| 13a || 192.83 | 193.05 | +0%˄ || 5.19 | 5.18 | -0%˄ | (run time too short) |
| 13b || 232.66 | 234.84 | +1%˄ || 4.30 | 4.26 | -1%˄ | (run time too short) |
| 13c || 196.65 | 199.26 | +1%˄ || 5.09 | 5.02 | -1%˄ | (run time too short) |
| 13d || 500.78 | 501.95 | +0%˄ || 2.00 | 1.99 | -0%˄ | (run time too short) |
| 14a || 98.89 | 99.56 | +1%˄ || 10.11 | 10.04 | -1%˄ | (run time too short) |
| 14b || 113.37 | 112.83 | -0%˄ || 8.82 | 8.86 | +0%˄ | (run time too short) |
| 14c || 208.98 | 207.59 | -1%˄ || 4.79 | 4.82 | +1%˄ | (run time too short) |
| 15a || 55.38 | 55.14 | -0%˄ || 18.06 | 18.13 | +0%˄ | (run time too short) |
| 15b || 55.72 | 55.66 | -0%˄ || 17.95 | 17.97 | +0%˄ | (run time too short) |
| 15c || 46.66 | 46.61 | -0%˄ || 21.43 | 21.45 | +0%˄ | (run time too short) |
| 15d || 44.07 | 44.05 | -0%˄ || 22.69 | 22.70 | +0%˄ | (run time too short) |
| 16a || 1335.71 | 1321.84 | -1% || 0.75 | 0.76 | +1% | 0.0375 |
| 16b || 2500.17 | 2403.70 | -4% || 0.40 | 0.42 | +4% | 0.0000 |
+| 16c || 1541.29 | 1465.99 | -5% || 0.65 | 0.68 | +5% | 0.0000 |
| 16d || 1474.35 | 1438.30 | -2% || 0.68 | 0.70 | +3% | 0.0000 |
| 17a || 456.28 | 458.51 | +0%˄ || 2.19 | 2.18 | -0%˄ | (run time too short) |
| 17b || 373.82 | 374.73 | +0%˄ || 2.68 | 2.67 | -0%˄ | (run time too short) |
| 17c || 357.59 | 357.83 | +0%˄ || 2.80 | 2.79 | -0%˄ | (run time too short) |
| 17d || 424.68 | 427.50 | +1%˄ || 2.35 | 2.34 | -1%˄ | (run time too short) |
| 17e || 1539.99 | 1502.31 | -2% || 0.65 | 0.67 | +3% | 0.0000 |
| 17f || 887.90 | 883.12 | -1%˄ || 1.13 | 1.13 | +1%˄ | (run time too short) |
| 18a || 141.86 | 144.53 | +2%˄ || 7.05 | 6.92 | -2%˄ | (run time too short) |
| 18b || 102.69 | 104.58 | +2%˄ || 9.74 | 9.56 | -2%˄ | (run time too short) |
| 18c || 202.66 | 203.00 | +0%˄ || 4.93 | 4.93 | -0%˄ | (run time too short) |
| 19a || 224.59 | 230.08 | +2%˄ || 4.45 | 4.35 | -2%˄ | (run time too short) |
| 19b || 187.65 | 190.74 | +2%˄ || 5.33 | 5.24 | -2%˄ | (run time too short) |
| 19c || 268.04 | 273.77 | +2%˄ || 3.73 | 3.65 | -2%˄ | (run time too short) |
| 19d || 654.14 | 655.05 | +0%˄ || 1.53 | 1.53 | -0%˄ | (run time too short) |
| 1a || 12.39 | 12.40 | +0%˄ || 80.69 | 80.62 | -0%˄ | (run time too short) |
| 1b || 14.11 | 14.08 | -0%˄ || 70.85 | 70.99 | +0%˄ | (run time too short) |
| 1c || 13.78 | 14.22 | +3%˄ || 72.56 | 70.33 | -3%˄ | (run time too short) |
| 1d || 11.85 | 11.86 | +0%˄ || 84.35 | 84.33 | -0%˄ | (run time too short) |
| 20a || 625.72 | 620.53 | -1%˄ || 1.60 | 1.61 | +1%˄ | (run time too short) |
| 20b || 667.99 | 655.06 | -2%˄ || 1.50 | 1.53 | +2%˄ | (run time too short) |
| 20c || 375.45 | 381.04 | +1%˄ || 2.66 | 2.62 | -1%˄ | (run time too short) |
| 21a || 109.98 | 108.97 | -1%˄ || 9.09 | 9.18 | +1%˄ | (run time too short) |
| 21b || 102.00 | 101.80 | -0%˄ || 9.80 | 9.82 | +0%˄ | (run time too short) |
| 21c || 113.08 | 112.52 | -0%˄ || 8.84 | 8.89 | +1%˄ | (run time too short) |
| 22a || 174.94 | 173.50 | -1%˄ || 5.72 | 5.76 | +1%˄ | (run time too short) |
| 22b || 159.99 | 158.06 | -1%˄ || 6.25 | 6.33 | +1%˄ | (run time too short) |
| 22c || 234.55 | 233.91 | -0%˄ || 4.26 | 4.28 | +0%˄ | (run time too short) |
| 22d || 355.42 | 356.67 | +0%˄ || 2.81 | 2.80 | -0%˄ | (run time too short) |
| 23a || 49.27 | 49.55 | +1%˄ || 20.30 | 20.18 | -1%˄ | (run time too short) |
| 23b || 53.75 | 53.88 | +0%˄ || 18.60 | 18.56 | -0%˄ | (run time too short) |
| 23c || 51.82 | 52.04 | +0%˄ || 19.30 | 19.21 | -0%˄ | (run time too short) |
| 24a || 218.61 | 225.62 | +3%˄ || 4.57 | 4.43 | -3%˄ | (run time too short) |
| 24b || 213.30 | 219.73 | +3%˄ || 4.69 | 4.55 | -3%˄ | (run time too short) |
| 25a || 112.20 | 114.73 | +2%˄ || 8.91 | 8.72 | -2%˄ | (run time too short) |
| 25b || 70.16 | 72.48 | +3%˄ || 14.25 | 13.80 | -3%˄ | (run time too short) |
| 25c || 254.93 | 254.93 | -0%˄ || 3.92 | 3.92 | +0%˄ | (run time too short) |
| 26a || 184.54 | 184.59 | +0%˄ || 5.42 | 5.42 | -0%˄ | (run time too short) |
| 26b || 153.09 | 154.18 | +1%˄ || 6.53 | 6.49 | -1%˄ | (run time too short) |
| 26c || 311.91 | 312.23 | +0%˄ || 3.21 | 3.20 | -0%˄ | (run time too short) |
| 27a || 104.66 | 104.73 | +0%˄ || 9.55 | 9.55 | -0%˄ | (run time too short) |
| 27b || 97.34 | 96.74 | -1%˄ || 10.27 | 10.34 | +1%˄ | (run time too short) |
| 27c || 113.14 | 112.51 | -1%˄ || 8.84 | 8.89 | +1%˄ | (run time too short) |
| 28a || 210.96 | 208.10 | -1%˄ || 4.74 | 4.81 | +1%˄ | (run time too short) |
| 28b || 65.86 | 65.83 | -0%˄ || 15.18 | 15.19 | +0%˄ | (run time too short) |
| 28c || 172.87 | 173.42 | +0%˄ || 5.78 | 5.77 | -0%˄ | (run time too short) |
| 29a || 183.82 | 189.58 | +3%˄ || 5.44 | 5.27 | -3%˄ | (run time too short) |
| 29b || 776.05 | 786.66 | +1%˄ || 1.29 | 1.27 | -1%˄ | (run time too short) |
| 29c || 220.86 | 227.16 | +3%˄ || 4.53 | 4.40 | -3%˄ | (run time too short) |
| 2a || 31.11 | 31.06 | -0%˄ || 32.14 | 32.19 | +0%˄ | (run time too short) |
| 2b || 29.22 | 29.16 | -0%˄ || 34.22 | 34.29 | +0%˄ | (run time too short) |
| 2c || 25.83 | 25.86 | +0%˄ || 38.72 | 38.66 | -0%˄ | (run time too short) |
| 2d || 53.17 | 52.99 | -0%˄ || 18.80 | 18.87 | +0%˄ | (run time too short) |
| 30a || 104.45 | 106.08 | +2%˄ || 9.57 | 9.43 | -2%˄ | (run time too short) |
| 30b || 110.01 | 111.37 | +1%˄ || 9.09 | 8.98 | -1%˄ | (run time too short) |
| 30c || 172.53 | 174.71 | +1%˄ || 5.80 | 5.72 | -1%˄ | (run time too short) |
| 31a || 91.16 | 93.08 | +2%˄ || 10.97 | 10.74 | -2%˄ | (run time too short) |
| 31b || 88.61 | 90.67 | +2%˄ || 11.28 | 11.03 | -2%˄ | (run time too short) |
| 31c || 104.06 | 105.41 | +1%˄ || 9.61 | 9.49 | -1%˄ | (run time too short) |
| 32a || 17.03 | 17.08 | +0%˄ || 58.72 | 58.54 | -0%˄ | (run time too short) |
| 32b || 33.22 | 33.39 | +1%˄ || 30.10 | 29.95 | -1%˄ | (run time too short) |
| 33a || 38.14 | 38.01 | -0%˄ || 26.22 | 26.31 | +0%˄ | (run time too short) |
| 33b || 26.15 | 26.13 | -0%˄ || 38.23 | 38.26 | +0%˄ | (run time too short) |
| 33c || 44.20 | 44.81 | +1%˄ || 22.62 | 22.31 | -1%˄ | (run time too short) |
| 3a || 61.31 | 61.36 | +0%˄ || 16.31 | 16.30 | -0%˄ | (run time too short) |
| 3b || 17.90 | 18.56 | +4%˄ || 55.86 | 53.89 | -4%˄ | (run time too short) |
| 3c || 226.85 | 224.67 | -1%˄ || 4.41 | 4.45 | +1%˄ | (run time too short) |
| 4a || 136.39 | 136.64 | +0%˄ || 7.33 | 7.32 | -0%˄ | (run time too short) |
| 4b || 17.10 | 17.83 | +4%˄ || 58.47 | 56.09 | -4%˄ | (run time too short) |
| 4c || 194.02 | 191.60 | -1%˄ || 5.15 | 5.22 | +1%˄ | (run time too short) |
| 5a || 42.93 | 43.00 | +0%˄ || 23.29 | 23.25 | -0%˄ | (run time too short) |
| 5b || 46.81 | 47.54 | +2%˄ || 21.36 | 21.03 | -2%˄ | (run time too short) |
| 5c || 88.95 | 88.24 | -1%˄ || 11.24 | 11.33 | +1%˄ | (run time too short) |
| 6a || 177.87 | 172.21 | -3%˄ || 5.62 | 5.81 | +3%˄ | (run time too short) |
| 6b || 189.44 | 183.14 | -3%˄ || 5.28 | 5.46 | +3%˄ | (run time too short) |
| 6c || 170.74 | 164.22 | -4%˄ || 5.86 | 6.09 | +4%˄ | (run time too short) |
| 6d || 394.59 | 389.04 | -1%˄ || 2.53 | 2.57 | +1%˄ | (run time too short) |
| 6e || 178.08 | 171.71 | -4%˄ || 5.62 | 5.82 | +4%˄ | (run time too short) |
| 6f || 697.24 | 690.20 | -1%˄ || 1.43 | 1.45 | +1%˄ | (run time too short) |
| 7a || 109.85 | 110.96 | +1%˄ || 9.10 | 9.01 | -1%˄ | (run time too short) |
| 7b || 103.27 | 104.46 | +1%˄ || 9.68 | 9.57 | -1%˄ | (run time too short) |
| 7c || 578.00 | 587.97 | +2%˄ || 1.73 | 1.70 | -2%˄ | (run time too short) |
| 8a || 169.15 | 172.09 | +2%˄ || 5.91 | 5.81 | -2%˄ | (run time too short) |
| 8b || 164.94 | 167.53 | +2%˄ || 6.06 | 5.97 | -2%˄ | (run time too short) |
| 8c || 1852.58 | 1858.06 | +0% || 0.54 | 0.54 | -0% | 0.2862 |
| 8d || 323.99 | 321.95 | -1%˄ || 3.09 | 3.11 | +1%˄ | (run time too short) |
| 9a || 244.34 | 251.10 | +3%˄ || 4.09 | 3.98 | -3%˄ | (run time too short) |
| 9b || 152.01 | 155.08 | +2%˄ || 6.58 | 6.45 | -2%˄ | (run time too short) |
| 9c || 272.04 | 278.09 | +2%˄ || 3.68 | 3.60 | -2%˄ | (run time too short) |
| 9d || 420.06 | 421.73 | +0%˄ || 2.38 | 2.37 | -0%˄ | (run time too short) |
+---------++----------+----------+--------++----------+----------+--------+----------------------+
| Sum || 29896.75 | 29694.57 | -1% || | | | |
| Geomean || | | || | | -0% | |
+---------++----------+----------+--------++----------+----------+--------+----------------------+
| Notes || ˄ Execution stopped due to max runs reached |
+---------++----------+----------+--------++----------+----------+--------+----------------------+hyriseBenchmarkJoinOrder - multi-threaded, ordered, 1 client, 28 cores Sum of avg. item runtimes: -0% || Geometric mean of throughput changes: +0%Configuration Overview - click to expand +Configuration Overview---------+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | /home/Martin.Boissier/hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkJoinOrder_43b228e1bc09ffbd5956577684cb99d0e060df1c_mt_ordered.json | /home/Martin.Boissier/hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkJoinOrder_556e8941a1168263ed4aedb7bf6bb4aaf25e8e28_mt_ordered.json |
+-------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 43b228e1bc09ffbd5956577684cb99d0e060df1c-dirty | 556e8941a1168263ed4aedb7bf6bb4aaf25e8e28-dirty |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| compiler | clang 15.0.7 | clang 15.0.7 |
| cores | 28 | 28 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-06 01:36:47 | 2024-05-06 05:15:32 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 60000000000 | 60000000000 |
| max_runs | 50 | 50 |
| time_unit | ns | ns |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [0, 0, 0, 28] | [0, 0, 0, 28] |
| verify | False | False |
| warmup_duration | 1000000000 | 1000000000 |
+-------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+ +---------++----------+----------+--------++----------+----------+--------+----------------------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+---------++----------+----------+--------++----------+----------+--------+----------------------+
| 10a || 43.85 | 43.24 | -1%˄ || 22.74 | 23.06 | +1%˄ | (run time too short) |
| 10b || 24.39 | 25.06 | +3%˄ || 40.80 | 39.70 | -3%˄ | (run time too short) |
| 10c || 143.12 | 143.02 | -0%˄ || 6.98 | 6.98 | +0%˄ | (run time too short) |
| 11a || 28.02 | 28.16 | +0%˄ || 35.58 | 35.43 | -0%˄ | (run time too short) |
| 11b || 25.98 | 26.29 | +1%˄ || 38.34 | 37.91 | -1%˄ | (run time too short) |
| 11c || 30.03 | 30.03 | +0%˄ || 33.14 | 33.13 | -0%˄ | (run time too short) |
| 11d || 34.59 | 34.47 | -0%˄ || 28.77 | 28.87 | +0%˄ | (run time too short) |
+| 12a || 30.49 | 29.10 | -5%˄ || 32.62 | 34.21 | +5%˄ | (run time too short) |
| 12b || 20.16 | 20.22 | +0%˄ || 49.26 | 49.16 | -0%˄ | (run time too short) |
| 12c || 60.51 | 60.36 | -0%˄ || 16.48 | 16.52 | +0%˄ | (run time too short) |
| 13a || 144.86 | 144.77 | -0%˄ || 6.90 | 6.90 | +0%˄ | (run time too short) |
+| 13b || 109.64 | 103.95 | -5%˄ || 9.11 | 9.61 | +5%˄ | (run time too short) |
| 13c || 96.85 | 98.40 | +2%˄ || 10.31 | 10.15 | -2%˄ | (run time too short) |
| 13d || 388.69 | 385.64 | -1%˄ || 2.57 | 2.59 | +1%˄ | (run time too short) |
| 14a || 62.08 | 62.75 | +1%˄ || 16.08 | 15.90 | -1%˄ | (run time too short) |
| 14b || 75.12 | 73.68 | -2%˄ || 13.29 | 13.55 | +2%˄ | (run time too short) |
| 14c || 186.76 | 184.87 | -1%˄ || 5.35 | 5.40 | +1%˄ | (run time too short) |
| 15a || 33.60 | 33.75 | +0%˄ || 29.64 | 29.53 | -0%˄ | (run time too short) |
| 15b || 29.32 | 29.93 | +2%˄ || 33.93 | 33.25 | -2%˄ | (run time too short) |
| 15c || 31.17 | 30.52 | -2%˄ || 31.91 | 32.61 | +2%˄ | (run time too short) |
| 15d || 36.60 | 36.60 | +0%˄ || 27.20 | 27.20 | -0%˄ | (run time too short) |
| 16a || 740.14 | 743.48 | +0%˄ || 1.35 | 1.34 | -0%˄ | (run time too short) |
| 16b || 1800.92 | 1780.23 | -1% || 0.56 | 0.56 | +1% | 0.0163 |
| 16c || 897.51 | 896.87 | -0%˄ || 1.11 | 1.11 | +0%˄ | (run time too short) |
| 16d || 868.28 | 867.68 | -0%˄ || 1.15 | 1.15 | +0%˄ | (run time too short) |
| 17a || 207.60 | 210.60 | +1%˄ || 4.81 | 4.74 | -1%˄ | (run time too short) |
| 17b || 180.77 | 179.88 | -0%˄ || 5.53 | 5.55 | +0%˄ | (run time too short) |
| 17c || 151.44 | 152.57 | +1%˄ || 6.60 | 6.55 | -1%˄ | (run time too short) |
| 17d || 149.42 | 149.13 | -0%˄ || 6.68 | 6.70 | +0%˄ | (run time too short) |
| 17e || 742.77 | 741.39 | -0%˄ || 1.35 | 1.35 | +0%˄ | (run time too short) |
| 17f || 333.25 | 328.93 | -1%˄ || 3.00 | 3.04 | +1%˄ | (run time too short) |
| 18a || 61.52 | 61.49 | -0%˄ || 16.22 | 16.23 | +0%˄ | (run time too short) |
| 18b || 58.66 | 57.84 | -1%˄ || 17.01 | 17.25 | +1%˄ | (run time too short) |
| 18c || 94.73 | 94.24 | -1%˄ || 10.54 | 10.59 | +1%˄ | (run time too short) |
| 19a || 115.40 | 117.29 | +2%˄ || 8.66 | 8.52 | -2%˄ | (run time too short) |
| 19b || 77.83 | 78.59 | +1%˄ || 12.83 | 12.71 | -1%˄ | (run time too short) |
| 19c || 129.37 | 129.96 | +0%˄ || 7.72 | 7.69 | -0%˄ | (run time too short) |
| 19d || 517.85 | 512.97 | -1%˄ || 1.93 | 1.95 | +1%˄ | (run time too short) |
| 1a || 11.16 | 11.07 | -1%˄ || 88.42 | 89.16 | +1%˄ | (run time too short) |
| 1b || 6.48 | 6.51 | +0%˄ || 151.21 | 150.60 | -0%˄ | (run time too short) |
| 1c || 6.38 | 6.38 | -0%˄ || 153.26 | 153.01 | -0%˄ | (run time too short) |
| 1d || 9.06 | 9.00 | -1%˄ || 108.59 | 109.49 | +1%˄ | (run time too short) |
| 20a || 87.17 | 87.10 | -0%˄ || 11.46 | 11.47 | +0%˄ | (run time too short) |
-| 20b || 64.66 | 68.50 | +6%˄ || 15.45 | 14.58 | -6%˄ | (run time too short) |
| 20c || 86.34 | 85.94 | -0%˄ || 11.56 | 11.61 | +0%˄ | (run time too short) |
| 21a || 32.37 | 32.04 | -1%˄ || 30.79 | 31.09 | +1%˄ | (run time too short) |
| 21b || 29.93 | 30.32 | +1%˄ || 33.29 | 32.87 | -1%˄ | (run time too short) |
| 21c || 34.03 | 34.15 | +0%˄ || 29.28 | 29.16 | -0%˄ | (run time too short) |
| 22a || 70.29 | 69.18 | -2%˄ || 14.20 | 14.42 | +2%˄ | (run time too short) |
| 22b || 58.93 | 59.18 | +0%˄ || 16.94 | 16.87 | -0%˄ | (run time too short) |
| 22c || 136.29 | 135.05 | -1%˄ || 7.33 | 7.40 | +1%˄ | (run time too short) |
| 22d || 241.62 | 241.27 | -0%˄ || 4.14 | 4.14 | +0%˄ | (run time too short) |
| 23a || 24.33 | 24.07 | -1%˄ || 40.89 | 41.34 | +1%˄ | (run time too short) |
-| 23b || 32.93 | 34.46 | +5%˄ || 30.27 | 28.92 | -4%˄ | (run time too short) |
| 23c || 26.23 | 26.78 | +2%˄ || 37.94 | 37.15 | -2%˄ | (run time too short) |
| 24a || 68.85 | 69.10 | +0%˄ || 14.50 | 14.45 | -0%˄ | (run time too short) |
| 24b || 62.74 | 62.42 | -1%˄ || 15.90 | 15.99 | +1%˄ | (run time too short) |
| 25a || 59.89 | 61.08 | +2%˄ || 16.65 | 16.32 | -2%˄ | (run time too short) |
| 25b || 40.13 | 38.74 | -3%˄ || 24.82 | 25.72 | +4%˄ | (run time too short) |
| 25c || 163.21 | 160.97 | -1%˄ || 6.12 | 6.21 | +1%˄ | (run time too short) |
| 26a || 92.04 | 92.02 | -0%˄ || 10.85 | 10.85 | +0%˄ | (run time too short) |
| 26b || 71.17 | 71.12 | -0%˄ || 14.02 | 14.03 | +0%˄ | (run time too short) |
| 26c || 151.02 | 150.16 | -1%˄ || 6.61 | 6.65 | +1%˄ | (run time too short) |
| 27a || 30.59 | 30.77 | +1%˄ || 32.56 | 32.38 | -1%˄ | (run time too short) |
| 27b || 30.74 | 30.66 | -0%˄ || 32.40 | 32.48 | +0%˄ | (run time too short) |
| 27c || 34.34 | 34.12 | -1%˄ || 29.02 | 29.21 | +1%˄ | (run time too short) |
| 28a || 151.88 | 149.75 | -1%˄ || 6.58 | 6.67 | +1%˄ | (run time too short) |
| 28b || 34.96 | 34.54 | -1%˄ || 28.50 | 28.85 | +1%˄ | (run time too short) |
| 28c || 119.72 | 119.65 | -0%˄ || 8.34 | 8.35 | +0%˄ | (run time too short) |
| 29a || 63.89 | 64.53 | +1%˄ || 15.63 | 15.47 | -1%˄ | (run time too short) |
| 29b || 110.99 | 111.31 | +0%˄ || 9.00 | 8.97 | -0%˄ | (run time too short) |
| 29c || 66.82 | 68.13 | +2%˄ || 14.93 | 14.64 | -2%˄ | (run time too short) |
| 2a || 32.78 | 33.04 | +1%˄ || 30.36 | 30.12 | -1%˄ | (run time too short) |
| 2b || 30.63 | 30.24 | -1%˄ || 32.47 | 32.89 | +1%˄ | (run time too short) |
| 2c || 23.13 | 23.13 | +0%˄ || 42.93 | 42.95 | +0%˄ | (run time too short) |
| 2d || 58.36 | 58.02 | -1%˄ || 17.09 | 17.18 | +1%˄ | (run time too short) |
| 30a || 53.62 | 54.43 | +2%˄ || 18.60 | 18.32 | -1%˄ | (run time too short) |
| 30b || 44.86 | 45.61 | +2%˄ || 22.23 | 21.86 | -2%˄ | (run time too short) |
| 30c || 114.23 | 113.51 | -1%˄ || 8.74 | 8.80 | +1%˄ | (run time too short) |
| 31a || 40.85 | 40.85 | -0%˄ || 24.40 | 24.38 | -0%˄ | (run time too short) |
| 31b || 38.90 | 38.64 | -1%˄ || 25.62 | 25.79 | +1%˄ | (run time too short) |
| 31c || 47.16 | 46.82 | -1%˄ || 21.14 | 21.29 | +1%˄ | (run time too short) |
| 32a || 11.61 | 11.59 | -0%˄ || 85.00 | 85.17 | +0%˄ | (run time too short) |
| 32b || 40.15 | 40.37 | +1%˄ || 24.80 | 24.67 | -1%˄ | (run time too short) |
| 33a || 15.94 | 16.35 | +3%˄ || 62.16 | 60.64 | -2%˄ | (run time too short) |
| 33b || 13.89 | 13.73 | -1%˄ || 71.22 | 72.09 | +1%˄ | (run time too short) |
| 33c || 23.60 | 24.54 | +4%˄ || 42.14 | 40.49 | -4%˄ | (run time too short) |
| 3a || 42.76 | 44.18 | +3%˄ || 23.31 | 22.56 | -3%˄ | (run time too short) |
| 3b || 10.85 | 10.81 | -0%˄ || 91.07 | 91.48 | +0%˄ | (run time too short) |
| 3c || 89.69 | 87.50 | -2%˄ || 11.13 | 11.41 | +3%˄ | (run time too short) |
| 4a || 78.47 | 75.98 | -3%˄ || 12.72 | 13.13 | +3%˄ | (run time too short) |
| 4b || 9.32 | 9.03 | -3%˄ || 105.68 | 108.96 | +3%˄ | (run time too short) |
+| 4c || 99.29 | 94.06 | -5%˄ || 10.05 | 10.61 | +6%˄ | (run time too short) |
| 5a || 33.28 | 33.28 | -0%˄ || 29.94 | 29.94 | +0%˄ | (run time too short) |
| 5b || 27.61 | 27.74 | +0%˄ || 36.05 | 35.88 | -0%˄ | (run time too short) |
| 5c || 55.72 | 53.54 | -4%˄ || 17.90 | 18.64 | +4%˄ | (run time too short) |
| 6a || 33.30 | 33.33 | +0%˄ || 29.95 | 29.92 | -0%˄ | (run time too short) |
| 6b || 57.14 | 57.86 | +1%˄ || 17.47 | 17.26 | -1%˄ | (run time too short) |
| 6c || 31.15 | 31.21 | +0%˄ || 32.01 | 31.95 | -0%˄ | (run time too short) |
| 6d || 160.23 | 160.78 | +0%˄ || 6.23 | 6.21 | -0%˄ | (run time too short) |
| 6e || 32.99 | 33.16 | +1%˄ || 30.22 | 30.09 | -0%˄ | (run time too short) |
| 6f || 484.46 | 481.13 | -1%˄ || 2.06 | 2.08 | +1%˄ | (run time too short) |
| 7a || 33.77 | 33.87 | +0%˄ || 29.52 | 29.44 | -0%˄ | (run time too short) |
| 7b || 33.31 | 33.68 | +1%˄ || 29.90 | 29.60 | -1%˄ | (run time too short) |
| 7c || 318.17 | 316.30 | -1%˄ || 3.14 | 3.16 | +1%˄ | (run time too short) |
| 8a || 40.46 | 40.78 | +1%˄ || 24.63 | 24.44 | -1%˄ | (run time too short) |
| 8b || 37.56 | 37.31 | -1%˄ || 26.53 | 26.72 | +1%˄ | (run time too short) |
| 8c || 1188.01 | 1174.34 | -1%˄ || 0.84 | 0.85 | +1%˄ | (run time too short) |
| 8d || 247.47 | 246.91 | -0%˄ || 4.04 | 4.05 | +0%˄ | (run time too short) |
| 9a || 147.83 | 147.68 | -0%˄ || 6.76 | 6.77 | +0%˄ | (run time too short) |
| 9b || 85.30 | 86.54 | +1%˄ || 11.70 | 11.54 | -1%˄ | (run time too short) |
| 9c || 143.91 | 144.48 | +0%˄ || 6.94 | 6.91 | -0%˄ | (run time too short) |
| 9d || 342.65 | 341.29 | -0%˄ || 2.92 | 2.93 | +0%˄ | (run time too short) |
+---------++----------+----------+--------++----------+----------+--------+----------------------+
| Sum || 15528.88 | 15465.72 | -0% || | | | |
| Geomean || | | || | | +0% | |
+---------++----------+----------+--------++----------+----------+--------+----------------------+
| Notes || ˄ Execution stopped due to max runs reached |
+---------++----------+----------+--------++----------+----------+--------+----------------------+hyriseBenchmarkJoinOrder - multi-threaded, shuffled, 28 clients, 28 cores Sum of avg. item runtimes: -0% || Geometric mean of throughput changes: +0%Configuration Overview - click to expand +Configuration Overview---------+----------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | /home/Martin.Boissier/hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkJoinOrder_43b228e1bc09ffbd5956577684cb99d0e060df1c_mt.json | /home/Martin.Boissier/hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkJoinOrder_556e8941a1168263ed4aedb7bf6bb4aaf25e8e28_mt.json |
+-------------------------------+----------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 43b228e1bc09ffbd5956577684cb99d0e060df1c-dirty | 556e8941a1168263ed4aedb7bf6bb4aaf25e8e28-dirty |
| benchmark_mode | Shuffled | Shuffled |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 28 | 28 |
| compiler | clang 15.0.7 | clang 15.0.7 |
| cores | 28 | 28 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-06 01:52:15 | 2024-05-06 05:30:59 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 1200000000000 | 1200000000000 |
| max_runs | -1 | -1 |
| time_unit | ns | ns |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [0, 0, 0, 28] | [0, 0, 0, 28] |
| verify | False | False |
| warmup_duration | 0 | 0 |
+-------------------------------+----------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------+ +---------++----------+----------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+---------++----------+----------+--------++----------+----------+--------+---------+
+| 10a || 185.65 | 163.02 | -12% || 0.63 | 0.63 | +1% | 0.2646 |
| 10b || 120.09 | 120.07 | -0% || 0.63 | 0.63 | +0% | 0.9993 |
+| 10c || 479.92 | 455.75 | -5% || 0.63 | 0.63 | +0% | 0.4748 |
-| 11a || 97.93 | 104.67 | +7% || 0.63 | 0.63 | +0% | 0.6781 |
-| 11b || 91.62 | 100.22 | +9% || 0.63 | 0.63 | +0% | 0.5835 |
| 11c || 91.56 | 90.46 | -1% || 0.63 | 0.63 | +0% | 0.9343 |
-| 11d || 83.47 | 96.57 | +16% || 0.63 | 0.63 | +0% | 0.3042 |
| 12a || 154.23 | 160.56 | +4% || 0.63 | 0.63 | +0% | 0.7626 |
| 12b || 87.74 | 88.40 | +1% || 0.63 | 0.63 | +0% | 0.9696 |
-| 12c || 269.66 | 290.28 | +8% || 0.63 | 0.63 | +0% | 0.4283 |
| 13a || 389.61 | 399.23 | +2% || 0.63 | 0.63 | +1% | 0.7220 |
-| 13b || 282.95 | 364.24 | +29% || 0.63 | 0.63 | +0% | 0.0120 |
+| 13c || 296.80 | 245.90 | -17% || 0.63 | 0.63 | +0% | 0.0283 |
| 13d || 757.65 | 760.03 | +0% || 0.63 | 0.63 | +0% | 0.9467 |
| 14a || 369.74 | 361.64 | -2% || 0.63 | 0.63 | +1% | 0.8211 |
+| 14b || 381.45 | 358.00 | -6% || 0.63 | 0.63 | +0% | 0.4786 |
+| 14c || 498.20 | 457.83 | -8% || 0.63 | 0.63 | +0% | 0.2251 |
| 15a || 118.20 | 114.74 | -3% || 0.63 | 0.63 | +0% | 0.8532 |
-| 15b || 106.54 | 114.61 | +8% || 0.63 | 0.63 | +0% | 0.6175 |
+| 15c || 125.52 | 115.62 | -8% || 0.63 | 0.63 | +0% | 0.5369 |
-| 15d || 129.20 | 140.26 | +9% || 0.63 | 0.63 | +0% | 0.5198 |
| 16a || 1368.82 | 1347.48 | -2% || 0.63 | 0.63 | +1% | 0.6384 |
| 16b || 2606.60 | 2631.52 | +1% || 0.63 | 0.63 | +1% | 0.5921 |
| 16c || 1561.28 | 1540.71 | -1% || 0.63 | 0.63 | +0% | 0.6642 |
+| 16d || 1585.04 | 1484.96 | -6% || 0.63 | 0.63 | +0% | 0.0518 |
| 17a || 705.91 | 729.27 | +3% || 0.63 | 0.63 | +0% | 0.5429 |
+| 17b || 596.22 | 529.97 | -11% || 0.63 | 0.63 | +0% | 0.0618 |
+| 17c || 568.70 | 515.70 | -9% || 0.63 | 0.63 | +0% | 0.1469 |
| 17d || 547.52 | 551.17 | +1% || 0.63 | 0.63 | +0% | 0.9247 |
| 17e || 1383.87 | 1366.51 | -1% || 0.63 | 0.63 | +0% | 0.7096 |
| 17f || 966.66 | 1006.81 | +4% || 0.63 | 0.63 | +0% | 0.3987 |
-| 18a || 226.84 | 238.45 | +5% || 0.63 | 0.63 | +0% | 0.6037 |
-| 18b || 331.15 | 366.69 | +11% || 0.63 | 0.63 | +0% | 0.2749 |
-| 18c || 495.27 | 537.43 | +9% || 0.63 | 0.63 | +1% | 0.2666 |
| 19a || 533.33 | 510.62 | -4% || 0.63 | 0.63 | +0% | 0.5691 |
+| 19b || 336.53 | 319.27 | -5% || 0.63 | 0.63 | +0% | 0.5517 |
+| 19c || 577.65 | 550.24 | -5% || 0.63 | 0.63 | +0% | 0.5065 |
+| 19d || 1223.09 | 1160.15 | -5% || 0.63 | 0.63 | +0% | 0.1671 |
+| 1a || 43.14 | 39.28 | -9% || 0.63 | 0.63 | +0% | 0.7237 |
+| 1b || 42.56 | 34.07 | -20% || 0.63 | 0.63 | +0% | 0.1829 |
+| 1c || 51.60 | 42.56 | -18% || 0.63 | 0.63 | +0% | 0.3954 |
| 1d || 39.21 | 38.85 | -1% || 0.63 | 0.63 | +0% | 0.9645 |
-| 20a || 411.37 | 440.16 | +7% || 0.63 | 0.63 | +0% | 0.4051 |
+| 20b || 400.56 | 375.51 | -6% || 0.63 | 0.63 | +0% | 0.4950 |
+| 20c || 372.97 | 320.56 | -14% || 0.63 | 0.63 | +0% | 0.0701 |
| 21a || 172.27 | 174.39 | +1% || 0.63 | 0.63 | +0% | 0.9317 |
+| 21b || 160.29 | 129.95 | -19% || 0.63 | 0.63 | +0% | 0.1678 |
| 21c || 163.57 | 167.49 | +2% || 0.63 | 0.63 | +0% | 0.8709 |
+| 22a || 446.08 | 379.44 | -15% || 0.63 | 0.63 | +0% | 0.0750 |
-| 22b || 327.10 | 368.42 | +13% || 0.63 | 0.63 | +0% | 0.2410 |
+| 22c || 589.68 | 544.44 | -8% || 0.63 | 0.63 | +0% | 0.2907 |
| 22d || 710.08 | 733.46 | +3% || 0.63 | 0.63 | +0% | 0.6002 |
-| 23a || 127.21 | 133.01 | +5% || 0.63 | 0.63 | +0% | 0.7416 |
+| 23b || 134.37 | 118.55 | -12% || 0.63 | 0.63 | +0% | 0.4695 |
| 23c || 160.08 | 158.92 | -1% || 0.63 | 0.63 | +0% | 0.9570 |
-| 24a || 345.77 | 400.77 | +16% || 0.63 | 0.63 | +0% | 0.0848 |
| 24b || 271.87 | 274.67 | +1% || 0.63 | 0.63 | +1% | 0.9248 |
+| 25a || 366.67 | 328.68 | -10% || 0.63 | 0.63 | +0% | 0.2236 |
+| 25b || 205.40 | 180.88 | -12% || 0.63 | 0.63 | +0% | 0.3271 |
-| 25c || 689.03 | 753.49 | +9% || 0.63 | 0.63 | +0% | 0.1308 |
| 26a || 356.75 | 346.67 | -3% || 0.63 | 0.63 | +0% | 0.7472 |
| 26b || 231.05 | 220.76 | -4% || 0.63 | 0.63 | +0% | 0.6293 |
| 26c || 489.62 | 501.65 | +2% || 0.63 | 0.63 | +0% | 0.6902 |
| 27a || 172.19 | 171.76 | -0% || 0.63 | 0.63 | +0% | 0.9858 |
+| 27b || 161.59 | 151.18 | -6% || 0.63 | 0.63 | +0% | 0.6242 |
+| 27c || 187.66 | 172.61 | -8% || 0.63 | 0.63 | +0% | 0.5849 |
| 28a || 555.54 | 577.76 | +4% || 0.63 | 0.63 | +0% | 0.5562 |
+| 28b || 234.85 | 208.64 | -11% || 0.63 | 0.63 | +0% | 0.3237 |
| 28c || 481.20 | 502.18 | +4% || 0.63 | 0.63 | +0% | 0.5499 |
-| 29a || 274.30 | 298.25 | +9% || 0.63 | 0.63 | +1% | 0.4117 |
-| 29b || 336.03 | 365.97 | +9% || 0.63 | 0.63 | +1% | 0.3824 |
-| 29c || 326.57 | 397.40 | +22% || 0.63 | 0.63 | +0% | 0.0503 |
+| 2a || 126.69 | 89.70 | -29% || 0.63 | 0.63 | +0% | 0.0035 |
+| 2b || 125.23 | 117.56 | -6% || 0.63 | 0.63 | +0% | 0.6812 |
-| 2c || 81.25 | 101.68 | +25% || 0.63 | 0.63 | +0% | 0.1186 |
-| 2d || 187.19 | 205.64 | +10% || 0.63 | 0.63 | +0% | 0.3630 |
+| 30a || 322.58 | 262.34 | -19% || 0.63 | 0.63 | +1% | 0.0558 |
| 30b || 288.90 | 281.80 | -2% || 0.63 | 0.63 | +0% | 0.8104 |
-| 30c || 539.30 | 569.94 | +6% || 0.63 | 0.63 | +0% | 0.4398 |
| 31a || 267.30 | 257.68 | -4% || 0.63 | 0.63 | +0% | 0.7810 |
-| 31b || 213.11 | 259.59 | +22% || 0.63 | 0.63 | +1% | 0.1327 |
| 31c || 294.52 | 300.47 | +2% || 0.63 | 0.63 | +0% | 0.8572 |
-| 32a || 37.11 | 49.73 | +34% || 0.63 | 0.63 | +0% | 0.0851 |
| 32b || 137.75 | 134.05 | -3% || 0.63 | 0.63 | +1% | 0.8110 |
-| 33a || 128.91 | 135.97 | +5% || 0.63 | 0.63 | +0% | 0.7593 |
-| 33b || 74.03 | 87.78 | +19% || 0.63 | 0.63 | +0% | 0.3395 |
+| 33c || 171.80 | 133.24 | -22% || 0.63 | 0.63 | +0% | 0.1336 |
+| 3a || 248.67 | 237.06 | -5% || 0.63 | 0.63 | +0% | 0.6992 |
+| 3b || 87.44 | 67.90 | -22% || 0.63 | 0.63 | +0% | 0.2563 |
+| 3c || 392.10 | 362.47 | -8% || 0.63 | 0.63 | +0% | 0.4355 |
-| 4a || 227.02 | 250.25 | +10% || 0.63 | 0.63 | +0% | 0.3191 |
-| 4b || 55.60 | 65.85 | +18% || 0.63 | 0.63 | +0% | 0.4713 |
+| 4c || 290.22 | 275.06 | -5% || 0.63 | 0.63 | +0% | 0.6149 |
-| 5a || 170.93 | 186.97 | +9% || 0.63 | 0.63 | +0% | 0.4911 |
| 5b || 129.13 | 129.48 | +0% || 0.63 | 0.63 | +0% | 0.9840 |
+| 5c || 257.93 | 230.12 | -11% || 0.63 | 0.63 | +1% | 0.2966 |
-| 6a || 121.46 | 132.21 | +9% || 0.63 | 0.63 | +0% | 0.5459 |
| 6b || 168.38 | 161.90 | -4% || 0.63 | 0.63 | +0% | 0.7158 |
-| 6c || 96.09 | 127.83 | +33% || 0.63 | 0.63 | +0% | 0.0220 |
+| 6d || 507.12 | 479.35 | -5% || 0.63 | 0.63 | +0% | 0.4340 |
+| 6e || 136.32 | 128.92 | -5% || 0.63 | 0.63 | +0% | 0.6728 |
-| 6f || 956.37 | 1019.48 | +7% || 0.63 | 0.63 | +0% | 0.0753 |
+| 7a || 210.32 | 176.12 | -16% || 0.63 | 0.63 | +0% | 0.1892 |
-| 7b || 151.78 | 180.83 | +19% || 0.63 | 0.63 | +0% | 0.2097 |
| 7c || 933.14 | 973.09 | +4% || 0.63 | 0.63 | +0% | 0.3646 |
-| 8a || 193.15 | 224.86 | +16% || 0.63 | 0.63 | +0% | 0.2027 |
+| 8b || 167.04 | 138.72 | -17% || 0.63 | 0.63 | +0% | 0.2262 |
| 8c || 1839.36 | 1905.08 | +4% || 0.63 | 0.63 | +0% | 0.1435 |
+| 8d || 604.67 | 540.94 | -11% || 0.63 | 0.63 | +0% | 0.0687 |
| 9a || 572.68 | 598.05 | +4% || 0.63 | 0.63 | +0% | 0.5436 |
+| 9b || 463.05 | 421.96 | -9% || 0.63 | 0.63 | +0% | 0.2720 |
-| 9c || 559.36 | 628.08 | +12% || 0.63 | 0.63 | +0% | 0.0999 |
+| 9d || 983.99 | 924.57 | -6% || 0.63 | 0.63 | +0% | 0.1856 |
+---------++----------+----------+--------++----------+----------+--------+---------+
| Sum || 44289.45 | 44095.70 | -0% || | | | |
| Geomean || | | || | | +0% | |
+---------++----------+----------+--------++----------+----------+--------+---------+hyriseBenchmarkStarSchema - single-threaded Sum of avg. item runtimes: -1% || Geometric mean of throughput changes: +1%Configuration Overview - click to expand +Configuration Overview---+-----------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | /home/Martin.Boissier/hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkStarSchema_43b228e1bc09ffbd5956577684cb99d0e060df1c_st.json | /home/Martin.Boissier/hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkStarSchema_556e8941a1168263ed4aedb7bf6bb4aaf25e8e28_st.json |
+-------------------------+-----------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 43b228e1bc09ffbd5956577684cb99d0e060df1c-dirty | 556e8941a1168263ed4aedb7bf6bb4aaf25e8e28-dirty |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| compiler | clang 15.0.7 | clang 15.0.7 |
| cores | 0 | 0 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-06 02:12:42 | 2024-05-06 05:51:25 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 60000000000 | 60000000000 |
| max_runs | 50 | 50 |
| scale_factor | 10.0 | 10.0 |
| time_unit | ns | ns |
| using_scheduler | False | False |
| verify | False | False |
| warmup_duration | 1000000000 | 1000000000 |
+-------------------------+-----------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------+ +---------++----------+---------+--------++----------+----------+--------+----------------------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+---------++----------+---------+--------++----------+----------+--------+----------------------+
| 1.1 || 329.33 | 332.50 | +1%˄ || 3.04 | 3.01 | -1%˄ | (run time too short) |
| 1.2 || 156.11 | 159.02 | +2%˄ || 6.41 | 6.29 | -2%˄ | (run time too short) |
+| 1.3 || 157.34 | 145.97 | -7%˄ || 6.36 | 6.85 | +8%˄ | (run time too short) |
| 2.1 || 559.94 | 557.42 | -0%˄ || 1.79 | 1.79 | +0%˄ | (run time too short) |
| 2.2 || 298.40 | 295.01 | -1%˄ || 3.35 | 3.39 | +1%˄ | (run time too short) |
| 2.3 || 200.77 | 198.97 | -1%˄ || 4.98 | 5.03 | +1%˄ | (run time too short) |
| 3.1 || 3031.41 | 3005.88 | -1% || 0.33 | 0.33 | +1% | 0.5178 |
| 3.2 || 300.69 | 297.14 | -1%˄ || 3.33 | 3.37 | +1%˄ | (run time too short) |
| 3.3 || 153.06 | 149.53 | -2%˄ || 6.53 | 6.69 | +2%˄ | (run time too short) |
| 3.4 || 145.34 | 142.55 | -2%˄ || 6.88 | 7.01 | +2%˄ | (run time too short) |
| 4.1 || 2992.40 | 2972.08 | -1% || 0.33 | 0.34 | +1% | 0.3774 |
| 4.2 || 878.17 | 883.71 | +1%˄ || 1.14 | 1.13 | -1%˄ | (run time too short) |
| 4.3 || 274.01 | 267.48 | -2%˄ || 3.65 | 3.74 | +2%˄ | (run time too short) |
+---------++----------+---------+--------++----------+----------+--------+----------------------+
| Sum || 9476.96 | 9407.28 | -1% || | | | |
| Geomean || | | || | | +1% | |
+---------++----------+---------+--------++----------+----------+--------+----------------------+
| Notes || ˄ Execution stopped due to max runs reached |
+---------++----------+---------+--------++----------+----------+--------+----------------------+hyriseBenchmarkStarSchema - multi-threaded, ordered, 1 client, 28 cores Sum of avg. item runtimes: -0% || Geometric mean of throughput changes: +0%Configuration Overview - click to expand +Configuration Overview---------+-------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | /home/Martin.Boissier/hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkStarSchema_43b228e1bc09ffbd5956577684cb99d0e060df1c_mt_ordered.json | /home/Martin.Boissier/hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkStarSchema_556e8941a1168263ed4aedb7bf6bb4aaf25e8e28_mt_ordered.json |
+-------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 43b228e1bc09ffbd5956577684cb99d0e060df1c-dirty | 556e8941a1168263ed4aedb7bf6bb4aaf25e8e28-dirty |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| compiler | clang 15.0.7 | clang 15.0.7 |
| cores | 28 | 28 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-06 02:18:18 | 2024-05-06 05:56:59 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 60000000000 | 60000000000 |
| max_runs | 50 | 50 |
| scale_factor | 10.0 | 10.0 |
| time_unit | ns | ns |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [0, 0, 0, 28] | [0, 0, 0, 28] |
| verify | False | False |
| warmup_duration | 1000000000 | 1000000000 |
+-------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------+ +---------++----------+---------+--------++----------+----------+--------+----------------------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+---------++----------+---------+--------++----------+----------+--------+----------------------+
| 1.1 || 102.47 | 104.37 | +2%˄ || 9.75 | 9.57 | -2%˄ | (run time too short) |
| 1.2 || 88.55 | 90.26 | +2%˄ || 11.27 | 11.06 | -2%˄ | (run time too short) |
| 1.3 || 84.95 | 84.30 | -1%˄ || 11.75 | 11.84 | +1%˄ | (run time too short) |
| 2.1 || 337.10 | 341.55 | +1%˄ || 2.97 | 2.93 | -1%˄ | (run time too short) |
| 2.2 || 166.48 | 164.66 | -1%˄ || 6.00 | 6.07 | +1%˄ | (run time too short) |
| 2.3 || 60.34 | 59.19 | -2%˄ || 16.53 | 16.86 | +2%˄ | (run time too short) |
| 3.1 || 831.80 | 840.04 | +1%˄ || 1.20 | 1.19 | -1%˄ | (run time too short) |
| 3.2 || 158.18 | 157.50 | -0%˄ || 6.32 | 6.34 | +0%˄ | (run time too short) |
| 3.3 || 81.19 | 80.34 | -1%˄ || 12.29 | 12.42 | +1%˄ | (run time too short) |
| 3.4 || 76.16 | 75.36 | -1%˄ || 13.11 | 13.25 | +1%˄ | (run time too short) |
| 4.1 || 704.89 | 694.92 | -1%˄ || 1.42 | 1.44 | +1%˄ | (run time too short) |
| 4.2 || 389.69 | 390.18 | +0%˄ || 2.57 | 2.56 | -0%˄ | (run time too short) |
| 4.3 || 117.29 | 114.96 | -2%˄ || 8.51 | 8.69 | +2%˄ | (run time too short) |
+---------++----------+---------+--------++----------+----------+--------+----------------------+
| Sum || 3199.10 | 3197.62 | -0% || | | | |
| Geomean || | | || | | +0% | |
+---------++----------+---------+--------++----------+----------+--------+----------------------+
| Notes || ˄ Execution stopped due to max runs reached |
+---------++----------+---------+--------++----------+----------+--------+----------------------+hyriseBenchmarkStarSchema - multi-threaded, shuffled, 28 clients, 28 cores Sum of avg. item runtimes: -0% || Geometric mean of throughput changes: +0%Configuration Overview - click to expand +Configuration Overview---------+-----------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | /home/Martin.Boissier/hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkStarSchema_43b228e1bc09ffbd5956577684cb99d0e060df1c_mt.json | /home/Martin.Boissier/hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkStarSchema_556e8941a1168263ed4aedb7bf6bb4aaf25e8e28_mt.json |
+-------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 43b228e1bc09ffbd5956577684cb99d0e060df1c-dirty | 556e8941a1168263ed4aedb7bf6bb4aaf25e8e28-dirty |
| benchmark_mode | Shuffled | Shuffled |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 28 | 28 |
| compiler | clang 15.0.7 | clang 15.0.7 |
| cores | 28 | 28 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-06 02:21:41 | 2024-05-06 06:00:22 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 1200000000000 | 1200000000000 |
| max_runs | -1 | -1 |
| scale_factor | 10.0 | 10.0 |
| time_unit | ns | ns |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [0, 0, 0, 28] | [0, 0, 0, 28] |
| verify | False | False |
| warmup_duration | 0 | 0 |
+-------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------+ +---------++----------+----------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+---------++----------+----------+--------++----------+----------+--------+---------+
| 1.1 || 632.85 | 632.78 | -0% || 2.17 | 2.17 | +0% | 0.9976 |
-| 1.2 || 357.54 | 396.27 | +11% || 2.17 | 2.17 | +0% | 0.0297 |
| 1.3 || 341.91 | 348.70 | +2% || 2.17 | 2.17 | +0% | 0.6732 |
| 2.1 || 1611.31 | 1576.18 | -2% || 2.16 | 2.17 | +0% | 0.3418 |
| 2.2 || 839.49 | 852.46 | +2% || 2.16 | 2.17 | +0% | 0.6349 |
| 2.3 || 296.67 | 291.10 | -2% || 2.17 | 2.17 | +0% | 0.6995 |
| 3.1 || 2531.29 | 2544.61 | +1% || 2.16 | 2.17 | +0% | 0.7268 |
| 3.2 || 765.15 | 764.89 | -0% || 2.16 | 2.17 | +0% | 0.9912 |
| 3.3 || 391.00 | 407.56 | +4% || 2.17 | 2.17 | +0% | 0.3771 |
+| 3.4 || 428.03 | 401.59 | -6% || 2.17 | 2.17 | +0% | 0.1771 |
| 4.1 || 2196.09 | 2187.26 | -0% || 2.16 | 2.17 | +0% | 0.8125 |
| 4.2 || 1939.11 | 1857.37 | -4% || 2.16 | 2.17 | +0% | 0.0431 |
| 4.3 || 577.74 | 600.66 | +4% || 2.17 | 2.17 | +0% | 0.2694 |
+---------++----------+----------+--------++----------+----------+--------+---------+
| Sum || 12908.18 | 12861.42 | -0% || | | | |
| Geomean || | | || | | +0% | |
+---------++----------+----------+--------++----------+----------+--------+---------+ |
|
@Bouncner looks fine System nemea - click to expand
Commit Info and Build Time
hyriseBenchmarkTPCH - single-threaded, SF 10.0 Sum of avg. item runtimes: -0% || Geometric mean of throughput changes: -1%Configuration Overview - click to expand +Configuration Overview----+----------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | /home/Daniel.Lindner/hyrise/clang15-release/benchmark_all_results/hyriseBenchmarkTPCH_f8f0db40e8efb4834f89e97e1cd086d8c0bf7f91_st.json | /home/Daniel.Lindner/hyrise/clang15-release/benchmark_all_results/hyriseBenchmarkTPCH_b3b0c86fc42c8c6f8631df85704d47162d01641d_st.json |
+--------------------------+----------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | f8f0db40e8efb4834f89e97e1cd086d8c0bf7f91-dirty | b3b0c86fc42c8c6f8631df85704d47162d01641d-dirty |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| clustering | None | None |
| compiler | clang 15.0.7 | clang 15.0.7 |
| cores | 0 | 0 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-22 13:56:07 | 2024-05-22 17:32:39 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 60000000000 | 60000000000 |
| max_runs | 50 | 50 |
| scale_factor | 10.0 | 10.0 |
| time_unit | ns | ns |
| use_prepared_statements | False | False |
| using_scheduler | False | False |
| verify | False | False |
| warmup_duration | 1000000000 | 1000000000 |
+--------------------------+----------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------+ +----------++----------+----------+--------++----------+----------+--------+----------------------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+----------++----------+----------+--------++----------+----------+--------+----------------------+
| TPC-H 01 || 6487.39 | 6486.41 | -0% || 0.15 | 0.15 | +0% | 0.9929 |
-| TPC-H 02 || 43.83 | 50.34 | +15%˄ || 22.81 | 19.87 | -13%˄ | (run time too short) |
| TPC-H 03 || 1389.95 | 1346.21 | -3% || 0.72 | 0.74 | +3% | 0.0245 |
| TPC-H 04 || 1420.44 | 1434.78 | +1% || 0.70 | 0.70 | -1% | 0.4264 |
| TPC-H 05 || 2207.30 | 2190.44 | -1% || 0.45 | 0.46 | +1% | 0.2842 |
+| TPC-H 06 || 166.74 | 153.62 | -8%˄ || 6.00 | 6.51 | +9%˄ | (run time too short) |
| TPC-H 07 || 813.77 | 835.66 | +3%˄ || 1.23 | 1.20 | -3%˄ | (run time too short) |
| TPC-H 08 || 503.00 | 522.91 | +4%˄ || 1.99 | 1.91 | -4%˄ | (run time too short) |
| TPC-H 09 || 5123.48 | 5165.29 | +1% || 0.20 | 0.19 | -1% | 0.1802 |
+| TPC-H 10 || 1700.15 | 1305.53 | -23% || 0.59 | 0.77 | +30% | 0.0000 |
| TPC-H 11 || 60.92 | 63.48 | +4%˄ || 16.41 | 15.75 | -4%˄ | (run time too short) |
| TPC-H 12 || 916.07 | 940.14 | +3%˄ || 1.09 | 1.06 | -3%˄ | (run time too short) |
| TPC-H 13 || 5184.25 | 5004.15 | -3% || 0.19 | 0.20 | +4% | 0.0053 |
| TPC-H 14 || 417.70 | 435.80 | +4%˄ || 2.39 | 2.29 | -4%˄ | (run time too short) |
+| TPC-H 15 || 198.72 | 188.33 | -5%˄ || 5.03 | 5.31 | +6%˄ | (run time too short) |
| TPC-H 16 || 564.78 | 586.89 | +4%˄ || 1.77 | 1.70 | -4%˄ | (run time too short) |
| TPC-H 17 || 219.17 | 223.36 | +2%˄ || 4.56 | 4.48 | -2%˄ | (run time too short) |
| TPC-H 18 || 1732.81 | 1795.67 | +4% || 0.58 | 0.56 | -4% | 0.1609 |
| TPC-H 19 || 216.25 | 224.33 | +4%˄ || 4.62 | 4.46 | -4%˄ | (run time too short) |
-| TPC-H 20 || 494.77 | 554.95 | +12%˄ || 2.02 | 1.80 | -11%˄ | (run time too short) |
-| TPC-H 21 || 3871.73 | 4061.83 | +5% || 0.26 | 0.25 | -5% | 0.0005 |
| TPC-H 22 || 425.57 | 429.98 | +1%˄ || 2.35 | 2.33 | -1%˄ | (run time too short) |
+----------++----------+----------+--------++----------+----------+--------+----------------------+
| Sum || 34158.79 | 34000.10 | -0% || | | | |
| Geomean || | | || | | -1% | |
+----------++----------+----------+--------++----------+----------+--------+----------------------+
| Notes || ˄ Execution stopped due to max runs reached |
+----------++----------+----------+--------++----------+----------+--------+----------------------+hyriseBenchmarkTPCH - single-threaded, SF 0.01 Sum of avg. item runtimes: +3% || Geometric mean of throughput changes: -5%Configuration Overview - click to expand +Configuration Overview----+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | /home/Daniel.Lindner/hyrise/clang15-release/benchmark_all_results/hyriseBenchmarkTPCH_f8f0db40e8efb4834f89e97e1cd086d8c0bf7f91_st_s01.json | /home/Daniel.Lindner/hyrise/clang15-release/benchmark_all_results/hyriseBenchmarkTPCH_b3b0c86fc42c8c6f8631df85704d47162d01641d_st_s01.json |
+--------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | f8f0db40e8efb4834f89e97e1cd086d8c0bf7f91-dirty | b3b0c86fc42c8c6f8631df85704d47162d01641d-dirty |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| clustering | None | None |
| compiler | clang 15.0.7 | clang 15.0.7 |
| cores | 0 | 0 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-22 14:11:53 | 2024-05-22 17:48:28 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 60000000000 | 60000000000 |
| max_runs | 50 | 50 |
| scale_factor | 0.009999999776482582 | 0.009999999776482582 |
| time_unit | ns | ns |
| use_prepared_statements | False | False |
| using_scheduler | False | False |
| verify | False | False |
| warmup_duration | 1000000000 | 1000000000 |
+--------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+ +----------++----------+---------+--------++----------+----------+--------+----------------------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+----------++----------+---------+--------++----------+----------+--------+----------------------+
| TPC-H 01 || 5.34 | 5.34 | +0%˄ || 187.37 | 187.24 | -0%˄ | (run time too short) |
-| TPC-H 02 || 4.29 | 4.69 | +9%˄ || 233.07 | 213.26 | -8%˄ | (run time too short) |
-| TPC-H 03 || 0.55 | 0.59 | +8%˄ || 1809.40 | 1682.19 | -7%˄ | (run time too short) |
-| TPC-H 04 || 0.39 | 0.45 | +14%˄ || 2534.43 | 2226.20 | -12%˄ | (run time too short) |
-| TPC-H 05 || 0.98 | 1.05 | +7%˄ || 1018.51 | 954.90 | -6%˄ | (run time too short) |
-| TPC-H 06 || 0.18 | 0.19 | +8%˄ || 5606.63 | 5191.29 | -7%˄ | (run time too short) |
| TPC-H 07 || 11.51 | 11.87 | +3%˄ || 86.87 | 84.25 | -3%˄ | (run time too short) |
| TPC-H 08 || 16.11 | 16.70 | +4%˄ || 62.05 | 59.89 | -3%˄ | (run time too short) |
+| TPC-H 09 || 3.92 | 3.36 | -14%˄ || 254.93 | 297.71 | +17%˄ | (run time too short) |
+| TPC-H 10 || 0.78 | 0.67 | -14%˄ || 1281.77 | 1483.50 | +16%˄ | (run time too short) |
-| TPC-H 11 || 0.19 | 0.22 | +16%˄ || 5213.46 | 4498.76 | -14%˄ | (run time too short) |
-| TPC-H 12 || 0.58 | 0.61 | +6%˄ || 1727.60 | 1629.31 | -6%˄ | (run time too short) |
| TPC-H 13 || 2.05 | 2.09 | +2%˄ || 488.03 | 478.65 | -2%˄ | (run time too short) |
-| TPC-H 14 || 0.32 | 0.35 | +10%˄ || 3084.26 | 2813.06 | -9%˄ | (run time too short) |
| TPC-H 15 || 1.31 | 1.37 | +4%˄ || 758.27 | 727.25 | -4%˄ | (run time too short) |
| TPC-H 16 || 2.30 | 2.34 | +2%˄ || 434.77 | 427.42 | -2%˄ | (run time too short) |
-| TPC-H 17 || 0.72 | 0.87 | +21%˄ || 1383.69 | 1147.86 | -17%˄ | (run time too short) |
| TPC-H 18 || 1.13 | 1.18 | +4%˄ || 884.45 | 848.74 | -4%˄ | (run time too short) |
| TPC-H 19 || 5.93 | 6.05 | +2%˄ || 168.46 | 165.19 | -2%˄ | (run time too short) |
| TPC-H 20 || 2.74 | 2.86 | +4%˄ || 365.05 | 349.49 | -4%˄ | (run time too short) |
-| TPC-H 21 || 1.01 | 1.19 | +17%˄ || 985.49 | 840.67 | -15%˄ | (run time too short) |
-| TPC-H 22 || 1.17 | 1.23 | +5%˄ || 850.86 | 813.68 | -4%˄ | (run time too short) |
+----------++----------+---------+--------++----------+----------+--------+----------------------+
| Sum || 63.51 | 65.25 | +3% || | | | |
-| Geomean || | | || | | -5% | |
+----------++----------+---------+--------++----------+----------+--------+----------------------+
| Notes || ˄ Execution stopped due to max runs reached |
+----------++----------+---------+--------++----------+----------+--------+----------------------+hyriseBenchmarkTPCH - multi-threaded, ordered, 1 client, 28 cores, SF 10.0 Sum of avg. item runtimes: -6% || Geometric mean of throughput changes: +8%Configuration Overview - click to expand +Configuration Overview---------+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | /home/Daniel.Lindner/hyrise/clang15-release/benchmark_all_results/hyriseBenchmarkTPCH_f8f0db40e8efb4834f89e97e1cd086d8c0bf7f91_mt_ordered.json | /home/Daniel.Lindner/hyrise/clang15-release/benchmark_all_results/hyriseBenchmarkTPCH_b3b0c86fc42c8c6f8631df85704d47162d01641d_mt_ordered.json |
+-------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | f8f0db40e8efb4834f89e97e1cd086d8c0bf7f91-dirty | b3b0c86fc42c8c6f8631df85704d47162d01641d-dirty |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| clustering | None | None |
| compiler | clang 15.0.7 | clang 15.0.7 |
| cores | 28 | 28 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-22 14:12:18 | 2024-05-22 17:48:54 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 60000000000 | 60000000000 |
| max_runs | 50 | 50 |
| scale_factor | 10.0 | 10.0 |
| time_unit | ns | ns |
| use_prepared_statements | False | False |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [0, 0, 28] | [0, 0, 28] |
| verify | False | False |
| warmup_duration | 1000000000 | 1000000000 |
+-------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+ +----------++----------+----------+--------++----------+----------+--------+----------------------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+----------++----------+----------+--------++----------+----------+--------+----------------------+
+| TPC-H 01 || 4677.52 | 4676.93 | -0% || 0.20 | 0.21 | +7% | (run time too short) |
+| TPC-H 02 || 63.49 | 62.05 | -2%˄ || 14.52 | 16.07 | +11%˄ | (run time too short) |
| TPC-H 03 || 606.73 | 589.45 | -3%˄ || 1.63 | 1.70 | +4%˄ | (run time too short) |
| TPC-H 04 || 517.21 | 506.23 | -2%˄ || 1.92 | 1.97 | +3%˄ | (run time too short) |
| TPC-H 05 || 516.19 | 513.89 | -0%˄ || 1.92 | 1.95 | +1%˄ | (run time too short) |
+| TPC-H 06 || 64.33 | 60.78 | -6%˄ || 14.41 | 16.41 | +14%˄ | (run time too short) |
+| TPC-H 07 || 273.37 | 265.88 | -3%˄ || 3.59 | 3.76 | +5%˄ | (run time too short) |
| TPC-H 08 || 251.27 | 251.54 | +0%˄ || 3.90 | 3.97 | +2%˄ | (run time too short) |
| TPC-H 09 || 1698.05 | 1651.62 | -3% || 0.58 | 0.61 | +4% | 0.0004 |
+| TPC-H 10 || 1007.54 | 541.83 | -46%˄ || 0.99 | 1.85 | +87%˄ | (run time too short) |
-| TPC-H 11 || 58.68 | 62.49 | +6%˄ || 15.57 | 15.97 | +3%˄ | (run time too short) |
| TPC-H 12 || 459.59 | 455.43 | -1%˄ || 2.15 | 2.19 | +2%˄ | (run time too short) |
+| TPC-H 13 || 2365.01 | 2222.22 | -6% || 0.42 | 0.45 | +8% | 0.0000 |
+| TPC-H 14 || 146.07 | 144.64 | -1%˄ || 6.60 | 6.91 | +5%˄ | (run time too short) |
+| TPC-H 15 || 147.75 | 141.24 | -4%˄ || 6.54 | 7.07 | +8%˄ | (run time too short) |
| TPC-H 16 || 629.19 | 627.52 | -0%˄ || 1.58 | 1.59 | +1%˄ | (run time too short) |
+| TPC-H 17 || 77.24 | 76.46 | -1%˄ || 12.09 | 13.05 | +8%˄ | (run time too short) |
+| TPC-H 18 || 2486.13 | 2170.62 | -13% || 0.40 | 0.46 | +15% | 0.0000 |
+| TPC-H 19 || 128.18 | 124.05 | -3%˄ || 7.53 | 8.05 | +7%˄ | (run time too short) |
-| TPC-H 20 || 199.20 | 209.25 | +5%˄ || 4.90 | 4.78 | -3%˄ | (run time too short) |
| TPC-H 21 || 851.20 | 874.55 | +3%˄ || 1.17 | 1.14 | -2%˄ | (run time too short) |
| TPC-H 22 || 141.58 | 143.94 | +2%˄ || 6.80 | 6.94 | +2%˄ | (run time too short) |
+----------++----------+----------+--------++----------+----------+--------+----------------------+
+| Sum || 17365.52 | 16372.61 | -6% || | | | |
+| Geomean || | | || | | +8% | |
+----------++----------+----------+--------++----------+----------+--------+----------------------+
| Notes || ˄ Execution stopped due to max runs reached |
+----------++----------+----------+--------++----------+----------+--------+----------------------+hyriseBenchmarkTPCH - multi-threaded, shuffled, 28 clients, 28 cores, SF 10.0 Sum of avg. item runtimes: -2% || Geometric mean of throughput changes: +2%Configuration Overview - click to expand +Configuration Overview---------+----------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | /home/Daniel.Lindner/hyrise/clang15-release/benchmark_all_results/hyriseBenchmarkTPCH_f8f0db40e8efb4834f89e97e1cd086d8c0bf7f91_mt.json | /home/Daniel.Lindner/hyrise/clang15-release/benchmark_all_results/hyriseBenchmarkTPCH_b3b0c86fc42c8c6f8631df85704d47162d01641d_mt.json |
+-------------------------------+----------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | f8f0db40e8efb4834f89e97e1cd086d8c0bf7f91-dirty | b3b0c86fc42c8c6f8631df85704d47162d01641d-dirty |
| benchmark_mode | Shuffled | Shuffled |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 28 | 28 |
| clustering | None | None |
| compiler | clang 15.0.7 | clang 15.0.7 |
| cores | 28 | 28 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-22 14:23:38 | 2024-05-22 18:00:15 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 1200000000000 | 1200000000000 |
| max_runs | -1 | -1 |
| scale_factor | 10.0 | 10.0 |
| time_unit | ns | ns |
| use_prepared_statements | False | False |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [0, 0, 28] | [0, 0, 28] |
| verify | False | False |
| warmup_duration | 0 | 0 |
+-------------------------------+----------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------+ +----------++----------+----------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+----------++----------+----------+--------++----------+----------+--------+---------+
+| TPC-H 01 || 6734.99 | 6386.85 | -5% || 0.54 | 0.55 | +2% | 0.0035 |
+| TPC-H 02 || 345.55 | 272.38 | -21% || 0.55 | 0.56 | +2% | 0.1662 |
-| TPC-H 03 || 2241.41 | 2411.12 | +8% || 0.55 | 0.56 | +2% | 0.2910 |
| TPC-H 04 || 2075.95 | 2030.58 | -2% || 0.55 | 0.56 | +2% | 0.7644 |
-| TPC-H 05 || 3124.89 | 3296.65 | +5% || 0.54 | 0.56 | +2% | 0.4088 |
+| TPC-H 06 || 651.96 | 620.49 | -5% || 0.55 | 0.56 | +2% | 0.7194 |
| TPC-H 07 || 2666.78 | 2680.86 | +1% || 0.54 | 0.56 | +2% | 0.9435 |
+| TPC-H 08 || 2018.93 | 1909.74 | -5% || 0.54 | 0.56 | +2% | 0.5568 |
+| TPC-H 09 || 5500.26 | 5109.36 | -7% || 0.55 | 0.56 | +2% | 0.0935 |
+| TPC-H 10 || 3246.71 | 2961.42 | -9% || 0.54 | 0.55 | +2% | 0.1483 |
-| TPC-H 11 || 351.79 | 471.33 | +34% || 0.55 | 0.56 | +2% | 0.1025 |
+| TPC-H 12 || 2269.80 | 2071.48 | -9% || 0.55 | 0.56 | +2% | 0.2468 |
| TPC-H 13 || 4920.44 | 4916.74 | -0% || 0.54 | 0.55 | +2% | 0.9764 |
+| TPC-H 14 || 1040.24 | 952.19 | -8% || 0.55 | 0.56 | +2% | 0.4801 |
+| TPC-H 15 || 624.37 | 543.10 | -13% || 0.55 | 0.56 | +2% | 0.3149 |
-| TPC-H 16 || 1746.84 | 1840.09 | +5% || 0.55 | 0.56 | +2% | 0.5071 |
| TPC-H 17 || 661.98 | 661.34 | -0% || 0.55 | 0.56 | +2% | 0.9944 |
| TPC-H 18 || 3705.77 | 3557.02 | -4% || 0.55 | 0.56 | +2% | 0.2346 |
| TPC-H 19 || 905.06 | 914.44 | +1% || 0.55 | 0.56 | +2% | 0.9316 |
| TPC-H 20 || 1376.18 | 1407.26 | +2% || 0.55 | 0.56 | +2% | 0.8101 |
| TPC-H 21 || 3901.37 | 4057.12 | +4% || 0.55 | 0.56 | +2% | 0.4561 |
-| TPC-H 22 || 768.53 | 902.35 | +17% || 0.55 | 0.56 | +2% | 0.1814 |
+----------++----------+----------+--------++----------+----------+--------+---------+
| Sum || 50879.78 | 49973.93 | -2% || | | | |
| Geomean || | | || | | +2% | |
+----------++----------+----------+--------++----------+----------+--------+---------+hyriseBenchmarkTPCDS - single-threaded Sum of avg. item runtimes: +7% || Geometric mean of throughput changes: -2%Configuration Overview - click to expand +Configuration Overview---+-----------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | /home/Daniel.Lindner/hyrise/clang15-release/benchmark_all_results/hyriseBenchmarkTPCDS_f8f0db40e8efb4834f89e97e1cd086d8c0bf7f91_st.json | /home/Daniel.Lindner/hyrise/clang15-release/benchmark_all_results/hyriseBenchmarkTPCDS_b3b0c86fc42c8c6f8631df85704d47162d01641d_st.json |
+-------------------------+-----------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | f8f0db40e8efb4834f89e97e1cd086d8c0bf7f91-dirty | b3b0c86fc42c8c6f8631df85704d47162d01641d-dirty |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| compiler | clang 15.0.7 | clang 15.0.7 |
| cores | 0 | 0 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-22 14:45:17 | 2024-05-22 18:21:55 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 60000000000 | 60000000000 |
| max_runs | 50 | 50 |
| time_unit | ns | ns |
| using_scheduler | False | False |
| verify | False | False |
| warmup_duration | 1000000000 | 1000000000 |
+-------------------------+-----------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------+ +----------++----------+----------+----------++----------+----------+----------+----------------------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+----------++----------+----------+----------++----------+----------+----------+----------------------+
-| 01 || 231.23 | 247.59 | +7%˄ || 4.32 | 4.04 | -7%˄ | (run time too short) |
-| 03 || 70.08 | 81.55 | +16%˄ || 14.27 | 12.26 | -14%˄ | (run time too short) |
-| 06 || 178.19 | 198.62 | +11%˄ || 5.61 | 5.03 | -10%˄ | (run time too short) |
-| 07 || 296.88 | 324.61 | +9%˄ || 3.37 | 3.08 | -9%˄ | (run time too short) |
| 09 || 694.77 | 704.05 | +1%˄ || 1.44 | 1.42 | -1%˄ | (run time too short) |
-| 10 || 139.25 | 175.26 | +26%˄ || 7.18 | 5.71 | -21%˄ | (run time too short) |
| 13 || 502.19 | 520.02 | +4%˄ || 1.99 | 1.92 | -3%˄ | (run time too short) |
-| 15 || 107.67 | 115.89 | +8%˄ || 9.29 | 8.63 | -7%˄ | (run time too short) |
-| 16 || 66.80 | 82.96 | +24%˄ || 14.97 | 12.05 | -19%˄ | (run time too short) |
-| 17 || 313.68 | 342.76 | +9%˄ || 3.19 | 2.92 | -8%˄ | (run time too short) |
-| 19 || 101.12 | 120.83 | +19%˄ || 9.89 | 8.28 | -16%˄ | (run time too short) |
-| 25 || 166.53 | 193.97 | +16%˄ || 6.00 | 5.16 | -14%˄ | (run time too short) |
| 26 || 136.12 | 140.24 | +3%˄ || 7.35 | 7.13 | -3%˄ | (run time too short) |
| 28 || 595.93 | 607.19 | +2%˄ || 1.68 | 1.65 | -2%˄ | (run time too short) |
| 29 || 482.47 | 498.29 | +3%˄ || 2.07 | 2.01 | -3%˄ | (run time too short) |
| 31 || 1318.13 | 1350.71 | +2% || 0.76 | 0.74 | -2% | 0.0000 |
-| 32 || 31.74 | 40.67 | +28%˄ || 31.51 | 24.59 | -22%˄ | (run time too short) |
-| 34 || 159.13 | 171.37 | +8%˄ || 6.28 | 5.84 | -7%˄ | (run time too short) |
| 35 || 612.45 | 633.23 | +3%˄ || 1.63 | 1.58 | -3%˄ | (run time too short) |
| 37 || 525.18 | 515.08 | -2%˄ || 1.90 | 1.94 | +2%˄ | (run time too short) |
-| 39a || 1561.12 | 1794.22 | +15% || 0.64 | 0.56 | -13% | 0.0000 |
-| 39b || 1531.66 | 1754.06 | +15% || 0.65 | 0.57 | -13% | 0.0000 |
| 41 || 286.15 | 294.14 | +3%˄ || 3.49 | 3.40 | -3%˄ | (run time too short) |
-| 42 || 81.50 | 100.90 | +24%˄ || 12.27 | 9.91 | -19%˄ | (run time too short) |
| 43 || 977.90 | 978.75 | +0%˄ || 1.02 | 1.02 | -0%˄ | (run time too short) |
-| 45 || 113.79 | 120.36 | +6%˄ || 8.79 | 8.31 | -5%˄ | (run time too short) |
| 48 || 1052.91 | 1058.37 | +1%˄ || 0.95 | 0.94 | -1%˄ | (run time too short) |
-| 50 || 115.53 | 135.85 | +18%˄ || 8.66 | 7.36 | -15%˄ | (run time too short) |
-| 52 || 79.86 | 98.93 | +24%˄ || 12.52 | 10.11 | -19%˄ | (run time too short) |
-| 55 || 76.88 | 96.52 | +26%˄ || 13.01 | 10.36 | -20%˄ | (run time too short) |
+| 62 || 562.25 | 536.89 | -5%˄ || 1.78 | 1.86 | +5%˄ | (run time too short) |
-| 65 || 1575.67 | 1748.92 | +11% || 0.63 | 0.57 | -10% | 0.0000 |
-| 69 || 139.04 | 172.84 | +24%˄ || 7.19 | 5.79 | -20%˄ | (run time too short) |
-| 73 || 76.11 | 95.79 | +26%˄ || 13.14 | 10.44 | -21%˄ | (run time too short) |
| 79 || 473.40 | 491.55 | +4%˄ || 2.11 | 2.03 | -4%˄ | (run time too short) |
| 81 || 151.56 | 152.73 | +1%˄ || 6.60 | 6.55 | -1%˄ | (run time too short) |
-| 82 || 554.83 | 633.49 | +14%˄ || 1.80 | 1.58 | -12%˄ | (run time too short) |
-| 83 || 35.10 | 38.33 | +9%˄ || 28.49 | 26.09 | -8%˄ | (run time too short) |
+| 85 -> 84 || 172.03 | 13.80 | -92%˄ || 5.81 | 72.47 | +1147%˄ | (run time too short) |
+| 88 -> 85 || 625.23 | 159.35 | -75%˄ || 1.60 | 6.28 | +292%˄ | (run time too short) |
-| 91 -> 88 || 13.59 | 777.26 | +5621%˄ || 73.59 | 1.29 | -98%˄ | (run time too short) |
+| 92 -> 91 || 25.33 | 15.06 | -41%˄ || 39.47 | 66.41 | +68%˄ | (run time too short) |
+| 93 -> 92 || 3814.35 | 30.57 | -99%˄ || 0.26 | 32.70 | +12374%˄ | (run time too short) |
-| 94 -> 93 || 37.41 | 3857.65 | +10213%˄ || 26.73 | 0.26 | -99%˄ | (run time too short) |
+| 95 -> 94 || 4939.90 | 47.06 | -99%˄ || 0.20 | 21.25 | +10397%˄ | (run time too short) |
-| 96 -> 95 || 59.30 | 5101.98 | +8503%˄ || 16.86 | 0.20 | -99%˄ | (run time too short) |
+| 97 -> 96 || 2640.85 | 78.33 | -97%˄ || 0.38 | 12.77 | +3271%˄ | (run time too short) |
-| 99 -> 97 || 1077.67 | 4289.91 | +298%˄ || 0.93 | 0.23 | -75%˄ | (run time too short) |
+----------++----------+----------+----------++----------+----------+----------+----------------------+
-| Sum || 29580.49 | 31738.46 | +7% || | | | |
| Geomean || | | || | | -2% | |
+----------++----------+----------+----------++----------+----------+----------+----------------------+
| Notes || ˄ Execution stopped due to max runs reached |
+----------++----------+----------+----------++----------+----------+----------+----------------------+hyriseBenchmarkTPCDS - multi-threaded, ordered, 1 client, 28 cores Sum of avg. item runtimes: -9% || Geometric mean of throughput changes: +16%Configuration Overview - click to expand +Configuration Overview---------+-------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | /home/Daniel.Lindner/hyrise/clang15-release/benchmark_all_results/hyriseBenchmarkTPCDS_f8f0db40e8efb4834f89e97e1cd086d8c0bf7f91_mt_ordered.json | /home/Daniel.Lindner/hyrise/clang15-release/benchmark_all_results/hyriseBenchmarkTPCDS_b3b0c86fc42c8c6f8631df85704d47162d01641d_mt_ordered.json |
+-------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | f8f0db40e8efb4834f89e97e1cd086d8c0bf7f91-dirty | b3b0c86fc42c8c6f8631df85704d47162d01641d-dirty |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| compiler | clang 15.0.7 | clang 15.0.7 |
| cores | 28 | 28 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-22 15:04:03 | 2024-05-22 18:41:14 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 60000000000 | 60000000000 |
| max_runs | 50 | 50 |
| time_unit | ns | ns |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [0, 0, 28] | [0, 0, 28] |
| verify | False | False |
| warmup_duration | 1000000000 | 1000000000 |
+-------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------+ +----------++----------+----------+---------++----------+----------+---------+----------------------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+----------++----------+----------+---------++----------+----------+---------+----------------------+
| 01 || 226.46 | 234.87 | +4%˄ || 4.33 | 4.26 | -2%˄ | (run time too short) |
-| 03 || 38.72 | 40.76 | +5%˄ || 23.77 | 24.43 | +3%˄ | (run time too short) |
| 06 || 195.99 | 194.63 | -1%˄ || 4.97 | 5.13 | +3%˄ | (run time too short) |
| 07 || 172.89 | 176.83 | +2%˄ || 5.62 | 5.65 | +1%˄ | (run time too short) |
+| 09 || 103.12 | 101.31 | -2%˄ || 9.18 | 9.86 | +7%˄ | (run time too short) |
+| 10 || 51.18 | 48.84 | -5%˄ || 17.13 | 20.42 | +19%˄ | (run time too short) |
| 13 || 257.08 | 257.54 | +0%˄ || 3.81 | 3.88 | +2%˄ | (run time too short) |
| 15 || 68.08 | 69.27 | +2%˄ || 13.88 | 14.40 | +4%˄ | (run time too short) |
+| 16 || 45.71 | 44.65 | -2%˄ || 19.79 | 22.35 | +13%˄ | (run time too short) |
| 17 || 146.04 | 147.87 | +1%˄ || 6.57 | 6.76 | +3%˄ | (run time too short) |
+| 19 || 72.01 | 70.02 | -3%˄ || 12.87 | 14.26 | +11%˄ | (run time too short) |
| 25 || 110.24 | 112.49 | +2%˄ || 8.75 | 8.88 | +2%˄ | (run time too short) |
+| 26 || 92.38 | 92.96 | +1%˄ || 10.12 | 10.74 | +6%˄ | (run time too short) |
| 28 || 93.48 | 94.70 | +1%˄ || 10.11 | 10.55 | +4%˄ | (run time too short) |
+| 29 || 411.73 | 398.39 | -3%˄ || 2.40 | 2.51 | +5%˄ | (run time too short) |
| 31 || 474.04 | 459.50 | -3%˄ || 2.09 | 2.18 | +4%˄ | (run time too short) |
+| 32 || 39.48 | 37.01 | -6%˄ || 22.76 | 26.92 | +18%˄ | (run time too short) |
+| 34 || 96.31 | 95.52 | -1%˄ || 9.83 | 10.45 | +6%˄ | (run time too short) |
| 35 || 377.29 | 366.18 | -3%˄ || 2.61 | 2.73 | +4%˄ | (run time too short) |
| 37 || 160.02 | 160.92 | +1%˄ || 6.07 | 6.21 | +2%˄ | (run time too short) |
| 39a || 772.79 | 771.62 | -0%˄ || 1.29 | 1.30 | +1%˄ | (run time too short) |
| 39b || 756.80 | 740.57 | -2%˄ || 1.31 | 1.35 | +3%˄ | (run time too short) |
+| 41 || 458.14 | 358.45 | -22%˄ || 2.16 | 2.79 | +29%˄ | (run time too short) |
+| 42 || 58.59 | 55.44 | -5%˄ || 16.16 | 17.99 | +11%˄ | (run time too short) |
| 43 || 430.26 | 429.09 | -0%˄ || 2.30 | 2.33 | +1%˄ | (run time too short) |
+| 45 || 56.43 | 57.15 | +1%˄ || 16.17 | 17.45 | +8%˄ | (run time too short) |
| 48 || 443.21 | 440.60 | -1%˄ || 2.23 | 2.27 | +2%˄ | (run time too short) |
+| 50 || 109.54 | 105.70 | -4%˄ || 8.70 | 9.45 | +9%˄ | (run time too short) |
+| 52 || 60.23 | 58.02 | -4%˄ || 15.60 | 17.19 | +10%˄ | (run time too short) |
+| 55 || 54.20 | 53.17 | -2%˄ || 16.47 | 18.75 | +14%˄ | (run time too short) |
+| 62 || 319.46 | 307.52 | -4%˄ || 3.08 | 3.25 | +6%˄ | (run time too short) |
+| 65 || 1737.25 | 1478.14 | -15% || 0.57 | 0.68 | +19% | 0.0000 |
+| 69 || 75.54 | 76.90 | +2%˄ || 12.24 | 12.98 | +6%˄ | (run time too short) |
+| 73 || 69.01 | 60.96 | -12%˄ || 13.58 | 16.37 | +21%˄ | (run time too short) |
| 79 || 442.14 | 433.86 | -2%˄ || 2.23 | 2.30 | +3%˄ | (run time too short) |
| 81 || 192.67 | 193.56 | +0%˄ || 5.07 | 5.16 | +2%˄ | (run time too short) |
| 82 || 191.73 | 191.46 | -0%˄ || 5.08 | 5.22 | +3%˄ | (run time too short) |
+| 83 || 50.62 | 47.87 | -5%˄ || 17.94 | 20.83 | +16%˄ | (run time too short) |
+| 85 -> 84 || 64.29 | 16.35 | -75%˄ || 14.50 | 60.63 | +318%˄ | (run time too short) |
+| 88 -> 85 || 80.47 | 63.42 | -21%˄ || 11.73 | 15.73 | +34%˄ | (run time too short) |
-| 91 -> 88 || 20.88 | 83.12 | +298%˄ || 39.14 | 12.02 | -69%˄ | (run time too short) |
+| 92 -> 91 || 36.78 | 20.54 | -44%˄ || 23.75 | 48.40 | +104%˄ | (run time too short) |
+| 93 -> 92 || 629.54 | 37.77 | -94%˄ || 1.58 | 26.38 | +1575%˄ | (run time too short) |
-| 94 -> 93 || 42.50 | 622.12 | +1364%˄ || 21.79 | 1.61 | -93%˄ | (run time too short) |
+| 95 -> 94 || 666.20 | 41.19 | -94%˄ || 1.49 | 24.21 | +1525%˄ | (run time too short) |
-| 96 -> 95 || 55.82 | 666.97 | +1095%˄ || 16.75 | 1.50 | -91%˄ | (run time too short) |
+| 97 -> 96 || 1673.57 | 54.60 | -97%˄ || 0.58 | 18.28 | +3034%˄ | (run time too short) |
-| 99 -> 97 || 626.36 | 1509.71 | +141%˄ || 1.58 | 0.66 | -58%˄ | (run time too short) |
+----------++----------+----------+---------++----------+----------+---------+----------------------+
+| Sum || 13407.28 | 12180.12 | -9% || | | | |
+| Geomean || | | || | | +16% | |
+----------++----------+----------+---------++----------+----------+---------+----------------------+
| Notes || ˄ Execution stopped due to max runs reached |
+----------++----------+----------+---------++----------+----------+---------+----------------------+hyriseBenchmarkTPCDS - multi-threaded, shuffled, 28 clients, 28 cores Sum of avg. item runtimes: +2% || Geometric mean of throughput changes: -4%Configuration Overview - click to expand +Configuration Overview---------+-----------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | /home/Daniel.Lindner/hyrise/clang15-release/benchmark_all_results/hyriseBenchmarkTPCDS_f8f0db40e8efb4834f89e97e1cd086d8c0bf7f91_mt.json | /home/Daniel.Lindner/hyrise/clang15-release/benchmark_all_results/hyriseBenchmarkTPCDS_b3b0c86fc42c8c6f8631df85704d47162d01641d_mt.json |
+-------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | f8f0db40e8efb4834f89e97e1cd086d8c0bf7f91-dirty | b3b0c86fc42c8c6f8631df85704d47162d01641d-dirty |
| benchmark_mode | Shuffled | Shuffled |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 28 | 28 |
| compiler | clang 15.0.7 | clang 15.0.7 |
| cores | 28 | 28 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-22 15:15:52 | 2024-05-22 18:53:06 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 1200000000000 | 1200000000000 |
| max_runs | -1 | -1 |
| time_unit | ns | ns |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [0, 0, 28] | [0, 0, 28] |
| verify | False | False |
| warmup_duration | 0 | 0 |
+-------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------+ +----------++----------+----------+---------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+----------++----------+----------+---------++----------+----------+--------+---------+
-| 01 || 566.11 | 696.15 | +23% || 0.57 | 0.54 | -4% | 0.0454 |
| 03 || 266.70 | 261.72 | -2% || 0.57 | 0.54 | -4% | 0.9040 |
+| 06 || 764.78 | 723.90 | -5% || 0.57 | 0.55 | -4% | 0.5612 |
-| 07 || 799.90 | 877.68 | +10% || 0.57 | 0.54 | -4% | 0.2166 |
| 09 || 733.16 | 741.66 | +1% || 0.57 | 0.54 | -4% | 0.9047 |
-| 10 || 545.14 | 621.87 | +14% || 0.57 | 0.54 | -4% | 0.2039 |
| 13 || 1349.02 | 1408.30 | +4% || 0.57 | 0.54 | -4% | 0.4976 |
+| 15 || 466.81 | 444.40 | -5% || 0.57 | 0.54 | -4% | 0.7215 |
+| 16 || 396.72 | 351.34 | -11% || 0.57 | 0.55 | -4% | 0.3418 |
-| 17 || 978.02 | 1072.52 | +10% || 0.57 | 0.55 | -4% | 0.2594 |
| 19 || 526.00 | 538.14 | +2% || 0.57 | 0.54 | -4% | 0.8222 |
-| 25 || 766.10 | 894.50 | +17% || 0.57 | 0.54 | -4% | 0.0720 |
-| 26 || 431.53 | 497.39 | +15% || 0.57 | 0.54 | -4% | 0.2426 |
-| 28 || 1102.23 | 1231.95 | +12% || 0.57 | 0.54 | -4% | 0.1816 |
-| 29 || 1202.66 | 1311.57 | +9% || 0.57 | 0.54 | -4% | 0.1827 |
| 31 || 2045.15 | 2056.93 | +1% || 0.57 | 0.55 | -4% | 0.9135 |
-| 32 || 154.02 | 201.69 | +31% || 0.57 | 0.55 | -4% | 0.1272 |
-| 34 || 632.31 | 687.29 | +9% || 0.57 | 0.55 | -4% | 0.3727 |
| 35 || 1596.90 | 1649.68 | +3% || 0.57 | 0.55 | -4% | 0.5579 |
| 37 || 716.66 | 735.22 | +3% || 0.57 | 0.55 | -4% | 0.7441 |
-| 39a || 2140.47 | 2256.81 | +5% || 0.57 | 0.54 | -4% | 0.1999 |
-| 39b || 2069.74 | 2238.12 | +8% || 0.57 | 0.55 | -4% | 0.1080 |
+| 41 || 2933.23 | 2024.90 | -31% || 0.57 | 0.54 | -4% | 0.0000 |
-| 42 || 402.09 | 442.35 | +10% || 0.57 | 0.55 | -4% | 0.4823 |
-| 43 || 1304.67 | 1445.94 | +11% || 0.57 | 0.54 | -4% | 0.0582 |
-| 45 || 576.63 | 679.78 | +18% || 0.57 | 0.55 | -4% | 0.1383 |
-| 48 || 1808.43 | 2133.57 | +18% || 0.57 | 0.54 | -4% | 0.0012 |
-| 50 || 824.87 | 936.42 | +14% || 0.57 | 0.54 | -4% | 0.1537 |
-| 52 || 397.30 | 427.28 | +8% || 0.57 | 0.55 | -4% | 0.4997 |
-| 55 || 310.01 | 408.86 | +32% || 0.57 | 0.54 | -4% | 0.0169 |
-| 62 || 831.61 | 899.59 | +8% || 0.57 | 0.55 | -4% | 0.1998 |
| 65 || 3332.22 | 3300.76 | -1% || 0.57 | 0.54 | -4% | 0.6472 |
| 69 || 855.74 | 893.40 | +4% || 0.57 | 0.55 | -4% | 0.6702 |
| 73 || 444.66 | 454.22 | +2% || 0.57 | 0.55 | -4% | 0.8686 |
-| 79 || 1143.40 | 1388.62 | +21% || 0.57 | 0.54 | -4% | 0.0002 |
-| 81 || 486.67 | 514.85 | +6% || 0.57 | 0.54 | -4% | 0.5980 |
| 82 || 885.16 | 910.72 | +3% || 0.57 | 0.54 | -4% | 0.6906 |
+| 83 || 330.27 | 259.02 | -22% || 0.57 | 0.55 | -4% | 0.1291 |
+| 85 -> 84 || 675.68 | 121.76 | -82% || 0.57 | 0.55 | -4% | 0.0000 |
+| 88 -> 85 || 1850.79 | 751.43 | -59% || 0.57 | 0.55 | -4% | 0.0000 |
-| 91 -> 88 || 142.99 | 1996.04 | +1296% || 0.57 | 0.54 | -4% | 0.0000 |
+| 92 -> 91 || 268.46 | 227.80 | -15% || 0.57 | 0.55 | -4% | 0.4499 |
+| 93 -> 92 || 1718.09 | 202.49 | -88% || 0.57 | 0.55 | -4% | 0.0000 |
-| 94 -> 93 || 337.25 | 1847.99 | +448% || 0.57 | 0.54 | -4% | 0.0000 |
+| 95 -> 94 || 1905.94 | 352.17 | -82% || 0.57 | 0.55 | -4% | 0.0000 |
-| 96 -> 95 || 347.28 | 1945.70 | +460% || 0.57 | 0.54 | -4% | 0.0000 |
+| 97 -> 96 || 3345.14 | 349.00 | -90% || 0.57 | 0.54 | -4% | 0.0000 |
-| 99 -> 97 || 1184.59 | 3621.08 | +206% || 0.57 | 0.54 | -4% | 0.0000 |
+----------++----------+----------+---------++----------+----------+--------+---------+
| Sum || 48893.32 | 50034.55 | +2% || | | | |
| Geomean || | | || | | -4% | |
+----------++----------+----------+---------++----------+----------+--------+---------+hyriseBenchmarkTPCC - single-threaded Sum of avg. item runtimes: +0% || Geometric mean of throughput changes: +0%Configuration Overview - click to expand +Configuration Overview---+----------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | /home/Daniel.Lindner/hyrise/clang15-release/benchmark_all_results/hyriseBenchmarkTPCC_f8f0db40e8efb4834f89e97e1cd086d8c0bf7f91_st.json | /home/Daniel.Lindner/hyrise/clang15-release/benchmark_all_results/hyriseBenchmarkTPCC_b3b0c86fc42c8c6f8631df85704d47162d01641d_st.json |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | f8f0db40e8efb4834f89e97e1cd086d8c0bf7f91-dirty | b3b0c86fc42c8c6f8631df85704d47162d01641d-dirty |
| benchmark_mode | Shuffled | Shuffled |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| compiler | clang 15.0.7 | clang 15.0.7 |
| cores | 0 | 0 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-22 15:36:15 | 2024-05-22 19:13:28 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 60000000000 | 60000000000 |
| max_runs | -1 | -1 |
| scale_factor | 10 | 10 |
| time_unit | ns | ns |
| using_scheduler | False | False |
| verify | False | False |
| warmup_duration | 0 | 0 |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------+ +--------------++----------+---------+--------++----------+----------+--------+----------------------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+--------------++----------+---------+--------++----------+----------+--------+----------------------+
-| Delivery || 57.61 | 57.59 | -0% || 1.65 | 1.50 | -9% | (run time too short) |
| New-Order || 41.91 | 41.79 | -0% || 18.35 | 18.60 | +1% | 0.8237 |
+| Order-Status || 1.90 | 1.97 | +4% || 1.63 | 2.02 | +23% | (run time too short) |
| Payment || 7.20 | 7.36 | +2% || 17.55 | 17.21 | -2% | 0.0066 |
-| Stock-Level || 3.50 | 3.44 | -2% || 1.60 | 1.45 | -9% | (run time too short) |
+--------------++----------+---------+--------++----------+----------+--------+----------------------+
| Sum || 112.13 | 112.15 | +0% || | | | |
| Geomean || | | || | | +0% | |
+--------------++----------+---------+--------++----------+----------+--------+----------------------+hyriseBenchmarkTPCC - multi-threaded, shuffled, 28 clients, 10 warehouses, 28 cores (high contention) Sum of avg. item runtimes: +14% || Geometric mean of throughput changes: +8%Configuration Overview - click to expand +Configuration Overview---------+-------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | /home/Daniel.Lindner/hyrise/clang15-release/benchmark_all_results/hyriseBenchmarkTPCC_f8f0db40e8efb4834f89e97e1cd086d8c0bf7f91_mt_highcont.json | /home/Daniel.Lindner/hyrise/clang15-release/benchmark_all_results/hyriseBenchmarkTPCC_b3b0c86fc42c8c6f8631df85704d47162d01641d_mt_highcont.json |
+-------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | f8f0db40e8efb4834f89e97e1cd086d8c0bf7f91-dirty | b3b0c86fc42c8c6f8631df85704d47162d01641d-dirty |
| benchmark_mode | Shuffled | Shuffled |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 28 | 28 |
| compiler | clang 15.0.7 | clang 15.0.7 |
| cores | 28 | 28 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-22 15:37:22 | 2024-05-22 19:14:35 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 600000000000 | 600000000000 |
| max_runs | -1 | -1 |
| scale_factor | 10 | 10 |
| time_unit | ns | ns |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [0, 0, 28] | [0, 0, 28] |
| verify | False | False |
| warmup_duration | 0 | 0 |
+-------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------+ +--------------++----------+---------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+--------------++----------+---------+--------++----------+----------+--------+---------+
-| Delivery || 113.38 | 129.70 | +14% || 21.87 | 22.44 | +3% | 0.0000 |
| unsucc.: || 1.29 | 1.77 | +37% || 5.94 | 8.75 | +47% | |
+| New-Order || 72.16 | 78.98 | +9% || 246.87 | 262.39 | +6% | 0.0000 |
| unsucc.: || 5.11 | 6.70 | +31% || 66.05 | 86.88 | +32% | |
+| Order-Status || 4.70 | 6.01 | +28% || 27.82 | 31.45 | +13% | 0.0000 |
+| Payment || 14.11 | 14.90 | +6% || 172.51 | 187.35 | +9% | 0.0000 |
| unsucc.: || 2.40 | 2.94 | +22% || 126.53 | 147.47 | +17% | |
+| Stock-Level || 6.65 | 10.64 | +60% || 27.82 | 31.10 | +12% | 0.0000 |
+--------------++----------+---------+--------++----------+----------+--------+---------+
-| Sum || 210.99 | 240.23 | +14% || | | | |
+| Geomean || | | || | | +8% | |
+--------------++----------+---------+--------++----------+----------+--------+---------+hyriseBenchmarkTPCC - multi-threaded, shuffled, 10 clients, 10 warehouses, 28 cores (low contention) Sum of avg. item runtimes: -3% || Geometric mean of throughput changes: +20%Configuration Overview - click to expand +Configuration Overview---------+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | /home/Daniel.Lindner/hyrise/clang15-release/benchmark_all_results/hyriseBenchmarkTPCC_f8f0db40e8efb4834f89e97e1cd086d8c0bf7f91_mt_lowcont.json | /home/Daniel.Lindner/hyrise/clang15-release/benchmark_all_results/hyriseBenchmarkTPCC_b3b0c86fc42c8c6f8631df85704d47162d01641d_mt_lowcont.json |
+-------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | f8f0db40e8efb4834f89e97e1cd086d8c0bf7f91-dirty | b3b0c86fc42c8c6f8631df85704d47162d01641d-dirty |
| benchmark_mode | Shuffled | Shuffled |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 10 | 10 |
| compiler | clang 15.0.7 | clang 15.0.7 |
| cores | 28 | 28 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-22 15:47:31 | 2024-05-22 19:24:44 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 600000000000 | 600000000000 |
| max_runs | -1 | -1 |
| scale_factor | 10 | 10 |
| time_unit | ns | ns |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [0, 0, 28] | [0, 0, 28] |
| verify | False | False |
| warmup_duration | 0 | 0 |
+-------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+ +--------------++----------+---------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+--------------++----------+---------+--------++----------+----------+--------+---------+
+| Delivery || 92.98 | 91.10 | -2% || 8.34 | 9.79 | +17% | 0.0000 |
| unsucc.: || 1.20 | 1.15 | -4% || 0.51 | 0.82 | +62% | |
+| New-Order || 66.48 | 64.77 | -3% || 91.73 | 109.12 | +19% | 0.0000 |
| unsucc.: || 4.58 | 3.98 | -13% || 7.74 | 10.26 | +33% | |
+| Order-Status || 7.28 | 6.78 | -7% || 8.84 | 10.76 | +22% | 0.0000 |
+| Payment || 21.43 | 19.84 | -7% || 75.08 | 88.46 | +18% | 0.0000 |
| unsucc.: || 2.46 | 2.29 | -7% || 19.99 | 24.93 | +25% | |
+| Stock-Level || 10.39 | 9.55 | -8% || 8.84 | 10.91 | +23% | 0.0000 |
+--------------++----------+---------+--------++----------+----------+--------+---------+
| Sum || 198.54 | 192.05 | -3% || | | | |
+| Geomean || | | || | | +20% | |
+--------------++----------+---------+--------++----------+----------+--------+---------+hyriseBenchmarkJoinOrder - single-threaded Sum of avg. item runtimes: +3% || Geometric mean of throughput changes: -7%Configuration Overview - click to expand +Configuration Overview---+---------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | /home/Daniel.Lindner/hyrise/clang15-release/benchmark_all_results/hyriseBenchmarkJoinOrder_f8f0db40e8efb4834f89e97e1cd086d8c0bf7f91_st.json | /home/Daniel.Lindner/hyrise/clang15-release/benchmark_all_results/hyriseBenchmarkJoinOrder_b3b0c86fc42c8c6f8631df85704d47162d01641d_st.json |
+-------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | f8f0db40e8efb4834f89e97e1cd086d8c0bf7f91-dirty | b3b0c86fc42c8c6f8631df85704d47162d01641d-dirty |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| compiler | clang 15.0.7 | clang 15.0.7 |
| cores | 0 | 0 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-22 15:57:38 | 2024-05-22 19:34:51 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 60000000000 | 60000000000 |
| max_runs | 50 | 50 |
| time_unit | ns | ns |
| using_scheduler | False | False |
| verify | False | False |
| warmup_duration | 1000000000 | 1000000000 |
+-------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+ +---------++----------+----------+--------++----------+----------+--------+----------------------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+---------++----------+----------+--------++----------+----------+--------+----------------------+
| 10a || 107.60 | 109.07 | +1%˄ || 9.29 | 9.17 | -1%˄ | (run time too short) |
| 10b || 67.20 | 67.21 | +0%˄ || 14.88 | 14.88 | -0%˄ | (run time too short) |
| 10c || 338.52 | 332.17 | -2%˄ || 2.95 | 3.01 | +2%˄ | (run time too short) |
| 11a || 95.32 | 96.57 | +1%˄ || 10.49 | 10.35 | -1%˄ | (run time too short) |
| 11b || 90.85 | 93.47 | +3%˄ || 11.01 | 10.70 | -3%˄ | (run time too short) |
-| 11c || 20.06 | 25.14 | +25%˄ || 49.86 | 39.78 | -20%˄ | (run time too short) |
-| 11d || 22.16 | 27.21 | +23%˄ || 45.11 | 36.74 | -19%˄ | (run time too short) |
-| 12a || 27.90 | 29.33 | +5%˄ || 35.84 | 34.09 | -5%˄ | (run time too short) |
-| 12b || 29.41 | 41.54 | +41%˄ || 34.00 | 24.07 | -29%˄ | (run time too short) |
| 12c || 61.09 | 63.32 | +4%˄ || 16.37 | 15.79 | -4%˄ | (run time too short) |
-| 13a || 178.53 | 191.95 | +8%˄ || 5.60 | 5.21 | -7%˄ | (run time too short) |
-| 13b || 223.83 | 242.11 | +8%˄ || 4.47 | 4.13 | -8%˄ | (run time too short) |
-| 13c || 185.26 | 204.24 | +10%˄ || 5.40 | 4.90 | -9%˄ | (run time too short) |
| 13d || 485.23 | 501.37 | +3%˄ || 2.06 | 1.99 | -3%˄ | (run time too short) |
-| 14a || 94.95 | 99.24 | +5%˄ || 10.53 | 10.08 | -4%˄ | (run time too short) |
| 14b || 109.90 | 112.47 | +2%˄ || 9.10 | 8.89 | -2%˄ | (run time too short) |
| 14c || 205.53 | 208.46 | +1%˄ || 4.87 | 4.80 | -1%˄ | (run time too short) |
-| 15a || 51.18 | 55.36 | +8%˄ || 19.54 | 18.06 | -8%˄ | (run time too short) |
-| 15b || 52.71 | 55.75 | +6%˄ || 18.97 | 17.94 | -5%˄ | (run time too short) |
-| 15c || 40.49 | 46.56 | +15%˄ || 24.70 | 21.47 | -13%˄ | (run time too short) |
-| 15d || 38.19 | 44.10 | +15%˄ || 26.18 | 22.67 | -13%˄ | (run time too short) |
| 16a || 1309.48 | 1309.67 | +0% || 0.76 | 0.76 | -0% | 0.9740 |
| 16b || 2414.22 | 2447.18 | +1% || 0.41 | 0.41 | -1% | 0.0119 |
| 16c || 1471.20 | 1487.34 | +1% || 0.68 | 0.67 | -1% | 0.0194 |
| 16d || 1434.58 | 1437.44 | +0% || 0.70 | 0.70 | -0% | 0.6295 |
| 17a || 437.61 | 455.61 | +4%˄ || 2.29 | 2.19 | -4%˄ | (run time too short) |
-| 17b || 354.83 | 373.27 | +5%˄ || 2.82 | 2.68 | -5%˄ | (run time too short) |
-| 17c || 338.20 | 356.79 | +5%˄ || 2.96 | 2.80 | -5%˄ | (run time too short) |
-| 17d || 406.86 | 426.56 | +5%˄ || 2.46 | 2.34 | -5%˄ | (run time too short) |
| 17e || 1482.10 | 1502.24 | +1% || 0.67 | 0.67 | -1% | 0.0000 |
| 17f || 859.41 | 877.28 | +2%˄ || 1.16 | 1.14 | -2%˄ | (run time too short) |
-| 18a || 131.11 | 144.31 | +10%˄ || 7.63 | 6.93 | -9%˄ | (run time too short) |
| 18b || 101.96 | 104.09 | +2%˄ || 9.81 | 9.61 | -2%˄ | (run time too short) |
| 18c || 197.69 | 202.91 | +3%˄ || 5.06 | 4.93 | -3%˄ | (run time too short) |
-| 19a || 210.83 | 229.28 | +9%˄ || 4.74 | 4.36 | -8%˄ | (run time too short) |
-| 19b || 178.30 | 191.65 | +7%˄ || 5.61 | 5.22 | -7%˄ | (run time too short) |
-| 19c || 252.90 | 273.65 | +8%˄ || 3.95 | 3.65 | -8%˄ | (run time too short) |
| 19d || 637.39 | 653.14 | +2%˄ || 1.57 | 1.53 | -2%˄ | (run time too short) |
-| 1a || 9.03 | 12.51 | +39%˄ || 110.70 | 79.91 | -28%˄ | (run time too short) |
-| 1b || 11.45 | 14.08 | +23%˄ || 87.29 | 71.03 | -19%˄ | (run time too short) |
-| 1c || 12.73 | 14.27 | +12%˄ || 78.52 | 70.05 | -11%˄ | (run time too short) |
-| 1d || 8.51 | 11.94 | +40%˄ || 117.50 | 83.75 | -29%˄ | (run time too short) |
| 20a || 612.54 | 629.31 | +3%˄ || 1.63 | 1.59 | -3%˄ | (run time too short) |
| 20b || 656.27 | 664.87 | +1%˄ || 1.52 | 1.50 | -1%˄ | (run time too short) |
-| 20c || 361.44 | 382.59 | +6%˄ || 2.77 | 2.61 | -6%˄ | (run time too short) |
| 21a || 110.60 | 111.10 | +0%˄ || 9.04 | 9.00 | -0%˄ | (run time too short) |
| 21b || 101.46 | 103.72 | +2%˄ || 9.86 | 9.64 | -2%˄ | (run time too short) |
| 21c || 112.22 | 114.11 | +2%˄ || 8.91 | 8.76 | -2%˄ | (run time too short) |
| 22a || 169.36 | 172.95 | +2%˄ || 5.90 | 5.78 | -2%˄ | (run time too short) |
| 22b || 153.98 | 157.20 | +2%˄ || 6.49 | 6.36 | -2%˄ | (run time too short) |
| 22c || 228.10 | 231.51 | +1%˄ || 4.38 | 4.32 | -1%˄ | (run time too short) |
| 22d || 348.43 | 354.01 | +2%˄ || 2.87 | 2.82 | -2%˄ | (run time too short) |
-| 23a || 43.31 | 49.22 | +14%˄ || 23.09 | 20.32 | -12%˄ | (run time too short) |
-| 23b || 48.61 | 53.94 | +11%˄ || 20.57 | 18.54 | -10%˄ | (run time too short) |
-| 23c || 45.85 | 52.06 | +14%˄ || 21.81 | 19.21 | -12%˄ | (run time too short) |
-| 24a || 203.66 | 224.90 | +10%˄ || 4.91 | 4.45 | -9%˄ | (run time too short) |
-| 24b || 197.66 | 218.62 | +11%˄ || 5.06 | 4.57 | -10%˄ | (run time too short) |
-| 25a || 104.70 | 114.09 | +9%˄ || 9.55 | 8.77 | -8%˄ | (run time too short) |
-| 25b || 64.53 | 72.28 | +12%˄ || 15.50 | 13.83 | -11%˄ | (run time too short) |
| 25c || 246.45 | 254.70 | +3%˄ || 4.06 | 3.93 | -3%˄ | (run time too short) |
-| 26a || 170.56 | 187.61 | +10%˄ || 5.86 | 5.33 | -9%˄ | (run time too short) |
-| 26b || 139.70 | 155.38 | +11%˄ || 7.16 | 6.44 | -10%˄ | (run time too short) |
-| 26c || 294.60 | 314.23 | +7%˄ || 3.39 | 3.18 | -6%˄ | (run time too short) |
| 27a || 103.93 | 107.01 | +3%˄ || 9.62 | 9.34 | -3%˄ | (run time too short) |
| 27b || 96.14 | 98.87 | +3%˄ || 10.40 | 10.11 | -3%˄ | (run time too short) |
| 27c || 112.13 | 114.43 | +2%˄ || 8.92 | 8.74 | -2%˄ | (run time too short) |
| 28a || 202.79 | 209.54 | +3%˄ || 4.93 | 4.77 | -3%˄ | (run time too short) |
-| 28b || 62.35 | 65.94 | +6%˄ || 16.04 | 15.16 | -5%˄ | (run time too short) |
| 28c || 168.83 | 173.18 | +3%˄ || 5.92 | 5.77 | -3%˄ | (run time too short) |
-| 29a || 167.50 | 189.02 | +13%˄ || 5.97 | 5.29 | -11%˄ | (run time too short) |
| 29b || 749.41 | 775.63 | +3%˄ || 1.33 | 1.29 | -3%˄ | (run time too short) |
-| 29c || 204.02 | 226.76 | +11%˄ || 4.90 | 4.41 | -10%˄ | (run time too short) |
-| 2a || 25.87 | 30.79 | +19%˄ || 38.65 | 32.47 | -16%˄ | (run time too short) |
-| 2b || 24.13 | 29.04 | +20%˄ || 41.43 | 34.44 | -17%˄ | (run time too short) |
-| 2c || 20.89 | 25.72 | +23%˄ || 47.87 | 38.88 | -19%˄ | (run time too short) |
-| 2d || 48.07 | 53.21 | +11%˄ || 20.80 | 18.79 | -10%˄ | (run time too short) |
-| 30a || 97.25 | 105.94 | +9%˄ || 10.28 | 9.44 | -8%˄ | (run time too short) |
-| 30b || 102.22 | 113.11 | +11%˄ || 9.78 | 8.84 | -10%˄ | (run time too short) |
-| 30c || 164.10 | 172.10 | +5%˄ || 6.09 | 5.81 | -5%˄ | (run time too short) |
-| 31a || 82.76 | 92.72 | +12%˄ || 12.08 | 10.78 | -11%˄ | (run time too short) |
-| 31b || 81.98 | 90.49 | +10%˄ || 12.20 | 11.05 | -9%˄ | (run time too short) |
-| 31c || 94.62 | 104.94 | +11%˄ || 10.57 | 9.53 | -10%˄ | (run time too short) |
-| 32a || 12.99 | 17.19 | +32%˄ || 76.95 | 58.17 | -24%˄ | (run time too short) |
-| 32b || 29.09 | 33.51 | +15%˄ || 34.37 | 29.84 | -13%˄ | (run time too short) |
-| 33a || 33.11 | 37.60 | +14%˄ || 30.20 | 26.59 | -12%˄ | (run time too short) |
-| 33b || 20.92 | 26.27 | +26%˄ || 47.80 | 38.06 | -20%˄ | (run time too short) |
-| 33c || 39.77 | 44.29 | +11%˄ || 25.15 | 22.58 | -10%˄ | (run time too short) |
-| 3a || 58.39 | 61.25 | +5%˄ || 17.12 | 16.33 | -5%˄ | (run time too short) |
-| 3b || 15.01 | 18.44 | +23%˄ || 66.63 | 54.23 | -19%˄ | (run time too short) |
| 3c || 222.89 | 223.92 | +0%˄ || 4.49 | 4.47 | -0%˄ | (run time too short) |
| 4a || 134.73 | 136.22 | +1%˄ || 7.42 | 7.34 | -1%˄ | (run time too short) |
-| 4b || 14.12 | 17.92 | +27%˄ || 70.82 | 55.81 | -21%˄ | (run time too short) |
| 4c || 188.51 | 188.94 | +0%˄ || 5.30 | 5.29 | -0%˄ | (run time too short) |
| 5a || 42.48 | 43.00 | +1%˄ || 23.54 | 23.26 | -1%˄ | (run time too short) |
| 5b || 46.72 | 46.81 | +0%˄ || 21.40 | 21.36 | -0%˄ | (run time too short) |
| 5c || 87.21 | 87.57 | +0%˄ || 11.47 | 11.42 | -0%˄ | (run time too short) |
| 6a || 166.88 | 172.67 | +3%˄ || 5.99 | 5.79 | -3%˄ | (run time too short) |
| 6b || 178.79 | 184.24 | +3%˄ || 5.59 | 5.43 | -3%˄ | (run time too short) |
| 6c || 159.14 | 164.53 | +3%˄ || 6.28 | 6.08 | -3%˄ | (run time too short) |
| 6d || 381.82 | 388.62 | +2%˄ || 2.62 | 2.57 | -2%˄ | (run time too short) |
| 6e || 165.98 | 172.27 | +4%˄ || 6.02 | 5.80 | -4%˄ | (run time too short) |
| 6f || 678.55 | 688.88 | +2%˄ || 1.47 | 1.45 | -1%˄ | (run time too short) |
-| 7a || 86.23 | 110.72 | +28%˄ || 11.60 | 9.03 | -22%˄ | (run time too short) |
-| 7b || 79.74 | 104.07 | +31%˄ || 12.54 | 9.61 | -23%˄ | (run time too short) |
| 7c || 561.78 | 577.04 | +3%˄ || 1.78 | 1.73 | -3%˄ | (run time too short) |
| 8a || 167.18 | 171.90 | +3%˄ || 5.98 | 5.82 | -3%˄ | (run time too short) |
| 8b || 163.29 | 167.43 | +3%˄ || 6.12 | 5.97 | -2%˄ | (run time too short) |
| 8c || 1802.84 | 1809.22 | +0% || 0.55 | 0.55 | -0% | 0.1008 |
-| 8d || 292.23 | 321.48 | +10%˄ || 3.42 | 3.11 | -9%˄ | (run time too short) |
-| 9a || 238.24 | 249.76 | +5%˄ || 4.20 | 4.00 | -5%˄ | (run time too short) |
| 9b || 149.14 | 154.58 | +4%˄ || 6.70 | 6.47 | -4%˄ | (run time too short) |
-| 9c || 264.32 | 277.02 | +5%˄ || 3.78 | 3.61 | -5%˄ | (run time too short) |
| 9d || 409.82 | 422.98 | +3%˄ || 2.44 | 2.36 | -3%˄ | (run time too short) |
+---------++----------+----------+--------++----------+----------+--------+----------------------+
| Sum || 28701.26 | 29689.01 | +3% || | | | |
-| Geomean || | | || | | -7% | |
+---------++----------+----------+--------++----------+----------+--------+----------------------+
| Notes || ˄ Execution stopped due to max runs reached |
+---------++----------+----------+--------++----------+----------+--------+----------------------+hyriseBenchmarkJoinOrder - multi-threaded, ordered, 1 client, 28 cores Sum of avg. item runtimes: +1% || Geometric mean of throughput changes: +11%Configuration Overview - click to expand +Configuration Overview---------+-----------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | /home/Daniel.Lindner/hyrise/clang15-release/benchmark_all_results/hyriseBenchmarkJoinOrder_f8f0db40e8efb4834f89e97e1cd086d8c0bf7f91_mt_ordered.json | /home/Daniel.Lindner/hyrise/clang15-release/benchmark_all_results/hyriseBenchmarkJoinOrder_b3b0c86fc42c8c6f8631df85704d47162d01641d_mt_ordered.json |
+-------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | f8f0db40e8efb4834f89e97e1cd086d8c0bf7f91-dirty | b3b0c86fc42c8c6f8631df85704d47162d01641d-dirty |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| compiler | clang 15.0.7 | clang 15.0.7 |
| cores | 28 | 28 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-22 16:21:53 | 2024-05-22 19:59:54 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 60000000000 | 60000000000 |
| max_runs | 50 | 50 |
| time_unit | ns | ns |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [0, 0, 28] | [0, 0, 28] |
| verify | False | False |
| warmup_duration | 1000000000 | 1000000000 |
+-------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------+ +---------++----------+----------+--------++----------+----------+--------+----------------------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+---------++----------+----------+--------++----------+----------+--------+----------------------+
+| 10a || 45.34 | 43.44 | -4%˄ || 19.78 | 22.95 | +16%˄ | (run time too short) |
+| 10b || 25.21 | 25.35 | +1%˄ || 32.93 | 39.21 | +19%˄ | (run time too short) |
| 10c || 141.00 | 143.36 | +2%˄ || 6.92 | 6.97 | +1%˄ | (run time too short) |
+| 11a || 29.12 | 28.60 | -2%˄ || 30.48 | 34.84 | +14%˄ | (run time too short) |
+| 11b || 27.87 | 26.41 | -5%˄ || 32.49 | 37.75 | +16%˄ | (run time too short) |
+| 11c || 29.88 | 30.14 | +1%˄ || 30.26 | 33.03 | +9%˄ | (run time too short) |
+| 11d || 34.25 | 34.63 | +1%˄ || 24.67 | 28.74 | +17%˄ | (run time too short) |
+| 12a || 30.12 | 29.76 | -1%˄ || 29.56 | 33.43 | +13%˄ | (run time too short) |
-| 12b || 19.17 | 20.57 | +7%˄ || 47.45 | 48.30 | +2%˄ | (run time too short) |
+| 12c || 61.96 | 60.36 | -3%˄ || 14.32 | 16.52 | +15%˄ | (run time too short) |
| 13a || 141.86 | 144.15 | +2%˄ || 6.80 | 6.93 | +2%˄ | (run time too short) |
-| 13b || 100.53 | 111.23 | +11%˄ || 9.54 | 8.98 | -6%˄ | (run time too short) |
| 13c || 96.03 | 99.25 | +3%˄ || 9.92 | 10.06 | +1%˄ | (run time too short) |
| 13d || 383.20 | 394.88 | +3%˄ || 2.58 | 2.53 | -2%˄ | (run time too short) |
+| 14a || 63.99 | 62.17 | -3%˄ || 14.20 | 16.05 | +13%˄ | (run time too short) |
+| 14b || 76.08 | 74.03 | -3%˄ || 12.35 | 13.49 | +9%˄ | (run time too short) |
| 14c || 184.22 | 184.35 | +0%˄ || 5.27 | 5.42 | +3%˄ | (run time too short) |
+| 15a || 35.86 | 33.30 | -7%˄ || 24.57 | 29.91 | +22%˄ | (run time too short) |
+| 15b || 30.46 | 29.76 | -2%˄ || 28.54 | 33.46 | +17%˄ | (run time too short) |
+| 15c || 30.43 | 29.77 | -2%˄ || 28.37 | 33.43 | +18%˄ | (run time too short) |
+| 15d || 35.88 | 36.53 | +2%˄ || 24.58 | 27.25 | +11%˄ | (run time too short) |
| 16a || 732.78 | 741.61 | +1%˄ || 1.36 | 1.35 | -1%˄ | (run time too short) |
| 16b || 1793.24 | 1798.71 | +0% || 0.55 | 0.56 | +1% | 0.4964 |
| 16c || 894.09 | 904.97 | +1%˄ || 1.11 | 1.10 | -1%˄ | (run time too short) |
| 16d || 869.70 | 871.73 | +0%˄ || 1.14 | 1.15 | +0%˄ | (run time too short) |
| 17a || 208.56 | 209.60 | +0%˄ || 4.68 | 4.77 | +2%˄ | (run time too short) |
| 17b || 176.05 | 180.84 | +3%˄ || 5.52 | 5.52 | +0%˄ | (run time too short) |
| 17c || 148.93 | 155.03 | +4%˄ || 6.53 | 6.44 | -1%˄ | (run time too short) |
| 17d || 145.73 | 149.80 | +3%˄ || 6.59 | 6.67 | +1%˄ | (run time too short) |
| 17e || 739.82 | 745.73 | +1%˄ || 1.34 | 1.38 | +3%˄ | (run time too short) |
| 17f || 328.52 | 330.20 | +1%˄ || 3.00 | 3.03 | +1%˄ | (run time too short) |
| 18a || 59.00 | 61.42 | +4%˄ || 16.17 | 16.25 | +0%˄ | (run time too short) |
+| 18b || 63.02 | 58.43 | -7%˄ || 14.50 | 17.07 | +18%˄ | (run time too short) |
| 18c || 93.45 | 95.32 | +2%˄ || 10.11 | 10.47 | +4%˄ | (run time too short) |
| 19a || 115.66 | 116.48 | +1%˄ || 8.24 | 8.57 | +4%˄ | (run time too short) |
+| 19b || 76.04 | 77.78 | +2%˄ || 12.19 | 12.84 | +5%˄ | (run time too short) |
| 19c || 126.05 | 130.34 | +3%˄ || 7.63 | 7.66 | +0%˄ | (run time too short) |
| 19d || 512.45 | 514.85 | +0%˄ || 1.93 | 1.94 | +1%˄ | (run time too short) |
+| 1a || 10.57 | 11.29 | +7%˄ || 58.03 | 87.29 | +50%˄ | (run time too short) |
+| 1b || 5.98 | 6.88 | +15%˄ || 98.51 | 142.51 | +45%˄ | (run time too short) |
+| 1c || 6.00 | 6.39 | +7%˄ || 98.46 | 153.56 | +56%˄ | (run time too short) |
+| 1d || 8.27 | 9.15 | +11%˄ || 98.59 | 107.46 | +9%˄ | (run time too short) |
-| 20a || 83.61 | 88.42 | +6%˄ || 11.32 | 11.30 | -0%˄ | (run time too short) |
+| 20b || 67.60 | 69.34 | +3%˄ || 13.62 | 14.40 | +6%˄ | (run time too short) |
| 20c || 83.47 | 86.53 | +4%˄ || 11.26 | 11.54 | +2%˄ | (run time too short) |
+| 21a || 34.83 | 32.54 | -7%˄ || 24.70 | 30.62 | +24%˄ | (run time too short) |
+| 21b || 31.66 | 30.38 | -4%˄ || 25.60 | 32.78 | +28%˄ | (run time too short) |
+| 21c || 36.52 | 34.73 | -5%˄ || 24.58 | 28.68 | +17%˄ | (run time too short) |
+| 22a || 69.10 | 69.81 | +1%˄ || 13.66 | 14.30 | +5%˄ | (run time too short) |
+| 22b || 58.05 | 58.75 | +1%˄ || 16.21 | 16.99 | +5%˄ | (run time too short) |
| 22c || 135.55 | 137.88 | +2%˄ || 7.11 | 7.24 | +2%˄ | (run time too short) |
+| 22d || 246.49 | 240.80 | -2%˄ || 3.97 | 4.15 | +5%˄ | (run time too short) |
+| 23a || 23.68 | 24.92 | +5%˄ || 32.92 | 39.91 | +21%˄ | (run time too short) |
+| 23b || 34.78 | 32.68 | -6%˄ || 24.68 | 30.51 | +24%˄ | (run time too short) |
+| 23c || 26.23 | 26.14 | -0%˄ || 32.93 | 38.08 | +16%˄ | (run time too short) |
+| 24a || 68.81 | 68.22 | -1%˄ || 13.67 | 14.63 | +7%˄ | (run time too short) |
+| 24b || 60.67 | 61.96 | +2%˄ || 15.23 | 16.11 | +6%˄ | (run time too short) |
| 25a || 58.29 | 60.73 | +4%˄ || 16.22 | 16.42 | +1%˄ | (run time too short) |
+| 25b || 40.85 | 38.93 | -5%˄ || 20.94 | 25.60 | +22%˄ | (run time too short) |
| 25c || 161.76 | 162.20 | +0%˄ || 6.01 | 6.16 | +2%˄ | (run time too short) |
-| 26a || 87.09 | 92.25 | +6%˄ || 10.92 | 10.82 | -1%˄ | (run time too short) |
-| 26b || 66.80 | 70.94 | +6%˄ || 14.13 | 14.07 | -0%˄ | (run time too short) |
| 26c || 144.74 | 150.85 | +4%˄ || 6.59 | 6.62 | +0%˄ | (run time too short) |
+| 27a || 32.55 | 30.76 | -5%˄ || 24.72 | 32.42 | +31%˄ | (run time too short) |
+| 27b || 31.76 | 30.48 | -4%˄ || 25.48 | 32.69 | +28%˄ | (run time too short) |
+| 27c || 37.05 | 34.22 | -8%˄ || 24.59 | 29.12 | +18%˄ | (run time too short) |
| 28a || 150.75 | 149.92 | -1%˄ || 6.43 | 6.66 | +4%˄ | (run time too short) |
+| 28b || 37.38 | 35.49 | -5%˄ || 24.72 | 28.08 | +14%˄ | (run time too short) |
| 28c || 119.40 | 119.57 | +0%˄ || 8.10 | 8.35 | +3%˄ | (run time too short) |
+| 29a || 62.19 | 64.43 | +4%˄ || 14.81 | 15.50 | +5%˄ | (run time too short) |
| 29b || 108.79 | 110.30 | +1%˄ || 8.82 | 9.06 | +3%˄ | (run time too short) |
| 29c || 66.08 | 68.27 | +3%˄ || 14.18 | 14.62 | +3%˄ | (run time too short) |
+| 2a || 32.10 | 31.81 | -1%˄ || 24.68 | 31.30 | +27%˄ | (run time too short) |
| 2b || 29.36 | 30.36 | +3%˄ || 31.64 | 32.76 | +4%˄ | (run time too short) |
+| 2c || 22.05 | 23.03 | +4%˄ || 32.92 | 43.13 | +31%˄ | (run time too short) |
+| 2d || 58.52 | 58.22 | -1%˄ || 16.20 | 17.13 | +6%˄ | (run time too short) |
+| 30a || 54.88 | 54.24 | -1%˄ || 16.43 | 18.39 | +12%˄ | (run time too short) |
+| 30b || 47.62 | 45.54 | -4%˄ || 19.16 | 21.90 | +14%˄ | (run time too short) |
| 30c || 112.16 | 114.37 | +2%˄ || 8.53 | 8.73 | +2%˄ | (run time too short) |
+| 31a || 44.78 | 41.17 | -8%˄ || 19.77 | 24.21 | +22%˄ | (run time too short) |
+| 31b || 42.31 | 38.85 | -8%˄ || 19.85 | 25.66 | +29%˄ | (run time too short) |
+| 31c || 46.10 | 46.68 | +1%˄ || 19.71 | 21.36 | +8%˄ | (run time too short) |
+| 32a || 10.82 | 11.49 | +6%˄ || 49.85 | 85.86 | +72%˄ | (run time too short) |
+| 32b || 39.40 | 40.28 | +2%˄ || 23.62 | 24.73 | +5%˄ | (run time too short) |
+| 33a || 16.28 | 16.07 | -1%˄ || 49.33 | 61.70 | +25%˄ | (run time too short) |
+| 33b || 13.64 | 13.91 | +2%˄ || 49.33 | 71.12 | +44%˄ | (run time too short) |
+| 33c || 24.91 | 24.43 | -2%˄ || 32.88 | 40.70 | +24%˄ | (run time too short) |
+| 3a || 46.49 | 43.35 | -7%˄ || 19.44 | 23.00 | +18%˄ | (run time too short) |
+| 3b || 10.90 | 10.97 | +1%˄ || 49.81 | 89.93 | +81%˄ | (run time too short) |
| 3c || 86.80 | 88.75 | +2%˄ || 10.89 | 11.25 | +3%˄ | (run time too short) |
+| 4a || 77.02 | 77.13 | +0%˄ || 12.20 | 12.94 | +6%˄ | (run time too short) |
+| 4b || 9.11 | 8.98 | -1%˄ || 83.58 | 109.77 | +31%˄ | (run time too short) |
-| 4c || 99.91 | 105.30 | +5%˄ || 9.50 | 9.48 | -0%˄ | (run time too short) |
+| 5a || 35.65 | 32.26 | -10%˄ || 23.97 | 30.88 | +29%˄ | (run time too short) |
+| 5b || 28.54 | 28.49 | -0%˄ || 31.05 | 34.90 | +12%˄ | (run time too short) |
+| 5c || 58.45 | 53.07 | -9%˄ || 15.49 | 18.79 | +21%˄ | (run time too short) |
+| 6a || 32.65 | 33.67 | +3%˄ || 24.88 | 29.62 | +19%˄ | (run time too short) |
+| 6b || 54.95 | 56.90 | +4%˄ || 16.55 | 17.54 | +6%˄ | (run time too short) |
+| 6c || 29.67 | 31.46 | +6%˄ || 28.62 | 31.69 | +11%˄ | (run time too short) |
| 6d || 156.59 | 160.15 | +2%˄ || 6.19 | 6.24 | +1%˄ | (run time too short) |
+| 6e || 32.00 | 33.66 | +5%˄ || 25.52 | 29.62 | +16%˄ | (run time too short) |
| 6f || 480.92 | 483.35 | +1%˄ || 2.06 | 2.07 | +0%˄ | (run time too short) |
+| 7a || 38.24 | 33.96 | -11%˄ || 24.11 | 29.35 | +22%˄ | (run time too short) |
+| 7b || 37.64 | 33.02 | -12%˄ || 24.71 | 30.18 | +22%˄ | (run time too short) |
| 7c || 314.40 | 317.45 | +1%˄ || 3.13 | 3.15 | +1%˄ | (run time too short) |
+| 8a || 40.56 | 40.76 | +0%˄ || 21.89 | 24.45 | +12%˄ | (run time too short) |
+| 8b || 37.26 | 37.76 | +1%˄ || 24.60 | 26.40 | +7%˄ | (run time too short) |
| 8c || 1180.22 | 1184.12 | +0%˄ || 0.84 | 0.84 | +0%˄ | 0.3419 |
| 8d || 244.12 | 248.24 | +2%˄ || 4.02 | 4.03 | +0%˄ | (run time too short) |
| 9a || 148.01 | 149.50 | +1%˄ || 6.51 | 6.68 | +3%˄ | (run time too short) |
+| 9b || 86.90 | 85.37 | -2%˄ || 10.89 | 11.70 | +7%˄ | (run time too short) |
| 9c || 144.64 | 144.59 | -0%˄ || 6.67 | 6.91 | +3%˄ | (run time too short) |
| 9d || 340.72 | 342.22 | +0%˄ || 2.89 | 2.92 | +1%˄ | (run time too short) |
+---------++----------+----------+--------++----------+----------+--------+----------------------+
| Sum || 15450.16 | 15555.91 | +1% || | | | |
+| Geomean || | | || | | +11% | |
+---------++----------+----------+--------++----------+----------+--------+----------------------+
| Notes || ˄ Execution stopped due to max runs reached |
+---------++----------+----------+--------++----------+----------+--------+----------------------+hyriseBenchmarkJoinOrder - multi-threaded, shuffled, 28 clients, 28 cores Sum of avg. item runtimes: +3% || Geometric mean of throughput changes: -2%Configuration Overview - click to expand +Configuration Overview---------+---------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | /home/Daniel.Lindner/hyrise/clang15-release/benchmark_all_results/hyriseBenchmarkJoinOrder_f8f0db40e8efb4834f89e97e1cd086d8c0bf7f91_mt.json | /home/Daniel.Lindner/hyrise/clang15-release/benchmark_all_results/hyriseBenchmarkJoinOrder_b3b0c86fc42c8c6f8631df85704d47162d01641d_mt.json |
+-------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | f8f0db40e8efb4834f89e97e1cd086d8c0bf7f91-dirty | b3b0c86fc42c8c6f8631df85704d47162d01641d-dirty |
| benchmark_mode | Shuffled | Shuffled |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 28 | 28 |
| compiler | clang 15.0.7 | clang 15.0.7 |
| cores | 28 | 28 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-22 16:37:11 | 2024-05-22 20:15:23 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 1200000000000 | 1200000000000 |
| max_runs | -1 | -1 |
| time_unit | ns | ns |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [0, 0, 28] | [0, 0, 28] |
| verify | False | False |
| warmup_duration | 0 | 0 |
+-------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+ +---------++----------+----------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+---------++----------+----------+--------++----------+----------+--------+---------+
-| 10a || 166.42 | 186.09 | +12% || 0.61 | 0.60 | -2% | 0.3496 |
-| 10b || 131.10 | 145.43 | +11% || 0.61 | 0.60 | -2% | 0.5370 |
-| 10c || 457.37 | 502.24 | +10% || 0.61 | 0.60 | -1% | 0.2357 |
-| 11a || 108.96 | 119.23 | +9% || 0.61 | 0.60 | -1% | 0.5395 |
-| 11b || 102.27 | 133.79 | +31% || 0.61 | 0.60 | -2% | 0.1322 |
-| 11c || 83.62 | 95.81 | +15% || 0.61 | 0.60 | -1% | 0.3140 |
-| 11d || 91.50 | 97.75 | +7% || 0.61 | 0.60 | -2% | 0.6142 |
+| 12a || 133.29 | 117.85 | -12% || 0.61 | 0.60 | -2% | 0.3040 |
-| 12b || 76.76 | 86.25 | +12% || 0.61 | 0.60 | -2% | 0.4902 |
| 12c || 281.23 | 287.26 | +2% || 0.61 | 0.60 | -1% | 0.8274 |
-| 13a || 404.24 | 435.47 | +8% || 0.61 | 0.60 | -2% | 0.3279 |
-| 13b || 284.05 | 326.82 | +15% || 0.61 | 0.60 | -2% | 0.1248 |
+| 13c || 287.20 | 267.67 | -7% || 0.61 | 0.60 | -2% | 0.4220 |
| 13d || 831.93 | 851.22 | +2% || 0.61 | 0.60 | -2% | 0.6590 |
| 14a || 332.70 | 341.99 | +3% || 0.61 | 0.60 | -2% | 0.7455 |
-| 14b || 362.32 | 398.97 | +10% || 0.61 | 0.60 | -2% | 0.3263 |
-| 14c || 490.48 | 514.14 | +5% || 0.61 | 0.60 | -2% | 0.5021 |
+| 15a || 119.74 | 107.41 | -10% || 0.61 | 0.60 | -1% | 0.4342 |
+| 15b || 137.06 | 98.71 | -28% || 0.61 | 0.60 | -2% | 0.0536 |
-| 15c || 118.79 | 141.07 | +19% || 0.61 | 0.60 | -2% | 0.2451 |
+| 15d || 148.76 | 137.86 | -7% || 0.61 | 0.60 | -2% | 0.5376 |
| 16a || 1431.45 | 1412.49 | -1% || 0.61 | 0.60 | -2% | 0.7183 |
| 16b || 2742.89 | 2747.25 | +0% || 0.61 | 0.60 | -2% | 0.9326 |
-| 16c || 1561.10 | 1697.87 | +9% || 0.61 | 0.60 | -2% | 0.0129 |
| 16d || 1568.57 | 1562.01 | -0% || 0.61 | 0.60 | -1% | 0.9040 |
-| 17a || 756.09 | 821.05 | +9% || 0.61 | 0.60 | -2% | 0.1753 |
-| 17b || 550.85 | 619.66 | +12% || 0.61 | 0.60 | -2% | 0.0801 |
| 17c || 560.47 | 536.55 | -4% || 0.61 | 0.60 | -2% | 0.5378 |
-| 17d || 520.22 | 610.06 | +17% || 0.61 | 0.60 | -2% | 0.0212 |
| 17e || 1481.41 | 1484.29 | +0% || 0.61 | 0.60 | -1% | 0.9570 |
| 17f || 1043.15 | 1005.68 | -4% || 0.61 | 0.60 | -2% | 0.4560 |
-| 18a || 244.10 | 275.23 | +13% || 0.61 | 0.60 | -1% | 0.3052 |
+| 18b || 375.11 | 327.28 | -13% || 0.61 | 0.60 | -2% | 0.1806 |
+| 18c || 569.38 | 526.72 | -7% || 0.61 | 0.60 | -2% | 0.3214 |
| 19a || 516.61 | 500.24 | -3% || 0.61 | 0.60 | -2% | 0.6676 |
| 19b || 348.81 | 358.11 | +3% || 0.61 | 0.60 | -1% | 0.7779 |
-| 19c || 552.96 | 641.08 | +16% || 0.61 | 0.60 | -2% | 0.0362 |
| 19d || 1273.21 | 1246.00 | -2% || 0.61 | 0.60 | -2% | 0.5750 |
-| 1a || 39.04 | 53.23 | +36% || 0.61 | 0.60 | -2% | 0.1631 |
-| 1b || 36.74 | 55.13 | +50% || 0.61 | 0.60 | -2% | 0.0347 |
-| 1c || 40.89 | 45.86 | +12% || 0.61 | 0.60 | -2% | 0.6247 |
| 1d || 45.69 | 47.42 | +4% || 0.61 | 0.60 | -2% | 0.9121 |
+| 20a || 480.35 | 401.62 | -16% || 0.61 | 0.60 | -2% | 0.0347 |
| 20b || 348.43 | 350.03 | +0% || 0.61 | 0.60 | -1% | 0.9594 |
-| 20c || 319.30 | 342.78 | +7% || 0.61 | 0.60 | -2% | 0.3527 |
-| 21a || 130.38 | 185.10 | +42% || 0.61 | 0.60 | -1% | 0.0272 |
| 21b || 122.94 | 128.47 | +4% || 0.61 | 0.60 | -1% | 0.7746 |
+| 21c || 210.45 | 176.75 | -16% || 0.61 | 0.60 | -1% | 0.2333 |
+| 22a || 471.82 | 409.38 | -13% || 0.61 | 0.60 | -1% | 0.1411 |
-| 22b || 344.31 | 393.37 | +14% || 0.61 | 0.60 | -2% | 0.2141 |
-| 22c || 522.46 | 556.25 | +6% || 0.61 | 0.60 | -1% | 0.4284 |
-| 22d || 707.44 | 818.58 | +16% || 0.61 | 0.60 | -2% | 0.0258 |
-| 23a || 144.81 | 167.47 | +16% || 0.61 | 0.60 | -2% | 0.3029 |
+| 23b || 122.24 | 99.79 | -18% || 0.61 | 0.60 | -2% | 0.1462 |
-| 23c || 150.81 | 176.22 | +17% || 0.61 | 0.60 | -2% | 0.2511 |
-| 24a || 359.70 | 425.37 | +18% || 0.61 | 0.60 | -1% | 0.0999 |
-| 24b || 257.13 | 270.52 | +5% || 0.61 | 0.60 | -2% | 0.6383 |
-| 25a || 362.38 | 399.75 | +10% || 0.61 | 0.60 | -1% | 0.3219 |
-| 25b || 188.18 | 222.65 | +18% || 0.61 | 0.60 | -2% | 0.2788 |
-| 25c || 741.18 | 810.36 | +9% || 0.61 | 0.60 | -2% | 0.1492 |
+| 26a || 401.20 | 380.97 | -5% || 0.61 | 0.60 | -2% | 0.5583 |
-| 26b || 224.77 | 240.33 | +7% || 0.61 | 0.60 | -2% | 0.5595 |
-| 26c || 486.93 | 593.85 | +22% || 0.61 | 0.60 | -1% | 0.0087 |
| 27a || 171.28 | 171.97 | +0% || 0.61 | 0.60 | -2% | 0.9765 |
-| 27b || 141.87 | 163.59 | +15% || 0.61 | 0.60 | -2% | 0.3152 |
| 27c || 182.22 | 179.30 | -2% || 0.61 | 0.60 | -2% | 0.9021 |
+| 28a || 587.36 | 547.12 | -7% || 0.61 | 0.60 | -1% | 0.3181 |
-| 28b || 223.08 | 234.98 | +5% || 0.61 | 0.60 | -1% | 0.6556 |
+| 28c || 552.73 | 518.56 | -6% || 0.61 | 0.60 | -2% | 0.3872 |
-| 29a || 309.12 | 324.37 | +5% || 0.61 | 0.60 | -2% | 0.6614 |
+| 29b || 372.63 | 335.61 | -10% || 0.61 | 0.60 | -1% | 0.2688 |
| 29c || 325.69 | 338.53 | +4% || 0.61 | 0.60 | -2% | 0.6861 |
+| 2a || 133.44 | 126.09 | -6% || 0.61 | 0.60 | -2% | 0.6981 |
| 2b || 128.72 | 125.31 | -3% || 0.61 | 0.60 | -2% | 0.8469 |
+| 2c || 108.30 | 82.03 | -24% || 0.61 | 0.60 | -2% | 0.0943 |
+| 2d || 219.76 | 203.74 | -7% || 0.61 | 0.60 | -2% | 0.5251 |
| 30a || 323.28 | 318.01 | -2% || 0.61 | 0.60 | -2% | 0.8796 |
| 30b || 293.91 | 293.20 | -0% || 0.61 | 0.60 | -2% | 0.9828 |
-| 30c || 506.05 | 574.17 | +13% || 0.61 | 0.60 | -2% | 0.0681 |
-| 31a || 256.21 | 284.37 | +11% || 0.61 | 0.60 | -2% | 0.3960 |
| 31b || 245.33 | 235.87 | -4% || 0.61 | 0.60 | -2% | 0.7624 |
-| 31c || 318.34 | 339.73 | +7% || 0.61 | 0.60 | -2% | 0.6019 |
+| 32a || 58.77 | 52.07 | -11% || 0.61 | 0.60 | -2% | 0.5648 |
+| 32b || 171.69 | 152.98 | -11% || 0.61 | 0.60 | -2% | 0.3265 |
-| 33a || 100.80 | 119.69 | +19% || 0.61 | 0.60 | -2% | 0.2883 |
| 33b || 84.02 | 87.03 | +4% || 0.61 | 0.60 | -2% | 0.8313 |
-| 33c || 141.80 | 165.64 | +17% || 0.61 | 0.60 | -2% | 0.2743 |
| 3a || 233.68 | 232.55 | -0% || 0.61 | 0.60 | -1% | 0.9681 |
-| 3b || 51.98 | 90.21 | +74% || 0.61 | 0.60 | -2% | 0.0566 |
| 3c || 354.53 | 343.12 | -3% || 0.61 | 0.60 | -2% | 0.7271 |
+| 4a || 250.76 | 238.64 | -5% || 0.61 | 0.60 | -2% | 0.6230 |
-| 4b || 60.97 | 70.92 | +16% || 0.61 | 0.60 | -2% | 0.5430 |
-| 4c || 270.41 | 310.17 | +15% || 0.61 | 0.60 | -2% | 0.1470 |
-| 5a || 191.48 | 212.05 | +11% || 0.61 | 0.60 | -1% | 0.4419 |
-| 5b || 124.62 | 134.09 | +8% || 0.61 | 0.60 | -2% | 0.5911 |
-| 5c || 282.18 | 314.91 | +12% || 0.61 | 0.60 | -2% | 0.3294 |
+| 6a || 142.19 | 123.01 | -13% || 0.61 | 0.60 | -2% | 0.2378 |
+| 6b || 185.08 | 168.72 | -9% || 0.61 | 0.60 | -2% | 0.4325 |
+| 6c || 140.60 | 123.87 | -12% || 0.61 | 0.60 | -2% | 0.4889 |
| 6d || 549.47 | 537.56 | -2% || 0.61 | 0.60 | -2% | 0.7690 |
+| 6e || 156.78 | 136.70 | -13% || 0.61 | 0.60 | -2% | 0.2833 |
| 6f || 1076.96 | 1041.11 | -3% || 0.61 | 0.60 | -2% | 0.3823 |
-| 7a || 146.30 | 194.47 | +33% || 0.61 | 0.60 | -2% | 0.0474 |
+| 7b || 144.27 | 136.47 | -5% || 0.61 | 0.60 | -2% | 0.6734 |
+| 7c || 1034.20 | 941.25 | -9% || 0.61 | 0.60 | -1% | 0.0479 |
| 8a || 209.15 | 210.79 | +1% || 0.61 | 0.60 | -1% | 0.9541 |
-| 8b || 163.71 | 204.55 | +25% || 0.61 | 0.60 | -2% | 0.0956 |
-| 8c || 1930.36 | 2056.02 | +7% || 0.61 | 0.60 | -2% | 0.0084 |
-| 8d || 563.98 | 618.48 | +10% || 0.61 | 0.60 | -2% | 0.1197 |
-| 9a || 586.03 | 670.31 | +14% || 0.61 | 0.60 | -1% | 0.0749 |
| 9b || 364.31 | 378.50 | +4% || 0.61 | 0.60 | -2% | 0.6291 |
| 9c || 627.38 | 643.27 | +3% || 0.61 | 0.60 | -2% | 0.7404 |
| 9d || 976.57 | 989.02 | +1% || 0.61 | 0.60 | -2% | 0.7958 |
+---------++----------+----------+--------++----------+----------+--------+---------+
| Sum || 45280.09 | 46610.59 | +3% || | | | |
| Geomean || | | || | | -2% | |
+---------++----------+----------+--------++----------+----------+--------+---------+hyriseBenchmarkStarSchema - single-threaded Sum of avg. item runtimes: +7% || Geometric mean of throughput changes: -10%Configuration Overview - click to expand +Configuration Overview---+----------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | /home/Daniel.Lindner/hyrise/clang15-release/benchmark_all_results/hyriseBenchmarkStarSchema_f8f0db40e8efb4834f89e97e1cd086d8c0bf7f91_st.json | /home/Daniel.Lindner/hyrise/clang15-release/benchmark_all_results/hyriseBenchmarkStarSchema_b3b0c86fc42c8c6f8631df85704d47162d01641d_st.json |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | f8f0db40e8efb4834f89e97e1cd086d8c0bf7f91-dirty | b3b0c86fc42c8c6f8631df85704d47162d01641d-dirty |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| compiler | clang 15.0.7 | clang 15.0.7 |
| cores | 0 | 0 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-22 16:57:39 | 2024-05-22 20:35:49 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 60000000000 | 60000000000 |
| max_runs | 50 | 50 |
| scale_factor | 10.0 | 10.0 |
| time_unit | ns | ns |
| using_scheduler | False | False |
| verify | False | False |
| warmup_duration | 1000000000 | 1000000000 |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------+ +---------++----------+---------+--------++----------+----------+--------+----------------------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+---------++----------+---------+--------++----------+----------+--------+----------------------+
-| 1.1 || 328.49 | 368.96 | +12%˄ || 3.04 | 2.71 | -11%˄ | (run time too short) |
-| 1.2 || 155.09 | 162.58 | +5%˄ || 6.45 | 6.15 | -5%˄ | (run time too short) |
| 1.3 || 154.88 | 159.77 | +3%˄ || 6.46 | 6.26 | -3%˄ | (run time too short) |
-| 2.1 || 544.52 | 572.18 | +5%˄ || 1.84 | 1.75 | -5%˄ | (run time too short) |
| 2.2 || 288.82 | 301.34 | +4%˄ || 3.46 | 3.32 | -4%˄ | (run time too short) |
-| 2.3 || 192.80 | 204.33 | +6%˄ || 5.19 | 4.89 | -6%˄ | (run time too short) |
-| 3.1 || 2832.96 | 3044.31 | +7% || 0.35 | 0.33 | -7% | 0.0000 |
-| 3.2 || 257.03 | 301.09 | +17%˄ || 3.89 | 3.32 | -15%˄ | (run time too short) |
-| 3.3 || 115.84 | 153.64 | +33%˄ || 8.63 | 6.51 | -25%˄ | (run time too short) |
-| 3.4 || 108.85 | 144.62 | +33%˄ || 9.19 | 6.91 | -25%˄ | (run time too short) |
| 4.1 || 2898.34 | 2958.72 | +2% || 0.35 | 0.34 | -2% | 0.0750 |
-| 4.2 || 849.03 | 904.17 | +6%˄ || 1.18 | 1.11 | -6%˄ | (run time too short) |
-| 4.3 || 239.71 | 277.45 | +16%˄ || 4.17 | 3.60 | -14%˄ | (run time too short) |
+---------++----------+---------+--------++----------+----------+--------+----------------------+
-| Sum || 8966.35 | 9553.16 | +7% || | | | |
-| Geomean || | | || | | -10% | |
+---------++----------+---------+--------++----------+----------+--------+----------------------+
| Notes || ˄ Execution stopped due to max runs reached |
+---------++----------+---------+--------++----------+----------+--------+----------------------+hyriseBenchmarkStarSchema - multi-threaded, ordered, 1 client, 28 cores Sum of avg. item runtimes: -1% || Geometric mean of throughput changes: +4%Configuration Overview - click to expand +Configuration Overview---------+------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | /home/Daniel.Lindner/hyrise/clang15-release/benchmark_all_results/hyriseBenchmarkStarSchema_f8f0db40e8efb4834f89e97e1cd086d8c0bf7f91_mt_ordered.json | /home/Daniel.Lindner/hyrise/clang15-release/benchmark_all_results/hyriseBenchmarkStarSchema_b3b0c86fc42c8c6f8631df85704d47162d01641d_mt_ordered.json |
+-------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | f8f0db40e8efb4834f89e97e1cd086d8c0bf7f91-dirty | b3b0c86fc42c8c6f8631df85704d47162d01641d-dirty |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| compiler | clang 15.0.7 | clang 15.0.7 |
| cores | 28 | 28 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-22 17:03:03 | 2024-05-22 20:41:30 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 60000000000 | 60000000000 |
| max_runs | 50 | 50 |
| scale_factor | 10.0 | 10.0 |
| time_unit | ns | ns |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [0, 0, 28] | [0, 0, 28] |
| verify | False | False |
| warmup_duration | 1000000000 | 1000000000 |
+-------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+ +---------++----------+---------+--------++----------+----------+--------+----------------------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+---------++----------+---------+--------++----------+----------+--------+----------------------+
+| 1.1 || 104.97 | 104.20 | -1%˄ || 8.98 | 9.58 | +7%˄ | (run time too short) |
+| 1.2 || 89.80 | 89.72 | -0%˄ || 10.46 | 11.13 | +6%˄ | (run time too short) |
+| 1.3 || 85.68 | 85.66 | -0%˄ || 11.03 | 11.65 | +6%˄ | (run time too short) |
| 2.1 || 340.46 | 340.46 | -0%˄ || 2.89 | 2.94 | +1%˄ | (run time too short) |
| 2.2 || 163.90 | 165.99 | +1%˄ || 5.90 | 6.02 | +2%˄ | (run time too short) |
| 2.3 || 58.91 | 60.57 | +3%˄ || 16.12 | 16.46 | +2%˄ | (run time too short) |
| 3.1 || 860.17 | 830.16 | -3%˄ || 1.16 | 1.20 | +4%˄ | (run time too short) |
| 3.2 || 155.49 | 157.22 | +1%˄ || 6.23 | 6.35 | +2%˄ | (run time too short) |
| 3.3 || 78.71 | 80.01 | +2%˄ || 12.07 | 12.47 | +3%˄ | (run time too short) |
+| 3.4 || 85.42 | 75.34 | -12%˄ || 11.01 | 13.25 | +20%˄ | (run time too short) |
| 4.1 || 699.08 | 692.77 | -1%˄ || 1.42 | 1.44 | +2%˄ | (run time too short) |
| 4.2 || 382.79 | 386.98 | +1%˄ || 2.57 | 2.58 | +0%˄ | (run time too short) |
| 4.3 || 112.93 | 117.18 | +4%˄ || 8.47 | 8.52 | +1%˄ | (run time too short) |
+---------++----------+---------+--------++----------+----------+--------+----------------------+
| Sum || 3218.33 | 3186.26 | -1% || | | | |
| Geomean || | | || | | +4% | |
+---------++----------+---------+--------++----------+----------+--------+----------------------+
| Notes || ˄ Execution stopped due to max runs reached |
+---------++----------+---------+--------++----------+----------+--------+----------------------+hyriseBenchmarkStarSchema - multi-threaded, shuffled, 28 clients, 28 cores Sum of avg. item runtimes: +2% || Geometric mean of throughput changes: -2%Configuration Overview - click to expand +Configuration Overview---------+----------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | /home/Daniel.Lindner/hyrise/clang15-release/benchmark_all_results/hyriseBenchmarkStarSchema_f8f0db40e8efb4834f89e97e1cd086d8c0bf7f91_mt.json | /home/Daniel.Lindner/hyrise/clang15-release/benchmark_all_results/hyriseBenchmarkStarSchema_b3b0c86fc42c8c6f8631df85704d47162d01641d_mt.json |
+-------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | f8f0db40e8efb4834f89e97e1cd086d8c0bf7f91-dirty | b3b0c86fc42c8c6f8631df85704d47162d01641d-dirty |
| benchmark_mode | Shuffled | Shuffled |
| build_type | release | release |
| chunk_indexes | False | False |
| chunk_size | 65535 | 65535 |
| clients | 28 | 28 |
| compiler | clang 15.0.7 | clang 15.0.7 |
| cores | 28 | 28 |
| data_preparation_cores | 0 | 0 |
| date | 2024-05-22 17:06:22 | 2024-05-22 20:44:52 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| max_duration | 1200000000000 | 1200000000000 |
| max_runs | -1 | -1 |
| scale_factor | 10.0 | 10.0 |
| time_unit | ns | ns |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [0, 0, 28] | [0, 0, 28] |
| verify | False | False |
| warmup_duration | 0 | 0 |
+-------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------+ +---------++----------+----------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+---------++----------+----------+--------++----------+----------+--------+---------+
| 1.1 || 649.03 | 659.20 | +2% || 2.12 | 2.08 | -2% | 0.6751 |
| 1.2 || 372.76 | 366.14 | -2% || 2.12 | 2.08 | -2% | 0.6990 |
| 1.3 || 343.68 | 358.74 | +4% || 2.12 | 2.08 | -2% | 0.3806 |
| 2.1 || 1640.83 | 1658.78 | +1% || 2.12 | 2.08 | -2% | 0.6561 |
| 2.2 || 876.23 | 883.73 | +1% || 2.12 | 2.08 | -2% | 0.8004 |
| 2.3 || 333.59 | 345.40 | +4% || 2.12 | 2.08 | -2% | 0.5313 |
-| 3.1 || 2464.10 | 2610.54 | +6% || 2.12 | 2.08 | -2% | 0.0003 |
| 3.2 || 819.46 | 815.79 | -0% || 2.12 | 2.08 | -2% | 0.8974 |
| 3.3 || 436.78 | 418.42 | -4% || 2.12 | 2.08 | -2% | 0.3860 |
+| 3.4 || 431.05 | 395.62 | -8% || 2.12 | 2.08 | -2% | 0.0693 |
-| 4.1 || 2166.52 | 2296.65 | +6% || 2.12 | 2.08 | -2% | 0.0014 |
| 4.2 || 1948.06 | 1964.07 | +1% || 2.12 | 2.08 | -2% | 0.7211 |
| 4.3 || 614.21 | 640.53 | +4% || 2.12 | 2.08 | -2% | 0.2767 |
+---------++----------+----------+--------++----------+----------+--------+---------+
| Sum || 13096.31 | 13413.61 | +2% || | | | |
| Geomean || | | || | | -2% | |
+---------++----------+----------+--------++----------+----------+--------+---------+ |
|
If we do not want to have colored output in the end or in |
dey4ss
approved these changes
May 23, 2024
Bouncner
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The output of
benchmark_all.shuses the--githubflag (enables GitHub's diff format that shows improving queries in green, regressing queries in red).The
compare_benchmark.py(called bybenchmark_all.sh) uses a post-processing step in which the color coding is checked to add GitHub markdown accordingly. But when the script is called frombenchmark_all.sh, no color codes are emitted as the usedtermcolorpackage checks for a terminal that supports colors.This PR changes the
compare_benchmark.pyscript to force a colorful output in all situations.