Skip to content

Commit

Permalink
Refactor OrcReadBenchmark
Browse files Browse the repository at this point in the history
  • Loading branch information
@yucai
yucai committed Sep 28, 2018
1 parent 3b7395f commit 4b6ea87
Show file tree
Hide file tree
Showing 2 changed files with 227 additions and 157 deletions.
188 changes: 188 additions & 0 deletions sql/hive/benchmarks/OrcReadBenchmark-results.txt
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,188 @@
================================================================================================
SQL Single Column Scan
================================================================================================

Java HotSpot(TM) 64-Bit Server VM 1.8.0_162-b12 on Mac OS X 10.13.6
Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

SQL Single TINYINT Column Scan: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Native ORC MR 1192 / 1251 13.2 75.8 1.0X
Native ORC Vectorized 149 / 165 105.3 9.5 8.0X
Native ORC Vectorized with copy 146 / 156 107.7 9.3 8.2X
Hive built-in ORC 1419 / 1424 11.1 90.2 0.8X

Java HotSpot(TM) 64-Bit Server VM 1.8.0_162-b12 on Mac OS X 10.13.6
Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

SQL Single SMALLINT Column Scan: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Native ORC MR 1168 / 1176 13.5 74.2 1.0X
Native ORC Vectorized 151 / 156 104.2 9.6 7.7X
Native ORC Vectorized with copy 154 / 159 102.3 9.8 7.6X
Hive built-in ORC 1726 / 1742 9.1 109.7 0.7X

Java HotSpot(TM) 64-Bit Server VM 1.8.0_162-b12 on Mac OS X 10.13.6
Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

SQL Single INT Column Scan: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Native ORC MR 1203 / 1240 13.1 76.5 1.0X
Native ORC Vectorized 208 / 215 75.7 13.2 5.8X
Native ORC Vectorized with copy 212 / 220 74.3 13.5 5.7X
Hive built-in ORC 1782 / 1820 8.8 113.3 0.7X

Java HotSpot(TM) 64-Bit Server VM 1.8.0_162-b12 on Mac OS X 10.13.6
Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

SQL Single BIGINT Column Scan: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Native ORC MR 1226 / 1245 12.8 77.9 1.0X
Native ORC Vectorized 261 / 267 60.3 16.6 4.7X
Native ORC Vectorized with copy 263 / 269 59.8 16.7 4.7X
Hive built-in ORC 1804 / 1869 8.7 114.7 0.7X

Java HotSpot(TM) 64-Bit Server VM 1.8.0_162-b12 on Mac OS X 10.13.6
Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

SQL Single FLOAT Column Scan: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Native ORC MR 1279 / 1283 12.3 81.3 1.0X
Native ORC Vectorized 297 / 304 52.9 18.9 4.3X
Native ORC Vectorized with copy 297 / 303 53.0 18.9 4.3X
Hive built-in ORC 2086 / 2089 7.5 132.6 0.6X

Java HotSpot(TM) 64-Bit Server VM 1.8.0_162-b12 on Mac OS X 10.13.6
Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

SQL Single DOUBLE Column Scan: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Native ORC MR 1396 / 1506 11.3 88.7 1.0X
Native ORC Vectorized 361 / 375 43.5 23.0 3.9X
Native ORC Vectorized with copy 361 / 381 43.6 22.9 3.9X
Hive built-in ORC 2108 / 2136 7.5 134.0 0.7X


================================================================================================
Int and String Scan
================================================================================================

Java HotSpot(TM) 64-Bit Server VM 1.8.0_162-b12 on Mac OS X 10.13.6
Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

Int and String Scan: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Native ORC MR 3205 / 3216 3.3 305.7 1.0X
Native ORC Vectorized 1978 / 2216 5.3 188.6 1.6X
Native ORC Vectorized with copy 2067 / 2155 5.1 197.1 1.6X
Hive built-in ORC 4373 / 4413 2.4 417.1 0.7X


================================================================================================
Partitioned Table Scan
================================================================================================

Java HotSpot(TM) 64-Bit Server VM 1.8.0_162-b12 on Mac OS X 10.13.6
Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

Partitioned Table: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Data column - Native ORC MR 1462 / 1487 10.8 93.0 1.0X
Data column - Native ORC Vectorized 269 / 272 58.5 17.1 5.4X
Data column - Native ORC Vectorized with copy 274 / 318 57.4 17.4 5.3X
Data column - Hive built-in ORC 2127 / 2138 7.4 135.2 0.7X
Partition column - Native ORC MR 1092 / 1098 14.4 69.4 1.3X
Partition column - Native ORC Vectorized 52 / 57 304.6 3.3 28.3X
Partition column - Native ORC Vectorized with copy 51 / 58 305.6 3.3 28.4X
Partition column - Hive built-in ORC 1349 / 1353 11.7 85.8 1.1X
Both columns - Native ORC MR 1472 / 1515 10.7 93.6 1.0X
Both columns - Native ORC Vectorized 309 / 328 51.0 19.6 4.7X
Both column - Native ORC Vectorized with copy 356 / 370 44.2 22.6 4.1X
Both columns - Hive built-in ORC 2175 / 2184 7.2 138.3 0.7X


================================================================================================
Repeated String Scan
================================================================================================

Java HotSpot(TM) 64-Bit Server VM 1.8.0_162-b12 on Mac OS X 10.13.6
Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

Repeated String: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Native ORC MR 1313 / 1323 8.0 125.2 1.0X
Native ORC Vectorized 258 / 271 40.7 24.6 5.1X
Native ORC Vectorized with copy 397 / 406 26.4 37.9 3.3X
Hive built-in ORC 1983 / 2009 5.3 189.2 0.7X


================================================================================================
String with Nulls Scan
================================================================================================

Java HotSpot(TM) 64-Bit Server VM 1.8.0_162-b12 on Mac OS X 10.13.6
Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

String with Nulls Scan (0.0%): Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Native ORC MR 2492 / 2721 4.2 237.7 1.0X
Native ORC Vectorized 775 / 807 13.5 73.9 3.2X
Native ORC Vectorized with copy 1009 / 1105 10.4 96.3 2.5X
Hive built-in ORC 3890 / 4018 2.7 371.0 0.6X

Java HotSpot(TM) 64-Bit Server VM 1.8.0_162-b12 on Mac OS X 10.13.6
Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

String with Nulls Scan (0.5%): Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Native ORC MR 2278 / 2322 4.6 217.3 1.0X
Native ORC Vectorized 941 / 950 11.1 89.7 2.4X
Native ORC Vectorized with copy 1201 / 1211 8.7 114.6 1.9X
Hive built-in ORC 2903 / 2931 3.6 276.9 0.8X

Java HotSpot(TM) 64-Bit Server VM 1.8.0_162-b12 on Mac OS X 10.13.6
Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

String with Nulls Scan (0.95%): Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Native ORC MR 1237 / 1247 8.5 118.0 1.0X
Native ORC Vectorized 334 / 340 31.4 31.9 3.7X
Native ORC Vectorized with copy 412 / 415 25.5 39.3 3.0X
Hive built-in ORC 1656 / 1697 6.3 158.0 0.7X


================================================================================================
Single Column Scan From Wide Columns
================================================================================================

Java HotSpot(TM) 64-Bit Server VM 1.8.0_162-b12 on Mac OS X 10.13.6
Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

Single Column Scan from 100 columns: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Native ORC MR 1105 / 1111 0.9 1053.7 1.0X
Native ORC Vectorized 129 / 135 8.1 122.9 8.6X
Native ORC Vectorized with copy 126 / 140 8.3 120.5 8.7X
Hive built-in ORC 371 / 378 2.8 354.1 3.0X

Java HotSpot(TM) 64-Bit Server VM 1.8.0_162-b12 on Mac OS X 10.13.6
Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

Single Column Scan from 200 columns: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Native ORC MR 2150 / 2169 0.5 2049.9 1.0X
Native ORC Vectorized 231 / 238 4.5 219.8 9.3X
Native ORC Vectorized with copy 225 / 239 4.7 214.9 9.5X
Hive built-in ORC 578 / 588 1.8 551.0 3.7X

Java HotSpot(TM) 64-Bit Server VM 1.8.0_162-b12 on Mac OS X 10.13.6
Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

Single Column Scan from 300 columns: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Native ORC MR 3379 / 3414 0.3 3222.6 1.0X
Native ORC Vectorized 375 / 395 2.8 358.1 9.0X
Native ORC Vectorized with copy 390 / 407 2.7 372.1 8.7X
Hive built-in ORC 838 / 846 1.3 799.1 4.0X


Loading

0 comments on commit 4b6ea87

Please sign in to comment.