Skip to content

Commit

Permalink
[SPARK-20735][SQL][TEST] Enable cross join in TPCDSQueryBenchmark
Browse files Browse the repository at this point in the history
## What changes were proposed in this pull request?

Since [SPARK-17298](https://issues.apache.org/jira/browse/SPARK-17298), some queries (q28, q61, q77, q88, q90) in the test suites fail with a message "_Use the CROSS JOIN syntax to allow cartesian products between these relations_".

This benchmark is used as a reference model for Spark TPC-DS, so this PR aims to enable the correct configuration in `TPCDSQueryBenchmark.scala`.

## How was this patch tested?

Manual. (Run TPCDSQueryBenchmark)

Author: Dongjoon Hyun <dongjoon@apache.org>

Closes #17977 from dongjoon-hyun/SPARK-20735.
  • Loading branch information
dongjoon-hyun authored and gatorsmile committed May 15, 2017
1 parent 499ba2c commit bbd163d
Showing 1 changed file with 1 addition and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ object TPCDSQueryBenchmark {
.set("spark.driver.memory", "3g")
.set("spark.executor.memory", "3g")
.set("spark.sql.autoBroadcastJoinThreshold", (20 * 1024 * 1024).toString)
.set("spark.sql.crossJoin.enabled", "true")

val spark = SparkSession.builder.config(conf).getOrCreate()

Expand Down

0 comments on commit bbd163d

Please sign in to comment.