[SPARK-8138] [SQL] Improves error message when conflicting partition columns are found

[liancheng]

This PR improves the error message shown when conflicting partition column names are detected. This can be particularly annoying and confusing when there are a large number of partitions while a handful of them happened to contain unexpected temporary file(s). Now all suspicious directories are listed as below:

java.lang.AssertionError: assertion failed: Conflicting partition column names detected:

        Partition column name list #0: b, c, d
        Partition column name list #1: b, c
        Partition column name list #2: b

For partitioned table directories, data files should only live in leaf directories. Please check the following directories for unexpected files:

        file:/tmp/foo/b=0
        file:/tmp/foo/b=1
        file:/tmp/foo/b=1/c=1
        file:/tmp/foo/b=0/c=0

[liancheng]

cc @rxin

[SparkQA]

Test build #34081 has finished for PR 6610 at commit c77fa4e.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • trait TypeCheckResult
  • case class TypeCheckFailure(message: String) extends TypeCheckResult
  • abstract class UnaryArithmetic extends UnaryExpression
  • case class UnaryMinus(child: Expression) extends UnaryArithmetic
  • case class Sqrt(child: Expression) extends UnaryArithmetic
  • case class Abs(child: Expression) extends UnaryArithmetic
  • case class BitwiseNot(child: Expression) extends UnaryArithmetic
  • case class MaxOf(left: Expression, right: Expression) extends BinaryArithmetic
  • case class MinOf(left: Expression, right: Expression) extends BinaryArithmetic
  • case class Atan2(left: Expression, right: Expression)
  • case class Hypot(left: Expression, right: Expression)
  • case class EqualTo(left: Expression, right: Expression) extends BinaryComparison

[yhuai]

Let's create a jira for this.

[liancheng]

Filed SPARK-8138 for this and updated PR title.

[liancheng]

@yhuai As discussed offline, now we give a more descriptive and help message with a list of all suspicious non-leaf partition directories.

[SparkQA]

Test build #34389 has finished for PR 6610 at commit afbdb14.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[rxin]

move this out, make it accept arguments in the form of collections, and write unit test for this function.

this function is way too complicated to not have unit tests.

[liancheng]

retest this please

[SparkQA]

Test build #35651 has finished for PR 6610 at commit a149250.

• This patch fails Scala style tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #35656 has finished for PR 6610 at commit 7d05f2c.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[liancheng]

Merging to master.