[SPARK-5441][pyspark] Make SerDeUtil PairRDD to Python conversions more robust

[ghost]

SerDeUtil.pairRDDToPython and SerDeUtil.pythonToPairRDD now both support empty RDDs by checking the result of take(1) instead of calling first which throws an exception.

[AmplabJenkins]

Can one of the admins verify this patch?

[pwendell]

Hey thanks for this - mind adding a regression test that fails on the old code?

[ghost]

I've added two regression tests which I made sure failed beforehand and succeed now.

[JoshRosen]

Jenkins, this is ok to test.

[SparkQA]

Test build #26243 has started for PR 4236 at commit a531c0c.

• This patch merges cleanly.

[SparkQA]

Test build #26243 has finished for PR 4236 at commit a531c0c.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26243/
Test PASSed.

[JoshRosen]

Thanks for adding tests. This looks good to me, so I'm going to merge it into master (1.3.0) and mark it for later backport into branch-1.2 (I'd commit it now, but we're in the middle of the 1.2.1 voting period right now, so we've placed a hold on merging in that branch until the vote passes).

[JoshRosen]

Minor nit: we usually place the Spark imports in their own section, separate from third-party library imports like Scalatest. I'll just fix this myself on merge, but I thought I'd mention it for future patches.

[JoshRosen]

I've cherry-picked this to branch-1.2 (1.2.2).