Skip to content

Commit

Permalink
[SPARK-43663][CONNECT][PS] Enable SeriesParityTests.test_compare
Browse files Browse the repository at this point in the history
### What changes were proposed in this pull request?

This PR proposes to enable `SeriesParityTests.test_compare` for Spark Connect.

### Why are the changes needed?

We should support the existing pandas-on-Spark APIs for Spark Connect as much as possible.

### Does this PR introduce _any_ user-facing change?

Yes, `Series.compare` is now available on Spark Connect.

### How was this patch tested?

Reusing the existing UT.

Closes #41567 from itholic/SPARK-43633.

Authored-by: itholic <haejoon.lee@databricks.com>
Signed-off-by: Ruifeng Zheng <ruifengz@apache.org>
  • Loading branch information
itholic authored and zhengruifeng committed Jun 14, 2023
1 parent b174aaa commit 75291d6
Show file tree
Hide file tree
Showing 2 changed files with 1 addition and 5 deletions.
2 changes: 1 addition & 1 deletion python/pyspark/pandas/frame.py
Original file line number Diff line number Diff line change
Expand Up @@ -4020,7 +4020,7 @@ def empty(self) -> bool:
"""
return (
len(self._internal.column_labels) == 0
or self._internal.resolved_copy.spark_frame.rdd.isEmpty()
or self._internal.resolved_copy.spark_frame.isEmpty()
)

@property
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,6 @@


class SeriesParityComputeTests(SeriesComputeMixin, PandasOnSparkTestUtils, ReusedConnectTestCase):
@unittest.skip("TODO(SPARK-43663): Enable SeriesParityTests.test_compare.")
def test_compare(self):
super().test_compare()

@unittest.skip(
"TODO(SPARK-43611): Fix unexpected `AnalysisException` from Spark Connect client."
)
Expand Down

0 comments on commit 75291d6

Please sign in to comment.