Skip to content

Commit

Permalink
[SPARK-45121][CONNECT][PS] Support Series.empty for Spark Connect
Browse files Browse the repository at this point in the history
### What changes were proposed in this pull request?

This PR proposes to support Series.empty for Spark Connect by removing JVM dependency.

### Why are the changes needed?

Increase API coverage for Spark Connect.

### Does this PR introduce _any_ user-facing change?

`Series.empty` is available on Spark Connect.

### How was this patch tested?

Added UT.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes apache#42877 from itholic/SPARK-45121.

Authored-by: Haejoon Lee <haejoon.lee@databricks.com>
Signed-off-by: Ruifeng Zheng <ruifengz@apache.org>
  • Loading branch information
itholic authored and zhengruifeng committed Sep 12, 2023
1 parent 3148511 commit 5d2d915
Show file tree
Hide file tree
Showing 2 changed files with 3 additions and 1 deletion.
2 changes: 1 addition & 1 deletion python/pyspark/pandas/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -505,7 +505,7 @@ def empty(self) -> bool:
>>> ps.DataFrame({}, index=list('abc')).index.empty
False
"""
return self._internal.resolved_copy.spark_frame.rdd.isEmpty()
return self._internal.resolved_copy.spark_frame.isEmpty()

@property
def hasnans(self) -> bool:
Expand Down
2 changes: 2 additions & 0 deletions python/pyspark/pandas/tests/series/test_series.py
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,8 @@ def test_empty_series(self):
self.assert_eq(ps.from_pandas(pser_a), pser_a)
self.assert_eq(ps.from_pandas(pser_b), pser_b)

self.assertTrue(pser_a.empty)

def test_all_null_series(self):
pser_a = pd.Series([None, None, None], dtype="float64")
pser_b = pd.Series([None, None, None], dtype="str")
Expand Down

0 comments on commit 5d2d915

Please sign in to comment.