[SPARK-44907][PYTHON][CONNECT] DataFrame.join` should throw IllegalAr…

…gumentException for invalid join types

### What changes were proposed in this pull request?
`DataFrame.join` should throw IllegalArgumentException for invalid join types

### Why are the changes needed?
all valid join types have already been supported, for an unknown one, should throw `IllegalArgumentException` now

### Does this PR introduce _any_ user-facing change?
yes

### How was this patch tested?
enabled UT

### Was this patch authored or co-authored using generative AI tooling?
NO

Closes apache#42603 from zhengruifeng/test_df_join_type.

Authored-by: Ruifeng Zheng <ruifengz@apache.org>
Signed-off-by: Ruifeng Zheng <ruifengz@apache.org>

python/pyspark/sql/connect/plan.py

@@ -41,7 +41,12 @@
     LiteralExpression,
 )
 from pyspark.sql.connect.types import pyspark_types_to_proto_types, UnparsedDataType
-from pyspark.errors import PySparkTypeError, PySparkNotImplementedError, PySparkRuntimeError
+from pyspark.errors import (
+    PySparkTypeError,
+    PySparkNotImplementedError,
+    PySparkRuntimeError,
+    IllegalArgumentException,
+)
 
 if TYPE_CHECKING:
     from pyspark.sql.connect._typing import ColumnOrName
@@ -854,7 +859,7 @@ def __init__(
         elif how == "cross":
             join_type = proto.Join.JoinType.JOIN_TYPE_CROSS
         else:
-            raise PySparkNotImplementedError(
+            raise IllegalArgumentException(
                 error_class="UNSUPPORTED_JOIN_TYPE",
                 message_parameters={"join_type": how},
             )

python/pyspark/sql/tests/connect/test_parity_dataframe.py

@@ -26,11 +26,6 @@ class DataFrameParityTests(DataFrameTestsMixin, ReusedConnectTestCase):
     def test_help_command(self):
         super().test_help_command()
 
-    # Spark Connect throws NotImplementedError tests expects IllegalArgumentException
-    @unittest.skip("Fails in Spark Connect, should enable.")
-    def test_invalid_join_method(self):
-        super().test_invalid_join_method()
-
     # TODO(SPARK-41527): Implement DataFrame.observe
     @unittest.skip("Fails in Spark Connect, should enable.")
     def test_observe(self):