Skip to content

Commit

Permalink
Removes setup.py install target for pyspark connect as it is no longe…
Browse files Browse the repository at this point in the history
…r necessary due to fixed import
  • Loading branch information
elijahbenizzy committed Aug 24, 2023
1 parent 5a9a4f8 commit fd42544
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ def load_requirements():
"ray": ["ray>=2.0.0", "pyarrow"],
"pyspark": [
# we have to run these dependencies cause Spark does not check to ensure the right target was called
"pyspark[pandas_on_spark,connect,sql]",
"pyspark[pandas_on_spark,sql]",
# This is problematic, see https://stackoverflow.com/questions/76072664/convert-pyspark-dataframe-to-pandas-dataframe-fails-on-timestamp-column
"pandas<2.0",
], # I'm sure they'll add support soon,
Expand Down

0 comments on commit fd42544

Please sign in to comment.