Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove tests specific to Spark 3.1.x #11160

Open
jlowe opened this issue Jul 9, 2024 · 0 comments · May be fixed by #11316
Open

Remove tests specific to Spark 3.1.x #11160

jlowe opened this issue Jul 9, 2024 · 0 comments · May be fixed by #11316
Assignees
@sameerz
Labels
good first issue Good for newcomers tech debt test Only impacts tests

Comments

@jlowe
Copy link
Member

jlowe commented Jul 9, 2024

After #11159 Spark 3.1.x is no longer supported, but there are tests remaining that are specific to Spark 3.1.x. These tests should be removed. One way of identifying these tests is to check what is calling spark_version() and checking for 3.1.x or < 3.2.0. In addition the following predicate functions in spark_session.py should be removed:

  • is_before_spark_312
  • is_before_spark_313
  • is_before_spark_314
  • is_before_spark_320
@jlowe jlowe added ? - Needs Triage Need team to review and classify test Only impacts tests tech debt good first issue Good for newcomers labels Jul 9, 2024
@mattahrens mattahrens removed the ? - Needs Triage Need team to review and classify label Jul 9, 2024
@jlowe jlowe mentioned this issue Jul 11, 2024
Merged
@NvTimLiu NvTimLiu mentioned this issue Aug 5, 2024
Open
@sameerz sameerz linked a pull request Aug 12, 2024 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sameerz sameerz

Labels
good first issue Good for newcomers tech debt test Only impacts tests
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Remove test checks for Spark versions before 3.2.0 sameerz/spark-rapids
3 participants
@jlowe @mattahrens @sameerz