Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEA] Scala 2.13 build should emit an error message when trying to build with unsupported shim #9563

Closed
NVnavkumar opened this issue Oct 27, 2023 · 0 comments · Fixed by #9568
Closed

[FEA] Scala 2.13 build should emit an error message when trying to build with unsupported shim #9563

NVnavkumar opened this issue Oct 27, 2023 · 0 comments · Fixed by #9568
Assignees
@gerashegalov
Labels
build Related to CI / CD or cleanly building Scala 2.13 Issues related to Scala 2.13 binaries task Work required that improves the product but is not user facing tech debt

Comments

@NVnavkumar
Copy link
Collaborator

Now that #8592 is merged, we can build the repository with Scala 2.13. However, we only support Apache Spark 3.3.0 and higher (not Cloudera or Databricks or older versions). We should emit a clear error message when trying to build with an unsupported version, when someone specifies this through -Dbuildver:

cd scala2.13
mvn -Dbuildver=311 verify

Should emit an error:

Unsupported Spark version: $buildver. Spark RAPIDS with Scala 2.13 only supports Apache Spark 3.3.0 and higher.

Same error should show up if we try to use a Databricks related profile:

mvn -PnoSnapshotsWithDatabricks ...

I don't think we should remove these profiles in Scala 2.13 since Spark 4.0.0 will switch to Scala 2.13 and ultimately Databricks and Cloudera will pick that up.
@NVnavkumar NVnavkumar added ? - Needs Triage Need team to review and classify build Related to CI / CD or cleanly building task Work required that improves the product but is not user facing tech debt labels Oct 27, 2023
gerashegalov added a commit to gerashegalov/spark-rapids that referenced this issue Oct 28, 2023
@gerashegalov
Enforce Apache 3.3.0+ for Scala 2.13
523dc76 
Fixes NVIDIA#9563

Signed-off-by: Gera Shegalov <gera@apache.org>
@gerashegalov gerashegalov mentioned this issue Oct 28, 2023
Merged
@gerashegalov gerashegalov self-assigned this Oct 28, 2023
@gerashegalov gerashegalov removed the ? - Needs Triage Need team to review and classify label Oct 28, 2023
jlowe pushed a commit that referenced this issue Oct 30, 2023
@gerashegalov
Enforce Apache 3.3.0+ for Scala 2.13 (#9568)
2cc202a 
* Enforce Apache 3.3.0+ for Scala 2.13

Fixes #9563

Signed-off-by: Gera Shegalov <gera@apache.org>

* Fix comment

Signed-off-by: Gera Shegalov <gera@apache.org>

---------

Signed-off-by: Gera Shegalov <gera@apache.org>
@gerashegalov gerashegalov added the Scala 2.13 Issues related to Scala 2.13 binaries label Oct 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@gerashegalov gerashegalov

Labels
build Related to CI / CD or cleanly building Scala 2.13 Issues related to Scala 2.13 binaries task Work required that improves the product but is not user facing tech debt
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Enforce Apache 3.3.0+ for Scala 2.13 gerashegalov/spark-rapids
2 participants
@gerashegalov @NVnavkumar