Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More compilation fixes for Spark 4.0.0 [databricks] #10978

Merged
merged 7 commits into from
Jun 5, 2024

Conversation

razajafri
Copy link
Collaborator

@razajafri razajafri commented Jun 4, 2024

This PR solves the last bit of the compilation errors, when building for Spark 4.0.

fixes #10972

razajafri added 2 commits June 4, 2024 14:30
Signed-off-by: Raza Jafri <rjafri@nvidia.com>
mythrocks
mythrocks previously approved these changes Jun 4, 2024
Copy link
Collaborator

@mythrocks mythrocks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, so far.

Here's the latest build break:

[INFO] Compiling 421 Scala sources and 59 Java sources to /home/runner/work/spark-rapids/spark-rapids/sql-plugin/target/spark312/classes ...
Error: ] /home/runner/work/spark-rapids/spark-rapids/sql-plugin/src/main/java/com/nvidia/spark/rapids/CudfUnsafeRow.java:31: cannot find symbol
  symbol:   class VariantVal
  location: package org.apache.spark.unsafe.types
Error: ] /home/runner/work/spark-rapids/spark-rapids/sql-plugin/src/main/java/com/nvidia/spark/rapids/CudfUnsafeRow.java:404: cannot find symbol
  symbol:   class VariantVal
  location: class com.nvidia.spark.rapids.CudfUnsafeRow
Error: ] /home/runner/work/spark-rapids/spark-rapids/sql-plugin/src/main/java/com/nvidia/spark/rapids/CudfUnsafeRow.java:403: method does not override or implement a method from a supertype

@mythrocks mythrocks self-requested a review June 4, 2024 22:37
@razajafri
Copy link
Collaborator Author

Sorry, this should be shimmed!

@razajafri razajafri dismissed mythrocks’s stale review June 4, 2024 22:42

This should be shimmed

This reverts commit dc4823b.
@razajafri
Copy link
Collaborator Author

build

@mythrocks
Copy link
Collaborator

Reviewing the latest version now.

Copy link
Collaborator

@mythrocks mythrocks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, that looks better. We use the trait for the CudfUnsafeRow object, and Base for the class. That works for me.

@razajafri razajafri merged commit 4d3b346 into NVIDIA:branch-24.08 Jun 5, 2024
44 checks passed
@razajafri razajafri deleted the SP-9259-compile-failures branch June 5, 2024 08:09
@sameerz sameerz added the Spark 4.0+ Spark 4.0+ issues label Jun 5, 2024
SurajAralihalli pushed a commit to SurajAralihalli/spark-rapids that referenced this pull request Jul 12, 2024
* More compilation fixes

* Signing off

Signed-off-by: Raza Jafri <rjafri@nvidia.com>

* Added shim for UnsafeCudfRow

* POM changes

Signed-off-by: Raza Jafri <rjafri@nvidia.com>

* Added shim for UnsafeCudfRow

* Revert "POM changes"

This reverts commit dc4823b.

---------

Signed-off-by: Raza Jafri <rjafri@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Spark 4.0+ Spark 4.0+ issues
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Spark 4.0 compile errors
3 participants