-
I am builidng a custom spark docker image to use spark connect in kubernetes. the base image of my docker image is I found this #3704 But looks like the config The docker file for reference
Will be helpful if someone explains what is going wrong here. |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments 9 replies
-
we haven't tested with Spark connect but we have seen some a similar issue with our jar that has multiple shims have classloader issues. I would suggest rebuilding spark-rapids for just the single spark release (3.5.0) that you are using to see if it fixes this issue: Please let us know if this works for you. |
Beta Was this translation helpful? Give feedback.
-
Related to #9375 |
Beta Was this translation helpful? Give feedback.
we haven't tested with Spark connect but we have seen some a similar issue with our jar that has multiple shims have classloader issues.
I would suggest rebuilding spark-rapids for just the single spark release (3.5.0) that you are using to see if it fixes this issue:
https://github.com/NVIDIA/spark-rapids/blob/branch-24.02/CONTRIBUTING.md#building-a-distribution-for-a-single-spark-release
Please let us know if this works for you.