Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix Miscellaneous Expression tests on Databricks 14.3 #11537

Closed
razajafri opened this issue Sep 27, 2024 · 0 comments · Fixed by #11670
Closed

Fix Miscellaneous Expression tests on Databricks 14.3 #11537

razajafri opened this issue Sep 27, 2024 · 0 comments · Fixed by #11670
Assignees
@mythrocks

Comments

@razajafri
Copy link
Collaborator

Build the plugin against the Databricks 14.3 cluster using #11467. Once built successfully run the Misc Expression tests by TESTS=misc_expr_test.py jenkins/databricks/test.sh

The following tests fail

[gw4] [ 61%] FAILED ../../src/main/python/misc_expr_test.py::test_raise_error
mythrocks added a commit to mythrocks/spark-rapids that referenced this issue Oct 28, 2024
@mythrocks
Fix misc_expr_test for [databricks] 14.3
40639b0 
Fixes NVIDIA#11537.

This commit addresses the failure of the `test_raise_error` test
in `misc_expr_test.py` for Databricks 14.3.

This is an extension of NVIDIA#11129, where this test was skipped for
Apache Spark 4.0.  The failure on Databricks 14.3 shares the same
cause as in Spark 4.0, i.e. a backward-incompatible Spark change
in the signature of RaiseError, as introduced in
https://issues.apache.org/jira/browse/SPARK-44838.

The work to support this change in a Spark-RAPIDS shim will be
tracked in NVIDIA#10969.  This test will be skipped until that work
is completed.

Signed-off-by: MithunR <mithunr@nvidia.com>
@mythrocks mythrocks mentioned this issue Oct 28, 2024
Merged
@mythrocks mythrocks self-assigned this Oct 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mythrocks mythrocks

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix misc_expr_test for [databricks] 14.3 mythrocks/spark-rapids
2 participants
@mythrocks @razajafri