Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix tests failures in sort_test.py #11027

Closed
Tracked by #11004
razajafri opened this issue Jun 8, 2024 · 1 comment · Fixed by #11099
Closed
Tracked by #11004

Fix tests failures in sort_test.py #11027

razajafri opened this issue Jun 8, 2024 · 1 comment · Fixed by #11099
Assignees
@mythrocks
Labels
bug Something isn't working Spark 4.0+ Spark 4.0+ issues

Comments

@razajafri
Copy link
Collaborator 

FAILED ../../../../integration_tests/src/main/python/sort_test.py::test_orderby_with_processing
FAILED ../../../../integration_tests/src/main/python/sort_test.py::test_orderby_with_processing_and_limit
FAILED ../../../../integration_tests/src/main/python/sort_test.py::test_single_nested_orderby_with_processing_and_limit
@razajafri razajafri added bug Something isn't working ? - Needs Triage Need team to review and classify labels Jun 8, 2024
@razajafri razajafri mentioned this issue Jun 8, 2024
Open
@razajafri razajafri added the Spark 4.0+ Spark 4.0+ issues label Jun 8, 2024
@mattahrens mattahrens removed the ? - Needs Triage Need team to review and classify label Jun 11, 2024
@mythrocks mythrocks self-assigned this Jun 12, 2024
@mythrocks
Copy link
Collaborator

These tests succeed with ANSI mode disabled.

mythrocks added a commit to mythrocks/spark-rapids that referenced this issue Jun 26, 2024
@mythrocks
Spark 4: Handle ANSI mode in sort_test.py
d35d9f0 
Fixes NVIDIA#11027.

With ANSI mode enabled (like the default in Spark 4), one sees that some
tests in `sort_test.py` fail, because they expect ANSI mode to be off.

This commit disables running those tests with ANSI enabled, and add a
separate test for ANSI on/off.

Signed-off-by: MithunR <mithunr@nvidia.com>
@mythrocks mythrocks mentioned this issue Jun 26, 2024
Merged
wjxiz1992 added a commit to nvliyuan/yuali-spark-rapids that referenced this issue Jul 5, 2024
@wjxiz1992 @xieshuaihu
@mythrocks @liurenjie1024 @firestarman
Max min by (#43)
66cf815 
* Fix match error in RapidsShuffleIterator.scala [scala2.13] (NVIDIA#11115)

Signed-off-by: xieshuaihu <xieshuaihu@gmail.com>

* Spark 4: Handle ANSI mode in sort_test.py (NVIDIA#11099)

* Spark 4: Handle ANSI mode in sort_test.py

Fixes NVIDIA#11027.

With ANSI mode enabled (like the default in Spark 4), one sees that some
tests in `sort_test.py` fail, because they expect ANSI mode to be off.

This commit disables running those tests with ANSI enabled, and add a
separate test for ANSI on/off.

Signed-off-by: MithunR <mithunr@nvidia.com>

* Refactored not to use disable_ansi_mode.

These tests need not be revisited.  They test all combinations of ANSI mode,
including overflow failures.

Signed-off-by: MithunR <mithunr@nvidia.com>

---------

Signed-off-by: MithunR <mithunr@nvidia.com>

* Introduce LORE framework. (NVIDIA#11084)

* Introduce lore id

* Introduce lore id

* Fix type

* Fix type

* Conf

* style

* part

* Dump

* Introduce lore framework

* Add tests.

* Rename test case

Signed-off-by: liurenjie1024 <liurenjie2008@gmail.com>

* Fix AQE test

* Fix style

* Use args to display lore info.

* Fix build break

* Fix path in loreinfo

* Remove path

* Fix comments

* Update configs

* Fix comments

* Fix config

---------

Signed-off-by: liurenjie1024 <liurenjie2008@gmail.com>

* Support minBy on GPU

Signed-off-by: Firestarman <firestarmanllc@gmail.com>

---------

Signed-off-by: xieshuaihu <xieshuaihu@gmail.com>
Signed-off-by: MithunR <mithunr@nvidia.com>
Signed-off-by: liurenjie1024 <liurenjie2008@gmail.com>
Signed-off-by: Firestarman <firestarmanllc@gmail.com>
Co-authored-by: xieshuaihu <xieshuaihu@gmail.com>
Co-authored-by: MithunR <mithunr@nvidia.com>
Co-authored-by: Renjie Liu <liurenjie2008@gmail.com>
Co-authored-by: Firestarman <firestarmanllc@gmail.com>
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mythrocks mythrocks

Labels
bug Something isn't working Spark 4.0+ Spark 4.0+ issues
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Spark 4: Handle ANSI mode in sort_test.py mythrocks/spark-rapids
3 participants
@mattahrens @mythrocks @razajafri