Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

integration tests assume incompat config is false, we should verify this #4893

Closed
tgravescs opened this issue Mar 2, 2022 · 0 comments · Fixed by #4917
Closed

integration tests assume incompat config is false, we should verify this #4893

tgravescs opened this issue Mar 2, 2022 · 0 comments · Fixed by #4917
Assignees
Labels
test Only impacts tests

Comments

@tgravescs
Copy link
Collaborator

Describe the bug
I was seeing test failures in one of our integration test environment where we were running on an actual cluster. Specifically with test test_hash_groupby_typed_imperative_agg_without_gpu_implementation_fallback. This test assumes that the spark.rapids.sql.incompatibleOps.enabled=false. In this test environment that flag had been turned to true in the global spark configs which caused the test to fail. In this case it wasn't obvious to me that this test relied on that behavior.

If the tests are assuming this is false we should have a global check for it and fail with useful message if it finds its not set that way.

Note that we have a check https://github.com/NVIDIA/spark-rapids/blob/branch-22.04/integration_tests/src/main/python/asserts.py#L219 that doesn't like setting the conf manually.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
test Only impacts tests
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants