Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] get_json_test.py::test_get_json_object_quoted_question fails on Spark 4 with mismatched output #11130

Open
mythrocks opened this issue Jul 3, 2024 · 0 comments
Assignees
Labels
bug Something isn't working Spark 4.0+ Spark 4.0+ issues

Comments

@mythrocks
Copy link
Collaborator

get_json_test.py::test_get_json_object_quoted_question fails on Spark 4 with mismatched output:

---------------------------- Captured stderr setup -----------------------------
2024-07-02 23:57:30 INFO     Running test 'src/main/python/get_json_test.py::test_get_json_object_quoted_question[DATAGEN_SEED=1719964646, TZ=UTC]'
------------------------------ Captured log setup ------------------------------
INFO     __pytest_worker_logger__:spark_init_internal.py:256 Running test 'src/main/python/get_json_test.py::test_get_json_object_quoted_question[DATAGEN_SEED=1719964646, TZ=UTC]'
----------------------------- Captured stdout call -----------------------------
### CPU RUN ###
### GPU RUN ###
### COLLECT: GPU TOOK 0.5124051570892334 CPU TOOK 1.7997944355010986 ###
--- CPU OUTPUT
+++ GPU OUTPUT
@@ -1 +1 @@
-Row(question='QUESTION')
+Row(question=None)

Seems to work alright on prior Spark versions. We'll need this functionality sorted out before we get a clean run in Spark 4.

I'm disabling this test as part of #11129.

@mythrocks mythrocks added bug Something isn't working ? - Needs Triage Need team to review and classify labels Jul 3, 2024
@mattahrens mattahrens added Spark 4.0+ Spark 4.0+ issues and removed ? - Needs Triage Need team to review and classify labels Jul 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Spark 4.0+ Spark 4.0+ issues
Projects
None yet
Development

No branches or pull requests

3 participants