Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-25935][SQL] Allow null rows for bad records from JSON parsers #23543

Closed
wants to merge 1 commit into from

Conversation

MaxGekk
Copy link
Member

@MaxGekk MaxGekk commented Jan 14, 2019

What changes were proposed in this pull request?

This PR reverts #22938 per discussion in #23325

Closes #23325

@MaxGekk
Copy link
Member Author

MaxGekk commented Jan 14, 2019

@cloud-fan @HyukjinKwon @attilapiros Please, take a look at the PR.

@MaxGekk MaxGekk changed the title [SPARK-25935][SQL] Return null rows from JSON parser [SPARK-25935][SQL] Allow null rows for bad records from JSON/CSV parsers Jan 14, 2019
@MaxGekk MaxGekk force-pushed the return-nulls-from-json-parser branch from 6574db0 to 69a4284 Compare January 15, 2019 00:00
@HyukjinKwon
Copy link
Member

Yup, I agree with it for now.

@SparkQA
Copy link

SparkQA commented Jan 15, 2019

Test build #101209 has finished for PR 23543 at commit 6574db0.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Jan 15, 2019

Test build #101210 has finished for PR 23543 at commit 69a4284.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@cloud-fan
Copy link
Contributor

thanks, merging to master!

@cloud-fan cloud-fan closed this in 33b5039 Jan 15, 2019
@bersprockets
Copy link
Contributor

Should the PR title be "Allow null rows for bad records from JSON parsers"?

@cloud-fan cloud-fan changed the title [SPARK-25935][SQL] Allow null rows for bad records from JSON/CSV parsers [SPARK-25935][SQL] Allow null rows for bad records from JSON parsers Jan 15, 2019
@cloud-fan
Copy link
Contributor

good catch! I've updated the title

jackylee-ch pushed a commit to jackylee-ch/spark that referenced this pull request Feb 18, 2019
## What changes were proposed in this pull request?

This PR reverts  apache#22938 per discussion in apache#23325

Closes apache#23325

Closes apache#23543 from MaxGekk/return-nulls-from-json-parser.

Authored-by: Maxim Gekk <max.gekk@gmail.com>
Signed-off-by: Wenchen Fan <wenchen@databricks.com>
@MaxGekk MaxGekk deleted the return-nulls-from-json-parser branch August 17, 2019 13:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants