Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] GPU CSV read does not honor ignoreCorruptFiles or ignoreMissingFiles #4802

Closed
jlowe opened this issue Feb 16, 2022 · 0 comments · Fixed by #4809
Closed

[BUG] GPU CSV read does not honor ignoreCorruptFiles or ignoreMissingFiles #4802

jlowe opened this issue Feb 16, 2022 · 0 comments · Fixed by #4809
Assignees
@wbo4958
Labels
bug Something isn't working

Comments

@jlowe
Copy link
Contributor

jlowe commented Feb 16, 2022

Relates to #4727. If the GPU performs a CSV read operation, it should honor the spark.sql.files.ignoreCorruptFiles and spark.sql.files.ignoreMissingFiles settings.
@jlowe jlowe added bug Something isn't working ? - Needs Triage Need team to review and classify labels Feb 16, 2022
@wbo4958 wbo4958 self-assigned this Feb 17, 2022
@wbo4958 wbo4958 mentioned this issue Feb 17, 2022
Merged
@sameerz sameerz removed the ? - Needs Triage Need team to review and classify label Feb 21, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@wbo4958 wbo4958

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add ignoreCorruptFiles for ORC readers [databricks] wbo4958/spark-rapids
3 participants
@wbo4958 @jlowe @sameerz