-
Notifications
You must be signed in to change notification settings - Fork 932
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[FEA] Support casting of LIST type to STRING in JSON #15278
Comments
Hello @revans2, would you please share a bit about the case where a list type is coerced to string? I understand the map type case, but why would you want this for the top-level list rather than only for its child values? |
@GregoryKimball Spark allows anything to be coerced into a string, if the user asks for it. Also Spark has requirements to treat quoted and unquoted strings differently for essentially all data types, so the schema we pass to the JSON parser has all strings at the leaf nodes. So this can end up being a very common problem, and we have no way to detect it and fall back to the CPU because the issue is in the data, not in anything we have access to at planning time. |
Is your feature request related to a problem? Please describe.
This is a follow on issue for #14936 (comment)
Really we just want to be able to request that a column or child column, be returned as a string and it works, even if the data is a nested type.
The text was updated successfully, but these errors were encountered: