Handle utf-16 encoding gracefully when querying Snowflake #38
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This stemmed from an issue we saw in production, a result we were
getting included some garbled characters because of a UTF-8 and UTF-16
difference.
We were getting the string:
Now, Â = U+00C2 and ® = U+00AE in UTF-8.
However, what we really wanted was:
As the  character is a mistaken extra byte.
Here, we introduce a function to properly handle this and return the
desired results using the latin1 encoding with UTF-8.
I have tried a couple ways of going about testing this, but the coverageso far does not allow me an easy way to even hit this branch of the
functions in result or snowflex itself. I will confer with someone else
to get a real test written.
JIRA ticket is
AUTO-4025
Here is validation run on this using just
iex
to demonstrate: