-
Notifications
You must be signed in to change notification settings - Fork 3
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
This PR fixes the array/slice handling in the Parquet filetype by slightly updating the Parquet schema definition. It's possible to represent UUID types with the proper logical type but it just didn't work with Athena. Also had to do some weird things like defining the uuid as **22-byte** FIXED_LEN_BYTE_ARRAY (UUIDs are 16 bytes) but giving it a 16-byte Go slice (not a Go array) and Base64 decoders in reverse transformers etc... And all this didn't even work when it came to `UUIDArray`s... So I gave up on that for now. JSON logicaltype seems to somewhat work with DuckDB but doesn't with Athena so I removed that as well.
- Loading branch information
Showing
4 changed files
with
70 additions
and
41 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters