Error parsing nested struct types when using s3.to_parquet

[nivf33]

Hi,

I'm using awswrangler 1.10.0
I've encountered with the following issue when using s3.to_parquet.
I'm trying to append a row as DataFrame to Athena, which one of the fields is of the following type:
{'Name': 'test', 'Type': 'array<struct<a:struct<id:string,name:string>,b:struct<id:string,name:string>>>'}

I'm getting the following error:
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/awswrangler/_data_types.py", line 58, in athena2pyarrow raise exceptions.UnsupportedType(f"Unsupported Athena type: {dtype}") awswrangler.exceptions.UnsupportedType: Unsupported Athena type: strin

I've debugged the code and the issue seems to be from the _data_types.py line 55.
This code is extracting each iteration another layer of the wrapping elements, but it doesn't support nested ones, this is why it gets an error of "strin" after splitting by ',' and the whole structure is ruined.

The issue seems to happen when the table already exists and I'm trying to append more data do it.

[igorborgest]

Hi @nivf33, thanks for reporting it.

It already was fixed in our development branch (PR above 👆 ). Could you give it a try?

pip install git+https://github.com/awslabs/aws-data-wrangler.git@feature/nested-struct

[igorborgest]

Released on version 2.1.0

[nivf33]

Hi Igor, sorry for the delay
I'm trying to run the command you've mentioned but I'm getting the following:
WARNING: Did not find branch or tag 'feature/nested-struct', assuming revision or ref.

am I missing anything?
Thanks

[igorborgest]

Oh sorry, the development branch already was deleted. Now you can use it directly from our official release into version 2.1.0

[nivf33]

great it works, thanks!