You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
When I try to load the data via read_parquet_table I get an error that magic byte is missing.
ArrowInvalid: Parquet magic bytes not found in footer. Either the file is corrupted or this is not a parquet file.
The table was created by pyspark write.saveAsTable and can be read by pyspark or athena.
maybe wrangler tries to read non parquet files like SUCCESS etc (see below).
when I read with the same table with read_parquet: awswrangler.s3.read_parquet(path='path', path_suffix='.parquet') it works but when I omit the path_suffix I get the same error
Describe the bug
When I try to load the data via read_parquet_table I get an error that magic byte is missing.
ArrowInvalid: Parquet magic bytes not found in footer. Either the file is corrupted or this is not a parquet file.
The table was created by pyspark write.saveAsTable and can be read by pyspark or athena.
maybe wrangler tries to read non parquet files like SUCCESS etc (see below).
when I read with the same table with read_parquet:
awswrangler.s3.read_parquet(path='path', path_suffix='.parquet')
it works but when I omit thepath_suffix
I get the same errorTo Reproduce
private data cannot share.
The text was updated successfully, but these errors were encountered: