-
Notifications
You must be signed in to change notification settings - Fork 697
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Question regarding wr.athena.to_iceberg #2782
Comments
The temporary table is created/deleted from the same database. As the temp table has a random name, you must have Lake Formation permissions to create, describe and delete any table in that database |
@jaidisido Thanks! I finally got what the problem was thanks to your answer. Now I am facing a different problem, but looking at wrangler's source code I think I know what's happening. |
The |
I understand, but that's not the point I was trying to make. You do not need to add a new parameter, you just have to pass the index parameter from outer method to inner method. If you'd like, I can fork and add a pull request so that you can check what I mean and evaluate if it makes sense. |
I am having a hard time trying to figure where the to_iceberg method tries to create and subsequently destroy the temporary table needed for the INSERT INTO … SELECT statement.
wr.athena.to_iceberg(df=data, index=True, database=os.getenv("GLUE_DATABASE"), table=os.getenv("GLUE_TABLE").lower(), merge_cols=[time_measure], workgroup=athena_workgroup, encryption="SSE-KMS", kms_key=kms_key)
This is how I am using the method. I have set up the IAM role to have necessary IAM permissions on the destination bucket, the corresponding glue db and table, as well as LakeFormation permissions on said db and table.
The code still raises the following exceptions.
botocore.errorfactory.AccessDeniedException: An error occurred (AccessDeniedException) when calling the GetTable operation: Insufficient Lake Formation permission(s) on temp_table_dca47e409f4a494781e27ea08cc1f74c
botocore.errorfactory.AccessDeniedException: An error occurred (AccessDeniedException) when calling the DeleteTable operation: Insufficient Lake Formation permission(s): Required Drop on temp_table_dca47e409f4a494781e27ea08cc1f74c
I was wondering, could it be that the temp tables are created in a different db on which I did not set enough LakeFormation permissions? I tried adding LakeFormation permissions to the relevant IAM role even on default db, but nothing changed.
Any help is appreciated!
The text was updated successfully, but these errors were encountered: