Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SNOW-733213: Add ability for write_pandas to truncate table instead of drop table #1413

Closed
sfc-gh-cconner opened this issue Jan 26, 2023 · 5 comments · Fixed by #1833
Closed
Labels
enhancement The issue is a request for improvement or a new feature good first issue triaged

Comments

@sfc-gh-cconner
Copy link
Contributor

Please answer these questions before submitting your issue. Thanks!

  1. What version of Python are you using?

    3.9.0

  2. What operating system and processor architecture are you using?

macOS-13.1-x86_64-i386-64bit

  1. What are the component versions in the environment (pip freeze)?

snowflake-connector-python==2.9.0

  1. What did you do?

Using write_pandas with overwrite=true, the table on Snowflake side is dropped and then recreated. This was decided in this git issue:

#1175

The decision was made in order to maintain the dataframe schema. However, drop tables loses grants on the table.

  1. What did you expect to see?

We would like to have an option that truncates the table instead of dropping the table in order to maintain grants.

@github-actions github-actions bot changed the title Add ability for write_pandas to truncate table instead of drop table SNOW-733213: Add ability for write_pandas to truncate table instead of drop table Jan 26, 2023
@sfc-gh-sfan sfc-gh-sfan added enhancement The issue is a request for improvement or a new feature good first issue and removed bug labels Jan 26, 2023
@sfc-gh-sfan
Copy link
Contributor

Thanks for reporting! This is actually a good onboarding issue :)

@LeonMar97
Copy link

Hi, was that issue already resolved? seems like there is just discrepancy with the documentation regarding when the table truncates.

@sfc-gh-sfan
Copy link
Contributor

This should have been resolved per this docstring:

overwrite: When true, and if auto_create_table is true, then it drops the table. Otherwise, it
truncates the table. In both cases it will replace the existing contents of the table with that of the passed in
Pandas DataFrame.

@ttzhou
Copy link

ttzhou commented Sep 3, 2023

Reading the code though, it appears to be doing "drop and swap/rename", not truncating. I don't see any "TRUNCATE" string anywhere else in that file...

This is on 3.1.1

@andreqaugusto
Copy link
Contributor

andreqaugusto commented Nov 3, 2023

per my personal experience and as @ttzhou pointed out, the code does not behave as the docstring says it so... I submitted a (probable) fix to this issue that makes the code behave like the docstring says it would be.

I thought about only fixing the docstring, but I think that we should have the option of truncating the table instead of just dropping it (because we lose any comments or defaults or conditions of the original table).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement The issue is a request for improvement or a new feature good first issue triaged
Projects
None yet
7 participants