Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SNOW-590657: Add overwrite option to write_pandas #1141

Closed
simonwardjones opened this issue May 15, 2022 · 2 comments
Closed

SNOW-590657: Add overwrite option to write_pandas #1141

simonwardjones opened this issue May 15, 2022 · 2 comments

Comments

@simonwardjones
Copy link

What is the current behavior?

When using write_pandas method with auto_create_table as True the code generates a create statement by inferring the column types from the staged dataframe (parquet) file.

The code snippet (from pandas_tools.py) that does this is here:

create_table_sql = (
    f"CREATE {'TEMP ' if create_temp_table else ''}TABLE IF NOT EXISTS {location} "
    f"({create_table_columns})"
    f" /* Python:snowflake.connector.pandas_tools.write_pandas() */ "
)

What is the desired behavior?

It would be handy if we could add an extra argument to the function overwrite that would enable this to be a create or replace with something like:

create_table_sql = (
    f"CREATE {'OR REPLACE' if overwrite else ''} "
    f"{'TEMP ' if create_temp_table else ''} TABLE"
    f"{'IF NOT EXISTS' if not overwrite else ''} {location} "
    f"({create_table_columns})"
    f" /* Python:snowflake.connector.pandas_tools.write_pandas() */ "
)

How would this improve snowflake-connector-python?

It would enable users to overwrite a table with a dataframe (with potentially different shape and column definitions) without having to run an explicit drop statement first.

References, Other Background

@github-actions github-actions bot changed the title Add overwrite option to write_pandas SNOW-590657: Add overwrite option to write_pandas May 15, 2022
@simonwardjones
Copy link
Author

Just a though if this were to be added there might want to be a validation that auto_create_table is not False and overwrite = True

@simonwardjones
Copy link
Author

I am happy to raise a PR but just wanted to get some feedback first

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant