Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Copying to Redshift fails if one of the primary key columns includes a hyphen #2350

Closed
mc51 opened this issue Jun 16, 2023 · 1 comment · Fixed by #2351
Closed

Copying to Redshift fails if one of the primary key columns includes a hyphen #2350

mc51 opened this issue Jun 16, 2023 · 1 comment · Fixed by #2351
Labels
bug Something isn't working

Comments

@mc51
Copy link
Contributor

mc51 commented Jun 16, 2023

Describe the bug

Copying to Redshift (awswrangler.redshift.copy) fails if one of the primary key columns (primary_keys) in the table has a hyphen in its name. The CREATE TABLE statement will then fail with the following error message:

ProgrammingError: {'S': 'ERROR', 'C': '42601', 'M': 'syntax error at or near "-"', 'P': '1026', 'F': '/home/ec2-user/padb/src/pg/src/backend/parser/parser_scan.l', 'L': '732', 'R': 'yyerror'}

This is because when creating the SQL Query for this, the column names are not escaped in the PRIMARY KEY () part.

Since this is an easy fix, I'll also open a PR for this myself today.

How to Reproduce

Create a pandas data frame where one of the columns is e.g. my-column-name and then use this column in the primary_keys parameter of the awswrangler.redshift.copy method.

Expected behavior

No error and table is written.

Your project

No response

Screenshots

No response

OS

Linux

Python version

3.9

AWS SDK for pandas version

2.17.0

Additional context

No response

@jaidisido
Copy link
Contributor

Thanks for raising this, I believe there is a suggestion to add a unit test in the PR. Happy to merge afterwards

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants