Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add dtype keyword to read_sql_query to control per column dtypes. #10285

Closed
Gerenuk opened this issue Jun 5, 2015 · 3 comments · Fixed by #37546
Closed

Add dtype keyword to read_sql_query to control per column dtypes. #10285

Gerenuk opened this issue Jun 5, 2015 · 3 comments · Fixed by #37546
Assignees
Labels
Enhancement IO SQL to_sql, read_sql, read_sql_query
Milestone

Comments

@Gerenuk
Copy link

Gerenuk commented Jun 5, 2015

When reading from SQL queries - especially when reading chunk-wise - issues with type conversion can occur.

It would be ideal if one could specify the types of the columns, so that type conversions go right.

A very common case is reading pure float results chunk-wise from a large table. The nuisance comes in when and chunk contains NULL values only in a column. In such cases these NULL values are stored as None rather than float("nan"). I see that this is an issue with inconsistent NULL types in pandas and missing type information in read_sql_query.
Another case could be when text columns happen to contain numbers only in a chunk.

Specifying types for a query manually could resolve these issues.

@jorisvandenbossche jorisvandenbossche added the IO SQL to_sql, read_sql, read_sql_query label Jun 5, 2015
@jorisvandenbossche
Copy link
Member

This can be done equivalent to the dtype argument in read_csv

@jorisvandenbossche jorisvandenbossche added this to the Someday milestone Aug 11, 2015
@TomAugspurger TomAugspurger modified the milestones: Someday, Contributions Welcome Jun 6, 2019
@TomAugspurger TomAugspurger changed the title Add control over types in read_sql_query (to resolve NULL inconsistency) Add dtype keyword to read_sql_query to control per column dtypes. Jun 6, 2019
@avinashpancham
Copy link
Contributor

take

@asandeep
Copy link

asandeep commented Sep 3, 2021

@avinashpancham Looks like this happens with read_sql as well. Are there plans to add a similar dtypes argument to read_sql as well?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement IO SQL to_sql, read_sql, read_sql_query
Projects
None yet
Development

Successfully merging a pull request may close this issue.

8 participants