Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DataFrame.from_records incorrectly up-converts dtypes to object. #2623

Closed
dalejung opened this issue Jan 1, 2013 · 4 comments
Closed

DataFrame.from_records incorrectly up-converts dtypes to object. #2623

dalejung opened this issue Jan 1, 2013 · 4 comments
Assignees
Labels
Milestone

Comments

@dalejung
Copy link
Contributor

dalejung commented Jan 1, 2013

rows = []
rows.append([pd.datetime(2010, 1, 1), 1])
rows.append([pd.datetime(2010, 1, 2), 'hi']) # test col upconverts to obj

df2_obj = pd.DataFrame.from_records(rows, columns=['date', 'test'])

print df2_obj.date.dtype # object

If the 'hi' is changed to an int then df2_obj.date.dtype is equal to M8[ns]

@ghost ghost assigned wesm Jan 20, 2013
@wesm
Copy link
Member

wesm commented Jan 20, 2013

Oh boy. This is caused by a real snake nest in pandas/core/internals.py. Punting til post 0.10.1 =/

@jreback
Copy link
Contributor

jreback commented Mar 21, 2013

@dalejung

This is fixed in current master (though I think I will add some tests in any event),
can you verify

In [6]: rows = []
In [7]: rows.append([pd.datetime(2010, 1, 1), 1])
In [8]: rows.append([pd.datetime(2010, 1, 2), 'hi']) # test col upconverts to obj
In [9]: df2_obj = pd.DataFrame.from_records(rows, columns=['date', 'test'])
In [10]: df2_obj.dtypes
Out[10]: 
date    datetime64[ns]
test            object
dtype: object

Ints


In [12]: In [7]: rows.append([pd.datetime(2010, 1, 1), 1])
In [13]: In [8]: rows.append([pd.datetime(2010, 1, 2),1])
In [14]: df2_int = pd.DataFrame.from_records(rows, columns=['date', 'test'])
In [15]: df2_int.dtypes
Out[15]: 
date    datetime64[ns]
test             int64
dtype: object

@dalejung
Copy link
Contributor Author

Yar! Just tested against master and it works.

@jreback
Copy link
Contributor

jreback commented Mar 22, 2013

great!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants