Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parsing of time series text files with date/time split across multiple columns #1174

Closed
wesm opened this issue Apr 30, 2012 · 3 comments
Closed
Labels
Datetime Datetime data dtype Enhancement
Milestone

Comments

@wesm
Copy link
Member

wesm commented Apr 30, 2012

No description provided.

@timmie
Copy link
Contributor

timmie commented May 2, 2012

Reference:
https://groups.google.com/forum/?fromgroups#!topic/pydata/pZjQMX_avmY

Example from tsfromtxt (http://pytseries.sourceforge.net/core.timeseries.io.html):


data = "year, month, a, b\n 2001, 01, 0.0, 10.\n 2001, 02, 1.1, 11."
datecols = (1,2,3)
dateconverter = lambda y, m, d: ts.Date('D', year=int(y), month=int(m),
day=int(d))
dateconverter = lambda y, m: Date('M', year=int(y), month=int(m))
data_ts = ts.tsfromtxt('file.csv', skiprows=1, missing='999.9',
delimiter=';', datecols=datecols, dateconverter=dateconverter)

@wesm
Copy link
Member Author

wesm commented May 14, 2012

This isn't addressed by #1186 and #1227 yet, need to think about parsing to periods and all that, the tsfromtxt functionality is not bad here, post #1186, the strings are concatenated together and parsed, which isn't exactly what you'd want here

@changhiskhan
Copy link
Contributor

In #1186, string concatenation and then parsing is just the really naive default behavior. If you specify a date parsing function it first passes the separate columns in for custom handling and then only do concat+parse if it fails.

@timmie if you're still interested this issue might be a good way to form the start of the pandas.io.converters module. Otherwise we would also appreciate more examples on common examples you've encountered that splits datetime across multiple columns

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Datetime Datetime data dtype Enhancement
Projects
None yet
Development

No branches or pull requests

3 participants