-
-
Notifications
You must be signed in to change notification settings - Fork 18.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Errors with nosetests-3.3 pandas.io.tests.test_yahoo.TestYahoo pandas, version b0ee363408336 #2847
Comments
Interesting, I developed those tests (and the methods behind them), and they were passing previously, and certainly before I submitted the PR. Regarding the decode issue, I know there were changes to how text and data are interpreted in Python 3 (see: http://docs.python.org/3.0/whatsnew/3.0.html#text-vs-data-instead-of-unicode-vs-8-bit), but I mostly develop in Python 2.7, and was believing so much that Travis would catch things like that for me. It didn't, I guess. Thanks for the heads up and suggested fixes, I'll implement them soon to see how they affect code in 2.7. Anyone's thoughts on this would be appreciated. Regarding the failed tests to incorrect hardcoded data, I am somewhat at a loss, as I referenced specific dates and date ranges to ensure that these particular values would be absolute. These also previously passed tests and I can only think that there was either a copy-paste slip right before submitting or I am overlooking some aspect of data queries from Yahoo! Finance. Either way, your values seem correct, and I'll included them in a PR soon to see if we can squash theses errors. Still, I am going to dig a bit deeper to make sure I'm not missing something more fundamental. Thanks! |
Thanks @lodagro and @selasley for the feedback. It was something fundamental:heavy_exclamation_mark: With Yahoo's details regarding their adjusted close calculations being quite lacking, I spent a little bit more time digging around. Based off the number of threads, blog posts, and white papers I discovered, it turns out I wasn't alone, and I though it helpful to comment a bit on the topic. I was incorrect in my assumption of how yahoo calculates its adjusted close price, which I had previously assumed to be forward adjusted (i.e., based off of the starting date of the date range of interest and projected forward). As it turns out, it's a blanket backwards adjusted calculation from the present, and includes adjustments for all dividends and stock splits from the present day. The error from expected = [18.38, 27.45, 24.54] #error
expected = [18.38, 27.45, 24.28] #corrected value refers to result = pan.Close.ix['01-18-12'][['GE', 'MSFT', 'INTC']].tolist() The story is, the same day I submitted PR #2795 on Feb. 5th, Intel had a dividend payout of 0.225, which then caused all adjusted close prices before this date to be recalculated and updated, resulting in the small change of value observed, from result = pan.Ret_Index.ix['01-18-11':'01-21-11'][['GE', 'INTC', 'MSFT']] is a result of the return index values being dependent on Intel's adjusted close price. As such, note that references to any value associated with the adjusted close price from yahoo are not absolute, and I will be updating the tests in Oh, if you'd like to learn more, checkout some helpful discussions on the topic:
EDIT: Also, I'll checkout using |
travis uses |
Could add the -v flag to nosetests, to see in the travis logs which tests ran/skip. |
I have done for debugging in the past,but travis's UI is sluggish at best |
@selasley, would you like to open a PR for the decode('utf8') bit? I'll merge it. IIUC, fixing the test vectors will still have the test failing come next dividend |
closed via 44b2495 |
I think get_quote_yahoo and get_components_yahoo in data.py need to decode the bytes returned by urllib.request.urlopen(urlStr).readlines() and urllib.urlopen(urlStr).read().strip().strip('"').split('"\r\n"') in python3.3
Also, the expected INTC values in test_get_data() in test_yahoo.py do not agree with the values Yahoo returns. The Adj Close value for 1-18-12 is 24.28 and the calculated Ret_Index values do not agree with the values in the d array.
The tests pass after making these changes. I have not tested the changes in python2.x
original errors
nosetests-3.3 pandas.io.tests.test_yahoo.TestYahoo pandas
.EEE
ERROR: test_get_quote (pandas.io.tests.test_yahoo.TestYahoo)
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/pandas-0.11.0.dev_b0ee363-py3.3-macosx-10.6-intel.egg/pandas/io/tests/test_yahoo.py", line 49, in test_get_quote
df = web.get_quote_yahoo(pd.Series(['GOOG', 'AAPL', 'GOOG']))
File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/pandas-0.11.0.dev_b0ee363-py3.3-macosx-10.6-intel.egg/pandas/io/data.py", line 118, in get_quote_yahoo
fields = line.strip().split(',')
TypeError: Type str doesn't support the buffer API
ERROR: test_get_components (pandas.io.tests.test_yahoo.TestYahoo)
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/pandas-0.11.0.dev_b0ee363-py3.3-macosx-10.6-intel.egg/pandas/io/tests/test_yahoo.py", line 57, in test_get_components
df = web.get_components_yahoo('^DJI') #Dow Jones
File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/pandas-0.11.0.dev_b0ee363-py3.3-macosx-10.6-intel.egg/pandas/io/data.py", line 245, in get_components_yahoo
strip('"').split('"\r\n"'))
TypeError: Type str doesn't support the buffer API
ERROR: test_get_data (pandas.io.tests.test_yahoo.TestYahoo)
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/pandas-0.11.0.dev_b0ee363-py3.3-macosx-10.6-intel.egg/pandas/io/tests/test_yahoo.py", line 87, in test_get_data
dfi = web.get_components_yahoo('^DJI')
File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/pandas-0.11.0.dev_b0ee363-py3.3-macosx-10.6-intel.egg/pandas/io/data.py", line 245, in get_components_yahoo
strip('"').split('"\r\n"'))
TypeError: Type str doesn't support the buffer API
Ran 4 tests in 0.962s
FAILED (errors=3)
The text was updated successfully, but these errors were encountered: