expose tiingo split and dividend data

[ethanbsmith]

the current tiingo getSymbols implementation only returns an OHLCV result. It does allow for an adjusted or un-adjusted result, but it does not expose the divCash or splitFactor columns. I'd like to expose this data so that I can use it in other processes. Specifically, i have some other data sources that only expose unadjusted prices. These sources need to be adjusted for splits (not divs), so i need access to the split factor

There are a few considerations in how to go about this:

1. getSymbols.yahoo already includes an extra .Adjusted column, in addition to the standard OHLCV columns, so it would not be unreasonable to just add these 2 columns to the end of the existing output. This would probably be the least invasive and simplest to implement, but would also create a go forward support issue that might interfere with a more radical API redesign approach

2. Another option would be to include a .Adjusted column to make it similar to the yahoo output, and to add parameters to the getSymbols call to control which adjustment to include, similar to how adjRatios works. This would be a bit more complicated to implement, but could work across a number of sources. the biggest downside would be that it wouldn't allow use of split/divs/both in different situations

3. this brings up the larger question of the framework design as a whole and how getSplits(), getDividends, adjRatios should work for sources other than yahoo. I certainly don't want to break existing functionality and design patterns, but it would seems silly to me to extend the existing 3 round trip call patterns to tiingo, when the data is already coming back as part of the getSymbols call.

4. For reference, the alphavatange api does not expose div and split data directly, so it's not really a consideration here

any thoughts on the best way to go about this. I'm happy to do the lifting once a direction/approach is accepted

[joshuaulrich]

Thanks for this suggestion! I definitely think we should expose Tiingo's split and dividend data.

Here are some thoughts, off the top of my head:

1. I'm not excited about adding the columns to the getSymbols() output. I would prefer that getSymbols() provide consistent output for all implemented data sources. This isn't a "no", but I would prefer to avoid this if we can.
2. I like this approach. Some of the getSymbols() 'methods' check for an adjust argument, but it's not documented. I would prefer to explicitly expose and document this as a feature. That should make it easier to implement. Can you elaborate on the downside you're thinking of? It's not clear to me.
3. I agree we should avoid making 3 calls when only 1 is needed. That said, it would be nice to provide getSplits() and getDividends() methods for Tiingo. It seems like exposing the adjust argument would allow us to only make one call for Tiingo.

[braverock]

1. I'm not excited about adding the columns to the getSymbols() output. I would prefer that getSymbols() provide consistent output for all implemented data sources. This isn't a "no", but I would prefer to avoid this if we can.

This is only true for OHLC outputs. There are getSymbols methods which have been developed for a variety of non-OHLC data and commercial data sources which return different data.

I think it certainly makes sense to have consistency in outputs, but we should also not be afraid to provide additional valuable information where that makes sense. For example, in a continuous futures contract, it is very useful to know which contract you are using as the front month, and the volumes of both the front and near contracts. That type of data is meaningless in stock OHLC data. In adjusted stock data, the data becomes less useful as you go further back in time and the number of adjustments rises. This is why most people who use adjusted data for real analysis have some continuously back-adjusted model, so that when you look at the most recent data, it always matches currently observed prices. I think we can maintain consistency while still allowing for more advanced features and standards over time.

[ethanbsmith]

If you consider the adjustment solution space as having 4 modes: unadjusted; splits adjusted; div adjusted; splits and div adjusted

What im looking for is a solution model that supports switching between modes without a round trip to the server

Also, while I see the adjustments mainly applied to listed equity OHLCV, as i describe in my use case above, I need the adjustment multipliers/values in order to apply them to other time series (transactions records).

I agree with @braverock that adjustments are not a panacea.I i also think that people reasonably can do their own research and choose how/when they want to they want to apply them... When you use a data API, you need to understand the data

One last point: there is upside to exposing the splits and divs as data points beyond just adjustments. Having these as data points allows searching/subsetting/visualization/analysis around these data points

[ethanbsmith]

so, here are the solutions discussed so far:

1. Add new .CashDividend and .SplitFactor to existing output for both adjusted = T & F on tiingo output

2. Add .Adjusted column to the exiting output on adjusted = F. not sure what to do with adjusted = T. Maybe formalize a .Unadjusted column to be included with adjusted results.

I'd like to add the following additional ideas for consideration:

3. Extend the what pattern and allow the caller to explicitly request one or both of the additional columns. this could be extended to existing providers as well.If this is selected, it could also be combined with 2

4. Formalize meaning of existing .Adjusted to ALWAYS mean split & div adjusted. Formalize new .CashDividend and .SplitFactor and retrofit these columns into all existing providers that could support them (either directly or through derivation)

[ethanbsmith]

Proposal:

1. formally add adjusted argument to getSymbols
2. add AdjFactor to output. The would be defined as the number than when multiplied by the unadjusted price gives the Split and Dividend adjusted price (divided for volume). This factor will allow you to easily move bi-directionally between adjusted and unadjusted values. The current solution of putting the adjusted Close in the Adjusted column does not allow you to convert from adjusted back to unadjusted. This would be added to yahoo, av, csv, rda, SQLLite, tiingo for now
3. Leave the existing Adjusted output column in yahoo and av, but make notes these will eventually be removed and that code should be modified to use the AdjFactor
4. Modify functions to use use the new AdjFactor column as a standard: as.quantmod.OHLC adjustOHLC to.period Ad has.Ad getSymbols.skeleton
5. move extending getSplits and getDividends into a separate fix and just accept the multiple round trips

https://github.com/joshuaulrich/quantmod/search?q=adjusted

[ethanbsmith]

While the Adjusted can be used for full adjusting of prices, it cant be used for adjusting volumes, which should only be split adjusted. what do you think about adding the splits and dividends as attributes to the OHLCV data?

[joshuaulrich]

I was actually thinking the exact same thing!

[ethanbsmith]

something like:

• add split and dividends attributes
• modify adjustOHLC() to allow using the attributes as an option
• not sure about to.period() and its ilk. should they carry forward the attributes, drop them, or try modifying them.

(feel free to edit)