Initial release 0.8.0 🎈

[maread99]

Why is the initial release 0.8? Because a huge amount of work went into this before the first release and I'd like to think (aka hope) it won't be too long before a 1.0 release will be warranted.

I don't know how popular, or not, this package will be. It would certainly benefit from a growing user base -> the more users the greater the potential for the underlying calendars to be maintained and the more accurate the datasets will be for everyone. We'll see.

Please do use this discussion to offer any feedback on the package, bad and good, it'll all be appreciated.

[ValueRaider]

Feedback: performance

Why does PriceYahoo(...) consume almost 6 seconds? I suspect market_prices is fetching all interval sizes up to 1d, but wouldn't lazy-loading be better in PricesYahoo.get?

Can I pass in a request_cache session somewhere?

[maread99]

Hi @ValueRaider,

It's nothing to do with getting prices and everything to do with instantiating and evaluating calendars.

Within the PricesYahoo initialization a calendar is created for each exchange that corresponds with the symbols. Each of these calendars is created to cover the maximum period over which prices could be requested for any passed symbol associated with that exchange. For each calendar, trading indexes are then created at each 'base interval' and it evaluates for each session which intervals are compatible over all calendars (so if for one session one calendar opens at 09:00 and another at 09:30 it will conclude that prices cannot be requested at 1 hour interval for that session).

This calendar creation and evaluation work is all a front-loaded, one-off cost. Locally they usually load for me in 2 to 3 seconds. After creating the instance, requesting prices via .get should be quick. market_prices is very efficient with price data. For any given instance of PricesYahoo prices are cached and any row for any base interval will be requested from Yahoo only once (return is stored in memory), only exception (pretty much) being the most recent row if any exchange is open (i.e. if that price is subject to change).

So, to some extent the initialization time is related to the number of different exchanges that the symbols cover (not the number of symbols) and the period over which prices are available for the symbols. An instance for a single symbol that has 3 years of price history will therefore load quicker than one for four symbols, each on a different exchange, with long price histories and one of which trades 24 hours...

Rather than have market_prices create calendars, preexisting calendars can be passed through to the 'calendars' kwarg of PricesYahoo. So if you've got a symbols list that includes instruments with, say, a 50 year price history but you know that you're only going to request prices for the last 5 years then the initialization will be quicker if you create the calendars yourself and pass them through (you'll just get a CalendarTooShortWarning raised advising of the earliest date for which prices can be requested given the calendars' limits).

Hope that helps.

[ValueRaider]

What performance do you see for this? 2x calls on same exchange. I see 2nd call speedup is ~15%, indicates little cache reuse.

PricesYahoo("AAPL")
PricesYahoo("AAPL")

[ValueRaider]

I had no idea that was possible. I should have dug into the tutorials.

[maread99]

@ValueRaider, if you can, let me know how you get on using market_prices with a large number of symbols (my usage and the tests are focused on relatively few symbols).

Cheers

[ValueRaider]

Without caching I'm unlikely to test a large number, don't want to draw the wrath of Yahoo's spam blocker.

[maread99]

For any given instance of PricesYahoo prices are cached.

[ValueRaider]

Sure, but I write my code in script files not interactive sessions, so cache would be cleared when file ends.