Proposed new way to dynamically add and test out indicators without modifying pandas_ta itself

[locupleto]

Hi KJ,

Basically this pull-request concerns what I asked about in this closed issue #264

I added a new module custom.py with all required functionality and an example indicator that can be used as a template (the same one used in the issue ni.py). I put ni.py in the examples folder.

The usage is described in the import_dir-function of the custom module.

Import a directory of custom indicators into pandas_ta

Args:
    dir_path (str): Full path to your indicator tree
    verbose (bool): If True verbose output of results

This method allows you to experiment and develop your own technical analysis
indicators in a separate local directory of your choice but use them seamlessly 
together with the existing pandas_ta functions just like if they were part of 
pandas_ta.

If you at some late point would like to push them into the pandas_ta library
you can do so very easily by following the step by step instruction here
https://github.com/twopirllc/pandas-ta/issues/264.

Let's get started!

1. Loading the 'ta' module:
>>> import pandas as pd
>>> import pandas_ta as ta

2. Create an empty directory on your machine where you want to work with your
indicators. Invoke pandas_ta.custom.import_dir once to pre-populate it with 
sub-folders for all available indicator categories, e.g.:

>>> import os
>>> from os.path import abspath, join, expanduser
>>> from pandas_ta.custom import create_dir, import_dir
>>> my_dir = abspath(join(expanduser("~"), "my_indicators"))
>>> create_dir(my_dir)

3. You can now create your own custom indicator e.g. by copying existing 
ones from pandas_ta core module and modifying them. Each custom indicator 
should have a unique name and have both a function and a method defined
within the module. In essence these modules should look exactly like the
standard indicators available in categories under the pandas_ta-folder.

The only difference will be an addition of a matching class method that
will be imported dynamically to the AnalysisIndicators class and a
call to the utility function that binds the indicator name to pandas_ta.

For an example of the correct structure, look at the example ni.py in the 
examples folder.

The ni.py indicator is a trend indicator so therefoe we drop it into the 
sub-folder named trend. Thus we have a folder structure like this:

~/my_indicators/
│
├── candles/
.
.
└── trend/
.      └── ni.py
.
└── volume/

4. We can now dynamically load all our custom indicators located in our
designated indicators directory like this:

>>> import_dir(my_dir)

If your custom indicator loaded succesfully then it should behave exactly
like all other native indicators in pandas_ta, including help functions.

Kind regards
Urban

[twopirllc]

Hey @locupleto,

Thanks for creating this feature! This looks cool! 😎 I remember discussing it in #264. I will try and merge it in soon after clearing up some TA Lib integration issue that I have been working on. I will keep you posted. I appreciate the contribution!

Thanks,
KJ

[locupleto]

Wow! That's great Kevin! It goes without saying but feel free to alter anything I submitted as you please. I'm just happy to be able to contribute something to your awesome project. :-)

A quick side note; I have attached an example indicator directory containing a couple of indicators in different categories that I simply copied from core.py and renamed. Also the test-cases I ran against that. I don't know if it helps you in any way but there they are anyway.

Please feel free to ask me if you have questions or I can help you
Examples.zip
in any way.
Cheers
Urban

[locupleto]

Hi again Kevin,

Just a quick update. I fixed a bug when importing multiple times within the same Python session. Also I removed the need to call the bind()-function yourself from the custom indicator modules. Comments and the ni.py module in the examples folder were updated to reflect this.

So ignore the examples I sent you previously; here are the updated ones to go with the current versions.

Kind Regards
Urban
Updated Examples.zip

[twopirllc]

Hey @locupleto,

Thanks for the update. I plan to make this my next inclusion as soon as I resolve Issue #285. I apologize for the delay and I appreciate your patience.

Thanks,
KJ

[twopirllc]

@locupleto,

Now that I think about it. 🤔 This is a also good place to also eventually include a YAML config for Pandas TA as I once tried to poll in #258.

KJ

[locupleto]

Hi @twopirllc, no worries. It sure took me a long time to get around to writing the code after discussing the idea with you in issue #264. Actually I didn't get around to do it until my summer vacation started in July. So take your time. 😎

Yeah, sure, why not. I think an application directory would be a very natural place for a user to place both strategy configs as well as any custom indicators of theirs. Looking forward to see what you will come up with Kevin.

Urban

[locupleto]

@twopirllc - I just want to share a thought with you. In the other two commercial trading platforms I use; in addition to user indicators there is also the concept user functions. That is; the user has the ability to create general custom functions that in turn can be used by other custom indicators and functions.

In the solution I proposed this isn't really possible. You can still define utility functions within your custom indicator module which will be loaded together with the main function and method of the module. But if you rely on any of those functions in another indicator module; then your success would rely on the import order of your custom indicators modules which isn't ideal to say the least.

Maybe this isn't an important enough feature to implement (at least now). It all depends on where you want to go with pandas_ta but it may be something to have in mind if you want to go that way.

Urban

[twopirllc]

@locupleto,

I just want to share a thought with you. In the other two commercial trading platforms I use; in addition to user indicators there is also the concept user functions. That is; the user has the ability to create general custom functions that in turn can be used by other custom indicators and functions.

Yeah. I am aware of this.

In the solution I proposed this isn't really possible. You can still define utility functions within your custom indicator module which will be loaded together with the main function and method of the module. But if you rely on any of those functions in another indicator module; then your success would rely on the import order of your custom indicators modules which isn't ideal to say the least.

I can see that.

Maybe this isn't an important enough feature to implement (at least now). It all depends on where you want to go with pandas_ta but it may be something to have in mind if you want to go that way.

For now probably not. Sadly, I do not even get time to create new features nor new indicators anymore. But if you have any ideas on how to approach this, I'm all ears.

KJ

[locupleto]

I totally get that Kevin, and a totally agree. Let's keep it simple. I was just thinking out loud. 🙂

And if the need/demand should arise for a more elaborate solution that also covers user functions then I will take a stab at that too.

Cheers,
Urban

[twopirllc]

@locupleto,

I totally get that Kevin, and a totally agree. Let's keep it simple. I was just thinking out loud. 🙂

No worries. I like forward thinking, it creates brainstorming ideas. 😎

And if the need/demand should arise for a more elaborate solution that also covers user functions then I will take a stab at that too.

That would be awesome and appreciated. 😎

Cheers,
KJ

[twopirllc]

@locupleto,

Also I pushed it to the development branch if you want to make sure I haven't altered it too much before I update main. I will be making changes over time as I figure out how I want to incorporate it into the DataFrame extension and hopefully a yaml config to boot.

$ pip install -U git+https://github.com/twopirllc/pandas-ta.git@development

In the mean time, please let me know that it is working as intended.

Thanks,
KJ

[locupleto]

Hi @twopirllc - it seems to work as intended.

I made a new venv:

numpy==1.21.1
pandas==1.3.1
pandas-ta==0.3.14b0
python-dateutil==2.8.2
pytz==2021.1
six==1.16.0

and tested both the test-cases I used previously as well as some manual load/re-load from a python terminal session. All seems to work.

Thanks a lot Kevin,
Urban

[twopirllc]

@locupleto,

Great! Thanks for taking the time to check it out letting me know. You're welcome! Thanks for contributing! I appreciate it! 😎

Thanks,
KJ