Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Model Complexity Computations #171

Closed
antoinecarme opened this issue Jul 1, 2021 · 4 comments
Closed

Update Model Complexity Computations #171

antoinecarme opened this issue Jul 1, 2021 · 4 comments

Comments

@antoinecarme
Copy link
Owner

antoinecarme commented Jul 1, 2021

PyAF uses a notion of complexity for each component of the model (signal transformation, trend, cycle.seasonal, AR, ... )

The complexity value is used when two models have almost the same performance (MAPE) , the less complex model is kept in this case (Occam's razor).

By introducing new models (Croston, XGBoost, LightGBM), the notion of complexity has to be dependent of each component and the overall weighting is no longer relevant.

this change will not impact too much the output of PyAF as the complexity is not always used in the model selection.

@antoinecarme
Copy link
Owner Author


~/dev/python/packages/timeseries/pyaf$ grep mComplexity pyaf/TS/SignalDecomposition*.py
pyaf/TS/SignalDecomposition_AR.py:        self.mComplexity = None;
pyaf/TS/SignalDecomposition_AR.py:        self.mComplexity = 0;
pyaf/TS/SignalDecomposition_Cycle.py:        self.mComplexity = None;
pyaf/TS/SignalDecomposition_Cycle.py:        self.mComplexity = 0;
pyaf/TS/SignalDecomposition_Cycle.py:        self.mComplexity = 1;
pyaf/TS/SignalDecomposition_Cycle.py:        self.mComplexity = 2;
pyaf/TS/SignalDecomposition_Trend.py:        self.mComplexity = None;
pyaf/TS/SignalDecomposition_Trend.py:        self.mComplexity = 0;
pyaf/TS/SignalDecomposition_Trend.py:        self.mComplexity = 2;
pyaf/TS/SignalDecomposition_Trend.py:        self.mComplexity = 3;
pyaf/TS/SignalDecomposition_Trend.py:        self.mComplexity = 3;
pyaf/TS/SignalDecomposition_Trend.py:        self.mComplexity = 1;
pyaf/TS/SignalDecomposition_Trend.py:        self.mComplexity = 1;

~/dev/python/packages/timeseries/pyaf$ grep mComplexity pyaf/TS/Scikit_Models*.py
        self.mComplexity = len(self.mInputNamesAfterSelection)
        self.mComplexity = P;
        self.mComplexity = 2*P;
        self.mComplexity = 2*P;
        self.mComplexity = 2*P;

@antoinecarme
Copy link
Owner Author

antoinecarme commented Jul 1, 2021

Overall weighting (weights should be 1, each model uses its own weight and defines its own complexity)

     def getComplexity(self):
-        lComplexity = 32 * self.mTransformation.mComplexity +  16 * self.mTrend.mComplexity + 4 * self.mCycle.mComplexity + 1 * self.mAR.mComplexity;

antoinecarme added a commit that referenced this issue Jul 1, 2021
Overall weighting (weights should be 1, each model uses its own weight)
Now, the complexity of a model is the sum of the complexities of all its components
@antoinecarme
Copy link
Owner Author

New values

~/dev/python/packages/timeseries/pyaf$ grep mComplexity pyaf/TS/SignalDecomposition*.py
pyaf/TS/SignalDecomposition_AR.py:        self.mComplexity = None;
pyaf/TS/SignalDecomposition_AR.py:        self.mComplexity = 0;
pyaf/TS/SignalDecomposition_Cycle.py:        self.mComplexity = None;
pyaf/TS/SignalDecomposition_Cycle.py:        self.mComplexity = 0;
pyaf/TS/SignalDecomposition_Cycle.py:        self.mComplexity = len(self.mEncodedValueDict.keys())
pyaf/TS/SignalDecomposition_Cycle.py:        self.mComplexity = 0
pyaf/TS/SignalDecomposition_Cycle.py:            self.mComplexity = len(lDict.keys())
pyaf/TS/SignalDecomposition_Trend.py:        self.mComplexity = None;
pyaf/TS/SignalDecomposition_Trend.py:        self.mComplexity = 0;
pyaf/TS/SignalDecomposition_Trend.py:        self.mComplexity = 2;
pyaf/TS/SignalDecomposition_Trend.py:        self.mComplexity = iWindow;
pyaf/TS/SignalDecomposition_Trend.py:        self.mComplexity = iWindow;
pyaf/TS/SignalDecomposition_Trend.py:        self.mComplexity = 1;
pyaf/TS/SignalDecomposition_Trend.py:        self.mComplexity = 3;

antoinecarme added a commit that referenced this issue Jul 1, 2021
Updated complexities for trends.
antoinecarme added a commit that referenced this issue Jul 1, 2021
Updated complexities for cycles.
@antoinecarme
Copy link
Owner Author

New weighting

    def getComplexity(self):
        lComplexity = self.mTransformation.mComplexity +  self.mTrend.mComplexity + self.mCycle.mComplexity + self.mAR.mComplexity;
        return lComplexity;     

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant