add hermite, hermite normalized and laguerre basis function. Update c…

…hangelog

.pre-commit-config.yaml

@@ -1,42 +1,42 @@
 # See https://pre-commit.com for more information
 # See https://pre-commit.com/hooks.html for more hooks
 default_language_version:
-    python: python3.11
+  python: python3.12
 repos:
-    - repo: https://github.com/pre-commit/pre-commit-hooks
-      rev: v4.4.0
-      hooks:
-          - id: check-added-large-files
-            args: ["--maxkb=15000"]
-          - id: check-toml
-          - id: check-yaml
-            args:
-                - --unsafe
-          - id: end-of-file-fixer
-            exclude: ^(docs/|examples/|images/)
-          - id: trailing-whitespace
-            args: [--markdown-linebreak-ext=md]
-    - repo: https://github.com/psf/black-pre-commit-mirror
-      rev: 24.2.0
-      hooks:
-          - id: black # use black-jupyter to run in notebooks too
-            args: [--config=./pyproject.toml]
-            # It is recommended to specify the latest version of Python
-            # supported by your project here, or alternatively use
-            # pre-commit's default_language_version, see
-            # https://pre-commit.com/#top_level-default_language_version
-    - repo: https://github.com/psf/black-pre-commit-mirror
-      rev: 24.2.0
-      hooks:
-          - id: black-jupyter
-            args: [--config=./pyproject.toml]
-            # It is recommended to specify the latest version of Python
-            # supported by your project here, or alternatively use
-            # pre-commit's default_language_version, see
-            # https://pre-commit.com/#top_level-default_language_version
-            language_version: python3.11
-    - repo: https://github.com/charliermarsh/ruff-pre-commit
-      rev: v0.1.13
-      hooks:
-          - id: ruff
-            types_or: [python] # you can add notebook and other files using pyi, jupyter
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v4.4.0
+    hooks:
+      - id: check-added-large-files
+        args: ["--maxkb=15000"]
+      - id: check-toml
+      - id: check-yaml
+        args:
+          - --unsafe
+      - id: end-of-file-fixer
+        exclude: ^(docs/|examples/|images/)
+      - id: trailing-whitespace
+        args: [--markdown-linebreak-ext=md]
+  - repo: https://github.com/psf/black-pre-commit-mirror
+    rev: 24.2.0
+    hooks:
+      - id: black # use black-jupyter to run in notebooks too
+        args: [--config=./pyproject.toml]
+        # It is recommended to specify the latest version of Python
+        # supported by your project here, or alternatively use
+        # pre-commit's default_language_version, see
+        # https://pre-commit.com/#top_level-default_language_version
+  - repo: https://github.com/psf/black-pre-commit-mirror
+    rev: 24.2.0
+    hooks:
+      - id: black-jupyter
+        args: [--config=./pyproject.toml]
+        # It is recommended to specify the latest version of Python
+        # supported by your project here, or alternatively use
+        # pre-commit's default_language_version, see
+        # https://pre-commit.com/#top_level-default_language_version
+        language_version: python3.12
+  - repo: https://github.com/charliermarsh/ruff-pre-commit
+    rev: v0.1.13
+    hooks:
+      - id: ruff
+        types_or: [python] # you can add notebook and other files using pyi, jupyter

CHANGELOG.md

@@ -1,5 +1,58 @@
 # Changes in SysIdentPy
 
+## v0.5.0
+
+### CONTRIBUTORS
+
+- wilsonrljr
+- nataliakeles
+- LeWerner42
+- Suyash Gaikwad
+
+
+### CHANGES
+
+This update introduces major features and changes. There is a guide to help you update your code to the new version. Depending on your model definition, you might not need to change anything. I decided to go directly to version v0.5.0 instead of providing incremental updates (0.3.5, 0.3.6, etc.) because the breaking changes are easy to fix and the new features are highly beneficial. This release provides crucial groundwork for the future development of SysIdentPy, making easier to add new features and improve the code, setting the stage for a robust and feature-complete 1.0.0 release in the feature.
+
+
+- **New Features:**
+  - **MAJOR**: Add Bilinear Basis Function (thanks nataliakeles). Now the user can use Bilinear NARX models for forecasting.
+  - **MAJOR**: Add Legendre polynomial basis function. Now the user can use Legendre NARX models for forecasting.
+  - **MAJOR**: Bounded Variables Least Squares algorithm for parameter estimation.
+  - **MAJOR**: Least Squares Minimal Residual algorithm for parameter estimation.
+  - **MAJOR**: Error Reduction Ratio algorithm enhancement for FROLS model structure selection. Users can now set an `err_tol` value to stop the algorithm when the sum of the ERR values reaches this threshold, offering a faster alternative to Information Criteria algorithms. A new example is available in the documentation.
+  - **MAJOR**: New Bernstein basis function available, allowing users to choose between Polynomial, Fourier, and Bernstein.
+  - **MAJOR**: v0.1 of the companion book "Nonlinear System Identification: Theory and Practice With SysIdentPy." This open-source book serves as robust documentation for the SysIdentPy package and a friendly introduction to Nonlinear System Identification and Timeseries Forecasting. There are case studies in the book that were not included in the documentation at the time of the update release. The book will always feature more in-depth studies and will be updated regularly with additional case studies.
+
+- **Documentation:**
+  - Files related to v.3.* doc removed.
+  - Improved formatting in mathematical equations.
+  - Fixed typos and grammatical errors in README.md (thanks Suyash Gaikwad and LeWerner42)
+  - Remove book assets from main repository. The assets were moved to sysidentpy-data repository to keep main repository cleaner and lighter.
+  - Fix cover book link in readme. Also change x2_val to x_valid in examples of how to use in readme.
+  - Add Pix method as an alternative for brazilian sponsors.
+  - Fix code documentation for basis function (it was not showing up in the docs before).
+
+- **Datasets:**
+  - Datasets are now available in a separate repository.
+
+- **API Changes:**
+  - add deprecated messages for bias and n in Bersntein basis function. Both parameters will be removed in v0.6.0. Use `include_bias` and `degree`, respectively, instead.
+  - Deploy-docs.yml: Change option to make a clean build of the documentation.
+  - Deploy-docs.yml: Change python version to deploy docs.
+  - Added support for Python 3.13.
+  - Update mkdocstrings dependency version
+  - Change Polynomial check from class name to isinstance method in every class.
+  - Remove support for torch==2.4.0 due to pip error in pytorch side. I'll check if it was solved before allow newer versions of pytorch.
+  - Make "main" the new default branch. Master branch removed.
+  - Change actions from master to main branch.
+  - Split basis function classes into multiples files (one for each basis).
+  - Fix redundant bias check on bersntein basis.
+  - Fix docstring math notation in basis functions docstring.
+  - Remove requirements.txt file.
+  - Extensive code refactoring, including type hint improvements, docstring enhancements, removal of unused code, and other behind-the-scenes changes to support new features.
+
+
 ## v0.4.0
 
 ### CONTRIBUTORS

sysidentpy/basis_function/__init__.py

@@ -3,5 +3,17 @@
 from ._fourier import Fourier
 from ._bersntein import Bersntein
 from ._bilinear import Bilinear
+from ._laguerre import Laguerre
+from ._hermite import Hermite
+from ._hermite_normalized import HermiteNormalized
 
-__all__ = ["Bersntein", "Bilinear", "Fourier", "Legendre", "Polynomial"]
+__all__ = [
+    "Bersntein",
+    "Bilinear",
+    "Fourier",
+    "Legendre",
+    "Laguerre",
+    "Hermite",
+    "HermiteNormalized",
+    "Polynomial",
+]

sysidentpy/basis_function/_hermite.py

@@ -0,0 +1,147 @@
+"""Hermite Basis Function for NARMAX models."""
+
+from typing import Optional
+
+import numpy as np
+from scipy.special import eval_hermite
+
+from sysidentpy.basis_function.basis_function_base import BaseBasisFunction
+
+
+class Hermite(BaseBasisFunction):
+    r"""Build Hermite basis function expansion.
+
+    This class constructs a feature matrix consisting of Hermite polynomial basis
+    functions up to a specified degree. Hermite polynomials, denoted by $H_n(x)$,
+    are orthogonal polynomials over the interval $(-\infty, \infty)$ with respect
+    to the weight function $w(x) = e^{-x^2}$. These polynomials are widely used in
+    probability theory, quantum mechanics, and numerical analysis, particularly in
+    solving the quantum harmonic oscillator and in the field of statistics.
+
+    **Physicist's Hermite polynomials** $H_n(x)$, often used in physics:
+    $$
+    H_n(x) = (-1)^n e^{x^2} \frac{d^n}{dx^n} e^{-x^2}
+    $$
+
+    The Hermite polynomial $H_n(x)$ of degree $n$ can be also defined by the following
+    recurrence relation:
+
+    $$
+    H_0(x) = 1
+    $$
+
+    $$
+    H_1(x) = 2x
+    $$
+
+    $$
+    H_{n+1}(x) = 2x H_n(x) - 2n H_{n-1}(x)
+    $$
+
+    where $H_n(x)$ represents the Hermite polynomial of degree $n$.
+
+    Parameters
+    ----------
+    degree : int, default=2
+        The maximum degree of the Hermite polynomial basis functions to be generated.
+
+    include_bias : bool, default=True
+        Whether to include the bias (constant) term in the output feature matrix.
+
+    ensemble : bool, default=False
+        If True, the original data is concatenated with the polynomial features.
+
+    Notes
+    -----
+    The number of features in the output matrix increases as the degree of the
+    polynomial increases, which can lead to a high-dimensional feature space.
+    Consider using dimensionality reduction techniques if overfitting becomes an issue.
+
+    References
+    ----------
+    - Wikipedia: Hermite polynomial
+        https://en.wikipedia.org/wiki/Hermite_polynomials
+    - Scipy: https://docs.scipy.org/doc/scipy/reference/generated/scipy.special.eval_hermite.html
+
+    """
+
+    def __init__(
+        self,
+        degree: int = 1,
+        include_bias: bool = True,
+        ensemble: bool = False,
+    ):
+        self.degree = degree
+        self.include_bias = include_bias
+        self.ensemble = ensemble
+
+    def _hermite_expansion(self, data: np.ndarray):
+        num_samples = data.shape[0]
+        basis = np.zeros((num_samples, self.degree + 1))
+        for n in range(self.degree + 1):
+            basis[:, n] = eval_hermite(n, data)
+        return basis
+
+    def fit(
+        self,
+        data: np.ndarray,
+        max_lag: int = 1,
+        ylag: int = 1,
+        xlag: int = 1,
+        model_type: str = "NARMAX",
+        predefined_regressors: Optional[np.ndarray] = None,
+    ):
+        # remove intercept (because data always have the intercept)
+        data = data[max_lag:, 1:]
+
+        n_features = data.shape[1]
+        psi = [self._hermite_expansion(data[:, col]) for col in range(n_features)]
+        # remove P0(x) = 1 from every column expansion
+        psi = [basis[:, 1:] for basis in psi]
+        psi = np.hstack(psi)
+        psi = np.nan_to_num(psi, 0)
+        if self.include_bias:
+            bias_column = np.ones((psi.shape[0], 1))
+            psi = np.hstack((bias_column, psi))
+
+        if self.ensemble:
+            psi = np.column_stack([data, psi])
+
+        if predefined_regressors is None:
+            return psi
+
+        return psi[:, predefined_regressors]
+
+    def transform(
+        self,
+        data: np.ndarray,
+        max_lag: int = 1,
+        ylag: int = 1,
+        xlag: int = 1,
+        model_type: str = "NARMAX",
+        predefined_regressors: Optional[np.ndarray] = None,
+    ):
+        """Build Bersntein Basis Functions.
+
+        Parameters
+        ----------
+        data : ndarray of floats
+            The lagged matrix built with respect to each lag and column.
+        max_lag : int
+            Maximum lag of list of regressors.
+        ylag : ndarray of int
+            The range of lags according to user definition.
+        xlag : ndarray of int
+            The range of lags according to user definition.
+        model_type : str
+            The type of the model (NARMAX, NAR or NFIR).
+        predefined_regressors: ndarray
+            Regressors to be filtered in the transformation.
+
+        Returns
+        -------
+        X_tr : {ndarray, sparse matrix} of shape (n_samples, n_features)
+            Transformed array.
+
+        """
+        return self.fit(data, max_lag, ylag, xlag, model_type, predefined_regressors)