Merge pull request #20 from derb12/development

Merge development for v0.8.0

.bumpversion.cfg

@@ -1,5 +1,5 @@
 [bumpversion]
-current_version = 0.7.0
+current_version = 0.8.0
 commit = False
 tag = False
 

.gitignore

@@ -87,4 +87,7 @@ ENV/
 # files output by code/miscellaneous files
 Raw Data/
 *.doc*
-*.xls*
+*.xls*
+
+# ignore svg files since the ones generated by Inkscape contain filepaths/usernames
+*.svg

CHANGELOG.rst

@@ -2,6 +2,84 @@
 Changelog
 =========
 
+Version 0.8.0 (2021-12-07)
+--------------------------
+
+This is a minor version with new features, bug fixes, deprecations,
+and documentation improvements.
+
+New Features
+~~~~~~~~~~~~
+
+* Added more efficient ways for creating the spline basis, and now solve penalized
+  spline equations as a banded system rather than as a sparse system. Compared to
+  version 0.7.0, spline.mixture_model, spline.irsqr, and morphological.mpspline are
+  ~60-90% faster when numba is installed and ~10-70% faster without numba.
+* Made several calculations in spline.mixture_model more efficient, further reducing the
+  time by ~60-70% compared to the timings above without numba. The total time reduction
+  from version 0.7.0 for spline.mixture_model without numba is ~50-90%.
+* Added penalized spline versions of all Whittaker-smoothing-based algorithms
+  (pspline_asls, pspline_iasls, pspline_airpls, pspline_arpls, pspline_drpls, pspline_iarpls,
+  pspline_aspls, pspline_psalsa, and pspline_derpsalsa) to pybaselines.spline.
+
+Bug Fixes
+~~~~~~~~~
+
+* Was not multiplying the penalty in whittaker.iasls by `lam_1`.
+* The output weights for polynomial.quant_reg and polynomial.loess are now squared
+  before returning since the square root of the weights are used internally.
+* The output weights and polynomial coefficients (if `return_coef` is True) for
+  polynomial.loess are now sorted to match the original order of the input x-values.
+* The output weights for optimizers.optimize_extended_range are now truncated and
+  sorted before returning to match the original order and length of the input x-values.
+* smooth.noise_median now works with a `smooth_half_window` value of 0 to give no smoothing.
+
+Other Changes
+~~~~~~~~~~~~~
+
+* Officially list Python 3.10 as supported.
+* pybaselines is now available to install using conda from the conda-forge channel.
+* Changed a factor in the weighting for whittaker.aspls to better match the
+  implementation in literature.
+* Allow inputting x-values for all penalized spline functions rather than assuming
+  evenly spaced measurements.
+* optimizers.adaptive_minmax now allows separate `constrained_fraction` and
+  `constrained_weight` values for for the left and right edges.
+* The error raised by optimizers.collab_pls if the input data is not 2-dimensional
+  is now more explicit.
+
+Deprecations/Breaking Changes
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+* No longer allow negative or array-like values for the penalty multipliers in
+  Whittaker-smoothing-based functions, penalized spline functions, morphological.jbcd,
+  or misc.beads. Array-like penalty values are technically valid; however, they change the
+  symmetry of the banded linear system, so additional code will have to be added in a
+  later version to ensure the setup is correct before re-allowing array-like values.
+* Deprecated passing keyword arguments to all functions in pybaselines.optimizers.
+  Passing additional keyword arguments will raise an error starting in version 0.10.0
+  or 1.0.0, whichever comes first (the same deprecation for optimize_extended_range made
+  in version 0.7.0 is also pushed back to 0.10.0 or 1.0.0).
+* For spline algorithms, the min and max x-values are now included as inner knots when
+  creating the spline basis rather than counting them as the first outer knots. To match
+  the number of knots from previous versions, the `num_knots` parameter should add 2 to
+  the `num_knots` used in previous versions.
+* Formally deprecated pybaselines.window, which was replaced by pybaselines.smooth in
+  version 0.6.0. pybaselines.window will be removed in version 1.0.
+* Removed optimize_window from pybaselines.morphological, which was deprecated in
+  version 0.6.0
+* Removed the code for allowing array-like `half_window` or `smooth_half_window` values
+  for morphological.rolling_ball, which was deprecated in version 0.7.0.
+
+Documentation/Examples
+~~~~~~~~~~~~~~~~~~~~~~
+
+* Added more examples to the documentation for fitting noisy data and exploring
+  penalized spline parameters.
+* Added an introduction for the splines category in the algorithms section of the
+  documentation.
+
+
 Version 0.7.0 (2021-10-28)
 --------------------------
 

LICENSES_bundled.txt

@@ -3,7 +3,7 @@ licensed sources, which are listed below.
 
 
 Source: ported MATLAB code from https://www.mathworks.com/matlabcentral/fileexchange/27429-background-correction
-(accessed March 18, 2021)
+(last accessed March 18, 2021)
 Function: pybaselines.polynomial.penalized_poly
 License: 2-clause BSD
 
@@ -33,8 +33,8 @@ ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
 POSSIBILITY OF SUCH DAMAGE.
 
 
-Source: https://gist.github.com/agramfort/850437 (accessed March 25, 2021)
-Function: pybaselines.polynomial.loess and all related functions
+Source: https://gist.github.com/agramfort/850437 (last accessed March 25, 2021)
+Functions: pybaselines.polynomial._loess_low_memory and pybaselines.polynomial._loess_first_loop
 License: 3-clause BSD
 
 # Authors: Alexandre Gramfort <alexandre.gramfort@telecom-paristech.fr>
@@ -71,7 +71,7 @@ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
 Source: ported MATLAB code from
 https://www.mathworks.com/matlabcentral/fileexchange/49974-beads-baseline-estimation-and-denoising-with-sparsity
-(accessed June 28, 2021)
+(last accessed June 28, 2021)
 Function: pybaselines.misc.beads and all related functions
 License: 3-clause BSD
 
@@ -101,7 +101,7 @@ OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
 OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
 
-Source: bandmat (https://github.com/MattShannon/bandmat) (accessed July 10, 2021)
+Source: bandmat (https://github.com/MattShannon/bandmat, last accessed July 10, 2021)
 Function: pybaselines.misc._banded_dot_banded
 License: 3-clause BSD
 
@@ -160,3 +160,39 @@ HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
 WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
 FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
 OTHER DEALINGS IN THE SOFTWARE.
+
+
+Source: SciPy (https://github.com/scipy/scipy, last accessed November 6, 2021)
+File: pybaselines._spline_utils.py
+License: 3-clause BSD
+
+Copyright (c) 2001-2002 Enthought, Inc.  2003-2019, SciPy Developers.
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions
+are met:
+
+1. Redistributions of source code must retain the above copyright
+   notice, this list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above
+   copyright notice, this list of conditions and the following
+   disclaimer in the documentation and/or other materials provided
+   with the distribution.
+
+3. Neither the name of the copyright holder nor the names of its
+   contributors may be used to endorse or promote products derived
+   from this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

README.rst

@@ -2,9 +2,17 @@
 pybaselines
 ===========
 
+.. image:: https://github.com/derb12/pybaselines/raw/main/docs/images/logo.png
+    :alt: Logo
+    :align: center
+
 .. image:: https://img.shields.io/pypi/v/pybaselines.svg
     :target: https://pypi.python.org/pypi/pybaselines
-    :alt: Most Recent Version
+    :alt: Current Pypi Version
+
+.. image:: https://img.shields.io/conda/vn/conda-forge/pybaselines.svg
+    :target: https://anaconda.org/conda-forge/pybaselines
+    :alt: Current conda Version
 
 .. image:: https://github.com/derb12/pybaselines/actions/workflows/python-test.yml/badge.svg
     :target: https://github.com/derb12/pybaselines/actions
@@ -14,10 +22,6 @@ pybaselines
     :target: https://pybaselines.readthedocs.io
     :alt: Documentation Status
 
-.. image:: https://img.shields.io/badge/license-BSD%203--Clause-blue.svg
-    :target: https://github.com/derb12/pybaselines/tree/main/LICENSE.txt
-    :alt: BSD 3-clause license
-
 .. image:: https://img.shields.io/pypi/pyversions/pybaselines.svg
     :target: https://pypi.python.org/pypi/pybaselines
     :alt: Supported Python versions
@@ -41,82 +45,18 @@ pybaselines is a library of algorithms for the baseline correction of experiment
 Introduction
 ------------
 
-pybaselines provides many different baseline correction algorithms for fitting baselines
-to data from experimental techniques such as Raman, FTIR, NMR, XRD, PIXE, etc. The aim of
-the project is to provide a semi-unified API to allow quickly testing and comparing
-multiple baseline correction algorithms to find the best one for a set of data.
-
-pybaselines has 45+ baseline correction algorithms. Whenever possible, the original
-names of the algorithms were used. The algorithms are grouped accordingly:
-
-* Polynomial methods (pybaselines.polynomial)
-
-  * poly (Regular Polynomial)
-  * modpoly (Modified Polynomial)
-  * imodpoly (Improved Modified Polynomial)
-  * penalized_poly (Penalized Polynomial)
-  * loess (Locally Estimated Scatterplot Smoothing)
-  * quant_reg (Quantile Regression)
-  * goldindec (Goldindec Method)
-
-* Whittaker-smoothing-based methods (pybaselines.whittaker)
-
-  * asls (Asymmetric Least Squares)
-  * iasls (Improved Asymmetric Least Squares)
-  * airpls (Adaptive Iteratively Reweighted Penalized Least Squares)
-  * arpls (Asymmetrically Reweighted Penalized Least Squares)
-  * drpls (Doubly Reweighted Penalized Least Squares)
-  * iarpls (Improved Asymmetrically Reweighted Penalized Least Squares)
-  * aspls (Adaptive Smoothness Penalized Least Squares)
-  * psalsa (Peaked Signal's Asymmetric Least Squares Algorithm)
-  * derpsalsa (Derivative Peak-Screening Asymmetric Least Squares Algorithm)
-
-* Morphological methods (pybaselines.morphological)
-
-  * mpls (Morphological Penalized Least Squares)
-  * mor (Morphological)
-  * imor (Improved Morphological)
-  * mormol (Morphological and Mollified Baseline)
-  * amormol (Averaging Morphological and Mollified Baseline)
-  * rolling_ball (Rolling Ball Baseline)
-  * mwmv (Moving Window Minimum Value)
-  * tophat (Top-hat Transformation)
-  * mpspline (Morphology-Based Penalized Spline)
-  * jbcd (Joint Baseline Correction and Denoising)
-
-* Smoothing-based methods (pybaselines.smooth)
-
-  * noise_median (Noise Median method)
-  * snip (Statistics-sensitive Non-linear Iterative Peak-clipping)
-  * swima (Small-Window Moving Average)
-
-* Spline methods (pybaselines.spline)
-
-  * mixture_model (Mixture Model)
-  * irsqr (Iterative Reweighted Spline Quantile Regression)
-  * corner_cutting (Corner-Cutting Method)
-  * ipsa (Iterative Polynomial Smoothing Algorithm)
-  * ria (Range Independent Algorithm)
-
-* Baseline/Peak Classification methods (pybaselines.classification)
-
-  * dietrich (Dietrich's Classification Method)
-  * golotvin (Golotvin's Classification Method)
-  * std_distribution (Standard Deviation Distribution)
-  * fastchrom (FastChrom's Baseline Method)
-  * cwt_br (Continuous Wavelet Transform Baseline Recognition)
-  * fabc (Fully Automatic Baseline Correction)
+pybaselines is a Python library that provides many different algorithms for
+performing baseline correction on data from experimental techniques such as
+Raman, FTIR, NMR, XRD, XRF, PIXE, etc. The aim of the project is to provide a
+semi-unified API to allow quickly testing and comparing multiple baseline
+correction algorithms to find the best one for a set of data.
 
-* Optimizers (pybaselines.optimizers)
-
-  * collab_pls (Collaborative Penalized Least Squares)
-  * optimize_extended_range
-  * adaptive_minmax (Adaptive MinMax)
-
-* Miscellaneous methods (pybaselines.misc)
-
-  * interp_pts (Interpolation between points)
-  * beads (Baseline Estimation And Denoising with Sparsity)
+pybaselines has 50+ baseline correction algorithms. These include popular algorithms,
+such as AsLS, airPLS, ModPoly, and SNIP, as well as many lesser known algorithms. Most
+algorithms are adapted directly from literature, although there are a few that are unique
+to pybaselines, such as penalized spline versions of Whittaker-smoothing-based algorithms.
+The full list of implemented algorithms can be found in the
+`documentation <https://pybaselines.readthedocs.io/en/latest/introduction.html>`_.
 
 
 Installation
@@ -125,43 +65,33 @@ Installation
 Stable Release
 ~~~~~~~~~~~~~~
 
-pybaselines is easily installed from `pypi <https://pypi.org/project/pybaselines>`_
+pybaselines can be installed from `pypi <https://pypi.org/project/pybaselines>`_
 using `pip <https://pip.pypa.io>`_, by running the following command in the terminal:
 
 .. code-block:: console
 
     pip install pybaselines
 
-To also install the `optional dependencies`_ when installing pybaselines, run:
+pybaselines can alternatively be installed from the
+`conda-forge <https://anaconda.org/conda-forge/pybaselines>`_ channel using conda by running:
 
 .. code-block:: console
 
-    pip install pybaselines[full]
-
+    conda install -c conda-forge pybaselines
 
-.. _optional dependencies: https://pybaselines.readthedocs.io/en/latest/installation.html#optional-dependencies
 
 Development Version
 ~~~~~~~~~~~~~~~~~~~
 
-The sources for pybaselines can be downloaded from the `Github repo`_.
-
-The public repository can be cloned using:
+The sources for pybaselines can be downloaded from the `GitHub repo`_.
+To install the current version of pybaselines from GitHub, run:
 
 .. code-block:: console
 
-    git clone https://github.com/derb12/pybaselines.git
+    pip install git+https://github.com/derb12/pybaselines.git#egg=pybaselines
 
 
-Once the repository is downloaded, it can be installed with:
-
-.. code-block:: console
-
-    cd pybaselines
-    pip install .
-
-
-.. _Github repo: https://github.com/derb12/pybaselines
+.. _GitHub repo: https://github.com/derb12/pybaselines
 
 
 Dependencies
@@ -175,11 +105,17 @@ and the following libraries:
 
 
 All of the required libraries should be automatically installed when
-installing pybaselines using either of the two installation methods above.
+installing pybaselines using any of the installation methods above.
+
+The `optional dependencies <https://pybaselines.readthedocs.io/en/latest/installation.html#optional-dependencies>`_
+for pybaselines are listed in the documentation . To also install the optional
+dependencies when installing pybaselines with pip, run:
 
-The optional dependencies for pybaselines are listed in the
-`documentation <https://pybaselines.readthedocs.io/en/latest/installation.html#optional-dependencies>`_.
+.. code-block:: console
+
+    pip install pybaselines[full]
 
+If installing with conda, the optional dependencies have to be specified manually.
 
 Quick Start
 -----------

docs/algorithms/index.rst

@@ -21,8 +21,8 @@ reference listing for any algorithm.
    polynomial
    whittaker
    morphological
-   smooth
    spline
+   smooth
    classification
    optimizers
    misc