release: release v1.2.0

changelog.md

@@ -1,18 +1,19 @@
 # Changelog
 
-## v1.2.0 | t.b.d.
+## v1.2.0 | 2023-08-15
 
 #### New features
 
-* Added `KymoTrack.plot_fit()` and `KymoTrackGroup.plot_fit()` to show the fitted model obtained from gaussian refinement.
 * Added fitting mode `"simultaneous"` to `lk.refine_tracks_gaussian()` which enforces optimization bounds between the peak positions. This helps prevent `lk.refine_tracks_gaussian()` from reassigning points to the wrong track when a track momentarily disappears and `overlap_strategy` is set to `"multiple"` and `refine_missing_frames` is set to `True`. When fitting mode is set to `"simultaneous"`, bounds ensure that the individual Gaussians cannot switch position. In addition, this mode uses a better initial guess for the peak amplitudes based on the maximum photon count observed in each range.
-* Added the optional parameter `loss_function` to [`fit_power_spectrum()`](https://lumicks-pylake.readthedocs.io/en/v1.1.0/_api/lumicks.pylake.fit_power_spectrum.html#lumicks.pylake.fit_power_spectrum). Implemented loss functions are `"gaussian"` (default) and `"lorentzian"`. The default corresponds to regular least-squares fitting, whereas `"lorentzian"` invokes a robust fitting method that is less susceptible to spurious peaks in the power spectrum which comes at the cost of a small bias in the estimates for a spectrum without noise peaks. Furthermore, no estimates of the errors in the fitted parameters are provided. This is beta functionality. While usable, this has not yet been tested in a large number of different scenarios. The API can still be subject to change without any prior deprecation notice!
-* Added `PowerSpectrum.identify_peaks()` method to the [`PowerSpectrum`](https://lumicks-pylake.readthedocs.io/en/v1.0.0/_api/lumicks.pylake.force_calibration.power_spectrum.PowerSpectrum.html) class. This method uses probability to identify peaks in the spectrum that are not due to the movement of beads in an optical trap. This is beta functionality. While usable, this has not yet been tested in a large number of different scenarios. The API can still be subject to change without any prior deprecation notice!
+* Added the option to take into account discretization effects in [`DwelltimeModel`](https://lumicks-pylake.readthedocs.io/en/v1.2.0/_api/lumicks.pylake.DwelltimeModel.html) by passing a `discretization_timestep` to the model when constructing it.
+* Added the option to take into account discretization effects when performing dwell time analysis on a [`KymoTrackGroup`](https://lumicks-pylake.readthedocs.io/en/v1.2.0/_api/lumicks.pylake.kymotracker.kymotrack.KymoTrackGroup.html). Simply pass `discrete_model=True` to `KymoTrackGroup.fit_binding_times()` to make use of this new functionality.
+* Added the optional parameter `loss_function` to [`fit_power_spectrum()`](https://lumicks-pylake.readthedocs.io/en/v1.2.0/_api/lumicks.pylake.fit_power_spectrum.html#lumicks.pylake.fit_power_spectrum). Implemented loss functions are `"gaussian"` (default) and `"lorentzian"`. The default corresponds to regular least-squares fitting, whereas `"lorentzian"` invokes a robust fitting method that is less susceptible to spurious peaks in the power spectrum which comes at the cost of a small bias in the estimates for a spectrum without noise peaks. Furthermore, no estimates of the errors in the fitted parameters are provided. This is beta functionality. While usable, this has not yet been tested in a large number of different scenarios. The API can still be subject to change without any prior deprecation notice!
+* Added `PowerSpectrum.identify_peaks()` method to the [`PowerSpectrum`](https://lumicks-pylake.readthedocs.io/en/v1.2.0/_api/lumicks.pylake.force_calibration.power_spectrum.PowerSpectrum.html) class. This method uses probability to identify peaks in the spectrum that are not due to the movement of beads in an optical trap. This is beta functionality. While usable, this has not yet been tested in a large number of different scenarios. The API can still be subject to change without any prior deprecation notice!
+* Added `KymoTrack.plot_fit()` and `KymoTrackGroup.plot_fit()` to show the fitted model obtained from gaussian refinement.
+* Added the ability to specify a cropping region when exporting to an h5-file using `file.save_as(filename, crop_time_range=(starting_timestamp, ending_timestamp))`.
+* Added method to create colormaps approximating a color from emission wavelength. See [`lk.colormaps.from_wavelength()`](https://lumicks-pylake.readthedocs.io/en/v1.2.0/_api/lumicks.pylake.colormaps.html#lumicks.pylake.from_wavelength) for more information.
 * Added support for accessing `Kymo`, `Scan` and `PointScan` by path (e.g. `file["Kymograph"]["my_kymo"]` or `file["Kymograph/my_kymo"]`).
 * Added support for slicing `PointScan`.
-* Added the ability to specify a cropping region when exporting to an h5-file using `file.save_as(filename, crop_time_range=(starting_timestamp, ending_timestamp))`.
-* Added the option to take into account discretization effects in [`DwelltimeModel`](https://lumicks-pylake.readthedocs.io/en/v1.1.0/_api/lumicks.pylake.DwelltimeModel.html) by passing a `discretization_timestep` to the model when constructing it.
-* Added the option to take into account discretization effects when performing dwell time analysis on a [`KymoTrackGroup`](https://lumicks-pylake.readthedocs.io/en/v1.1.0/_api/lumicks.pylake.kymotracker.kymotrack.KymoTrackGroup.html). Simply pass `discrete_model=True` to `KymoTrackGroup.fit_binding_times()` to make use of this new functionality.
 
 #### Bug fixes
 
@@ -24,13 +25,16 @@
 
 #### Other changes
 
-* Dropped `opencv` dependency which was only used for calculating rotation matrices and performing the affine transformations required for image alignment. Pylake now uses `scikit-image` for this purpose.
-* Use Unicode characters for µ and ² when plotting rather than TeX strings.
-* Deprecated fitting mode `"multiple"` in `lk.refine_tracks_gaussian()` as it could lead to spurious track crossings. See the entry for the fitting mode `"simultaneous"` under `New Features` for more information.
 * [`File.save_as()`](https://lumicks-pylake.readthedocs.io/en/latest/_api/lumicks.pylake.File.html#lumicks.pylake.File.save_as) data now allows passing in a single string for the `omit_data` parameter.
 * Gracefully handle empty `Scan` after slicing. Previously, a slice operation on a `Scan` that resulted in no frames remaining raised a `NotImplementedError`. Now it returns an `EmptyScan`.
 * Improved performance of `Scan.pixel_time_seconds`, `Kymo.pixel_time_seconds` and `Kymo.line_time_seconds`.
+* Dropped `opencv` dependency which was only used for calculating rotation matrices and performing the affine transformations required for image alignment. Pylake now uses `scikit-image` for this purpose.
 * Marked functions that take file paths as arguments with the `os.PathLike` type hint to idicate that `pathlib.Path` and similar types are also accepted (not just `str`).
+* Use Unicode characters for µ and ² when plotting rather than TeX strings.
+
+#### Deprecations
+
+* Deprecated fitting mode `"multiple"` in `lk.refine_tracks_gaussian()` as it could lead to spurious track crossings. See the entry for the fitting mode `"simultaneous"` under `New Features` for more information.
 
 ## v1.1.1 | 2023-06-13
 

docs/tutorial/force_calibration.rst

@@ -39,7 +39,7 @@ The various theoretical models that can be used to fit these data are described
 We can download the data needed for this tutorial directly from Zenodo using Pylake.
 Since we don't want it in our working folder, we'll put it in a folder called `"test_data"`::
 
-    filenames = lk.download_from_doi("10.5281/zenodo.7729824", "test_data")
+    filenames = lk.download_from_doi("10.5281/zenodo.7729823", "test_data")
 
 Undoing the previous calibration
 --------------------------------
@@ -465,6 +465,8 @@ When working with small beads or at high laser powers, it is important to verify
 Sometimes, the filtering effect has been characterized independently.
 In that case, the arguments `fixed_diode` and `fixed_alpha` can be passed to :func:`~lumicks.pylake.calibrate_force()` to fix these parameters to their predetermined values.
 
+.. _robust_fitting:
+
 Robust fitting
 --------------
 
@@ -473,7 +475,7 @@ Blocking or windowing the power spectrum ensures that this assumption is close e
 Occasionally, the power spectrum might show a spurious noise peak.
 Such a peak is an outlier in the expected behavior of the spectrum and therefore interferes with the assumption of having a Gaussian error distribution.
 As a result, the fit is skewed. In those cases, it can be beneficial to do a robust fit. When a robust fit is performed, one assumes that the probability of encountering one or multiple outliers is non-negligible.
-By taking this into account during fitting, the fit can be made more robust to outliers in the data. The following example illustrates the method. 
+By taking this into account during fitting, the fit can be made more robust to outliers in the data. The following example illustrates the method.
 
 To see this effect, let's load a dataset of uncalibrated force sensor data of a 4.4 μm bead showing Brownian motion while being trapped. In particular, look at the `Force 2y` sensor signal::
 
@@ -523,19 +525,19 @@ Now plot the robust fit::
 
 .. image:: figures/force_calibration/power_spectrum_noise_peak_robust.png
 
-Notice how the model now follows the power spectrum nearly perfectly. The value for `f_diode` has increased significantly, now that it is not abused to reduce the error induced by the outlier. 
+Notice how the model now follows the power spectrum nearly perfectly. The value for `f_diode` has increased significantly, now that it is not abused to reduce the error induced by the outlier.
 
 This example shows that a robust fitting method is less likely to fail on outliers in the power spectrum data. It is therefore a fair question why one would not use it all the time?
 Robust fitting leads to a small bias in the fit results for which Pylake has no correction.
 Least-squares fitting also leads to a bias, but this bias is known (:cite:`norrelykke2010power`) and can be corrected with `bias_correction=True`.
 Secondly, for least-squares fitting, methods exist to estimate the expected standard errors in the estimates of the free parameters, which are implemented in the least-squares fitting routines that Pylake uses :cite:`press1990numerical`.
 These error estimates are not implemented for robust fitting, and as such, the fit results will show `nan` for the error estimates after a robust fit.
-However, as will be shown below, the robust fitting results may be used as a start to identify outliers automatically, in order to exclude these from a second, regular least-squares, fit. 
+However, as will be shown below, the robust fitting results may be used as a start to identify outliers automatically, in order to exclude these from a second, regular least-squares, fit.
 
 Automated spurious peak detection
 ---------------------------------
 
-We will continue the tutorial with the results of the previous section. If you did not yet do that part of the tutorial, please go back and execute the code examples in that section. 
+We will continue the tutorial with the results of the previous section. If you did not yet do that part of the tutorial, please go back and execute the code examples in that section.
 
 We still have the power spectrum `ps` that was created without blocking or windowing. Here we will use it to identify the peak and automatically obtain frequency exclusion ranges.
 We will use the method :meth:`~lumicks.pylake.force_calibration.power_spectrum.PowerSpectrum.identify_peaks()` in order to do so.
@@ -594,4 +596,4 @@ The default values of `loss_function='gaussian'` and `bias_correction=True` ensu
 
 .. image:: figures/force_calibration/power_spectrum_no_noise_peak.png
 
-Notice that no skewing occurs, and that the values of `fc`, `D` and `f_diode` are now closer to values found via robust fitting in the section above. 
+Notice that no skewing occurs, and that the values of `fc`, `D` and `f_diode` are now closer to values found via robust fitting in the section above.

docs/whatsnew/1.2.0/1_2_0.rst

@@ -3,36 +3,67 @@ Pylake 1.2.0
 
 .. only:: html
 
-Here's a sneak peak at some of the highlights from the upcoming Pylake `v1.2.0` release...
+Pylake `v1.2.0` has been released with new features and improvements to existing analyses. Here’s some of the highlights:
+
+Improved binding time analysis
+------------------------------
 
 Correctly use the minimum observable time in dwell time analysis
-----------------------------------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 In earlier versions of Pylake, binding time analysis would return biased estimates when analyzing kymographs with very few events.
-This was because `KymoTrackGroup.fit_binding_times()` relied on the assumption that the shortest track in the `KymoTrackGroup` represented the minimum observable dwell time.
-This assumption is likely valid for kymographs with many tracks but problematic when few events occur per kymograph. In this case, binding times will be underestimated.
-For more information see the :doc:`changelog</changelog>`.
+This was because :meth:`~lumicks.pylake.kymotracker.kymotrack.KymoTrackGroup.fit_binding_times()` relied on the assumption that the shortest track in the
+:class:`~lumicks.pylake.kymotracker.kymotrack.KymoTrackGroup` represented the minimum observable dwell time.
+This assumption is likely valid for kymographs with many tracks but problematic when few events occur per kymograph. In this case, binding
+times will be underestimated. For more information see the :doc:`changelog</changelog>`.
 
 .. important::
 
     The old (incorrect) behavior is maintained as default until the next major release (`v2.0.0`) to ensure
     backward compatibility. To enable the fixed behavior, specify `observed_minimum=False` when calling
-    `KymoTrackGroup.fit_binding_times()`.
+    :meth:`~lumicks.pylake.kymotracker.kymotrack.KymoTrackGroup.fit_binding_times()`.
 
     Note that CSVs exported from the kymotracker widget  before `v1.2.0` will contain insufficient metadata
-    to make use of the improved analysis. To create this metadata, use `lk.filter_tracks()` on the group with a specified `min_length` before further analysis.
+    to make use of the improved analysis. To create this metadata, use :func:`~lumicks.pylake.filter_tracks()` on the group with a specified
+    `min_length` before further analysis. CSVs exported starting from `v1.2.0` contain an additional data column with this data
+    under the header `"minimum_length (-)"`.
 
 Correct for discretization in binding lifetime analysis
--------------------------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+You can now take into account the discretized nature of observed binding times of tracked particles. When lifetimes are short
+(compared to the scan line time) this can significantly improve results by removing bias from the fitted lifetimes.
+Simply use `discrete_model=True` with
+:meth:`~lumicks.pylake.kymotracker.kymotrack.KymoTrackGroup.fit_binding_times()`.
+
+Plot `KymoTrack` Gaussian fit
+-----------------------------
+
+You can now easily plot the fitting results from :func:`~lumicks.pylake.refine_tracks_gaussian` for a particular scan line using
+:meth:`KymoTrack.plot_fit()<lumicks.pylake.kymotracker.kymotrack.KymoTrack.plot_fit>` or
+:meth:`KymoTrackGroup.plot_fit()<lumicks.pylake.kymotracker.kymotrack.KymoTrackGroup.plot_fit>`.
+
+.. figure:: group_plot_fit.png
+
+    Gaussian fits to signal peaks along a scan line of a tracked kymograph.
+
+Refine gaussian simultaneous
+----------------------------
 
-You can now take into account the discretized nature of observed binding times of tracked particles. In some cases, this can significantly improve results by removing bias from the fitted lifetimes.
-Simply use `discrete_model=True` with :meth:`~lumicks.pylake.kymotracker.kymotrack.KymoTrackGroup.fit_binding_times()`.
+The new fitting mode `"simultaneous"` was introduced to :func:`~lumicks.pylake.refine_tracks_gaussian` which enforces optimization bounds between
+peak positions when two tracks are close together. This ensures that individual gaussians cannot switch positions between two
+tracks (which was a faulty behavior that could occur using the previous "multiple" method). For more information see the :doc:`changelog</changelog>`.
+
+.. important::
+
+    The fitting mode `"simultaneous"` is the recommended flag for refining tracks that may be close together. The previous
+    `"multiple"` option is deprecated and will be removed in a future release.
 
 Generate colormaps according to emission wavelength
 ---------------------------------------------------
 
 By default, single-channel images arising from fluorophores excited with the red, green, and blue lasers
-are plotted with the corresponding `lk.colormaps.red`, `lk.colormaps.green`, and `lk.colormaps.blue`
+are plotted with the corresponding `~lumicks.pylake.colormaps.red` `lk.colormaps.red`, `lk.colormaps.green`, and `lk.colormaps.blue`
 colormaps, respectively. However, the actual light emitted is always red-shifted from the excitation color.
 Now you can plot single-channel images with the approximate color of the signal emitted based on the
 emission wavelength using the `from_wavelength()` method of :data:`~lumicks.pylake.colormaps`.
@@ -42,9 +73,21 @@ emission wavelength using the `from_wavelength()` method of :data:`~lumicks.pyla
     Kymographs showing tracks in three color channels using the default colormaps (left) and colormaps
     corresponding to the actual emission colors (right).
 
+Robust force calibration
+------------------------
+
+Added a new fitting method to deal with spurious noise peaks in power spectra during force calibration.
+See the :ref:`Force Calibration tutorial<robust_fitting>` for more details!
+
+.. figure:: robust_fitting.png
+
+    Fitting a power spectrum with a noise peak at ~20,000 Hz. Top panel: using the standard passive calibration, we can see
+    that the fit is skewed at high frequency end. Bottom panel: using the robust fitting method, the skewness is removed.
+
 Cropping h5 files
 -----------------
 
-You can now use `File.save_as(crop_time_range=(start_timestamp, stop_timestamp))` to export a specific time range to a new `h5` file.
+You can now use :meth:`lk.File.save_as(crop_time_range=(start_timestamp, stop_timestamp))<lumicks.pylake.File.save_as>`
+to export a specific time range to a new `h5` file.
 This can be useful for when you want to export a specific part of the timeline or a partial kymograph for instance.
 Exporting a partial file helps keep file size down and makes it easier to share only the relevant parts of your data with others.

lumicks/pylake/__about__.py

@@ -1,5 +1,5 @@
 __title__ = "lumicks.pylake"
-__version__ = "1.1.1"
+__version__ = "1.2.0"
 __summary__ = "Bluelake data analysis tools"
 __url__ = "https://github.com/lumicks/pylake"