Merge remote-tracking branch 'upstream/main' into stc

* upstream/main: MAINT: Fix CircleCI error (mne-tools#11205) [circle deploy] Add regression-based approach to removing EOG artifacts (mne-tools#11046) [DOC, MRG] Minor documentation improvements and remove glossary entry for array-like (mne-tools#11207) Fix `include_tmax` not considered in `mne.io.Raw.crop` to check `tmax` in bounds (mne-tools#11204) MAINT: Fix notebook backend (mne-tools#11206) MRG: Fix displayed Raw duration in Jupyter notebook (mne-tools#11203) add EpochsSpectrum.average() (mne-tools#11198) reduce memory [circle deploy] (mne-tools#11199) [BUG, MRG] Fix bug in find_bads_muscle (mne-tools#11197) BUG: Fix bug with long legend labels (mne-tools#11181)
alexrockhill · Sep 28, 2022 · e1414ad · e1414ad
2 parents ed63788 + 35e466f
commit e1414ad
Show file tree

Hide file tree

Showing 47 changed files with 940 additions and 224 deletions.
diff --git a/.circleci/config.yml b/.circleci/config.yml
@@ -19,6 +19,9 @@ jobs:
           default: "false"
       docker:
         - image: cimg/base:current-22.04
+      # medium 2 vCPUs, 4GB mem; medium+ 3vCPUs 6GB mem; large 4 vCPUs 8GB mem
+      # https://circleci.com/docs/configuration-reference#resourceclass
+      resource_class: medium+
       steps:
         - restore_cache:
             keys:

diff --git a/azure-pipelines.yml b/azure-pipelines.yml
@@ -162,7 +162,7 @@ stages:
     variables:
       DISPLAY: ':99'
       OPENBLAS_NUM_THREADS: '1'
-      TEST_OPTIONS: "--tb=short --cov=mne --cov-report=xml --cov-report=html --cov-append -vv mne/viz/_brain mne/viz/backends mne/viz/tests/test_evoked.py mne/gui"
+      TEST_OPTIONS: "--tb=short --cov=mne --cov-report=xml --cov-report=html --cov-append -vv mne/viz/_brain mne/viz/backends mne/viz/tests/test_evoked.py mne/gui mne/report"
     steps:
     - bash: ./tools/setup_xvfb.sh
       displayName: 'Install Ubuntu dependencies'

diff --git a/doc/changes/latest.inc b/doc/changes/latest.inc
@@ -54,6 +54,7 @@ Enhancements
 
 Bugs
 ~~~~
+- Fix bug in :meth:`mne.io.Raw.crop` where argument ``include_tmax`` was not considered in checking ``tmax`` in bounds (:gh:`11196` by `Lukas Gemein`_)
 - Fix bug in :func:`mne.io.read_raw_eeglab` where unlabeled fiducials causde reading errors (:gh:`11074` by :newcontrib:`Sebastiaan Mathot`)
 - Fix bug in :func:`mne.time_frequency.read_csd` that returned ``projs`` as a list of dict instead of :class:`mne.Projection` (:gh:`11072` by :newcontrib:`Chetan Gohil`)
 - Fix bug in :func:`mne.decoding.TimeFrequency` that prevented cloning if constructor arguments were modified (:gh:`11004` by :newcontrib:`Daniel Carlström Schad`)
@@ -65,13 +66,16 @@ Bugs
 - Fix bug in :func:`mne.minimum_norm.apply_inverse_epochs` where the average EEG projector was not checked properly (:gh:`11182` by `Eric Larson`_)
 - Fix bug in :func:`mne.viz.plot_filter` when plotting filters created using ``output='ba'`` mode with ``compensation`` turned on. (:gh:`11040` by `Marian Dovgialo`_)
 - Fix bugs in documentation of surface :class:`~mne.SourceSpaces` (:gh:`11171` by `Eric Larson`_)
+- Fix bug in :func:`mne.viz.plot_compare_evokeds` where automatic legend labels could be excessively long; they are now abbreviated with ``...`` when necessary (:gh:`11181` by `Eric Larson`_)
 - Fix bugs with ``verbose='error'`` not being used properly and allowing warnings through (:gh:`11193` by `Eric Larson`_)
 - Fix bug in :func:`mne.io.read_raw_bti` where EEG, EMG, and H/VEOG channels were not detected properly, and many non-ECG channels were called ECG. The logic has been improved, and any channels of unknown type are now labeled as ``misc`` (:gh:`11102` by `Eric Larson`_)
 - Fix bug in :func:`mne.viz.plot_topomap` when providing ``sphere="eeglab"`` (:gh:`11081` by `Mathieu Scheltienne`_)
 - Fix bug in :meth:`mne.Dipole.to_mri` where MRI RAS rather than MRI surface RAS was returned (:gh:`11185` by `Eric Larson`_)
 - Fix bug in :meth:`epochs.save <mne.Epochs.save>` where the ``verbose`` parameter defaulted to ``True`` instead of ``None`` (:gh:`11191` by `Eric Larson`_)
 - The string and HTML representation of :class:`mne.preprocessing.ICA` reported incorrect values for the explained variance. This information has been removed from the representations, and should instead be retrieved via the new :meth:`~mne.preprocessing.ICA.get_explained_variance_ratio` method (:gh:`11141` by `Richard Höchenberger`_)
 - Fix bug in :meth:`mne.Evoked.plot` and related methods where a ``np.nan`` location value in any channel causes spatial colours to fail (:gh:`6870` by `Simeon Wong`_)
+- Fix bug in :meth:`mne.preprocessing.ICA.find_bads_muscle` where epochs caused an error when passed as the ``inst`` (:gh:`11197` by `Alex Rockhill`_)
+- The duration of raw data sometimes wasn't displayed correctly in Jupyter notebooks by omitting fractions of a second. We now always round up to the next full second so a duration of less than 1 second will not be displayed as a duration of zero anymore (:gh:`11203` by `Richard Höchenberger`_)
 
 API changes
 ~~~~~~~~~~~
@@ -82,3 +86,5 @@ API changes
 - New classes :class:`~mne.time_frequency.Spectrum` and :class:`~mne.time_frequency.EpochsSpectrum`, created via new methods :meth:`Raw.compute_psd()<mne.io.Raw.compute_psd>`, :meth:`Epochs.compute_psd()<mne.Epochs.compute_psd>`, and :meth:`Evoked.compute_psd()<mne.Evoked.compute_psd>` (:gh:`10184` by `Daniel McCloy`_)
 - The ``mne.epochs.add_channels_epochs`` function has been deprecated in favor of :meth:`epochs.add_channels <mne.Epochs.add_channels>` (:gh:`11180` by `Eric Larson`_)
 - The PSD functions that operate on Raw/Epochs/Evoked instances (``mne.time_frequency.psd_welch`` and ``mne.time_frequency.psd_multitaper``) are deprecated; for equivalent functionality create :class:`~mne.time_frequency.Spectrum` or :class:`~mne.time_frequency.EpochsSpectrum` objects instead and then run ``spectrum.get_data(return_freqs=True)`` (:gh:`10184` by `Daniel McCloy`_)
+- Added new class  :class:`mne.preprocessing.EOGRegression` to allow more flexibility when using regression to reduce EOG artifacts (:gh:`11046` by `Marijn van Vliet`_)
+- New parameter ``exclude`` added to :func:`mne.preprocessing.regress_artifact` to prevent regression from being applied to certain channels (:gh:`11046` by `Marijn van Vliet`_)
diff --git a/doc/conf.py b/doc/conf.py
@@ -181,7 +181,7 @@
     'file-like': ':term:`file-like <python:file object>`',
     'iterator': ':term:`iterator <python:iterator>`',
     'path-like': ':term:`path-like`',
-    'array-like': ':term:`array-like`',
+    'array-like': ':term:`array_like <numpy:array_like>`',
     'Path': ':class:`python:pathlib.Path`',
     'bool': ':class:`python:bool`',
     # Matplotlib
@@ -237,6 +237,7 @@
     'Transform': 'mne.transforms.Transform',
     'Coregistration': 'mne.coreg.Coregistration',
     'Figure3D': 'mne.viz.Figure3D',
+    'EOGRegression': 'mne.preprocessing.EOGRegression',
     'Spectrum': 'mne.time_frequency.Spectrum',
     'EpochsSpectrum': 'mne.time_frequency.EpochsSpectrum',
     # dipy
@@ -975,6 +976,10 @@ def reset_warnings(gallery_conf, fname):
     warnings.filterwarnings(
         'ignore', message=r'iteritems is deprecated.*Use \.items instead\.',
         category=FutureWarning)
+    # pandas in 50_epochs_to_data_frame.py
+    warnings.filterwarnings(
+        'ignore', message=r'invalid value encountered in cast',
+        category=RuntimeWarning)
 
     # In case we use np.set_printoptions in any tutorials, we only
     # want it to affect those:

diff --git a/doc/glossary.rst b/doc/glossary.rst
@@ -22,13 +22,6 @@ general neuroimaging concepts. If you think a term is missing, please consider
         :ref:`tut-events-vs-annotations` for a short tutorial.
         See also :term:`events`.
 
-    array-like
-        Something that acts like – or can be converted to – a
-        :class:`NumPy array <numpy.ndarray>`.
-        This includes (but is not limited to)
-        :class:`arrays <numpy.ndarray>`, `lists <list>`, and
-        `tuples <tuple>`.
-
     beamformer
         A beamformer is a popular source estimation approach that uses a set of
         spatial filters (beamformer weights) to compute time courses of sources

diff --git a/doc/preprocessing.rst b/doc/preprocessing.rst
@@ -70,6 +70,7 @@ Projections:
 
    ICA
    Xdawn
+   EOGRegression
    annotate_amplitude
    annotate_break
    annotate_movement
@@ -99,6 +100,7 @@ Projections:
    oversampled_temporal_projection
    peak_finder
    read_ica
+   read_eog_regression
    realign_raw
    regress_artifact
    corrmap

diff --git a/doc/references.bib b/doc/references.bib
@@ -2351,6 +2351,16 @@ @article{StenroosHauk2013
 	month = nov,
 	year = {2013},
 	keywords = {Electroencephalography, Inverse problem, Magnetoencephalography, Minimum-norm estimation, Skull conductivity},
-	pages = {265--272},
-	file = {Full Text:/Users/larsoner/Zotero/storage/VJAIPDFL/Stenroos and Hauk - 2013 - Minimum-norm cortical source estimation in layered.pdf:application/pdf;ScienceDirect Snapshot:/Users/larsoner/Zotero/storage/FR9YZCJZ/S1053811913004333.html:text/html},
+	pages = {265--272}
+}
+
+@article{CroftBarry2000,
+  title = {Removal of ocular artifact from the {EEG}: a review},
+  author = {Croft, R. J. and Barry, R. J.},
+  year = {2000},
+  journal = {Clinical Neurophysiology},
+  volume = {30},
+  number = {1},
+  pages = {5--19},
+  doi = {10.1016/S0987-7053(00)00055-1}
 }
diff --git a/doc/visualization.rst b/doc/visualization.rst
@@ -59,6 +59,7 @@ Visualization
    plot_projs_joint
    plot_raw
    plot_raw_psd
+   plot_regression_weights
    plot_sensors
    plot_snr_estimate
    plot_source_estimates

diff --git a/examples/datasets/brainstorm_data.py b/examples/datasets/brainstorm_data.py
@@ -34,7 +34,7 @@
 raw_path = (data_path / 'MEG' / 'bst_raw' /
             'subj001_somatosensory_20111109_01_AUX-f.ds')
 # Here we crop to half the length to save memory
-raw = read_raw_ctf(raw_path).crop(0, 180).load_data()
+raw = read_raw_ctf(raw_path).crop(0, 120).load_data()
 raw.plot()
 
 # set EOG channel

diff --git a/examples/preprocessing/eog_regression.py b/examples/preprocessing/eog_regression.py
@@ -0,0 +1,78 @@
+# -*- coding: utf-8 -*-
+"""
+=======================================
+Reduce EOG artifacts through regression
+=======================================
+
+Reduce artifacts by regressing the EOG channels onto the rest of the channels
+and then subtracting the EOG signal.
+
+This is a quick example to show the most basic application of the technique.
+See the :ref:`tutorial <tut-artifact-regression>` for a more thorough
+explanation that demonstrates more advanced approaches.
+"""
+
+# Author: Marijn van Vliet <w.m.vanvliet@gmail.com>
+#
+# License: BSD (3-clause)
+
+# %%
+# Import packages and load data
+# -----------------------------
+#
+# We begin as always by importing the necessary Python modules and loading some
+# data, in this case the :ref:`MNE sample dataset <sample-dataset>`.
+
+import mne
+from mne.datasets import sample
+from mne.preprocessing import EOGRegression
+from matplotlib import pyplot as plt
+
+print(__doc__)
+
+data_path = sample.data_path()
+raw_fname = data_path / 'MEG' / 'sample' / 'sample_audvis_filt-0-40_raw.fif'
+
+# Read raw data
+raw = mne.io.read_raw_fif(raw_fname, preload=True)
+events = mne.find_events(raw, 'STI 014')
+
+# Highpass filter to eliminate slow drifts
+raw.filter(0.3, None, picks='all')
+
+# %%
+# Perform regression and remove EOG
+# ---------------------------------
+
+# Fit the regression model
+weights = EOGRegression().fit(raw)
+raw_clean = weights.apply(raw, copy=True)
+
+# Show the filter weights in a topomap
+weights.plot()
+
+# %%
+# Before/after comparison
+# -----------------------
+# Let's compare the signal before and after cleaning with EOG regression. This
+# is best visualized by extracting epochs and plotting the evoked potential.
+
+tmin, tmax = -0.1, 0.5
+event_id = {'visual/left': 3, 'visual/right': 4}
+evoked_before = mne.Epochs(raw, events, event_id, tmin, tmax,
+                           baseline=(tmin, 0)).average()
+evoked_after = mne.Epochs(raw_clean, events, event_id, tmin, tmax,
+                          baseline=(tmin, 0)).average()
+
+# Create epochs after EOG correction
+epochs_after = mne.Epochs(raw_clean, events, event_id, tmin, tmax,
+                          baseline=(tmin, 0))
+evoked_after = epochs_after.average()
+
+fig, ax = plt.subplots(nrows=3, ncols=2, figsize=(10, 7),
+                       sharex=True, sharey='row')
+evoked_before.plot(axes=ax[:, 0], spatial_colors=True)
+evoked_after.plot(axes=ax[:, 1], spatial_colors=True)
+fig.subplots_adjust(top=0.905, bottom=0.09, left=0.08, right=0.975,
+                    hspace=0.325, wspace=0.145)
+fig.suptitle('Before --> After')
diff --git a/mne/conftest.py b/mne/conftest.py
@@ -27,7 +27,7 @@
 from mne.io import read_raw_fif, read_raw_ctf, read_raw_nirx, read_raw_snirf
 from mne.stats import cluster_level
 from mne.utils import (_pl, _assert_no_instances, numerics, Bunch,
-                       _check_qt_version, _TempDir)
+                       _check_qt_version, _TempDir, check_version)
 
 # data from sample dataset
 from mne.viz._figure import use_browser_backend
@@ -123,6 +123,9 @@ def pytest_configure(config):
     ignore:.*cmap function will be deprecated.*:
     # joblib hasn't updated to avoid distutils
     ignore:.*distutils package is deprecated.*:DeprecationWarning
+    # nbclient
+    ignore:Passing a schema to Validator\.iter_errors is deprecated.*:
+    ignore:Unclosed context <zmq.asyncio.Context.*:ResourceWarning
     """.format(first_kind)  # noqa: E501
     for warning_line in warning_lines.split('\n'):
         warning_line = warning_line.strip()
@@ -426,18 +429,21 @@ def _check_pyqtgraph(request):
             pytest.skip(f'mne_qt_browser {ver} requires PyQt5, API is {api}')
 
 
-@pytest.mark.pgtest
 @pytest.fixture
 def pg_backend(request, garbage_collect):
     """Use for pyqtgraph-specific test-functions."""
     _check_pyqtgraph(request)
+    from mne_qt_browser._pg_figure import MNEQtBrowser
     with use_browser_backend('qt') as backend:
         backend._close_all()
         yield backend
         backend._close_all()
         # This shouldn't be necessary, but let's make sure nothing is stale
         import mne_qt_browser
         mne_qt_browser._browser_instances.clear()
+        if check_version('mne_qt_browser', min_version='0.4'):
+            _assert_no_instances(
+                MNEQtBrowser, f'Closure of {request.node.name}')
 
 
 @pytest.fixture(params=[
@@ -872,7 +878,10 @@ def _nbclient():
 }""", as_version=4)
     client = NotebookClient(nb, km=km)
     yield client
-    client._cleanup_kernel()
+    try:
+        client._cleanup_kernel()
+    except Exception:
+        pass
 
 
 @pytest.fixture(scope='function')

diff --git a/mne/forward/_compute_forward.py b/mne/forward/_compute_forward.py
@@ -2,7 +2,7 @@
 # Authors: Matti Hämäläinen <msh@nmr.mgh.harvard.edu>
 #          Alexandre Gramfort <alexandre.gramfort@inria.fr>
 #          Martin Luessi <mluessi@nmr.mgh.harvard.edu>
-#          Eric Larson <larsoner@uw.edu>
+#          Eric Larson <larson.eric.d@gmail.com>
 #          Mark Wronkiewicz <wronk@uw.edu>
 #
 # License: BSD-3-Clause

diff --git a/mne/forward/_field_interpolation.py b/mne/forward/_field_interpolation.py
@@ -1,7 +1,7 @@
 # -*- coding: utf-8 -*-
 # Authors: Matti Hämäläinen <msh@nmr.mgh.harvard.edu>
 #          Alexandre Gramfort <alexandre.gramfort@inria.fr>
-#          Eric Larson <larsoner@uw.edu>
+#          Eric Larson <larson.eric.d@gmail.com>
 
 # The computations in this code were primarily derived from Matti Hämäläinen's
 # C code.

diff --git a/mne/forward/_lead_dots.py b/mne/forward/_lead_dots.py
@@ -1,5 +1,5 @@
 # Authors: Matti Hämäläinen <msh@nmr.mgh.harvard.edu>
-#          Eric Larson <larsoner@uw.edu>
+#          Eric Larson <larson.eric.d@gmail.com>
 #          Mainak Jas <mainak.jas@telecom-paristech.fr>
 #
 # License: BSD-3-Clause

diff --git a/mne/forward/_make_forward.py b/mne/forward/_make_forward.py
@@ -1,7 +1,7 @@
 # Authors: Matti Hämäläinen <msh@nmr.mgh.harvard.edu>
 #          Alexandre Gramfort <alexandre.gramfort@inria.fr>
 #          Martin Luessi <mluessi@nmr.mgh.harvard.edu>
-#          Eric Larson <larsoner@uw.edu>
+#          Eric Larson <larson.eric.d@gmail.com>
 #
 # License: BSD-3-Clause
 

diff --git a/mne/io/base.py b/mne/io/base.py
@@ -1263,7 +1263,7 @@ def crop(self, tmin=0.0, tmax=None, include_tmax=True, *, verbose=None):
                              % (tmin, tmax))
         if tmin < 0.0:
             raise ValueError('tmin (%s) must be >= 0' % (tmin,))
-        elif tmax > max_time:
+        elif tmax - int(not include_tmax) / self.info['sfreq'] > max_time:
             raise ValueError('tmax (%s) must be less than or equal to the max '
                              'time (%0.4f sec)' % (tmax, max_time))
 
@@ -1745,9 +1745,15 @@ def _repr_html_(self, caption=None):
         basenames = [
             os.path.basename(f) for f in self._filenames if f is not None
         ]
-        m, s = divmod(self._last_time - self.first_time, 60)
-        h, m = divmod(m, 60)
-        duration = f'{int(h):02d}:{int(m):02d}:{int(s):02d}'
+
+        # https://stackoverflow.com/a/10981895
+        duration = timedelta(seconds=self.times[-1])
+        hours, remainder = divmod(duration.seconds, 3600)
+        minutes, seconds = divmod(remainder, 60)
+        seconds += duration.microseconds / 1e6
+        seconds = np.ceil(seconds)  # always take full seconds
+
+        duration = f'{int(hours):02d}:{int(minutes):02d}:{int(seconds):02d}'
         raw_template = repr_templates_env.get_template('raw.html.jinja')
         return raw_template.render(
             info_repr=self.info._repr_html_(caption=caption),

diff --git a/mne/io/ctf/constants.py b/mne/io/ctf/constants.py
@@ -1,7 +1,7 @@
 """CTF constants."""
 
 # Authors: Matti Hämäläinen <msh@nmr.mgh.harvard.edu>
-#          Eric Larson <larsoner@uw.edu>
+#          Eric Larson <larson.eric.d@gmail.com>
 #
 # License: BSD-3-Clause
 

diff --git a/mne/io/ctf/ctf.py b/mne/io/ctf/ctf.py
@@ -1,7 +1,7 @@
 """Conversion tool from CTF to FIF."""
 
 # Authors: Matti Hämäläinen <msh@nmr.mgh.harvard.edu>
-#          Eric Larson <larsoner@uw.edu>
+#          Eric Larson <larson.eric.d@gmail.com>
 #
 # License: BSD-3-Clause
 

diff --git a/mne/io/ctf/res4.py b/mne/io/ctf/res4.py
@@ -1,7 +1,7 @@
 """Read .res4 files."""
 
 # Authors: Matti Hämäläinen <msh@nmr.mgh.harvard.edu>
-#          Eric Larson <larsoner@uw.edu>
+#          Eric Larson <larson.eric.d@gmail.com>
 #
 # License: BSD-3-Clause
 

diff --git a/mne/io/fiff/tests/test_raw_fiff.py b/mne/io/fiff/tests/test_raw_fiff.py
@@ -1049,7 +1049,8 @@ def test_crop():
     tmaxs /= sfreq
     tmins /= sfreq
 
-    # going in revere order so the last fname is the first file (need it later)
+    # going in reverse order so the last fname is the first file (need it
+    # later)
     raws = [None] * len(tmins)
     for ri, (tmin, tmax) in enumerate(zip(tmins, tmaxs)):
         raws[ri] = raw.copy().crop(tmin, tmax)
@@ -1074,6 +1075,19 @@ def test_crop():
     with pytest.raises(ValueError, match='No samples.*when include_tmax=Fals'):
         raw.crop(0, 0, include_tmax=False)
 
+    # edge cases cropping to exact duration +/- 1 sample
+    data = np.zeros((1, 100))
+    info = create_info(1, 100)
+    raw = RawArray(data, info)
+    with pytest.raises(ValueError, match='tmax \\(1\\) must be less than or '):
+        raw.copy().crop(tmax=1, include_tmax=True)
+    raw1 = raw.copy().crop(tmax=1 - 1 / raw.info['sfreq'], include_tmax=True)
+    assert raw.n_times == raw1.n_times
+    raw2 = raw.copy().crop(tmax=1, include_tmax=False)
+    assert raw.n_times == raw2.n_times
+    raw3 = raw.copy().crop(tmax=1 - 1 / raw.info['sfreq'], include_tmax=False)
+    assert raw.n_times - 1 == raw3.n_times
+
 
 @testing.requires_testing_data
 def test_resample_equiv():