Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add regression-based approach to removing EOG artifacts #11046

Merged
merged 84 commits into from
Sep 27, 2022

Conversation

wmvanvliet
Copy link
Contributor

@wmvanvliet wmvanvliet commented Aug 18, 2022

This adds the mne.preprocessing.eog_regression function to regress EOG channels out of other channels. This provides a way to remove eye artifacts without zapping out a complete PCA or ICA component from the data. This could be useful when you don't have many channels.

closes #11028

Todo:

  • Test effectiveness of the approach
  • Example
  • Modify existing tutorial
  • Decide on API: single eog_regression function or .fit()/.apply() object.
  • Add unit tests
  • Update "What's new"

@wmvanvliet wmvanvliet marked this pull request as draft August 18, 2022 13:55
@wmvanvliet wmvanvliet requested a review from cbrnr August 18, 2022 13:56
@larsoner
Copy link
Member

Can you make a sibling PR to deprecate https://github.com/mne-tools/mne-sandbox/blob/master/mne_sandbox/preprocessing/eog.py#L6 ?

Ignore codespell for now, @drammock is on it

@wmvanvliet
Copy link
Contributor Author

The example script shows the function in action on the MNE-sample data. For some more thorough evaluation, perhaps https://osf.io/2qgrd/ is better suited. This dataset contains EEG data with the subject performing guided eye movements as well as periods of rest. The idea is that the EOG correction procedure should remove as much signal variation during the eye movements as possible while keeping the resting state data as-is. Here is a script that operates on this dataset: https://gist.github.com/wmvanvliet/8c9bc44d46ebf51a54ea0ef35d97398b

And here are the results of that script:
fig1

Both ICA and this new regression approach seem to perform excellent.

@wmvanvliet
Copy link
Contributor Author

@larsoner If you have time, could you take a look at the picks parameter of this function and the way references are done? I know there have been changes regarding how we do these things, but I haven't been keeping tabs on it and now I'm unsure.

@larsoner
Copy link
Member

For some more thorough evaluation, perhaps https://osf.io/2qgrd/ is better suited.

Do you think it's worth adding this as a new mne.datasets dataset and adding to a tutorial?

examples/preprocessing/plot_eog_regression.py Outdated Show resolved Hide resolved
mne/preprocessing/eog.py Outdated Show resolved Hide resolved
mne/preprocessing/eog.py Outdated Show resolved Hide resolved
mne/preprocessing/eog.py Outdated Show resolved Hide resolved
mne/preprocessing/eog.py Outdated Show resolved Hide resolved
mne/preprocessing/eog.py Outdated Show resolved Hide resolved
mne/preprocessing/eog.py Outdated Show resolved Hide resolved
@cbrnr
Copy link
Contributor

cbrnr commented Aug 18, 2022

The comparison figures for the external dataset look nice. Can you show which figures the example produces? If they are good enough (i.e. show that regression effectively removes ocular artifacts), then I would not add the new dataset.

@wmvanvliet
Copy link
Contributor Author

Output from the example script:
Figure_1
Figure_2
Figure_3

mne/preprocessing/eog.py Outdated Show resolved Hide resolved
@cbrnr
Copy link
Contributor

cbrnr commented Aug 19, 2022

Indeed the figures from the other dataset demonstrate the effectiveness much better. I do like the regression weight from the example though. Would it be possible to create two similar figures like you showed for the other dataset in the left column (eye blinks before and after removal)?

@wmvanvliet
Copy link
Contributor Author

Here is the eog_regression function applied to the blinks:
Figure_4
Figure_5

@agramfort
Copy link
Member

@wmvanvliet can you share the link to the rendered doc when you have it? 🙏

@cbrnr
Copy link
Contributor

cbrnr commented Aug 19, 2022

Here is the eog_regression function applied to the blinks:

Nice! Please include this figure in the example!

@wmvanvliet
Copy link
Contributor Author

For some more thorough evaluation, perhaps https://osf.io/2qgrd/ is better suited.

Do you think it's worth adding this as a new mne.datasets dataset and adding to a tutorial?

If we just stick with plain regression, without implementing fancier techniques such as RAAA or SGEYESUB, then the sample dataset is actually fine I think.

@wmvanvliet wmvanvliet self-assigned this Aug 22, 2022
@cbrnr
Copy link
Contributor

cbrnr commented Sep 20, 2022

@agramfort working on it right now!

Copy link
Contributor

@cbrnr cbrnr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work @wmvanvliet! I have some minor suggestions, after that this is good to go! Thanks a lot for this great addition!

mne/preprocessing/_regress.py Outdated Show resolved Hide resolved
mne/preprocessing/_regress.py Outdated Show resolved Hide resolved
mne/preprocessing/_regress.py Outdated Show resolved Hide resolved
mne/preprocessing/_regress.py Outdated Show resolved Hide resolved
mne/preprocessing/_regress.py Outdated Show resolved Hide resolved
Copy link
Contributor

@cbrnr cbrnr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, forgot the example. Only some minor things though.

examples/preprocessing/eog_regression.py Outdated Show resolved Hide resolved
examples/preprocessing/eog_regression.py Outdated Show resolved Hide resolved
examples/preprocessing/eog_regression.py Outdated Show resolved Hide resolved
examples/preprocessing/eog_regression.py Show resolved Hide resolved
examples/preprocessing/eog_regression.py Outdated Show resolved Hide resolved
@agramfort
Copy link
Member

agramfort commented Sep 26, 2022 via email

@wmvanvliet
Copy link
Contributor Author

CI not happy yet :(

@cbrnr
Copy link
Contributor

cbrnr commented Sep 27, 2022

Can you post links to the rendered docs?

@wmvanvliet
Copy link
Contributor Author

CI pass! All is ready from my end, unless someone has some more nitpicks.

@cbrnr
Copy link
Contributor

cbrnr commented Sep 27, 2022

You still have raw.apply_proj() in the tutorial, is there a reason why you didn't delete it?

@cbrnr
Copy link
Contributor

cbrnr commented Sep 27, 2022

Also, these lines in the example should be formatted as code:

Screen Shot 2022-09-27 at 10 56 42

I don't have time to go through all things that I mentioned previously, but it would be nice that in case you do not want to implement a comment/suggestion to at least let me know.

@wmvanvliet
Copy link
Contributor Author

Sorry for being overzealous in marking your comments as resolved. I indeed missed some things. I went over them again and I think I got them all now.

@wmvanvliet
Copy link
Contributor Author

@cbrnr
Copy link
Contributor

cbrnr commented Sep 27, 2022

No problem @wmvanvliet, thanks for going through everything once again!

Re the example, sorry that I keep nitpicking, but I think the line Import packages and load the MNE sample data. should also be a comment as part of the code. So the first code block should start with

# Author: Marijn van Vliet <w.m.vanvliet@gmail.com>

and end with

raw.filter(0.3, None, picks='all')

@wmvanvliet
Copy link
Contributor Author

That was intentional. Why should that sentence be a comment and not just a paragraph block?

@cbrnr
Copy link
Contributor

cbrnr commented Sep 27, 2022

That's fine of course, but right now it is not a sentence, that's why I thought it was supposed to be a comment. Can you change it to a proper sentence?

@wmvanvliet
Copy link
Contributor Author

Done :)

@cbrnr
Copy link
Contributor

cbrnr commented Sep 27, 2022

Perfect, thanks a lot!

@larsoner larsoner merged commit 12cf874 into mne-tools:main Sep 27, 2022
@larsoner
Copy link
Member

Thanks @wmvanvliet !

@wmvanvliet wmvanvliet deleted the eog-regression branch September 27, 2022 22:53
larsoner added a commit to drammock/mne-python that referenced this pull request Sep 28, 2022
* upstream/main:
  MAINT: Fix CircleCI error (mne-tools#11205) [circle deploy]
  Add regression-based approach to removing EOG artifacts (mne-tools#11046)
  [DOC, MRG] Minor documentation improvements and remove glossary entry for array-like (mne-tools#11207)
  Fix `include_tmax` not considered in `mne.io.Raw.crop` to check `tmax` in bounds (mne-tools#11204)
  MAINT: Fix notebook backend (mne-tools#11206)
  MRG: Fix displayed Raw duration in Jupyter notebook (mne-tools#11203)
  add EpochsSpectrum.average() (mne-tools#11198)
larsoner added a commit to alexrockhill/mne-python that referenced this pull request Sep 28, 2022
* upstream/main:
  MAINT: Fix CircleCI error (mne-tools#11205) [circle deploy]
  Add regression-based approach to removing EOG artifacts (mne-tools#11046)
  [DOC, MRG] Minor documentation improvements and remove glossary entry for array-like (mne-tools#11207)
  Fix `include_tmax` not considered in `mne.io.Raw.crop` to check `tmax` in bounds (mne-tools#11204)
  MAINT: Fix notebook backend (mne-tools#11206)
  MRG: Fix displayed Raw duration in Jupyter notebook (mne-tools#11203)
  add EpochsSpectrum.average() (mne-tools#11198)
  reduce memory [circle deploy] (mne-tools#11199)
  [BUG, MRG] Fix bug in find_bads_muscle (mne-tools#11197)
  BUG: Fix bug with long legend labels (mne-tools#11181)
larsoner added a commit to drammock/mne-python that referenced this pull request Sep 28, 2022
…-args

* upstream/main:
  Fix mesh display in tutorial (mne-tools#11200)
  MAINT: Add arm64 CI using CirrusCI (mne-tools#11209)
  Fix spatial colors (mne-tools#11201)
  MAINT: Fix CircleCI error (mne-tools#11205) [circle deploy]
  Add regression-based approach to removing EOG artifacts (mne-tools#11046)
  [DOC, MRG] Minor documentation improvements and remove glossary entry for array-like (mne-tools#11207)
  Fix `include_tmax` not considered in `mne.io.Raw.crop` to check `tmax` in bounds (mne-tools#11204)
  MAINT: Fix notebook backend (mne-tools#11206)
  MRG: Fix displayed Raw duration in Jupyter notebook (mne-tools#11203)
  add EpochsSpectrum.average() (mne-tools#11198)
larsoner added a commit to larsoner/mne-python that referenced this pull request Oct 11, 2022
* upstream/main: (64 commits)
  MAINT: Better check (mne-tools#11229)
  MAINT: Fix link and update instantiation note (mne-tools#11228)
  BUG: Add estimated fiducials when missing / assumed head coords (mne-tools#11212)
  Fix tfr db (mne-tools#11223)
  MAINT: Update link (mne-tools#11222)
  add CPGRL doc section (mne-tools#11216)
  Don't insert superfluous newlines in subprocess log messages (mne-tools#11219)
  purge _get_args helper func (mne-tools#11215)
  Standardize topomap args (mne-tools#11123)
  MAINT: Ensure no datasets are downloaded in tests (mne-tools#11213)
  MAINT: Fix Cirrus caching (mne-tools#11211)
  Fix mesh display in tutorial (mne-tools#11200)
  MAINT: Add arm64 CI using CirrusCI (mne-tools#11209)
  Fix spatial colors (mne-tools#11201)
  MAINT: Fix CircleCI error (mne-tools#11205) [circle deploy]
  Add regression-based approach to removing EOG artifacts (mne-tools#11046)
  [DOC, MRG] Minor documentation improvements and remove glossary entry for array-like (mne-tools#11207)
  Fix `include_tmax` not considered in `mne.io.Raw.crop` to check `tmax` in bounds (mne-tools#11204)
  MAINT: Fix notebook backend (mne-tools#11206)
  MRG: Fix displayed Raw duration in Jupyter notebook (mne-tools#11203)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Development

Successfully merging this pull request may close these issues.

Addition of a non-projection-based method of removing EOG and ECG artifacts.
5 participants