Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add preprocess by shank #151

Merged
merged 8 commits into from
Dec 15, 2023
Merged

Add preprocess by shank #151

merged 8 commits into from
Dec 15, 2023

Conversation

JoeZiminski
Copy link
Member

@JoeZiminski JoeZiminski commented Dec 11, 2023

This PR adds the ability to preprocess a recording by group. This entails splitting the recording, applying preprocessing, and then aggregating at the end.

Initially, for each step the recording was split, preprocessed, then aggregated. However, this led to very slow behaviour when using the get_traces() function, it does not appear aggregate_channels is best used in this way. Instead, the recording is split at the start. For each preporcessing step, the preproecssing is applied, and a aggregated version of the recording is stored on preprocess_data for provenance. However, the split recording are kept and each preprocessing step is applied to these separately. Then at the end, this split recording is finally reaggregated. This means that for each stored preprocessing step the data is aggregated and it is aggregated in the final step, but for each step the aggregation only happens once.

@JoeZiminski JoeZiminski mentioned this pull request Dec 13, 2023
@JoeZiminski JoeZiminski force-pushed the add_preprocess_by_shank branch 4 times, most recently from 1030d15 to 934de87 Compare December 15, 2023 11:58
@JoeZiminski JoeZiminski force-pushed the add_preprocess_by_shank branch from 60a4b45 to a05eb73 Compare December 15, 2023 17:23
@JoeZiminski JoeZiminski merged commit 9802eb2 into dev Dec 15, 2023
13 checks passed
@JoeZiminski JoeZiminski deleted the add_preprocess_by_shank branch December 15, 2023 17:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant