template fitting problem with Kampff dense array

[zhiwen10]

Hi Marius,
Thank you for the great software for spike sorting. I'm using kilosort2 to look at data from high density probes recorded by Kampff lab. The probes contain 255+1 sites with 5x5 um each site and spacing at 1 um. That is 17x15 array packed within 102 x 84 um space. As you can imagine, many spikes have waveform across 100-200 channels.

Kilosort2 result observation:
Here is the problem with kilosort2 for this data. In the final result, for a cluster that has waveform across 100-200 channels, there are always a dozen of other clusters that have similar waveform shapes with huge CCG peak near 0ms (with a closer look, these clusters have peaks 5-10 samples away from each other, less than 1ms) . Screenshots attached. Among these clusters, there is usually one cluster with nice template that is matching ok with the waveform, but templates of the rest clusters are usually quite wrong.

Problem diagnosis:
If I understand correctly, kilosort2 seems to only use nearby 32 channels at most for template fitting, peal off scaled template, and look for other spikes in residue data at the same spike moment iteratively. While this is beneficial to look at temporally overlapping spikes for probes with large spacing, it causes problem with dense probes as the residue waveform that belongs to the same spike subtracted will be fitted again a couple of times for other templates. I tried to change the size of the template by changing sigmaMask value from 0 to10000, it did not help. I then attempted to modify NchanNear/Nnearest value from 32 to 256, the code broke at learnTemplates (mexMpNU8 specifically). It worked for NchanNear/Nnearest =64 though. There are two issues coming with using only nearby 32 channels, 1) repeated fitting of residual waveform that belong to the same spike 2) not using high resolution data in the rest of the channels for spike sorting. I'd like to point out that Kilosort1 does not seem to have this problem, at least not as much. But we 'd like to use many functionalities implemented in Kilosort2, such as drifting correction, automated determination of template numbers.

Could you please kindly let me know if I'm thinking correctly about the issue, and how can I amend this with Kilosort2? Many thanks, Zhiwen

[marius10p]

Hi Zhiwen, yes, you have correctly identified the issue. The max number of channels is currently "baked in" at several places including in low-level CUDA code. To expose that parameter, I would need to do a careful sweep of the entire codebase, which I don't have time for right now, maybe in March. I'll keep you updated. Kilosort1 by default would use all the channels with minimal runtime penalty, but that was no longer possible in Kilosort2 due to the new algorithms.

[zhiwen10]

Thank you for your response, Marius. I tried to change this value in several locations, including inside CUDA code, without success. Maybe there are a couple of places where it is less obvious, or incompatible. I'm having some difficulties with CUDA GPU debugging in Visual Studio. It will be great if you can help expose this parameter. I will catch up with you on this again soon, hopefully in March.

[marius10p]

Just letting you know that this will take a little longer. We are doing major changes to Kilosort2, and removing some of these baked in parameters along the way, but it won't be ready for a while.

[zhiwen10]

Thanks for updating, Marius. Looking forward to the new version.