Segmentation violation during Saving results to Phy

[timsainb]

I'm running into some issues with a segmentation violation in a subset of my data. I'm running kilosort2 from SpikeInterface. A subset of my datasets yield this segmentation error, and I can't figure out the cause.

This appears to happen more frequently when I split the sort up into a subset of channels, rather than sorting the whole dataset together.

Time  89s. Optimizing templates ...
Time  89s. Optimizing templates ...
88.85 sec, 1 / 165 batches, 16 units, nspks: 3.7000, mu: 12.1608, nst0: 416
88.85 sec, 1 / 165 batches, 16 units, nspks: 3.7000, mu: 12.1608, nst0: 416
105.55 sec, 101 / 165 batches, 16 units, nspks: 99.3983, mu: 12.5286, nst0: 350
105.55 sec, 101 / 165 batches, 16 units, nspks: 99.3983, mu: 12.5286, nst0: 350
Elapsed time is 115.737678 seconds.
Elapsed time is 115.737678 seconds.
Number of spikes before applying cutoff: 137302
Number of spikes before applying cutoff: 137302
Finished compressing time-varying templates
Finished compressing time-varying templates
initialized spike counts
initialized spike counts
Found 0 splits, checked 1/16 clusters, nccg 0
Found 0 splits, checked 1/16 clusters, nccg 0
Finished splitting. Found 4 splits, checked 20/20 clusters, nccg 0
Finished splitting. Found 4 splits, checked 20/20 clusters, nccg 0
Found 0 splits, checked 1/20 clusters, nccg 0
Found 0 splits, checked 1/20 clusters, nccg 0
Finished splitting. Found 0 splits, checked 20/20 clusters, nccg 0
Finished splitting. Found 0 splits, checked 20/20 clusters, nccg 0
found 0 good units
found 0 good units
Saving results to Phy
Saving results to Phy

--------------------------------------------------------------------------------
       Segmentation violation detected at Thu Dec 03 09:27:35 2020 -0800
--------------------------------------------------------------------------------

Configuration:
  Crash Decoding           : Disabled - No sandbox or build area path
  Crash Mode               : continue (default)
  Default Encoding         : UTF-8
  Deployed                 : false
  GNU C Library            : 2.27 stable
  Graphics Driver          : Unknown software
  Graphics card 1          : 0x10de ( 0x10de ) 0x102d Version 440.100.0.0 (0-0-0)
  Graphics card 2          : 0x10de ( 0x10de ) 0x102d Version 440.100.0.0 (0-0-0)
  Graphics card 3          : 0x102b ( 0x102b ) 0x534 Version 0.0.0.0 (0-0-0)
  Java Version             : Java 1.8.0_202-b08 with Oracle Corporation Java HotSpot(TM) 64-Bit Server VM mixed mode
  MATLAB Architecture      : glnxa64
Number of spikes before applying cutoff: 137302
Number of spikes before applying cutoff: 137302
Finished compressing time-varying templates
Finished compressing time-varying templates
initialized spike counts
initialized spike counts
Found 0 splits, checked 1/16 clusters, nccg 0
Found 0 splits, checked 1/16 clusters, nccg 0
Finished splitting. Found 4 splits, checked 20/20 clusters, nccg 0
Finished splitting. Found 4 splits, checked 20/20 clusters, nccg 0
Found 0 splits, checked 1/20 clusters, nccg 0
Found 0 splits, checked 1/20 clusters, nccg 0
Finished splitting. Found 0 splits, checked 20/20 clusters, nccg 0
Finished splitting. Found 0 splits, checked 20/20 clusters, nccg 0
found 0 good units
found 0 good units
Saving results to Phy
Saving results to Phy

--------------------------------------------------------------------------------
       Segmentation violation detected at Thu Dec 03 09:27:35 2020 -0800
--------------------------------------------------------------------------------

Configuration:
  Crash Decoding           : Disabled - No sandbox or build area path
  Crash Mode               : continue (default)
  Default Encoding         : UTF-8
  Deployed                 : false
  GNU C Library            : 2.27 stable
  Graphics Driver          : Unknown software
  Graphics card 1          : 0x10de ( 0x10de ) 0x102d Version 440.100.0.0 (0-0-0)
  Graphics card 2          : 0x10de ( 0x10de ) 0x102d Version 440.100.0.0 (0-0-0)
  Graphics card 3          : 0x102b ( 0x102b ) 0x534 Version 0.0.0.0 (0-0-0)
  Java Version             : Java 1.8.0_202-b08 with Oracle Corporation Java HotSpot(TM) 64-Bit Server VM mixed mode
  MATLAB Architecture      : glnxa64
  MATLAB Entitlement ID    : 3164272
  MATLAB Root              : /sscf/ssrde-storage/apps/matlab/R2020a
  MATLAB Version           : 9.8.0.1323502 (R2020a)
  OpenGL                   : software
  Operating System         : Ubuntu 18.04.5 LTS
  Process ID               : 25006
  Processor ID             : x86 Family 6 Model 45 Stepping 7, GenuineIntel
  Session Key              : 92cc7a72-3aff-4eed-998a-98e3a254a11d
  Static TLS mitigation    : Disabled: Unnecessary
  Window System            : No active display

Fault Count: 1


Abnormal termination:
Segmentation violation

Register State (from fault):
  RAX = 0000000000641300  RBX = 00007f0d4fffe520
  RCX = 0000000000000000  RDX = 00007f0d4fffe42c
  RSP = 00007f0d4fffe400  RBP = 00007f0d4fffe400
  RSI = 0000000000000001  RDI = 00007f0d65448260

   R8 = 00007f0d4400f2b8   R9 = 0000000000653a10
  R10 = 0000000000000001  R11 = 00007f0d44007350
  R12 = 00007f0d4fffe530  R13 = 00007f0d4fffe850
  R14 = 00007f0d4fffe800  R15 = 00007f0d44009070

  RIP = 00007f0d653dab96  EFL = 0000000000010202

   CS = 0033   FS = 0000   GS = 0000

Stack Trace (from fault):
[  0] 0x00007f0d653dab96 /sscf/ssrde-storage/apps/matlab/R2020a/bin/glnxa64/matlab_startup_plugins/graphics/opengl_startup
_plugin/../../../../../bin/glnxa64/libmwddux.so+00240534
[  1] 0x00007f0d653f094b /sscf/ssrde-storage/apps/matlab/R2020a/bin/glnxa64/matlab_startup_plugins/graphics/opengl_startup
_plugin/../../../../../bin/glnxa64/libmwddux.so+00330059 _ZNKSt8__detail9_Map_baseIiSt4pairIKiNSt7__cxx1112basic_stringIcS
t11char_traitsIcESaIcEEEESaIS9_ENS_10_Select1stESt8equal_toIiESt4hashIiENS_18_Mod_range_hashingENS_20_Default_ranged_hashE
NS_20_Prime_rehash_policyENS_17_Hashtable_traitsILb0ELb0ELb1EEELb1EE2atERS2_+00000027
[  2] 0x00007f0d653db803 /sscf/ssrde-storage/apps/matlab/R2020a/bin/glnxa64/matlab_startup_plugins/graphics/opengl_startup
_plugin/../../../../../bin/glnxa64/libmwddux.so+00243715 _ZN4ddux17getDataStreamNameB5cxx11ENS_10DataStreamE+00000083
[  3] 0x00007f0d54c39bfb /sscf/ssrde-storage/apps/matlab/R2020a/bin/glnxa64/ddux/ddux_impl/mwddux_impl.so+00863227
[  4] 0x00007f0d54c0d300 /sscf/ssrde-storage/apps/matlab/R2020a/bin/glnxa64/ddux/ddux_impl/mwddux_impl.so+00680704
[  5] 0x00007f0d54ca8eea /sscf/ssrde-storage/apps/matlab/R2020a/bin/glnxa64/ddux/ddux_impl/mwddux_impl.so+01318634
[  6] 0x00007f0d54c18c0b /sscf/ssrde-storage/apps/matlab/R2020a/bin/glnxa64/ddux/ddux_impl/mwddux_impl.so+00728075
[  7] 0x00007f0d6d55cdec /sscf/ssrde-storage/apps/matlab/R2020a/bin/glnxa64/libmwms.so+02690540
[  8] 0x00007f0d54c10e46 /sscf/ssrde-storage/apps/matlab/R2020a/bin/glnxa64/ddux/ddux_impl/mwddux_impl.so+00695878
[  9] 0x00007f0d6de6ca02 /sscf/ssrde-storage/apps/matlab/R2020a/bin/glnxa64/libmwboost_thread.so.1.70.0+00064002
[ 10] 0x00007f0d6ee326db              /lib/x86_64-linux-gnu/libpthread.so.0+00030427
[ 11] 0x00007f0d6e5c3a3f                    /lib/x86_64-linux-gnu/libc.so.6+01186367 clone+00000063

** This crash report has been saved to disk as /home/AD/tsainbur/matlab_crash_dump.25006-1 **



MATLAB is exiting because of fatal error

[marius10p]

Sorry, running Kilosort from SpikeInterface is not supported by us, so please contact the SpikeInterface folks.