Possible missing streams in CedIO

[h-mayorquin]

Hi, I am looking at the gin data for CedIO and I found the following:

from pathlib import Path
from neo import CedIO

DATA_PATH = Path("/home/heberto/ephy_testing_data/")  # Gin data
file_path = DATA_PATH / "spike2" / "m365_1sec.smrx"

neo_recorder = CedIO(filename=str(file_path))
neo_recorder.header["signal_streams"]

With the output indicating only one stream:

array([('0 69chans', '0')], dtype=[('name', '<U64'), ('id', '<U64')])

However, if I looked at the last ten channels:

neo_recorder.header["signal_channels"][-10:]

The output is:

array([('RhdD-59', '62', 30030.03003003, 'int16', 'mV', 1.95312500e-04, 0.  , '0'),
       ('RhdD-60', '63', 30030.03003003, 'int16', 'mV', 1.95312500e-04, 0.  , '0'),
       ('RhdD-61', '64', 30030.03003003, 'int16', 'mV', 1.95312500e-04, 0.  , '0'),
       ('RhdD-62', '65', 30030.03003003, 'int16', 'mV', 1.95312500e-04, 0.  , '0'),
       ('RhdD-63', '66', 30030.03003003, 'int16', 'mV', 1.95312500e-04, 0.  , '0'),
       ('CED_Mech', '67', 30030.03003003, 'int16', 'g', 2.04467773e-03, 0.  , '0'),
       ('LFP', '68', 30030.03003003, 'int16', 'V', 5.03540039e-05, 1.65, '0'),
       ('MechTTL', '70', 30030.03003003, 'int16', 'V', 5.03540039e-05, 1.65, '0'),
       ('MechStim', '71', 30030.03003003, 'int16', 'V', 5.03540039e-05, 1.65, '0'),
       ('Laser', '72', 30030.03003003, 'int16', 'V', 5.03540039e-05, 1.65, '0')],
      dtype=[('name', '<U64'), ('id', '<U64'), ('sampling_rate', '<f8'), ('dtype', '<U16'), ('units', '<U64'), ('gain', '<f8'), ('offset', '<f8'), ('stream_id', '<U64')])

It appears to me that from channel 67 onward we are a dealing with a different modality. Is this correct?

[samuelgarcia]

This channels looks differents but they belong to the same stream:

• same sampling rate
• same t_start
• same duration
  In short the same (or same group) of ADC.
  The neo do not split channels depending modality. Even it is obvisous that latest channels are onther block of channel than the first part

[h-mayorquin]

I see, what is exactly the definition of stream? The only definition that I see on the docs are:

Now the concept of a signal stream is used to handle channel groups for signals. This enhances the way the annotation and array_annotation attributes are rendered at neo.io level.

https://neo.readthedocs.io/en/latest/releases/0.10.0.html?highlight=stream#rawio-modules

That seems to be lacking something.

Btw, @samuelgarcia , the channel 67 over there has units of grams, this probably triggers the warning unecessarily in:
https://github.com/SpikeInterface/spikeinterface/blob/5c5ce219bc8f0901d9d2890f6e29e5de24f99a51/spikeinterface/extractors/neoextractors/neobaseextractor.py#L68-L71

[apdavison]

@samuelgarcia shouldn't "same units" also be part of the definition? Would these channels with units of millivolts, grams and volts be separated into separate AnalogSignals at the neo.io level?

[samuelgarcia]

@apdavison
At neo.rawio no the units is not taken in account because generally one stream is one buffer, splitting this in several sub stream per units woulb be inefficient and lots of code hard to maintain.
At neo.io level this is somehow already done. There is not the stream concept at all. Signal streams are automatically split into AnalogSignals that inherently have the same units.
I think this choice is good:

• neo.rawio : low level, fast buffer acces, not split on something logical (source type, units, same probe) only same ADC
• neo.io: high level, buffer are split on untis

@h-mayorquin I do not thing have several units in the same tream is a problem because the splitting channel on some any grouping is quite easy for any other toolbox (neo.io or spikeinterface)

[apdavison]

@samuelgarcia thanks for the clarification!

[h-mayorquin]

@samuelgarcia thanks for taking the time to clarify.

It seems that this was caused by my misunderstanding of what constitutes a stream. I also read in
https://neo.readthedocs.io/en/stable/rawio.html

That:
AnalogSignals must have the same characteristcs across all Blocks and Segments: sampling_rate, shape[1], dtype:

Which does sound similar to what you are hinting at? What is ACD?

I can add some words along the lines of the discussion here to:

https://neo.readthedocs.io/en/stable/rawio.html

If you think this would be useful and close this issue (or just close the issue).

[JuliaSprenger]

This should be addressed in #1178

[JuliaSprenger]

This might be also relevant for #1178