participants.tsv files are erroneously expected to always have more than one column

[SebastianSpeer]

Dear MNE BIDS team,

I'm trying to read in data via mne-bids. Everything works fine until the channel locations are read in.

This is what my channels tsv file looks like:

This is the error message I'm receiving:

> ---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-77-d54a13bf5b31> in <module>
      4 bids_path = BIDSPath(subject=subject, task ='xxxx',suffix= suffix,
      5                       datatype=datatype, root=bids_root)
----> 6 raw = read_raw_bids(bids_path=bids_path, verbose=False)

~/anaconda3/lib/python3.8/site-packages/mne_bids/read.py in read_raw_bids(bids_path, extra_params, verbose)
    711     subject = f"sub-{bids_path.subject}"
    712     if op.exists(participants_tsv_fpath):
--> 713         raw = _handle_participants_reading(participants_tsv_fpath, raw,
    714                                            subject, verbose=verbose)
    715     else:

~/anaconda3/lib/python3.8/site-packages/mne_bids/read.py in _handle_participants_reading(participants_fname, raw, subject, verbose)
    172 def _handle_participants_reading(participants_fname, raw,
    173                                  subject, verbose=None):
--> 174     participants_tsv = _from_tsv(participants_fname)
    175     subjects = participants_tsv['participant_id']
    176     row_ind = subjects.index(subject)

~/anaconda3/lib/python3.8/site-packages/mne_bids/tsv_handler.py in _from_tsv(fname, dtypes)
    138     data = np.loadtxt(fname, dtype=str, delimiter='\t',
    139                       comments=None, encoding='utf-8-sig')
--> 140     column_names = data[0, :]
    141     info = data[1:, :]
    142     data_dict = OrderedDict()

IndexError: too many indices for array: array is 1-dimensional, but 2 were indexed

Could this issue be due to an incorrectly formatted tsv file?

[welcome]

Hello! 👋 Thanks for opening your first issue here! ❤️ We will try to get back to you soon. 🚴🏽‍♂️

[adam2392]

Hi @SebastianSpeer , it seems your error message is due to the participants.tsv file actually. Can you share that file here?

[SebastianSpeer]

@adam2392 , This is what it looks like:

github does not support tsv files so I can't upload it as such. I can upload it as csv if needed?

[sappelhoff]

Thanks @SebastianSpeer, I think this is a bug on our side. We somehow always expected participants.tsv files to contain at least two columns (ID, and sex, ... or age, ...), but having only the ID column is definitely valid.

In [3]: data = np.loadtxt("no.tsv", dtype=str, delimiter="\t", comments=None, en
   ...: coding="utf-8-sig")

In [4]: data
Out[4]: array(['participant_id', 'sub-01', 'sub-02', 'sub-03'], dtype='<U14')

In [5]: data.shape
Out[5]: (4,) <-- We are expecting a 2D shape here

In [6]: cat no.tsv
participant_id
sub-01
sub-02
sub-03

The fast fix for us would be to use something like np.atleast_2d in the code, I think

[sappelhoff]

@SebastianSpeer do you perhaps want to prepare a fix and make a pull request?

[sappelhoff]

Oups, I forgot that this was already fixed in #886

[smalltimer]

@SebastianSpeer This will be fixed in the next version bump. Until then, visit the file mne_bids/tsv_handler.py and change the line:

data = np.loadtxt(fname, dtype=str, delimiter='\t', 
                      comments=None, encoding='utf-8-sig')

to:

data = np.loadtxt(fname, dtype=str, delimiter='\t', ndmin=2,
                      comments=None, encoding='utf-8-sig')

Voilà.

[skjerns]

I'm working (presumably) on the same dataset (EEGManyPipelines study), and there is a further error with this .tsv file that is not fixed with simply loading the data with ndim=2:

  File "<decorator-gen-584>", line 24, in make_report

  File "c:\users\user\anaconda3\lib\site-packages\mne_bids\report.py", line 500, in make_report
    participant_summary = _summarize_participants_tsv(root)

  File "c:\users\user\anaconda3\lib\site-packages\mne_bids\report.py", line 232, in _summarize_participants_tsv
    n_age_unknown = len(p_ages)

TypeError: object of type 'NoneType' has no len()

However, fixing this did not fix the problem with loading the dataset, as the participants contain -, which appareantly is not allowed? Seems like this dataset is not up to BIDS standards?

ValueError: Unallowed `-`, `_`, or `/` found in key/value pair subject: sub-022