Error downloading dandi dataset - cannot import name 'NWBFile' from 'pynwb'

[magland]

Hi, I am trying to download this dataset:

https://dandiarchive.org/dandiset/000053

I installed dandi CLI using pip (I also tried with conda and ran into a different problem).

I am using Linux.

Here's the error:

$ dandi download DANDI:000053/0.210819.0345
Error: cannot import name 'NWBFile' from 'pynwb' (unknown location)

Thanks!

[satra]

if not already in a new virtual/conda env, would it be possible to try in a clean environment? seems like one of the two packages was not installed properly.

[magland]

Thanks @satra . That worked.

But now I am realizing the dataset is too large for me to download (> 1TB). Is there a way to load a subset of the ephys recording (I'm interested in the raw ephys traces).

tag: @bendichter

[satra]

yes, a few ways:

1. use datalad : datalad clone https://github.com/dandisets/000053 - this creates a shall clone of the structure. and you can fetch any particular file with datalad get and remove it with datalad drop. you can also try to read it without get/drop if you install datalad fuse, which will fetch any bytes behind the scenes for you.
2. you use the dandi python api to read any file. also one can read any nwb file in a streaming manner. see: https://pynwb.readthedocs.io/en/stable/tutorials/advanced_io/streaming.html
3. you can also use the client to download any specific file by giving it the url.

[magland]

Thanks @satra!
For reference, I am downloading one of the smaller files (~75 GB) using this command

dandi download https://api.dandiarchive.org/api/assets/2d8d2eb4-dff2-4bfd-998c-5d6dc1c5a611/download/

While that's downloading, I will check out the streaming option using the Python API.

[yarikoptic]

@magland might be of interest a convenience we provided recently within dandi to stream those remote assets https://dandi.readthedocs.io/en/latest/modref/dandiapi.html#dandi.dandiapi.BaseRemoteBlobAsset.as_readable which would return our tiny shim using fsspec (https://github.com/dandi/dandi-cli/blob/master/dandi/misctypes.py#L319) if you are to work with them in Python and do not need/care about local caching etc.

edit: adding example now at dandi/dandi-cli#1250