Sharing mesoscale calcium imaging in nwb format

[DaveOC90]

Hi All,

I would like to share mesoscale optical calcium imaging data on dandi, but after reading the docs I am unsure about the best way of adhering to nwb. Apologies in advance for the long post.

TLDR; whats the best way to share large amounts of 2D x Time optical data (currently in nifti format) using nwb.

More info:

Data Description
The data are currently in nifti format, and in a BIDs like organization, that is to say, they are organized by subject id, session, and run (afaik there is no bids standard for these type of data yet). The data are 2D x Time. We have at least 6 runs/scans per session, and around 31 sessions. The 10 minute runs can yield ~5.5 -6 GB of data so they are usually split into three tif files when output (we converted to nii). A further complication is that they were acquired using a dual wavelength protocol, which means signal sensitive frames were interleaved with signal insensitive frames, We have split each tif file into two separate nii files. So for now we have 6 image files (as nii) per run, and at least 6 runs per session, and 31 sessions across 23 subjects.

Solution?
It seemed like an image series class object might be best for these, but I dont know how to resolve that with the requirement for all session data to be contained in one nwb file (am I mistaken?). Can I link to external nii files from an nwb file and keep the nii files as is? I am not wedded to nii, and happy to convert the data, I am just concerned about aggregating a lot of imaging data into one file, our computers may struggle with it. I am also pretty new to hdf5 so I may be missing something obvious.

Thanks for any help you can offer.

Dave

[CodyCBakerPhD]

Hi Dave, I'm glad to hear you're interested in using NWB and DANDI to share your data! If you haven't seen it yet, I would recommend starting out with a quick glance through the NWB Overview.

I am not wedded to nii, and happy to convert the data, I am just concerned about aggregating a lot of imaging data into one file

To share the data on the DANDIArchive, this data must be in NWB format: please reach out on the NWB Helpdesk if you have any further questions.

and in a BIDs like organization,

Thankfully, one of the final steps before performing the dandi upload is calling dandi organize on your NWBFiles, which will automatically structure the folder into the way DANDI requires.

our computers may struggle with it. I am also pretty new to hdf5 so I may be missing something obvious.

The great thing about NWB files (indeed, thanks to the HDF5 backend) is that all data access is performed lazily; that is, it only loads particular selections of data on-demand. You can open the file and explore its high-level contents without overwhelming your RAM.

Another great thing about NWB is we make it easy to use lossless compression on the data contained in the file, so all your TIFF stacks ought to be reduced 20%-30% in total size (based on past personal experience) without any change to the data itself, which can free up a lot of space on your hard drives.

It seemed like an image series class object might be best for these,

Assuming this is 2-photon data (sorry if you mentioned that above and I missed it), I would recommend the TwoPhotonSeries type. If not, then I would try to make every attempt to use the other types in the Optical Physiology section of the schema.

As for the frame-splitting, I'm not 100% sure, but I would probably just have each signal-sensitive condition added as a separate series. Might be worth asking over among the ophys experts on the NWB side.

but I dont know how to resolve that with the requirement for all session data to be contained in one nwb file (am I mistaken?).

Indeed, a single NWBFile typically contains all of the data associated with a single session's worth of data. Also, one very important thing to ensure is that all time references throughout the file are synchronized to the session_start_time field. You can find other similar suggestions at the NWB Best Practices page.

We're here to help throughout this process, so again, if you have any other questions, please don't hesitate to reach out at the NWB Helpdesk or anywhere else more specific in the NWB ecosystem.

Best,
Cody Baker, PhD
Sr. Neuro-data Scientist at CatalystNeuro

[DaveOC90]

Hi Cody,

Thanks for you response. I still have questions, but I realize the nwb helpdesk is more appropriate for this enquiry so I will post there.

Cheers,

Dave