Move metadata to Subject and extend for age_in_days attribute

[pickles-bread-and-butter]

Included subject information in EcephysLabMetaData object. This information would be better in Subject. In fact, there are already fields for "sex", "strain", "genotype", and "date_of_birth" in the Subject object. If you want to pre-compute the 'age_in_days" and provide that as an attribute, the best way to do that would be to extend Subject.

AllenSDK/allensdk/brain_observatory/ecephys/nwb/AIBS_ecephys_extension.yaml

Lines 29 to 50 in a040661

	- neurodata_type_def: EcephysLabMetaData
	neurodata_type_inc: LabMetaData
	doc: "metadata for ecephys sessions"
	attributes:
	- name: specimen_name
	dtype: text
	doc: full name of this specimen
	- name: age_in_days
	dtype: float
	doc: age of this subject, in days
	- name: full_genotype
	dtype: text
	doc: "long-form description of this subjects genotype"
	- name: strain
	dtype: text
	doc: "this subjects strain"
	- name: sex
	dtype: text
	doc: "this subjects sex"
	- name: stimulus_name
	dtype: text
	doc: "the name of the stimulus set used for this session"

Split off from: #1330

[pickles-bread-and-butter]

@bendichter what subject are you referring to in the above? Is a subject object being saved to the NWB file and you're referring to moving the age_in_days to this object?

[rly]

@isaak-willett PyNWB allows you to save a pynwb.file.Subject object to the NWB file: https://pynwb.readthedocs.io/en/latest/pynwb.file.html#pynwb.file.Subject

specimen_name -> subject_id (I think?)
age_in_days -> age -- I would specify in the description that the age is "in days". (We are working on a better way to standardize age values.)
full_genotype -> genotype
sex -> sex

The strain field is not yet included on Subject types in NWB, but you could create an extension of the Subject type with a strain field:

- neurodata_type_def: AIBSSubject
  neurodata_type_inc: Subject
  doc: "Allen Institute for Brain Science subject with strain information"
  attributes:
  - name: strain
    dtype: text
    doc: "the subject's strain"

Then, the EcephysLabMetaData type would just have:

 - neurodata_type_def: EcephysLabMetaData 
   neurodata_type_inc: LabMetaData 
   doc: "Metadata for ecephys sessions" 
   attributes: 
   - name: stimulus_name 
     dtype: text 
     doc: "the name of the stimulus set used for this session"

Then, with the changes in #1425, https://github.com/AllenInstitute/AllenSDK/blob/internal/allensdk/brain_observatory/ecephys/nwb/__init__.py could be simplified to:

import os

from pynwb import load_namespaces, get_class

namespace_path = os.path.abspath(os.path.join(os.path.dirname(__file__), "AIBS_ecephys_namespace.yaml"))
load_namespaces(namespace_path)

EcephysProbe = get_class('EcephysProbe', 'AIBS_ecephys')
EcephysLabMetaData = get_class('EcephysLabMetaData', 'AIBS_ecephys')
AIBSSubject = get_class('AIBSSubject', 'AIBS_ecephys')

def to_dict(self):
    out = {}
    for key in self.__nwbfields__:
        if key != 'name':
            out[key] = getattr(self, key)
    return out


EcephysLabMetaData.to_dict = to_dict

Note the addition of the line AIBSSubject = get_class('AIBSSubject', 'AIBS_ecephys'). The new class AIBSSubject would be used throughout the allensdk wherever pynwb.file.Subject was used previously, i.e., pynwb.subject = AIBSSubject(...). And any code that references the subject metadata that was previously in EcephysLabMetaData would have to be updated to refer to nwbfile.subject instead.

Let me know if that doesn't make sense...

[bendichter]

I agree with everything @rly has said here, but I'd go with more of a step-wise approach.

To me, the highest priority issue here is that there are several fields that were added to the AIBS extension that already exist in the Subject object. This is a problem because programs that are searching for information like age or sex will look in the Subject object for this information, not in the custom AIBS extension. Let's start with those. You write these with:

from datetime import datetime
from dateutil.tz import tzlocal
from pynwb import NWBFile, Subject

nwbfile = NWBFile('session_description','identifier',datetime.now(tzlocal())) 
nwbfile.subject = Subject(subject_id="this_subject_id", age="age in days", genotype="this_genotype", sex='M')

and read with, e.g.

nwbfile.subject.age 
'age in days'

That's simple enough, but I think the challenge here is going to be making sure the allensdk can handle data in this new location and in the old location, ensuring that you can make this change while still being able to read data that you have already released. I'd be interested to hear your thoughts on how you might implement this.

Then there are two issues that I see as orthogonal/separable:

Then there is the issue of auto-generating the class vs. manually generating the class. I agree this would be better as auto than manual, which is what it currently is, but let's save that for a separate issue.

There is also the idea of extending NWB's Subject to include additional subject-specific meta-data. This would be preferable to what is done now, which is adding the info to the generic lab metadata extension, but to me this is a lower priority, since we would not expect this to be uniform across groups anyway, or to be queried by programs outside of Allen.

[njmei]

Resolved by: #1549