This documentation describes how to use our pre-trained model to segment the left and right hippocampi from brain MRI scans. The model was trained on data from the ADNI dataset (for more information see the model card below). This data is publicly available via their website, but users must sign a Data Use Agreement in order to gain access. We do not provide access to the data. The following description assumes the user has their own dataset to evaluate/ retrain the model on.
Please note that this model is intended for research purposes only. You are responsible for the performance, the necessary testing, and if needed any regulatory clearance for any of the models produced by this toolbox.
The following instructions assume you have completed the preceding setup steps in the InnerEye README, in particular, Setting up Azure Machine Learning.
To evaluate this model on your own data, you will first need to register an Azure ML Dataset. You can follow the instructions in the InnerEye repo for creating datasets in order to do this.
The saved weights from the trained Hippocampus model can be downloaded along with the source code used to train it from our github releases page.
To evaluate the model in Azure ML, you must first resgister an Azure ML Model. To register the Hippocampus model in your AML Workspace, unpack the source code downloaded in the previous step and follow InnerEye's instructions to upload models to Azure ML.
WORKSPACE="fill with your workspace name"
GROUP="fill with your resource group name"
SUBSCRIPTION="fill with your subscription ID"
python InnerEye/Scripts/move_model.py \
--action upload \
--path . \
--workspace_name $WORKSPACE \
--resource_group $GROUP \
--subscription_id $SUBSCRIPTION \
--model_id Hippocampus:118You can evaluate the model either in Azure ML or locally using the downloaded checkpoint files. These 2 scenarios are described in more detail, along with instructions in testing an existing model.
For example, to evalute the model on your Dataset in Azure ML:
CLUSTER="fill with your cluster name"
DATASET_ID="fill with your dataset name"
python InnerEye/ML/runner.py \
--azure_dataset_id $DATASET_ID \
--model Hippocampus \
--model_id Hippocampus:111 \
--experiment_name evaluate_hippocampus_model \
--azureml \
--no-train \
--cluster $CLUSTER
--restrict_subjects=0,0,+It is possible to apply connected compoents as a post-processing step, although by default this is disabled. To enable, update the property largest_connected_component_foreground_classes of the Hippocampus class in InnerEye/ML/configs/segmentation/Hippocampus.py
To deploy this model, see the instructions in the InnerEye README.
- Organiation: Biomedical Imaging Team at Micrsoft Research, Cambridge UK.
- Model date: 5th July 2022.
- Model version: 0.1 . Currently only one version of the model has been released. We do not expect to update the model frequently.
- Model type: 3d UNET ensemble of 3d UNet. Training details are as described in this paper.
- Training details: 5 fold ensemble model. Trained on a subsection of the ADNI dataset (described in detail below)
- License: The model is released under MIT license as described here.
- Contect: innereyeinfo@microsoft.com.
This model has been trained on a subset of the ADNI dataset. There have been various phases of ADNI spanning different time periods. In this Model Card we refer to the original, or ADNI 1, stud.y This dataset comprises scans and metadata from patients between the ages of 55-90 from 57 different sites across the US and Canada source. Therefore a major limitation of this model would be the ability to generalise to patients outside of this demographic. Another limitation is that The MRI protocol for ADNI1 (which was collected between 2004-2009) focused on imaging on 1.5T scanners source. Modern scanners may have higher field strengths and therefore different levels of contrast which could lead to different performance from the results we report.
The results of this model have not been validated by clinical experts. We expect the user to evaluate the result
This model is for research purposes only. It is intended to be used for the task of segmentating hippocampi from brain MRI scans. Any other task is out of scope for this model.
The model was trained on 998 pairs of MRI segmentation + segmentation. The model was further validated on 127 pairs of images and tested on 125 pairs. A further 317 pairs were retained as a held-out test set for the final evaluation of the model, which is what we report performance on.
All of this data comes from the Alzheimers Disease Neuroimaging Initiative study link to website. The data is publicly available, but requires signing a Data Use Agreement before access is granted.
The segmentations were also downloaded from the ADNI dataset. They were created semi-automatically using software from Medtronic Surgical Navigation Technologies. Further information is available on the ADNI website.
Note that due to the ADNI Data Usage Agreement we are only able to share aggregate-level metrics from our evaluation. Evaluation is performed on a heldout test set of 252 pairs of MRI + segmentation pairs from the ADNI dataset.
Dice Score for Left and Right hippocampus respectively on a held-out test set of :
| Structure | count | DiceNumeric_mean | DiceNumeric_std | DiceNumeric_min | DiceNumeric_max | HausdorffDistance_mm_mean | HausdorffDistance_mm_std | HausdorffDistance_mm_min | HausdorffDistance_mm_max | MeanDistance_mm_mean | MeanDistance_mm_std | MeanDistance_mm_min | MeanDistance_mm_max |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| hippocampus_L | 252 | 0.918 | 0.022 | 0.819 | 0.953 | 2.206 | 0.812 | 1.206 | 6.964 | 0.168 | 0.054 | 0.096 | 0.399 |
| hippocampus_R | 252 | 0.918 | 0.024 | 0.816 | 0.954 | 2.185 | 0.706 | 1.206 | 4.966 | 0.168 | 0.055 | 0.094 | 0.433 |