Validation against VoxelMorph

[YipengHu]

https://github.com/voxelmorph/voxelmorph

What are the differences?
Anything in VM is not supported in deepreg?

[mathpluscode]

It can be worth to reproduce the results and compare the metrics against VM! @acasamitjana is it in part of your plan already?

[acasamitjana]

Yes, it was one of the models I wanted to compare with.

[acasamitjana]

They recently published some work for hyperparameter tuning. Was anyone working on it? https://arxiv.org/pdf/2101.01035.pdf

[mathpluscode]

wow, I think using open-sourced data, and benchmark with published paper can be an entirely independent project! what do you think @acasamitjana @YipengHu

When I say project, I mean https://github.com/DeepRegNet/DeepReg/projects

[YipengHu]

They recently published some work for hyperparameter tuning. Was anyone working on it? https://arxiv.org/pdf/2101.01035.pdf

This is interesting but i think a bigger problem (outside of neuroimaging maybe) is that

• the goal of registration is not exactly minimising the loss, i.e. to get the absolutely smallest loss possible
• regularisation is very very important in many applications to prevent non-physical deformation
• you can often get a smaller overall loss by using a smaller weight on regularisation
• but doing so, there is no guarantee the amount of regularisation (therefore the resulting deformation) is physically plausible
• one can, however, to have a validation set such the metrics representing independent TREs are optimised in a meta-learning style tuning, but this approach usually requires (preferably task-level-independent) labels - something rarely available in medical imaging applications

[acasamitjana]

Yes, I agree the trade-off between image similarity and regularization is very application-dependent and the minimum is not necesseraly the optimal for registration. However, (semi-)automatic methods in this line may be helpful to, at least, get some intuition on the hyperparameter values.

Also, I mentioned this because I thought someone in the team was interested/working on hyperparameter tuning.

[mathpluscode]

For further discussions, please go to https://github.com/DeepRegNet/Benchmark