Randomness in calculation of FD/Motion Parameters #226

muschellij2 · 2018-07-12T13:23:30Z

I've been trying to make examples that address the reproducibility of the results, especially in cases with registration variance and randomness. I have taken some ideas from the discussion #210 and ANTsX/ANTs#444 to at least address the ability to single thread an operation (knowing that it will reduce speed).

Inconsistent values in FD (maybe others)

I've made it to about antsrMotionCalculation, which seems to give the same motion-corrected image and DVARS calculation consistently (which would fail using testthat if otherwise), but not framewise displacement (FD). The FD command is calculated on .antsMotionCorrStats0 (with stupidoff = 0), which calls antsMotionCorrStats. This is due to the motion correction parameters changing, but only the parameters 4-6. See below. If you run the command and don't get output, change the 3 to a higher number and you will see differences:

library(ANTsR)
Sys.setenv(ITK_GLOBAL_DEFAULT_NUMBER_OF_THREADS = 1)
set.seed(120)
simimg<-makeImage(rep(5,4), rnorm(5^4))
testthat::expect_equal(mean(simimg), 0.0427369860965759)

for (i in 1:3) {
  res = antsrMotionCalculation( simimg )
  res2 = antsrMotionCalculation( simimg )
  testthat::expect_equal(res$moco_img, res2$moco_img)
  testthat::expect_equal(res$dvars, res2$dvars)
  
  testthat::expect_equal(res$moco_mask, res2$moco_mask)
  
  check = all(res$moco_params == res2$moco_params)
  if (!check) {
    print(res$moco_params)
    print(res2$moco_params)
  }
  testthat::expect_equal(res$moco_params[, 1:3], res$moco_params[, 1:3])

  check = all(res$fd == res2$fd)
  if (!check) {
    print(res$fd)
    print(res2$fd)
  }
}

     MOCOparam1 MOCOparam2 MOCOparam3 MOCOparam4 MOCOparam5 MOCOparam6
[1,]  0.8070312 -2.7196524 -7.7220206   0.000000 0.00000000   0.000000
[2,]  0.4357748  0.9194809  0.3162477   1.694678 0.03968869  -1.526623
[3,] -3.5773988  0.7025864 -1.7924935   0.000000 0.00000000   0.000000
[4,]  3.4404235 -2.8812182  2.7187538   0.000000 0.00000000   0.000000
[5,]  2.5139089  0.2271850  2.9871078   0.000000 0.00000000   0.000000
     MOCOparam1 MOCOparam2 MOCOparam3 MOCOparam4 MOCOparam5 MOCOparam6
[1,]  0.8070312 -2.7196524 -7.7220206  0.0000000   0.000000  0.0000000
[2,]  0.6281858  0.7154721  0.4558244 -0.5385924  -2.039912  0.7108642
[3,] -3.5773988  0.7025864 -1.7924935  0.0000000   0.000000  0.0000000
[4,]  3.4404235 -2.8812182  2.7187538  0.0000000   0.000000  0.0000000
[5,]  2.5139089  0.2271850  2.9871078  0.0000000   0.000000  0.0000000
  MeanDisplacement MaxDisplacement
1         8.865212        8.899663
2         4.557541        4.656692
3         9.079914        9.079914
4         3.254630        3.254630
5         0.000000        0.000000
  MeanDisplacement MaxDisplacement
1         8.825520        8.914181
2         4.720911        4.789618
3         9.079914        9.079914
4         3.254630        3.254630
5         0.000000        0.000000

From the documentation, moco_params is a "Data frame of translation parameters.", but it is likely translation and rotation. It seems as though MOCOparam4-6 are likely rotation parameters, but they are not giving the same result. Any insights into what would be causing this, or is this a by-product of the above randomness. This behavior could potentially largely affect which scans people keep in an analysis. Any thoughts @stnava @ntustison @cookpa ?

The text was updated successfully, but these errors were encountered:

stnava · 2018-07-12T13:31:45Z

Have you tried setting ANTS_RANDOM_SEED and your environment?

…

On Thu, Jul 12, 2018, 09:23 John Muschelli ***@***.***> wrote: I've been trying to make examples that address the reproducibility of the results, especially in cases with registration variance and randomness. I have taken some ideas from the discussion #210 <#210> and ANTsX/ANTs#444 <ANTsX/ANTs#444> to at least address the ability to single thread an operation (knowing that it will reduce speed). Inconsistent values in FD (maybe others) I've made it to about antsrMotionCalculation, which seems to give the same motion-corrected image and DVARS calculation consistently, but not framewise displacement (FD). The FD command is calculated on .antsMotionCorrStats0 (with stupidoff = 0), which calls antsMotionCorrStats. This is due to the motion correction parameters changing, but only the parameters 4-6. See below. If you run the command and don't get output, change the 3 to a higher number and you will see differences: library(ANTsR) Sys.setenv(ITK_GLOBAL_DEFAULT_NUMBER_OF_THREADS = 1) set.seed(120)simimg<-makeImage(rep(5,4), rnorm(5^4))testthat::expect_equal(mean(simimg), 0.0427369860965759) for (i in 1:3) { res = antsrMotionCalculation( simimg ) res2 = antsrMotionCalculation( simimg ) testthat::expect_equal(res$moco_img, res2$moco_img) testthat::expect_equal(res$dvars, res2$dvars) testthat::expect_equal(res$moco_mask, res2$moco_mask) check = all(res$moco_params == res2$moco_params) if (!check) { print(res$moco_params) print(res2$moco_params) } testthat::expect_equal(res$moco_params[, 1:3], res$moco_params[, 1:3]) check = all(res$fd == res2$fd) if (!check) { print(res$fd) print(res2$fd) } } MOCOparam1 MOCOparam2 MOCOparam3 MOCOparam4 MOCOparam5 MOCOparam6 [1,] 0.8070312 -2.7196524 -7.7220206 0.000000 0.00000000 0.000000 [2,] 0.4357748 0.9194809 0.3162477 1.694678 0.03968869 -1.526623 [3,] -3.5773988 0.7025864 -1.7924935 0.000000 0.00000000 0.000000 [4,] 3.4404235 -2.8812182 2.7187538 0.000000 0.00000000 0.000000 [5,] 2.5139089 0.2271850 2.9871078 0.000000 0.00000000 0.000000 MOCOparam1 MOCOparam2 MOCOparam3 MOCOparam4 MOCOparam5 MOCOparam6 [1,] 0.8070312 -2.7196524 -7.7220206 0.0000000 0.000000 0.0000000 [2,] 0.6281858 0.7154721 0.4558244 -0.5385924 -2.039912 0.7108642 [3,] -3.5773988 0.7025864 -1.7924935 0.0000000 0.000000 0.0000000 [4,] 3.4404235 -2.8812182 2.7187538 0.0000000 0.000000 0.0000000 [5,] 2.5139089 0.2271850 2.9871078 0.0000000 0.000000 0.0000000 MeanDisplacement MaxDisplacement1 8.865212 8.8996632 4.557541 4.6566923 9.079914 9.0799144 3.254630 3.2546305 0.000000 0.000000 MeanDisplacement MaxDisplacement1 8.825520 8.9141812 4.720911 4.7896183 9.079914 9.0799144 3.254630 3.2546305 0.000000 0.000000 From the documentation, moco_params is a "Data frame of translation parameters.", but it is likely translation and rotation. It seems as though MOCOparam4-6 are likely rotation parameters, but they are not giving the same result. Any insights into what would be causing this, or is this a by-product of the above randomness. This behavior could potentially largely affect which scans people keep in an analysis. Any thoughts? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#226>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AATyfk9ZFQ19xkDPRmihdUS7RIgNpMkpks5uF03TgaJpZM4VM1y1> .

muschellij2 · 2018-07-12T13:33:23Z

No, but I will. Is this behavior expected for such a small image and only a rigid-body transform though? John

…

On Thu, Jul 12, 2018 at 9:31 AM, stnava ***@***.***> wrote: Have you tried setting ANTS_RANDOM_SEED and your environment? On Thu, Jul 12, 2018, 09:23 John Muschelli ***@***.***> wrote: > I've been trying to make examples that address the reproducibility of the > results, especially in cases with registration variance and randomness. I > have taken some ideas from the discussion #210 > <#210> and ANTsX/ANTs#444 > <ANTsX/ANTs#444> to at least address the > ability to single thread an operation (knowing that it will reduce speed). > Inconsistent values in FD (maybe others) > > I've made it to about antsrMotionCalculation, which seems to give the > same motion-corrected image and DVARS calculation consistently, but not > framewise displacement (FD). The FD command is calculated on > .antsMotionCorrStats0 (with stupidoff = 0), which calls > antsMotionCorrStats. This is due to the motion correction parameters > changing, but only the parameters 4-6. See below. If you run the command > and don't get output, change the 3 to a higher number and you will see > differences: > > library(ANTsR) > Sys.setenv(ITK_GLOBAL_DEFAULT_NUMBER_OF_THREADS = 1) > set.seed(120)simimg<-makeImage(rep(5,4), rnorm(5^4))testthat::expect_equal(mean(simimg), 0.0427369860965759) > for (i in 1:3) { > res = antsrMotionCalculation( simimg ) > res2 = antsrMotionCalculation( simimg ) > testthat::expect_equal(res$moco_img, res2$moco_img) > testthat::expect_equal(res$dvars, res2$dvars) > > testthat::expect_equal(res$moco_mask, res2$moco_mask) > > check = all(res$moco_params == res2$moco_params) > if (!check) { > print(res$moco_params) > print(res2$moco_params) > } > testthat::expect_equal(res$moco_params[, 1:3], res$moco_params[, 1:3]) > > check = all(res$fd == res2$fd) > if (!check) { > print(res$fd) > print(res2$fd) > } > } > > MOCOparam1 MOCOparam2 MOCOparam3 MOCOparam4 MOCOparam5 MOCOparam6 > [1,] 0.8070312 -2.7196524 -7.7220206 0.000000 0.00000000 0.000000 > [2,] 0.4357748 0.9194809 0.3162477 1.694678 0.03968869 -1.526623 > [3,] -3.5773988 0.7025864 -1.7924935 0.000000 0.00000000 0.000000 > [4,] 3.4404235 -2.8812182 2.7187538 0.000000 0.00000000 0.000000 > [5,] 2.5139089 0.2271850 2.9871078 0.000000 0.00000000 0.000000 > MOCOparam1 MOCOparam2 MOCOparam3 MOCOparam4 MOCOparam5 MOCOparam6 > [1,] 0.8070312 -2.7196524 -7.7220206 0.0000000 0.000000 0.0000000 > [2,] 0.6281858 0.7154721 0.4558244 -0.5385924 -2.039912 0.7108642 > [3,] -3.5773988 0.7025864 -1.7924935 0.0000000 0.000000 0.0000000 > [4,] 3.4404235 -2.8812182 2.7187538 0.0000000 0.000000 0.0000000 > [5,] 2.5139089 0.2271850 2.9871078 0.0000000 0.000000 0.0000000 > MeanDisplacement MaxDisplacement1 8.865212 8.8996632 4.557541 4.6566923 9.079914 9.0799144 3.254630 3.2546305 0.000000 0.000000 > MeanDisplacement MaxDisplacement1 8.825520 8.9141812 4.720911 4.7896183 9.079914 9.0799144 3.254630 3.2546305 0.000000 0.000000 > > From the documentation, moco_params is a "Data frame of translation > parameters.", but it is likely translation and rotation. It seems as though > MOCOparam4-6 are likely rotation parameters, but they are not giving the > same result. Any insights into what would be causing this, or is this a > by-product of the above randomness. This behavior could potentially largely > affect which scans people keep in an analysis. Any thoughts? > > — > You are receiving this because you are subscribed to this thread. > Reply to this email directly, view it on GitHub > <#226>, or mute the thread > <https://github.com/notifications/unsubscribe-auth/ AATyfk9ZFQ19xkDPRmihdUS7RIgNpMkpks5uF03TgaJpZM4VM1y1> > . > — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#226 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABBnrkBf369tAuF5pm_fUrF_TJVtszlOks5uF0_DgaJpZM4VM1y1> .

stnava · 2018-07-12T13:35:32Z

Maybe .

…

On Thu, Jul 12, 2018, 09:33 John Muschelli ***@***.***> wrote: No, but I will. Is this behavior expected for such a small image and only a rigid-body transform though? John On Thu, Jul 12, 2018 at 9:31 AM, stnava ***@***.***> wrote: > Have you tried setting ANTS_RANDOM_SEED and your environment? > > On Thu, Jul 12, 2018, 09:23 John Muschelli ***@***.***> > wrote: > > > I've been trying to make examples that address the reproducibility of the > > results, especially in cases with registration variance and randomness. I > > have taken some ideas from the discussion #210 > > <#210> and ANTsX/ANTs#444 > > <ANTsX/ANTs#444> to at least address the > > ability to single thread an operation (knowing that it will reduce > speed). > > Inconsistent values in FD (maybe others) > > > > I've made it to about antsrMotionCalculation, which seems to give the > > same motion-corrected image and DVARS calculation consistently, but not > > framewise displacement (FD). The FD command is calculated on > > .antsMotionCorrStats0 (with stupidoff = 0), which calls > > antsMotionCorrStats. This is due to the motion correction parameters > > changing, but only the parameters 4-6. See below. If you run the command > > and don't get output, change the 3 to a higher number and you will see > > differences: > > > > library(ANTsR) > > Sys.setenv(ITK_GLOBAL_DEFAULT_NUMBER_OF_THREADS = 1) > > set.seed(120)simimg<-makeImage(rep(5,4), rnorm(5^4))testthat::expect_equal(mean(simimg), > 0.0427369860965759) > > for (i in 1:3) { > > res = antsrMotionCalculation( simimg ) > > res2 = antsrMotionCalculation( simimg ) > > testthat::expect_equal(res$moco_img, res2$moco_img) > > testthat::expect_equal(res$dvars, res2$dvars) > > > > testthat::expect_equal(res$moco_mask, res2$moco_mask) > > > > check = all(res$moco_params == res2$moco_params) > > if (!check) { > > print(res$moco_params) > > print(res2$moco_params) > > } > > testthat::expect_equal(res$moco_params[, 1:3], res$moco_params[, 1:3]) > > > > check = all(res$fd == res2$fd) > > if (!check) { > > print(res$fd) > > print(res2$fd) > > } > > } > > > > MOCOparam1 MOCOparam2 MOCOparam3 MOCOparam4 MOCOparam5 MOCOparam6 > > [1,] 0.8070312 -2.7196524 -7.7220206 0.000000 0.00000000 0.000000 > > [2,] 0.4357748 0.9194809 0.3162477 1.694678 0.03968869 -1.526623 > > [3,] -3.5773988 0.7025864 -1.7924935 0.000000 0.00000000 0.000000 > > [4,] 3.4404235 -2.8812182 2.7187538 0.000000 0.00000000 0.000000 > > [5,] 2.5139089 0.2271850 2.9871078 0.000000 0.00000000 0.000000 > > MOCOparam1 MOCOparam2 MOCOparam3 MOCOparam4 MOCOparam5 MOCOparam6 > > [1,] 0.8070312 -2.7196524 -7.7220206 0.0000000 0.000000 0.0000000 > > [2,] 0.6281858 0.7154721 0.4558244 -0.5385924 -2.039912 0.7108642 > > [3,] -3.5773988 0.7025864 -1.7924935 0.0000000 0.000000 0.0000000 > > [4,] 3.4404235 -2.8812182 2.7187538 0.0000000 0.000000 0.0000000 > > [5,] 2.5139089 0.2271850 2.9871078 0.0000000 0.000000 0.0000000 > > MeanDisplacement MaxDisplacement1 8.865212 8.8996632 4.557541 4.6566923 > 9.079914 9.0799144 3.254630 3.2546305 0.000000 0.000000 > > MeanDisplacement MaxDisplacement1 8.825520 8.9141812 4.720911 4.7896183 > 9.079914 9.0799144 3.254630 3.2546305 0.000000 0.000000 > > > > From the documentation, moco_params is a "Data frame of translation > > parameters.", but it is likely translation and rotation. It seems as > though > > MOCOparam4-6 are likely rotation parameters, but they are not giving the > > same result. Any insights into what would be causing this, or is this a > > by-product of the above randomness. This behavior could potentially > largely > > affect which scans people keep in an analysis. Any thoughts? > > > > — > > You are receiving this because you are subscribed to this thread. > > Reply to this email directly, view it on GitHub > > <#226>, or mute the thread > > <https://github.com/notifications/unsubscribe-auth/ > AATyfk9ZFQ19xkDPRmihdUS7RIgNpMkpks5uF03TgaJpZM4VM1y1> > > . > > > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > <#226 (comment)>, or mute > the thread > < https://github.com/notifications/unsubscribe-auth/ABBnrkBf369tAuF5pm_fUrF_TJVtszlOks5uF0_DgaJpZM4VM1y1 > > . > — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#226 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AATyfsY5wwO2vMETuK0I5sNHnTczO2DVks5uF1AkgaJpZM4VM1y1> .

muschellij2 · 2018-07-12T13:44:54Z

Setting that seed resulted in the same output consistently, but this is still an FD change of 4% (0.2/4.5 of mean displacement) is large when not set.

stnava · 2018-07-12T14:15:31Z

Good.

…

On Thu, Jul 12, 2018, 09:44 John Muschelli ***@***.***> wrote: Setting that seed resulted in the same output consistently, but this is still an FD change of 4% (0.2/4.5 of mean displacement) is large. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#226 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AATyfhDCSFfYECYpXaebnVpz9vmVCVJuks5uF1LWgaJpZM4VM1y1> .

muschellij2 · 2018-07-16T23:54:47Z

After changes in #221 are merged, this should be able to be closed.

muschellij2 mentioned this issue Jul 16, 2018

Reproducible kmeans segmentation ANTsX/ANTsRCore#51

Closed

stnava closed this as completed Jul 21, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Randomness in calculation of FD/Motion Parameters #226

Randomness in calculation of FD/Motion Parameters #226

muschellij2 commented Jul 12, 2018 •

edited

Loading

stnava commented Jul 12, 2018 via email

muschellij2 commented Jul 12, 2018 via email

stnava commented Jul 12, 2018 via email

muschellij2 commented Jul 12, 2018 •

edited

Loading

stnava commented Jul 12, 2018 via email

muschellij2 commented Jul 16, 2018

Randomness in calculation of FD/Motion Parameters #226

Randomness in calculation of FD/Motion Parameters #226

Comments

muschellij2 commented Jul 12, 2018 • edited Loading

Inconsistent values in FD (maybe others)

stnava commented Jul 12, 2018 via email

muschellij2 commented Jul 12, 2018 via email

stnava commented Jul 12, 2018 via email

muschellij2 commented Jul 12, 2018 • edited Loading

stnava commented Jul 12, 2018 via email

muschellij2 commented Jul 16, 2018

muschellij2 commented Jul 12, 2018 •

edited

Loading

muschellij2 commented Jul 12, 2018 •

edited

Loading