Sometimes inv_metric does not return one metric per chain, sometimes metrics are incomplete

[maxmantei]

Describe the bug
The inv_metric() method sometimes does not return the inverse metric for each chain. For some chains it might be missing. Also, when using "dense_e" sometimes the inverse metric, which is returned by inv_metric is just a vector for some of the chains.

To Reproduce
Sometimes the inverse metric is missing for some chains. Note that these examples are all with "dense_e", but I first encountered the issue using the default "diag_e" (on a bigger model). The following "test" mostly runs fine with "diag_e".

for (i in 1:100){
    M <- cmdstanr_example(method = "sample", metric = "dense_e")$inv_metric()
    if (length(M) != 4){
        print(paste("Try:", i))
        print(M)
        break
    }
}

The code above returned this for me:

[1] "Try: 4"
$`1`
             [,1]        [,2]        [,3]         [,4]
[1,]  0.040256200 -0.00335270  0.00198159  0.000593459
[2,] -0.003352700  0.05839920 -0.00385252  0.001137230
[3,]  0.001981590 -0.00385252  0.05351410 -0.015615500
[4,]  0.000593459  0.00113723 -0.01561550  0.080952100

$`2`
            [,1]         [,2]         [,3]       [,4]
[1,]  0.04249720 -0.006099360  0.002226680  0.0037958
[2,] -0.00609936  0.058991800 -0.000284421 -0.0126412
[3,]  0.00222668 -0.000284421  0.047187900 -0.0109423
[4,]  0.00379580 -0.012641200 -0.010942300  0.0670333

$`3`
             [,1]         [,2]        [,3]         [,4]
[1,]  0.043669300 -0.003417680  0.00840543 -0.000542557
[2,] -0.003417680  0.066449300 -0.00681561  0.000112086
[3,]  0.008405430 -0.006815610  0.05900440 -0.017976800
[4,] -0.000542557  0.000112086 -0.01797680  0.064814900

The inverse metric of chain 4 is missing.

Also, sometimes only a vector is returned. This is probably only a problem with "dense_e". A slightly modified loop:

for (i in 1:100){
    M <- cmdstanr_example(method = "sample", metric = "dense_e")$inv_metric()
    if (length(M) == 4 & length(unlist(M)) != 4^3){ # specifically fishing for incomplete metrics
        print(paste("Try:", i))
        print(M)
        break
    }
}

would return:

[1] "Try: 28"
$`1`
            [,1]        [,2]        [,3]        [,4]
[1,]  0.04526390 -0.00109848 -0.00569870  0.00195247
[2,] -0.00109848  0.06756560  0.00723948 -0.01093820
[3,] -0.00569870  0.00723948  0.05556670 -0.01843860
[4,]  0.00195247 -0.01093820 -0.01843860  0.07086560

$`2`
            [,1]        [,2]        [,3]        [,4]
[1,]  0.04910670  0.00248248 -0.00133962  0.00178766
[2,]  0.00248248  0.05477440 -0.00463148 -0.00226593
[3,] -0.00133962 -0.00463148  0.05054930 -0.01137160
[4,]  0.00178766 -0.00226593 -0.01137160  0.06099220

$`3`
          [,1]        [,2]        [,3]      [,4]
[1,] 0.0423699 -0.00359629 -0.00259332 0.0012797

$`4`
            [,1]        [,2]        [,3]        [,4]
[1,]  0.04071840 -0.00758207  0.00105500  0.00323377
[2,] -0.00758207  0.06613060  0.00640958 -0.01011680
[3,]  0.00105500  0.00640958  0.06165830 -0.01464940
[4,]  0.00323377 -0.01011680 -0.01464940  0.07564350

Note that for chain 3 it only returns the first row of the matrix (not the diagonal).

In combination this leads to either missing, or incomplete inverse metrics relatively frequently.

Expected behavior
I'd expect inv_matrix to always return an inverse metric for all the chains that were run. I'd also expect the resulting matrix to be a full matrix if a dense metric was used in fitting.

Operating system
Ubuntu 18.04 (through WSL2 on Windows 10)

CmdStanR version number

> packageVersion("cmdstanr")
[1] ‘0.2.2’

[rok-cesnovar]

Thanks, Max for reporting with an easily repeatable example. The problem is with values in scientific format:

# Elements of inverse mass matrix:
# 0.0442316, -0.000878945, **6.14707e-05**, -0.00134108
# -0.000878945, 0.0583718, **-4.91874e-05**, -0.00825635
# **6.14707e-05**, **-4.91874e-05**, 0.046496, -0.0169065
# -0.00134108, -0.00825635, -0.0169065, 0.0691499

It seems that it doesn't read lines in scientific format.

[jgabry]

Thanks, Max for reporting with an easily repeatable example.

Indeed!

The problem is with values in scientific format

Thanks for figuring it out so quickly. Do you know if this is only a potential problem with inv_metric or is this something that could come up in other places and we've been lucky not to see it yet?

[rok-cesnovar]

Only here, because the inv metric has this weird output format and we need special handling for it (biggest problem being we dont know when it ends).