Error in tergm_MCMC_slave(state, eta.comb, control, verbose) : MCMCDyn failed with error code 3

[RodrigoAnderle]

Hi Everyone,
I am running a network model with Epimodel package. This package use tergm functions for estimation, diagnostics and epidemiological modeling.
I have generate a network model with the function netest(), and start the diagnostics with netd function, when I got this error.
I didn't find any reference about what means "code 3".

I try to debug a little, but I have no clue about what could be. Any ideas?

My network information:
EpiModel Network Estimation

Model class: netest
Estimation Method: ERGM with Edges Approximation

Model Form

Formation: ~edges + nodemix("sex") + absdiff("age") + absdiff("income")
Target Statistics: 135979 10742 6934 679895 26683535
Constraints: ~.

Dissolution: ~offset(edges)
Target Statistics: 20

The estimation results
Call:
ergm(formula = formation, constraints = constraints, offset.coef = coef.form,
target.stats = target.stats, eval.loglik = FALSE, control = set.control.ergm,
verbose = verbose, basis = nw)

Monte Carlo Maximum Likelihood Results:

             Estimate Std. Error MCMC %  z value Pr(>|z|)    

edges -1.155e+01 2.986e-02 0 -386.896 < 1e-04 ***
mix.sex.1.2 -1.961e+00 3.048e-02 0 -64.335 < 1e-04 ***
mix.sex.2.2 -2.079e+00 7.053e-02 0 -29.481 < 1e-04 ***
absdiff.age -1.039e-01 1.893e-03 0 -54.874 < 1e-04 ***
absdiff.income 3.873e-04 1.340e-04 0 2.891 0.00384 **

Dissolution Coefficients

Dissolution Model: ~offset(edges)
Target Statistics: 20
Crude Coefficient: 2.944439
Mortality/Exit Rate: 0
Adjusted Coefficient: 2.944439

The diagnostics I tried was
dx <- netdx(est, nsims = 10,
dynamic = T, ncores = 10,
nsteps = 500)

> sessionInfo()
R version 4.2.1 (2022-06-23)
Platform: x86_64-redhat-linux-gnu (64-bit)
Running under: Oracle Linux Server 8.6

Matrix products: default
BLAS/LAPACK: /usr/lib64/libopenblaso-r0.3.15.so

locale:
[1] LC_CTYPE=pt_BR.UTF-8 LC_NUMERIC=C
[3] LC_TIME=pt_BR.UTF-8 LC_COLLATE=pt_BR.UTF-8
[5] LC_MONETARY=pt_BR.UTF-8 LC_MESSAGES=pt_BR.UTF-8
[7] LC_PAPER=pt_BR.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=pt_BR.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] parallel stats graphics grDevices utils datasets methods base

other attached packages:
[1] doParallel_1.0.17 iterators_1.0.14 foreach_1.5.2
[4] patchwork_1.1.2 data.table_1.14.8 haven_2.5.2
[7] lubridate_1.9.2 forcats_1.0.0 stringr_1.5.0
[10] dplyr_1.1.2 purrr_1.0.1 readr_2.1.4
[13] tidyr_1.3.0 tibble_3.2.1 tidyverse_2.0.0
[16] ggplot2_3.4.2 EpiModel_2.4.0 statnet.common_4.9.0
[19] tergm_4.2.0 ergm_4.5.0 networkDynamic_0.11.3
[22] network_1.18.1 deSolve_1.38

loaded via a namespace (and not attached):
[1] tidygraph_1.2.3 splines_4.2.1 Rdpack_2.5
[4] ergm.ego_1.1.0 robustbase_0.99-0 pillar_1.9.0
[7] lattice_0.20-45 glue_1.6.2 digest_0.6.31
[10] RColorBrewer_1.1-3 rbibutils_2.2.15 egor_1.23.3
[13] colorspace_2.1-0 Matrix_1.4-1 survey_4.2-1
[16] pkgconfig_2.0.3 ergm.multi_0.2.0 scales_1.2.1
[19] tzdb_0.4.0 timechange_0.2.0 generics_0.1.3
[22] cachem_1.0.8 withr_2.5.0 lazyeval_0.2.2
[25] cli_3.6.1 survival_3.3-1 magrittr_2.0.3
[28] memoise_2.0.1 fansi_1.0.4 nlme_3.1-157
[31] MASS_7.3-57 tools_4.2.1 hms_1.1.3
[34] mitools_2.4 lifecycle_1.0.3 networkLite_1.0.5
[37] trust_0.1-8 munsell_0.5.0 compiler_4.2.1
[40] rlang_1.1.1 grid_4.2.1 rstudioapi_0.14
[43] igraph_1.5.1 gtable_0.3.3 codetools_0.2-18
[46] DBI_1.1.3 R6_2.5.1 lpSolveAPI_5.5.2.0-17.10
[49] rle_0.9.2 fastmap_1.1.1 utf8_1.2.3
[52] srvyr_1.2.0 ape_5.7-1 stringi_1.7.12
[55] Rcpp_1.0.10 vctrs_0.6.3 DEoptimR_1.1-2
[58] tidyselect_1.2.0 coda_0.19-4

[sfloresalvarado]

Hi! Did you find out what that error means? I am also getting the same, using Epimodel, and this question is the only site where I found it mentioned.

[RodrigoAnderle]

Sorry, I didn't. I was working with different municipalities, and just one was given this problem. So, I just drop off this one. I will check the code to share here.

[krivit]

Generally, things shouldn't fail at that level, so it might be a bug. Can you provide code to reproduce? Thanks in advance!

[sfloresalvarado]

I will as soon as possible, since I don't have my computer right now. Meanwhile, I would like to add that I first tried the simulation with a network with 100,000 nodes and I got that error, but when I tried exactly the same code with n = 1,000, there was no issue. Could it be related to the size of my network?

[sfloresalvarado]

Hi again. I prepared the code you asked for. Thanks in advance for your time.

Initialization

num_nodes <- 1e5

## base sim
nw.sim <- network_initialize(n = num_nodes)

Formulas

target_stats <- c(88984, 11495, 20680, 19561, 13283, 8439, 3628, 1278, 580, 45708, 22047)

formation_formula <- formula(~ edges +
                               nodefactor("edad") +
                               nodefactor("sexo") +
                               nodematch("edad") )

dissolution_formula <- dissolution_coefs(~ offset(edges),
                                         duration = 5.471683)

Simulation

set.seed(0)

netest <- netest(
  nw = nw.sim,
  formation = formation_formula,
  target.stats = target_stats,
  coef.diss = dissolution_formula,
  verbose = F
)

dx <- netdx(x = netest, 
            dynamic = T, 
            nsims = 5, 
            nsteps = 140, 
            ncores = parallel::detectCores(),
            nwstats.formula = formation_formula,
            keep.tedgelist = T) # here I got the error

Here I got the error "Error en dosim(): task 1 failed - "MCMCDyn failed with error code 3""; I changed target_stats to target_stats *.5 and the error changed to "Error en tergm_MCMC_slave(state, eta.comb, control, verbose): MCMCDyn failed with error code 3". This happens with num_nodes == 1e5 | (1e5 - 1) | 5e4; but it does not when num_nodes == 1e4 | 1e3.

[krivit]

I see. It looks like the TERGM simulation error messages aren't very informative. This message indicates that the code ran out of space for the list of changes. These can be adjusted by the MCMC.maxchanges parameter to control.simulate.tergm().

[krivit]

The problem appears to be that MCMC.maxchanges needed to be higher; in general, it should be higher for larger networks, models with faster evolution, and long simulations.

tergm simulation will provide more informative error messages going forward.

[smjenness]

Thanks @krivit : this has been a not uncommon error for a while so a more informative message would be helpful. Also, do you think it would be OK for us (EpiModel) to set the default for MCMC.maxchanges to be a higher value? Are there any computational tradeoffs in doing so, even for smaller networks? Thanks

[krivit]

Thanks @krivit : this has been a not uncommon error for a while so a more informative message would be helpful. Also, do you think it would be OK for us (EpiModel) to set the default for MCMC.maxchanges to be a higher value? Are there any computational tradeoffs in doing so, even for smaller networks? Thanks

Only memory. See #123 .