TypeError: cannot pickle 'starry._c_ops.Ops' object

[fbartolic]

The eclipsing binary example notebook with MCMC is failing at the NUTS sampling step. It has something to do with the new version of PyMC3 (3.9) because it works fine with PyMC3 3.7.

The following cell fails with the error TypeError: cannot pickle 'starry._c_ops.Ops' object:

with model:
    trace = pm.sample(
        tune=500,
        draws=500,
        start=map_soln,
        chains=4,
        step=xo.get_dense_nuts_step(target_accept=0.9),
    )

It works if you set chains=1.

Starry version: 1.0.1.dev161+g9c462a2c
exoplanet version: 0.3.3
PyMC3 version: 3.9.3
macOS Catalina
install from source

[rodluger]

Interesting. Mark Popinchalk messaged me today with the same error. Agreed it's something about the new pymc3 version. He found that setting chains=1 fixes the issue if you need a quick workaround. @dfm any ideas about this? Are things in exoplanet breaking as well?

[dfm]

forkserver strikes again!

pymc-devs/pymc#3844

and

exoplanet-dev/exoplanet#111

[dfm]

It's all the same issue and it's really a Python issue, not pymc3 really. I've found that using fork instead of forkserver works fine in general.

[rodluger]

Looking at @bmorris3's comment, the workaround is

import multiprocessing as mp

pm.sample(
    ..., 
    mp_ctx=mp.get_context("fork")
)

[dfm]

Yes - I expect that that will do it.

[fbartolic]

Looking at @bmorris3's comment, the workaround is

import multiprocessing as mp

pm.sample(
    ..., 
    mp_ctx=mp.get_context("fork")
)

Thanks for the fix, this works!

[rodluger]

FYI I'm implementing simple pickling support in #270, to be merged into master soon. Should fix this issue without needing to switch to fork.