Replacing PyMC3 plots w/ Arviz plots & sigma Param change [Part 7] #19

CloudChaoszero · 2021-01-19T03:14:57Z

Description

The following is a large PR breakdown of PR #16.

Replace PyMC3 dependent plots with arviz plots in case studies & examples.

The issue is a 3-part step for the following:
- Initiative via Issue Remove Plots & Diagnostics from PyMC Code Base pymc#4371 (Leave open)

Replace parameter sd with sigma (e.g. some examples have pm.Normal(...sd=...)

Closes issue Use sigma instead of sd #2

…h sigma

review-notebook-app · 2021-01-19T03:15:02Z

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

review-notebook-app · 2021-01-29T16:18:25Z

View / edit / reply to this conversation on ReviewNB

twiecki commented on 2021-01-29T16:18:25Z
----------------------------------------------------------------

Where did the output of summary() go?

CloudChaoszero commented on 2021-02-02T05:15:43Z
----------------------------------------------------------------

I ignored this, as there was an error in the above cell that I was not able to run. I did not how to go about resolving it, seen below.

Section: "Create fine model and perform inference"

RuntimeError                              Traceback (most recent call last)
<ipython-input-23-17659000b718> in <module>
     29     method_names.append("Metropolis")
     30     traces.append(
---> 31         pm.sample(
     32             draws=ndraws,
     33             step=step_metropolis,
.........
~/opt/anaconda3/envs/pymc3-dev/lib/python3.8/site-packages/pymc3/parallel_sampling.py in recv_draw(processes, timeout)
    357             else:
    358                 error = RuntimeError("Chain %s failed." % proc.chain)
--> 359             raise error from old_error
    360         elif msg[0] == "writing_done":
    361             proc._readable = True
RuntimeError: Chain 0 failed.

CloudChaoszero commented on 2021-02-02T05:54:14Z
----------------------------------------------------------------

Small update:

I found this PR (#3844) and added:

        mp_ctx="forkserver",
        pickle_backend='dill',
        return_inferencedata=True

However, now I am getting the following:

Multiprocess sampling (2 chains in 2 jobs)
MLDA: [x, Intercept]
 0.00% [0/8000 00:00<00:00 Sampling 2 chains, 0 divergences]

RemoteTraceback                           Traceback (most recent call last)

RemoteTraceback:

"""

Traceback (most recent call last):

File "/Users/CloudChaoszero/opt/anaconda3/envs/pymc3-dev/lib/python3.8/site-packages/theano/compile/function/types.py", line 970, in call

self.fn()

File "/Users/CloudChaoszero/opt/anaconda3/envs/pymc3-dev/lib/python3.8/site-packages/theano/gof/op.py", line 912, in rval

r = p(n, [x[0] for x in i], o)

File "<ipython-input-5-06c97d253f6f>", line 33, in perform

NameError: name 'np' is not defined

review-notebook-app · 2021-01-29T16:18:26Z

View / edit / reply to this conversation on ReviewNB

twiecki commented on 2021-01-29T16:18:25Z
----------------------------------------------------------------

Also misses the outputs of the samplers.

review-notebook-app · 2021-01-29T16:18:26Z

View / edit / reply to this conversation on ReviewNB

twiecki commented on 2021-01-29T16:18:26Z
----------------------------------------------------------------

no plot

CloudChaoszero commented on 2021-02-02T05:16:59Z
----------------------------------------------------------------

This and other parts of this file are subsequent to the error from the first comment, unfortunately. Let me know what next steps I should take in this case. :)

CloudChaoszero commented on 2021-02-07T06:43:24Z
----------------------------------------------------------------

acknowledeged now :D

review-notebook-app · 2021-01-29T16:18:27Z

View / edit / reply to this conversation on ReviewNB

twiecki commented on 2021-01-29T16:18:27Z
----------------------------------------------------------------

no plot

twiecki · 2021-01-29T16:18:28Z

examples/samplers/MLDA_variance_reduction_linear_regression.ipynb

@@ -533,7 +533,7 @@
    }


Should probably add the model context to this and below to future proof this and get rid of the annoying warning.

Reply via ReviewNB

Sounds great. I quite didn't understand what the FutureWarning recommended, but I am happy to update the description as a PR.

Here was my take on what the FutureWarning recommended haha:

with model: trace2_az = az.from_pymc3(trace2) az.summary(trace2_az)

twiecki · 2021-01-29T16:19:11Z

Reviewed all of these, looks great but see comments.

CloudChaoszero · 2021-02-02T05:15:44Z

I ignored this, as there was an error in the above cell that I was not able to run. I did not how to go about resolving it, seen below.

Section: "Create fine model and perform inference"

RuntimeError                              Traceback (most recent call last)
<ipython-input-23-17659000b718> in <module>
     29     method_names.append("Metropolis")
     30     traces.append(
---> 31         pm.sample(
     32             draws=ndraws,
     33             step=step_metropolis,
.........
~/opt/anaconda3/envs/pymc3-dev/lib/python3.8/site-packages/pymc3/parallel_sampling.py in recv_draw(processes, timeout)
    357             else:
    358                 error = RuntimeError("Chain %s failed." % proc.chain)
--> 359             raise error from old_error
    360         elif msg[0] == "writing_done":
    361             proc._readable = True
RuntimeError: Chain 0 failed.

View entire conversation on ReviewNB

CloudChaoszero · 2021-02-02T05:17:00Z

This and other parts of this file are subsequent to the error from the first comment, unfortunately. Let me know what next steps I should take in this case. :)

View entire conversation on ReviewNB

CloudChaoszero · 2021-02-02T05:54:15Z

Small update:

I found this PR and added:

        mp_ctx="forkserver",
        pickle_backend='dill',
        return_inferencedata=True

However, now I am getting the following:

Multiprocess sampling (2 chains in 2 jobs)
MLDA: [x, Intercept]
 0.00% [0/8000 00:00<00:00 Sampling 2 chains, 0 divergences]

RemoteTraceback                           Traceback (most recent call last)

RemoteTraceback:

"""

Traceback (most recent call last):

File "/Users/CloudChaoszero/opt/anaconda3/envs/pymc3-dev/lib/python3.8/site-packages/theano/compile/function/types.py", line 970, in call

self.fn()

File "/Users/CloudChaoszero/opt/anaconda3/envs/pymc3-dev/lib/python3.8/site-packages/theano/gof/op.py", line 912, in rval

r = p(n, [x[0] for x in i], o)

File "<ipython-input-5-06c97d253f6f>", line 33, in perform

NameError: name 'np' is not defined

View entire conversation on ReviewNB

twiecki · 2021-02-02T07:54:05Z

Can you try updating PyMC3 and Theano? Otherwise this seems like a Aesara bug. CC @brandonwillard

brandonwillard · 2021-02-02T17:37:58Z

That environment's version of Theano is too old; you can tell by the module name: theano.gof.op.

CloudChaoszero · 2021-02-04T11:06:30Z

@brandonwillard Thanks for the response! Hmm, unfortunately I get some more errors. I updated the particular notebook to show the error, seen below.

(If the details are too much for this PR, I can submit a detailed issue in Aesara? Let me know!)

Adding these params in pm.sample()

    method_names.append("Metropolis")
    traces.append(
        pm.sample(
            draws=ndraws,
            step=step_metropolis,
            chains=nchains,
            tune=nburn,
            discard_tuned_samples=discard_tuning,
            random_seed=sampling_seed,
            start=MAP,
            mp_ctx="forkserver",
            pickle_backend='dill',
        )

I get

due to dill pickle'ing. Well, I suspect that is the reason.

If I do not add the params

            mp_ctx="forkserver",
            pickle_backend='dill',

then I get a chained 0 error 😢

brandonwillard · 2021-02-04T22:17:46Z

Please, provide the complete trace as formatted markdown. If the trace is too large, it can be wrapped in HTML <details> tags (e.g. here).

CloudChaoszero · 2021-02-07T01:59:47Z

@brandonwillard Thanks for that! (useful tip 🔥 )

Here is the Gist for when we use

            mp_ctx="forkserver",
            pickle_backend='dill',

https://gist.github.com/CloudChaoszero/2ffbc8efa6157786102c8a4659bf5c11

Here is the Gist for when we do not use the params, originally in the notebook
https://gist.github.com/CloudChaoszero/6078a62bad062cc9ab541e150c914d49

brandonwillard · 2021-02-07T02:25:39Z

I don't see any connections to Theano-PyMC, but a quick search did yield this open dill issue.

CloudChaoszero · 2021-02-07T06:20:27Z

I don't see any connections to Theano-PyMC, but a quick search did yield this open dill issue.

@brandonwillard Thanks for checking it out. After some further research, I founda. workaround is to use cores param to help out with this. I found this workaround from the PyMC3 Discourse, found here.

Here is what I added at the end of each pm.sample():
cores != None...i.e.

            cores=1,
            mp_ctx="forkserver",

CloudChaoszero · 2021-02-07T06:43:24Z

acknowledeged now :D

View entire conversation on ReviewNB

CloudChaoszero · 2021-02-07T06:47:54Z

As a note, I updated my package dependencies and did more research this time on navigating the errors encountered with pymc3.

In particular, this PR uses cores=1 in pm.sample() and some other small changes.

Though the perf runs long, it was successfully. 😄

Deps

pymc3                     3.11.0                    dev_0    <develop>
theano-pymc               1.1.2                    pypi_0    pypi

twiecki · 2021-02-07T20:02:34Z

Thanks @CloudChaoszero!

🎨 Update plots from pymc to arviz dependencies & replace param sd wit…

a257fb0

…h sigma

twiecki reviewed Jan 29, 2021

View reviewed changes

🐛 Run complete MLDA Gravity Surv. file w/error

1f810d9

🎨 Update notebook to run under arviz dependency and cores=1 pm.sample()

a92d073

🎨 Utilize from_pymc() with arviz summary and update notebook

72c74f0

CloudChaoszero requested a review from twiecki February 7, 2021 06:44

twiecki approved these changes Feb 7, 2021

View reviewed changes

twiecki merged commit f624891 into pymc-devs:main Feb 7, 2021

CloudChaoszero deleted the replace-pymc3-arviz-plots_part7 branch April 5, 2021 01:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Replacing PyMC3 plots w/ Arviz plots & sigma Param change [Part 7] #19

Replacing PyMC3 plots w/ Arviz plots & sigma Param change [Part 7] #19

CloudChaoszero commented Jan 19, 2021

review-notebook-app bot commented Jan 19, 2021

review-notebook-app bot commented Jan 29, 2021 •

edited

Loading

review-notebook-app bot commented Jan 29, 2021 •

edited

Loading

review-notebook-app bot commented Jan 29, 2021 •

edited

Loading

review-notebook-app bot commented Jan 29, 2021 •

edited

Loading

twiecki Jan 29, 2021 •

edited

Loading

CloudChaoszero Feb 7, 2021

twiecki commented Jan 29, 2021

CloudChaoszero commented Feb 2, 2021

CloudChaoszero commented Feb 2, 2021

CloudChaoszero commented Feb 2, 2021

twiecki commented Feb 2, 2021

brandonwillard commented Feb 2, 2021

CloudChaoszero commented Feb 4, 2021

brandonwillard commented Feb 4, 2021

CloudChaoszero commented Feb 7, 2021 •

edited

Loading

brandonwillard commented Feb 7, 2021

CloudChaoszero commented Feb 7, 2021

CloudChaoszero commented Feb 7, 2021

CloudChaoszero commented Feb 7, 2021

twiecki commented Feb 7, 2021

Replacing PyMC3 plots w/ Arviz plots & sigma Param change [Part 7] #19

Replacing PyMC3 plots w/ Arviz plots & sigma Param change [Part 7] #19

Conversation

CloudChaoszero commented Jan 19, 2021

Description

review-notebook-app bot commented Jan 19, 2021

review-notebook-app bot commented Jan 29, 2021 • edited Loading

Section: "Create fine model and perform inference"

review-notebook-app bot commented Jan 29, 2021 • edited Loading

review-notebook-app bot commented Jan 29, 2021 • edited Loading

review-notebook-app bot commented Jan 29, 2021 • edited Loading

twiecki Jan 29, 2021 • edited Loading

Choose a reason for hiding this comment

CloudChaoszero Feb 7, 2021

Choose a reason for hiding this comment

twiecki commented Jan 29, 2021

CloudChaoszero commented Feb 2, 2021

Section: "Create fine model and perform inference"

CloudChaoszero commented Feb 2, 2021

CloudChaoszero commented Feb 2, 2021

twiecki commented Feb 2, 2021

brandonwillard commented Feb 2, 2021

CloudChaoszero commented Feb 4, 2021

brandonwillard commented Feb 4, 2021

CloudChaoszero commented Feb 7, 2021 • edited Loading

brandonwillard commented Feb 7, 2021

CloudChaoszero commented Feb 7, 2021

CloudChaoszero commented Feb 7, 2021

CloudChaoszero commented Feb 7, 2021

twiecki commented Feb 7, 2021

review-notebook-app bot commented Jan 29, 2021 •

edited

Loading

review-notebook-app bot commented Jan 29, 2021 •

edited

Loading

review-notebook-app bot commented Jan 29, 2021 •

edited

Loading

review-notebook-app bot commented Jan 29, 2021 •

edited

Loading

twiecki Jan 29, 2021 •

edited

Loading

CloudChaoszero commented Feb 7, 2021 •

edited

Loading