Use the new marginal sampler for latent gaussian models in the GP notebook

[juanitorduz]

Closes #238

Aim: Use the data from https://blackjax-devs.github.io/blackjax/examples/GP_EllipticalSliceSampler.html to illustrate the implementation introduced in #247

For now, I have a running notebook that runs the example. I still wanted to push this branch and create a draft PR for feedback. What is missing (IMO):

General

• Add text explaining the method and the results. I will do a very complete (verbose?) explanation for a newbie like me. We can trim the text to make it more succinct.
• Convert notebook to md (jupytext?)

Details

• I have used the implementation based on the unit-test of Add Gradient based sampler for latent Gaussian models #247 . At the moment I am setting the arbitrary value of alpha=0.5 when defining the kernel. Should we use a calibration step as illustrated in https://github.com/blackjax-devs/blackjax/blob/main/examples/LogisticRegressionWithLatentGaussianSampler.md ? I meant delta and not alpha 🤦

Any feedback is welcome :)

[codecov]

Codecov Report

Merging #398 (4004742) into main (28167ad) will increase coverage by 0.12%.
The diff coverage is n/a.

❗ Current head 4004742 differs from pull request most recent head 36f9e3b. Consider uploading reports for the commit 36f9e3b to get more accurate results

@@            Coverage Diff             @@
##             main     #398      +/-   ##
==========================================
+ Coverage   89.86%   89.99%   +0.12%     
==========================================
  Files          45       47       +2     
  Lines        2161     2188      +27     
==========================================
+ Hits         1942     1969      +27     
  Misses        219      219              

Impacted Files	Coverage Δ
blackjax/__init__.py	100.00% <0.00%> (ø)
blackjax/base.py	100.00% <0.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[rlouf]

All sounds good! And yes, you will need to use jupytext to convert the notebook to md.

[AdrienCorenflos]

@juanitorduz sorry I didn't see this

At the moment I am setting the arbitrary value of alpha=0.5 when defining the kernel. Should we use a calibration step as illustrated in https://github.com/blackjax-devs/blackjax/blob/main/examples/LogisticRegressionWithLatentGaussianSampler.md ?

alpha is a parameter in the prior, I am not sure what you mean by "calibration step". The calibration step in the notebook mentioned is for the delta parameter which corresponds (in a loose sense) to the step-size of MALA and other similar algos. Are you trying to learn alpha?

[juanitorduz]

Ahh! Thanks for the input! I was indeed confused! Now it makes sense. I will add the text and work on the notebook next week.
Thanks for the input!

[AdrienCorenflos]

no pb

[AdrienCorenflos]

On a side note, good choice of example. The original paper considers GP classification and it shows that their method (mGrad, which is the one I implemented here) dominates the elliptical slice sampler (Ellipt):

[juanitorduz]

I think I have a first initial complete iteration ready for review :)

[rlouf]

It would be nice to have a quick example of how the sampler works, like in the Elliptical Slice Sampling notebook.

[juanitorduz]

Yeah! I want to do this... unfortunately I have not been able to access the paper (I am trying through old access from my university). Is there a pre-print available?

[rlouf]

Yes! https://arxiv.org/abs/1610.09641

[AdrienCorenflos]

https://arxiv.org/abs/1610.09641

[juanitorduz]

OMG 🤦 I can not believe I did not find this! I think I was keen to get the published version. Anyway, let me give a read and try to write this section myself, and then I'll collect feedback. Thanks!

[AdrienCorenflos]

Btw, this is the "executive summary" of the method I have in the paper I am currently writing.

[rlouf]

It looks great! I modified the titles to match the notebook with the Elliptical Slice sampler (we may merge them eventually). It would also be nice to have a short explanation of how the sampler works if you feel like writing it.

[juanitorduz]

@rlouf @AdrienCorenflos In 687f295 I tried to express in my words how I understand the sampler works. I have a math background, which is far from this area so I feel it is at the moment out of my reach to have a very detailed and transparent description of the details behind the sampler (if you want to talk about geometric analysis I feel a bit better prepared hehe). Hence, feel free to modify / add or erase parts that you believe are confusing or incorrect.

[AdrienCorenflos]

@rlouf @AdrienCorenflos In 687f295 I tried to express in my words how I understand the sampler works. I have a math background, which is far from this area so I feel it is at the moment out of my reach to have a very detailed and transparent description of the details behind the sampler (if you want to talk about geometric analysis I feel a bit better prepared hehe). Hence, feel free to modify / add or erase parts that you believe are confusing or incorrect.

I made some changes to the explanation of the algo.
https://github.com/AdrienCorenflos/blackjax/tree/patch-2
@rlouf if you could check and merge?

[rlouf]

I'll take a look early next week, thanks.

[rlouf]

I added the changes suggested by @AdrienCorenflos (and added him as co-author) and rebased the commits. It looks good to me, can you take a look one last time before I merge?

[juanitorduz]

Thank you for the feedback and corrections @AdrienCorenflos ! LGTM.

[rlouf]

Merging now, great work everyone!