Implement No-U-Turn Sampler (NUTS)

[ben18785]

This is actually the algorithm (not HMC, although they are similar) that powers Stan and Pymc3. Look at the latest papers on Stan/Pymc3 to see what variants they implement.

[ben18785]

I know how to do this now as have implemented it elsewhere. Am adding to the paper project.

[martinjrobins]

I'm planning on implementing this. Based on discussions with Ben, it looks very difficult to fit in the ask-tell interface since it uses recursion. Seems like the simplest thing to do would be to write the original algorithm as a coroutine that yields whenever a likelihood evaluation is needed.

Looks like the syntax of coroutines is currently in flux (https://docs.python.org/3/library/asyncio-task.html). The old generator based syntax (using the yield keyword) is scheduled for removal in python 3.10. The new async def coroutines were added in python 3.5. @MichaelClerx, are we still supporting 2.7 (I notice we are still testing it)?

Also, @ben18785: should I be implementing the nuts in this issue or in #1001?

As an aside, I'm impressed that googling "pints nuts" leads me to #1001, I think it was about 5th in the list :)

[MichaelClerx]

Awesome! Thanks for picking this up @martinjrobins

I might be OK with at least one method not being ask-and-tell, if it really doesn't fit?

Not sure about Python 2.7. I imagine some academic users will be stuck with 2.7 for a while, so I'd like to keep it compatible for a while. At the very least, even if we don't test, I'd like 2.7 to be able to import pints, even if some methods don't work

[martinjrobins]

I need at least the yield from keyword, which was introduced in python 3.3, presumably we can just not import the NUTS sampler for python versions less than 3.3 (so everything else works for older version, just not nuts)?

[MichaelClerx]

Yeah I'm happy with a solution like that :-)

E.g., in the init

if sys.hexversion < 0x03050000:
    warn about unsupported python version

then further down

if sys.hexversion >= 0x03030000:
    from .mcmc.nuts import NUTSSampler

and in the tests:

@unittest.skipIf(sys.hexversion < 0x03030000, 'Old Python version')
class NUTSTest(unittest.TestCase):
    ...