Vine: Jupyter Notebook Integration and Best Practices

[btovar]

Users of notebooks often re-evaluate cells. If a cell redefines a vine manager, two undesirable things may happen:

• If port is set, then the cell raises an exception because the manager cannot bind to the port.
• If the manager is free to choose another port, then many manager instances run at the same time.

Ideally, the Manager constructor would check if a manager on a port already exits and simply warn the user that a previous manager definition is being used. (Or some mechanism like that.)

[dthain]

I think we need to think more comprehensively about how to use notebook environments to deploy distributed/parallel applications. As you point out, it doesn't make sense to reinstantiate the manager in every cell. Maybe it belongs in the notebook kernel instead? We have pending CSSI proposal on notebook/workflow integration, I'll send it to you separately...

[btovar]

Sounds good!

[dthain]

There would appear to be a variety of ways to use TaskVine within a notebook, some of which work, and others which do not. At a minimum, we should document and provide an example of how TV can be used correctly, with some warnings re things that don't work.