Jupyter Notebook 7 compatibility (discussion)

[kiyoon]

Describe the bug
This plugin is not compatible with Jupyter Notebook 7 (as it follows the Jupyter Lab design).

To Reproduce
pip install notebook==7.0.0.a18

Output of jupynium --version

0.2.0

Additional Context
Currently, this plugin doesn't support Jupyter Lab. The reason is that Jupyter Lab doesn't provide a front-end API to interact using Selenium. A possible way may be writing a Jupyter Lab extension, and communicate with the extension, but it will be much more complicated to set up Jupynium that way..

Problems:

• Writing a Jupynium extension requires one extra version to match. It requires Jupynium plugin version and the Jupyter Lab extension version to exactly match. So whenever you update Jupynium, you'll be required to update all of your server's Jupyter Lab extension.
• It will be one more configuration required. Instead of communicating with the browser, now you'll need to communicate with the Jupyter Lab extension, then the extension needs to listen to a socket.
• It will be harder to maintain. The contributors need understanding in all lua, python, typescript and Neovim / Jupyter Lab extension development experiences.

I may be wrong here, but if anyone can find a Jupyter Lab front-end method without using an extension please let me know. You can try with Jupyter Notebook like this:

1. Press F12 to open a developer console
2. Write Jupyter.notebook. and see the APIs like setting text on cells etc.

(see https://stackoverflow.com/questions/51411871/where-is-a-docs-for-jupyter-front-end-extensions-javascript-api)

This is simply what I need, and it looks like the new Jupyter Lab and Jupyter Notebook 7 doesn't support this.

Suggested Solution
Luckily, it looks like we can still use nbclassic (pip install nbclassic) and it will still serve the classic notebook UI. We will need to change the default address to localhost:8888/nbclassic, or maybe Jupynium can detect if the server running is not compatible and fallback to the nbclassic URL. This and Notebook 6 are supported for 2 more years.

[sho-87]

if there is the additional requirement of a JLab extension plus a python package, could they not both be bundled with jupynium to ensure the versions match (in the same way that the python package is currently bundled)?

the silver lining is that moving over to JLab has some perks as it buys a lot of additional functionality for free, like the variable explorer, a larger plugin ecosystem, cell tags for easier config and export to multiple formats (eg slides etc)

I guess it depends what the long term plans for jupynium are. it wouldn't make sense to tie jupynium EOL to NB6 2 year timeline if you're wanting to keep the project going for longer - you'd have to switch over sooner or later anyway.

if you're considering the switch, then this is a good time to think about how integrated into the jupyter ecosystem you'd want to be. for example, is there a single implementation that would give jupynium access to their wider ecosystem like jupyter notebook, jupyter lab, jupyter hub? rearchitecting the plugin for that level of support is more work for sure, but maybe its a time saver in the long term?

[kiyoon]

@sho-87 It's not as easy as that because you can have multiple environments in one system even (I don't know if that matters), and it could be in a different server.

I asked for help and one of the Jupyter devs guided me about the equivalent functions, but it's still not clear how it can be done.

First of all, I have no experience with web development, javascript, typescript and understanding how Jupyter Lab extensions work is not easy for me. I spent some time on it but I couldn't get it to build with compilation errors.

Even if I manage to build it, it wouldn't directly expose an API so that Selenium can call it. The Jupyter extension has to be run as a server and listen to RPC responses. In current Jupynium version,

nvim -> Jupynium plugin -> Jupyter Notebook (localhost:8888)

With Jupyter Lab, the plugin has to both be able to communicate with the extension and the browser. It makes it very complicated to set up in remote servers.

nvim -> Jupynium plugin -> Jupyter extension (localhost:9999) -> Jupyter Lab (localhost:8888)
                    |
                    ---> Jupyter Lab (localhost:8888)

I just wish that they didn't remove the front-end APIs for this reason. I may be missing something so any help with directions and setting up will be very helpful. In the long run it makes sense to adopt Jupyter Lab ecosystem but also after 2 years I hope that there's a cleaner solution.

[kiyoon]

Another thing I figured out is that you also need to package (build) Jupyter Lab extensions. So you need NodeJS dependencies with react etc. installed on your system. You also need Jupyter lab installed already because they use their modified version of npm called jlpm etc.

It will be kind of like how CoC or markdown-preview.nvim is doing, like you can choose to resolve those dependencies or you get infrequent updates using prebuilt pip packaged extension.

I may have to ditch Selenium and use Playwright because that's what Jupyter Lab devs use, and by that time I think there will be a complete rewrite and a brand-new name (Jupywright? 😂)

[fecet]

notebook 7 is released, we should add version requirement now

[kiyoon]

@fecet what do you think about the solution #86 ?

[raffaem]

can we pin this and ask for help in the README?

[fecet]

Just found this https://github.com/jtpio/ipylab could be helpful to support jupyter lab. The most important thing for us (in my view) is adding a cell, it's looks tricky but still possible jupyterlab/jupyterlab#13080 (comment).

If can help you are willing to porting to this.

[singledoggy]

Actually I tried jupyter_ascending, it's possible to utilize the Jupytext¶, I don't know if this information can help.

[raffaem]

This project seems to support JupyterLab: https://github.com/SUSTech-data/neopyter

[statquant]

Hello @kiyoon, are you going to support jupyter-lab eventually ?
Many organisations (mine included) are pivoting to solely use jupyter-lab

[kiyoon]

@statquant Unfortunately this is out of scope for the project. The implementation will be totally different to support it. But please check out neopyter as they seemed to have nailed it.