Voila-render source files

[SylvainCorlay]

This is a usecase that is not handled by the voila, and was supported by @pbugnion's ipywidgets_server.

To handle that case, we need to figure out how to

• decide which kernelspec to use for a given source file (should it be infered from the file extensions?).
• should voila actually insert the content of the file into a single-cell notebook, or handle script file completely differently from notebooks?

[choldgraf]

A potentially wacky solution would be to use jupytext (for python files) to convert into notebooks (and optionally populate cells in a modular fashion using jupytext syntax) before serving

[maartenbreddels]

In the context of the material design widgets, I see also a use case where 'cell output' does not exist. Only widgets, with a particular 'id', on where that widget should end up in the template.

For the source file, I think the simplest approach would be to see it as a single cell and fully rely on display (or the last expression) for visualization. Using display also allows for markdown etc.

[choldgraf]

Have y'all explored the idea of using cell metadata for this? That's what I use in jupyter book and it works pretty nicely (e.g.: https://jupyter.org/jupyter-book/features/hiding.html#hiding-code-cells-and-displaying-a-button-to-show-them)

[maartenbreddels]

I think jupytext support would be nice, but I think we should also have pure Python support, or C++ even. We could rely on using display(widget), and simply see the source file as 1 cell.

[maartenbreddels]

I think if we want to support this, we may want to reconsider the extension. What if we have both foobar.py and foobar.ipynb in a directory, what should render at /voila/render/foobar?

[SylvainCorlay]

@choldgraf we sat together for a few minutes with @mwouts and got this to work with jupytext!

The two action items to get this in are

• in voila: re-add the file extension to URLs so that we differentiate between foobar.py and foobar.ipynb.
• in jupytext: make jupytext's notebook server extension work with jupyter_server as well.

[maartenbreddels]

I think this is good material for the dashboarding workshops!

[choldgraf]

amazing! very cool :-)

[mwouts]

@choldgraf , it's good to see you here! I see that you contribute to every fun project 😄. And thanks for mentioning Jupytext!

@SylvainCorlay, is it correct that you would like

• a jupyter_server_config.d/jupytext.json file that activates Jupytext's server extension on the ServerApp
• and that the server extension should set a contents manager that derives from the current app's content manager class (e.g. LargeFileManager in jupyter_server.services.contents.largefilemanager rather than from LargeFileManager in notebook.services.contents.largefilemanager) ?

[SylvainCorlay]

@SylvainCorlay, is it correct that you would like

That is correct!

[mwouts]

Perfect. I have prepared (but not tested) this branch.

In that branch, the TextFileContentsManager is created with a function that takes the parent class as its argument, so we can call that function with jupyter_server's default contents manager. I am sure there could be a better way to do this, please let me know if you have any idea!

Also, in the server extension, can you think of a better way than app.cm_class is jupyter_server's cm_class to detect that the app is a ServerApp and not a NotebookApp ?

[mwouts]

I have prepared a RC for this:

pip install jupytext==1.2.0rc1

That version derives a Jupytext contents manager from the ServerApp's one. Could you please let me know if it makes the job? Thanks!

Also I see that I never commented on this:

decide which kernelspec to use for a given source file (should it be infered from the file extensions?).

Jupytext's contents manager will do that for you. If there's no kernel in the text file, then it will choose one kernel that matches the inferred notebook language.

[maartenbreddels]

How should we proceed, do we want native .py support (basically requiring you to use display(), or inject the code into in a single cell). Should we rely on jupytext only, or document how to do this with jupytext.

@mwouts could you lay out what would be the advantage of jupytext over simple .py support (I think I can find a few points, but better for the original author to make these points).

[mwouts]

Nice question @maartenbreddels . That's a good opportunity to review my expectations for Jupytext 😄

My expectations are:

• Jupytext can create sensible notebooks from many file types, and in particular
  • random Python scripts (cell breaks will happen on blank lines in-between Python paragraphs)
  • scripts in any other language, as long as it is a language supported by Jupytext (to be exact, multiline comments are supported only for Python)
  • scripts created with nbconvert
  • Spyder/Hydrogen/etc scripts (cells delimited with # %%)
  • Sphinx Gallery scripts
  • Markdown documents

If you want to test that yourself on random files, the simplest way is probably to install Jupytext and see how your favorite files open as notebooks in Jupyter (simple click in Jupyter notebook, right-click and Open as Notebook in JupyterLab).

• Jupytext supports both notebook and cell metadata. Notebook medata will allow an user to configure a specific kernel. And you probably also have good use of cell metadata in Voila.

• Using Jupytext in Voila should require no work in Voila other than using the jupyter server contents manager. You're not even expected to take a dependency on Jupytext. Precisely, having Jupytext >= 1.2.0 installed will activate Jupytext's contents manager in Jupyter server (give a try to jupyter server, there should be a line in the logs about that), and that content manager can open any of the above files as a notebook.

[SylvainCorlay]

I think we can close this one as it was implemented.