Improve introspection capabilities of Sage Jupyter Kernel

[embray]

Originally from https://groups.google.com/d/msg/sage-devel/8qr20g0EM5c/U9Kug99SBQAJ

Currently introspecting Sage objects in Jupyter (either the IPython console or the Notebook) with the obj? syntax is little different from the standard Python help(obj) output.

In particular on the Notebook this is not great because it means equations are not rendered--this is a major deficiency compared to SageNB.

The interface for Jupyter kernels does allow customizing what is sent back to the client upon object introspection. My understanding is that this can send back a multi-part MIME message in multiple formats (e.g. plain text and HTML) and the client will pick the most appropriate format to display. So on the Notebook, for example, it will prefer HTML-formatted help if any.

In this case the thing to do might be to look up the appropriate page for an object in Sage's reference docs, and return the HTML for that object, taking care to make sure that MathJax rendering is applied, and any images returned as well. I'm not exactly sure about the details of making this work yet (images, for example, might have to be embedded as base64).

Relevant upstream issues:

• Math in pager contents not rendered jupyter/notebook#4808

Part of #29889.

CC: @egourgoulhon @zerline

Component: documentation

Author: Erik Bray

Branch/Commit: [uhttps://user-images.githubusercontent.com/676149/216875645-789923ad-e325-42ea-9fbe-37b055e6063c.png) @ ec1e13a

Issue created by migration from https://trac.sagemath.org/ticket/25015

[williamstein]

comment:2

Related issue for CoCalc: sagemathinc/cocalc#2764

I'll implement something compatible with what you do here...

[embray]

comment:4

I should add, that while I assigned myself this task OpenDreamKit is currently hiring for a position (we're actually interviewing candidates today) under which this work could very likely fall as well. So depending on how things go I might point someone else in the right direction to work on this...

[embray]

comment:9

Really mostly what this has to do with is ensuring that help(obj) and obj? where obj is some object in the sage package displays the corresponding HTML documentation for that object if possible.

So we would need a way to find which page in the HTML docs contains the API documentation for that object, and return the HTML for that page (and specifically the section for that object) to Jupyter.

This would involve modifying what the Sage Jupyter kernel returns in response to inspect_request messages. Right now we just return a text/plain response containing the same text that would be displayed on the console. What we really want is to return a "MIME bundle": a dict of "text/plain" and "text/html".

The one thing I'm less sure about is the best way to format the HTML. Would it make sense to extract HTML from Sage's HTML docs (but in this case we also need to ensure that all the relevant stylesheets, javascript, etc. are loaded). Or do we do something like an <iframe> and just embed the URL to the appropriate doc page in that? The <iframe> approach might be simpler, but I don't know how well it will work. This might also be something worth asking on the Jupyter mailing list in case anyone has any advice or experience they could share...

[jhpalmieri]

comment:10

The system with the legacy Sage notebook is to run sphinxify on the docstring to produce html documentation. Rather than try to extract from the built documentation, shouldn't you just keep doing this? The issue may be making sure that mathjax is loaded.

[nthiery]

comment:11

I don't have an informed opinion for how to proceed.
But either way, it's useful to recover the URL of the relevant chunk of Sage documentation, to at least include a "Jump to full/online documentation" link, enabling the user to browse the whole documentation from that entry point.

Thank you for pushing this forward!

[embray]

comment:12

Retargeting some of my tickets.

[embray]

comment:13

Removing most of the rest of my open tickets out of the 8.7 milestone, which should be closed.

[embray]

comment:14

Finally have some progress on a working prototype of this (something I just felt compelled to work on today).

It turns out to be much more challenging that I originally expected :(

It turns out that the inspect_request message mentioned in the ticket description is only used, in the case of the IPython/Sage kernels, when pressing Shift-Tab to show a tooltip for the object under the cursor. It has nothing to do, in this case, with what happens when run obj?. That is handled entirely deep within the IPython input transformers, which converts this to a call to:

get_ipython().magic('pinfo obj')

In other words, obj? is equivalent to running the magic %pinfo obj. The implementation of the pinfo magic, then, is a complex affair which generates the help output as a string and passes that string to the IPython pager.

So the trick is the actually override how the pinfo magic works (which Sage already does a little bit) to have it pass the pager a "mime-bundle" containing both plain text and HTML, where the HTML has been generated by Sage's sphinxify() function.

On top of all that it requires a little more hacking to get the math displayed right. My prototype mostly works now, save for also loading the necessary CSS for the HTML docs to be displayed better.

Here's a screenshot showing an example of the in-progress work:

As you can see, one of the main problems is a lack of margins between paragraphs. Not shown in the screenshot, but code highlighting in the examples is broken. This is all due to not having the standard CSS load. Some non-standard LaTeX macros probably won't render either, though this is a more general problem with have right now in the Jupyter notebook.

[embray]

Author: Erik Bray

[embray]

Branch: u/embray/ticket-25015

[embray]

comment:15

Attachment: ticket-25015.png

Here's the in-progress version if anyone wants to try it out.

---

New commits:

c09d2da	Add strip_math_delims option to sphinxify so that its current default
2800281	Add sphinxify_mimebundle function for sending docstrings as text+HTML
1a56e10	Add the necessary default settings to enable HTML output in the Jupyter

[embray]

Commit: 1a56e10

[embray]

comment:16

Another shortcoming I noticed is that sphinxify() makes no effort to resolve references, so if one docstring refers to another function or something, that reference won't be linked.

It would be nice if this could instead produce an actual link. Perhaps that should be tackled as a separate issue though.

[egourgoulhon]

comment:17

Waouh! I've just checked it and it looks already very nice, even without the code highlighting in the doctests. Other shortcomings one might notice:

• the LaTeX macros \RR, \CC, etc. don't render
• the plots produced with sphinx_plot don't show up
• if one clicks on the button "Ouvrir le paginateur dans une fenêtre externe" ("Open the pager in external window" I guess --- sorry, my Jupyter is in French) in the top right of the help page, then the aspect change and the maths are printed twice, with standard LaTeX fonts and with some ugly fonts.

Anyway, all the above seem very minor inconveniences in perspective of the current state, which is ASCII-only documentation. So IMHO we could adopt your "prototype" as is and leave further improvements for other tickets.

[embray]

comment:18

Thank you for testing it out!

I think fixing some of these issues is going to require new features in a Jupyter extension, in particular to load the JavaScript and CSS necessary fo fix some of those issues.

The double math-rendering thing is odd. It might be related also this upstream issue I opened. It hasn't gotten any notice yet though. Maybe it will get more if I go ahead and make a PR to propose a fix...

[embray]

comment:19

Replying to @egourgoulhon:

• the LaTeX macros \RR, \CC, etc. don't render

By the way: This is a problem in the Jupyter notebook in general. Try making a text cell including them--they won't render properly there either, at least in my experience.

In SageNB, as well as in the Sphinx docs, we manually define these macros from:

def latex_extra_preamble():
    r"""
    Return the string containing the user-configured preamble,
    ``sage_latex_macros``, and any user-configured macros.  This is
    used in the :meth:`~Latex.eval` method for the :class:`Latex`
    class, and in :func:`_latex_file_`; it follows either
    ``LATEX_HEADER`` or ``SLIDE_HEADER`` (defined at the top of this
    file) which is a string containing the documentclass and standard
    usepackage commands.

    EXAMPLES::

        sage: from sage.misc.latex import latex_extra_preamble
        sage: print(latex_extra_preamble())
        <BLANKLINE>
        \newcommand{\ZZ}{\Bold{Z}}
        \newcommand{\NN}{\Bold{N}}
        \newcommand{\RR}{\Bold{R}}
        \newcommand{\CC}{\Bold{C}}
        \newcommand{\QQ}{\Bold{Q}}
        \newcommand{\QQbar}{\overline{\QQ}}
        \newcommand{\GF}[1]{\Bold{F}_{#1}}
        \newcommand{\Zp}[1]{\Bold{Z}_{#1}}
        \newcommand{\Qp}[1]{\Bold{Q}_{#1}}
        \newcommand{\Zmod}[1]{\ZZ/#1\ZZ}
        \newcommand{\CDF}{\Bold{C}}
        \newcommand{\CIF}{\Bold{C}}
        \newcommand{\CLF}{\Bold{C}}
        \newcommand{\RDF}{\Bold{R}}
        \newcommand{\RIF}{\Bold{I} \Bold{R}}
        \newcommand{\RLF}{\Bold{R}}
        \newcommand{\Bold}[1]{\mathbf{#1}}
        <BLANKLINE>
    """
    from sage.misc.latex_macros import sage_latex_macros
    return "\n".join([_Latex_prefs._option['preamble'],
                     "\n".join(sage_latex_macros()),
                     _Latex_prefs._option['macros']])

Somehow we need to have the Sage Jupyter Notebook extension output some JavaScript to define these in MathJax. I'm not really sure how that works though. Someone with more experience with both Juypter Extensions and MathJax should chime in on that...

[williamstein]

comment:20

Tiny side note:

This is a problem in the Jupyter notebook in general.

For what it is worth, these macros work in our implementation of Jupyter (for CoCalc), with any kernel. Also, we use KaTeX by default, which is faster for math rendering. In addition, we cache rendering, so if you render the same formula again, it's much faster.

It might be easier just to get a PR into Jupyter that predefines some useful latex macros. You would also want to support nbconvert.

[jhpalmieri]

comment:21

These macros are defined in src/sage/misc/latex_macros.py. That file has a function sage_mathjax_macros, which contains versions of these suitable for use with MathJax, and that is what gets used in the legacy Sage notebook. I would hope there is a way to get the Jupyter notebook to use them.

[zerline]

comment:22

A few elements about Sphinx code highlighting:

1. Sphinx uses pygments "friendly" style, with some very small changes.

2. At documentation build stage, Sphinx creates a CSS file called pygments.css:

from builders/html.py

    def copy_static_files(self):
        # type: () -> None
        try:
            # copy static files
            logger.info(bold(__('copying static files... ')), nonl=True)
            ensuredir(path.join(self.outdir, '_static'))
            # first, create pygments style file
            with open(path.join(self.outdir, '_static', 'pygments.css'), 'w') as f:
                f.write(self.highlighter.get_stylesheet())  # type: ignore

3. You can find the resulting CSS file in the HTML documentation tree, ie on

http://doc.sagemath.org/html/en/reference/_static/pygments.css

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

3067b7f	Add strip_math_delims option to sphinxify so that its current default
5ed3d00	Add sphinxify_mimebundle function for sending docstrings as text+HTML
ec1e13a	Add the necessary default settings to enable HTML output in the Jupyter

[sagetrac-git]

Changed commit from 1a56e10 to ec1e13a

[embray]

comment:24

I see; we probably don't want to rely on the full HTML docs being built/installed in order to load this stylesheet in the Jupyter kernel though; I'll look at that copy_static_files function and see if we can easily replicated its functionality, e.g., when installing the kernel.

---

New commits:

3067b7f	Add strip_math_delims option to sphinxify so that its current default
5ed3d00	Add sphinxify_mimebundle function for sending docstrings as text+HTML
ec1e13a	Add the necessary default settings to enable HTML output in the Jupyter

[egourgoulhon]

comment:25

May I ask about the status of this? It would be so nice to have it in the Sage 9.0 release. Even if it is not fully perfect, it is much much better than having the pure ASCII documentation in Jupyter notebooks...

[embray]

comment:26

I haven't had time to work on it so if someone would like to take it over they are welcome to. I think improving some of the CSS stuff should still be done.

Since this might render some documentation less legible (e.g. due to the CSS or latex issues) it might be good to have a global function in Sage to enable/disable this (but it could be enabled by default in most cases I think...)