Extension to autogenerate API reference docs for Python

[SamEdwardes]

What

It would be awesome if quarto had an extension to turn Python docstrings into organized and rendered HTML pages automatically. This could enable quarto to be used by Python package authors who want or need to create automatically generated API reference.

Current state

Right now the biggest tools in the python documentation eco system are:

Tool	Auto API Generation	Notes
Sphinx	Yes - autodoc	- The most commonly used tool in the Python eco system - Very powerful and nice looking API reference generation - Does not feel modern - Docs are written in RST (you can write MD but it requires some extra work) - Example: https://pandas.pydata.org/docs/reference/index.html#
MkDocs	No - but you can do it with mkdocstrings. However, I think the output of mkdocstrings is not as nice or as powerful as what you get with sphinx	- Generates really nice looking sites with material for mkdocs - No built in API reference generation - No built in code execution - Example: https://fastapi.tiangolo.com

Why Quarto?

• It is much easier to get started then sphinx
• The writing experience is similar to mkdocs - however the ability for quarto to execute python code could make it really easy to write tutorials and "article style" documentation
• If quarto had the ability to auto generated API reference that was as powerful and looked as good as sphinx it could be the best tool for Python documentation.

Questions

• Does anyone know of existing quarto extensions that do this?
• I assume this should be an extension and not core functionality.
• Are there any approaches to do this yet for the R language? I saw this (ecodown, `pkgdown` in Quarto? #1591) from Edgar, but I am not sure if there are plans to use this outside of RStudio or continue to develop it.

[SamEdwardes]

@edgararuiz have you put any thought into this for Python? I am interested to know what you think or how you would approach this.

[SamEdwardes]

@machow I am also curious about what you think. How you built the pins docs look really clean. Did you combine sphinx autodoc with jupyter book? Can we do the same with quarto?

https://github.com/rstudio/pins-python/tree/main/docs

[machow]

Ah, I looked deeper into this exact topic today! jupyterbook is built on top of sphinx, so pins-python was able to use that.

For quarto, we'd probably want to generate markdown (or HTML). There are a couple relevant tools:

• nbdev, generates API docs in quarto from jupyter notebooks. They have code to parse docstrings, and AFAICT a custom notebook processor and executor.
• mkdocstring. Popular tool that builds API docs on top of mkdocs. It's a mkdocs plugin, and seems a bit tricky to modify to work with quarto. But it seems like a promising avenue.

There may be more, and ideally they'd be able to do these things:

• Allow you to specify which classes, methods, or functions you want to include
• Allow interlinking mentions of functions in docstrings to each other
• Handle a range of cases, including docstrings set dynamically
• Be able to parse the full numpydoc spec

[SamEdwardes]

Thanks for the details @machow. Very cool how jupyter-book lets you incorporate rst files and use the sphinx directives/syntax. I also did not know nbdev outputs to quarto.

I think there is a need in the python system for a lower-level tool that can:

• parse doc strings
• output them to different formats
• plus all the things you mentioned:
  • Allow you to specify which classes, methods, or functions you want to include
  • Allow interlinking mentions of functions in docstrings to each other
  • Handle a range of cases, including docstrings set dynamically
  • Be able to parse the full numpydoc spec

If this tool was written in Python, and exposed via a Python API and/or the command line it could then be integrated with a bunch of tools like quarto, jupyter-book, etc.

Here is an example of what the API could look like.

from pins import board_temp
from newdoctool import document_function

# Document a single function
document_function(board_temp, output_format="html", theme="cosmos")

Imagine you include the above snippet inside a quarto file, the output of the code block would be markdown or html that quarto would render.

Or from the command line

python -m newdoctool document function --function-name board_temp --module-path ./pins/boards.py

You could have functions that document an entire module.

from pins import board
from newdoctool import document_module

# Document an entire module
document_module(board, output_format="markdown", theme="cosmos", page_per_function=False)

Or from the command line

python -m newdoctool document module --module-path ./pins/boards.py

You could have functions that automatically document an entire package.

import pins
from newdoctool import document_package

# Document an entire package
document_package(
    board, 
    output_format="markdown", 
    theme="cosmos", 
    page_per_function=False,
    page_per_module=True
)

Or from the command line

python -m newdoctool document package --package-path ./pins

[mcanouil]

• Does anyone know of existing quarto extensions that do this?
• I assume this should be an extension and not core functionality.

Far as I know, there is none and it would be a quite big extension (in number of lines of code). I am not sure it could be an extension as in the extension mechanism of Quarto. But more of a pkgdown like approach for Python and Quarto.
There is docstring-to-markdown which can be a start to a Quarto website documentation I guess.

• Are there any approaches to do this yet for the R language? I saw this (ecodown, `pkgdown` in Quarto? #1591) from Edgar, but I am not sure if there are plans to use this outside of RStudio or continue to develop it.

ecodown is in fact not an R package to do what pkgdown is doing for R packages documentation in Rmarkdown.

It could indeed be really nice to be able to build documentation website for Python/Julia/R libraries/packages natively.
This would allow to make one single website with multiple tools that might play well together, instead of having multiple independent websites.

[d-chambers]

I am also very interested in replacing sphinx with quarto, and the API generation seems to be the only missing component.

[cscheid]

@d-chambers Try out nbdev. maybe? It was mentioned upthread and is designed with this usecase precisely in mind

[SamEdwardes]

This looks promising: https://nbdev.fast.ai/tutorials/modular_nbdev.html

from nbdev.showdoc import show_doc
from numpy import all
show_doc(all)

I will try it out!

[d-chambers]

Yes, that does look like exactly what I am looking for. Thanks @SamEdwardes and @cscheid.

I guess the only missing piece would be linking between the tutorial docs and API docs. Suppose we are documenting numpy, something like this would be useful;

# tutorial.qmd

Numpy does basic statistics, like [median](np.median). 
 or maybe
[median](`np.median`)

where the link refers to the median API page. I suppose the easiest way to do this is to write a project-specific quarto filter that would replace the ref with the appropriate API markdown path?

[d-chambers]

A quick update:

I ended up rolling my own solution for this. It was both more challenging and interesting than I initially anticipated. The implementation can be found in the DASCore library, and here is the documentation page.

It has a few components:

1. A script to index the structure of the library and create a json file which maps a library address (e.g. dascore.Patch) to the corresponding API markdown file (e.g., docs/api/dascore/Patch.qmd)
2. A script to render the markdown files.
3. Some custom CSS for styling.
4. A panflute filter for applying the mapping created by the index filter.

Now the project has auto-generated API pages (though far from perfect) and internal cross-references work like this: [SomeClass](`library.SomeClass`) . There are still a lot of things I would like to improve, primarily making this functionality into its own package with better testing, but my efforts so far serve as a proof of concept. I am hopeful quarto will be a viable alternative to sphinx for documenting python libraries in the near future.

[machow]

Hey! If you're interested, I've been working on a library to generate API docs called quartodoc.

I think its aims are similar, especially focused on being able to fully port a sphinx site. Would love to collaborate, since there are a lot of weird pieces to sphinx 😓.

Here's its strategy right now:

1. Crossrefs index: Generate a sphinx inventory file for mapping cross-references. sphinx sites often generate these, so this would allow linking to existing python doc sites as well (e.g. python.org documentation)
2. Docstring parsing: Use griffe to read and parse docstrings
3. Render: Use tree visitors to go from griffe parsed docstrings -> markdown or html (similar to pkgdown)
4. Do crossrefs: Working on a rough lua filter to do links based on sphinx inventory files.

Here are docs on how it reads/renders docstrings, and what's there so far for crossrefs

[SamEdwardes]

@machow this looks awesome! I feel like this is a gap in the python ecosystem. I am excited to see where it goes :)

[d-chambers]

@machow,

This looks great! You are really taking the time to think through it and design a nice package, unlike just hacking something together like I did. After looking at the sphinx inventory stuff the term "dumpster fire" comes to mind, but compatibility with other python packages is probably worth it, especially if sphobjinv does the heavy lifting for you.

I started to write a Lua filter, but, after profiling, I found the python filter only accounted for about 25% of the build time, so it wasn't worth it to spend a few days learning Lua.

I will try to find some time to look though quartodoc and discuss more with you over there. Thanks for sharing!

[machow]

Your panflute stuff is really helpful to look at! I couldn't figure it out, so dove into the world of lua, but staying in python definitely would have been a lot faster 😬.

In case it's helpful--I've added a rough example of rendering markdown pages for dascore to the quartodoc examples :). I wasn't able to install dascore (something related to hdf5 and my computer being an M1 mac?), but griffe does static analysis, so I just used pip install dascore --no-dependencies.

There were a few functions it couldn't render (would be quick to fix in the renderer), and I didn't generate an inventory file, but should hopefully give a feel for how griffe works!

[d-chambers]

Thanks for making me aware of the M1 dascore installation issue, I will look into it.

Also, huge thanks for creating the dascore example. I will try to play with it more once finals are over next week.

[mcanouil]

Closing this as completed, see https://github.com/machow/quartodoc for example.

[SamEdwardes]

Nice - https://github.com/machow/quartodoc looks awesome. I am excited to try it.