Read the Docs Addons enabled by default and injected Sphinx context removal

[humitos]

Hi all 👋🏼. I want to communicate we are deprecating auto-injected Sphinx context and enabling Read the Docs Addons by default on October 7th, 2024. Read the public announcement at our blog post Read the Docs Addons enabled by default.

I'm opening this issue here as a way to connect with other Sphinx theme authors and coordinate the required changes on those themes before reaching the deprecation date and make the transition as smooth as possible:

• @jbms & @2bndy5 (jbms/sphinx-immaterial)
• @agoose77 & @choldgraf (executablebooks/sphinx-book-theme)
• @pradyunsg (pradyunsg/furo)
• @drammock (pydata/pydata-sphinx-theme)
• @mgeier (mgeier/insipid-sphinx-theme)

Feel free to ping other theme authors you may know here so they are aware of the upcoming changes as well.

Summary of the changes

Read the Docs will stop installing readthedocs-sphinx-ext Python package by default and stop appending extra configurations to the Sphinx's conf.py when building projects.

We've created projects on Read the Docs to show the Sphinx configuration for the old and new behavior. This allow us to understand what are the differences (search for html_context on each page, as an example):

• Page with current and deprecated behavior: https://test-builds.readthedocs.io/en/latest/sphinx-context.html
• Page with new default behavior: https://test-builds-disable-sphinx-manipulation.readthedocs.io/en/latest/sphinx-context.html

Details of the changes

When building on Read the Docs, there are some changes performed automatically via a Sphinx configuration file by conf.py.tmpl:

• There are some workarounds to support old versions of Sphinx
• Injects plenty of variables in the html_context
• Adds _static directory to html_static_path if it exists
• Sets alabaster theme if there is no theme defined
• Set html_baseurl as canonical_url if Sphinx >=1.8
• Install readthedocs_ext.readthedocs as a Sphinx extension
• Install readthedocs_ext.external_version_warning if the Read the Docs version is external
• Define some LaTeX configurations based on project's language

Template variables available via html_context

If you are depending on any of this variables injected into the HTML context, you will need to perform some changes on your theme. Note there are some environment variables that Read the Docs passes to the build process that you may be able to use for this purpose. As an example, note that html_context["commit"] won't injected anymore, but you can use the environment variable READTHEDOCS_GIT_COMMIT_HASH instead to get this value.

Read the Docs embeded flyout replaced by new addons

We are moving away from the HTML blob injection we are currently doing to generate the flyout into a #readthedocs-embed-flyout selector. We already implemented Read the Docs Addons as a replacement to this approach which is modular and customizable.

The removal of the HTML blob injection means that if the theme were using an explicit selector to decide where to inject the flyout menu, it has to be migrated to listen to the readthedocs-addons-data-ready JavaScript event to get all the Read the Docs data in a JSON-like¹ object and generate the flyout as you prefer. Note that you can style and render the flyout as you prefer --even splitting it into different selectors, placement or integrate it more with your theme.

Example integrations using the JavaScript custom event:

• Read the Docs Sphinx theme kept the exact look&feel than before.
• CPython documentation doesn't render the flyout as-is but split it into version and language selectors at the top-left of the page.

You are free to integrate this data as you prefer, the only requirement here is to mention the documentation is hosted by Read the Docs somewhere in the page 😄

EthicalAds

If your theme is using a specific placement for the EthicalAd as described in our documentation via #ethical-ad-placement, it should keep working properly.

If that's not the case, Read the Docs will try to place the ad in the best placement possible based on some heuristic.

---

Hopefully, I was able to communicate this clearly enough 😅 . I'm happy to keep this issue open or open one issue per theme, depending on your needs, and I'm happy to collaborate with you all with this migration to the new Read the Docs Addons. Let me know if you have any doubt or if you want me to help you with anything related to this work 👍🏼

Footnotes

1. Unfortunately, not yet documented 😓, but you can find an example of it at https://docs.readthedocs.io/_/addons/?client-version=0.17.0&api-version=1&project-slug=docs&version-slug=stable ↩

[2bndy5]

I'm glad to see this getting finalized. I've been periodically checking on this progress, but I was hesitant to implement anything without a definitive date. 👍🏼

Is there a plan to document the "JSON-like" data? (possibly with a schema?)

[humitos]

Is there a plan to document the "JSON-like" data? (possibly with a schema?)

I have not started documenting this yet. However, we've implemented a way for theme author to pin to a particular version of the JSON-like data, so it won't change over time for that theme even if we keep updating this object for any reason. To pin the version we require the theme to define it via:

<meta name="readthedocs-addons-api-version" content="1">

I don't have too much experience with schemas, but I know that @stsewd have done something similar for our readthedocs.yaml file. So, I think we would follow something similar at some point --but we will definitely start with the documentation first.

[humitos]

@2bndy5 oh, by the way, there is a Material for MkDocs example I built that uses the new addons integration to build the version selector at the top (as Mike plugin does) that you can find at https://test-builds.readthedocs.io/en/mkdocs-material/ to check out how it looks in case you want to follow something similar.

[2bndy5]

I noticed that. I'm more inclined to replace/augment the repo info both in header bar and fly-out hamburger menu (on mobile).

My last attempt didn't go so well (using old approach).

[daltzctr]

What exactly is the purpose behind readthedocs_ext.readthedocs and should users start adding this dependency to their requirements and conf.py?

What exactly does the flyout injection? The extension or readthedocs addons?

Does this back-break older builds of documentation?

[humitos]

What exactly is the purpose behind readthedocs_ext.readthedocs and should users start adding this dependency to their requirements and conf.py?

The purpose of that extension was "to give Read the Docs a way to execute code at build time inside the Sphinx process to integrate better with the platform". This is not required anymore and users should not install this extension.

What exactly does the flyout injection? The extension or readthedocs addons?

Since we moved away from installing that extension and we don't have control on the Sphinx build process anymore, all the integration was migrated to JavaScript and happens on the client side now. The JavaScript file that contains all the addons is injected via a Cloudflare Worker when serving the documentation.

BTW, the repository for the addons is at https://github.com/readthedocs/addons, in case you want to chime in.

One of the goals for this was to make the platform being documentation agnostic. Now, Read the Docs supports any documentation tool and all of them have the same integrations in place 🎉

Does this back-break older builds of documentation?

It shouldn't, but if it does, please let us know and we will try to fix it.

You can test this integration by enabling the addons in your project and then hitting an old documentation version.