Integrating sphinx gallery to run documentation

[agoscinski]

Draft trying out sphinx gallery. I chose sphinx gallery because most of the documentation are already ipynb that can be directly transferred to sphinx-gallery compatible python files while with sphinx-jupyter you need to transform them manually to .rst files.

Locally it worked, but need to check if it works also on readthedocs. For now it is just transforming the concept/task.ipynb.

TODOs:

• Document how to convert ipynb to sphinx python in CONTRIBUTING.md
• move all ipynb in the concept section to gallery
• clean up
  • the README.txt are not necessary can move back to emtpy gallery headers
  • conf.py a bit ugly
  • rm mv html script

For future PRs

• Merge sphinx requirements.txt with the docs optional environment (will do in another PR)
• add conda dependency for aiida services to start a rabbitmq server for examples that submit jobs https://docs.readthedocs.io/en/stable/guides/conda.html#effective-use-of-channels
• Convert remaining ipynb to sphinx-gallery scripts, there might be issues with the running time with some of them.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 79.37%. Comparing base (5937b88) to head (6c92221).
Report is 35 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #234      +/-   ##
==========================================
+ Coverage   75.75%   79.37%   +3.62%     
==========================================
  Files          70       62       -8     
  Lines        4615     4567      -48     
==========================================
+ Hits         3496     3625     +129     
+ Misses       1119      942     -177     

Flag	Coverage Δ
python-3.11	79.24% <ø> (+3.57%)	⬆️
python-3.12	79.26% <ø> (?)
python-3.9	79.30% <ø> (+3.57%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[agoscinski]

For getting the widget to work in the CI my first approach was to implement the repr_html but this did not work for some reasons (I assume because of some JS code?). Then I moved to just moving the html files. There is already a function in the conf.pythat takes care of moving the html files in the docs/source/ folder to the docs/build/folder, but the html files are generated in the gallery source folderdocs/galleryon RTD (on my machine it was in thedocs/source/` folder, I dont know why). So now I added a movement of the html files from the gallery source folder to the build folder.

I thought about how to integrate the generated files into the the docs/source. Because the generation of the files creates a page with a gallery, it is best to always create a subfolder for this as it would otherwise overwrite the index.rstfile. One can control the index.rst a bit by providing one in the gallery source folder, but the style always is a bit adapted for a gallery page introducing fragments in the table of content sidebar. So the solution which works is just to reference the files from autogen folder in the index.rst (see docs/source/concept/index.rst). This has the effect that the url changes to autogen/task.html but I think that is fine for now. There is for sure a fix for this, but I leave this for some future PR.

Note for me:
There seems to be bug in sphinx-gallery when using custom index.rst. the README.txt approach worked on the other hand. See related PR sphinx-gallery/sphinx-gallery#1071

[superstar54]

Nice work!

[superstar54]

Can we copy the dependencies from the docs to the requirements.txt, so that we only need to run

pip install -r docs/requirements.txt

[agoscinski]

Yes I want to do it in another PR. But I actually wanted to merge them to the extras .[docs], as we anyway have to specify the building commands now.

[superstar54]

Is this for debug purpose?

[agoscinski]

I would say it is on the info level. I would keep it, because it useful information and makes more transparent what happens to the user. I did not realize that the html files by this are copied until I saw the code. Maybe there is a better way to integrate it with the other sphinx messages, but for now this is okay.

[agoscinski]

Actually there seems to be sphinx logging utils. I make an issue for this.
https://www.sphinx-doc.org/en/master/extdev/logging.html

[agoscinski]

Made an issue #244