DEV - Refactor CI and environment setup

[trallard]

This PR addresses some of the concerns/issues raised in #1292, more details, caveats, and TODOs below.

What is in this PR

1. A refactored CI (currently in a tests_tox.yml file; if accepted, this should completely replace the existing tests.yml, but I thought I would leave it here in the interim so folks could compare as the CI gets executed. Some notable items:
   • There was a lot of duplication as we set Python and other things multiple times, so I created an action that performs that essential setup task and avoids duplication
   • The current CI workflow involves calling nox using bash scripts to install things, calling sphinx-build directly, etc. Here, we use tox for all the jobs and steps, providing syntax and tool consistency and removing cognitive overload for maintainers and contributors.
   • I made some style improvements to make all the workflow files more readable and easier to follow and added some nice emojis, which are great when looking at the actions CI over and over.
   • Added macos-14 to our testing matrices; even with that and other combinations of OSes, python and Sphinx versions, even with these changes this CI suite is between 3.5 and 4 minutes faster than the existing one
   • Gets rid of codecov upload, which has been a burden and instead uploads results to GH artifacts and prints a summary
2. tox.ini, which is the base of this refactor - this would mean tox will effectively replace nox; therefore, the noxfile.py would be removed. The reasons why I am proposing this replacement are:
   • tox is faster than nox, and I have found that the should_install step in the noxfile is flaky, and I have had to nuke my env multiple times.
   • tox is also very good at handling rebuilds and keeping track of dependencies changes which is something we have been struggling with using nox
   • We need to test against multiple versions of Python and Sphinx, and tox is very flexible and designed for this kind of complex setups
   • With the current tox.ini configuration, we can install PST in development mode for tasks like testing, profiling, etc. and perform other functions by installing a packaged version of PST; this is mostly for building our docs, which more closely mimics how any other user would use PST, thus helping with testing and validation on that end too
   • We can use tox for both CI, local development, docs, compiling, lining, and all (actually, we drop the pre-commit action dependency here and do our lint)

What it does not do

1. We still use stb and sphinx-build as they serve different purposes - sbt helps with the compilation of assets and all node-related things as well as packaging, while sphinx-build only generates our docs in HTML format. So, we are keeping these two dependencies as we need both.
2. It Does not address figure out how to control node version in CIs #998 but makes it easier for me or someone else to extend and test against different versions of node

Still to do

• Delete obsolete files github_actions_install.sh
• Remove obsolete dependencies nox -> deferred to other PR
• Update documentation as needed to replace use of nox

Deferred to other PR

To move this PR and facilitate the merge, I have decided to move this other significant amount of work to another PR.
Note that this means this PR does not get rid of nox, so until this follow-up PR, we have nox and tox, but I have started to move the documentation here to include the use of tox for development tasks and the like.

• Delete obsolete files - noxfile.py
• Build on the work in refactor: update babel build #1294 to add the babel build pieces and update actions
• I want to look into the docs-live set-up, as noted in can we simplify how our tests are defined and run? #1292 This is extremely slow (Sphinx tho), so I am not sure if I can make any improvements. But it can be so slow that I cannot use it or decide not to use it.

Questions

1. We use Python 3.10 in Readthedocs (see readthedocs.yml) and Python 3.12 by default for some of our docs building tasks. Do we want to update the version in RTD for consistency?

If you made it all the way here you deserve a cookie 🍪, thank you 💜

[trallard]

Thanks @Carreau I'd be happy to get this in. I also updated the description to better reflect the content and status of the work.

If @drammock gives a thumbs up let's merge.

[drammock]

I'm on board with this. I've left a few comments/questions on things that looked odd to me; not sure if they're mistakes or just showing my lack of familiarity with tox / actions / etc.

[drammock]

bit confusing, expected 6.1. why not use the tr -d . trick on sphinx version too?

[trallard]

Honestly I cannot remember why I did this but can change to match the rest

[Carreau]

Minor conflict with #1826 that added timeouts. I merge with main, but timeouts may needed to be readded to actions in the future.

[trallard]

I have addressed all the review comments, so if CI is happy, we could merge @drammock

Edit I broke stuff so will fix it

[drammock]

I won't have a chance to look again until Tuesday at the earliest, so please feel free to merge. I don't need to look again. Sent from Proton Mail Android

…

-------- Original Message --------

On 5/23/24 15:00, Tania Allard wrote: I have addressed all the review comments, so if CI is happy, we could merge ***@***.***(https://github.com/drammock) — Reply to this email directly, [view it on GitHub](#1759 (comment)), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/AAN2AUYNT4IJ7YJH55Z257TZDZDHVAVCNFSM6AAAAABFZMWHKSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMRXHEZDKMRUGQ). You are receiving this because you were mentioned.Message ID: ***@***.***>

[Carreau]

Suggested change

	py39-sphinx61-docs: {py39-sphinx61-tests[deps]}
	py39-sphinx61-docs: py39-sphinx61-tests[deps]

I think this is the issue as the error message is

/opt/hostedtoolcache/Python/3.9.19/x64/bin/uv pip install './{py39-sphinx61-tests[deps]}'
error: Failed to read from the distribution cache
  Caused by: failed to query metadata of file `/home/runner/work/pydata-sphinx-theme/pydata-sphinx-theme/{py39-sphinx61-tests[deps]}`

[trallard]

Thanks for all your help, @Carreau 🟣 - I fixed the issue with the deps, and the CI worked well.
I just pushed a change to add the new ubuntu-24.4, so if this works, I will merge it as is. If it fails, I will revert and merge without the latest commit.