tests failing with sphinx dev version

[drammock]

our CI job running Sphinx 7.3.0 (current dev version) started failing today, like this:

  + WARNING: while setting up extension sphinx.addnodes: node class 'toctree' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'desc' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'desc_signature' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'desc_signature_line' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'desc_content' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'desc_inline' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'desc_name' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'desc_addname' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'desc_type' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'desc_returns' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'desc_parameterlist' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'desc_type_parameter_list' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'desc_parameter' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'desc_type_parameter' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'desc_optional' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'desc_annotation' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'desc_sig_literal_char' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'desc_sig_literal_number' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'desc_sig_keyword_type' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'desc_sig_punctuation' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'desc_sig_name' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'desc_sig_literal_string' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'desc_sig_keyword' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'desc_sig_operator' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'desc_sig_space' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'versionmodified' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'seealso' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'productionlist' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'production' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'index' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'centered' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'acks' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'hlist' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'hlistcol' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'compact_paragraph' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'glossary' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'only' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'start_of_file' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'highlightlang' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'tabular_col_spec' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'pending_xref' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'number_reference' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'download_reference' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'literal_emphasis' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'literal_strong' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.addnodes: node class 'manpage' is already registered, its visitors will be overridden
  + WARNING: while setting up extension sphinx.domains.changeset: directive 'deprecated' is already registered, it will be overridden
  + WARNING: while setting up extension sphinx.domains.changeset: directive 'versionadded' is already registered, it will be overridden
  + WARNING: while setting up extension sphinx.domains.changeset: directive 'versionchanged' is already registered, it will be overridden
  + WARNING: while setting up extension sphinx.domains.index: directive 'index' is already registered, it will be overridden
  + WARNING: while setting up extension sphinx.domains.index: role 'index' is already registered, it will be overridden
  + WARNING: while setting up extension sphinx.domains.math: role 'eq' is already registered, it will be overridden

Looks like probably (?) an upstream bug, but I haven't dug into it yet. Any takers?

[12rambau]

I fel on this issue in Sphinx tracker: sphinx-doc/sphinx#9841, could this be related to an incompatibility between sphinx and latests docutils ?

[drammock]

Both my local setup and CI setup are using sphinx@1785fc93520cc7595f740f3270f715d17a67c7df and docutils 0.20.1 --- I'll try downgrading to 0.20.0

[drammock]

with 0.20.0 I still only see one warning:

Unexpected warning: WARNING: The config value `myst_enable_extensions' has type `list'; expected `Any'.

[drammock]

ok I can repro locally (I forgot that the error was in pytest, not in build site). working on it...

[drammock]

the problem, I think, is something to do with sphinx_build_factory. Any individual test, run by itself, will pass. But every subsequent test run in that session will fail, unless the test does sphinx_build.build(no_warning=False) (which we occasionally do if we want to check for the presence of a particular warning).

I suspect something along the lines of "the sphinx app is supposed to be a singleton and it's not" (which I saw mentioned in rstcheck/rstcheck#113).

[AA-Turner]

I changed Sphinx's own test set up recently (sphinx-doc/sphinx@462404c), would that be an impact?

A

[drammock]

I changed Sphinx's own test set up recently (sphinx-doc/sphinx@462404c), would that be an impact?

A

indeed that commit is likely the problem. If I locally check out sphinx@841f2bd9b70e3559daf34c0d33210b1d24b621c1 (the immediately preceeding commit) then our tests pass. If I had to guess, I suspect it is the changes to SphinxTestApp.cleanup() that are to blame here.

[drammock]

temporary band-aid is pushed in f420c13 and it appears to have worked

@12rambau note that I (perhaps callously) force-pushed to main to shoe-horn that commit before the release bump commit. Hope that doesn't trip you up.

[AA-Turner]

I've pushed sphinx-doc/sphinx@e3b613a, hopefully it serves as a fix?

A