Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sphinx 5.1.0 exception: pop from an empty deque #10701

Closed
eivindjahren opened this issue Jul 24, 2022 · 20 comments · Fixed by #10709 or spack/spack#31756
Closed

Sphinx 5.1.0 exception: pop from an empty deque #10701

eivindjahren opened this issue Jul 24, 2022 · 20 comments · Fixed by #10709 or spack/spack#31756

Comments

@eivindjahren
Copy link

Describe the bug

As of 5.1.0 our doc build at github.com/equinor/ert stopped working and gives us the following unhandled exception:

Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/sphinx/events.py", line 94, in emit
    results.append(listener.handler(self.app, *args))
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/sphinx/ext/napoleon/__init__.py", line 385, in _process_docstring
    docstring = NumpyDocstring(result_lines, app.config, app, what, name,
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/sphinx/ext/napoleon/docstring.py", line 1147, in __init__
    super().__init__(docstring, config, app, what, name, obj, options)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/sphinx/ext/napoleon/docstring.py", line 214, in __init__
    self._parse()
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/sphinx/ext/napoleon/docstring.py", line 594, in _parse
    res = self._parse_attribute_docstring()
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/sphinx/ext/napoleon/docstring.py", line 626, in _parse_attribute_docstring
    _type, _desc = self._consume_inline_attribute()
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/sphinx/ext/napoleon/docstring.py", line 303, in _consume_inline_attribute
    line = self._lines.popleft()
IndexError: pop from an empty deque
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/sphinx/cmd/build.py", line [27](https://github.com/equinor/ert/runs/7488615143?check_suite_focus=true#step:8:28)7, in build_main
    app.build(args.force_all, filenames)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/sphinx/application.py", line 349, in build
    self.builder.build_update()
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 300, in build_update
    self.build(to_build,
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 316, in build
    updated_docnames = set(self.read())
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 423, in read
    self._read_serial(docnames)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 444, in _read_serial
    self.read_doc(docname)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 497, in read_doc
    publisher.publish()
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/docutils/core.py", line 217, in publish
    self.document = self.reader.read(self.source, self.parser,
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/sphinx/io.py", line 104, in read
    self.parse()
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/docutils/readers/__init__.py", line 78, in parse
    self.parser.parse(self.input, document)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/sphinx/parsers.py", line 78, in parse
    self.statemachine.run(inputlines, document, inliner=self.inliner)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 170, in run
    results = StateMachineWS.run(self, input_lines, input_offset,
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/docutils/statemachine.py", line 239, in run
    context, next_state, result = self.check_line(
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/docutils/statemachine.py", line 451, in check_line
    return method(match, context, next_state)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 2769, in underline
    self.section(title, source, style, lineno - 1, messages)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 327, in section
    self.new_subsection(title, lineno, messages)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 393, in new_subsection
    newabsoffset = self.nested_parse(
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line [28](https://github.com/equinor/ert/runs/7488615143?check_suite_focus=true#step:8:29)1, in nested_parse
    state_machine.run(block, input_offset, memo=self.memo,
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 196, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/docutils/statemachine.py", line 239, in run
    context, next_state, result = self.check_line(
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/docutils/statemachine.py", line 451, in check_line
    return method(match, context, next_state)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 2344, in explicit_markup
    self.explicit_list(blank_finish)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 2369, in explicit_list
    newline_offset, blank_finish = self.nested_list_parse(
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 318, in nested_list_parse
    state_machine.run(block, input_offset, memo=self.memo,
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 196, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/docutils/statemachine.py", line 239, in run
    context, next_state, result = self.check_line(
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/docutils/statemachine.py", line 451, in check_line
    return method(match, context, next_state)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 2647, in explicit_markup
    nodelist, blank_finish = self.explicit_construct(match)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 2354, in explicit_construct
    return method(self, expmatch)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 2096, in directive
    return self.run_directive(
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 2146, in run_directive
    result = directive_instance.run()
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/sphinx/ext/autodoc/directive.py", line 148, in run
    documenter.generate(more_content=self.content)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/sphinx/ext/autodoc/__init__.py", line 1788, in generate
    return super().generate(more_content=more_content,
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/sphinx/ext/autodoc/__init__.py", line 954, in generate
    self.document_members(all_members)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/sphinx/ext/autodoc/__init__.py", line 1779, in document_members
    super().document_members(all_members)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/sphinx/ext/autodoc/__init__.py", line 8[30](https://github.com/equinor/ert/runs/7488615143?check_suite_focus=true#step:8:31), in document_members
    documenter.generate(
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/sphinx/ext/autodoc/__init__.py", line 951, in generate
    self.add_content(more_content)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/sphinx/ext/autodoc/__init__.py", line 602, in add_content
    for i, line in enumerate(self.process_doc(docstrings)):
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/sphinx/ext/autodoc/__init__.py", line 547, in process_doc
    self.env.app.emit('autodoc-process-docstring',
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/sphinx/application.py", line [46](https://github.com/equinor/ert/runs/7488615143?check_suite_focus=true#step:8:47)0, in emit
    return self.events.emit(event, *args, allowed_exceptions=allowed_exceptions)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/sphinx/events.py", line 105, in emit
    raise ExtensionError(__("Handler %r for event %r threw an exception") %
sphinx.errors.ExtensionError: Handler <function _process_docstring at 0x7fe6a9[57](https://github.com/equinor/ert/runs/7488615143?check_suite_focus=true#step:8:58)2280> for event 'autodoc-process-docstring' threw an exception (exception: pop from an empty deque)
Extension error (sphinx.ext.napoleon):
Handler <function _process_docstring at 0x7fe6a95[72](https://github.com/equinor/ert/runs/7488615143?check_suite_focus=true#step:8:73)2[80](https://github.com/equinor/ert/runs/7488615143?check_suite_focus=true#step:8:81)> for event 'autodoc-process-docstring' threw an exception (exception: pop from an empty deque)
Error: Process completed with exit code 2.

How to Reproduce

Following works in my environment, if there are issues, see https://github.com/equinor/ert/blob/ec015e87ef511f45826ae53c0aae1d6e1a99b9f1/.github/workflows/build.yml#L195 for how its
done in CI.

$ git clone https://github.com/equinor/ert
$ cd ert
$ pip install .
$ pip install -r dev_requirements.txt
$ sphinx-build -n -v -E -W ./docs/rst/manual ./tmp/ert_docs

Expected behavior

Documentation is built

Your project

github.com/equinor/ert

Screenshots

No response

OS

Ubuntu-latest

Python version

3.8

Sphinx version

5.1.0

Sphinx extensions

"sphinx.ext.intersphinx", "sphinx.ext.mathjax", "sphinx.ext.ifconfig", "sphinx.ext.githubpages", "sphinx.ext.autodoc", "sphinx.ext.napoleon", "sphinxarg.ext", "sphinx.ext.todo", "sphinxcontrib.datatemplates",

Extra tools

No response

Additional context

Our project uses some custom plugins, and we are trying to figure out if they are acting up.

@AA-Turner
Copy link
Member

This is related to the napolean.iterators change (#9856) it seems, cc @anntzer

A

@AA-Turner
Copy link
Member

AA-Turner commented Jul 24, 2022

@eivindjahren I can't install ERT on Windows to reproduce (Python 3.9; pip 22.2):

(venv) I:\Development\sphinx-bugfixing\ert>pip install .
Processing i:\development\sphinx-bugfixing\ert
  Installing build dependencies ... error
  error: subprocess-exited-with-error

  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [18 lines of output]
      Collecting setuptools
        Using cached setuptools-63.2.0-py3-none-any.whl (1.2 MB)
      Collecting setuptools_scm
        Downloading setuptools_scm-7.0.5-py3-none-any.whl (42 kB)
           -------------------------------------- 42.9/42.9 kB 695.1 kB/s eta 0:00:00
      Collecting wheel
        Using cached wheel-0.37.1-py2.py3-none-any.whl (35 kB)
      Collecting scikit-build
        Downloading scikit_build-0.15.0-py2.py3-none-any.whl (77 kB)
           ---------------------------------------- 77.5/77.5 kB 1.1 MB/s eta 0:00:00
      Collecting cmake
        Downloading cmake-3.22.5-py2.py3-none-win_amd64.whl (38.1 MB)
           ---------------------------------------- 38.1/38.1 MB 1.2 MB/s eta 0:00:00
      Collecting ninja
        Downloading ninja-1.10.2.3-py2.py3-none-win_amd64.whl (292 kB)
           ------------------------------------ 292.6/292.6 kB 950.7 kB/s eta 0:00:00
      ERROR: Could not find a version that satisfies the requirement ecl (from versions: none)
      ERROR: No matching distribution found for ecl
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Any suggestions?

A

@eivindjahren
Copy link
Author

@AA-Turner unfortunately we don't support windows. I am guessing the problem is that it's not finding a windows wheel for ecl.

eivindjahren added a commit to eivindjahren/ert that referenced this issue Jul 24, 2022
sphinx-5.1.0 broke our build. Issue raised here:
sphinx-doc/sphinx#10701. Pinning for now.
@yarikoptic
Copy link

yarikoptic commented Jul 24, 2022

seems due to @propertyes having no docstring, e.g. in our case:

  File "/home/yoh/proj/datalad/datalad-maint/venvs/dev3/lib/python3.10/site-packages/sphinx/ext/napoleon/docstring.py", line 626, in _parse_attribute_docstring
    _type, _desc = self._consume_inline_attribute()
  File "/home/yoh/proj/datalad/datalad-maint/venvs/dev3/lib/python3.10/site-packages/sphinx/ext/napoleon/docstring.py", line 303, in _consume_inline_attribute
    line = self._lines.popleft()
IndexError: pop from an empty deque
> /home/yoh/proj/datalad/datalad-maint/venvs/dev3/lib/python3.10/site-packages/sphinx/ext/napoleon/docstring.py(303)_consume_inline_attribute()
-> line = self._lines.popleft()
(Pdb) up
> /home/yoh/proj/datalad/datalad-maint/venvs/dev3/lib/python3.10/site-packages/sphinx/ext/napoleon/docstring.py(626)_parse_attribute_docstring()
-> _type, _desc = self._consume_inline_attribute()
(Pdb) 
> /home/yoh/proj/datalad/datalad-maint/venvs/dev3/lib/python3.10/site-packages/sphinx/ext/napoleon/docstring.py(594)_parse()
-> res = self._parse_attribute_docstring()
(Pdb) 
> /home/yoh/proj/datalad/datalad-maint/venvs/dev3/lib/python3.10/site-packages/sphinx/ext/napoleon/docstring.py(214)__init__()
-> self._parse()
(Pdb) 
> /home/yoh/proj/datalad/datalad-maint/venvs/dev3/lib/python3.10/site-packages/sphinx/ext/napoleon/docstring.py(1147)__init__()
-> super().__init__(docstring, config, app, what, name, obj, options)
(Pdb) p name
'datalad.support.archives.ExtractedArchive.is_extracted'
(Pdb) 
[2]  + 1098197 suspended  DATALAD_SPHINX_RUN=1 sphinx-build -b html -d build/doctrees -W source -v -P 
(dev3) (datalad-py3.7) 1 39129 ->148 [2].....................................:Sun 24 Jul 2022 01:56:21 PM EDT:.
(git)lena:~datalad/datalad-maint[maint]docs
$> git grep -4 'def is_extracted'
(dev3) (datalad-py3.7) 1 39130 ->1 [2].....................................:Sun 24 Jul 2022 01:56:32 PM EDT:.
(git)lena:~datalad/datalad-maint[maint]docs
$> git -C .. grep -4 'def is_extracted'
datalad/support/archives.py-    def stamp_path(self):
datalad/support/archives.py-        return self._path + self.STAMP_SUFFIX
datalad/support/archives.py-
datalad/support/archives.py-    @property
datalad/support/archives.py:    def is_extracted(self):
datalad/support/archives.py-        return exists(self.path) and exists(self.stamp_path) \
datalad/support/archives.py-            and os.stat(self.stamp_path).st_mtime >= os.stat(self.path).st_mtime
datalad/support/archives.py-
datalad/support/archives.py-    def assure_extracted(self):

so adding dummy docstring to those properties worksaround the issue

edit: a quick dirty monkey patching workaround https://github.com/datalad/datalad/pull/6883/files#diff-008dcb3426febd767787b1521f1fe33086313b927ea37eaab86df5fa88a51698R356 with hope that 5.1.1 will soon fix this issue properly.

yarikoptic added a commit to yarikoptic/datalad that referenced this issue Jul 24, 2022
See sphinx-doc/sphinx#10701 for more information.
I have not checked if rendered docs are anyhow affected etc. I hope the issue
would get fixed asap since should affect lots of sphinx users
@adamjstewart
Copy link
Contributor

@PeterBaker0
Copy link

Similar error on essentially blank project following guide from https://py-pkgs.org/

PeterBaker0 added a commit to gbrrestoration/mds-is-client-tools that referenced this issue Jul 25, 2022
@PeterBaker0
Copy link

edit: a quick dirty monkey patching workaround https://github.com/datalad/datalad/pull/6883/files#diff-008dcb3426febd767787b1521f1fe33086313b927ea37eaab86df5fa88a51698R356 with hope that 5.1.1 will soon fix this issue properly.
@yarikoptic just confirming this fixed the issue for me as well - cheers

eivindjahren added a commit to equinor/ert that referenced this issue Jul 25, 2022
sphinx-5.1.0 broke our build. Issue raised here:
sphinx-doc/sphinx#10701. Pinning for now.
@MatthijsBurgh
Copy link
Contributor

sebix pushed a commit to certtools/intelmq that referenced this issue Jul 25, 2022
add workaround of @yarikoptic to conf.py for Sphinx 5.1.0
sebix pushed a commit to certtools/intelmq that referenced this issue Jul 25, 2022
add workaround of @yarikoptic to conf.py for Sphinx 5.1.0

fixes #2224
oleksandr-pavlyk added a commit to IntelPython/dpctl that referenced this issue Jul 25, 2022
Due to sphinx-doc/sphinx#10701 documentation build of dpctl started failing
with release of Sphinx 5.1.0.

The exception raised is triggered by missing docstrings for property class
attributes.

This change adds such docstrings and now local documentation build goes through
as expected.
sebix added a commit to certtools/intelmq that referenced this issue Jul 25, 2022
@phiadaarr
Copy link

I plan to release 5.1.1 tomorrow, details in #10710. Please test the 5.1.x branch as soon as possible, as set out in #10710.

I had the same problem and the current version of 5.1.x seems to fix it. Thanks!

@vfazio
Copy link

vfazio commented Jul 26, 2022

#10709 seems to address the issue in testing. I guess my concern is that no unit tests caught this scenario, so i'm curious if that's something that could be added to ensure we don't see a reoccurrence if the implementation is changed in the future

@AA-Turner
Copy link
Member

Sphinx 5.1.1 has been released, see #10710.

A

@adamjstewart
Copy link
Contributor

https://www.sphinx-doc.org/en/master/changes.html doesn't actually list the changes in 5.1.1

@AA-Turner
Copy link
Member

You're too quick for CI, the website is currently rendering!

A

@AA-Turner
Copy link
Member

Now live: https://www.sphinx-doc.org/en/master/changes.html#release-5-1-1-released-jul-26-2022

@eivindjahren
Copy link
Author

eivindjahren commented Jul 27, 2022

Issue is resolved on our end 🥳 ! Thanks for the help @AA-Turner !

@cacti77
Copy link

cacti77 commented Jul 27, 2022

Happy to report my Azure DevOps docs builds are passing again now that Sphinx 5.1.1 is being picked up. Thanks for fixing this so quickly, @AA-Turner!

@phiadaarr
Copy link

For me, too. Thanks!

1ucian0 added a commit to 1ucian0/qiskit-terra that referenced this issue Jul 28, 2022
mergify bot pushed a commit to Qiskit/qiskit that referenced this issue Jul 28, 2022
mergify bot pushed a commit to Qiskit/qiskit that referenced this issue Jul 28, 2022
mergify bot added a commit to Qiskit/qiskit that referenced this issue Jul 28, 2022
sphinx-doc/sphinx#10701 is fixed
(cherry picked from commit e9f2a2b)

Co-authored-by: Luciano Bello <bel@zurich.ibm.com>
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 27, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.