Builds on Python 3.7 started failing due to a problem with PyYaml during pip install

[sphuber]

Describe the bug
Builds on Python 3.7 started failing as of yesterday. Seems to be that the version got upped from 3.7.7 to 3.7.8.

Which version of the action are you using?

• v1
• v2
• Some other tag (such as v2.0.1 or master)

Tested with both versions of the action. Appears for both v1 and v2

Environment

• self-hosted
• Linux
• Windows
• Mac

If applicable, please specify if you're using a container

Python Versions
3.7.8 only

To Reproduce
Check this build: https://github.com/aiidateam/aiida-quantumespresso/runs/854338665?check_suite_focus=true

Run/Repo Url
https://github.com/aiidateam/aiida-quantumespresso/runs/854338665?check_suite_focus=true

Additional context
We have been running our builds successfully using this action @v1 for Python 3.7. Since yesterday, the builds started failing at the step where we install python dependencies through pip. It seems that the Python version got upped from 3.7.7 to 3.7.8. It fails because pyyaml==5.3.1 is installed on the system and gets uninstalled by pip but that somehow fails. See the linked build for more information on the stack trace that I copy here:

  Running setup.py develop for aiida-quantumespresso
Successfully installed Mako-1.1.3 PyCifRW-4.4.1 aiida-core-1.3.0 aiida-quantumespresso aldjemy-0.9.1 alembic-1.4.2 ase-3.19.1 backcall-0.2.0 bcrypt-3.1.7 cffi-1.14.0 circus-0.16.1 click-completion-0.5.2 click-config-file-0.6.0 click-spinner-0.1.10 configobj-5.0.6 cryptography-2.9.2 cycler-0.10.0 decorator-4.4.2 django-2.2.14 elementpath-1.4.6 ete3-3.1.1 frozendict-1.2 furl-2.1.0 graphviz-0.14 ipython-7.16.1 ipython-genutils-0.2.0 jedi-0.17.1 jinja2-2.11.2 kiwipy-0.5.5 kiwisolver-1.2.0 matplotlib-3.2.2 monty-3.0.2 more-itertools-8.4.0 mpmath-1.1.0 networkx-2.4 numpy-1.17.5 orderedmultidict-1.0.1 packaging-20.4 palettable-3.3.0 pandas-1.0.5 paramiko-2.7.1 parso-0.7.0 pexpect-4.8.0 pg8000-1.15.3 pgsu-0.1.0 pgtest-1.3.2 pickleshare-0.7.5 pika-1.1.0 plotly-4.8.2 pluggy-0.13.1 plumpy-0.15.0 prompt-toolkit-3.0.5 psutil-5.7.0 psycopg2-binary-2.8.5 ptyprocess-0.6.0 py-1.9.0 pycparser-2.20 pygments-2.6.1 pymatgen-2020.7.3 pymysql-0.9.3 pynacl-1.4.0 pyparsing-2.4.7 pytest-5.4.3 pytest-datadir-1.3.1 pytest-regressions-1.0.6 python-editor-1.0.4 pytz-2019.3 pyyaml-5.1.2 pyzmq-19.0.1 qe-tools-1.1.4 reentry-1.3.1 retrying-1.3.3 ruamel.yaml-0.16.10 ruamel.yaml.clib-0.2.0 scipy-1.5.1 scramp-1.2.0 seekpath-1.9.7 shellingham-1.3.2 shortuuid-1.0.1 simplejson-3.17.0 spglib-1.15.1 sqlalchemy-1.3.18 sqlalchemy-utils-0.34.2 sqlparse-0.3.1 sympy-1.6.1 tabulate-0.8.7 topika-0.2.1 tornado-4.5.3 tqdm-4.47.0 traitlets-4.3.3 upf-to-json-0.9.2 wcwidth-0.2.5 wrapt-1.11.2 xmlschema-1.2.2
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.7.8/x64/bin/pip", line 8, in <module>
    sys.exit(main())
  File "/opt/hostedtoolcache/Python/3.7.8/x64/lib/python3.7/site-packages/pip/_internal/cli/main.py", line 75, in main
    return command.main(cmd_args)
  File "/opt/hostedtoolcache/Python/3.7.8/x64/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 114, in main
    return self._main(args)
  File "/opt/hostedtoolcache/Python/3.7.8/x64/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 226, in _main
    self.handle_pip_version_check(options)
  File "/opt/hostedtoolcache/Python/3.7.8/x64/lib/python3.7/site-packages/pip/_internal/cli/req_command.py", line 155, in handle_pip_version_check
    timeout=min(5, options.timeout)
  File "/opt/hostedtoolcache/Python/3.7.8/x64/lib/python3.7/site-packages/pip/_internal/cli/req_command.py", line 100, in _build_session
    index_urls=self._get_index_urls(options),
  File "/opt/hostedtoolcache/Python/3.7.8/x64/lib/python3.7/site-packages/pip/_internal/network/session.py", line 249, in __init__
    self.headers["User-Agent"] = user_agent()
  File "/opt/hostedtoolcache/Python/3.7.8/x64/lib/python3.7/site-packages/pip/_internal/network/session.py", line 159, in user_agent
    setuptools_version = get_installed_version("setuptools")
  File "/opt/hostedtoolcache/Python/3.7.8/x64/lib/python3.7/site-packages/pip/_internal/utils/misc.py", line 634, in get_installed_version
    working_set = pkg_resources.WorkingSet()
  File "/opt/hostedtoolcache/Python/3.7.8/x64/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 567, in __init__
    self.add_entry(entry)
  File "/opt/hostedtoolcache/Python/3.7.8/x64/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 623, in add_entry
    for dist in find_distributions(entry, True):
  File "/opt/hostedtoolcache/Python/3.7.8/x64/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 1983, in find_eggs_in_zip
    if metadata.has_metadata('PKG-INFO'):
  File "/opt/hostedtoolcache/Python/3.7.8/x64/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 1414, in has_metadata
    return self._has(path)
  File "/opt/hostedtoolcache/Python/3.7.8/x64/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 1854, in _has
    return zip_path in self.zipinfo or zip_path in self._index()
  File "/opt/hostedtoolcache/Python/3.7.8/x64/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 1731, in zipinfo
    return self._zip_manifests.load(self.loader.archive)
  File "/opt/hostedtoolcache/Python/3.7.8/x64/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 1688, in load
    mtime = os.stat(path).st_mtime
FileNotFoundError: [Errno 2] No such file or directory: '/opt/hostedtoolcache/Python/3.7.8/x64/lib/python3.7/site-packages/PyYAML-5.3.1-py3.7-linux-x86_64.egg'

Edit: I tried running with the version pinned to Python 3.7.7 and the problem disappeared: https://github.com/aiidateam/aiida-quantumespresso/runs/854391978?check_suite_focus=true

[ltalirz]

Just noting that this was already discussed here actions/runner-images#1202

As mentioned there, uninstalling pyyaml 5.3.1 actually works fine.
The issue occurs when trying to install pyyaml 5.1.2 under python 3.7.8 (while installing pyyaml 5.1.2 under python 3.7.7 works fine).
I don't think this is the fault of the setup-python github action and suggest to close this issue.
Not sure whether it's worth opening an issue with pyyaml on this... at least for us, an easier solution might be to just upgrade pyyaml

[sphuber]

Thanks @ltalirz . Closing this then.

[Borda]

The same also here - https://github.com/PyTorchLightning/pytorch-lightning/runs/889998403
for all OS (ubuntu-18.04, windows-2019, macOS-10.15) and python 3.6, 3.7 and 3.8

[DataGreed]

I have the same problem, is there any fix for this?

I also see that there is a ticket opened in other repo actions/runner-images#1202 - should I follow it instead?

[ltalirz]

@DataGreed I think so - it looks like an issue with the environment on the runner (when setup-python actually reuses the one that is already present)