Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Upgrading an environment started from 2 base packages fails (only) with new resolver #9572

Open
bersbersbers opened this issue Feb 9, 2021 · 4 comments
Open

Upgrading an environment started from 2 base packages fails (only) with new resolver #9572

bersbersbers opened this issue Feb 9, 2021 · 4 comments
Labels
S: needs triage Issues/PRs that need to be triaged

Comments

@bersbersbers
Copy link

bersbersbers commented Feb 9, 2021 •
edited
Loading

TLDR:

pip install --upgrade argcomplete beautifulsoup4 chardet docx2txt EbookLib extract-msg IMAPClient lxml olefile pdfminer.six Pillow pip pycryptodome PyPDF2 python-pptx pytz setuptools six sortedcontainers soupsieve SpeechRecognition textract tzlocal xlrd XlsxWriter

fails while trying to install very old packages although newer packages are available. The command above is obtained from installing just two packages and then trying to upgrade all installed packages.

  • Clean environment and update pip (I tried 20.2.3, 21.0.1, 21.1.dev0):
pyenv global 3.8.7
pyenv virtualenv-delete temp
pyenv virtualenv temp
pyenv global temp
# pip install --upgrade pip
# python -m pip install -U "pip @ https://github.com/pypa/pip/archive/master.zip"
  • Initialize environment and verify it's consistent
pip install PyPDF2 textract
pip install PyPDF2 textract | sort

gives (for all pip versions I tested):

...
Requirement already satisfied: argcomplete==1.10.0 in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (from textract) (1.10.0)
Requirement already satisfied: beautifulsoup4==4.8.0 in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (from textract) (4.8.0)
Requirement already satisfied: chardet==3.0.4 in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (from textract) (3.0.4)
Requirement already satisfied: docx2txt==0.8 in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (from textract) (0.8)
Requirement already satisfied: EbookLib==0.17.1 in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (from textract) (0.17.1)
Requirement already satisfied: extract-msg==0.23.1 in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (from textract) (0.23.1)
Requirement already satisfied: imapclient==2.1.0 in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (from extract-msg==0.23.1->textract) (2.1.0)
Requirement already satisfied: lxml in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (from EbookLib==0.17.1->textract) (4.6.2)
Requirement already satisfied: olefile==0.46 in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (from extract-msg==0.23.1->textract) (0.46)
Requirement already satisfied: pdfminer.six==20181108 in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (from textract) (20181108)
Requirement already satisfied: Pillow>=3.3.2 in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (from python-pptx==0.6.18->textract) (8.1.0)
Requirement already satisfied: pycryptodome in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (from pdfminer.six==20181108->textract) (3.9.9)
Requirement already satisfied: PyPDF2 in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (1.26.0)
Requirement already satisfied: python-pptx==0.6.18 in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (from textract) (0.6.18)
Requirement already satisfied: pytz in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (from tzlocal==1.5.1->extract-msg==0.23.1->textract) (2021.1)
Requirement already satisfied: six==1.12.0 in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (from textract) (1.12.0)
Requirement already satisfied: sortedcontainers in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (from pdfminer.six==20181108->textract) (2.3.0)
Requirement already satisfied: soupsieve>=1.2 in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (from beautifulsoup4==4.8.0->textract) (2.1)
Requirement already satisfied: SpeechRecognition==3.8.1 in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (from textract) (3.8.1)
Requirement already satisfied: textract in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (1.6.3)
Requirement already satisfied: tzlocal==1.5.1 in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (from extract-msg==0.23.1->textract) (1.5.1)
Requirement already satisfied: xlrd==1.2.0 in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (from textract) (1.2.0)
Requirement already satisfied: XlsxWriter>=0.5.7 in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (from python-pptx==0.6.18->textract) (1.3.7)

Some time later, I try to upgrade my installation. I usually do this because pip is not yet able to consider installed packages:

pip list --format freeze | sed 's/==.*//' | xargs --no-run-if-empty echo pip install --upgrade

which gives (again, for all versions)

pip install --upgrade argcomplete beautifulsoup4 chardet docx2txt EbookLib extract-msg IMAPClient lxml olefile pdfminer.six Pillow pip pycryptodome PyPDF2 python-pptx pytz setuptools six sortedcontainers soupsieve SpeechRecognition textract tzlocal xlrd XlsxWriter

Running this command with pip 20.2.3 works, although it's ends up with an incompatible state:

Collecting argcompleteiter
  Using cached argcomplete-1.12.2-py2.py3-none-any.whl (38 kB)
Collecting beautifulsoup4
  Using cached beautifulsoup4-4.9.3-py3-none-any.whl (115 kB)
Collecting chardet
  Using cached chardet-4.0.0-py2.py3-none-any.whl (178 kB)
Requirement already up-to-date: docx2txt in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (0.8)
Requirement already up-to-date: EbookLib in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (0.17.1)
Collecting extract-msg
  Using cached extract_msg-0.28.1-py2.py3-none-any.whl (67 kB)
Collecting IMAPClient
  Using cached IMAPClient-2.2.0-py2.py3-none-any.whl (179 kB)
Requirement already up-to-date: lxml in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (4.6.2)
Requirement already up-to-date: olefile in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (0.46)
Collecting pdfminer.six
  Using cached pdfminer.six-20201018-py3-none-any.whl (5.6 MB)
Requirement already up-to-date: Pillow in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (8.1.0)
Collecting pip
  Using cached pip-21.0.1-py3-none-any.whl (1.5 MB)
Requirement already up-to-date: pycryptodome in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (3.9.9)
Requirement already up-to-date: PyPDF2 in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (1.26.0)
Requirement already up-to-date: python-pptx in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (0.6.18)
Requirement already up-to-date: pytz in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (2021.1)
Collecting setuptools
  Using cached setuptools-53.0.0-py3-none-any.whl (784 kB)
Collecting six
  Using cached six-1.15.0-py2.py3-none-any.whl (10 kB)
Requirement already up-to-date: sortedcontainers in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (2.3.0)
Requirement already up-to-date: soupsieve in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (2.1)
Requirement already up-to-date: SpeechRecognition in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (3.8.1)
Requirement already up-to-date: textract in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (1.6.3)
Collecting tzlocal
  Using cached tzlocal-2.1-py2.py3-none-any.whl (16 kB)
Collecting xlrd
  Using cached xlrd-2.0.1-py2.py3-none-any.whl (96 kB)
Requirement already up-to-date: XlsxWriter in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (1.3.7)
Collecting compressed-rtf>=1.0.6
  Using cached compressed_rtf-1.0.6.tar.gz (5.8 kB)
Collecting ebcdic>=1.1.1
  Using cached ebcdic-1.1.1-py2.py3-none-any.whl (128 kB)
Collecting cryptography
  Using cached cryptography-3.4.3-cp36-abi3-manylinux2014_x86_64.whl (3.2 MB)
Collecting cffi>=1.12
  Using cached cffi-1.14.4-cp38-cp38-manylinux1_x86_64.whl (411 kB)
Collecting pycparser
  Using cached pycparser-2.20-py2.py3-none-any.whl (112 kB)
Using legacy 'setup.py install' for compressed-rtf, since package 'wheel' is not installed.
Installing collected packages: argcomplete, beautifulsoup4, chardet, tzlocal, six, IMAPClient, compressed-rtf, ebcdic, extract-msg, pycparser, cffi, cryptography, pdfminer.six, pip, setuptools, xlrd
  Attempting uninstall: argcomplete
    Found existing installation: argcomplete 1.10.0
    Uninstalling argcomplete-1.10.0:
      Successfully uninstalled argcomplete-1.10.0
  Attempting uninstall: beautifulsoup4
    Found existing installation: beautifulsoup4 4.8.0
    Uninstalling beautifulsoup4-4.8.0:
      Successfully uninstalled beautifulsoup4-4.8.0
  Attempting uninstall: chardet
    Found existing installation: chardet 3.0.4
    Uninstalling chardet-3.0.4:
      Successfully uninstalled chardet-3.0.4
  Attempting uninstall: tzlocal
    Found existing installation: tzlocal 1.5.1
    Uninstalling tzlocal-1.5.1:
      Successfully uninstalled tzlocal-1.5.1
  Attempting uninstall: six
    Found existing installation: six 1.12.0
    Uninstalling six-1.12.0:
      Successfully uninstalled six-1.12.0
  Attempting uninstall: IMAPClient
    Found existing installation: IMAPClient 2.1.0
    Uninstalling IMAPClient-2.1.0:
      Successfully uninstalled IMAPClient-2.1.0
    Running setup.py install for compressed-rtf ... done
  Attempting uninstall: extract-msg
    Found existing installation: extract-msg 0.23.1
    Uninstalling extract-msg-0.23.1:
      Successfully uninstalled extract-msg-0.23.1
  Attempting uninstall: pdfminer.six
    Found existing installation: pdfminer.six 20181108
    Uninstalling pdfminer.six-20181108:
      Successfully uninstalled pdfminer.six-20181108
  Attempting uninstall: pip
    Found existing installation: pip 20.2.3
    Uninstalling pip-20.2.3:
      Successfully uninstalled pip-20.2.3
  Attempting uninstall: setuptools
    Found existing installation: setuptools 49.2.1
    Uninstalling setuptools-49.2.1:
      Successfully uninstalled setuptools-49.2.1
  Attempting uninstall: xlrd
    Found existing installation: xlrd 1.2.0
    Uninstalling xlrd-1.2.0:
      Successfully uninstalled xlrd-1.2.0
ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way that it resolves dependency conflicts.

We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default.

textract 1.6.3 requires argcomplete==1.10.0, but you'll have argcomplete 1.12.2 which is incompatible.
textract 1.6.3 requires beautifulsoup4==4.8.0, but you'll have beautifulsoup4 4.9.3 which is incompatible.
textract 1.6.3 requires chardet==3.0.4, but you'll have chardet 4.0.0 which is incompatible.
textract 1.6.3 requires extract-msg==0.23.1, but you'll have extract-msg 0.28.1 which is incompatible.
textract 1.6.3 requires pdfminer.six==20181108, but you'll have pdfminer-six 20201018 which is incompatible.
textract 1.6.3 requires six==1.12.0, but you'll have six 1.15.0 which is incompatible.
textract 1.6.3 requires xlrd==1.2.0, but you'll have xlrd 2.0.1 which is incompatible.
extract-msg 0.28.1 requires imapclient==2.1.0, but you'll have imapclient 2.2.0 which is incompatible.
Successfully installed IMAPClient-2.2.0 argcomplete-1.12.2 beautifulsoup4-4.9.3 cffi-1.14.4 chardet-4.0.0 compressed-rtf-1.0.6 cryptography-3.4.3 ebcdic-1.1.1 extract-msg-0.28.1 pdfminer.six-20201018 pip-21.0.1 pycparser-2.20 setuptools-53.0.0 six-1.15.0 tzlocal-2.1 xlrd-2.0.1

While running it with 21.0.1 and today's 21.1.dev0 gives this:

Requirement already satisfied: argcomplete in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (1.10.0)
Collecting argcomplete
  Using cached argcomplete-1.12.2-py2.py3-none-any.whl (38 kB)
Requirement already satisfied: beautifulsoup4 in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (4.8.0)
Collecting beautifulsoup4
  Using cached beautifulsoup4-4.9.3-py3-none-any.whl (115 kB)
Requirement already satisfied: chardet in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (3.0.4)
Collecting chardet
  Using cached chardet-4.0.0-py2.py3-none-any.whl (178 kB)
Requirement already satisfied: docx2txt in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (0.8)
Requirement already satisfied: EbookLib in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (0.17.1)
Requirement already satisfied: extract-msg in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (0.23.1)
Collecting extract-msg
  Using cached extract_msg-0.28.1-py2.py3-none-any.whl (67 kB)
Requirement already satisfied: IMAPClient in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (2.1.0)
Collecting IMAPClient
  Using cached IMAPClient-2.2.0-py2.py3-none-any.whl (179 kB)
Requirement already satisfied: lxml in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (4.6.2)
Requirement already satisfied: olefile in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (0.46)
Requirement already satisfied: pdfminer.six in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (20181108)
Collecting pdfminer.six
  Using cached pdfminer.six-20201018-py3-none-any.whl (5.6 MB)
Requirement already satisfied: Pillow in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (8.1.0)
Requirement already satisfied: pip in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (21.0.1)
Requirement already satisfied: pycryptodome in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (3.9.9)
Requirement already satisfied: PyPDF2 in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (1.26.0)
Requirement already satisfied: python-pptx in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (0.6.18)
Requirement already satisfied: pytz in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (2021.1)
Requirement already satisfied: setuptools in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (49.2.1)
Collecting setuptools
  Using cached setuptools-53.0.0-py3-none-any.whl (784 kB)
Requirement already satisfied: six in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (1.12.0)
Collecting six
  Using cached six-1.15.0-py2.py3-none-any.whl (10 kB)
Requirement already satisfied: sortedcontainers in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (2.3.0)
Requirement already satisfied: soupsieve in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (2.1)
Requirement already satisfied: SpeechRecognition in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (3.8.1)
Requirement already satisfied: textract in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (1.6.3)
Requirement already satisfied: tzlocal in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (1.5.1)
Collecting tzlocal
  Using cached tzlocal-2.1-py2.py3-none-any.whl (16 kB)
Requirement already satisfied: xlrd in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (1.2.0)
Collecting xlrd
  Using cached xlrd-2.0.1-py2.py3-none-any.whl (96 kB)
Requirement already satisfied: XlsxWriter in /home/bers/.pyenv/versions/3.8.7/envs/temp/lib/python3.8/site-packages (1.3.7)
Collecting compressed-rtf>=1.0.6
  Using cached compressed_rtf-1.0.6.tar.gz (5.8 kB)
Collecting ebcdic>=1.1.1
  Using cached ebcdic-1.1.1-py2.py3-none-any.whl (128 kB)
Collecting cryptography
  Using cached cryptography-3.4.3-cp36-abi3-manylinux2014_x86_64.whl (3.2 MB)
Collecting extract-msg
  Using cached extract_msg-0.23.1-py2.py3-none-any.whl (45 kB)
INFO: pip is looking at multiple versions of chardet to determine which version is compatible with other requirements. This could take a while.
Collecting chardet
  Using cached chardet-3.0.4-py2.py3-none-any.whl (133 kB)
INFO: pip is looking at multiple versions of beautifulsoup4 to determine which version is compatible with other requirements. This could take a while.
Collecting beautifulsoup4
  Using cached beautifulsoup4-4.8.0-py3-none-any.whl (97 kB)
INFO: pip is looking at multiple versions of argcomplete to determine which version is compatible with other requirements. This could take a while.
Collecting argcomplete
  Using cached argcomplete-1.10.0-py2.py3-none-any.whl (31 kB)
INFO: pip is looking at multiple versions of chardet to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of textract to determine which version is compatible with other requirements. This could take a while.
Collecting textract
  Using cached textract-1.6.3-py3-none-any.whl (21 kB)
  Using cached textract-1.6.1.tar.gz (16 kB)
  Using cached textract-1.6.0.tar.gz (15 kB)
  Using cached textract-1.5.0.tar.gz (15 kB)
  Using cached textract-1.4.0.tar.gz (14 kB)
  Using cached textract-1.3.0.tar.gz (14 kB)
  Using cached textract-1.2.0.tar.gz (14 kB)
  Using cached textract-1.1.0.tar.gz (13 kB)
  Using cached textract-1.0.0.tar.gz (10 kB)
  Using cached textract-0.5.1.tar.gz (8.2 kB)
    ERROR: Command errored out with exit status 1:
     command: /home/bers/.pyenv/versions/3.8.7/envs/temp/bin/python3.8 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-lsnp_upz/textract_8fcc0673a09547c3918058b34b3ed75d/setup.py'"'"'; __file__='"'"'/tmp/pip-install-lsnp_upz/textract_8fcc0673a09547c3918058b34b3ed75d/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-0y_ixjn6
         cwd: /tmp/pip-install-lsnp_upz/textract_8fcc0673a09547c3918058b34b3ed75d/
    Complete output (10 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-lsnp_upz/textract_8fcc0673a09547c3918058b34b3ed75d/setup.py", line 5, in <module>
        import textract
      File "/tmp/pip-install-lsnp_upz/textract_8fcc0673a09547c3918058b34b3ed75d/textract/__init__.py", line 3, in <module>
        from .parsers import process
      File "/tmp/pip-install-lsnp_upz/textract_8fcc0673a09547c3918058b34b3ed75d/textract/parsers/__init__.py", line 37
        except ImportError, e:
                          ^
    SyntaxError: invalid syntax
    ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/bd/84/a3919f562f7aaec51a2b115ed11e8bf442dee1a9b421633ceee5fe1963ca/textract-0.5.1.tar.gz#sha256=75a4ec51ad10642f73e13137f90efa2fd944f84a7c32285b53448ea2cf7d04d1 (from https://pypi.org/simple/textract/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
  Using cached textract-0.5.0.tar.gz (8.0 kB)
    ERROR: Command errored out with exit status 1:
     command: /home/bers/.pyenv/versions/3.8.7/envs/temp/bin/python3.8 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-lsnp_upz/textract_bf4be8b1cbd245fca075420cce8cbe0a/setup.py'"'"'; __file__='"'"'/tmp/pip-install-lsnp_upz/textract_bf4be8b1cbd245fca075420cce8cbe0a/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-yvopnmve
         cwd: /tmp/pip-install-lsnp_upz/textract_bf4be8b1cbd245fca075420cce8cbe0a/
    Complete output (10 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-lsnp_upz/textract_bf4be8b1cbd245fca075420cce8cbe0a/setup.py", line 5, in <module>
        import textract
      File "/tmp/pip-install-lsnp_upz/textract_bf4be8b1cbd245fca075420cce8cbe0a/textract/__init__.py", line 3, in <module>
        from .parsers import process
      File "/tmp/pip-install-lsnp_upz/textract_bf4be8b1cbd245fca075420cce8cbe0a/textract/parsers/__init__.py", line 37
        except ImportError, e:
                          ^
    SyntaxError: invalid syntax
    ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/8c/2f/e1e716a320afc3cf9ec1d3df47fdf8e43ed465faa0e09b9c87ccb9917a3d/textract-0.5.0.tar.gz#sha256=d155c29d10968b2e4fdcc23cae016b25b7f8b41b43d2dcdc1d49c108b92aaceb (from https://pypi.org/simple/textract/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
  Using cached textract-0.4.0.tar.gz (6.2 kB)
    ERROR: Command errored out with exit status 1:
     command: /home/bers/.pyenv/versions/3.8.7/envs/temp/bin/python3.8 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-lsnp_upz/textract_71dbd6c7f1cd41bcadfa63ef6828283e/setup.py'"'"'; __file__='"'"'/tmp/pip-install-lsnp_upz/textract_71dbd6c7f1cd41bcadfa63ef6828283e/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-fxp3v6wl
         cwd: /tmp/pip-install-lsnp_upz/textract_71dbd6c7f1cd41bcadfa63ef6828283e/
    Complete output (10 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-lsnp_upz/textract_71dbd6c7f1cd41bcadfa63ef6828283e/setup.py", line 5, in <module>
        import textract
      File "/tmp/pip-install-lsnp_upz/textract_71dbd6c7f1cd41bcadfa63ef6828283e/textract/__init__.py", line 3, in <module>
        from .parsers import process
      File "/tmp/pip-install-lsnp_upz/textract_71dbd6c7f1cd41bcadfa63ef6828283e/textract/parsers/__init__.py", line 23
        except ImportError, e:
                          ^
    SyntaxError: invalid syntax
    ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/d5/db/a76cd92425a9d430916460ac6f5c3178f93e5057e54874bcddec6b19ee4d/textract-0.4.0.tar.gz#sha256=d47e16dc1e1e4c6aa8b1817be5e5ed889aa891fb8844183f0f0834ab2190faf4 (from https://pypi.org/simple/textract/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
  Using cached textract-0.3.0.tar.gz (5.3 kB)
    ERROR: Command errored out with exit status 1:
     command: /home/bers/.pyenv/versions/3.8.7/envs/temp/bin/python3.8 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-lsnp_upz/textract_256901a546d7417d9a3d2a2063ec8243/setup.py'"'"'; __file__='"'"'/tmp/pip-install-lsnp_upz/textract_256901a546d7417d9a3d2a2063ec8243/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-mav23ufo
         cwd: /tmp/pip-install-lsnp_upz/textract_256901a546d7417d9a3d2a2063ec8243/
    Complete output (10 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-lsnp_upz/textract_256901a546d7417d9a3d2a2063ec8243/setup.py", line 5, in <module>
        import textract
      File "/tmp/pip-install-lsnp_upz/textract_256901a546d7417d9a3d2a2063ec8243/textract/__init__.py", line 3, in <module>
        from .parsers import process
      File "/tmp/pip-install-lsnp_upz/textract_256901a546d7417d9a3d2a2063ec8243/textract/parsers/__init__.py", line 23
        except ImportError, e:
                          ^
    SyntaxError: invalid syntax
    ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/4c/31/32b3bd8c0c42e927d20adbdaf5efcbe33783fa5bbf6120116eaf3f0ba5e9/textract-0.3.0.tar.gz#sha256=18bcdcc9edc187349a324e99626f07893dc2cf48493b92698afbb90e7edfea0f (from https://pypi.org/simple/textract/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
  Using cached textract-0.2.0.tar.gz (4.9 kB)
    ERROR: Command errored out with exit status 1:
     command: /home/bers/.pyenv/versions/3.8.7/envs/temp/bin/python3.8 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-lsnp_upz/textract_df3ae24382944409a23b98413b96be51/setup.py'"'"'; __file__='"'"'/tmp/pip-install-lsnp_upz/textract_df3ae24382944409a23b98413b96be51/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-bfwj90ze
         cwd: /tmp/pip-install-lsnp_upz/textract_df3ae24382944409a23b98413b96be51/
    Complete output (10 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-lsnp_upz/textract_df3ae24382944409a23b98413b96be51/setup.py", line 5, in <module>
        import textract
      File "/tmp/pip-install-lsnp_upz/textract_df3ae24382944409a23b98413b96be51/textract/__init__.py", line 3, in <module>
        from .parsers import process
      File "/tmp/pip-install-lsnp_upz/textract_df3ae24382944409a23b98413b96be51/textract/parsers/__init__.py", line 23
        except ImportError, e:
                          ^
    SyntaxError: invalid syntax
    ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/0b/1e/65e7d4bf6d9d9edad30fbb3e530fd137f43781a0618e0afde8559583547c/textract-0.2.0.tar.gz#sha256=c59da8e65100de20295ac623015f33c60f1d6dcfb4964058b344f7788b96d63a (from https://pypi.org/simple/textract/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
  Using cached textract-0.1.0.tar.gz (4.2 kB)
    ERROR: Command errored out with exit status 1:
     command: /home/bers/.pyenv/versions/3.8.7/envs/temp/bin/python3.8 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-lsnp_upz/textract_3050e9b390754aa281abec241d5a419f/setup.py'"'"'; __file__='"'"'/tmp/pip-install-lsnp_upz/textract_3050e9b390754aa281abec241d5a419f/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-y27443qe
         cwd: /tmp/pip-install-lsnp_upz/textract_3050e9b390754aa281abec241d5a419f/
    Complete output (10 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-lsnp_upz/textract_3050e9b390754aa281abec241d5a419f/setup.py", line 5, in <module>
        import textract
      File "/tmp/pip-install-lsnp_upz/textract_3050e9b390754aa281abec241d5a419f/textract/__init__.py", line 3, in <module>
        from .parsers import process
      File "/tmp/pip-install-lsnp_upz/textract_3050e9b390754aa281abec241d5a419f/textract/parsers/__init__.py", line 23
        except ImportError, e:
                          ^
    SyntaxError: invalid syntax
    ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/e6/1e/fe23e5771b7ac47671c23e70652c98dd7dce1cb1249d7fa2962ebadfadf9/textract-0.1.0.tar.gz#sha256=bee3b4c852b0641761681198558057cee2a8f779e6cd3507b62d20f181d4417f (from https://pypi.org/simple/textract/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. If you want to abort this run, you can press Ctrl + C to do so. To improve how pip performs, tell us what happened here: https://pip.pypa.io/surveys/backtracking
INFO: pip is looking at multiple versions of beautifulsoup4 to determine which version is compatible with other requirements. This could take a while.
Collecting argcomplete
  Using cached argcomplete-1.8.2-py2.py3-none-any.whl (34 kB)
Collecting chardet
  Using cached chardet-2.3.0-py2.py3-none-any.whl (180 kB)
Collecting python-pptx
  Using cached python-pptx-0.6.5.tar.gz (7.1 MB)
Collecting docx2txt
  Using cached docx2txt-0.6.tar.gz (2.1 kB)
Collecting beautifulsoup4
  Using cached beautifulsoup4-4.5.3-py3-none-any.whl (85 kB)
Collecting xlrd
  Using cached xlrd-1.0.0-py3-none-any.whl (143 kB)
Collecting EbookLib
  Using cached EbookLib-0.15.tar.gz (29 kB)
Collecting SpeechRecognition
  Using cached SpeechRecognition-3.6.3-py2.py3-none-any.whl (31.8 MB)
Collecting six
  Using cached six-1.10.0-py2.py3-none-any.whl (10 kB)
Collecting pocketsphinx==0.1.3
  Using cached pocketsphinx-0.1.3.zip (29.3 MB)
Collecting cffi>=1.12
  Using cached cffi-1.14.4-cp38-cp38-manylinux1_x86_64.whl (411 kB)
Collecting pycparser
  Using cached pycparser-2.20-py2.py3-none-any.whl (112 kB)
Using legacy 'setup.py install' for compressed-rtf, since package 'wheel' is not installed.
Using legacy 'setup.py install' for textract, since package 'wheel' is not installed.
Using legacy 'setup.py install' for docx2txt, since package 'wheel' is not installed.
Using legacy 'setup.py install' for EbookLib, since package 'wheel' is not installed.
Using legacy 'setup.py install' for python-pptx, since package 'wheel' is not installed.
Using legacy 'setup.py install' for pocketsphinx, since package 'wheel' is not installed.
Installing collected packages: pycparser, six, cffi, xlrd, tzlocal, SpeechRecognition, python-pptx, pocketsphinx, EbookLib, ebcdic, docx2txt, cryptography, compressed-rtf, chardet, beautifulsoup4, argcomplete, textract, setuptools, pdfminer.six, extract-msg
  Attempting uninstall: six
    Found existing installation: six 1.12.0
    Uninstalling six-1.12.0:
      Successfully uninstalled six-1.12.0
  Attempting uninstall: xlrd
    Found existing installation: xlrd 1.2.0
    Uninstalling xlrd-1.2.0:
      Successfully uninstalled xlrd-1.2.0
  Attempting uninstall: tzlocal
    Found existing installation: tzlocal 1.5.1
    Uninstalling tzlocal-1.5.1:
      Successfully uninstalled tzlocal-1.5.1
  Attempting uninstall: SpeechRecognition
    Found existing installation: SpeechRecognition 3.8.1
    Uninstalling SpeechRecognition-3.8.1:
      Successfully uninstalled SpeechRecognition-3.8.1
  Attempting uninstall: python-pptx
    Found existing installation: python-pptx 0.6.18
    Uninstalling python-pptx-0.6.18:
      Successfully uninstalled python-pptx-0.6.18
    Running setup.py install for python-pptx ... done
    Running setup.py install for pocketsphinx ... error
    ERROR: Command errored out with exit status 1:
     command: /home/bers/.pyenv/versions/3.8.7/envs/temp/bin/python3.8 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-lsnp_upz/pocketsphinx_f398f2f86e8f451ab8d80299c58150b3/setup.py'"'"'; __file__='"'"'/tmp/pip-install-lsnp_upz/pocketsphinx_f398f2f86e8f451ab8d80299c58150b3/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-duocl3he/install-record.txt --single-version-externally-managed --compile --install-headers /home/bers/.pyenv/versions/3.8.7/envs/temp/include/site/python3.8/pocketsphinx
         cwd: /tmp/pip-install-lsnp_upz/pocketsphinx_f398f2f86e8f451ab8d80299c58150b3/
    Complete output (29 lines):
    running install
    running build_ext
    building 'sphinxbase._ad' extension
    swigging swig/sphinxbase/ad.i to swig/sphinxbase/ad_wrap.c
    swig -python -modern -Ideps/sphinxbase/include -Ideps/sphinxbase/include/sphinxbase -Ideps/sphinxbase/include/android -Ideps/sphinxbase/swig -outdir sphinxbase -o swig/sphinxbase/ad_wrap.c swig/sphinxbase/ad.i
    creating build
    creating build/temp.linux-x86_64-3.8
    creating build/temp.linux-x86_64-3.8/swig
    creating build/temp.linux-x86_64-3.8/swig/sphinxbase
    creating build/temp.linux-x86_64-3.8/deps
    creating build/temp.linux-x86_64-3.8/deps/sphinxbase
    creating build/temp.linux-x86_64-3.8/deps/sphinxbase/src
    creating build/temp.linux-x86_64-3.8/deps/sphinxbase/src/libsphinxad
    gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DSPHINXBASE_EXPORTS -DPOCKETSPHINX_EXPORTS -DSPHINX_DLL -DHAVE_CONFIG_H -Ideps/sphinxbase/include -Ideps/sphinxbase/include/sphinxbase -Ideps/sphinxbase/include/android -I/home/bers/.pyenv/versions/3.8.7/envs/temp/include -I/home/bers/.pyenv/versions/3.8.7/include/python3.8 -c swig/sphinxbase/ad_wrap.c -o build/temp.linux-x86_64-3.8/swig/sphinxbase/ad_wrap.o -Wno-unused-label -Wno-strict-prototypes -Wno-parentheses -Wno-unused-but-set-variable -Wno-unused-variable -Wno-unused-result -Wno-sign-compare -Wno-misleading-indentation
    swig/sphinxbase/ad_wrap.c: In function ‘_wrap_Ad_readinto’:
    swig/sphinxbase/ad_wrap.c:3692:5: warning: ‘PyObject_AsWriteBuffer’ is deprecated [-Wdeprecated-declarations]
         res2 = PyObject_AsWriteBuffer(obj1, &buf2, &size2);
         ^~~~
    In file included from /home/bers/.pyenv/versions/3.8.7/include/python3.8/Python.h:147:0,
                     from swig/sphinxbase/ad_wrap.c:149:
    /home/bers/.pyenv/versions/3.8.7/include/python3.8/abstract.h:348:17: note: declared here
     PyAPI_FUNC(int) PyObject_AsWriteBuffer(PyObject *obj,
                     ^~~~~~~~~~~~~~~~~~~~~~
    gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DSPHINXBASE_EXPORTS -DPOCKETSPHINX_EXPORTS -DSPHINX_DLL -DHAVE_CONFIG_H -Ideps/sphinxbase/include -Ideps/sphinxbase/include/sphinxbase -Ideps/sphinxbase/include/android -I/home/bers/.pyenv/versions/3.8.7/envs/temp/include -I/home/bers/.pyenv/versions/3.8.7/include/python3.8 -c deps/sphinxbase/src/libsphinxad/ad_pulse.c -o build/temp.linux-x86_64-3.8/deps/sphinxbase/src/libsphinxad/ad_pulse.o -Wno-unused-label -Wno-strict-prototypes -Wno-parentheses -Wno-unused-but-set-variable -Wno-unused-variable -Wno-unused-result -Wno-sign-compare -Wno-misleading-indentation
    deps/sphinxbase/src/libsphinxad/ad_pulse.c:44:10: fatal error: pulse/pulseaudio.h: No such file or directory
     #include <pulse/pulseaudio.h>
              ^~~~~~~~~~~~~~~~~~~~
    compilation terminated.
    error: command 'gcc' failed with exit status 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /home/bers/.pyenv/versions/3.8.7/envs/temp/bin/python3.8 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-lsnp_upz/pocketsphinx_f398f2f86e8f451ab8d80299c58150b3/setup.py'"'"'; __file__='"'"'/tmp/pip-install-lsnp_upz/pocketsphinx_f398f2f86e8f451ab8d80299c58150b3/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-duocl3he/install-record.txt --single-version-externally-managed --compile --install-headers /home/bers/.pyenv/versions/3.8.7/envs/temp/include/site/python3.8/pocketsphinx Check the logs for full command output.

Given that the environment is in a consistent state with all dependencies fulfilled, I'd say a better behvaior for this upgrade command (than to downgrade to textract==0.5.1, which is the first error) would be to do nothing. (I do see how that may be difficult given that pip does not take installed packages into account.)
@bersbersbers
Copy link
Author

This may be a good test case for the related issue #4551.

@uranusjr
Copy link
Member

uranusjr commented Feb 9, 2021 •
edited
Loading

That’s basically what the new resolver is good for, it catches and prevents the inconsistent state from ever happening. You would end up with an inconsistent state after upgrading, so it refuses to upgrade and keep your environment in a consistent state. Everything seems to work as designed, from what I can tell.

@bersbersbers
Copy link
Author

@uranusjr I am not sure I agree. I will verify, but my understanding is that

  1. pip tries to actually downgrade textract (which only fails because the old version does not compile) without need to do so,
  2. pip uninstalls six and other packages, so I don't think the environment is consistent,
  3. pip actually errors with return code 1

This may be two issues coming together: unwanted downgrade plus installation failure. But I would not say everything is working as planned.

@bersbersbers
Copy link
Author

  1. Confirming that the inconsistent state is happening after the erroring pip install:
$ pip check
textract 1.6.3 has requirement python-pptx==0.6.18, but you have python-pptx 0.6.5.
textract 1.6.3 has requirement six==1.12.0, but you have six 1.10.0.
textract 1.6.3 has requirement SpeechRecognition==3.8.1, but you have speechrecognition 3.6.3.
textract 1.6.3 has requirement xlrd==1.2.0, but you have xlrd 1.0.0.
extract-msg 0.23.1 has requirement tzlocal==1.5.1, but you have tzlocal 2.1.
  1. A similar problem can be reproduced starting from a clean environment using
pip install --upgrade extract-msg textract

and with

pip install --upgrade textract extract-msg
  1. I think what may be happening here is that pip first considers extract-msg==0.28.1 and then finds no suitable version of textract to work with this. And this may be a general problem for pip: if package A==1.0 depends on B, if package A==2.0 depends on B==1.0, and if B has 1.0 and 2.0 available, two (upgrade) solutions are possible: you can install A 1.0 and B 2.0, or A 2.0 and B 1.0. Which one to prefer?

As a human, I see from pipdeptree that A (textract) depends on B (extract-msg) - so I would try the most current version of A (textract) first. It is likely that the user is interested in using that package (as it is not dependent on any other package), and it is possible that B has been installed only to serve As dependencies. Maybe pip could start with the most recent version of the top packages first?

@pradyunsg pradyunsg added the S: needs triage Issues/PRs that need to be triaged label Mar 6, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
S: needs triage Issues/PRs that need to be triaged
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@uranusjr @pradyunsg @bersbersbers