virtualenv + python-future = broken virtualenv

[asottile]

Simplest reproduction:

$ virtualenv venv
$ ./venv/bin/pip install future virtualenv
Collecting future
  Using cached future-0.14.3.tar.gz
Collecting virtualenv
  Using cached virtualenv-12.1.1-py2.py3-none-any.whl
Installing collected packages: future, virtualenv
  Running setup.py install for future
Successfully installed future-0.14.3 virtualenv-12.1.1
$ ./venv/bin/virtualenv -ppython3.4 venv34
Running virtualenv with interpreter /usr/bin/python3.4
Traceback (most recent call last):
  File "/home/anthony/venv/local/lib/python2.7/site-packages/virtualenv.py", line 8, in <module>
    import base64
  File "/usr/lib/python3.4/base64.py", line 9, in <module>
    import re
  File "/usr/lib/python3.4/re.py", line 336, in <module>
    import copyreg
  File "/home/anthony/venv/lib/python2.7/site-packages/copyreg/__init__.py", line 7, in <module>
    raise ImportError('This package should not be accessible on Python 3. '
ImportError: This package should not be accessible on Python 3. Either you are trying to run from the python-future src folder or your installation of python-future is corrupted.

[QuLogic]

This is a bug in virtualenv since it's muddling up the 2.7 and 3.4 module paths. There are several similar issues reported upstream:
pypa/virtualenv#745
pypa/virtualenv#671
pypa/virtualenv#625
pypa/virtualenv#697

Better to use pyvenv that comes with Python 3.3+ for now.

[asottile]

pyvenv isn't really an option for a codebase targetting 2+3 (especially considering the many edgecases that pyvenv is not-quite-correct for (and cannot be fixed due to being stranded in the stdlib)). Would there ever be potential plans to split the useful bits (backports, moves, etc.) from the shadowing of py3 module names?

[edschofield]

Thanks for mentioning this, Anthony. I think having a Python 2.7 site-packages folder accessible to a Python 3.4 interpreter will lead to a lot of breakage in general. So I agree with Elliott that this is a virtualenv bug. (Perhaps a resurfacing of bug #673.)

I must admit that I haven't used virtualenv for 2 years (since I discovered conda). I'll try to find some time to tinker with it further to see what's going on. But, reading pypa/virtualenv#697, I rather suspect it's a bird's nest of brittle hacks ...

[asottile]

Could you address the question? Probably fine to close as wontfix after answering I would guess.

Would there ever be potential plans to split the useful bits (backports, moves, etc.) from the shadowing of py3 module names?

[bukzor]

@QuLogic @edschofield This is actually an effect of the pythonpath mining that we do at our company. We've since factored it out.

[bukzor]

munging*

[asottile]

@bukzor it's actually not, see my reproduction

[sontek]

I just ran into this issue, looks like it was introducing in virtualenv 12.04, I've started pinning to virtualenv 12.02 without issues

[asottile]

fwiw, this is the commit that enabled it to work from virtualenv's side: pypa/virtualenv@73d46a8

It was however reverted later due to causing issues (?) on debian.

I've created my own wrapper script for virtualenv that also solves this problem: https://github.com/asottile/virtualenv-hax

[valerymelou]

Had the same issue. Downgrading to virtualenv 12.0.2 solved the problem.

[zruu]

Experiencing the same issue. My version is 13.1.2. Had to downgrade to 12.0.2 as valerymelou mentioned.

[adamteale]

downgrading to 12.0.2 worked for me too

[valerymelou]

Are they going to fix the bug or what?

[fernandojunior]

For Ubuntu 14.04, this combination worked for me:

• Install pip for python 3.4 (https://pip.pypa.io/en/stable/installing/):

$ wget https://bootstrap.pypa.io/get-pip.py -O - | sudo python3.4

• Replace the old virtualenv (python 2.7 to python 3.4) :

$ sudo pip3.4 install virtualenv

• Just to check:

$ head -n 1 /usr/local/bin/virtualenv
#!/usr/bin/python3

• Now, the following commands will run:

$ virtualenv venv34
Using base prefix '/usr'
New python executable in venv34/bin/python3
Also creating executable in venv34/bin/python
Installing setuptools, pip, wheel...done.

$ ./venv34/bin/pip install virtualenv
Collecting virtualenv
  Using cached virtualenv-13.1.2-py2.py3-none-any.whl
Installing collected packages: virtualenv
Successfully installed virtualenv-13.1.2

$ ./venv34/bin/virtualenv -p python venv
Running virtualenv with interpreter /usr/bin/python
New python executable in venv/bin/python
Installing setuptools, pip, wheel...done.

$ /usr/bin/python --version
Python 2.7.6

$ ./venv/bin/pip install future virtualenv
Collecting future
Collecting virtualenv
  Using cached virtualenv-13.1.2-py2.py3-none-any.whl
Installing collected packages: future, virtualenv
Successfully installed future-0.15.2 virtualenv-13.1.2

$ ./venv34/bin/pip install future
Collecting future
Installing collected packages: future
Successfully installed future-0.15.2

• To avoid compilation errors from Python extensions written in C or C++ (ref):

$ sudo apt-get update
$ sudo apt-get install -y build-essential
$ sudo apt-get install -y python3.4-dev
$ sudo apt-get install -y python3-software-properties

Now, tox runs beautifully :)

[rtorresve]

Hi guys I saw your opinions, and fernandojunior recomendations work for me however rigth now i update my virtualenv to 14.0.5 and the trouble are fixed in this version, currently i am working in my host with linuxmint 17.3 x64

[NataliaDiaz]

What to do when getting the same error but using conda environments?

[jmadler]

Closing as this conversation has stalled and there appears to be no remaining action for python-future. Please reopen if I may be mistaken :)

[asottile]

yep yep! This got worked around in the virtualenv project