Incorrect (broken) virtualenv layout with pypy3.8's new layout

[mgorny]

Issue

PyPy3.8 (currently 7.3.6rc1) supports a new install layout that resembles CPython more. That is, sys.prefix no longer needs being isolated, and site-packages are found in $prefix/lib/pypy3.8/site-packages. However, virtualenv tries to symlink everything from /usr/lib including the pypy3.8 directory. As a result, the user can't write to the site-packages directory in the venv.

I haven't tried running it as root but I can imagine it doing major mess if it virtualenv doesn't take any precautions from writing into system directories.

Environment

Provide at least:

• OS: Gentoo Linux
• pip list of the host python where virtualenv is installed: pip-list.txt

Output of the virtual environment creation

Make sure to run the creation with -vvv --with-traceback:

Full output: output.txt

tail:

1048 create virtualenv import hook file /tmp/z/lib/pypy3.8/site-packages/_virtualenv.pth [DEBUG api:95]
Traceback (most recent call last):
  File "/usr/lib/pypy3.8/runpy.py", line 198, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/pypy3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/usr/lib/pypy3.8/site-packages/virtualenv/__main__.py", line 80, in <module>
    run_with_catch()  # pragma: no cov
  File "/usr/lib/pypy3.8/site-packages/virtualenv/__main__.py", line 65, in run_with_catch
    run(args, options, env)
  File "/usr/lib/pypy3.8/site-packages/virtualenv/__main__.py", line 18, in run
    session = cli_run(args, options, env)
  File "/usr/lib/pypy3.8/site-packages/virtualenv/run/__init__.py", line 32, in cli_run
    of_session.run()
  File "/usr/lib/pypy3.8/site-packages/virtualenv/run/session.py", line 46, in run
    self._create()
  File "/usr/lib/pypy3.8/site-packages/virtualenv/run/session.py", line 53, in _create
    self.creator.run()
  File "/usr/lib/pypy3.8/site-packages/virtualenv/create/creator.py", line 171, in run
    self.create()
  File "/usr/lib/pypy3.8/site-packages/virtualenv/create/via_global_ref/builtin/via_global_self_do.py", line 101, in create
    super(ViaGlobalRefVirtualenvBuiltin, self).create()
  File "/usr/lib/pypy3.8/site-packages/virtualenv/create/via_global_ref/api.py", line 89, in create
    self.install_patch()
  File "/usr/lib/pypy3.8/site-packages/virtualenv/create/via_global_ref/api.py", line 96, in install_patch
    pth.write_text("import _virtualenv")
  File "/usr/lib/pypy3.8/pathlib.py", line 1255, in write_text
    with self.open(mode='w', encoding=encoding, errors=errors) as f:
  File "/usr/lib/pypy3.8/pathlib.py", line 1223, in open
    opener=self._opener)
  File "/usr/lib/pypy3.8/pathlib.py", line 1078, in _opener
    return self._accessor.open(self, flags, mode)
PermissionError: [Errno 13] Permission denied: PosixPath('/tmp/z/lib/pypy3.8/site-packages/_virtualenv.pth')

[mgorny]

Turns out there's a similar (yet somewhat opposite) problem in pypy's built-in venv module: https://foss.heptapod.net/pypy/pypy/-/issues/3553. Maybe it's better to wait for a fix there, in case something relevant changes.

[olliemath]

Can confirm major mess ensues when the system site packages dir is not owned by root 😭

[mattip]

Is this related?
I found that trying to change a seeded module does not seem to work. I opened an issue on PyPy where creating a virtualenv in /tmp/venv_pypy38, and then trying to update setuptools with pypy --upgrade 'setuptools<49.2.0' does not succeed (it cannot uninstall the seeded version):

Collecting setuptools<49.2.0
  Using cached setuptools-49.1.3-py3-none-any.whl (789 kB)
Installing collected packages: setuptools
  Attempting uninstall: setuptools
    Found existing installation: setuptools 58.1.0
    Not uninstalling setuptools at /home/matti/oss/pypy3.8-HEAD/lib/pypy3.8/site-packages, outside environment /tmp/venv_pypy38
    Can't uninstall 'setuptools'. No files were found to uninstall.
Successfully installed setuptools-58.1.0

[gaborbernat]

Released via 20.9.0.