Discrepancies in sys.path in Kernel vs Terminal

[DonJayamanne]

Summary:

• For non-global environments we don't need to append the user site packages (ie. the trailing paths in sys.path is not necessary).

For a conda environment:

# Kernel
python -c "import sys;print(sys.path)"
['/home/don/samples/pySamples/crap/misc',
 '/home/don/miniconda3/envs/env1/lib/python38.zip',
 '/home/don/miniconda3/envs/env1/lib/python3.8',
 '/home/don/miniconda3/envs/env1/lib/python3.8/lib-dynload',
 '',
 '/home/don/miniconda3/envs/env1/lib/python3.8/site-packages',
 '/home/don/miniconda3/envs/env1/lib/python3.8/site-packages/IPython/extensions',
 '/home/don/.ipython',
 '/home/don/.local/lib/python3.8/site-packages']
# Terminal
python -c "import sys;print(sys.path)"
['/home/don/samples/pySamples/crap/misc',
 '/home/don/miniconda3/envs/env1/lib/python38.zip',
 '/home/don/miniconda3/envs/env1/lib/python3.8',
 '/home/don/miniconda3/envs/env1/lib/python3.8/lib-dynload',
 '',
 '/home/don/.local/lib/python3.8/site-packages',
 '/home/don/miniconda3/envs/env1/lib/python3.8/site-packages',
 '/home/don/miniconda3/envs/env1/lib/python3.8/site-packages/IPython/extensions',
 '/home/don/.ipython']

For Virtual Env on windows

# Our kernel
['c:\\Development\\samples\\pySamples\\sample1', 
'C:\\Python39\\python39.zip', 
'C:\\Python39\\DLLs', 
'C:\\Python39\\lib', 
'C:\\Python39', 
'c:\\Development\\samples\\pySamples\\sample1\\.venv', 
'', 
'c:\\Development\\samples\\pySamples\\sample1\\.venv\\lib\\site-packages', 'c:\\Development\\samples\\pySamples\\sample1\\.venv\\lib\\site-packages\\win32', 'c:\\Development\\samples\\pySamples\\sample1\\.venv\\lib\\site-packages\\win32\\lib', 'c:\\Development\\samples\\pySamples\\sample1\\.venv\\lib\\site-packages\\Pythonwin', 'c:\\Development\\samples\\pySamples\\sample1\\.venv\\lib\\site-packages\\IPython\\extensions', 
'C:\\Users\\donja\\.ipython', 
'C:\\Users\\donja\\AppData\\Roaming\\Python\\Python39\\site-packages', 'C:\\Users\\donja\\AppData\\Roaming\\Python\\Python39\\site-packages\\win32', 'C:\\Users\\donja\\AppData\\Roaming\\Python\\Python39\\site-packages\\win32\\lib', 'C:\\Users\\donja\\AppData\\Roaming\\Python\\Python39\\site-packages\\Pythonwin']

# Jupyter Notebook
['C:\\Development\\samples\\pySamples\\sample1',
 'C:\\Python39\\python39.zip',
 'C:\\Python39\\DLLs',
 'C:\\Python39\\lib',
 'C:\\Python39',
 'c:\\Development\\samples\\pySamples\\sample1\\.venv',
 '',
 'c:\\Development\\samples\\pySamples\\sample1\\.venv\\lib\\site-packages',
 'c:\\Development\\samples\\pySamples\\sample1\\.venv\\lib\\site-packages\\win32',
 'c:\\Development\\samples\\pySamples\\sample1\\.venv\\lib\\site-packages\\win32\\lib',
 'c:\\Development\\samples\\pySamples\\sample1\\.venv\\lib\\site-packages\\Pythonwin',
 'c:\\Development\\samples\\pySamples\\sample1\\.venv\\lib\\site-packages\\IPython\\extensions',
 'C:\\Users\\donja\\.ipython']

For virtual env:

# Our kernel
['/home/don/samples/pySamples/crap/misc',
 '/home/don/.pyenv/versions/3.8.12/lib/python38.zip',
 '/home/don/.pyenv/versions/3.8.12/lib/python3.8',
 '/home/don/.pyenv/versions/3.8.12/lib/python3.8/lib-dynload',
 '',
 '/home/don/samples/pySamples/crap/.venv/lib/python3.8/site-packages',
 '/home/don/samples/pySamples/crap/.venv/lib/python3.8/site-packages/IPython/extensions',
 '/home/don/.ipython',
 '/home/don/.local/lib/python3.8/site-packages']

# Jupyter Notebook
['/home/don/samples/pySamples/crap/misc',
 '/home/don/.pyenv/versions/3.8.12/lib/python38.zip',
 '/home/don/.pyenv/versions/3.8.12/lib/python3.8',
 '/home/don/.pyenv/versions/3.8.12/lib/python3.8/lib-dynload',
 '',
 '/home/don/samples/pySamples/crap/.venv/lib/python3.8/site-packages',
 '/home/don/samples/pySamples/crap/.venv/lib/python3.8/site-packages/IPython/extensions',
 '/home/don/.ipython']

For global env

# Our Kernel
['c:\\Development\\samples\\pySamples\\sample1', 
'C:\\Python39\\python39.zip', 
'C:\\Python39\\DLLs', 
'C:\\Python39\\lib', 
'C:\\Python39', 
'', 
'C:\\Users\\donja\\AppData\\Roaming\\Python\\Python39\\site-packages', 'C:\\Users\\donja\\AppData\\Roaming\\Python\\Python39\\site-packages\\win32', 'C:\\Users\\donja\\AppData\\Roaming\\Python\\Python39\\site-packages\\win32\\lib', 'C:\\Users\\donja\\AppData\\Roaming\\Python\\Python39\\site-packages\\Pythonwin', 
'C:\\Python39\\lib\\site-packages', 
'C:\\Python39\\lib\\site-packages\\win32', 
'C:\\Python39\\lib\\site-packages\\win32\\lib', 
'C:\\Python39\\lib\\site-packages\\Pythonwin']

# Jupyter Notebook
['C:\\Development\\samples\\pySamples\\sample1\\crap',
 'C:\\Python39\\python39.zip',
 'C:\\Python39\\DLLs',
 'C:\\Python39\\lib',
 'C:\\Python39',
 '',
 'C:\\Users\\donja\\AppData\\Roaming\\Python\\Python39\\site-packages',
 'C:\\Users\\donja\\AppData\\Roaming\\Python\\Python39\\site-packages\\win32',
 'C:\\Users\\donja\\AppData\\Roaming\\Python\\Python39\\site-packages\\win32\\lib',
 'C:\\Users\\donja\\AppData\\Roaming\\Python\\Python39\\site-packages\\Pythonwin',
 'C:\\Python39\\lib\\site-packages',
 'C:\\Python39\\lib\\site-packages\\win32',
 'C:\\Python39\\lib\\site-packages\\win32\\lib',
 'C:\\Python39\\lib\\site-packages\\Pythonwin']

[DonJayamanne]

This could probably explain why we're seeing other global packages in virtual environments #9164

[DonJayamanne]

I.e. for non-global environment we don't need to add user site_packages into the sys.path at all.

[rchiodo]

So not entirely sure what the expected behavior is here, but I get different results from the terminal and from our stuff:

Conda env:

Ours:

['d:\\Source\\Testing\\Testing_3', 'c:\\Users\\aku91\\miniconda3\\envs\\condaEnvTest\\python39.zip', 'c:\\Users\\aku91\\miniconda3\\envs\\condaEnvTest\\DLLs', 'c:\\Users\\aku91\\miniconda3\\envs\\condaEnvTest\\lib', 'c:\\Users\\aku91\\miniconda3\\envs\\condaEnvTest', '', 'c:\\Users\\aku91\\miniconda3\\envs\\condaEnvTest\\lib\\site-packages', 'c:\\Users\\aku91\\miniconda3\\envs\\condaEnvTest\\lib\\site-packages\\win32', 'c:\\Users\\aku91\\miniconda3\\envs\\condaEnvTest\\lib\\site-packages\\win32\\lib', 'c:\\Users\\aku91\\miniconda3\\envs\\condaEnvTest\\lib\\site-packages\\Pythonwin', 'c:\\Users\\aku91\\miniconda3\\envs\\condaEnvTest\\lib\\site-packages\\IPython\\extensions', 'C:\\Users\\aku91\\.ipython']

Terminal/Jupyter

['', 'C:\\Users\\aku91\\miniconda3\\envs\\condaenvtest\\python39.zip', 'C:\\Users\\aku91\\miniconda3\\envs\\condaenvtest\\DLLs', 'C:\\Users\\aku91\\miniconda3\\envs\\condaenvtest\\lib', 'C:\\Users\\aku91\\miniconda3\\envs\\condaenvtest', 'C:\\Users\\aku91\\AppData\\Roaming\\Python\\Python39\\site-packages', 'C:\\Users\\aku91\\miniconda3\\envs\\condaenvtest\\lib\\site-packages', 'C:\\Users\\aku91\\miniconda3\\envs\\condaenvtest\\lib\\site-packages\\win32', 'C:\\Users\\aku91\\miniconda3\\envs\\condaenvtest\\lib\\site-packages\\win32\\lib', 'C:\\Users\\aku91\\miniconda3\\envs\\condaenvtest\\lib\\site-packages\\Pythonwin']

Where jupyter/terminal DO have the global site-packages in the list, but we DON'T.

I believe you changed our stuff to not have it on purpose?

[joyceerhl]

@DonJayamanne how should this be verified?

[rchiodo]

Not sure what the expected behavior here is. Reopening to look at next month.

[DonJayamanne]

I believe you changed our stuff to not have it on purpose?

Yes thats correct.

@rchiodo what you have here #9379 (comment) is what is expected.
Closing as verified.