Pytest discovery fails with ENOENT

[mattB1989]

Type: Bug

Behaviour

Expected vs. Actual

I expect the tests to get discovered, but instead I got a spawn error. There are other users who mentioned this before and managed to get around with restarting etc. but this does not seem to work. I tested in multiroot and simple folder, but I get the same issue.

Steps to reproduce:

This is the same as 15739

1. Delete/move ~/.vscode for fresh vscode user env.
2. Create simple python project with one pytest test and open vscode to folder with only the python extension and pytest auto discovery enabled.
3. Observe output in Python Test Log listed below.
4. Copy/paste command line from Python Test Log into linux terminal.
   Observe process runs successfully and discovers pytest test.

Diagnostic data

• Python version (& distribution if applicable, e.g. Anaconda): 3.11.6
• Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Conda
• Value of the python.languageServer setting: Default

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

2023-10-31 14:28:23.628 [info] > conda run -n common_env --no-capture-output python ~/.vscode-server/extensions/ms-python.python-2023.14.0/pythonFiles/get_output_via_markers.py ~/.vscode-server/extensions/ms-python.python-2023.14.0/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir ~/projects/dams-core -s --cache-clear test
2023-10-31 14:28:23.628 [info] cwd: ~/projects/dams-core
2023-10-31 14:28:23.633 [error] Error discovering pytest tests:
 [Error: spawn conda ENOENT
	at Process.ChildProcess._handle.onexit (node:internal/child_process:283:19)
	at onErrorNT (node:internal/child_process:478:16)
	at processTicksAndRejections (node:internal/process/task_queues:83:21)] {
  errno: -2,
  code: 'ENOENT',
  syscall: 'spawn conda',
  path: 'conda',
  spawnargs: [
    'run',
    '-n',
    'common_env',
    '--no-capture-output',
    'python',
    '/home/matbouchy/.vscode-server/extensions/ms-python.python-2023.14.0/pythonFiles/get_output_via_markers.py',
    '/home/matbouchy/.vscode-server/extensions/ms-python.python-2023.14.0/pythonFiles/testing_tools/run_adapter.py',
    'discover',
    'pytest',
    '--',
    '--rootdir',
    '/home/matbouchy/projects/dams-core',
    '-s',
    '--cache-clear',
    'test'
  ]
}

Running manually gives the right result:

(common_env) [14:07:09]matbouchy@pxssmtdam001 ~/projects/dams-core$ conda run -n common_env --no-capture-output python ~/.vscode-server/extensions/ms-python.python-2023.14.0/pythonFiles/get_output_via_markers.py ~/.vscode-server/extensions/ms-python.python-2023.14.0/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir ~/projects/dams-core -s --cache-clear test
>>>PYTHON-EXEC-OUTPUT
[{"rootid": ".", "root": "/home/matbouchy/projects/dams-core", "parents": [{"id": "./test", "kind": "folder", "name": "test", "parentid": ".", "relpath": "./test"}, {"id": "./test/dams_core", "kind": "folder", "name": "dams_core", "parentid": "./test", "relpath": "./test/dams_core"}, {"id": "./test/dams_core/data", "kind": "folder", "name": "data", "parentid": "./test/dams_core", "relpath": "./test/dams_core/data"}, {"id": "./test/dams_core/data/access", "kind": "folder", "name": "access", "parentid": "./test/dams_core/data", "relpath": "./test/dams_core/data/access"}, {"id": "./test/dams_core/data/access/test_accessor.py", "kind": "file", "name": "test_accessor.py", "parentid": "./test/dams_core/data/access", "relpath": "./test/dams_core/data/access/test_accessor.py"}], "tests": [{"id": "./test/dams_core/data/access/test_accessor.py::test_double_bitemporal_join", "name": "test_double_bitemporal_join", "source": "./test/dams_core/data/access/test_accessor.py:8", "markers": [], "parentid": "./test/dams_core/data/access/test_accessor.py"}]}]
<<<PYTHON-EXEC-OUTPUT

User Settings

Multiroot scenario, following user settings may not apply:

languageServer: "Pylance"

testing
• cwd: "/home/matbouchy/projects/dams-core"
• pytestArgs: ["test"]
• pytestEnabled: true

Extension version: 2023.14.0
VS Code version: Code 1.81.1 (6c3e3dba23e8fadc360aed75ce363ba185c49794, 2023-08-09T22:22:42.175Z)
OS version: Windows_NT x64 10.0.19045
Modes:
Remote OS version: Linux x64 4.18.0-305.25.1.el8_4.x86_64

System Info

Item	Value
CPUs	Intel(R) Xeon(R) Gold 6326 CPU @ 2.90GHz (16 x 2893)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: disabled_off direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok video_decode: enabled video_encode: enabled vulkan: disabled_off webgl: enabled webgl2: enabled webgpu: enabled
Load (avg)	undefined
Memory (System)	60.00GB (45.13GB free)
Process Argv	--crash-reporter-id a8e817e1-2d0c-4559-a5da-0b853dd18d7f
Screen Reader	no
VM	100%

Item	Value
Remote	SSH: pxssmtdam001
OS	Linux x64 4.18.0-305.25.1.el8_4.x86_64
CPUs	Intel(R) Xeon(R) Gold 6354 CPU @ 3.00GHz (72 x 800)
Memory (System)	251.32GB (206.56GB free)
VM	0%

A/B Experiments

vsliv368cf:30146710
vsreu685:30147344
python383cf:30185419
vspor879:30202332
vspor708:30202333
vspor363:30204092
vstes627:30244334
vslsvsres303:30308271
vserr242:30382549
pythontb:30283811
vsjup518:30340749
pythonptprofiler:30281270
vshan820:30294714
vstes263:30335439
vscorecescf:30445987
vscod805cf:30301675
binariesv615:30325510
bridge0708:30335490
bridge0723:30353136
vsaa593cf:30376535
pythonvs932:30410667
py29gd2263cf:30856253
vsclangdf:30486550
c4g48928:30535728
dsvsc012:30540252
pynewext54:30695312
azure-dev_surveyone:30548225
3biah626:30602489
f6dab269:30613381
a9j8j154:30646983
showlangstatbar:30737416
0bi6i642:30841073
03d35959:30757346
pythonfmttext:30731395
fixshowwlkth:30771522
showindicator:30805244
pythongtdpath:30769146
i26e3531:30792625
pythonnosmt12:30797651
pythonidxpt:30866567
pythonnoceb:30805159
copilotsettingt:30859503
asynctok:30869155
dsvsc013:30795093
dsvsc014:30804076
dsvsc015:30845448
pythontestfixt:30871694
pythonregdiag2:30871582
pyreplss1:30865275
pythoncet00cf:30874137
h48ei257:30870396
pythontbext0:30864172

[eleanorjboyd]

Hello! Can you try on our rewrite that redesigns the architecture of how teseting works? To try it yourself, add this setting to your users settings.json "python.experiments.optInto": ["pythonTestAdapter"].

You can confirm you have the rewrite enabled by setting "python.analysis.logLevel": "Trace", in your user settings then check for Experiment 'pythonTestAdapter' is active in your python logs.

If this doesn't fix your issue, please attach logs from the "python" output channel with the trace setting as decribed above.

[mattB1989]

Hi @eleanorjboyd , I followed your advice, no luck. Please see below the logs.

I am not fully sure, but conda itself is an alias in the shell, is the environment properly capturing this ? This is not my set up, this what happens when running conda init. It might be that the extension is using a different profile or not loading it properly.

which conda
conda ()
{
    \local cmd="${1-__missing__}";
    case "$cmd" in
        activate | deactivate)
            __conda_activate "$@"
        ;;
        install | update | upgrade | remove | uninstall)
            __conda_exe "$@" || \return;
            __conda_reactivate
        ;;
        *)
            __conda_exe "$@"
        ;;
    esac
}

2023-11-01 02:47:46.876 [info] Experiment 'pythonTestAdapter' is active
...
2023-11-01 02:47:57.298 [debug] Cached data exists KeyPrefix=Cache_Method_Output_S.getCondaVersion-Args=
2023-11-01 02:47:57.299 [debug] Found executable within conda env {"name":"common_env","prefix":"/export/home/anaconda/2022.10/envs/common_env"}
2023-11-01 02:47:57.310 [info] > conda run -n common_env --no-capture-output python ~/.vscode-server/extensions/ms-python.python-2023.14.0/pythonFiles/get_output_via_markers.py -m pytest -p vscode_pytest --collect-only test
2023-11-01 02:47:57.310 [info] cwd: ~/projects/dams-core
2023-11-01 02:47:57.311 [error] Error while trying to run pytest discovery, 
Error: spawn conda ENOENT


2023-11-01 02:47:57.314 [error] pytest test discovery error
 [Error: spawn conda ENOENT
	at Process.ChildProcess._handle.onexit (node:internal/child_process:283:19)
	at onErrorNT (node:internal/child_process:478:16)
	at processTicksAndRejections (node:internal/process/task_queues:83:21)] {
  errno: -2,
  code: 'ENOENT',
  syscall: 'spawn conda',
  path: 'conda',
  spawnargs: [
    'run',
    '-n',
    'common_env',
    '--no-capture-output',
    'python',
    '/home/matbouchy/.vscode-server/extensions/ms-python.python-2023.14.0/pythonFiles/get_output_via_markers.py',
    '-m',
    'pytest',
    '-p',
    'vscode_pytest',
    '--collect-only',
    'test'
  ]
}

[eleanorjboyd]

Do you think it could be related to: #17641 (comment), thinking the conda activation might not be working as expected leading to the error.

[mattB1989]

That's definitely possible. My hunch is that when you spawn you don't source the relevant profile. Can you detail what profile you would expect to be loaded ? It needs to find a PATH for instance, and as conda is defined as an alias, it needs to source that definition. Is there a way to run a "printenv && conda ..." ?

[eleanorjboyd]

@karrtikr, what are your thoughts here? This looks like another issue where it seems conda might not be activating correctly in the given scenario.

[karrtikr]

@mattB1989 We do not have access to terminal or terminal profile/init scripts when we run tests, could this be because of that?

[mattB1989]

@karrtikr I imagineso yes. To run a command like conda, it needs to be in the $PATH, and the way conda inits its shell is to add an alias.

conda init will initialize a shell permanently by writing some shell code in the relevant startup scripts of your shell (e.g. ~/.bashrc). This is done through different functions defined in conda.core.initialize, namely:

conda docs
I don't see how this can work without sourcing a profile somehow ?

[eleanorjboyd]

@karrtikr, assigning this to you as well so you can follow up while I am out of office- thanks!

[karrtikr]

Gotcha. Let me explore alternatives for you.

@eleanorjboyd Can you point me to where this command is run:

conda run -n common_env --no-capture-output python ~/.vscode-server/extensions/ms-python.python-2023.14.0/pythonFiles/get_output_via_markers.py -m pytest -p vscode_pytest --collect-only test

[karrtikr]

I think I found the issue, ideally we should be running these commands using a shell, but we're using exec without a shell:

vscode-python/src/client/testing/testController/pytest/pytestDiscoveryAdapter.ts

Line 101 in 6333752

const result = execService?.execObservable(execArgs, spawnOptions);

Adding the shell: true option here to spawnOptions should solve the issue, however it will also make testing slower as launching a shell can take some time.

[mattB1989]

I'm guessing this is compiled / transpiled so I can't easily test this, can I ?

[eleanorjboyd]

@karrtikr and @karthiknadig, should I switch this to use shell even if it slows down the testing? Not sure the best answer here.

[karrtikr]

I think we can potentially use execObservable to just open the shell once, and keep using the same shell for subsequent runs without ever closing it, that way we only pay the price of using the shell exactly once. I recall we used to do the same for isort for sorting imports.

That may be premature though, so I will say switch to using shell for now and explore the above in case it's slow enough to bother users.

[eleanorjboyd]

@karrtikr @paulacamargo25 do I need to switch to using shell just for the execObservable or do I also need to add it somewhere to work for debugging? Not sure if "shell: true" is used in debugging or if that is handled at another point.

[karrtikr]

I think debugger uses a shell, so I would suggest to solve this issue just for execObservable for now.