Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tests are discovered correctly but then an incorrect argument is supplied to PyTest, leading test-result discovery to fail (Git Submodules?) #17903

Closed
Mithrandir2k18 opened this issue Nov 2, 2021 · 1 comment
Assignees
Labels
area-testing bug Issue identified by VS Code Team member as probable bug

Comments

@Mithrandir2k18
Copy link

Mithrandir2k18 commented Nov 2, 2021

Environment data

  • VS Code version:
    Version: 1.61.2
    Commit: 6cba118ac49a1b88332f312a8f67186f7f3c1643
    Date: 2021-10-19T14:58:13.605Z
    Electron: 13.5.1
    Chrome: 91.0.4472.164
    Node.js: 14.16.0
    V8: 9.1.269.39-electron.0
    OS: Linux x64 5.4.0-90-generic snap
  • Extension version (available under the Extensions sidebar): v2021.10.1365161279
  • OS and version: Ubuntu 20.04
  • Python version (& distribution if applicable, e.g. Anaconda): 3.8.10
  • Type of virtual environment used (N/A | venv | virtualenv | conda | ...): None
  • Relevant/affected Python packages and their versions: PyTest 6.0.0
  • Relevant/affected Python-related VS Code extensions and their versions: VSCode Native Test Explorer
  • Value of the python.languageServer setting: Pylance

Expected behaviour

Given the following repo structure:

/work/repos/my_repo
|
-src/
    |
    - main.py
 - dependencies/
      |
     -my_git_submodule/
            - tests/
                   |
                   - test_other_main.py
           ...
-tests/
    |
    - conftest.py
    - test_sometests.py
    - integration_tests/
            |
            - test_integration_main.py

Discovered tests in workspace are run and their results displayed when doing CTRL+SHIFT+P: Test: Run All Tests.
Expect to run the command using the supplied path to the test folder.

Actual behaviour

Pytest fails to run, using the following command:

cwd: /work/repos/my_repo

Causing the error: ERROR - ModuleNotFoundError: No module named 'tests.integration_tests'
But it runs correctly when only running the tests in the tests folder and that can be achieved by clicking "Run All" on the "tests" subgroup:

> /usr/bin/python3 -m pytest --rootdir /work/repos/my_repo --override-ini junit_family=xunit1 --junit-xml=/tmp/tmp-14347RaAiZqepCFUO.xml ./tests
cwd: /work/repos/my_repo

This works without any issues, however this is not run when doing the VSCode Run All tests function and I have not been able to configure this working to command to be used.

Another way to get it working is ignoring the submodules in dependencies(they should be unit-tested using their own repos workflows) using --ignore-glob=dependencies/* in python.testing.pytestArgs. However I don't see why not manually ignoring submodules should break test(result) discovery.

> /usr/bin/python3 -m pytest --rootdir /work/repos/my_repo --override-ini junit_family=xunit1 --junit-xml=/tmp/tmp-14347jcCkQFFhe9c0.xml --ignore-glob=dependencies/* ./tests
cwd: /work/repos/my_repo

So the first command fails while the two other commands work. I am not sure if it is intended for the first command to fail.
When using pytest from the terminal, all that's required to get it to run is a simple pytest.ini like this:

[pytest]
testpaths = 
    tests

and the simply doing pytest with the repo as cwd, no ignoring the dependencies folder required.

Steps to reproduce:

Replicate above repo structure with Pytest files in /tests and in /tests/integration_tests and include another repo as a submodule that also contains its own tests.

@Mithrandir2k18 Mithrandir2k18 added bug Issue identified by VS Code Team member as probable bug triage-needed Needs assignment to the proper sub-team labels Nov 2, 2021
@karthiknadig karthiknadig self-assigned this Nov 2, 2021
@karthiknadig karthiknadig added triage area-testing and removed triage-needed Needs assignment to the proper sub-team labels Nov 2, 2021
@karthiknadig
Copy link
Member

This will be resolved with #17242

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Dec 3, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-testing bug Issue identified by VS Code Team member as probable bug
Projects
None yet
Development

No branches or pull requests

2 participants