Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pytest discovery fails when an unmatched end square bracket is given to parametrize #17951

Closed
1 task done
anderssonjohan opened this issue Nov 6, 2021 · 2 comments
Closed
1 task done

pytest discovery fails when an unmatched end square bracket is given to parametrize #17951

anderssonjohan opened this issue Nov 6, 2021 · 2 comments
Assignees
@karthiknadig
Labels
area-testing bug Issue identified by VS Code Team member as probable bug

Comments

@anderssonjohan
Copy link

anderssonjohan commented Nov 6, 2021 •
edited
Loading

VS Code version

1.61.2

Extension version

2021.11.1422169775

OS type

macOS

OS version

11.4

Python distribution

python.org

Python version

3.7.8

Language server

Pylance

Expected behaviour

Test discovery should not fail for the project when a single end square bracket in argument given to @pytest.mark.parametrize

Actual behaviour

Test discovery fails.

Steps to reproduce

Use an unmatched end square bracket (]) in a string argument to pytest.mark.parametrize and the test discovery will fail.

Example test:

@pytest.mark.parametrize("id", [("]")])
def test_parametrize(id):
    pass

Sample repo with this test where test discovery fails:
https://github.com/anderssonjohan/parametrize-discovery-error

Example of a discoverable test, where a pair of brackets works fine:

@pytest.mark.parametrize("id", [("[]")])
def test_parametrize(id):
    pass

Logs

Error 2021-11-06 09:12:19: Error discovering pytest tests:
 [r [Error]: ============================= test session starts ==============================
platform darwin -- Python 3.8.6, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /build/Documents/src/parametrize-discovery-error
The Python extension has run into an unexpected situation
while processing a pytest node during test discovery.  Please
Please open an issue at:
  https://github.com/microsoft/vscode-python/issues
and paste the following output there.

nodeid: ./tests/test_parametrize_discovery_error.py::test_parametrize[]]

traceback:
  File "/build/.vscode/extensions/ms-python.python-2021.11.1422169775/pythonFiles/testing_tools/run_adapter.py", line 22, in <module>
    main(tool, cmd, subargs, toolargs)
  File "/build/.vscode/extensions/ms-python.python-2021.11.1422169775/pythonFiles/testing_tools/adapter/__main__.py", line 100, in main
    parents, result = run(toolargs, **subargs)
  File "/build/.vscode/extensions/ms-python.python-2021.11.1422169775/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 30, in discover
    ec = _pytest_main(pytestargs, [_plugin])
  File "/usr/local/lib/python3.8/site-packages/_pytest/config/__init__.py", line 162, in main
    ret: Union[ExitCode, int] = config.hook.pytest_cmdline_main(
  File "/usr/local/lib/python3.8/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/usr/local/lib/python3.8/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/usr/local/lib/python3.8/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 316, in pytest_cmdline_main
    return wrap_session(config, _main)
  File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 269, in wrap_session
    session.exitstatus = doit(config, session) or 0
  File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 322, in _main
    config.hook.pytest_collection(session=session)
  File "/usr/local/lib/python3.8/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/usr/local/lib/python3.8/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/usr/local/lib/python3.8/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 333, in pytest_collection
    session.perform_collect()
  File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 637, in perform_collect
    hook.pytest_collection_modifyitems(
  File "/usr/local/lib/python3.8/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/usr/local/lib/python3.8/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/usr/local/lib/python3.8/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/build/.vscode/extensions/ms-python.python-2021.11.1422169775/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 92, in pytest_collection_modifyitems
    test, parents = self.parse_item(item)
  File "/build/.vscode/extensions/ms-python.python-2021.11.1422169775/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 77, in parse_item
    return parse_item(item)
  File "/build/.vscode/extensions/ms-python.python-2021.11.1422169775/pythonFiles/testing_tools/adapter/pytest/_pytest_item.py", line 161, in parse_item
    (nodeid, parents, fileid, testfunc, parameterized) = _parse_node_id(
  File "/build/.vscode/extensions/ms-python.python-2021.11.1422169775/pythonFiles/testing_tools/adapter/pytest/_pytest_item.py", line 144, in <lambda>
    _parse_node_id=(lambda *a: _parse_node_id(*a)),
  File "/build/.vscode/extensions/ms-python.python-2021.11.1422169775/pythonFiles/testing_tools/adapter/pytest/_pytest_item.py", line 382, in _parse_node_id
    testid, name, kind = next(nodes)
  File "/build/.vscode/extensions/ms-python.python-2021.11.1422169775/pythonFiles/testing_tools/adapter/pytest/_pytest_item.py", line 479, in _iter_nodes
    raise should_never_reach_here(
  File "/build/.vscode/extensions/ms-python.python-2021.11.1422169775/pythonFiles/testing_tools/adapter/pytest/_pytest_item.py", line 132, in should_never_reach_here
    traceback.print_stack()
collected 4 items

<Package tests>
  <Module test_parametrize_discovery_error.py>
    <Function test_parametrize[(]>
    <Function test_parametrize[)]>
    <Function test_parametrize[[]>
    <Function test_parametrize[]]>
The Python extension has run into an unexpected situation
while processing a pytest node during test discovery.  Please
Please open an issue at:
  https://github.com/microsoft/vscode-python/issues
and paste the following output there.

nodeid: ./tests/test_parametrize_discovery_error.py::test_parametrize[]]

traceback:
  File "/build/.vscode/extensions/ms-python.python-2021.11.1422169775/pythonFiles/testing_tools/run_adapter.py", line 22, in <module>
    main(tool, cmd, subargs, toolargs)
  File "/build/.vscode/extensions/ms-python.python-2021.11.1422169775/pythonFiles/testing_tools/adapter/__main__.py", line 100, in main
    parents, result = run(toolargs, **subargs)
  File "/build/.vscode/extensions/ms-python.python-2021.11.1422169775/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 30, in discover
    ec = _pytest_main(pytestargs, [_plugin])
  File "/usr/local/lib/python3.8/site-packages/_pytest/config/__init__.py", line 162, in main
    ret: Union[ExitCode, int] = config.hook.pytest_cmdline_main(
  File "/usr/local/lib/python3.8/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/usr/local/lib/python3.8/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/usr/local/lib/python3.8/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 316, in pytest_cmdline_main
    return wrap_session(config, _main)
  File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 269, in wrap_session
    session.exitstatus = doit(config, session) or 0
  File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 322, in _main
    config.hook.pytest_collection(session=session)
  File "/usr/local/lib/python3.8/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/usr/local/lib/python3.8/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/usr/local/lib/python3.8/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 333, in pytest_collection
    session.perform_collect()
  File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 641, in perform_collect
    hook.pytest_collection_finish(session=self)
  File "/usr/local/lib/python3.8/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/usr/local/lib/python3.8/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/usr/local/lib/python3.8/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/build/.vscode/extensions/ms-python.python-2021.11.1422169775/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 107, in pytest_collection_finish
    test, parents = self.parse_item(item)
  File "/build/.vscode/extensions/ms-python.python-2021.11.1422169775/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 77, in parse_item
    return parse_item(item)
  File "/build/.vscode/extensions/ms-python.python-2021.11.1422169775/pythonFiles/testing_tools/adapter/pytest/_pytest_item.py", line 161, in parse_item
    (nodeid, parents, fileid, testfunc, parameterized) = _parse_node_id(
  File "/build/.vscode/extensions/ms-python.python-2021.11.1422169775/pythonFiles/testing_tools/adapter/pytest/_pytest_item.py", line 144, in <lambda>
    _parse_node_id=(lambda *a: _parse_node_id(*a)),
  File "/build/.vscode/extensions/ms-python.python-2021.11.1422169775/pythonFiles/testing_tools/adapter/pytest/_pytest_item.py", line 382, in _parse_node_id
    testid, name, kind = next(nodes)
  File "/build/.vscode/extensions/ms-python.python-2021.11.1422169775/pythonFiles/testing_tools/adapter/pytest/_pytest_item.py", line 479, in _iter_nodes
    raise should_never_reach_here(
  File "/build/.vscode/extensions/ms-python.python-2021.11.1422169775/pythonFiles/testing_tools/adapter/pytest/_pytest_item.py", line 132, in should_never_reach_here
    traceback.print_stack()
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 637, in perform_collect
INTERNALERROR>     hook.pytest_collection_modifyitems(
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/pluggy/_hooks.py", line 265, in __call__
INTERNALERROR>     return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/pluggy/_manager.py", line 80, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/pluggy/_callers.py", line 60, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/pluggy/_result.py", line 60, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/pluggy/_callers.py", line 39, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/build/.vscode/extensions/ms-python.python-2021.11.1422169775/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 92, in pytest_collection_modifyitems
INTERNALERROR>     test, parents = self.parse_item(item)
INTERNALERROR>   File "/build/.vscode/extensions/ms-python.python-2021.11.1422169775/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 77, in parse_item
INTERNALERROR>     return parse_item(item)
INTERNALERROR>   File "/build/.vscode/extensions/ms-python.python-2021.11.1422169775/pythonFiles/testing_tools/adapter/pytest/_pytest_item.py", line 161, in parse_item
INTERNALERROR>     (nodeid, parents, fileid, testfunc, parameterized) = _parse_node_id(
INTERNALERROR>   File "/build/.vscode/extensions/ms-python.python-2021.11.1422169775/pythonFiles/testing_tools/adapter/pytest/_pytest_item.py", line 144, in <lambda>
INTERNALERROR>     _parse_node_id=(lambda *a: _parse_node_id(*a)),
INTERNALERROR>   File "/build/.vscode/extensions/ms-python.python-2021.11.1422169775/pythonFiles/testing_tools/adapter/pytest/_pytest_item.py", line 382, in _parse_node_id
INTERNALERROR>     testid, name, kind = next(nodes)
INTERNALERROR>   File "/build/.vscode/extensions/ms-python.python-2021.11.1422169775/pythonFiles/testing_tools/adapter/pytest/_pytest_item.py", line 479, in _iter_nodes
INTERNALERROR>     raise should_never_reach_here(
INTERNALERROR> NotImplementedError: Unexpected pytest node (see printed output).
INTERNALERROR> 
INTERNALERROR> During handling of the above exception, another exception occurred:
INTERNALERROR> 
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 269, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 322, in _main
INTERNALERROR>     config.hook.pytest_collection(session=session)
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/pluggy/_hooks.py", line 265, in __call__
INTERNALERROR>     return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/pluggy/_manager.py", line 80, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/pluggy/_callers.py", line 60, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/pluggy/_result.py", line 60, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/pluggy/_callers.py", line 39, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 333, in pytest_collection
INTERNALERROR>     session.perform_collect()
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 641, in perform_collect
INTERNALERROR>     hook.pytest_collection_finish(session=self)
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/pluggy/_hooks.py", line 265, in __call__
INTERNALERROR>     return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/pluggy/_manager.py", line 80, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/pluggy/_callers.py", line 60, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/pluggy/_result.py", line 60, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/pluggy/_callers.py", line 39, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/build/.vscode/extensions/ms-python.python-2021.11.1422169775/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 107, in pytest_collection_finish
INTERNALERROR>     test, parents = self.parse_item(item)
INTERNALERROR>   File "/build/.vscode/extensions/ms-python.python-2021.11.1422169775/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 77, in parse_item
INTERNALERROR>     return parse_item(item)
INTERNALERROR>   File "/build/.vscode/extensions/ms-python.python-2021.11.1422169775/pythonFiles/testing_tools/adapter/pytest/_pytest_item.py", line 161, in parse_item
INTERNALERROR>     (nodeid, parents, fileid, testfunc, parameterized) = _parse_node_id(
INTERNALERROR>   File "/build/.vscode/extensions/ms-python.python-2021.11.1422169775/pythonFiles/testing_tools/adapter/pytest/_pytest_item.py", line 144, in <lambda>
INTERNALERROR>     _parse_node_id=(lambda *a: _parse_node_id(*a)),
INTERNALERROR>   File "/build/.vscode/extensions/ms-python.python-2021.11.1422169775/pythonFiles/testing_tools/adapter/pytest/_pytest_item.py", line 382, in _parse_node_id
INTERNALERROR>     testid, name, kind = next(nodes)
INTERNALERROR>   File "/build/.vscode/extensions/ms-python.python-2021.11.1422169775/pythonFiles/testing_tools/adapter/pytest/_pytest_item.py", line 479, in _iter_nodes
INTERNALERROR>     raise should_never_reach_here(
INTERNALERROR> NotImplementedError: Unexpected pytest node (see printed output).

========================== 4 tests collected in 0.03s ==========================

Traceback (most recent call last):
  File "/build/.vscode/extensions/ms-python.python-2021.11.1422169775/pythonFiles/testing_tools/run_adapter.py", line 22, in <module>
    main(tool, cmd, subargs, toolargs)
  File "/build/.vscode/extensions/ms-python.python-2021.11.1422169775/pythonFiles/testing_tools/adapter/__main__.py", line 100, in main
    parents, result = run(toolargs, **subargs)
  File "/build/.vscode/extensions/ms-python.python-2021.11.1422169775/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 44, in discover
    raise Exception("pytest discovery failed (exit code {})".format(ec))
Exception: pytest discovery failed (exit code 3)

	at ChildProcess.<anonymous> (/build/.vscode/extensions/ms-python.python-2021.11.1422169775/out/client/extension.js:17:38446)
	at Object.onceWrapper (events.js:422:26)
	at ChildProcess.emit (events.js:315:20)
	at maybeClose (internal/child_process.js:1048:16)
	at Process.ChildProcess._handle.onexit (internal/child_process.js:288:5)]

Code of Conduct

  • I agree to follow this project's Code of Conduct
@anderssonjohan anderssonjohan added bug Issue identified by VS Code Team member as probable bug triage-needed Needs assignment to the proper sub-team labels Nov 6, 2021
@anderssonjohan anderssonjohan changed the title pytest discovery fails when parenthesis or bracket characters are given to parametrize pytest discovery fails when an unmatched end square bracket is given to parametrize Nov 7, 2021
@anderssonjohan
Copy link
Author

EDIT: Upgraded the extension to 2021.11 before submitting the issue. Now it seems it only fails on a single end square bracket. Updated the issue to reflect this.

It seems that _find_left_bracket is unable to parse this pytest collection:

<Module test_parametrize_discovery_error.py>
    <Function test_parametrize[]]>

@scwilbanks scwilbanks mentioned this issue Nov 7, 2021
Closed
@karthiknadig karthiknadig self-assigned this Nov 8, 2021
@karthiknadig karthiknadig added area-testing triage and removed triage-needed Needs assignment to the proper sub-team labels Nov 8, 2021
@karthiknadig
Copy link
Member

Duplicate of #17676

@karthiknadig karthiknadig marked this as a duplicate of #17676 Nov 8, 2021
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Dec 9, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@karthiknadig karthiknadig

Labels
area-testing bug Issue identified by VS Code Team member as probable bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@anderssonjohan @karthiknadig