-
Notifications
You must be signed in to change notification settings - Fork 40
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Issues with multiprocessing #242
Comments
Added an example to recreate the issue. When running the test it just keep retrying. |
Thanks @twsl ! Multiprocessing should definitely supported, if it is supported by pytest. As a workaround I guess that you can put the multiprocessing code in a separate module (for example the conftest.py or another file in the tests folder). That should prevent I'll have a look one of these days |
I was able to reproduce the issue. from multiprocessing import Pool
import pytest
from functools import partial
def f(a, b, t):
if t:
assert a ** 2 == b
else:
assert not (a ** 2 == b)
@pytest.mark.parametrize("x,y", [(2, 4), (3, 9)])
def test_f_xy(x, y):
pool = Pool(processes=2)
pool.starmap(partial(f), [(x, y, False), (x, y, True)]) Running it yields to the two tests failing as expected, and it completes correctly. While the following hangs as you point out: from pytest_cases import parametrize_with_cases
from multiprocessing import Pool
from functools import partial
class TestCases:
def case_1(self):
return 2, 4
def case_2(self):
return 3, 9
def f(a, b, t):
if t:
assert a ** 2 == b
else:
assert not (a ** 2 == b)
@parametrize_with_cases("x,y", cases=TestCases)
def test_f_xy(x, y):
pool = Pool(processes=2)
pool.starmap(partial(f), [(x, y, False), (x, y, True)]) |
Apparently we can detect if a process is the main process using this: try:
from multiprocessing import parent_process
except ImportError:
from multiprocessing import current_process
from multiprocessing.process import _MainProcess
def is_main_process():
return isinstance(current_process(), _MainProcess)
else:
def is_main_process():
return parent_process() is None We could therefore use this to only raise the However if we spawn a single process like below, then p = Process(target=partial(f), args=(x, y, True))
p.start()
p.join() So this workaround is not satisfaying. I will therefore simply comment the error-raising line, hoping that it will not make harder to debug a future situation where the config is None. |
Should be fixed in 3.6.8. let me know ! |
I am having issues with multiprocessing triggering this error message:
python-pytest-cases/src/pytest_cases/common_pytest.py
Line 647 in 4976c00
Here is a minimal example to recreate the issue.
Is multiprocessing not supported?
The text was updated successfully, but these errors were encountered: