Using a case function with no suffix leads to an unexpected hypen symbol in the test ids

[romanponomaryov]

Hi @smarie!
I've noticed that if I apply @parametrize or @pytest.mark.parametrize to my cases function like this:

@pytest.mark.parametrize('dummy_amount', [1, 0, -1])
def case_amount(dummy_amount):
    return dummy_amount

@parametrize_with_cases('dummy_amount', cases=".")
def test_amount(dummy_amount):
    <some code here>

then in the test report instead of

PASSED ....test_amount[1]
PASSED ....test_amount[0]
PASSED ....test_amount[-1]

I get

PASSED ....test_amount[-1]
PASSED ....test_amount[-0]
PASSED ....test_amount[--1]

Moving case function to another file makes no difference.
Is parametrizing cases even a proper way? I just don't want to create a separate case function for every case, and sometimes the test values are packed in separate variables, so I need a way to iterate through them. Parametrizing seems like a good solution, but then I'm getting the described problem.

Thanks!

[romanponomaryov]

I've looked into it, and it seems these lines are to blame:

python-pytest-cases/pytest_cases/case_parametrizer_new.py

Lines 299 to 300 in 1d33f2f

	return tuple(lazy_value(partial(case_fun, **c.funcargs), id="%s-%s" % (case_id, c.id), marks=c.marks)
	for c in meta._calls)

If I change it so the case where there is no case_id is being accounted for:

        return tuple(lazy_value(partial(case_fun, **c.funcargs), id="%s-%s" % (case_id, c.id) if case_id else "%s" % c.id, marks=c.marks)
                    for c in meta._calls)

my problem goes away

[smarie]

I @romanponomaryov , which version of pytest-cases do you use ? Indeed if I execute your original code, I can not reproduce the issue. I get:

test_issue_142.py::test_amount[amount-1] 
test_issue_142.py::test_amount[amount-0] 
test_issue_142.py::test_amount[amount--1]

Indeed the amount in the braquets denote the case id (extracted from the function name by removing the prefix), because you may have several cases.

Could it be that in you actual code, you have a case function named def case(...) or def case_(...) ? In that situation, indeed the case id becomes the empty string and what you describe can happen. Now is this something to take into account ? I am not convinced. Or if I take it into account I rather suggest to replace the empty case id with something like '<empty_case_id>', so that users understand it. What do you think ?

[romanponomaryov]

Hey @smarie , sorry for misinforming you.
The problem indeed only happens with one additional option - prefix. Otherwise I get the same result as you.
So, my code looks like this:

@pytest.mark.parametrize('dummy_amount', [1, 0, -1])
def case_amount(dummy_amount):
    return dummy_amount

@parametrize_with_cases('dummy_amount', prefix="case_amount", cases=".")
def test_amount(dummy_amount):
    <some code here>

I am using the latest version of pytest-cases (2.3.0)
As for the prefix option - I need to use it, because in my cases file there are many case functions, each needed for different test function in my tests file.

[smarie]

Now I understand and I am able to reproduce it. I fixed it for next version: "<empty_case_id>" will appear when the case id is empty.

Thanks a lot for spotting this issue !

[romanponomaryov]

Hi @smarie ,
I've managed a workaround for my problem, but only partly. There is still something there I don't understand, so if you could please help me or explain where to look.
So, I have a workaround for parametrizing single values:

@parametrize('dummy_amount', [0, 3, -4, -1])
def case_list(dummy_amount):
    return dummy_amount

remove_redundant_hyphen = lambda **args: "-".join(str(v)[1:] for n, v in args.items())

@parametrize_with_cases('dummy_amount', prefix="case_list", cases='.', idgen=remove_redundant_hyphen)
def test_list(dummy_amount):
    assert True

This works ok and I get in report:

PASSED ...test_experiment.py::test_list[0]
PASSED ...test_experiment.py::test_list[3]
PASSED ...test_experiment.py::test_list[-4]
PASSED ...test_experiment.py::test_list[-1]

But this does not work with parametrizing more than one value:

@parametrize('dummy_amount, exp_amount', [(0, 3), (-4, -1), (5, 8)])
def case_tuples(dummy_amount, exp_amount):
    return dummy_amount, exp_amount

remove_redundant_hyphen = lambda **args: "-".join(str(v)[1:] for n, v in args.items())

@parametrize_with_cases('dummy_amount, exp_amount', prefix="case_tuples", cases='.', idgen=remove_redundant_hyphen2)
def test_tuples(dummy_amount, exp_amount):
    assert True

and I get in report:

PASSED ...test_experiment.py::test_tuples[-0-3]
PASSED ...test_experiment.py::test_tuples[--4--1]
PASSED ...test_experiment.py::test_tuples[-5-8]

while I'm aiming at getting same result as with @pytest.mark.parametrize, namely:

PASSED ...test_experiment.py::test_tuples[0-3]
PASSED ...test_experiment.py::test_tuples[-4--1]
PASSED ...test_experiment.py::test_tuples[5-8]

The output in report seems to be not affected at all, no matter how I change the function provided to idgen or if I introduce ids parameter into @parametrize_with_cases , like this:

tuples = [(0, 3), (-4, -1), (5, 8)]

@parametrize_with_cases('dummy_amount, exp_amount', prefix="case_tuples", cases='.', ids=[str(x) for x in tuples])

[smarie]

Hi @romanponomaryov , I think that there was some misunderstanding here. The easiest workaround until the new release ships is just to use a non empty case id. So the case function names should have a suffix after the prefix, for example:

@parametrize('dummy_amount', [0, 3, -4, -1])
def case_list_a(dummy_amount):  # <------------ notice the name here
    return dummy_amount

def case_list_b():  # <------------ notice the name here
    return 12

@parametrize_with_cases('amount', prefix="case_list_", cases='.')
def test_list(amount):
    assert True

Yields:

test_tmp.py::test_list[a-0] 
test_tmp.py::test_list[a-3] 
test_tmp.py::test_list[a--4] 
test_tmp.py::test_list[a--1] 
test_tmp.py::test_list[b] 

I do not think that there is a need to remove the case function name from the test id in the particular situation where there is a single case function. Do you ? That would require quite cumbersome/specific code for something that does not bring much to the end user in my opinion..

[romanponomaryov]

Hey @smarie
"I do not think that there is a need to remove the case function name from the test id in the particular situation where there is a single case function." - for backwards compatibility with @pytest.mark.parametrize it would be ideal to have an option to mimick @pytest.mark.parametrize output. I think this is mostly useful for other plugins, as in my case.
In my specific case, I need this for pytest-dependency plugin. It ensures that my other test that depends on the test_tuples only runs if all the cases in the test_tuples passed, otherwise it skips. So this plugin takes the test ids of the test_tuples successful runs and compares them to the precompiled list of expected values of these runs and makes a decision. This precompiled list is made independently by a fixture function (this solution is given in pytest-dependency docs) , and the plugin depends on the output of @pytest.mark.parametrize, so it expects the same format of test ids. That's why I am trying to get the test ids made only with the values used in the test case because it's what @pytest.mark.parametrize outputs.