mutmut run failed to collect stats #336
Comments
Hmm. I guess mutmut should error out there already. Without that step working, running the tests clean is kinda useless.
Are you running tests that themselves modify tests in-place on disk? |
|
No, I'm not changing any test files in the I create tests for pytest in temporary folders and use pytest to run these tests. My assumption is that my tests fail for some reason when you try to collect your stats. But I have currently no idea why. |
|
I added a requirements.txt to my branch and tested it in a normal venv. My best guess at the moment is that your stats collector plugin causes some problems in combination with my tests. I will later take a look at it again. |
|
Getting closer... I discovered that when I go into Yea.. as I write that I get why. I don't use a |
src_path = (Path('mutants') / 'src')
source_path = (Path('mutants') / 'source')
if src_path.exists():
sys.path.insert(0, str(src_path.absolute()))
elif source_path.exists():
sys.path.insert(0, str(source_path.absolute))
else:
sys.path.insert(0, os.path.abspath('mutants'))instead of just inserting |
|
Now I discovered that my trampolines don't preserve generator semantics. So that's where I'm at right now. |
|
I released 3.1.0 with fixes for |
|
my generators return also values. I think you should change your trampoline code to something like this: if not mutant_under_test.startswith(prefix):
result=yield from orig(*args, **kwargs)
return result # for the yield case
mutant_name = mutant_under_test.rpartition('.')[-1]
result = yield from mutants[mutant_name](*args, **kwargs)
return result |
|
Ah. Good point. I pushed a fix for that. Even with that fix I crash here: File "/Users/boxed/Projects/inline-snapshot/mutants/src/inline_snapshot/_inline_snapshot.py", line 2297, in _get_changes
yield from _mutmut_yield_from_trampoline(object.__getattribute__(self, "xǁUndecidedValueǁ_get_changes__mutmut_orig"), object.__getattribute__(self, "xǁUndecidedValueǁ_get_changes__mutmut_mutants"), *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/boxed/Projects/inline-snapshot/mutants/src/inline_snapshot/_inline_snapshot.py", line 35, in _mutmut_yield_from_trampoline
result = yield from orig(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/boxed/Projects/inline-snapshot/mutants/src/inline_snapshot/_inline_snapshot.py", line 1541, in xǁUndecidedValueǁ_get_changes__mutmut_orig
yield from handle(self._ast_node, self._old_value)
File "/Users/boxed/Projects/inline-snapshot/mutants/src/inline_snapshot/_inline_snapshot.py", line 1522, in handle
for item in adapter.items(obj, node):
^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/boxed/Projects/inline-snapshot/mutants/src/inline_snapshot/_adapter/dict_adapter.py", line 294, in items
return _mutmut_trampoline(object.__getattribute__(self, "xǁDictAdapterǁitems__mutmut_orig"), object.__getattribute__(self, "xǁDictAdapterǁitems__mutmut_mutants"), *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/boxed/Projects/inline-snapshot/mutants/src/inline_snapshot/_adapter/dict_adapter.py", line 14, in _mutmut_trampoline
result = orig(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/Users/boxed/Projects/inline-snapshot/mutants/src/inline_snapshot/_adapter/dict_adapter.py", line 60, in xǁDictAdapterǁitems__mutmut_orig
assert isinstance(node, ast.Dict), f'{type(node)}, {ast.unparse(node.parent.parent)}'
^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: <class 'ast.Starred'>, result = orig(*args, **kwargs)I don't really understand how this happens. |
|
I asked me while reading your trampoline code if it could be possible to remove the trampoline indirection and replace it with if-else constructs in the functions, like: def add(a,b):
if __mutmut_use_orig(...):
return a+b
elif __mutmut_use_mutant(1,...):
return a-b
elif __mutmut_use_mutant(2,...):
return ... # some other mutantThis would allow it to use |
|
Hm. Yea I guess that could work too. The function bodies would be rather huge which can have other problems I guess. It would also make it more annoying to implement the code to get the diff for a mutant. I tried adding while 'mutmut' not in frame.f_code.co_name:
frame = frame.f_backinto |
|
are you not mutating |
|
I didn't realize until you said so right now that |
|
I removed the decorator and got mutated. I think the problem is that snapshot() has problems finding the correct frame. I will look at it later again. Maybe this helps you. |
|
Yea ok, so decorators stop mutation! I wrote a small test for that and yea. That should just result in fewer mutants though, not a fail like the traceback above I think.
|
made some progress. skipping the mutmut frames was a good idea, but I had to use: while "mutmut" in frame.f_code.co_name:
frame = frame.f_back
|
|
Doh! haha, yea my bad. Coding at night when you are sleep deprived because you're sick doesn't give you the best code :P |
|
That is maybe an other issue but I found it could be caused by the failing |
|
I found the problem: def assign(self, *args, **kwargs):
yield from _mutmut_yield_from_trampoline(object.__getattribute__(self, "xǁValueAdapterǁassign__mutmut_orig"), object.__getattribute__(self, "xǁValueAdapterǁassign__mutmut_mutants"), *args, **kwargs) This example is from my def assign(self, *args, **kwargs):
result=yield from _mutmut_yield_from_trampoline(object.__getattribute__(self, "xǁValueAdapterǁassign__mutmut_orig"), object.__getattribute__(self, "xǁValueAdapterǁassign__mutmut_mutants"), *args, **kwargs)
return result |
|
Ah. Fixed on main now. Now I can get mutmut to collect stats on your project, but running the clean tests fail on |
|
yes, I know. I will look into it. |
|
This is what I get when I run the following command in rootdir is It should be from I have the same problem when I run it with the test should be from |
|
I made some progress by setting the M mutmut/__main__.py
@@ -683,7 +683,7 @@ class ListAllTestsResult:
class PytestRunner(TestRunner):
def execute_pytest(self, params, **kwargs):
import pytest
- exit_code = int(pytest.main(params, **kwargs))
+ exit_code = int(pytest.main(["--rootdir=.",*params], **kwargs))
if exit_code == 4:
raise BadTestExecutionCommandsException(params)
return exit_codeI use now the correct inline-snapshot version in every case for all of my tests. The reason is that I write source files in temporary folders and execute them with pytest in a subprocess. |
And you want those subprocesses to have mutants turned on?
Not in this case I think. Even if it survives the conf (by say loading it), the registration of the trampoline hit is communicated in-process. I guess that could be changed to be a network call or a local pipe or something... With this conf in setup.cfg i get a bit further [mutmut]
also_copy=
conftest.py
pyproject.toml
do_not_mutate=
src/inline_snapshot/pytest_plugin.py |
I would be nice if it is possible, but it is not a must have. def record_trampoline_hit(name):
if not hasattr(mutmut,"config"):
returnThis causes no crash and I hope it is ok for mutmut. I think the only open problem is that mutmut does not copy my |
|
You can add it to the |
|
This issue turned into a pile of issues, many (all?) of which are solved, so I'm closing this. If there are problems still, it's better to create new issues for that. |
boxed
changed the title
I tried to test inline-snapshot with mutmut but got the following error:
I don't know if this problem is only caused by mutmut, because inline-snapshot does also some special things with pytest/assert-rewriting and some ast analysis which can have problems when the source code is modified before it is compiled.
I hope you can reproduce the problem in this branch with.
> hatch run mutmut runThe text was updated successfully, but these errors were encountered: