Ignore ABC abstract methods

[saroad2]

Is your feature request related to a problem? Please describe.
Sometimes you want to create an abstract class with abstract methods that will be overridden by other subclasses. For example:

import abc

class Foo(abc.ABC):
    @abc.abstractmethod
    def a():
        ...

class Bar(Foo):
    def a():
        return 1

Pay attention that while Bar.a content should be covered by test, the content of Foo.a should not since there is no actual gain in covering it with a test.

Describe the solution you'd like
Coverage should ignore abstract methods by default and should not require coverage for them.

Describe alternatives you've considered
If you think that sometimes abstract methods should be covered, maybe we can make this ability toggleable by a flag (--ignore-abstract) for example.

Additional context
This API is available only in Python 3, therefore we should make sure we disable it in Python 2.

[nedbat]

Will it work for you to update your .coveragerc file to exclude those methods? Like this:

[report]
exclude_lines:
    pragma: no cover
    @abc.abstractmethod

[saroad2]

No, but the following might will:

[report]
exclude_lines:
    pragma: no cover
    @abc.abstractmethod
    ...

I'll check it.

Even though it is a solid solution, I think that this case is general enough to be handled directly with a flag or a specific configuration.
What do you think?

[nedbat]

You don't want to add "...", that's a regex that matches almost every line of your code (se #1042). Try the setting I suggested, I think it will work for you.

I'm reluctant to add hard-coded options for language features, because different people approach them differently, and the language changes over time.

[nedbat]

I don't know if it would be helpful to add to the default excluded lines pattern?

[ktbarrett]

Another way to do it might be to automatically ignore "trivial" function bodies like mypy already does. "Trivial" function bodies, according to mypy, have a single pass, ..., or raise NotImplementedError statement. This may be possible with a regex, but making rules that handle formatting, comments, etc might be difficult. mypy operates on AST, which makes the implementation of detecting trivial functions trivial 😏.

[nedbat]

Maybe we could add something to the docs that suggest these sorts of regexes? I think regexes in the configuration file are powerful enough for what people need to do.

[ktbarrett]

#894 is blocking that.

[nedbat]

@ktbarrett can you show me some of the code you are working with, and what you would like to do?

[ktbarrett]

I'd like to ignore the bodies of abstract functions.

@abstractmethod
def method(self, arg: int) -> str:
    """docstring with code examples in it"""
    pass

Adding just pass to exclude_lines means I exclude things like

try:
    return may_fail()
except Exception:
    pass
return NotImplemented

I thought for a second I could write a regex that could detect such cases, but I now realize that isn't really reasonable/possible.

I've adjusted my current strategy to:

• add raise NotImplementedError and ... to exclude_lines
• by convention use only those statements for type stubs and abstractmethod bodies and not pass
• by convention never use those statements in executable code, but I can use pass
• update everything that currently does not follow this convention

[nedbat]

There's something people are missing in this discussion: since coverage.py v4.1, if you exclude a decorator, it excludes the entire decorated function. So the best exclude line here would be:

[report]
exclude_lines =
    @abstractmethod
    @abc.abstractmethod

Once you do that, the entire abstract method is excluded from coverage, no matter what statements it contains.

BTW, this is also being discussed in pytest-cov: pytest-dev/pytest-cov#428

[nedbat]

Also, I see that this behavior of excluding a decorator isn't mentioned in the docs! I will fix that.

[ktbarrett]

There is the possibility that abstractmethods could have behavior in them that is intended to be run with a super().method() call. I think it's kind of atypical, but possible. However, the decorator solution works for me because I don't do that.

[ktbarrett]

Also doesn't cover setters of abstractpropertys. I unfortunately have to use abstractproperty because mypy does not like layering property and abstractmethod. But it will be covered by the raise\s+NotImplementedError rule.

If someone were interested I would add the following to your exclude_lines:

    @(?:abc\.)?abstract(?:method|property|classmethod|staticmethod)
    @(?:typing\.)?overload

[nedbat]

I've added to the docs in c6d9eb9.