Issues with conditional imports

[JukkaL]

Even though #649 was closed, there still remains several issues related to conditional imports. This issue is a top-level issue for these.

Related, more specific issues:

• Can't redefine name imported from another module #591
• Conditional class import #1107
• mypy bug with try/except conditional imports #1153
• Conditional import of izip is problematic #1026

[gvanrossum]

Also #1289.

[JukkaL]

Also #1327.

[JukkaL]

And #1334.

[cjerdonek]

I found that the following pattern can be used as a workaround in cases where you want to set a failed import to None on a failed import and still have the revealed type be Union[_importlib_modulespec.ModuleType, None] afterwards:

try:
    from pip._vendor import colorama as _colorama
 except Exception:
    colorama = None
else:
    colorama = _colorama

This is to avoid the following error in the except block:

Incompatible types in assignment (expression has type "None", variable has type Module)

[ethanhs]

As a (more) general solution to this problem, perhaps we could treat blocks like:

try:
    from foo import A
except ImportError:
    A = None

as A = TypeVar('A', foo.A, None)

and

try:
    from foo import A
except ImportError:
    from bar import A # ( or even from bar import AA as A)

as A = TypeVar('A', foo.A, bar.A)

And so on.

This is nice because you can do things like

from typing import TypeVar


class A:
    def __add__(self, a: A) -> str:
        ...
        
class AA:
    def __add__(self, a: AA) -> str:
        ...
        
T = TypeVar('T', A, AA)

def add(a: T, b: T) -> T:
    return a + b

if A overrides __add__ and the like.

This would lead to two issues:

1. isinstance/issubclass would be broken, this should be easy to fix, as we just need to silence the warning for the special form (the type var would only exist in mypy)

2. This would lead to many more unsolved type variables (which currently will result in Anys). Perhaps we can have unsolved but bounded type variables generate more Unions of their bounds?

For example, if you call the add function above with two variables of type T, you will get an Any, which is clearly not desirable, but if we change things so that you get Union[A, AA], this may be possible?

Hopefully this suggestion makes sense, let me know if there are questions or holes I am missing.

[JukkaL]

An interesting idea! Treating conditionally imported types as type variables probably wouldn't work in some basic use cases, unfortunately. Here are some things that could cause trouble:

• Using an unbound type variable as the type of a variable is not supported.
• A base class such as List[A] would make the class a generic class, which could be surprising.
• Various error messages related to type variable could be confusing.
• Having a type variable as the return type (say, in def f() -> A: ...) would result in unresolved type variables on most call sites, as you pointed out.
• Fairly often a conditional import would be used for Python 2/3 compatibility, and only one of the target modules will exist on the target Python version.

Some of these could be improved/fixed, but there may be other problematic things that we don't know about yet.

[ilevkivskyi]

Another example is #8319 (closing it as a duplicate).

[vpogrebi]

I have an issue related to mypy, conditional import and validation of the __all package attribute (defined as a Tuple containing variable number of items depending on the success of the conditional import). Consider following within package's init.py:

try:
    from .__version__ import version as __version__
    __all__ = (
        '...........',
        '...........',
        '...........',
        '__version__'
    )
except ImportError:
    __all__ = (
        '...........',
        '...........',
        '...........'
    )

When this code is checked by mypy, it raises following error:

______________________________ units/__init__.py _______________________________
15: error: Incompatible types in assignment (expression has type "Tuple[str, str, str]", variable has type "Tuple[str, str, str, str]")
_________________________________ test session _________________________________
mypy exited with status 1.
=============================== warnings summary ===============================

This issue keeps me from building Python package in the presence of 'mypy' check; I had to explicitly disable mypy check in order to build this package.

I'd like to have mypy recognize possibility of variable-size Tuple in the presence of conditional import (and in general - same-name Tuple can be instantiated having different size depending on some conditions), or be able to ignore this error (unfortunately, mypy output does not report error number that can be ignored), or entirely exclude given file from mypy check (similarly to flake8's "exclude" in the setup.cfg).

[JukkaL]

@vpogrebi This looks a little different from the other issues discussed here. Can you open a separate issue about conditionally defined __all__? Here the correct fix may be easier (don't infer a fixed-length tuple type for __all__ as an ad hoc fix).

[vpogrebi]

New issue #8758 has been created.

[rodrigc]

@JukkaL While introducing mypy to the CI for the Twisted codebase, I ran into this problem which is similar to the examples mentioned in this issue, similar to the one brought up by @ethanhs:

twisted/twisted#1261 (comment)

What is the way to deal with this:

try:
    from foo import A
except ImportError:
    A = None

Is there something that needs to be fixed in mypy?
For now, I just put:

A = None # type:ignore

to turn off mypy errors on that line. As much as possible, I want to avoid turning off mypy like that.

The other option is to follow the example mentioned by @cjerdonek here: #1297 (comment)

and do:

try:
   from foo import A as _A
except ImportError:
   A = None
else:
   A = _A

[adam-grant-hendry]

I found that the following pattern can be used as a workaround in cases where you want to set a failed import to None on a failed import and still have the revealed type be Union[_importlib_modulespec.ModuleType, None] afterwards:

try:
    from pip._vendor import colorama as _colorama
 except Exception:
    colorama = None
else:
    colorama = _colorama

This is to avoid the following error in the except block:

Incompatible types in assignment (expression has type "None", variable has type Module)

Interesting. Adam Johnson tried something similar, which fails in this gist. I've tried yours in this gist and it does work.

Nice workaround.

[hauntsaninja]

Closing, see here: #1153 (comment)