Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to selectively disable --disallow-untyped-calls #15845

Merged
merged 4 commits into from
Aug 13, 2023
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions docs/source/command_line.rst
Original file line number Diff line number Diff line change
Expand Up @@ -350,6 +350,33 @@ definitions or calls.
This flag reports an error whenever a function with type annotations
calls a function defined without annotations.

.. option:: --untyped-call-exception

This flag allows to selectively disable :option:`--disallow-untyped-calls`
for functions and methods defined in specific packages, modules, or classes.
Note that each exception entry acts as a prefix. For example (assuming there
are no type annotations for ``numpy`` available):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Numpy's probably not a great example here, as they ship with a py.typed file these days and have pretty complete type annotations. Maybe one of matplotlib, requests or TensorFlow might be a better example? Third-party stubs exist for all three packages, but none of them ships with a py.typed file

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any example of a popular library might become a bad example over time, maybe limit ourselves to something generic like third_party_lib?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, let's use some generic name.


.. code-block:: python

# mypy --disallow-untyped-calls
# --untyped-call-exception=numpy.random
# --untyped-call-exception=foo.A
import numpy as np
from foo import A, B

np.random.gamma(1.0) # OK, function comes from package `numpy.random`
np.fft.fft([1, 2, 3]) # E: Call to untyped function "fft" in typed context

A().meth() # OK, method was defined in class `bar.A`
B().meth() # E: Call to untyped function "meth" in typed context

# file foo.py
class A:
def meth(self): pass
class B:
def meth(self): pass

.. option:: --disallow-untyped-defs

This flag reports an error whenever it encounters a function definition
Expand Down
33 changes: 32 additions & 1 deletion docs/source/config_file.rst
Original file line number Diff line number Diff line change
Expand Up @@ -490,7 +490,38 @@ section of the command line docs.
:default: False

Disallows calling functions without type annotations from functions with type
annotations.
annotations. Note that when used in per-module options, it enables/disables
this check **inside** the module(s) specified, not for functions that come
from that module(s), for example config like this:

.. code-block:: ini

[mypy]
disallow_untyped_calls = True

[mypy-some.library.*]
disallow_untyped_calls = False

will disable this check inside ``some.library``, not for your code that
imports ``some.library``. If you want to selectively disable this check for
all your code that imports ``some.library`` you should instead use
:confval:`untyped_call_exception`, for example:

.. code-block:: ini

[mypy]
disallow_untyped_calls = True
untyped_call_exception = some.library

.. confval:: untyped_call_exception
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems useful, but I'm not sure about the name of the option. Maybe we can come up with a better name? I don't any suggestions right now, but I'll think about this later.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also struggled some time with this. I would propose to not spend much time on this. People who want this will probably search for "mypy untyped call" or similar, so including these words in the option name should be good enough. So if you have a better idea, then I will be happy to update, otherwise let's just move on.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fwiw I'd go with untyped_call_allowlist. Clearly conveys the type as well ;-)

Copy link
Member Author

@ilevkivskyi ilevkivskyi Aug 11, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is what I considered at first, but then how wold we call corresponding command line option? --untyped-call-allowlist? All other options like --always-true, --always-false, --enable-error-code, --disable-error-code etc, are singular and should be repeated. So I would say we should try to follow this pattern. Finally, from reading comments in the issue, it looks like most people will need just 1 or maybe 2 exceptions, not more.


:type: comma-separated list of strings

Selectively excludes functions and methods defined in specific packages,
modules, and classes from action of :confval:`disallow_untyped_calls`.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mention that this also applies to submodules of packages (i.e. everything inside that prefix).

This also applies to all submodules of packages (i.e. everything inside
a given prefix). Note, this option does not support per-file configuration,
the exception list is defined globally for all your code.

.. confval:: disallow_untyped_defs

Expand Down
23 changes: 16 additions & 7 deletions mypy/checkexpr.py
Original file line number Diff line number Diff line change
Expand Up @@ -529,13 +529,6 @@ def visit_call_expr_inner(self, e: CallExpr, allow_none_return: bool = False) ->
callee_type = get_proper_type(
self.accept(e.callee, type_context, always_allow_any=True, is_callee=True)
)
if (
self.chk.options.disallow_untyped_calls
and self.chk.in_checked_function()
and isinstance(callee_type, CallableType)
and callee_type.implicit
):
self.msg.untyped_function_call(callee_type, e)

# Figure out the full name of the callee for plugin lookup.
object_type = None
Expand All @@ -561,6 +554,22 @@ def visit_call_expr_inner(self, e: CallExpr, allow_none_return: bool = False) ->
):
member = e.callee.name
object_type = self.chk.lookup_type(e.callee.expr)

if (
self.chk.options.disallow_untyped_calls
and self.chk.in_checked_function()
and isinstance(callee_type, CallableType)
and callee_type.implicit
):
if fullname is None and member is not None:
assert object_type is not None
fullname = self.method_fullname(object_type, member)
if not fullname or not any(
fullname == p or fullname.startswith(f"{p}.")
for p in self.chk.options.untyped_call_exception
):
self.msg.untyped_function_call(callee_type, e)

ret_type = self.check_call_expr_with_callee_type(
callee_type, e, fullname, object_type, member
)
Expand Down
18 changes: 18 additions & 0 deletions mypy/config_parser.py
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,20 @@ def validate_codes(codes: list[str]) -> list[str]:
return codes


def validate_package_allow_list(allow_list: list[str]) -> list[str]:
for p in allow_list:
msg = f"Invalid allow list entry: {p}"
if "*" in p:
raise argparse.ArgumentTypeError(
f"{msg} (entries are already prefixes so must not contain *)"
)
if "\\" in p or "/" in p:
raise argparse.ArgumentTypeError(
f"{msg} (entries must be packages like foo.bar not directories or files)"
)
return allow_list


def expand_path(path: str) -> str:
"""Expand the user home directory and any environment variables contained within
the provided path.
Expand Down Expand Up @@ -164,6 +178,9 @@ def split_commas(value: str) -> list[str]:
"plugins": lambda s: [p.strip() for p in split_commas(s)],
"always_true": lambda s: [p.strip() for p in split_commas(s)],
"always_false": lambda s: [p.strip() for p in split_commas(s)],
"untyped_call_exception": lambda s: validate_package_allow_list(
[p.strip() for p in split_commas(s)]
),
"enable_incomplete_feature": lambda s: [p.strip() for p in split_commas(s)],
"disable_error_code": lambda s: validate_codes([p.strip() for p in split_commas(s)]),
"enable_error_code": lambda s: validate_codes([p.strip() for p in split_commas(s)]),
Expand All @@ -187,6 +204,7 @@ def split_commas(value: str) -> list[str]:
"plugins": try_split,
"always_true": try_split,
"always_false": try_split,
"untyped_call_exception": lambda s: validate_package_allow_list(try_split(s)),
"enable_incomplete_feature": try_split,
"disable_error_code": lambda s: validate_codes(try_split(s)),
"enable_error_code": lambda s: validate_codes(try_split(s)),
Expand Down
17 changes: 16 additions & 1 deletion mypy/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,12 @@
from typing import IO, Any, Final, NoReturn, Sequence, TextIO

from mypy import build, defaults, state, util
from mypy.config_parser import get_config_module_names, parse_config_file, parse_version
from mypy.config_parser import (
get_config_module_names,
parse_config_file,
parse_version,
validate_package_allow_list,
)
from mypy.errorcodes import error_codes
from mypy.errors import CompileError
from mypy.find_sources import InvalidSourceList, create_source_list
Expand Down Expand Up @@ -675,6 +680,14 @@ def add_invertible_flag(
" from functions with type annotations",
group=untyped_group,
)
untyped_group.add_argument(
"--untyped-call-exception",
metavar="MODULE",
action="append",
default=[],
help="Disable --disallow-untyped-calls for functions/methods coming"
" from specific package, module, or class",
)
add_invertible_flag(
"--disallow-untyped-defs",
default=False,
Expand Down Expand Up @@ -1307,6 +1320,8 @@ def set_strict_flags() -> None:
% ", ".join(sorted(overlap))
)

validate_package_allow_list(options.untyped_call_exception)

# Process `--enable-error-code` and `--disable-error-code` flags
disabled_codes = set(options.disable_error_code)
enabled_codes = set(options.enable_error_code)
Expand Down
4 changes: 4 additions & 0 deletions mypy/options.py
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,10 @@ def __init__(self) -> None:
# Disallow calling untyped functions from typed ones
self.disallow_untyped_calls = False

# Always allow untyped calls for function coming from modules/packages
# in this list (each item effectively acts as a prefix match)
self.untyped_call_exception: list[str] = []

# Disallow defining untyped (or incompletely typed) functions
self.disallow_untyped_defs = False

Expand Down
55 changes: 55 additions & 0 deletions test-data/unit/check-flags.test
Original file line number Diff line number Diff line change
Expand Up @@ -2077,6 +2077,61 @@ y = 1
f(reveal_type(y)) # E: Call to untyped function "f" in typed context \
# N: Revealed type is "builtins.int"

[case testDisallowUntypedCallsAllowListFlags]
# flags: --disallow-untyped-calls --untyped-call-exception=foo --untyped-call-exception=bar.A
from foo import test_foo
from bar import A, B
from baz import test_baz
from foobar import bad

test_foo(42) # OK
test_baz(42) # E: Call to untyped function "test_baz" in typed context
bad(42) # E: Call to untyped function "bad" in typed context

a: A
b: B
a.meth() # OK
b.meth() # E: Call to untyped function "meth" in typed context
[file foo.py]
def test_foo(x): pass
[file foobar.py]
def bad(x): pass
[file bar.py]
class A:
def meth(self): pass
class B:
def meth(self): pass
[file baz.py]
def test_baz(x): pass

[case testDisallowUntypedCallsAllowListConfig]
# flags: --config-file tmp/mypy.ini
from foo import test_foo
from bar import A, B
from baz import test_baz

test_foo(42) # OK
test_baz(42) # E: Call to untyped function "test_baz" in typed context

a: A
b: B
a.meth() # OK
b.meth() # E: Call to untyped function "meth" in typed context
[file foo.py]
def test_foo(x): pass
[file bar.py]
class A:
def meth(self): pass
class B:
def meth(self): pass
[file baz.py]
def test_baz(x): pass

[file mypy.ini]
\[mypy]
disallow_untyped_calls = True
untyped_call_exception = foo, bar.A

[case testPerModuleErrorCodes]
# flags: --config-file tmp/mypy.ini
import tests.foo
Expand Down