Improving and evolving stub package suggestions

[JukkaL]

Currently mypy suggests some stub packages from typeshed using a manually curated list. Example:

t.py:1: error: Library stubs not installed for "click" (or incompatible with Python 3.8)
t.py:1: note: Hint: "python3 -m pip install types-click"
t.py:1: note: (or run "mypy --install-types" to install all missing stub packages)
t.py:1: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports

These suggested packages are treated somewhat specially -- the suggestion is shown even when using --ignore-missing-imports. This means that it can be slightly tricky to ignore these missing stubs, in case the user actually doesn't want to install the type packages for some reason (e.g. it targets the wrong version of the package or has incorrect annotations). It's possible to ignore the errors, but it needs to done in a per-module section of the config file, instead of using the global flag. The reason for the special treatment is that we didn't want users of older mypy versions silently lose access to these stubs that used to be bundled with mypy when they update a more recent mypy version that doesn't bundle the stubs.

It would be nice to also suggest stub packages maintained outside typeshed, and to gradually add more stub packages to suggest as more stubs are added to typeshed. Since it's not quite trivial to avoid installing these stubs (because of the special handling discussed above), any new stub that has issues could cause a lot of friction. Either users may need to deal with false positives related to the stub, manually ignore errors using # type: ignore, add a special exception to their config file (which they may not know how to do), or contribute fixes to the stubs.

Note that we know that the friction with new stubs is real -- I've spend a lot of time fixing stub issues after syncing fresh stubs from typeshed. This friction actually was one the primary motivating factors for not bundling stubs with mypy. We don't want to force all users to deal with issues in all newly contributed or modified stubs.

I see a few different options for dealing with the above issue:

1. Any/some additional stubs being suggested wouldn't get the special treatment, and global --ignore-missing-imports would work with them.
2. Provide option to always suggest known stubs, even if errors about these stubs would be ignored otherwise.
3. Somehow ensure that the suggested stubs are mature enough so issues should be rare.

Option 1 is pretty straightforward. It has the drawback that users that use --ignore-missing-imports to hide errors about missing stubs won't see any additional suggestions, and they may not be aware of the existence of these new stubs, making the new suggestions less useful.

Option 2 would partially address this by providing an option to force suggesting all known stub files (e.g. --always-suggest-stubs). With this option we'd suggest stubs even if there is a per-module setting to ignore missing imports, and it would possibly also ignore # type: ignore comments targeting import statements. Users can decide whether they prefer easier mypy updates or better stub coverage.

Option 3 is the trickiest one. We could combine this with option 2 by first adding new suggestions as "beta" or "experimental" stubs that are easier to ignore and perhaps wouldn't be suggested unless the user opts in via --always-suggest-stubs or a similar option. After beta stubs have been included in suggestions for some time (e.g. 6 months) and they don't have major known outstanding issues, they could be moved to a set of "stable" stubs that are suggested by default (at least when not using --ignore-missing-imports). Eventually at least stubs for popular libraries could be treated as "core" stubs and get the special treatment where mypy would suggest them even if --ignore-missing-imports is used.

[Garrett-R]

The reason for the special treatment is that we didn't want users of older mypy versions silently lose access to these stubs that used to be bundled with mypy when they update a more recent mypy version that doesn't bundle the stubs.

I think that's a nice crutch for migrating, but is a poor experience for all newcomers to mypy, which over time will come to dominate the user share. Perhaps now that we're well beyond 0.9xx, we can remove this "migration aid" in favor of a more friendly long-term design.

It would be nice to also suggest stub packages maintained outside typeshed

Keep in mind that the current pattern of "suggesting" is super aggressive:

• report an error for not having the package installed.
• Then when the user tries to globally disable this error (because they don't want to see it anymore), ignore their wish, and continue reporting it as an error (for certain packages)

so if we continue with these "suggestions", I believe we want to limit the damage and not add even more stubs to this. So, in that sense, I vote for (1) above, but as mentioned before, I think the special treatment should be removed altogether.

It has the drawback that users that use --ignore-missing-imports to hide errors about missing stubs won't see any additional suggestions

I don't see that as a downside, that's precisely the expected behavior. One idea is to add to the docs for --ignore-missing-imports a warning that they could miss lots of suggestions.

and it would possibly also ignore # type: ignore comments targeting import statements.

This doesn't sound great, I don't think Mypy should ever ignore an ignore.