Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use modular typeshed (unbundle third-party package stubs) #9973

Merged
merged 50 commits into from
Jan 26, 2021

Conversation

JukkaL
Copy link
Collaborator

@JukkaL JukkaL commented Jan 26, 2021

Support the new structure of typeshed (python/typeshed#2491) and only
bundle stdlib stubs with mypy.

Most stubs for third-party packages now need to be installed using pip (for
example, pip install types-requests). Add stubs for typing_extensions
and mypy_extensions as mypy dependencies, since these are
needed for basic operation.

Suggest a pip command to run if we encounter known missing stubs.

Add --install-types option that installs all missing stub packages. This
can be used as mypy --install-types to install missing stub packages from
the previous mypy run (no need to provide a full mypy command line).

This also replaces the typeshed git submodule with a partial copy of the
typeshed that only includes stdlib stubs.

Add a script to sync stubs from typeshed (misc/sync-typeshed.py).
This is still incomplete since typeshed hasn't actually migrated to the new
structure yet.

Work towards #9971.

Copy link
Member

@ilevkivskyi ilevkivskyi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. thanks!


.. note::

Starting in mypy 0.800, most third-party package stubs must be
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume you mean 0.900 here too.


.. note::

This is new in mypy 0.800. Previous mypy versions included a
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume you mean 0.900 here.

JukkaL added a commit that referenced this pull request Jan 26, 2021
I'll add mypy/typeshed back in another commit as a copy of a
typeshed subset.

This is preparations for #9971 and #9973.
JukkaL added 28 commits January 26, 2021 16:53
It can used in a normal mypy run (missing stub packages will be installed
at the end). We also record the missing stub packages in a file so that
another "mypy --install-types" run (with no other arguments) can be used
to just install stub packages, without type checking anything.
They are actually 3rd party modules, but they are special like some
stdlib modules so we bundle them with mypy.
This avoids picking up the backport of dataclasses when type
checking in Python 3.7 mode. It will still be picked up in
Python 3.6 mode.
This is currently needed for six.
@JukkaL
Copy link
Collaborator Author

JukkaL commented Jan 26, 2021

I rebased and force pushed in the hope of fixing the brokenness.

@JukkaL
Copy link
Collaborator Author

JukkaL commented Jan 26, 2021

I'm going to ignore the mypy primer failures for now. If they persist, I'll disable mypy primer temporarily.

@emmatyping
Copy link
Collaborator

This also replaces the typeshed git submodule with a partial copy of the
typeshed that only includes stdlib stubs.

Is the idea here that we don't want to include all of typeshed for releases? If that is the case we can keep the submodule and only include the stdlib folder in typeshed I believe (which would be a bit easier I think).

E.g. If we change this line to point to the stdlib folder of typeshed only that will be included on install.

@JukkaL
Copy link
Collaborator Author

JukkaL commented Jan 26, 2021

Is the idea here that we don't want to include all of typeshed for releases?

That's part of it, but it's not the only thing. Now that we only need the stdlib stubs, the size is small enough that we can reasonably just commit them all in the repo. This lets us drop the submodule, which simplifies workflows a little -- especially for new contributors who might have no previous experience with git submodules.

Also, if modular typeshed is successful, we'll eventually have 10x the current number of third-party stubs, and having so many stubs (that we don't need) inside the mypy repo, even as a submodule, could be awkward.

@JukkaL JukkaL merged commit 0e7b867 into master Jan 26, 2021
@JukkaL JukkaL deleted the modular-typeshed branch January 26, 2021 19:05
ilevkivskyi added a commit that referenced this pull request Jan 26, 2021
This is a follow up for #9973

Co-authored-by: Ivan Levkivskyi <ilevkivskyi@dropbox.com>
JukkaL added a commit that referenced this pull request Jan 27, 2021
These are no longer relevant after we removed the typeshed
submodule in #9973.
JukkaL added a commit that referenced this pull request Jan 27, 2021
These are no longer relevant after we removed the typeshed
submodule in #9973.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants