-
-
Notifications
You must be signed in to change notification settings - Fork 2.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Test stub files against real code #5028
Comments
This is not a new idea. Do you also have a plan for implementing this, or are you just requesting that the mypy core team takes this on? |
@gvanrossum I'm aware of this. I've chatted earlier today with some mypy core team members. Actually the plan described here is something we mostly came up together with @JukkaL. I do plan to implement this, or at least help with this (hopefully get the ball rolling during the sprints next week). That being said created the issue to discuss the general approach before anyone starts coding. I did not found a ticket dedicated existing, so created a new one. |
I've got an initial proof of concept version of this working (gaborbernat@a56f007). Will give it a test run trying to use it to type hint the https://github.com/tox-dev/tox code base, which contains a lot more edge cases I'm sure it's not handling yet. And fix the issues coming up. Once manage to do that I'll create the PR for us to review/discuss it (probably will take a week or so). |
Are there any updates on this? As someone using stub files exclusively, this is very unintuitive behavior (and hard to realize/easy to forget). It also makes type checking |
@gaborbernat are you taking your proof of concept further? It would be extremely useful. |
Yeah it's a work in progress. Sadly it's quite complicated to get it right and touched a few code places that were modified in the meantime ( so needed to rebase and resolve merge conflicts like three times by now), and my availability was also fluctuating. Long story short we agreed on design and implementation. Will probably be ready for merge by middle of April. Definitely commited to get it stable by PyCon Us in early May. |
So just an update on this. We have the ability to merge the stub into the source code via https://github.com/ambv/retype. The Issues with this approach are that you lose context of what content comes from the source and what content comes from the stub. mypy errors are hard to fix as errors are reported on the merged source information; mapping back to the original files can be troublesome. retype also fails at the first conflict so requires a lot of run-fix-run cycles. The plan going ahead is to make retype also report some line mappings (allowing at this point to generate more localized errors). Also, retype should run and report all errors rather than stop at first. The later issue is of a mediocre hard. The first is hard though. The problem is that at the moment retype parses stubs via the |
imho with python 2.7 and 3.5 EOL around the corner the use-case for external stubs seems to diminish. Already people are choosing to put their stubs separate from their code because they don't like the "aesthetic" of type annotations. In my opinion that makes this ticket describe an anti-feature, and just provides these people with ammunition to avoid applying types directly to their code. |
Well, I use stubs with python3.7 because sometimes stubs are really ugly. And I am happy that they live outside of the source code. Example: |
@graingert I don't quite understand your comment. At first it sounds like, with the introductions of annotations we don't need stubs anymore (I disagree), then it sounds like you actively want to prevent people from using type stubs by making it harder, thus this would be an anti-feature. Could you clarify? Type stubs have fallen out of favor with me, but especially for legacy code and where I currently work there is a lot of push back for in-code annotations and a solid type stub option would surely help getting their feet wet. |
It is my opinion that mypy should focus on fostering community support for inline type annotations as the default and preferred option |
Fair enough, but I don't think treating a feature, which has non-equivalent uses as a red headed step child is a good way to achieve that. It might just split the adoption of type hints in the community in general. Unless type stubs were always just meant as a holdover. But they seem way too useful and in some cases unavoidable for that. |
Stub files are necessary in cases where the libraries you are using do not have annotations and the stub library for them makes a type generic. Since the subscript operator isn't defined normally you can't inline the annotations all the time. Encountered this with the django stubs. |
@JukkaL that doesn't work for inheriting though. The specific case here is roughly you have a |
@ruler501 Yeah, inheritance is tricky. You can use this ugly workaround, however: from typing import TYPE_CHECKING
from foo import Field
if TYPE_CHECKING:
Base = Field[str]
else:
Base = Field
class Derived(Base):
... |
I'm thinking about augmenting stdlib ast module to have some lib2to3 features (because llibib2to3 is going away). If this is of interest to you: |
Note that mypy now ships with a tool called stubtest, which will compare stubs to what it gets from importing and inspecting the module at runtime. |
Is it documented? I tried it but I get |
It's not currently documented. (update: stubtest is documented here https://mypy.readthedocs.io/en/latest/stubtest.html) |
I just added type annotations to Werkzeug. Because inheriting from Also, I was tempted to use stub files everywhere, since inline annotations can get quite messy for complex code bases, but not being able to check the code itself made that not possible. |
For avoiding subclassing Generics specifically, you could try using an Yeah, stubtest is meant as a test for stubs, not for code :-) |
@graingert just pointed me at this ticket as part of adding type hints support to
|
Yup! To summarise the state of this issue: Checking stub files based on real code: stubtest does a decent job of this (it imports the module and uses runtime introspection). It's worked pretty well for typeshed. |
@hauntsaninja out of interest why didn't it catch the incorrect type of contextlib.nullcontext in typeshed? |
It did, there's just a backlog of allowlisted errors: https://github.com/python/typeshed/blob/50f5858a5628529b7a453c675e0f4b1eeccc704d/tests/stubtest_allowlists/py37.txt#L29 |
Ah awesome, thanks |
very new to typing and Python both but was just wondering if it is possible to integrate directly or as a plugin which transfers the type annotations from pyi files to source code using code transformations from something like LibCST and mypy checks for type checking in source in the next step? Of course, the solution in the description seems much better (for correctness reasons) but could this be used as a workaround in the meantime? |
Yup, |
There are a few use cases when stubs may be compelling over type annotations or type comments:
However at the moment this has some limitations: stub files are used only to check clients of the stub, not the code itself.
In case of the above files mypy will warn about bad usage of inside
client.py
, but not ina.py
(that is, mypy does not validate against the source files being annotated by the stub). This has serious drawbacks:Here I propose to implement a way to support testing stub files against the real code:
Merging the AST definitely is not trivial. The plan is to start with a POC implementation, that works for the most cases, and then see the viability of this approach. Note this would help a lot both stub maintainers I think (typeshed) and projects that need Python 2 support.
The text was updated successfully, but these errors were encountered: