Use with Pandas?

[dlozeve]

Hello, I recently started to use Pyright on my projects, where I intensively use Pandas. If I understand correctly, Pandas does not have type stubs, so I get constant false positive errors like this:

13: Pyright: "Grouper" is not a known member of module
20: Pyright: "__setitem__" method not defined on type "Series[Unknown]"
24: Pyright: "Timestamp" is not a known member of module

Is there a solution for this? Can I fix somehow the type checking for Pandas, or at least disable type checking for Pandas entirely (but not for the rest of the code)?

Thanks in advance!

[erictraut]

The pandas maintainers are working on incorporating types in an upcoming version of the library. Until then, you will need to do one of the following:

1. If you're primarily interested in completion suggestions, you can download and install a work-in-progress set of type stubs from this site.
2. You can disable the useLibraryCodeForTypes setting (which I presume you've enabled). This will tell pyright not to attempt to infer types from library sources (a common source of false positives) and treat all types as "unknown" (Any) when a type stub is not present. If your intent is to do type checking, this may be your only feasible option at this point. It will eliminate the false positives, but it will also prevent the type checker from checking any types that come from those libraries.

[dlozeve]

Thank you for your answer! I have disabled useLibraryCodeForTypes, and indeed all variables look like they have type Unknown (which is good for me, until we get proper type information).

However, I still get errors like "xxx" is not a known member of module when I use various things like pd.Grouper or pd.Timestamp. Note that this does not happen with other classes like pd.DataFrame. Is this an issue on Pandas side, or am I doing something wrong? Maybe these classes are exported differently than DataFrame, and Pyright doesn't know about them?

Actually, mypy returns the same errors, so I think Pyright behaves as expected here. Do you have any pointers on what might be happening?

[erictraut]

Here's the tracking issue in the pandas repo: pandas-dev/pandas#28142.

The "xxx" is not a known member of module error message implies that some type stub file is present. If you click on pd and hit F12, it will take you to the source of that type information. Is it a ".pyi" file?

[dlozeve]

Sorry, I should have mentioned this earlier, but I do not use VS Code, I'm in lsp-mode for Emacs. However if I understand correctly, F12 is "Go to definition", and when I do that in Emacs it tells me:

LSP :: Not found for: Grouper

(the exact same error I get for an undefined variable)

Note that I tested the same file in VS Code with the (poprietary) Pylance, and I get no errors. Is there a difference between Pyright and Pylance here? VS Code still has no type information (everything is Any), and pressing F12 on pd.Grouper just says "No definition found", which is the expected outcome I guess.

[erictraut]

Pylance is built on top of Pyright. Pylance chooses different default settings in a couple of cases, but other than that it should act the same as Pyright. Pylance also distributes some additional stub files (including partial stubs for pandas) that are not packaged with Pyright, but you can download these stubs yourself from this repo.

You can also uninstall Pylance and install the Pyright VS Code extension. That would be a better apples-to-apples comparison with the LSP you're running.

I'm interested in the type definition for pd, not Grouper. Could you "Go to definition" on that symbol?

[dlozeve]

Thanks! Indeed this solved my issue: with Pyright (not Pylance) in VS Code, the type definition of pd came from a pandas-stubs folder that belonged to the data-science-types package, that I installed a long time ago to fix a similar issue. Once I uninstalled it, the error disappeared.

Now there is no type checking whatsoever for Pandas, but I don't have any false positives. Thanks a lot for your help, it was great to have someone explaining it in detail!