Project Stubs in ./typings are not used

[CrazyIvan359]

Environment data

• Language Server version: v2021.11.1
• OS and version: Ubuntu 20.04.3 LTS
• Python version (& distribution if applicable, e.g. Anaconda): 3.8.10

Expected behaviour

Pylance uses project stub files for libraries.

Actual behaviour

Pylance does not appear to be using stubs for external libraries in ./typings. I have a project using some external libraries that are either not typed or not sufficiently typed to make strict type checking happy so I have typed the functions that I use to get accurate typing in my project. These stubs used to be used by pylance for type checking and auto-completion but are no longer are. I can't tell you an exact version that they used to work in but It was definitely working this time last year.

I have looked in the logs and see that pylance is finding the stubs folder but doesn't seem to be using them as I am seeing errors saying these functions are not typed. I have tried both settings of useLibraryCodeForTypes with no change in behaviour. Best guess is my own stubs are being ignored for some reason.

May be related to #1197

Code Snippet / Additional information

    handler_stdout = logging.StreamHandler()
    handler_stdout.setFormatter(
        colorlog.ColoredFormatter(
            _LOG_FORMAT_TERM,
            log_colors=_LOG_COLORS,
            secondary_log_colors=_LOG_SECONDARY_COLORS,
        )
    )

The above colorlog.ColoredFormatter instantiation comes up with the following errors from pylance. The library is not typed but you will find below my stub for it that was previously working.

• Type of "ColoredFormatter" is unknown Pylance(reportUnknownMemberType)
• "ColoredFormatter" is not a known member of module Pylance(reportGeneralTypeIssues)

This is my stub for ColoredFormatter which is the only function in this library I'm using:

import typing as t
import logging

class ColoredFormatter(logging.Formatter):
    def __init__(
        self,
        fmt: t.Optional[str] = ...,
        datefmt: t.Optional[str] = ...,
        style: t.Optional[str] = ...,
        log_colors: t.Optional[t.Dict[str, str]] = ...,
        reset: t.Optional[bool] = ...,
        secondary_log_colors: t.Optional[t.Dict[str, t.Dict[str, str]]] = ...,
    ) -> None: ...

[erictraut]

We make heavy use of internal type stubs, so I don't think this is broken.

We'll need additional details to help diagnose the problem you're seeing.

• What is the directory/folder structure within your typings director?
• Have you specified a value for the stubsPath configuration option to override the default location of ./typings?
• Is the package a "py.typed" library? In other words, does it contain a "py.typed" file in its root directory indicating that it has its own type information?

[judej]

Thanks for the report. Could you please add the information in the report as described in the troubleshooting guide?

[CrazyIvan359]

@erictraut thanks Eric,

Relevant folder structure looks like this:

root
 |- project-src
 |   |- pyrightconfig.json
 |- typings
 |   |- colorlog
 |       |- __init__.pyi
 |       |- colorlog.pyi
 |- project.code-workspace
 |- pyrightconfig.json

I've tried specifying stubPath: "./typings" in ./pyrightconfig.json and in the workspace file but no change. Previously it was specified in ./project-src/pyrightconfig.json as "./../typings" and I have also tried removing that to no avail.

No py.typed file in the library.

@judej what information did I miss from the guide? At this point I don't know what events I need to trigger to generate logs showing the issue so it seemed unhelpful to include that, otherwise I think I've included everything?

[erictraut]

From your directory layout, it looks like there's a package called colorlog, and it has a submodule that is also called colorlog. Is that correct?

What are the contents of typings/colorlog/__init__.pyi? Is that where you declare the ColoredFormatter class? Or is that class declared within the colorlog submodule? If it's in the submodule, do you re-export it from the top-level module (i.e. from within the __init__.pyi file) using a statement like from .colorlog import ColoredFormatter as ColoredFormatter or from .colorlog import *?

If you don't re-export it from the top-level module, then you'll need to import it from the submodule using a statement like from colorlog.colorlog import ColoredFormatter.

[CrazyIvan359]

All assumptions are correct. __init__.pyi re-exports the ColoredFormatter class that is defined in colorlog/colorlog.pyi and I confirmed it is using the correct import path.

[erictraut]

How is __init__.pyi indicating that the symbol should be re-exported? Is it using one of the two forms I mentioned above?

[CrazyIvan359]

This is all that's in __init__.pyi:

from colorlog.colorlog import ColoredFormatter

[erictraut]

Change that to from colorlog.colorlog import ColoredFormatter as ColoredFormatter, and it should work. PEP 484 indicates that symbols imported by a type stub shouldn't be considered re-exported unless they use a redundant alias form.

Modules and variables imported into the stub are not considered exported from the stub unless the import uses the import ... as ... form or the equivalent from ... import ... as ... form. (UPDATE: To clarify, the intention here is that only names imported using the form X as X will be exported, i.e. the name before and after as must be the same.)

[heejaechang]

I just tested and it works with x as x form

[CrazyIvan359]

@erictraut amazing! It is working again, thank you for helping me get it figured out.

Was pylance/pyright changed to adhere to this part of PEP 484? As I said originally, these stubs used to work without the X as X imports.

[erictraut]

Yes, a bug was fixed several months ago that addressed non-conformance with PEP 484.

[CrazyIvan359]

There it is. I'm glad I'm not crazy, it was starting to bug me that I couldn't get it working when it had been before.

Thanks again for the help