Intellisense is not active on python scripts without .py extension

[fevb]

Environment data

• Language Server version: 2020.12.2
• OS and version: Linux Ubuntu 18.04
• Python version: 2.7.17

Expected behaviour

Open a python script without a .py extension, e.g. run_test, intellisense should work (code navigation, autocompletion, etc).

Actual behaviour

• For the script (e.g. run_test) pylance intellisense does not work, even though it does get recognized by VSCode as a python script:
  

Additional information

• If I rename the file to run_test.py, then intellisense does work as expected.

• If I downgrade pylance to version 2020.11.2, then intellisense does work as expected also for scripts without the .py extension. Seems like some regression occurred between versions 2020.11.2 and 2020.12.2.

[jcristine]

Same issue. Up to a few days ago intellisense worked fine as long defined/declared the file with standard python "shebang line": #!/usr/bin/env python3

Now it only works if the file has a ".py" extension. Using Pylance 2020.12.2

[erictraut]

It is intended that the language server only attempt to parse files ending in ".py" and ".pyi". Previously, there was a missing check. There are a few reasons why this limitation is needed. First, it's important for the language server not to attempt to parse some random file. We had a bug report of a crash (out of memory condition) that occurred because the language server was attempting to parse a large binary file. Second, there are semantic differences between ".py" and ".pyi" files, and the language server needs to be able to distinguish between the two based on the file extension.

[jakebailey]

Now that I think about this, I completely forgot that it's not a problem to be missing the py extension. It's very common for scripts installed in PATH to not have one. I think we should consider treating any file that isn't pyi as Python source, and maybe detect issues with binaries some other way.

[erictraut]

Yeah, that sounds reasonable. What mechanism is "recognizing that this is a Python file"? Is the core Python extension reading the contents and using some heuristics?

[jcristine]

What mechanism is "recognizing that this is a Python file"?

How about: with standard python "shebang line": #!/usr/bin/env python3

[jakebailey]

Shebang lines for python are not standardized to that extent, no. We couldn't key it off of that specifically (need /bin versions, /usr/bin, non-python3, etc).

I don't think there's a great heuristic other than to cap at a certain file size; if we're looking for something more strict, every python file will likely start with an identifier (keywords fit the same character set) or #, but I know VSC has some special code that determines if a file is a source file or not.

With the compiled code bug fixed, though, I'd wonder if we even need to check anymore.

[jakebailey]

Another option is to always accept a file if it came from a user opening a file, but ban non py/pyi files stemming from imports. There's no possible way for a non-py/pyi file to be brought in via an import. If a user opens a 2GB binary file and has reclassified it as python, well, then that's on them. 🙂

[jakebailey]

For reference VSC does this to check the encoding: https://github.com/microsoft/vscode/blob/f74e473238aca7b79c08be761d99a0232838ca4c/src/vs/workbench/services/textfile/common/encoding.ts#L382

Note the "seems binary". As for detecting python itself, I believe it uses extension only, plus user overrides via config files or manual intervention. The only other auto-detection is when you copy and paste Python from a python file into an untitled file, but in that case it knows that you copied from Python and to make the new buffer Python too.

[jakebailey]

See also (in memory untitled files, which are extensionless): #737

[jcristine]

I changed the python language server to "Jedi" and all of the (many!) python scripts that I have work [intellisense] with it. Whatever they are doing may be an option? I'm not on MS Win so extensions seem redundant not to mention the body of code that I already have with (.py).

[jakebailey]

Yes, that option #739 (comment), to treat anything we're given as Python.

[rmccampbell]

This also doesn't recognize .pyw extensions. This should be easily fixable even if more general file extensions are not supported.

[rmccampbell]

Shebang lines for python are not standardized to that extent, no. We couldn't key it off of that specifically (need /bin versions, /usr/bin, non-python3, etc).

For the record the Python Launcher for Windows parses unix shebang lines to determine what python version to run. This isn't used to determine if the file is python in the first place however.

[jakebailey]

Per #795, this should be fixed in the next release.

microsoft/pyright@c03e8c3

[jakebailey]

This issue has been fixed in version 2021.1.0, which we've just released. You can find the changelog here: https://github.com/microsoft/pylance-release/blob/main/CHANGELOG.md#202110-6-january-2021

[DevilXD]

It appears that pyw files are still not recognized properly. I'm on v2021.12.2 using the VSCode extension, and changing main.py to main.pyw results in this error in another file: https://i.imgur.com/yF1yCsh.png

There was an issue about this, but it got closed and redirected to here: #639