ENOTDIR: not a directory, scandir '/opt/spark/python/lib/py4j-0.10.9-src.zip'

[pebabion]

Environment data

• Language Server version: 2021.3.0
• OS and version: linux x64
• Python version (and distribution if applicable, e.g. Anaconda):
• python.analysis.indexing: undefined
• python.analysis.typeCheckingMode: off

Expected behaviour

I have this error thrown back from Pylance and I have no understanding why it is happening.
In .bashrc, I do have this line (needed to run Spark).

export PYTHONPATH=$SPARK_HOME/python:$SPARK_HOME/python/lib/py4j-0.10.9-src.zip:$PYTHONPATH

Actual behaviour

Pylance is tell me that there is an error ENOTDIR.

Logs

Python Language Server Log

[Error - 2:56:42 pm] An internal error occurred while parsing file "/home/kelvin/projects/deployments/telkom/tests/test_impact_analysis.py": Error: ENOTDIR: not a directory, scandir '/opt/spark/python/lib/py4j-0.10.9-src.zip'
    at Object.readdirSync (fs.js:959:3)
    at Object.readdirSync (electron/js2c/asar_bundle.js:5:10231)
    at h.readdirEntriesSync (/home/kelvin/.vscode/extensions/ms-python.vscode-pylance-2021.3.0/dist/pyright-internal/src/common/fileSystem.ts:135:19)
    at t.PyrightFileSystem.processPartialStubPackages (/home/kelvin/.vscode/extensions/ms-python.vscode-pylance-2021.3.0/dist/pyright-internal/src/pyrightFileSystem.ts:150:46)
    at _0x419dcf._ensurePartialStubPackages (/home/kelvin/.vscode/extensions/ms-python.vscode-pylance-2021.3.0/dist/pyright-internal/src/analyzer/importResolver.ts:773:25)
    at _0x419dcf.resolveImport (/home/kelvin/.vscode/extensions/ms-python.vscode-pylance-2021.3.0/dist/pyright-internal/src/analyzer/importResolver.ts:119:14)
    at t.SourceFile._resolveImports (/home/kelvin/.vscode/extensions/ms-python.vscode-pylance-2021.3.0/dist/pyright-internal/src/analyzer/sourceFile.ts:1070:77)
    at callback (/home/kelvin/.vscode/extensions/ms-python.vscode-pylance-2021.3.0/dist/pyright-internal/src/analyzer/sourceFile.ts:556:47)
    at i.timeOperation (/home/kelvin/.vscode/extensions/ms-python.vscode-pylance-2021.3.0/dist/pyright-internal/src/common/timing.ts:40:20)
    at /home/kelvin/.vscode/extensions/ms-python.vscode-pylance-2021.3.0/dist/pyright-internal/src/analyzer/sourceFile.ts:555:48

Code Snippet / Additional information

XXX

[erictraut]

Thanks for the bug report. I see the cause of the error. It's a bug that was introduced in the last release.

@heejaechang, this is in the new partial stub virtual file system code. The function processPartialStubPackages is verifying that the path exists, but it's not verifying that it's a directory before calling readdirEntriesSync.

[erictraut]

This will be fixed in the next release.

[pebabion]

@erictraut thanks a lot for the quick response.

May I know when the next release is going to be?

In the meantime, I've reverted back to jedi to avoid the error

[erictraut]

We typically release a new version of Pylance each week on Wednesday.

[pebabion]

Many thanks!

[heejaechang]

@erictraut thank you for taking care of it. I wrongly assumed all paths given will be a directory.

[jakebailey]

Note that fix doesn't actually go and make zip files work; that'd be another enhancement to consider. We implemented it in MPLS (and likely via the VFS abstraction we can again; zip files are random accessable).

[jakebailey]

This issue has been fixed in version 2021.3.1, which we've just released. You can find the changelog here: https://github.com/microsoft/pylance-release/blob/main/CHANGELOG.md#202131-10-march-2021

If you need support for zip files specifically, we'd appreciate a new issue so we can take note of the enhancement request.