-
Notifications
You must be signed in to change notification settings - Fork 767
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Go to Symbol in Workspace fails with Heap out of memory and Exclusion patterns are not respected #228
Comments
Workspace symbols are currently being improved (#34), performance included. Note that if you mean What does this "Python > Workspace symbols: Exclusion Patterns" setting look like in the settings JSON file and what is its description? I'm pretty sure that this setting is only intended to control the Python extension's |
Says: "Pattern used to exclude files and folders from ctags See http://ctags.sourceforge.net/ctags.html." Pylance is a workspace symbol provider, but doesn't use ctags, so this option would have no effect. Looking more closely at your comment about it looking into |
And is there any option to exclude files from Go to Symbol in Workspace? I tried to put
as suggested here but Pylance still parsed site packages. But it is probably expected as the Pyright docs mentions
But the main problem is still the OOM error. I don't consider parsing site packages overzealous. E.g. PyCharm parses them too and it is useful to be able to search symbols also in site packages. I'd like to exclude them only temporarily so it is working at least for my project files until the memory issues are resolved. |
Parsing isn't overzealous, no, it's running various parts of the analysis that could be, and we need to a lot in order to correctly determine the types of some symbols. Right now, we effectively query the same analysis as everything else to get symbols and their types. If possible, it would be helpful to enable trace logging to get an idea if there's a specific file which is causing the overflow, as there could be a bug. Note that this is still workspace symbol search, which has a specific meaning within the LSP and is used by various clients to do things like test discovery and such, so we don't want to overload it to mean something it shouldn't. |
It failed with OOM during parsing But every time I use the workspace symbol search, there is cca 3s delay. E.g. I open symbol search and I type a few characters, then there is 3s delay, it shows symbols, then I continue with typing, again 3s delay, then it updates the symbols. Also it would be useful to be able to select which type of symbols should be searched, e.g. only classes, methods and functions. Similarly as it can be set e.g. in Outline settings. |
Did you run with trace logging such that we could see when it's running out of memory? Workspace symbol hanging I expect, but regular analysis I do not. "python.analysis.logLevel": "Trace" |
This time it failed on
|
We have mechanism in place that watches the heap size and empties our internal type cache if we are at risk of overflowing. However, this mechanism was not being invoked during a workspace symbol search, which explains the problem you're seeing. I've introduced a fix, and it will be in the next release. Thanks again for the bug report. |
I'm going to mark this as fixed in the next release; the overflow shouldn't happen anymore and further improvements to the workspace symbols are being tracked in #34 (including making it faster and not over-scan). |
This issue has been fixed in version 2020.8.1, which we've just released. You can find the changelog here: https://github.com/microsoft/pylance-release/blob/master/CHANGELOG.md#202081-13-august-2020 |
Unfortunately, I got the same error also with 2020.8.1:
|
Hm, that's shame. It's curious that the parse for that |
Apologies for not addressing your issue with this week's release. I think I understand why my initial fix didn't address it, and I've taken another run at the problem. I've added the cache high watermark detection in a few more places to cover the case that you are hitting. I'm not able to repro your problem, so I can't verify it directly, but I think this will prevent it. |
E.g. |
To replicate the issue it should be sufficient to use many packages in the environment. Here is example that also fails with OOM.
|
Thanks for the example. I'm not able to repro an out-of-heap-space error with the latest code. |
This issue has been fixed in version 2020.8.2, which we've just released. You can find the changelog here: https://github.com/microsoft/pylance-release/blob/master/CHANGELOG.md#202082-20-august-2020 |
Environment data
python:3.7.8-slim-buster
base imageExpected behaviour
Python > Workspace symbols: Exclusion Patterns
are ignored in Go to Symbol in Workspace.Actual behaviour
JavaScript heap out of memory
error./usr/local/lib/python3.7/site-packages
folder when there is**/site-packages/**
inPython > Workspace symbols: Exclusion Patterns
.Logs
The text was updated successfully, but these errors were encountered: