Intellisense stops working for python files when using the new 'Outline' feature

[derekwallace]

Environment data

• VS Code version: 1.24 insiders
• Extension version (available under the Extensions sidebar): latest
• OS and version: Win10
• Python version (& distribution if applicable, e.g. Anaconda): 3.6.5 - standard distribution
• Type of virtual environment used (N/A | venv | virtualenv | conda | ...): N/A
• Relevant/affected Python packages and their versions: XXX

Actual behavior

using network drive as project is hosted on a shared drive.
intellisense becomes un-responsive after a few python files are opened.
snippets also stops working
intellisense will work fine for other file types.

using process explorer i kill "python completion.py".
intellisense becomes responsive and works correct.

Expected behavior

intellisense always work.

Steps to reproduce:

[derekwallace]

initially things seem to work fine.
The new Outline also works.
The after a few minutes it all stops working.
The new OUTLINE does not refresh when you select a different file.

• selecting a new file should at a minimum clear the OUTLINE. but it stays with the value from some previous file which is very misleading.

killing the python completion.py sometimes works...

[DonJayamanne]

• Please check if there are any errors logged in the Python output window (View->Output, select Python from the dropdown)
• Please check if there are any errors logged in the console window Help-> Toggle Developer Tools
• Are you using any specific packages such as numpy, pandas or the like or any other 3rd party SDK (python packages)?

[alete89]

Can this be related with this question I did in SO? If this comment violate de rules, I'll remove. Thanks.

[derekwallace]

In the Developer Tools - Console i am seeing this. All my Extensions are no longer listed

In the View Output , Python is not listed

I need to restart COde now due to the 1st error seems to periodically appear as a notification.

Its a large project with N x individual python applications.
1 does use numpy (installed on Win10) and matplotlib (not installed on Win10)
Im not using a 3rd party SDK. I do have Atom, Sublime, PyCharm installed, WIngIDE (all part of an attemtp to find a good editor for Python development).

Im using VSCode on Windows. Im not using it for debug. just as a code editor (code complete, symbol Outline , highlighting etc..). All the python is executed on linux and hence i have a network drive to make the linux shared disk visible on windows.

[derekwallace]

As a further note.

1. ive Python linting disabled (in case its interferring with performance). ultimatly i want to turn this on.
2. In my workspace ive setup the Code to ignore some folders from Explorer. This is in theory to minimize the number of files.
3. the workspace has < 10K files, of which 1346 are python

thx
Derek

[derekwallace]

Does Jedi leave a DB file behind.
is it something you can inspect.

What i also notice is that the Outline in insiders also stops working.
It is like it gets "stuck" and no longer updates.
I assume this is implemented via Jedi

If i close all the open editors, open a new file, then the outline for the newly opened file is identical from to when it stopped working.

In following diagram you can see the open file is ccconfig_mod.py.
the outline is showing outline for a different file.

[joristaglio]

This is also happening to me, and it's incredibly frustrating. VScode-Python kills all intellisense for me.

[derekwallace]

A better solution to get intellisense back working is to just disable the extension Python.

With it disabled, intellisense seems to be working.
My code snippets are working.
Colour coding of Python still seems to work.

Obviously now ive lost

1. Linting
2. Outline

Extremely disappointing that it has come to this.
i want to use VSCode. I can live without the linting. but the Outline for symbols in the current file is so useful. at a glance you see all the methods.
i cant understnad why it cant work simply. Ive come from Notepad++ and it just works.

[joristaglio]

That's where I'm at as well. It's incredibly frustrating to have something so critical not work on the canonical extension for python language support in VSCode.

This happens for me regardless of virtual environment, python language, distribution, etc.

[patrys]

@joristaglio Please respect the devs, they're doing great work you're not paying for and they only have so many hands.

If you want to have this resolved earlier, contribute your time and help the Microsoft team reproduce this issue. I'm using Python in my everyday work yet I never managed to stumble upon this problem so it's likely that it only affects a small fraction of the community.

[derekwallace]

appreciate the devs and the effort.
im trying my best as well to convey the problem and provide as much information as i can to aid you guys. ive spent days trying to get this to work, to no avail.
what type of diagnostics can i do to help you out to debug the issue?

[MikhailArkhipov]

This depends on type of the project and source code. Completion process basically runs Jedi out of process. There is a shallow wrapper there over Jedi to handle out of process requests from the extension for the completion, tooltips etc. One way to try and repro (and maybe report directly in Jedi GH) is to import Jedi (it is in Python) and write short code that asks Jedi to return completions for a given file and location. Just see if it works as is.

[MikhailArkhipov]

@derekwallace - no, Jedi does not appear to cache results of persist a database.

[derekwallace]

I installed Jedi on PC.
I created a simple .py script to read one of the files in my workspace on my p:
Here is the script (del.py)

sFile = r'P:\perforce_Playground\derekw_de02-1777_rce_dev\tools\iipregr\bin\iipregrpkg\cascript_mod.py'
with open(sFile, "r") as fh:
   s = fh.read()

script = jedi.Script(source=s, line=50, column=1, path='MYFILE')
print(script.goto_definitions())

names = jedi.names(source=s)
for x in names:
   print(x)

I ran it locally outside VSCode in a Command shell.
Took about 2 sec to run

C:\derek\Python\Jedi>python del.py
Example2
[]
<Definition module collections>
<Definition module getpass>
<Definition module logging>
<Definition module os>

In VSCcde, i tried to run via "Run Python in terminal"
And it behaved the same.

For the record, here is a screen shot of del.py in VSCode and the outline is frozen with content from launch.json.

Close VSCOde, reopen, and you can see that Outline has del.py

Select cascript_mod.py (which i analyzed with del.py and Jedi).
Its now frozen and doesnt update.

at this point intellisense etc. is dead.

Is it in ANY way related to ripgrep running ?
Could this "block" other things running ....

[patrys]

I get similar reports in patrys/vscode-code-outline#134 (this is older than the built-in Outline mentioned above but works in the same way) which suggests that over time the SymbolProvider stops working.

[derekwallace]

I decided to create a brand new workspace (on network p: drive).
it consists of about 30 .py files. So small. This eliminates my concerns re size of my project workspace.

I have the same issue with Outline and symbols. Initially work for a few mins and then stop.
Never refresh. Intellisense stops working.

As a note, i have another issue open wrt performance of the Debugger. See #1744
I have this resolved and it came down to something in the launcher.json file .

I then created exact same workspace on c:
And outline, intelli all work fine.

[brettcannon]

@derekwallace We've had other reported issues with network drives. Could you provide the sample files that triggers the issue for you?

[derekwallace]

HI,
Ive created a small workspace. with 4 python files and a .yml file.
They were originally my project files, but ive trimmed them back to almost nothing, and removed all meaningful names etc. The files are not functional, but have imports, class , methods etc.

Attached the workspace
attached the workspace file
attached the user settings file
a_vscode_project_p.zip
settings.zip
a_vscode_project.zip

below , when i use the word "freeze" it means

1. Outline stops updating for Python files
2. Intellisense stops working in Python files

Here is what i do to trigger the issue.

1. Open the workspace

2. Open all 5 files

3. In, Open Editors select the files one by one randomly ...
   Initially the Outline updates for each file. can take 1-2 sec, can take 5 sec.
   Occasionally this alone can trigger the "freeze". But in this very small design it seems to rarely cause it.

4. now i start to edit some files.
   i normally go to a file and type "def" to see will intellisense pop up.
   I sometimes have to jump to one or more files and in each type "def".
   i might save file, i might not.
   eventually it will freeze (1-2 mins)

I cant find a specific pattern that causes it. but i can always get it to happen.

5. at this point selecting different .py files in Open Editors will not refresh the Outline
   intellisense is frozen

6. if i select the wal.yml file, the outline will refresh and show this docs outline
   if i select any .py file the outline does not update.

7. while doing the above repeatedly (i.e. close workspace, open workspace, and repeat)..... ive noticed that the outline can sometimes re-start working.
   The intellisense rarely restarts (if ever)

I hope this is of use.

Derek

[brettcannon]

I just realized something: where is that outline coming from? Do you have another extension installed to show that? If so can you disable all extensions but ours and see if this is still a problem?

[joristaglio]

@brettcannon I removed all extensions and the behavior continued with only the Python extension– and the behavior was not also triggered by any of my other extensions if I enabled them each one at a time.

[patrys]

@brettcannon Both outline implementations (my extension and the one built into VSCode) call "executeSymbolProvider" to obtain a list of symbols and their ranges.

[derekwallace]

@brettcannon disabled all extensions. Same behaviour.

A further note. Disable the Python extension, reload, and now intellisense is working (without Python knowledge, and outline etc...). But at least i get some code completion and my snippets work.

Enable Python and reload. Outline starts working, and quickly it stops and intellisese becomes unresponsive.

Its clearly the Python extension.

[derekwallace]

@DonJayamanne i downloaded and installed the developer version of the python extension. Unfortunately the issue remains. Same/similar behavior as before.
I notice that the intellisense can be responsive, then it can start to become laggy to the point it becomes unresponsive. it very much feels like that as you type a set of changes overwhelms intellisense that it cannot keep up and hence becomes laggy.

Error message in console. This seems to happen when intellisense becomes unresponsive and outline completely stops refreshing.

ERR Cannot read property 'id' of undefined: TypeError: Cannot read property 'id' of undefined
   at responses.forEach (C:\Users\derekw\.vscode-insiders\extensions\ms-python.python-2018.5.0-rc\out\client\providers\jediProxy.js:393:54)
   at Array.forEach (native)
   at SafeSubscriber.result.out.subscribe.output [as _next] (C:\Users\derekw\.vscode-insiders\extensions\ms-python.python-2018.5.0-rc\out\client\providers\jediProxy.js:387:31)
   at SafeSubscriber.__tryOrUnsub (C:\Users\derekw\.vscode-insiders\extensions\ms-python.python-2018.5.0-rc\node_modules\rxjs\Subscriber.js:242:16)
   at SafeSubscriber.next (C:\Users\derekw\.vscode-insiders\extensions\ms-python.python-2018.5.0-rc\node_modules\rxjs\Subscriber.js:189:22)
   at Subscriber._next (C:\Users\derekw\.vscode-insiders\extensions\ms-python.python-2018.5.0-rc\node_modules\rxjs\Subscriber.js:129:26)
   at Subscriber.next (C:\Users\derekw\.vscode-insiders\extensions\ms-python.python-2018.5.0-rc\node_modules\rxjs\Subscriber.js:93:18)
   at sendOutput (C:\Users\derekw\.vscode-insiders\extensions\ms-python.python-2018.5.0-rc\out\client\common\process\proc.js:51:32)
   at Socket.on (C:\Users\derekw\.vscode-insiders\extensions\ms-python.python-2018.5.0-rc\out\client\common\process\proc.js:54:47)
   at emitOne (events.js:96:13)
   at Socket.emit (events.js:191:7)
   at readableAddChunk (_stream_readable.js:178:18)
   at Socket.Readable.push (_stream_readable.js:136:10)
   at Pipe.onread (net.js:560:20)

[derekwallace]

@DonJayamanne To be sure i re-tried and i saw the same behaviour. At the moment that intellisense became un-responsive the below error appeared in console.

ERR Cannot read property 'id' of undefined: TypeError: Cannot read property 'id' of undefined
	at responses.forEach (C:\Users\derekw\.vscode-insiders\extensions\ms-python.python-2018.5.0-rc\out\client\providers\jediProxy.js:393:54)
	at Array.forEach (native)
	at SafeSubscriber.result.out.subscribe.output [as _next] (C:\Users\derekw\.vscode-insiders\extensions\ms-python.python-2018.5.0-rc\out\client\providers\jediProxy.js:387:31)
	at SafeSubscriber.__tryOrUnsub (C:\Users\derekw\.vscode-insiders\extensions\ms-python.python-2018.5.0-rc\node_modules\rxjs\Subscriber.js:242:16)
	at SafeSubscriber.next (C:\Users\derekw\.vscode-insiders\extensions\ms-python.python-2018.5.0-rc\node_modules\rxjs\Subscriber.js:189:22)
	at Subscriber._next (C:\Users\derekw\.vscode-insiders\extensions\ms-python.python-2018.5.0-rc\node_modules\rxjs\Subscriber.js:129:26)
	at Subscriber.next (C:\Users\derekw\.vscode-insiders\extensions\ms-python.python-2018.5.0-rc\node_modules\rxjs\Subscriber.js:93:18)
	at sendOutput (C:\Users\derekw\.vscode-insiders\extensions\ms-python.python-2018.5.0-rc\out\client\common\process\proc.js:51:32)
	at Socket.on (C:\Users\derekw\.vscode-insiders\extensions\ms-python.python-2018.5.0-rc\out\client\common\process\proc.js:54:47)
	at emitOne (events.js:96:13)
	at Socket.emit (events.js:191:7)
	at readableAddChunk (_stream_readable.js:178:18)
	at Socket.Readable.push (_stream_readable.js:136:10)
	at Pipe.onread (net.js:560:20)

[DonJayamanne]

@derekwallace

• Please could you uninstall that extension
• Delete all of the ms-python.python-* directories
• Download and install the development version once again & test this out.

[derekwallace]

@DonJayamanne Uninstalled and deleted the folders. Installed .vsix and the same behavior. Also get lots of errors from parso as its still 0.2.0

Installed

[derekwallace]

saw this error while testing again.

[IPC Library: Watcher] Uncaught Exception:  Service shutdown unexpectedly
t.log @ console.ts:136
(anonymous) @ ipc.cp.ts:157
e.fire @ event.ts:140
i @ event.ts:535
emitTwo @ events.js:106
emit @ events.js:194
process.nextTick @ internal/child_process.js:766
_combinedTickCallback @ internal/process/next_tick.js:73
_tickCallback @ internal/process/next_tick.js:104

[derekwallace]

@DonJayamanne Why is the issue closed?

[brettcannon]

@derekwallace because we updated to Parso 0.2.1 in master which you can test in either our development build or wait until our 2018.6.0 release.

[derekwallace]

I tested the development build of the python extension and it doesn't resolve the issue. I was asked to test it and provide feedback . I have done this and report that the issue persists. I don't see how the issue can be closed.

…

On Tue, 5 Jun 2018, 21:56 Brett Cannon, ***@***.***> wrote: @derekwallace <https://github.com/derekwallace> because we updated to Parso 0.2.1 in master which you can test in either our development build or wait until our 2018.6.0 release. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1721 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFUOVpmTSFu74Ra3HosOeQM5f2qxqGHaks5t5vCUgaJpZM4UJPik> .

[derekwallace]

Is there any update? Did the dev build include more than the parso update? Did it have something to denounce the outline update.
Thx.

[DonJayamanne]

@derekwallace
Please understand that we are doing our best to resolve issues. Also, at this stage you seem to be the only user experiencing this issue.

I understand this can be quite frustrating for you as well, hence can I suggest you just turn off the Outline view for the moment.

Is there any update?

You can see all of the updates to this issue here, please scroll up and you'll have the answers to all of your questions.
Also VS Code have added some additional fixes at their end (see here microsoft/vscode#51096 (comment)).

Again all of this information you see is on this issue (including the links to the VS Code issue).

Once again, please consider turn off (hiding) the Outline view for the moment.

[derekwallace]

I appreciate the support trying to resolve the issues. My colleagues have the same issue. I'd be confident others do as well. I have the developer build of python extension with parso update. I looked at the vscode update. Is this in the insiders build? How do I know I have this update? I see from other issues that you seem to be working on another python analysis extension. Is there any early access to this that I could try? Thx. Derek.

…

On Wed, 6 Jun 2018, 19:11 Don Jayamanne, ***@***.***> wrote: @derekwallace <https://github.com/derekwallace> Please understand that we are doing our best to resolve issues. Also, at this stage you seem to be the only user experiencing this issue. I understand this can be quite frustrating for you as well, hence can I suggest you just turn off the Outline view for the moment. Is there any update? You can see all of the updates to this issue here, please scroll up and you'll have the answers to all of your questions. Also VS Code have added some additional fixes at their end (see here microsoft/vscode#51096 (comment) <microsoft/vscode#51096 (comment)> ). *Again all of this information you see are on this issue (including the links to the VS Code issue).* Once again, please consider turn off (hiding) the Outline view for the moment. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1721 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFUOVjGBTUyP3EJ583ixkC4BgR-h4se2ks5t6BtFgaJpZM4UJPik> .

[brettcannon]

The parso update is in the latest public release of 2018.5.0. If you are installing the development build of the extension then the version would be 2018.6.0-alpha.

The new analysis engine is available for testing if you know the hidden setting to switch it on, but I'm not ready to give that out for this instance as we have not tested it yet with the Outline view.

[brettcannon]

This seems to have been fixed as I can't reproduce at least with a checkout build.

[myron]

I don't think it should be closed.

With OUTLINE enabled, python code auto-complete takes 10-15 seconds to produce a suggestion, then even longer to show function/method input arguments. Disabling Outline, makes everything normal again.

this is for VCcode 1.24.0 on Ubuntu 16.04, with python 3.6 over Anaconda (with all the latest packages, including parso 0.2.1)

[derekwallace]

I don't think it should be closed either. The updates have improved behaviour. In general I've found that the micro test project both outline and intellisense work. However on my large network drive project the outline is unusable and when enabled makes the intellisense completely unresponsive. Disabling and intellisense is just about workable. Sometimes responsive, Sometimes it's laggy, So there is still something going on that is interfering with normal behaviour.

…

On Tue, 12 Jun 2018, 18:38 myron, ***@***.***> wrote: I don't think it should be closed. With OUTLINE enabled, python code auto-complete takes 10-15 seconds to produce a suggestion, then even longer to show function/method input arguments. Disabling Outline, makes everything normal again. this is for VCcode 1.24.0 on Ubuntu 16.04, with python 3.6 over Anaconda (with all the latest packages, including parso 0.2.1) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1721 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFUOVrpgY6mobqFcdkP1ba-hdWJbyTJkks5t8ELtgaJpZM4UJPik> .

[brettcannon]

"there is still something going on that is interfering with normal behaviour", and that behaviour is the outline view. 😉 VS Code is basically hammering Jedi every 250 ms to re-parse the file and analyze it for symbols. IOW there's not much we can do here short of throttling even more or magically making Jedi faster (also realize the Outline view is in preview for VS Code for a reason 😉; we actually had to provide the feedback to even have the 250 ms throttle added as originally it was every key press triggering a re-scan).

We are hoping to have the Outline view supported by our experimental analysis engine soon which should help alleviate this performance bottleneck. Once #1934 is merged and we have verified the new analysis engine works with the outline view I can provide instructions here on how to flip it on.

[derekwallace]

Thanks Brett for providing clear explanation of what is happening. Do you think it would be possible that the 250 could be a setting so that users that have an issue would be able to investigate finding a sweet spot that works sufficiently for them? Or the ability to refresh the outline only on file load &save? If you want any feedback or roadtest of the new engine I'm willing to try. Thanks

…

On Wed, 13 Jun 2018, 14:57 Brett Cannon, ***@***.***> wrote: "there is still something going on that is interfering with normal behaviour", and that behaviour is the outline view. 😉 VS Code is basically hammering Jedi every 250 ms to re-parse the file and analyze it for symbols. IOW there's not much we can do here short of throttling even more or magically making Jedi faster (also realize the Outline view is in preview for VS Code for a reason 😉; we actually had to provide the feedback to even have the 250 ms throttle added as originally it was every key press triggering a re-scan). We are hoping to have the Outline view supported by our experimental analysis engine soon which should help alleviate this performance bottleneck. Once #1934 <#1934> is merged and we have verified the new analysis engine works with the outline view I can provide instructions here on how to flip it on. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1721 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFUOVnyJ4XGy5fhlwA1lBC13pUFPd8zMks5t8WCXgaJpZM4UJPik> .

[brettcannon]

The latest build of the new language server should have the Outline functioning as expected. You can download a development build of the extension to test it out.