VS Code crashes when running a cell that produces a lot of output

[ydmykr]

I am trying to print model summary in tensorflow and I think the model is large and it's crashing my notebook. The model is ResNet101.

The whole computer comes to a halt, memory usage goes up to 99% and VS Code crashes. I have 16 GB of ram, so I didn't think printing something large would actually eat all my ram. Also, because the kernel crashes, all the variables are lost like history = model.fit() which I need to fine-tune the model afterwards. Moreover, I need to print base_model summary in order to choose from which layer to fine-tune from.

Is there a way to print the summary in another way and save the entire notebook with the variables, so I can continue working? I have checkpoints for model weights, but I need to keep track of past epochs through history to resume training afterwards.

I will try using.py files, but I want to know if there is a way to solve this problem for jupyter.

[DonJayamanne]

Thanks for filing this issue and I'm sorry you are running into this.

• Is it VS Code or is it is the kernel that crashes?
• If you were to run this outside VS Code in classic jupyter via a terminal , do you experience the same problem?
• Can you run a cell at a time to check when exactly vscode crashes, that would help narrow down the issue

[ydmykr]

@DonJayamanne
1.Vscode crashed
2.It doesn't crash on classic jupyter
3.The cell that cause the problem is the one where I am trying to print model summary, It consumes 99% of my available memory and it will either crash VS Code or it will gray everything out

[DonJayamanne]

Thanks for your patience & I'm sorry about this issue, hopefully we can get to the bottom of this.

I'm assuming you are referring to the cell with the code base_model.summary
Is it possible to try the following:

• Not print any output at all
• Run the first cell, then clear all output using the command or icon in the toolbar
• Run the next cell and clear all of the output.
• Similarly continue till you get to the problematic cell, again, comment out the code where you are printing the model summary.

I have a suspicion the output from the second cell is very large & that could be chewing up resources.
Again, please ensure you clera all of the output after running each cell.

What I'd like to figure out is:

• Whether the memory usaage/crash is realted to the output or execution
• If we ensure the output is always cleared, and things work as expected (i.e. it does not crash), then we've narrowed down the issue to outputs causing increased memory usage and crashes vscode.

Finally

• Also, please could you share the output from the Jupyter output panel, use the command Jupyter: Show Output
  Run the first cell in the notebook and copy the logs and send the logs (please ensure you send the whole log). Shows informatino about versions, python used, and the like.
• Are you running the code against a local or remote kernel?

• memory usage goes up to 99% and VS Code crashes. I
  Please could you take a screenshot of this and upload that here, would like to check what vscode process is chewing up the memory. thanks

[mariosconsta]

I am the OP who posted this question on stack overflow and OP here created the issue for me, so I will answer your questions.
So after making a new notebook to test again, the same thing happens:

Sorry for the phone screenshot, computer froze. Also, please note that because the computer is getting so slow, it can't show me the exact usage number, but I guess it's gigabytes since its saying 99%

• Clearing outputs is not changing the outcome

I have a suspicion the output from the second cell is very large & that could be chewing up resources.
Again, please ensure you clera all of the output after running each cell.

• I have put verbose = 2 for model.fit so that output is not large. But eitherway, the same thing happens on another notebook where I do not train any model and the only output is base_model.summary()

Similarly continue till you get to the problematic cell, again, comment out the code where you are printing the model summary

Yep the problematic cell is the one where I print the summary(). Same thing happens on a brand new and small notebook that I just import ResNet101 and try to print it's summary.

What I'd like to figure out is:

Whether the memory usaage/crash is realted to the output or execution

I think it's execution and not output. Because even if 15-20secs has passed since I pressed run on that cell, after it finishes it shows 0.5s and when I run that cell again, it is taking less time which I believe is because the variable is already saved and it doesn't need to compute it again.

If we ensure the output is always cleared, and things work as expected (i.e. it does not crash), then we've narrowed down the issue to outputs causing increased memory usage and crashes vscode.

Unfortunately, that's not the issue

Are you running the code against a local or remote kernel?

Local Kernel

Also, please could you share the output from the Jupyter output panel, use the command Jupyter: Show Output
Run the first cell in the notebook and copy the logs and send the logs (please ensure you send the whole log). Shows informatinon about versions, python used, and the like.

logs.txt

[DonJayamanne]

Thanks for all of the information, please could you:

• Provide a very small sample code to replicate this at our end.
  You mentioned a small notebook, please provide that as a sample or something even smaller.
• Please provide all of the necessary python packages that need to be installed, if you are using conda, then even better as you could just provide the conda yaml file (do let us know the version of the python environment as well).
• If there are ay other files that we need, please could you provide those as well (feel free to email them to me, if you think they are they shoudl not be shared online, my email is on my github profile)

[mariosconsta]

Thanks for all of the information, please could you:

• Provide a very small sample code to replicate this at our end.
  You mentioned a small notebook, please provide that as a sample or something even smaller.
• Please provide all of the necessary python packages that need to be installed, if you are using conda, then even better as you could just provide the conda yaml file (do let us know the version of the python environment as well).
• If there are ay other files that we need, please could you provide those as well (feel free to email them to me, if you think they are they shoudl not be shared online, my email is on my github profile)

The zip folder should contain everything you need. The small demo notebook at around 100KB and a notepad with all my packages, you can run pip install with that notepad and it will install everything for you.

demo.zip

Also I am using venv, python version is 3.10.2, tensorflow version is the latest. If you can't reproduce it, I can take a video with my phone and show you step by step what is happening.

@DonJayamanne Let me know if you need anything else and I would be more than happy to assist

[DonJayamanne]

@mariosconsta
I've tried running this at my end, both on a Mac and Windows machine and have no problems.

Can you try the following:

• Step 1
  • Run the notebook in Jupyter notebooks, save the notebook and upload that saved notebook here.

  notebook where I do not train any model and the only output is base_model.summary()

  • Run the above notebook in Jupyter notebooks as well , save the notebook and upload that saved notebook here.
• Step 2
  • Exit vscode completely
  • Open the above two notebooks and see if you have similar perf issues in vscode.
  • Check the memory usage
  • Note: At this point, you are NOT running anything in VS Code, just opening the notebooks

Basically I'd like to get the two notebooks.

[mariosconsta]

Sorry for the late reply, I will test this out ASAP and let you know.

[mariosconsta]

@DonJayamanne

So with VS Code closed. I opened jupyter notebook and run the cell that outputs model summary. The cell ran instantly without trouble. I opened VS Code to try it again and the same thing happened.

notebooks.zip

I will take a video with my phone as its happening and upload it here as well.

[mariosconsta]

@DonJayamanne
I have uploaded 2 clips of the problem. The second clip is from my phone because I couldn't record normally. The first clip is showing jupyter running like a charm and the second one is VS Code consuming 100% of my RAM and then crashing.

There's also 2 screenshots on how VS Code looks after it recovers along with the error that sometimes pops up. The error doesn't pop up always.

https://drive.google.com/file/d/1GzOu8ZdX9CTZeN7guRcpZpYaWco5B445/view?usp=sharing

[DonJayamanne]

Will get a custom build of he extension shortly, so you can test with that
Once again, thanks for your patience and sorry for the inconvenience caused.

[DonJayamanne]

Please could you:

• Install VS Code Insiders
• Install the attached vsix using the command Extnsions: Install from VSIX
• Reload VS Code insiders
• Try running the notebook with this version
• Note: You might not see any outputs in any cell, however you should still see the green tick to indicate whether a cell ran successfully or not, along with the execution count

Please let me know how this goes.
ms-toolsai-jupyter-insiders.vsix.zip

[mariosconsta]

I will test it out first thing in the morning!

[mariosconsta]

@DonJayamanne Runs instantly as shown in the screenshot below. Like you said there's no output, but it does run like a charm and uses zero ram. Is there a way to get output from this version? Or should I continue using the version I currently am on.

P.S. I tried printing it as well to see if I get output but I didn't. The original code was just

base_model.summary()

Also these are my extensions:

Should I try switching to jupyter pre-release version on VS Code? (not insiders, the normal one)

EDIT: So using the pre-release version of jupyter did not fix the problem.

VS Code (main version not the one you sent) still uses 100% of my memory and after that, I get that visual bug. Like I said, this does not happen on insiders version and with the file you shared. Also it does not happen on stand alone jupyter notebook.

The cell might say 0.5s but it wasn't 0.5s, it took more than 15s to complete.

EDIT:

I am willing to do an AnyDesk session with you if you want to see it "live" if the recordings I sent you were not enough.

In conclusion, the version you sent works like a charm, the main version does not, this text is all over the place and I am sorry for that.

[mariosconsta]

Another issue is this:

Sometimes when it finishes without crashing, the output looks like that.

[DonJayamanne]

@mariosconsta Please could you try installing this VSIX as well and letting me know whether this still crashes.
Thanks again for your help and patience
ms-toolsai-jupyter-insiders.vsix (1).zip

[DonJayamanne]

I am willing to do an AnyDesk session with you if you want to see it "live" if the recordings I sent you were not enough.

Thanks, so far what we have going is working, i'm still trying to gather some more information before I start working on this issue.

[mariosconsta]

@DonJayamanne

Please could you try installing this VSIX as well and letting me know whether this still crashes.

Just like the last VSIX, works like a charm. The cell runs instantly without issues.

[mariosconsta]

Please could you try installing this VSIX as well and letting me know whether this still crashes.

Also like before, no output. Could this be the issue? The output? I did some testing and I was able to output large text without issue using a for loop.

You can check the recording here:
demo2.zip

[DonJayamanne]

Internal note: Blocked on upstream issue microsoft/vscode#146768

Also like before, no output. Could this be the issue? The output? I did some testing and I was able to output large text without issue using a for loop.

@mariosconsta I'm sorry, but I dont follow you.
From what I understand with the VSIX i provided, it doesn't crash anymore, however hot output is displayed is that right.

Could this be the issue? The output? I

Currently the extension host seems to crash due to the size of the output generated.

[livan3li]

hi, has this problem been solved? @DonJayamanne

I have same problem right now. I have 24 GB ram and have recorded a video while problem is happening.
VS Code consumes all available memory and OS crashes if i dont close the VS CODE When i try to open a Jupyter Notebook which have 618 images in.

But weird thing is that when i try to open same notebook in Google Colab everything is perfect. No memory consumption and even more i can view all images very fast by sliding scroll bar very very fast and as a result no freeze, no stuck, no problem.

I think the problem might be all about loading all data into memory(or memory management), not from storage device(HDD/SSD). Question is how Google Colab can achive such a good performance.

Versions:
VS CODE
Version: 1.76.2 (user setup) Commit: ee2b180d582a7f601fa6ecfdad8d9fd269ab1884 Date: 2023-03-14T17:55:54.936Z Electron: 19.1.11 Chromium: 102.0.5005.196 Node.js: 16.14.2 V8: 10.2.154.26-electron.0 OS: Windows_NT x64 10.0.19045 Sandboxed: Yes

Jupyter v2023.2.1200692131

Jupyter Keymap v1.1.0

Jupyter Notebook Renderers v1.0.15

Jupyter Slide Show v0.1.5

Thanks.

vs_code_memory.mp4

[DonJayamanne]

@livan3li
Unfortunately this hasn't been fixed yet
please could you share this notebook
I'd like to try this at my end

Also, please could your disable all extensions, including Jupyter and see if you run into this issue
If you do then the problem is different, as this issue focuses on crashes doe to running of notebooks
However I do want to address your issue too, but would like to deal with that separately, for now let's chat here

[livan3li]

Ok. I've disabled all extensions related with Jupyter after this it's been opened as plain text file.

Here is the notebook link in drive which i have used in the video.

@livan3li Unfortunately this hasn't been fixed yet please could you share this notebook I'd like to try this at my end

Also, please could your disable all extensions, including Jupyter and see if you run into this issue If you do then the problem is different, as this issue focuses on crashes doe to running of notebooks However I do want to address your issue too, but would like to deal with that separately, for now let's chat here

[DonJayamanne]

I've disabled all extensions related with Jupyter after this it's been opened as plain text file.

What i meant was, disable all extensions (including Jupyter), but still open as a notebook.
You can still open the ipynb file as a notebook even with teh Jupyter extension disabled,
Please try to re-open the file as a notebook (instead of opening it as a plain text file)
Opening and rendering notebooks is a feature that is a core functionality in vscode

[DonJayamanne]

@rebornix aren't you already looking into some perf improvements for opening notebooks with large number of outputs?

[livan3li]

I've disabled all extensions related with Jupyter after this it's been opened as plain text file.

What i meant was, disable all extensions (including Jupyter), but still open as a notebook. You can still open the ipynb file as a notebook even with teh Jupyter extension disabled, Please try to re-open the file as a notebook (instead of opening it as a plain text file) Opening and rendering notebooks is a feature that is a core functionality in vscode

Sorry for that. I've done exactly what you said. I disabled all extensions (even Python) and tried to open the notebook but same thing happened again. VS Code consumes all available memory.

[tyePhDCandy]

I've disabled all extensions related with Jupyter after this it's been opened as plain text file.

What i meant was, disable all extensions (including Jupyter), but still open as a notebook. You can still open the ipynb file as a notebook even with teh Jupyter extension disabled, Please try to re-open the file as a notebook (instead of opening it as a plain text file) Opening and rendering notebooks is a feature that is a core functionality in vscode

Sorry for that. I've done exactly what you said. I disabled all extensions (even Python) and tried to open the notebook but same thing happened again. VS Code consumes all available memory.

you can check the subprocess under main process 'vscode' . see which subprocess consumes all that memory. I found out, on my end, the python subprocess under 'vscode' consumes loads of memory.

[tyePhDCandy]

Yeah I can verify as well that it does not work. I run my notebook again and the whole computer froze for 3 solid minutes and then:

this happens to me too.

[livan3li]

I've disabled all extensions related with Jupyter after this it's been opened as plain text file.

What i meant was, disable all extensions (including Jupyter), but still open as a notebook. You can still open the ipynb file as a notebook even with teh Jupyter extension disabled, Please try to re-open the file as a notebook (instead of opening it as a plain text file) Opening and rendering notebooks is a feature that is a core functionality in vscode

Sorry for that. I've done exactly what you said. I disabled all extensions (even Python) and tried to open the notebook but same thing happened again. VS Code consumes all available memory.

you can check the subprocess under main process 'vscode' . see which subprocess consumes all that memory. I found out, on my end, the python subprocess under 'vscode' consumes loads of memory.

Since i disabled all extensions, Python doesn't do that for me. But now under VS Code, again VS Code consumes all memory.

[livan3li]

I've disabled all extensions related with Jupyter after this it's been opened as plain text file.

What i meant was, disable all extensions (including Jupyter), but still open as a notebook. You can still open the ipynb file as a notebook even with teh Jupyter extension disabled, Please try to re-open the file as a notebook (instead of opening it as a plain text file) Opening and rendering notebooks is a feature that is a core functionality in vscode

Have you reported bug where it needs to be or i need to do that?

[DonJayamanne]

@tyePhDCandy
Thanks for continuing to provide feedback, however it gets a little confusing when we try different things
Please could you confirm the following:

• Disable all extensions
• Close VS Code
• Re-open VS Code
• Open the same large notebook
• VS Code hangs again

This will confirm that the freeze is due to loading of the notebook
The memory usage of the python process could be due to the large training job or the like.
Optionally you could try the same in Jupyter Notebook Application outside VS Code and see if there are similar high memory usages by that application, which would indicate the fact that it is expected (due to the code being executed)

[livan3li]

@tyePhDCandy Thanks for continuing to provide feedback, however it gets a little confusing when we try different things Please could you confirm the following:

• Disable all extensions
• Close VS Code
• Re-open VS Code
• Open the same large notebook
• VS Code hangs again

This will confirm that the freeze is due to loading of the notebook The memory usage of the python process could be due to the large training job or the like. Optionally you could try the same in Jupyter Notebook Application outside VS Code and see if there are similar high memory usages by that application, which would indicate the fact that it is expected (due to the code being executed)

I dont have any installed Jupyter Notebook app other than VS Code.
I've done as you said. Please watch all video. VS Code freezes two different ways.

1. When i scroll down to place where images appears. (Doesn't freeze when it's opened first time)
2. When i open the notebook (freezes asap it's opened)

vs_code_bug.mp4

[Megil-Zesty]

Also experiencing this issue - super frustrating.

Context: notebook essentially freezes when I print too much to a console or when trying to display a large pandas dataframe (by simply typing "df" in a cell and running it"). I have 104gb of ram and the dataframes are only consuming about 7gb.

[DonJayamanne]

@Megil-Zesty

• Please could you share some sample code that can be used to replicate this issue.

[rebornix]

This issue should now be fixed in latest Insiders and will be available in next week's Stable release. We will revisit this issue once you all get the latest fixes and reopen if necessary.

[jrieken]

Tested successfully with a dummy controller that produces 10000000 lines textual output. That might have done it but having author verification or crisp verification steps would help

[joserojasb]

Hi everyone! I've run into the white walls of doom just after a recent vscode update, on my Mac M1 Pro. Due to compliance reasons i'm unable to provide code examples, but some of the issues others have reported seem to keep popping up:

• It happens when running exhaustive calculations/displaying long outputs
• Other notebooks that previously ran fine seem now to be "corrupted", or at least this is a persistent issue even when rebooting
• I've purged vscode and reinstalled everything, to no avail

I hope this information is at least a little bit useful!

[r3m0t]

I have a repro here microsoft/vscode#182636 @jrieken

[johnomage]

I currently having this issue with version 1.81.0 😭😞🤬

@livan3li
@DonJayamanne
@DonJayamanne

Help !!!