MarkerService and ProblemsView do not scale well and block UI thread

[DustinCampbell]

• VSCode Version: 1.5.2
• OS Version: Tested Windows 10 and macOS 10.11

Repro steps:

1. Clone https://github.com/DustinCampbell/diagnostic-perf-test
2. Open the diagnostic-perf-test project in VS Code
3. F5 to compile and launch the extension under the debugger
4. In the debuggee process, select View->Problems to display the Problems pane.
5. In the debuggee process, select View->Command Palette... and execute the Populate Diagnostics command.

At this point, the VS Code debuggee window will stop responding as it populates a single diagnostic for 2000 files. This is very similar to the diagnostics produced for a medium- to large-sized C# solution (say, one generated by Unity) where there are a couple thousand files and each has a single "remove unnecessary using directives" diagnostic.

Expected result:

• Populating diagnostics should not scale exponentially by the number of files.
• Calling DiagnosticCollection.set(...) should periodically free up the UI thread while processing.

[jrieken]

The problem is that we call changeOne per uri+owner combination which leads to many events. Processing those events (problems view) takes a long time. We should not fire so many events...

CPU-20160919T115033.cpuprofile.zip

[jrieken]

We have found a couple of issues

• event storm by emitting in the loop
• the problems view sorting and filtering all elements

[jrieken]

@sandy081 I have pushed changes to send all events at once and to make reading of markers a little faster. that improves the situation a tiny bit. as discussed the culprit is the problems view. pls continue from here

[sandy081]

Thanks @jrieken for the changes. I will do the improvement is problems view as discussed.

[sandy081]

Root cause: Problems model is getting updated resource by resource and every time it is getting refreshed (filtered and sorted). So this is doing filtering and sorting as many times as the number of resources (in this case 2000). Fixed it by doing bulk update / refresh.

Also did some optimisations in the model for better performance. After all this, amount of time came down from 20s to 1s

Attaching the pre-fix and post-fix profiles here

[DustinCampbell]

nice!

[jrieken]

reopening for me to further improve MarkerService#read. That 1 second must go ;-)

[jrieken]

🐎 MarkerService#read from 1000ms down to 2.5ms. Slightly embarrassing it wasn't like this to begin with...

[jrieken]

@sandy081 The last big chunk is Tree#expand which you might wanna consult with @joaomoreno if it can be improved.

[sandy081]

I added another fix/improvement while applying filters.

Using Tree.expand on each item or Tree.expandAll is not making any difference.
Will discuss with @joaomoreno and see if I can improve this.

Otherwise Closing this since we improved performance drastically.