Search Service consumes 70% of my CPU on startup for quite some time

[dbaeumer]

Latest insider On Windows

• starte code-insiders on a vscode workspace

You get:

[dbaeumer]

@joaomoreno FYI

[joaomoreno]

I don't see the same process madness as @dbaeumer, but I do see this in the devtools:

cc @jrieken

[jrieken]

Yep, seeing the same

[jrieken]

Also these things seem to stick around for way to long...

[roblourens]

Do they pile up when you do a text search or just quickopen?

[jrieken]

I start the product. That's it. My guess is that extensions using the workspaceContains-event trigger a search for file names. And yes, they do pile up.

[roblourens]

I'm not seeing it, but I am seeing the error that @joaomoreno posted at least so I can start there.

[roblourens]

Can you grab the grab the args from a few of those rg processes with ps?

[jrieken]

nah, deleted the vscode-ripgrep module because my system was practically un-usable when running ./scripts/code often. Tho, I do remember seeing some glob patterns in htop

[roblourens]

Can you give me your extensions list?

[roblourens]

I fixed the error that @joaomoreno posted. @dbaeumer's issue is possibly doing too much work because of #34487. I never saw @jrieken's issue but please try again.

[jrieken]

This is just killing me. @roblourens to reproduce do this

• install the c#-extension or any extension with many workspace contains activation events
• have a large workspace, e.g. / or a folder with node modules
• start vscode and when it's up hit refresh, repeat 5 times
• 🐛 many rg processes

Yes, the activation of extensions is a little stupid and should merge the query per extension but the disk search should not spawn so many processes and eventually start queuing process execution and it should make sure to kill the processes (which is hard usually the child process must check if its parent pid is still alive and self-terminate)

[roblourens]

I can repro this on / now and we can at least shut down the rg processes properly when the window closes or reloads. I'm still confused as to why it's so slow for you just in the vscode folder. Those searches terminate for me in a few milliseconds. Does the screenshot show the vscode workspace or /? If it's vscode, that's a problem.

Also just to confirm, this isn't new, right? I see the same behavior with find. It might be more noticeable now, since rg is more efficient and parallelized, and will probably use higher CPU while it runs but complete sooner.

And also, there's yet another issue that Dirk shows where the rg processes terminate quickly, but the searchService uses high CPU for a long time.

To summarize this issue up to this point:

• console errors when launching with workspaceContains
  • Fixed
• rg processes not shut down when the windows closes or reloads
  • I can fix
• rg processes run slowly and pile up in vscode workspace for @jrieken
  • I don't repro, they complete and terminate almost immediately
• After rg processes terminate quickly, searchService takes high CPU for awhile, in vscode repo, on Windows, for @dbaeumer
  • I think I can repro, it's not nearly to the same degree but it's running longer than it takes on Mac. I can check whether we are doing something dumb.

[peabnuts123]

Just to chip in here, on my Macbook Pro the search service ramps up to high CPU usage (100% of a core) on startup and never terminates on both my work and home laptops. I'm not 100% confident this happens without any extensions but I am also not 100% sure that means it isn't vscode's problem.

[roblourens]

Pushed the change to ensure it's killed. I will open an issue just for that to make sure it gets verified. The rest of #34434 (comment) I will investigate in October.

[peabnuts123]

If anybody here has the exact problems that I described above, I tracked it down to the Omnisharp-vscode C# extension. Try disabling that if you have issues.

I've opened an issue here: dotnet/vscode-csharp#1770

[roblourens]

Thanks for the datapoint - No need to file an issue on individual extensions, this will impact any extension that uses the workspaceContains pattern to activate when the workspace contains a certain type of file. The C# extension triggers this in particular because it happens to use several of these.

You should see an improvement already in the latest insiders (https://az764295.vo.msecnd.net/insider/683a46efbab870527671f1ae55f9ebce5f3f5a3c/VSCode-darwin-insider-unsigned.zip) because we will batch these per extension. But it won't go away entirely.

[peabnuts123]

I see. I hadn't fully realised that there was a concrete issue with vscode itself that was causing these problems

[roblourens]

Discussion for this has also spread out into

#34487
#34711
and
#34823

[Chillee]

I've had a similar issue for a while. It seems to usually appear when I accidentally start vscode in the home directory, for example. The process does not close upon the vscode instance being closed.

[jrieken]

Pushed the change to ensure it's killed.

@roblourens I still see plenty of rg-processes (without a parent) on my machine.

It's from scripts/code.sh and they all look like this

13609 ttys001    0:48.65 /Users/jrieken/Code/vscode/node_modules/vscode-ripgrep/bin/rg --files --hidden --case-sensitive -g **/test* -g !Users/jrieken/Code/_samples/ConsoleAppDnx -
jrieken:~/Code$

[jrieken]

Unsure how these processes are meant to die but from my experience it is hopeless to try to manage from the outside, e.g. ext-host/renderer killing them on shutdown, because the "outside" my disappear without notice. Usually, we put something into the processes, like a pointer to the parent/owner, and when that disappears the processes shutdown themselves. Maybe you are already doing that tho...

For instance the extension host checks every 5 secs if its owning renderer process is still around

[roblourens]

How are you killing vscode? It's not bulletproof but should at least work for a normal shutdown.

We don't own the rg process, ideally I think we would run ripgrep as a library from a process we control, not yet possible.

[jrieken]

How are you killing vscode?

Depends, but during development and when I am in the console anyways often with Ctrl+C

It's not bulletproof but should at least work for a normal shutdown.

I don't think that's enough, processes crash and when that happens it will leak.

[roblourens]

That's true but I think it's the best we can do without making a change in ripgrep. But it sounds like there is still the real problem of the rg processes not terminating when they are actually finished, which I haven't been able to repro.

[chrmarti]

@jrieken You could use sudo opensnoop -n rg to check if it is still accessing files.

[chrmarti]

@jrieken Do you still see these? Could you post the full command line (the above seems truncated)? And maybe run the same command from the workspace folder (redirect stdout to /dev/null) to see what that does and outputs on stderr?

[roblourens]

Still an issue? I still haven't encountered this...