uriTransformer uses rg and it descends into directories with circular references - 100% CPU and connection eventually drops

[LiveFreeAndRoam]

I have had to address this same issue with VSCode Search (see my settings.json below) and also for Pylance issue microsoft/pylance-release#1939 (see pywrightconfig.json below). Now, I'm seeing this issue with uriTransformer.js.

How do I configure uriTransformer so that it excludes specified directories. Ideally, it should use my settings.json, search.exclude list, but it does not.

The evidence and trail is captured below. Please ask if you need more info.

• VSCode Version: v1.62.2
• Local OS Version: Windows 10
• Remote OS Version: SLES 15-SP2
• Remote Extension/Connection Type: SSH
• Logs: See below

Steps to Reproduce:

Start with a really big VSCode workspace (e.g. mine is a Yocto-based dev area). When yocto builds, it creates lots of deeply nested circular references in the build area. I have configured VSCode Search and Pylance to not descend into the directories where there are circular references. Now, I'm seeing uriTransformer.js gets stuck in a loop, presumably because it is not using those settings.

NOTE, these circular references are legitimate and won't be addressed (it's all part of a complex build process). When they get installed on the target, the circular references point to special devices.

Top

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
19047 user+     20   0   96172  43544   3552 S  1135 0.011  82:44.12 rg
22329 user+     20   0 4671644 1.206g  38140 R 100.0 0.321   9:56.93 node
18511 user+     20   0 10.251g 9.589g  41468 S 23.53 2.551   4:59.52 node

Process info PID 19047 (rg)

The strace below reveals this PID is rg and is not doing anything:

$ ps -Flww -p 19047
F S UID        PID  PPID  C PRI  NI ADDR SZ WCHAN    RSS PSR STIME TTY          TIME CMD
0 S user+    19047 18511 99  80   0 - 24649 -      43772  27 01:27 ?        02:31:02 /home/user/.vscode-server/bin/3a6960b964327f0e3882ce18fcebd07ed191b316/node_modules/vscode-ripgrep/bin/rg --files --hidden --case-sensitive -g **/*.ipynb -g !**/.git -g !**/.svn -g !**/.hg -g !**/CVS -g !**/.DS_Store -g !**/Thumbs.db --no-ignore --follow --no-config --no-ignore-global

$ strace -p 19047
strace: Process 19047 attached
futex(0x7f5ebf9ffb60, FUTEX_WAIT_PRIVATE, 1, NULL
^Cstrace: Process 19047 detached
 <detached ...>

Process info PPID 18511 (uriTransformer)

The strace below reveals the PPID is uriTransformer and it is stuck in a loop:

$ ps -Flww -p 18511
F S UID        PID  PPID  C PRI  NI ADDR SZ WCHAN    RSS PSR STIME TTY          TIME CMD
0 S user+    18511 12094 98  80   0 - 5572517 -    21549092 54 01:27 ?      00:16:54 /home/user/.vscode-server/bin/3a6960b964327f0e3882ce18fcebd07ed191b316/node /home/user/.vscode-server/bin/3a6960b964327f0e3882ce18fcebd07ed191b316/out/bootstrap-fork --type=extensionHost --uriTransformerPath=/home/user/.vscode-server/bin/3a6960b964327f0e3882ce18fcebd07ed191b316/out/vs/server/uriTransformer.js

$ strace -p 18511 -s 400 2>&1 | grep -i loop
read(63, "File system loop found: ./yocto/build/my-subsys/tmp/work/aarch64-fsl-linux/systemd/1_246.2-r0/build/test/sys/dev/char/189:385/4-1:1.1/subsystem/devices/usb3/3-1/3-1:1.0/input/input8/subsystem/event5/device/device/driver/LNXPWRBN:00/subsystem/devices/device:1a/physical_node/usb2/2-0:1.0/ep_81/subsystem/usbdev5.1_ep81/device/driver/module/holders/cdc_acm/drivers/usb:cdc_acm/5-2:1.0/tty/ttyACM0/subsyste"..., 65536) = 3662
read(63, "File system loop found: ./yocto/build/my-subsys/tmp/work/aarch64-fsl-linux/systemd/1_246.2-r0/build/test/sys/dev/char/189:385/4-1:1.1/subsystem/devices/usb3/3-1/3-1:1.0/input/input8/subsystem/event5/device/device/driver/LNXPWRBN:00/subsystem/devices/device:1a/physical_node/usb2/2-0:1.0/ep_81/subsystem/usbdev5.1_ep81/device/driver/module/holders/cdc_acm/drivers/usb:cdc_acm/5-2:1.0/tty/ttyACM0/subsyste"..., 65536) = 1240
read(63, "File system loop found: ./yocto/build/my-subsys/tmp/work/aarch64-fsl-linux/systemd/1_246.2-r0/build/test/sys/dev/char/189:385/4-1:1.1/subsystem/devices/usb3/3-1/3-1:1.0/input/input8/subsystem/event5/device/device/driver/LNXPWRBN:00/subsystem/devices/device:1a/physical_node/usb2/2-0:1.0/ep_81/subsystem/usbdev5.1_ep81/device/driver/module/holders/cdc_acm/drivers/usb:cdc_acm/5-2:1.0/tty/ttyACM0/subsyste"..., 65536) = 1242
read(63, "File system loop found: ./yocto/build/my-subsys/tmp/work/aarch64-fsl-linux/systemd/1_246.2-r0/build/test/sys/dev/char/189:385/4-1:1.1/subsystem/devices/usb3/3-1/3-1:1.0/input/input8/subsystem/event5/device/device/driver/LNXPWRBN:00/subsystem/devices/device:1a/physical_node/usb2/2-0:1.0/ep_81/subsystem/usbdev5.1_ep81/device/driver/module/holders/cdc_acm/drivers/usb:cdc_acm/5-2:1.0/tty/ttyACM0/subsyste"..., 65536) = 3735

... forever and ever ...

Workspace's .vscode/settings

{
  "files.watcherExclude": {
    "**/__pycache__": true,
    "**/my-cpld": true,
    "**/stuff": true,
    "**/yocto": true
  },
  "search.exclude": {
    "**/__pycache__": true,
    "**/yocto/build/my-subsys/tmp": true,
    "**/yocto/build/sstate-cache.1": true,
    "**/yocto/build/sstate-cache.2": true
  }
}

Worksapce pyrightconfig.json

{
  "exclude": [
    "**/node_modules",
    "**/__pycache__",
    ".git",
    "./yocto/build/my-subsys/tmp",
    "./yocto/build/sstate-cache.1",
    "./yocto/build/sstate-cache.2"
  ]
}

Does this issue occur when you try this locally?: Yes/No
Does this issue occur when you try this locally and all extensions are disabled?: Yes/No

[roblourens]

-g **/*.ipynb some extension is searching your whole workspace for ipynb files. What extensions do you have installed? Pylance and the python extension do not do that. The "uriTransformer" process you are pointing at is just the extension host.

I don't understand the circular references you mention, are they symlinks? You might want to disable the "follow symlinks" setting. Something in https://github.com/microsoft/vscode/wiki/Search-Issues might help here.

[LiveFreeAndRoam]

Thank you!

I have discovered the ipynb files belong to Jupyter Notebook Extension; I have referenced this issue on their Extension issues (https://github.com/microsoft/vscode-jupyter/issues/8290). In the meantime, I have disabled the "follow symlinks" to see if that works.

This raises a general issue where extensions are not honoring the search.exclude setting. Instead, they each want their own setting. Can this be addressed at the top-level rather than chasing down each extension?

[roblourens]

The jupyter extension is starting a search, that explains the search.

By default, search.exclude isn't used in the findFiles API that the extension is using here, this is a long standing issue that we hope to fix soon: microsoft/vscode#48674

files.exclude and .gitignore would be respected here though.