Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Significant slowdown when evaluating re.sub(...) functions in versions after 2021.9.1 #1979

Closed
ScottNealon opened this issue Oct 25, 2021 · 3 comments
Closed

Significant slowdown when evaluating re.sub(...) functions in versions after 2021.9.1 #1979

ScottNealon opened this issue Oct 25, 2021 · 3 comments
Labels
fixed in next version (main) A fix has been implemented and will appear in an upcoming version

Comments

@ScottNealon
Copy link

Environment data

  • Language Server version: 2021.9.2 or newer (bug does not exist in 2021.9.1 or 2021.9.0)
  • OS and version: Microsoft Windows 10 Enterprise, 10.0.18363 Build 18363
  • Python version (& distribution if applicable, e.g. Anaconda): 3.9.6

Expected behavior

Pylance provides close to real time syntax coloring.

Actual behavior

Pylance takes 22 seconds to provide syntax coloring for a simple script and up to 391 seconds for a more complicated script, during which time syntax coloring is not available to other scripts. Slowdown seems to involve regular expression substitution.

Example Setup

directory/
├── .vscode/
│   ├── settings.json
├── functional_script.py
├── super_slow_script.py
.vscode/settings.json

{
    "python.pythonPath": "C:\\Users\\{$username}\\AppData\\Local\\Programs\\Python\\Python39\\python.exe",
    "python.languageServer": "Pylance",
    "python.analysis.logLevel": "Trace"
}

functional_script.py

def main():
    print("Hello World")

if __name__ == "__main__":
    main()

super_slow_script.py

import os
import re

def get_text_in_file(file, file_paths):
    return "dummy string"

def get_included_files(text,file_paths):

    include_once= r'include_once\s+(\S+)'
    include= r'include\s+(\S+)'
    
    include_files = re.findall(include,text)
    include_once_files = re.findall(include_once,text)
    
    new_files = []
    imported_include_once_files = {}
    
    while include_once_files:
        for file in include_files:
            new_text = get_text_in_file(file, file_paths)
            text = re.sub(r'include\s+' + file, new_text[0], text)
               
        for file in include_once_files:
            if file not in imported_include_once_files: 
                new_text = get_text_in_file(file, file_paths)
                imported_include_once_files[file] = new_text 
                text = re.sub(r'include_once\s+'+file, new_text[0], text)
                
        include_files = re.findall(include,text)
        include_once_files = re.findall(include_once,text)
            
    return text, imported_include_once_files

Logs

Logs were generated by reloading VSCode while viewing the script in question.

functional_script.py, Any Language Server Version

[Info  - 1:14:48 PM] Pylance language server 2021.10.2 (pyright 1d97208c) starting
[Info  - 1:14:48 PM] Server root directory: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.10.2\dist
[Info  - 1:14:48 PM] No configuration file found.
[Info  - 1:14:48 PM] No pyproject.toml file found.
[Info  - 1:14:48 PM] Setting pythonPath for service "test": "C:\Users\{$username}\AppData\Local\Programs\Python\Python39\python.exe"
[Warn  - 1:14:48 PM] stubPath c:\Users\{$username}\Documents\Projects\test\typings is not a valid directory.
[Info  - 1:14:48 PM] Assuming Python version 3.9
[Info  - 1:14:48 PM] Assuming Python platform Windows
Search paths for c:\Users\{$username}\Documents\Projects\test
  c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.10.2\dist\typeshed-fallback\stdlib
  c:\Users\{$username}\Documents\Projects\test
  c:\Users\{$username}\Documents\Projects\test\typings
  c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.10.2\dist\typeshed-fallback\stubs\...
  c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.10.2\dist\bundled\stubs
  c:\Users\{$username}\AppData\Local\Programs\Python\Python39\DLLs
  c:\Users\{$username}\AppData\Local\Programs\Python\Python39\Lib
  c:\Users\{$username}\AppData\Local\Programs\Python\Python39
  c:\Users\{$username}\AppData\Local\Programs\Python\Python39\Lib\site-packages
[Info  - 1:14:49 PM] Searching for source files
[Info  - 1:14:49 PM] Found 2 source files
[Info  - 1:14:49 PM] Background analysis(1) root directory: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.10.2\dist
[Info  - 1:14:49 PM] Background analysis(1) started
Background analysis message: setConfigOptions
Background analysis message: setImportResolver
Background analysis message: ensurePartialStubPackages
Background analysis message: setTrackedFiles
Background analysis message: markAllFilesDirty
Background analysis message: setFileOpened
Background analysis message: getSemanticTokens full
[BG(1)] getSemanticTokens full at c:\Users\{$username}\Documents\Projects\test\functional_script.py ...
[BG(1)]   parsing: c:\Users\{$username}\Documents\Projects\test\functional_script.py (13ms)
[BG(1)]   parsing: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.10.2\dist\typeshed-fallback\stdlib\builtins.pyi [fs read 3ms] (73ms)
[BG(1)]   binding: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.10.2\dist\typeshed-fallback\stdlib\builtins.pyi (31ms)
[BG(1)]   binding: c:\Users\{$username}\Documents\Projects\test\functional_script.py (0ms)
[BG(1)]   parsing: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.10.2\dist\typeshed-fallback\stdlib\typing.pyi [fs read 1ms] (26ms)
[BG(1)]   binding: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.10.2\dist\typeshed-fallback\stdlib\typing.pyi (7ms)
[BG(1)]   parsing: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.10.2\dist\typeshed-fallback\stdlib\_typeshed\__init__.pyi [fs read 1ms] (8ms)
[BG(1)]   binding: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.10.2\dist\typeshed-fallback\stdlib\_typeshed\__init__.pyi (2ms)
[BG(1)]   parsing: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.10.2\dist\typeshed-fallback\stdlib\typing_extensions.pyi [fs read 1ms] (4ms)
[BG(1)]   binding: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.10.2\dist\typeshed-fallback\stdlib\typing_extensions.pyi (1ms)
[BG(1)]   parsing: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.10.2\dist\typeshed-fallback\stdlib\abc.pyi [fs read 0ms] (3ms)
[BG(1)]   binding: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.10.2\dist\typeshed-fallback\stdlib\abc.pyi (0ms)
[BG(1)] getSemanticTokens full at c:\Users\{$username}\Documents\Projects\test\functional_script.py (194ms)
Background analysis message: getSemanticTokens range
[BG(1)] getSemanticTokens range 0:0 - 4:10 at c:\Users\{$username}\Documents\Projects\test\functional_script.py (1ms)
Background analysis message: analyze
[BG(1)] analyzing: c:\Users\{$username}\Documents\Projects\test\functional_script.py ...
[BG(1)]   checking: c:\Users\{$username}\Documents\Projects\test\functional_script.py (5ms)
[BG(1)] analyzing: c:\Users\{$username}\Documents\Projects\test\functional_script.py (5ms)
Background analysis message: resumeAnalysis

super_slow_script.py, Language Server Version <= 2021.9.1, Functioning Normally

[Info  - 12:56:48 PM] Pylance language server 2021.9.1 (pyright e804f324) starting
[Info  - 12:56:48 PM] Server root directory: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.1\dist
[Info  - 12:56:48 PM] No configuration file found.
[Info  - 12:56:48 PM] No pyproject.toml file found.
[Info  - 12:56:48 PM] Setting pythonPath for service "test": "C:\Users\{$username}\AppData\Local\Programs\Python\Python39\python.exe"
[Warn  - 12:56:48 PM] stubPath c:\Users\{$username}\Documents\Projects\test\typings is not a valid directory.
[Info  - 12:56:48 PM] Assuming Python version 3.9
[Info  - 12:56:48 PM] Assuming Python platform Windows
Search paths for c:\Users\{$username}\Documents\Projects\test
  c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.1\dist\typeshed-fallback\stdlib
  c:\Users\{$username}\Documents\Projects\test
  c:\Users\{$username}\Documents\Projects\test\typings
  c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.1\dist\typeshed-fallback\stubs\...
  c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.1\dist\bundled\stubs
  c:\Users\{$username}\AppData\Local\Programs\Python\Python39\DLLs
  c:\Users\{$username}\AppData\Local\Programs\Python\Python39\Lib
  c:\Users\{$username}\AppData\Local\Programs\Python\Python39
  c:\Users\{$username}\AppData\Local\Programs\Python\Python39\Lib\site-packages
[Info  - 12:56:49 PM] Searching for source files
[Info  - 12:56:49 PM] Found 2 source files
[Info  - 12:56:49 PM] Background analysis(1) root directory: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.1\dist
[Info  - 12:56:49 PM] Background analysis(1) started
Background analysis message: setConfigOptions
Background analysis message: setImportResolver
Background analysis message: ensurePartialStubPackages
Background analysis message: setTrackedFiles
Background analysis message: markAllFilesDirty
Background analysis message: setFileOpened
Background analysis message: setFileOpened
Background analysis message: getSemanticTokens full
[BG(1)] getSemanticTokens full at c:\Users\{$username}\Documents\Projects\test\super_slow_script.py ...
[BG(1)]   parsing: c:\Users\{$username}\Documents\Projects\test\super_slow_script.py (18ms)
[BG(1)]   parsing: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.1\dist\typeshed-fallback\stdlib\builtins.pyi [fs read 3ms] (68ms)
[BG(1)]   binding: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.1\dist\typeshed-fallback\stdlib\builtins.pyi (31ms)
[BG(1)]   binding: c:\Users\{$username}\Documents\Projects\test\super_slow_script.py (1ms)
[BG(1)]   parsing: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.1\dist\typeshed-fallback\stdlib\typing.pyi [fs read 2ms] (23ms)
[BG(1)]   binding: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.1\dist\typeshed-fallback\stdlib\typing.pyi (9ms)
[BG(1)]   parsing: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.1\dist\typeshed-fallback\stdlib\_typeshed\__init__.pyi [fs read 1ms] (7ms)
[BG(1)]   binding: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.1\dist\typeshed-fallback\stdlib\_typeshed\__init__.pyi (2ms)
[BG(1)]   parsing: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.1\dist\typeshed-fallback\stdlib\typing_extensions.pyi [fs read 1ms] (2ms)
[BG(1)]   binding: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.1\dist\typeshed-fallback\stdlib\typing_extensions.pyi (1ms)
[BG(1)]   parsing: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.1\dist\typeshed-fallback\stdlib\os\__init__.pyi [fs read 1ms] (19ms)
[BG(1)]   binding: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.1\dist\typeshed-fallback\stdlib\os\__init__.pyi (7ms)
[BG(1)]   parsing: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.1\dist\typeshed-fallback\stdlib\re.pyi [fs read 1ms] (9ms)
[BG(1)]   binding: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.1\dist\typeshed-fallback\stdlib\re.pyi (1ms)
[BG(1)]   parsing: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.1\dist\typeshed-fallback\stdlib\abc.pyi [fs read 1ms] (2ms)
[BG(1)]   binding: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.1\dist\typeshed-fallback\stdlib\abc.pyi (0ms)
[BG(1)]   parsing: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.1\dist\typeshed-fallback\stdlib\enum.pyi [fs read 1ms] (2ms)
[BG(1)]   binding: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.1\dist\typeshed-fallback\stdlib\enum.pyi (0ms)
[BG(1)] getSemanticTokens full at c:\Users\{$username}\Documents\Projects\test\super_slow_script.py (348ms)
Background analysis message: getSemanticTokens full
[BG(1)] getSemanticTokens full at c:\Users\{$username}\Documents\Projects\test\functional_script.py ...
[BG(1)]   parsing: c:\Users\{$username}\Documents\Projects\test\functional_script.py (0ms)
[BG(1)]   binding: c:\Users\{$username}\Documents\Projects\test\functional_script.py (0ms)
[BG(1)] getSemanticTokens full at c:\Users\{$username}\Documents\Projects\test\functional_script.py (2ms)
Background analysis message: getSemanticTokens range
[BG(1)] getSemanticTokens range 0:0 - 33:44 at c:\Users\{$username}\Documents\Projects\test\super_slow_script.py (4ms)
Background analysis message: analyze
[BG(1)] analyzing: c:\Users\{$username}\Documents\Projects\test\super_slow_script.py ...
[BG(1)]   checking: c:\Users\{$username}\Documents\Projects\test\super_slow_script.py (5ms)
[BG(1)] analyzing: c:\Users\{$username}\Documents\Projects\test\super_slow_script.py (5ms)
[BG(1)] analyzing: c:\Users\{$username}\Documents\Projects\test\functional_script.py ...
[BG(1)]   checking: c:\Users\{$username}\Documents\Projects\test\functional_script.py (1ms)
[BG(1)] analyzing: c:\Users\{$username}\Documents\Projects\test\functional_script.py (1ms)
Background analysis message: resumeAnalysis
Background analysis message: getDiagnosticsForRange
Background analysis message: getDiagnosticsForRange
[FG] parsing: c:\Users\{$username}\Documents\Projects\test\super_slow_script.py (19ms)
[FG] parsing: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.1\dist\typeshed-fallback\stdlib\builtins.pyi [fs read 3ms] (67ms)
[FG] binding: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.1\dist\typeshed-fallback\stdlib\builtins.pyi (30ms)
[FG] binding: c:\Users\{$username}\Documents\Projects\test\super_slow_script.py (1ms)

super_slow_script.py, Language Server Version >= 2021.9.2, NOT Functioning Normally

[Info  - 12:46:05 PM] Pylance language server 2021.9.2 (pyright eca6f1a5) starting
[Info  - 12:46:05 PM] Server root directory: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.2\dist
[Info  - 12:46:05 PM] No configuration file found.
[Info  - 12:46:05 PM] No pyproject.toml file found.
[Info  - 12:46:05 PM] Setting pythonPath for service "test": "C:\Users\{$username}\AppData\Local\Programs\Python\Python39\python.exe"
[Warn  - 12:46:05 PM] stubPath c:\Users\{$username}\Documents\Projects\test\typings is not a valid directory.
[Info  - 12:46:06 PM] Assuming Python version 3.9
[Info  - 12:46:06 PM] Assuming Python platform Windows
Search paths for c:\Users\{$username}\Documents\Projects\test
  c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.2\dist\typeshed-fallback\stdlib
  c:\Users\{$username}\Documents\Projects\test
  c:\Users\{$username}\Documents\Projects\test\typings
  c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.2\dist\typeshed-fallback\stubs\...
  c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.2\dist\bundled\stubs
  c:\Users\{$username}\AppData\Local\Programs\Python\Python39\DLLs
  c:\Users\{$username}\AppData\Local\Programs\Python\Python39\Lib
  c:\Users\{$username}\AppData\Local\Programs\Python\Python39
  c:\Users\{$username}\AppData\Local\Programs\Python\Python39\Lib\site-packages
[Info  - 12:46:06 PM] Searching for source files
[Info  - 12:46:06 PM] Found 2 source files
[Info  - 12:46:06 PM] Background analysis(1) root directory: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.2\dist
[Info  - 12:46:06 PM] Background analysis(1) started
Background analysis message: setConfigOptions
Background analysis message: setImportResolver
Background analysis message: ensurePartialStubPackages
Background analysis message: setTrackedFiles
Background analysis message: markAllFilesDirty
Background analysis message: setFileOpened
Background analysis message: setFileOpened
Background analysis message: getSemanticTokens full
[BG(1)] getSemanticTokens full at c:\Users\{$username}\Documents\Projects\test\super_slow_script.py ...
[BG(1)]   parsing: c:\Users\{$username}\Documents\Projects\test\super_slow_script.py (19ms)
[BG(1)]   parsing: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.2\dist\typeshed-fallback\stdlib\builtins.pyi [fs read 4ms] (73ms)
[BG(1)]   binding: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.2\dist\typeshed-fallback\stdlib\builtins.pyi (30ms)
[BG(1)]   binding: c:\Users\{$username}\Documents\Projects\test\super_slow_script.py (1ms)
[BG(1)]   parsing: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.2\dist\typeshed-fallback\stdlib\typing.pyi [fs read 2ms] (23ms)
[BG(1)]   binding: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.2\dist\typeshed-fallback\stdlib\typing.pyi (8ms)
[BG(1)]   parsing: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.2\dist\typeshed-fallback\stdlib\_typeshed\__init__.pyi [fs read 3ms] (8ms)
[BG(1)]   binding: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.2\dist\typeshed-fallback\stdlib\_typeshed\__init__.pyi (2ms)
[BG(1)]   parsing: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.2\dist\typeshed-fallback\stdlib\typing_extensions.pyi [fs read 0ms] (2ms)
[BG(1)]   binding: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.2\dist\typeshed-fallback\stdlib\typing_extensions.pyi (1ms)
[BG(1)]   parsing: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.2\dist\typeshed-fallback\stdlib\os\__init__.pyi [fs read 1ms] (20ms)
[BG(1)]   binding: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.2\dist\typeshed-fallback\stdlib\os\__init__.pyi (6ms)
[BG(1)]   parsing: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.2\dist\typeshed-fallback\stdlib\re.pyi [fs read 1ms] (3ms)
[BG(1)]   binding: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.2\dist\typeshed-fallback\stdlib\re.pyi (1ms)
[BG(1)]   parsing: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.2\dist\typeshed-fallback\stdlib\abc.pyi [fs read 1ms] (1ms)
[BG(1)]   binding: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.2\dist\typeshed-fallback\stdlib\abc.pyi (1ms)
[BG(1)]   parsing: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.2\dist\typeshed-fallback\stdlib\enum.pyi [fs read 1ms] (3ms)
[BG(1)]   binding: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.2\dist\typeshed-fallback\stdlib\enum.pyi (1ms)
[BG(1)] getSemanticTokens full at c:\Users\{$username}\Documents\Projects\test\super_slow_script.py (22448ms)
[Info  - 12:46:29 PM] [BG(1)] Long operation: getSemanticTokens full at c:\Users\{$username}\Documents\Projects\test\super_slow_script.py (22448ms)
Background analysis message: getSemanticTokens full
[BG(1)] getSemanticTokens full at c:\Users\{$username}\Documents\Projects\test\functional_script.py ...
[BG(1)]   parsing: c:\Users\{$username}\Documents\Projects\test\functional_script.py (4ms)
[BG(1)]   binding: c:\Users\{$username}\Documents\Projects\test\functional_script.py (2ms)
[BG(1)] getSemanticTokens full at c:\Users\{$username}\Documents\Projects\test\functional_script.py (10ms)
Background analysis message: getSemanticTokens range
[BG(1)] getSemanticTokens range 0:0 - 33:44 at c:\Users\{$username}\Documents\Projects\test\super_slow_script.py (5ms)
Background analysis message: analyze
[BG(1)] analyzing: c:\Users\{$username}\Documents\Projects\test\super_slow_script.py ...
[BG(1)]   checking: c:\Users\{$username}\Documents\Projects\test\super_slow_script.py (2143ms)
[Info  - 12:46:31 PM] [BG(1)] Long operation: checking: c:\Users\{$username}\Documents\Projects\test\super_slow_script.py (2143ms)
[BG(1)] analyzing: c:\Users\{$username}\Documents\Projects\test\super_slow_script.py (2143ms)
[Info  - 12:46:31 PM] [BG(1)] Long operation: analyzing: c:\Users\{$username}\Documents\Projects\test\super_slow_script.py (2143ms)
Background analysis message: resumeAnalysis
[BG(1)] analyzing: c:\Users\{$username}\Documents\Projects\test\functional_script.py ...
[BG(1)]   checking: c:\Users\{$username}\Documents\Projects\test\functional_script.py (1ms)
[BG(1)] analyzing: c:\Users\{$username}\Documents\Projects\test\functional_script.py (1ms)
Background analysis message: resumeAnalysis
Background analysis message: getDiagnosticsForRange
Background analysis message: getDiagnosticsForRange
[FG] parsing: c:\Users\{$username}\Documents\Projects\test\super_slow_script.py (21ms)
[FG] parsing: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.2\dist\typeshed-fallback\stdlib\builtins.pyi [fs read 3ms] (80ms)
[FG] binding: c:\Users\{$username}\.vscode\extensions\ms-python.vscode-pylance-2021.9.2\dist\typeshed-fallback\stdlib\builtins.pyi (34ms)
[FG] binding: c:\Users\{$username}\Documents\Projects\test\super_slow_script.py (1ms)

Key line:

[Info  - 12:46:29 PM] [BG(1)] Long operation: getSemanticTokens full at c:\Users\{$username}\Documents\Projects\test\super_slow_script.py (22448ms)

Additional Notes

  • This is a stripped down version of the codebase I am working on. When I run this on my full codebase, the key line in question takes 391713ms to complete. During this time, syntax highlighting fails to function on any other python script in my window.
    • Any update to the slowdown file in question, either opening it or making changes to it requires full rerun, significantly affecting productivity.
  • Removing either of the following lines returns Pylance to standard operating speeds:
    • text = re.sub(r'include\s+' + file, new_text[0], text)
    • text = re.sub(r'include_once\s+'+file, new_text[0], text)
    • The strange thing is that removing EITHER of these lines resolves the issue, leaving one of them in is fine but both is not.
  • Removing "python.analysis.logLevel": "Trace" does not significantly affect speeds
  • Removing get_text_in_file(...) reduces time to 7295ms
    • Not replacing, just leaving Undefined error.

Current Work Around

Revert to Version 2021.9.1, no serious impact on productivity
@erictraut
Copy link
Contributor

erictraut commented Oct 25, 2021 •
edited

Thanks for the bug report. The details you've provided are very helpful. I'm able to repro the problem. The slowdown isn't nearly as bad on my machine, but it's still taking much longer than I would expect. I'll investigate further.

@erictraut erictraut mentioned this issue Oct 25, 2021
Closed
@judej judej added the needs investigation Could be an issue - needs investigation label Oct 26, 2021
@github-actions github-actions bot removed the triage label Oct 26, 2021
@erictraut
Copy link
Contributor

I've implemented some additional optimizations that improve the analysis performance for this code. Before these optimizations, it took >5800ms on my machine, and it now takes about 820ms. This will be included in the next release of pylance.

@erictraut erictraut added fixed in next version (main) A fix has been implemented and will appear in an upcoming version and removed needs investigation Could be an issue - needs investigation labels Nov 18, 2021
@bschnurr
Copy link
Member

bschnurr commented Dec 2, 2021

This issue has been fixed in version 2021.12.0, which we've just released. You can find the changelog here: CHANGELOG.md

@bschnurr bschnurr closed this as completed Dec 2, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
fixed in next version (main) A fix has been implemented and will appear in an upcoming version
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@judej @bschnurr @erictraut @ScottNealon