High memory usage on macOS (M1) in VS Code (HashiCorp Extension)

[ghost]

Server Version

0.31.1

Terraform Version

Terraform v1.4.6
on darwin_arm64

Client Version

Version: 1.77.3 (Universal)
Commit: 704ed70d4fd1c6bd6342c436f1ede30d1cff4710
Date: 2023-04-12T09:19:37.325Z
Electron: 19.1.11
Chromium: 102.0.5005.196
Node.js: 16.14.2
V8: 10.2.154.26-electron.0
OS: Darwin arm64 22.4.0
Sandboxed: No

Terraform Configuration Files

We use Terragrunt to manage configuration across our deployments, with two repositories:

• a "modules" repository that contains reusable modules
• an "inventories" repository that contains the deployment specs, by application/subsystem (where each application/subsystem is composed of multiple modules)

Log Output

GPG-encrypted gist: https://gist.github.com/jonbackhaus-mt/bafcffc3a1cbc510934d50bb6017c216

I attempted to profile terraform-ls for CPU and memory. The CPU profiler created an empty file; the memory profiler never even created a file.

Expected Behavior

Based on other sources, I would expect the memory utilization to be <1 GB.

Actual Behavior

Memory utilization is currently ~10 GB.

Steps to Reproduce

1. Open VS Code workspace, which includes both the "modules" repository and the "inventories" repository as top-level folders in the workspace.
2. terraform-ls immediately starts processing in the background and quickly starts consuming resources (high CPU usage; memory footprint grows from ~2 GB to ~10 GB).

Continuation of #1100, which has been locked

[radeksimko]

We intend to further investigate the performance issues, although the realistic expectation to set here is that amount of memory and CPU will generally always be proportional to the size of the workspace being processed, number of providers used etc., not static.

In the interest of transparency (and hope of building some empathy on all sides), I will say that we have not yet invested much effort into optimising for the higher end of workspace sizes. Identifying the different scales (esp. the higher end) and measuring how different parts of the language server behave under those scales is what we plan to do as part of #1056 We will also reach out to you all as part of that work.

The only workaround for the moment is to avoid opening large workspaces and rather open individual folders nested within the tree, or at least smaller part of the tree. I appreciate this may not be the answer you're looking for, but I hope the context is helpful.

[jpogran]

We know this has been a frustrating experience and appreciate your understanding while we gathered feedback and examples to diagnose this issue. We've created hashicorp/vscode-terraform#1557 as a central place to see updates on what we are doing to address performance issues in terraform-ls in both the short and long term and pinned it to the repo. We'll be adding more detail there as we implement fixes. As we work on this we'll be recording the content and then closing the individual issues so that everyone has one place to look at instead of searching for individual tickets for updates.

[jpogran]

We've released 2.27.2 which contains two fixes (#1369, #1372) in our continuing efforts to address this issue.

With the fix, we should be back to terraform-ls v0.29.2 levels of CPU usage while maintaining the memory improvements of v0.29.3.

If you have the time, please give it a try and let us know how it works for you. Please open a new issue rather than replying here, as this issue has gotten so many replies with different experiences it is hard to process.

I am going to close this as we're using hashicorp/vscode-terraform#1557 as a central tracking issue, so subscribe to that to see continued updates.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.