Provide opt out from preloaded schemas

[radeksimko]

Current Version

0.16.1

Background

Users generally expect completion, hover and most of other functionality to work "out of the box". This is why from X.Y.Z we started pre-baking schemas of official and partner providers (~150). While this makes it convenient for users and makes most of the functionality work almost straight away after initial load, it comes at a cost.

That cost is increased memory usage.

At this point we build in 34 (official) + 115 (partner) providers. This has the following impact on memory usage:

• total memory (represented as MemStats.Sys) before allocation: 69 MiB
• total memory (represented as MemStats.Sys) after allocation: 206 MiB

Preloading all mentioned schemas increases total memory usage by approximately 137 MiB. This is measured before and after this block of code:

terraform-ls/internal/schemas/preload_schemas.go

Lines 10 to 13 in c4ea46f

	pOut, vOut, err := PreloadedProviderSchemas()
	if pOut == nil || err != nil {
	return err
	}

So it seems to be just a result of unmarshaling the (currently ~26MB on disk) JSON file.

For many users this may be ok cost to pay for the convenience, for some it may not.

Use Case

User should have a way of opting out from builtin provider schema and reduce the memory footprint of the language server.

Proposal

Introduce a new flag -preload-schemas=false which will prevent builtin schema from ever being unmarshaled or otherwise loaded into memory.

[radeksimko]

We have significantly reduced the memory usage as part of #1071 which is included in the latest release (0.29.3). We now only load the schemas into the memory dynamically when needed. Therefore there should be no need to opt out anymore.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.