Enhance initial out of the box experience (functionality available without terraform init)

[radeksimko]

Problem Statement

Historical Context

With extension v2 and initial release of the language server the expected end-user workflow was that user would run terraform init to allow the language server to obtain all the metadata to use for completion, hover etc.

This does not represent a great "out-of-the-box experience" given that the extra step is not obvious to many users and even if it is, init will require the user to have valid config and access to any remote state at the time of init both of which add even more steps before the user can actually craft code.

We initially attempted to notify the user about this requirement via popup, which many users (rightly) found distracting and the popup was eventually removed in 0.16.0 as we also built a cache which holds schema for the latest versions of all official + partner providers (~150) at the time of the release.

See hashicorp/terraform-ls#354 and hashicorp/terraform-ls#424

Today

Most users receive reasonable "out of the box" experience thanks to the pre-built schema (without having to run init).

Many users can however still often experience many inaccuracies or entirely missing data depending on what versions of what providers do they use, unless they run init as part of their day-2-day workflow.

Less user-visible impact

The model of us pre-building schema for 150+ providers also doesn't scale very well (currently adds around 6-7 minutes to every release of LS which will only increase with more providers with the whole schema JSON having ~30MB) and we can't really apply the same approach to community providers.

Expected User Experience

Happy Path

User will be provided with all provider-dependent features using accurate data, such as

• completion within provider, resource or data blocks
• hover over any elements within provider, resource or data blocks
• go-to-references of resource or data blocks
• go-to-definition of resource or data blocks
• semantic highlighting of provider, resource or data blocks

as long as they use providers published in the Terraform Registry and have access to internet at the time of editing configuration.

Unhappy Path

User will be provided with a hint in the status bar that required Terraform version is not installed and will have option to install the latest Terraform version matching the requirement.

User will be provided with contextual code action to install a provider if no schema is found. The code action will be attached to the relevant provider, resource or data blocks and also to the relevant entry in required_providers block.

Proposal

• Display Terraform path, version and constraint in the status bar #697
• Expose targeted terraform init as code action
• Search registry providers via completion terraform-ls#194
• Use registry schema cache if no initialized data for providers terraform-ls#193

Resolving these would also improve pre-init experience but it might be a stretch:

• textDocument/complete: Get schemas for invalid configs terraform-ls#56
• textDocument/complete: Provide completion in incomplete blocks terraform-ls#57
• Source provider schemas directly from binaries via gRPC terraform-ls#355

Blocking Upstream Work

• Persisted and exposed provider schema via Terraform Registry API
• Targeted terraform init per provider in Terraform CLI