Adjust(able) job scheduler parallelism

[radeksimko]

Context

While discussing various reported performance issues and potential root causes, one of the theories which came up is the effective synchronous nature of the job scheduler.

terraform-ls/internal/langserver/handlers/service.go

Lines 472 to 480 in 1cc56a9

	svc.lowPrioIndexer = scheduler.NewScheduler(svc.stateStore.JobStore, 1, job.LowPriority)
	svc.lowPrioIndexer.SetLogger(svc.logger)
	svc.lowPrioIndexer.Start(svc.sessCtx)
	svc.logger.Printf("started low priority scheduler")
	svc.highPrioIndexer = scheduler.NewScheduler(svc.stateStore.JobStore, 1, job.HighPriority)
	svc.highPrioIndexer.SetLogger(svc.logger)
	svc.highPrioIndexer.Start(svc.sessCtx)
	svc.logger.Printf("started high priority scheduler")

As per the snippet linked above, we do set the parallelism of the two scheduler instances to just 1. This was an intentional decision made in the past, in an attempt to reduce the amount of work happening at the same time to (in turn) reduce the CPU/memory spikes. Some users continue reporting high CPU and memory usage, some just report slowness.

It is possible that increasing parallelism could speed up the processing, but naturally also increase the CPU and memory usage further.

Additionally, there is a large number of jobs which depend on each other (e.g. parsing has to finish before early decoding can start; we need to have decoded provider requirements before we know which providers to pull schema for etc.), meaning that increasing parallelism may not bring as much benefit.

Proposal

• Consider increasing the parallelism based on runtime.NumCPU() - i.e. accordingly to the amount of CPU available
• Consider making the parallelism configurable

Pre-requisite

It is important to first gain some confidence in the tracing logic as per #1056 or #1340 such that we don't just do this blindly but can confirm (or deny) with data, that increased (or adjusted) parallelism has the desired positive effect.