fix(cf): CF forkjoin threading improvements

[german-muzquiz]

CF provider creates a ForkJoinPool for each accounts defined, with a default max parallelism of 16.
This causes that in case of a large amount of accounts the number of threads increases significantly, for example having 50 accounts means 50 Applications FJP + 50 Routes FJP = 100 x 16 = 1600 threads.

This causes a lot of unesessary overhead because the only threads actually being used at any given moment are the ones used by running caching agents, which means that most of these threads are not used all the time.

The given change moves the ForkjoinPool to be a singleton bean at the provider level, so there's only one pool for all accounts, with a configurable max parallelism. In this way the full capacity of the pool can be used regardless of which agents are running at any given moment.

[zachsmith1]

LGTM -- One note: you will need to create a PR in halyard and update the credentials validator once this PR is merged/versioned. Anytime we touch the credentials in CD, halyard will need the accompanying change

[german-muzquiz]

@fieldju this is a candidate to be backported to version 1.23: presumably with the introduction of kork-credentials for cloudfoundry in 1.23, there's a period of time when there are more ManagedAccount cloudfoundry objects than real accounts, waiting to be garbage collected. Since each ManagedAccount object creates 2 ForkJoinPool objects with 16 threads each, this causes high spikes of threads while the garbage collector kicks-in, potentially bringing down clouddriver pod.

This is considered a performance regression for cloudfoundry provider in 1.23 because the spikes of threads were not seen at least in 1.20. This is documented in this bug.

Instead of assuming this is caused by the introduction of kork-credentials change and revert that change, this PR already fixes the root cause of the issue by defining a singleton ForkJoinPool for all cloudfoundry accounts.

The implication of backporting this PR will cause the cloudfoundry setting cloudfoundry.accounts[x].maxCapiConnectionsForCache to be ignored in the change from 1.23.3 to 1.23.4.

There is not an easy way to create test cases for performance regressions like this, but I did a manual test with the same configuration reported in the bug and I only see the expected amount of threads controlled by the new setting cloudfoundry.apiRequestParallelism:

[fieldju]

@Mergifyio backport release-1.23.x

[mergify]

Command backport release-1.23.x: success

Backports have been created

• #5161 fix(cf): CF forkjoin threading improvements (bp #5071) has been created for branch release-1.23.x