[Backport 2.4]Revert "Cluster manager task throttling feature [Final PR] (#5071)

[Bukhtawar]

This reverts commit e0d1c2e.

Signed-off-by: Bukhtawar Khan bukhtawa@amazon.com

Description

[Describe what this change achieves]

Issues Resolved

[List any issues this PR will resolve]

Check List

• New functionality includes testing.
  • All tests pass
• New functionality has been documented.
  • New functionality has javadoc added
• Commits are signed per the DCO using --signoff
• Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[andrross]

Why is this being reverted?

[andrross]

The release notes should be updated too:

https://github.com/opensearch-project/OpenSearch/blob/2.4/release-notes/opensearch.release-notes-2.4.0.md?plain=1#L8

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/6768/
• CommitID: 8f7b8ab
  Please examine the workflow log, locate, and copy-paste the failure below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/6770/
• CommitID: 19d15d0
  Please examine the workflow log, locate, and copy-paste the failure below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/6773/
• CommitID: d15d45e
  Please examine the workflow log, locate, and copy-paste the failure below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: SUCCESS ✅
• URL: https://build.ci.opensearch.org/job/gradle-check/6775/
• CommitID: 95ab855

[andrross]

I found the following explanation for this revert out-of-band. Going to capture it here for posterity in case folks want to understand why this was reverted from the 2.4 release:

we are modifying the way data nodes submit task to cluster manager and hence it requires bit data nodes and cluster manager understand the new format. Also since the cluster manager could now potentially throw a throttling exception it required data nodes to understand the new exception and hence data node and master node had to be coupled. However we should ideally have allowed mixed clusters to work without any BWC complexity, which was the miss

@Bukhtawar please correct or update as necessary