[Remote state] Need a tool to update cluster state stored in remote state

[dhwanilpatel]

Is your feature request related to a problem? Please describe

Sometime cluster state stored in remote store gets corrupted due to some code bug and it enters in state where we manually need to change the state stored in remote state to recover.

One example could be, some issue in to/from XContent of any custom metadata could lead to corrupted state in remote. There was one such bug in one of the custom metadata where manual update of cluster state was required. [Ref: #11679 ] .

Similar situation can occur for remote state as well, we will need tool to manually update the cluster state.

Describe the solution you'd like

There are some tools for updating local cluster state on disk, to start with we can support those cases in remote state [remove-settings/remove-customs].

Ref:

https://github.com/opensearch-project/OpenSearch/blob/main/server/src/main/java/org/opensearch/cluster/coordination/NodeToolCli.java#L62-L63

Related component

Cluster Manager

Describe alternatives you've considered

No response

Additional context

No response

[peternied]

[Triage - attendees 1 2 3 4]
Thanks for filing this issue, please feel free to submit a pull request.