Batch execute template and pipeline cluster state operations

[dakrone]

This commit changes the cluster state operations for templates (legacy, component, and composable) as well as ingest pipelines to be bulk executed. This means that they can be processed much faster when creating/updating many simultaneously.

Relates to #77505

[elasticmachine]

Pinging @elastic/es-data-management (Team:Data Management)

[elasticsearchmachine]

Hi @dakrone, I've created a changelog YAML for you.

[martijnvg]

The PR looks good to me.

I do wonder about testing.
Each type of operation is tested (e.g. putting a pipeline) on the unit level and integration level. I think batching implementation is tested as well in ClusterStateService is tested as well. So I think we're good here, right?

[martijnvg]

👍 to see these listeners being replaced by ActionListener

[dakrone]

So I think we're good here, right?

I think so too, I think adding a test specifically for this would add a ton of complexity.

[martijnvg]

I think adding a test specifically for this would add a ton of complexity.

Yes, I had a similar feeling that these tests require a lot of work without adding much benefit.

[martijnvg]

LGTM 👍

[DaveCTurner]

For future reference you can use ClusterStateTaskExecutorUtils to simplify the job of writing tests that execute batches of cluster state update tasks. I don't think it's that necessary here, the batching is pretty straightforward, but e.g. in #86446 I'm proposing a change to the IngestServiceTests which involves running the executor code too. The tests still only ever execute singleton batches, but hopefully it's obvious how to extend them to execute multiple tasks in a single batch too.