[Metrics Alerts] Send a recovery alert when an alert returns to an OK state

[Zacqary]

⚠️ Depends on #49405

As brought up in #64080

Send an alert like

My Alert - elasticsearch-master-0 is in a state of OK [Recovered]

Because system.load.1 is no longer greater than a threshold of 1.0 
(current value is 0.95); system.mem.usage is no longer greater than
a threshold of 85 (current value is 76)

but only once when the alert goes green. Don't renotify after.

[elasticmachine]

Pinging @elastic/logs-metrics-ui (Team:logs-metrics-ui)

[Zacqary]

Related: #49405

[Zacqary]

I'm not sure if this will work very well with renotify periods until full support for recovery action groups is added to the alerting plugin. We have to consolidate all of our alert fired/alert recovered/etc. messages within a single default action group, and the renotify period suppresses messages based on the action group ID, not on the alert state or any other metadata. This means recovery messages may get lost if the state changes between recovery intervals.

[Zacqary]

Marking as blocked, we're going to wait on the alerting plugin to implement a native solution before proceeding (#49405)