Test silent data loss from CCS limitations is no longer silent

[stacey-gammon]

#94965 (comment) describes the current limitations with CCS. Some of those issues result in data not showing up from remote clusters. ES recently merged support to ensure a warning is returned from the API in these situations. Kibana apps should now verify that they are correctly capturing and displaying the warning to users.

In situations where the app is using data search services, this may be blocked on #98764.

Area	CCS Limitation	Is a warning shown?
Discover (cc @timroes)	Data from incompatible cluster is silently lost
Canvas (cc @clintandrewhall)	Elasticsearch queries will timeout
Canvas (cc @clintandrewhall)	SQL queries will timeout
Canvas (cc @clintandrewhall)	EQL queries will timeout
Logs (cc @jasonrhodes )	Data loss from incompatible clusters
SIEM (cc @XavierM )	Timeline and Hosts page are missing data.
SIEM (cc @XavierM )	Alerts do not generate data.

[elasticmachine]

Pinging @elastic/kibana-presentation (Team:Presentation)

[elasticmachine]

Pinging @elastic/kibana-app (Team:KibanaApp)

[elasticmachine]

Pinging @elastic/kibana-app-services (Team:AppServices)

[elasticmachine]

Pinging @elastic/siem (Team:SIEM)

[timductive]

@crob611 @jb1b84 @kmartastic tagging you for context, I'm not sure if this is something we are waiting on App Services for.

[matschaffer]

Should we maybe add a row to capture the case where a viz might work on some subset of clusters?

For example we have a very large CCS deployment. Some of the remotes are currently disconnected (need to have certs reconfigured). So viz responses often contain something like this:

  "_clusters": {
    "total": 52,
    "successful": 49,
    "skipped": 3
  },

It'd be great to have the "skipped: 3" called out visually somewhere. Maybe a small warning icon or something?

[stacey-gammon]

I was mistaken. I thought ES already had support for returning errors in the CCS silent data loss situation, but that is not implemented yet, so these scenarios cannot be tested. Apologies for the needless ping. Closing until that is implemented.

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)