[Search] Telemetry makes too many requests

[lukasolson]

Kibana version: 7.9

Elasticsearch version: 7.9

Describe the bug:

In #70677, we introduced some telemetry on search requests (duration, success/failure, etc.). We make requests to the backend telemetry service. However, this introduces an unnecessary number of extra requests, as we could easily collect the telemtetry on the server rather than require an additional call from the browser.

Screenshots (if relevant):

[elasticmachine]

Pinging @elastic/kibana-app-arch (Team:AppArch)

[lizozom]

@elastic/kibana-app-arch @elastic/kibana-qa

This issue causes O(number of panels) of telemetry to be sent for each dashboard search.
Would like to get this in for 7.9.0. What's your opinion?

[LeeDr]

Are we logging the duration of every dashboard query or refresh? Or min/max/avg over some time period? I guess we need to see what the change looks like to know if it's something we should change to the server in 7.9.0 vs disabling for this release.