Expression Language Test Plan (Short Term)

[stacey-gammon]

Goal

Be confident that the expression language is stable enough to release into the wild, on top of a GA canvas, and the current visualizations.

Target: Have a plan ready within 2 - 4 weeks

Automation Plan

Track test completion times in build-stats (https://github.com/elastic/infra/issues/9270)

Doing this will allow us to view average test run times and how they change over time. Averaging them together will hopefully smooth out anomalies coming from network issues, etc. In general, it should help us recognize points in time where changes caused a performance hit.

This route won't help overall though with judging the timing with toggle on and toggle off, since the toggle will be off on master.

I'm (@stacey-gammon) currently investigating this.

To at least get the test completion times locally, this PR does the trick: https://github.com/elastic/kibana/pull/30285/files

Add tests for more complicated expressions

If we can set up the former so we can see the time over a given run, the next step would be to add some tests that include more interesting expressions that are specifically designed. For instance:

serverFn | commonFn | serverFn | commonFn

We could also create some test expressions that do something like send back a payload of data of the size requested.

This isn't necessary as part of the test plan. We can do this manually as well. It just might not be that much more effort to build it as a test and then it makes it much easier to write up steps that everyone can try out on their own environments.

We should explore #22139 too but I don't know if there is enough time to do this within 2 - 4 weeks.

Manual Test Plan

Copied over from the Plugin Review Issue Template

• Elasticsearch and Kibana Configuration tests
  • Multiple Kibana instances pointing to the same index
  • Multiple Kibana instances pointing to different indexes
• Test with cross cluster search or cross cluster replication?
• Performance and load tests
  • Many fields
  • Many indexes
  • Lots of data
  • Lots of saved objects
  • Large saved objects
  • Browser left running overnight (endurance tests)
• Tested behind a reverse proxy
  • Have some really rough directions on how to set this up locally here: https://docs.google.com/document/d/1bgp8DUUGi1gGvsCUOsQU4tAJoxdHoOwbzyu31KYqtcQ/edit
    really rough...

Going to try to go through some of the performance benchmarks here: https://docs.google.com/spreadsheets/d/1kPTqGKn1thG0TcZl5-BYhDN6Em40Ac7wrRosZDU0w1o/edit#gid=1802315937
-- nothing is set up yet though.

[elasticmachine]

Pinging @elastic/kibana-app

[ppisljar]

closing this as most of the above points have been done and we have separate issues tracking any remaining work