[Meta] Dashboard Migration Smoke Tests

[ThomThomson]

Smoke Tests

Functional migration smoke test suites should be created for each major panel type on dashboard (e.g. Lens, Visualize, TSVB, Maps). These smoke test suites should focus on detecting possible error cases which a missing migration could cause.

Each of these smoke test suites should:

1. Import a dashboard or a collection of dashboards with representative samples of the embeddable type covered, including panels which are by value, are by reference, contain drilldowns, etc.
2. Open each dashboard.
3. Confirm that all panels render.
4. Confirm that there are no error embeddables on the dashboard.
5. Confirm that each panel has an 'edit' action when it is in edit mode.
6. Any other smoke tests that could be added specifically for the panel type.

Embeddable Types

These tests should be written to cover

• Lens
• TSVB. (@crob611 )
• Other Visualize Panels. (@poffdeluxe)
• Maps. (@clintandrewhall)

For any coverage questions around Visualize and TSVB, @stratoula may be able to help.
For any coverage questions around Maps, @nreese may be able to help.
For any questions around how to implement the functional tests / importing files, @bhavyarm is an expert!

Instructions

The process for creating these smoke tests should be as follows:

1. Bootstrap and start Kibana 7.12.1 locally
2. Use esArchiver to load in a dataset that can be easily used in functional tests, for instance:
   • NODE_TLS_REJECT_UNAUTHORIZED=0 node --no-warnings scripts/es_archiver load getting_started/shakespeare --config test/functional/config.js --es-url=http://elastic:changeme@localhost:9200 --kibana-url=http://elastic:changeme@localhost:5601
   • Note: If you're using kibana with the random three-letter prefix, you'll want to add the lets to the kibana url and make sure it has a trialing slash else you'll get an error. For example: http://elastic:changeme@localhost:5601/vpq/
   • This could also be done through the functional test runner, via letting the test run load data, then quitting the run before it can clear it.
3. Build a representative dashboard with a sample of all the 7.12.1 features on that panel type.
4. Export that dashboard.
5. Add the ndjson export to the x-pack/test/functional/apps/dashboard/migration_smoke_tests/exports folder
6. Use the previously written smoke tests in x-pack/test/functional/apps/dashboard/migration_smoke_tests as an example of how to import the dashboard and cover its features.

[elasticmachine]

Pinging @elastic/kibana-presentation (Team:Presentation)

[ThomThomson]

Closing this as most of the vulnerable chart types are covered with migration smoke tests.