Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Dashboard] Adds migration smoke tests for TSVB panels on dashboard #105655

Merged
merged 7 commits into from
Jul 26, 2021

Conversation

crob611
Copy link
Contributor

@crob611 crob611 commented Jul 14, 2021

Summary

This adds some smoke tests for migration of TSVB By Value panels on a Dashboard.

There are two exported dashboards that get tested by being imported through Saved Object import and confirming the import was successful, and by visiting the dashboard and confirming there are no error panels.

One is created using 7.12.1. It has panels for Time Series, Metric, Top N, Gauge, Markdown, Table, and a Time Series w/ Annotations. It also includes some drilldowns on some panels and tests that those drill downs are also properly migrated.

The second is created using 7.13.3. It has a Time Series panel that is not using a Kibana Index Pattern.

@crob611 crob611 added review Feature:Dashboard Dashboard related features Team:Presentation Presentation Team for Dashboard, Input Controls, and Canvas loe:medium Medium Level of Effort v8.0.0 release_note:skip Skip the PR/issue when compiling release notes impact:high Addressing this issue will have a high level of impact on the quality/strength of our product. v7.15.0 labels Jul 14, 2021
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-presentation (Team:Presentation)

@crob611
Copy link
Contributor Author

crob611 commented Jul 14, 2021

@stratoula Would also love any feedback if you think of any other cases that might should be covered.

@stratoula
Copy link
Contributor

@crob611 it looks awesome! My only feedback is that maybe we should test a 7.13.3 TSVB viz which uses index pattern.
All the old visualizations (before 7.13) when migrated to newer versions they are using the old mechanism (not index pattern mode). So this functionality I think is checked from the 7.12 SOs. For me it makes more sense, to create a 7.13.3 migration which uses the index pattern mode and check that is migrated correctly.

Copy link
Contributor

@ThomThomson ThomThomson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code only review, so I haven't seen the TSVB charts used in these smoke tests - but from what I've heard, you and the Kibana app team were able to create a representative sample.

It will be good to get all of these tests into master, so that we can clean them up all at once after the testing harness is merged, as part of #106290. This PR duplicates coverage of dashboards from 7.13.1, and 7.12.1, so ideally we can write a generic system which loads all of these dashboards at once via kbn_archiver, then loops through them and runs the same smoke tests. At that point, testing migrations from a new version -> latest will be as easy as appending a new kbn_archiver export to the end of the file.

@crob611
Copy link
Contributor Author

crob611 commented Jul 22, 2021

@elasticmachine merge upstream

@crob611 crob611 added the auto-backport Deprecated - use backport:version if exact versions are needed label Jul 22, 2021
@crob611
Copy link
Contributor Author

crob611 commented Jul 22, 2021

@elasticmachine merge upstream

@crob611
Copy link
Contributor Author

crob611 commented Jul 23, 2021

@elasticmachine merge upstream

@crob611
Copy link
Contributor Author

crob611 commented Jul 26, 2021

@elasticmachine merge upstream

@kibanamachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
actions 117 - -117
advancedSettings 22 - -22
alerting 234 - -234
apm 39 - -39
apmOss 4 - -4
banners 9 - -9
bfetch 62 - -62
canvas 5 - -5
cases 407 - -407
charts 159 - -159
cloud 21 - -21
core 1080 - -1080
dashboard 137 - -137
dashboardEnhanced 50 - -50
dashboardMode 11 - -11
data 3162 - -3162
dataEnhanced 16 - -16
dataVisualizer 104 - -104
devTools 8 - -8
discover 55 - -55
discoverEnhanced 37 - -37
embeddable 384 - -384
embeddableEnhanced 14 - -14
encryptedSavedObjects 28 - -28
enterpriseSearch 2 - -2
esUiShared 90 - -90
eventLog 70 - -70
expressionError 12 - -12
expressionImage 4 - -4
expressionRepeatImage 28 - -28
expressionRevealImage 4 - -4
expressions 1569 - -1569
expressionShape 90 - -90
features 97 - -97
fileUpload 128 - -128
fleet 1033 - -1033
globalSearch 14 - -14
home 70 - -70
indexLifecycleManagement 4 - -4
indexManagement 157 - -157
indexPatternFieldEditor 29 - -29
infra 22 - -22
inspector 78 - -78
kibanaLegacy 62 - -62
kibanaReact 230 - -230
kibanaUtils 359 - -359
lens 190 - -190
licenseApiGuard 8 - -8
licenseManagement 3 - -3
licensing 42 - -42
lists 143 - -143
management 40 - -40
maps 203 - -203
mapsEms 75 - -75
metricsEntities 6 - -6
ml 274 - -274
monitoring 10 - -10
navigation 31 - -31
newsfeed 17 - -17
observability 219 - -219
osquery 11 - -11
presentationUtil 136 - -136
remoteClusters 4 - -4
reporting 132 - -132
rollup 20 - -20
ruleRegistry 60 - -60
runtimeFields 19 - -19
savedObjects 199 - -199
savedObjectsManagement 85 - -85
savedObjectsTagging 50 - -50
savedObjectsTaggingOss 50 - -50
screenshotMode 17 - -17
security 51 - -51
securityOss 9 - -9
securitySolution 1245 - -1245
share 83 - -83
snapshotRestore 22 - -22
spacesOss 5 - -5
stackAlerts 4 - -4
taskManager 25 - -25
telemetryCollectionManager 29 - -29
telemetryCollectionXpack 1 - -1
telemetryManagementSection 13 - -13
timelines 763 - -763
triggersActionsUi 228 - -228
uiActions 88 - -88
uiActionsEnhanced 147 - -147
urlForwarding 15 - -15
usageCollection 16 - -16
visTypeTimeseries 10 - -10
visualizations 229 - -229
total -15344

Any counts in public APIs

Total count of every any typed public API. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats any for more detailed information.

id before after diff
bfetch 1 - -1
charts 2 - -2
core 148 - -148
dashboard 1 - -1
data 98 - -98
dataVisualizer 3 - -3
embeddable 4 - -4
esUiShared 4 - -4
expressions 58 - -58
fileUpload 4 - -4
fleet 15 - -15
indexManagement 12 - -12
indexPatternFieldEditor 1 - -1
inspector 6 - -6
kibanaLegacy 3 - -3
kibanaReact 5 - -5
kibanaUtils 3 - -3
maps 2 - -2
mapsEms 1 - -1
ml 10 - -10
presentationUtil 3 - -3
reporting 1 - -1
savedObjects 3 - -3
savedObjectsTaggingOss 3 - -3
securitySolution 8 - -8
share 1 - -1
snapshotRestore 1 - -1
timelines 6 - -6
triggersActionsUi 1 - -1
uiActionsEnhanced 2 - -2
visTypeTimeseries 1 - -1
visualizations 13 - -13
total -424

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id before after diff
actions 7 - -7
advancedSettings 1 - -1
alerting 16 - -16
apm 30 - -30
bfetch 2 - -2
canvas 3 - -3
cases 14 - -14
charts 1 - -1
core 31 - -31
dashboard 9 - -9
data 64 - -64
dataEnhanced 2 - -2
devTools 2 - -2
discover 6 - -6
discoverEnhanced 2 - -2
embeddable 3 - -3
encryptedSavedObjects 3 - -3
esUiShared 1 - -1
eventLog 4 - -4
expressionError 2 - -2
expressionImage 1 - -1
expressionRevealImage 1 - -1
expressions 5 - -5
features 2 - -2
fileUpload 1 - -1
fleet 8 - -8
globalSearch 5 - -5
home 5 - -5
indexManagement 3 - -3
indexPatternFieldEditor 4 - -4
infra 3 - -3
inspector 4 - -4
kibanaLegacy 1 - -1
kibanaReact 4 - -4
kibanaUtils 8 - -8
lens 22 - -22
licensing 8 - -8
lists 38 - -38
management 5 - -5
maps 11 - -11
metricsEntities 1 - -1
ml 33 - -33
monitoring 2 - -2
navigation 2 - -2
observability 10 - -10
presentationUtil 5 - -5
reporting 14 - -14
ruleRegistry 9 - -9
runtimeFields 2 - -2
savedObjects 5 - -5
screenshotMode 1 - -1
security 6 - -6
securityOss 3 - -3
securitySolution 28 - -28
share 8 - -8
snapshotRestore 1 - -1
taskManager 8 - -8
telemetryCollectionManager 4 - -4
timelines 25 - -25
triggersActionsUi 19 - -19
uiActions 11 - -11
uiActionsEnhanced 10 - -10
usageCollection 2 - -2
visTypeTimeseries 3 - -3
visualizations 12 - -12
total -566
Unknown metric groups

API count

id before after diff
actions 117 - -117
advancedSettings 23 - -23
alerting 242 - -242
apm 39 - -39
apmOss 4 - -4
banners 9 - -9
bfetch 73 - -73
canvas 6 - -6
cases 445 - -445
charts 190 - -190
cloud 21 - -21
core 2359 - -2359
dashboard 160 - -160
dashboardEnhanced 51 - -51
dashboardMode 11 - -11
data 3716 - -3716
dataEnhanced 16 - -16
dataVisualizer 104 - -104
devTools 10 - -10
discover 81 - -81
discoverEnhanced 39 - -39
embeddable 456 - -456
embeddableEnhanced 14 - -14
encryptedSavedObjects 30 - -30
enterpriseSearch 2 - -2
esUiShared 92 - -92
eventLog 70 - -70
expressionError 12 - -12
expressionImage 4 - -4
expressionRepeatImage 28 - -28
expressionRevealImage 4 - -4
expressions 2003 - -2003
expressionShape 90 - -90
features 215 - -215
fileUpload 128 - -128
fleet 1128 - -1128
globalSearch 68 - -68
home 94 - -94
indexLifecycleManagement 4 - -4
indexManagement 162 - -162
indexPatternFieldEditor 31 - -31
infra 25 - -25
inspector 101 - -101
kibanaLegacy 66 - -66
kibanaReact 260 - -260
kibanaUtils 551 - -551
lens 206 - -206
licenseApiGuard 8 - -8
licenseManagement 3 - -3
licensing 117 - -117
lists 150 - -150
management 40 - -40
maps 204 - -204
mapsEms 75 - -75
metricsEntities 9 - -9
ml 278 - -278
monitoring 10 - -10
navigation 31 - -31
newsfeed 17 - -17
observability 219 - -219
osquery 11 - -11
presentationUtil 163 - -163
remoteClusters 4 - -4
reporting 133 - -133
rollup 20 - -20
ruleRegistry 60 - -60
runtimeFields 24 - -24
savedObjects 213 - -213
savedObjectsManagement 96 - -96
savedObjectsTagging 54 - -54
savedObjectsTaggingOss 89 - -89
screenshotMode 22 - -22
security 112 - -112
securityOss 12 - -12
securitySolution 1296 - -1296
share 123 - -123
snapshotRestore 23 - -23
spaces 106 - -106
spacesOss 72 - -72
stackAlerts 4 - -4
taskManager 52 - -52
telemetry 42 - -42
telemetryCollectionManager 29 - -29
telemetryCollectionXpack 1 - -1
telemetryManagementSection 14 - -14
timelines 882 - -882
triggersActionsUi 237 - -237
uiActions 127 - -127
uiActionsEnhanced 205 - -205
urlForwarding 15 - -15
usageCollection 57 - -57
visTypeTimeseries 10 - -10
visualizations 247 - -247
total -19276

References to deprecated APIs

id before after diff
actions 8 - -8
alerting 32 - -32
apm 7 - -7
canvas 53 - -53
cases 151 - -151
crossClusterReplication 2 - -2
dashboard 128 - -128
dashboardEnhanced 10 - -10
dataEnhanced 53 - -53
dataVisualizer 16 - -16
discover 102 - -102
discoverEnhanced 19 - -19
embeddable 2 - -2
encryptedSavedObjects 2 - -2
fleet 89 - -89
globalSearch 4 - -4
graph 2 - -2
indexLifecycleManagement 2 - -2
indexManagement 12 - -12
infra 292 - -292
lens 160 - -160
lists 103 - -103
maps 592 - -592
ml 140 - -140
observability 34 - -34
presentationUtil 2 - -2
savedObjects 6 - -6
savedObjectsManagement 18 - -18
savedObjectsTaggingOss 5 - -5
security 2 - -2
securitySolution 881 - -881
stackAlerts 104 - -104
timelines 76 - -76
transform 16 - -16
uptime 11 - -11
urlDrilldown 18 - -18
visTypeTimeseries 6 - -6
visualizations 32 - -32
total -3192

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@kibanamachine
Copy link
Contributor

💚 Backport successful

Status Branch Result
7.x

This backport PR will be merged automatically after passing CI.

kibanamachine added a commit to kibanamachine/kibana that referenced this pull request Jul 26, 2021
…lastic#105655)

* Adds migration smoke tests for TSVB panels on dashboard

* Adds index pattern TSVB panel

* Update 7.13.3 export

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
kibanamachine added a commit that referenced this pull request Jul 26, 2021
…105655) (#106745)

* Adds migration smoke tests for TSVB panels on dashboard

* Adds index pattern TSVB panel

* Update 7.13.3 export

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

Co-authored-by: Corey Robertson <corey.robertson@elastic.co>
streamich pushed a commit to vadimkibana/kibana that referenced this pull request Aug 8, 2021
…lastic#105655)

* Adds migration smoke tests for TSVB panels on dashboard

* Adds index pattern TSVB panel

* Update 7.13.3 export

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto-backport Deprecated - use backport:version if exact versions are needed Feature:Dashboard Dashboard related features impact:high Addressing this issue will have a high level of impact on the quality/strength of our product. loe:medium Medium Level of Effort release_note:skip Skip the PR/issue when compiling release notes review Team:Presentation Presentation Team for Dashboard, Input Controls, and Canvas v7.15.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants