Reduce the number of cluster start/stops for Metadata end to end tests #1186
Conversation
peternied
force-pushed
the
faster-meta-e2e
branch
from
ad90bda to
9cb6196
Compare
After recently reviewing metadata migration end to end test cases I noticed there was 56 different tests, this was a large increase, taking nearly 20 minutes of runtime. Reviewing the parameters of the tests I'm running evaluate and then immediately following migrate, this should reduce the test cases by nearly half and increase of coverage using http and snapshot sources. I have tweaked how we use templates moving back to the logic of legacy templates for ES 6.X and using both index and index compontent templates for future ES versions, which should be another 30% redunction. Signed-off-by: Peter Nied <peternied@hotmail.com>
peternied
force-pushed
the
faster-meta-e2e
branch
from
9cb6196 to
ec2bb14
Compare
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #1186 +/- ##
============================================
+ Coverage 80.93% 80.97% +0.03%
- Complexity 2995 2998 +3
============================================
Files 407 407
Lines 15241 15241
Branches 1021 1021
============================================
+ Hits 12336 12341 +5
+ Misses 2277 2275 -2
+ Partials 628 625 -3
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
|
@AndreKurait could you take a look at this change since you most recently increased the number of test cases? Note; the code coverage numbers are unchanged with this version, but 10 minutes faster ⚡ |
peternied
requested review from
AndreKurait,
chelma,
gregschohn,
lewijacn,
mikaylathompson and
sumobrian
as code owners
| void metadataCommand(ContainerVersion sourceVersion, ContainerVersion targetVersion, TransferMedium medium, | ||
| MetadataCommands command, TemplateType templateType) { | ||
| void metadataCommand(ContainerVersion sourceVersion, ContainerVersion targetVersion, TransferMedium medium) { | ||
| var templateType = VersionMatchers.isES_6_X.test(sourceVersion.getVersion()) |
There was a problem hiding this comment.
Where are you testing TemplateType.Legacy on ES 7? There was a bug in that flow that these tests were added thereafter
There was a problem hiding this comment.
Could you review this matrix for the test cover for template types? I'll update the code to include the correct set of items
| ES Version | Template Type(s) |
|---|---|
| ES 6.X | Legacy |
| ES 7.X | Legacy & Template & Component Template |
| OS 1.X | Legacy & Template & Component Template |
| OS 2.X | Legacy & Template & Component Template |
There was a problem hiding this comment.
Yes, that's correct.
There's also a difference from a Template with an inline mapping vs Template linked to a Component Template so they should each be tested.
Description
After recently reviewing metadata migration end to end test cases I
noticed there was 56 different tests, this was a large increase, taking
nearly 20 minutes of runtime. Reviewing the parameters of the tests
I'm running evaluate and then immediately following migrate, this
should reduce the test cases by nearly half and increase of coverage
using http and snapshot sources.
I have tweaked how we use templates moving back to the logic of
legacy templates for ES 6.X and using both index and index component
templates for future ES versions, which should be another 30%
reduction.
Issues
Improvement
Previous
This Change
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.
Package Tests Failures Ignored Duration Success rate
org.opensearch.migrations 56 0 0 20m54.98s 100%Package Tests Failures Ignored Duration Success rate
org.opensearch.migrations 24 0 0 9m43.71s 100%