Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Continue to evaulate / migrate after failures #1037

Merged
merged 8 commits into from
Oct 9, 2024

Conversation

peternied
Copy link
Member

Description

Updates the behavior of the metadata processing for templates and indexes to collect all the results and then return failure details after all items have been processed. When items are not created they will appear in the output with a reason and exception if needed be. Here is the output from an updated evaulate test case where the index already_exists was already on the target cluster and won't be created.

Clusters:
   Source:
      Remote Cluster: ELASTICSEARCH 6.8.23 ConnectionContext(uri=http://localhost:36188, protocol=HTTP, insecure=false, compressionSupported=false)

   Target:
      Remote Cluster: OPENSEARCH 2.14.0 ConnectionContext(uri=http://localhost:36187, protocol=HTTP, insecure=false, compressionSupported=false)


Migration Candidates:
   Index Templates:
      simple_index_template

   Component Templates:
      <NONE FOUND>

   Indexes:
      blog_2023, movies_2023
      WARN - already_exists already exists

   Aliases:
      alias1, movies-alias


Results:
   0 issue(s) detected

Check List

  • New functionality includes testing
    • All tests pass, including unit test, integration test and doctest
  • New functionality has been documented
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Updates the behavoir of the metadata processing for templates and
indexes to collect all the results and then return failure details after
all items have been processed.

Signed-off-by: Peter Nied <peternied@hotmail.com>
Copy link

codecov bot commented Sep 30, 2024

Codecov Report

Attention: Patch coverage is 76.62338% with 36 lines in your changes missing coverage. Please review.

Project coverage is 80.17%. Comparing base (baa7bc2) to head (c0e242d).
Report is 6 commits behind head on main.

Files with missing lines Patch % Lines
...version_os_2_11/GlobalMetadataCreator_OS_2_11.java 20.00% 10 Missing and 2 partials ⚠️
...opensearch/migrations/metadata/CreationResult.java 65.00% 7 Missing ⚠️
...search/migrations/bulkload/worker/IndexRunner.java 64.70% 6 Missing ⚠️
...rch/migrations/commands/MigratorEvaluatorBase.java 50.00% 4 Missing and 1 partial ⚠️
...oad/transformers/IndexTransformationException.java 0.00% 2 Missing ⚠️
...grations/bulkload/worker/IndexMetadataResults.java 0.00% 0 Missing and 2 partials ⚠️
...main/java/org/opensearch/migrations/cli/Items.java 97.91% 1 Missing ⚠️
...bulkload/version_os_2_11/IndexCreator_OS_2_11.java 96.15% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #1037      +/-   ##
============================================
- Coverage     80.54%   80.17%   -0.38%     
- Complexity     2736     2745       +9     
============================================
  Files           365      367       +2     
  Lines         13614    13743     +129     
  Branches        941      949       +8     
============================================
+ Hits          10966    11018      +52     
- Misses         2068     2149      +81     
+ Partials        580      576       -4     
Flag Coverage Δ
gradle-test 78.08% <76.62%> (-0.51%) ⬇️
python-test 90.24% <ø> (+0.13%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Peter Nied <peternied@hotmail.com>
Copy link
Member

@AndreKurait AndreKurait left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the cleanup, i'm good for merging once the isolatedTests are fixed

Signed-off-by: Peter Nied <peternied@hotmail.com>
Signed-off-by: Peter Nied <peternied@hotmail.com>
Signed-off-by: Peter Nied <peternied@hotmail.com>
This reverts commit bfafa20.

Signed-off-by: Peter Nied <peternied@hotmail.com>
Signed-off-by: Peter Nied <peternied@hotmail.com>
Signed-off-by: Peter Nied <peternied@hotmail.com>
@peternied peternied force-pushed the proceed-with-failures branch from 6ad40bf to c0e242d Compare October 9, 2024 20:25
@peternied peternied merged commit 7307579 into opensearch-project:main Oct 9, 2024
13 of 14 checks passed
@peternied peternied deleted the proceed-with-failures branch October 9, 2024 20:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants