Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix ignore_missing parameter has no effect in rename ingest processor #9725

Merged
merged 3 commits into from
Sep 14, 2023

Conversation

gaobinlong
Copy link
Collaborator

Description

Fix the bug of ignore_missing parameter has no effect when using template snippet in rename ingest processor, if the field in the template snippet {{field}} does not exist, the resolved field path is '', then document.hasField method will throw path cannot be null nor empty error directly even though the parameter ignore_missing is true. So we should check the resolve field path in advance, if it's empty and ignore_missing is true, exits quietly.

Related Issues

#9687

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff
  • Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

…t in rename ingest processor

Signed-off-by: Gao Binlong <gbinlong@amazon.com>
@github-actions
Copy link
Contributor

github-actions bot commented Sep 4, 2023

Compatibility status:

Checks if related components are compatible with change b19ed23

Incompatible components

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/reporting.git]

@github-actions
Copy link
Contributor

github-actions bot commented Sep 4, 2023

Gradle Check (Jenkins) Run Completed with:

Signed-off-by: Gao Binlong <gbinlong@amazon.com>
@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@codecov
Copy link

codecov bot commented Sep 11, 2023

Codecov Report

Merging #9725 (b19ed23) into main (c100c0c) will decrease coverage by 0.09%.
Report is 6 commits behind head on main.
The diff coverage is 100.00%.

@@             Coverage Diff              @@
##               main    #9725      +/-   ##
============================================
- Coverage     71.16%   71.08%   -0.09%     
+ Complexity    58114    58048      -66     
============================================
  Files          4824     4824              
  Lines        273897   273900       +3     
  Branches      39910    39911       +1     
============================================
- Hits         194926   194708     -218     
- Misses        62573    62865     +292     
+ Partials      16398    16327      -71     
Files Changed Coverage Δ
...main/java/org/opensearch/common/collect/Tuple.java 64.70% <ø> (ø)
...java/org/opensearch/common/crypto/DataKeyPair.java 0.00% <ø> (ø)
...h/common/crypto/DecryptedRangedStreamProvider.java 0.00% <ø> (ø)
...g/opensearch/compress/spi/CompressionProvider.java 100.00% <ø> (ø)
...cli/src/main/java/org/opensearch/cli/Terminal.java 74.73% <100.00%> (ø)
.../org/opensearch/ingest/common/RenameProcessor.java 97.29% <100.00%> (+0.23%) ⬆️

... and 478 files with indirect coverage changes

@gaobinlong
Copy link
Collaborator Author

Could anyone help to review this PR? Thanks!

Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: gaobinlong <gbl_long@163.com>
@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@reta
Copy link
Collaborator

reta commented Sep 12, 2023

@msfroh any comments from your side? thank you!

@msfroh
Copy link
Collaborator

msfroh commented Sep 14, 2023

@msfroh any comments from your side? thank you!

Looks good to me! 👍 Thanks @gaobinlong !

@msfroh msfroh merged commit 921cd0c into opensearch-project:main Sep 14, 2023
@reta reta added the backport 2.x Backport to 2.x branch label Sep 14, 2023
opensearch-trigger-bot bot pushed a commit that referenced this pull request Sep 14, 2023
…#9725)

* Fix ignore_missing parameter has no effect when using template snippet in rename ingest processor

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Small change

Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: gaobinlong <gbl_long@163.com>

---------

Signed-off-by: Gao Binlong <gbinlong@amazon.com>
Signed-off-by: gaobinlong <gbl_long@163.com>
Co-authored-by: Andriy Redko <drreta@gmail.com>
(cherry picked from commit 921cd0c)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
reta added a commit that referenced this pull request Sep 14, 2023
…#9725) (#10043)

* Fix ignore_missing parameter has no effect when using template snippet in rename ingest processor



* Small change




---------




(cherry picked from commit 921cd0c)

Signed-off-by: Gao Binlong <gbinlong@amazon.com>
Signed-off-by: gaobinlong <gbl_long@163.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Andriy Redko <drreta@gmail.com>
sarthakaggarwal97 pushed a commit to sarthakaggarwal97/OpenSearch that referenced this pull request Sep 20, 2023
…opensearch-project#9725)

* Fix ignore_missing parameter has no effect when using template snippet in rename ingest processor

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Small change

Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: gaobinlong <gbl_long@163.com>

---------

Signed-off-by: Gao Binlong <gbinlong@amazon.com>
Signed-off-by: gaobinlong <gbl_long@163.com>
Co-authored-by: Andriy Redko <drreta@gmail.com>
brusic pushed a commit to brusic/OpenSearch that referenced this pull request Sep 25, 2023
…opensearch-project#9725)

* Fix ignore_missing parameter has no effect when using template snippet in rename ingest processor

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Small change

Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: gaobinlong <gbl_long@163.com>

---------

Signed-off-by: Gao Binlong <gbinlong@amazon.com>
Signed-off-by: gaobinlong <gbl_long@163.com>
Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: Ivan Brusic <ivan.brusic@flocksafety.com>
vikasvb90 pushed a commit to vikasvb90/OpenSearch that referenced this pull request Oct 10, 2023
…opensearch-project#9725)

* Fix ignore_missing parameter has no effect when using template snippet in rename ingest processor

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Small change

Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: gaobinlong <gbl_long@163.com>

---------

Signed-off-by: Gao Binlong <gbinlong@amazon.com>
Signed-off-by: gaobinlong <gbl_long@163.com>
Co-authored-by: Andriy Redko <drreta@gmail.com>
shiv0408 pushed a commit to Gaurav614/OpenSearch that referenced this pull request Apr 25, 2024
…opensearch-project#9725)

* Fix ignore_missing parameter has no effect when using template snippet in rename ingest processor

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Small change

Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: gaobinlong <gbl_long@163.com>

---------

Signed-off-by: Gao Binlong <gbinlong@amazon.com>
Signed-off-by: gaobinlong <gbl_long@163.com>
Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x Backport to 2.x branch
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants