Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Forwardport] Bump Apache Lucene to 9.11.1 (#14576) #14581

Merged

Conversation

reta
Copy link
Collaborator

@reta reta commented Jun 27, 2024

Forwardport of #14576 to main

Copy link
Contributor

❌ Gradle check result for 6fd5357: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
(cherry picked from commit 0095fd1)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
@reta reta force-pushed the backport/backport-14576-to-2.x branch from 6fd5357 to 5ee1e15 Compare June 27, 2024 17:22
Copy link
Contributor

✅ Gradle check result for 5ee1e15: SUCCESS

Copy link

codecov bot commented Jun 27, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 71.62%. Comparing base (f70fd71) to head (5ee1e15).
Report is 1 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main   #14581      +/-   ##
============================================
- Coverage     71.68%   71.62%   -0.07%     
- Complexity    62161    62180      +19     
============================================
  Files          5125     5125              
  Lines        292445   292445              
  Branches      42257    42257              
============================================
- Hits         209652   209469     -183     
- Misses        65478    65723     +245     
+ Partials      17315    17253      -62     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@andrross andrross merged commit 243e8db into opensearch-project:main Jun 27, 2024
31 checks passed
@nibix
Copy link

nibix commented Jun 28, 2024

Sorry for crashing here :) I just found this because I am struggling with resolving locally the artifact lucene-test-framework/9.12.0-snapshot-847316d when compiling tests for the security plugin.

This snapshot id was introduced here. Is there something I can do to fix this?

nils@teal:~/ope/security$ ./gradlew clean compileTestJava
=======================================
OpenSearch Build Hamster says Hello!
  Gradle Version        : 8.8
  OS Info               : Linux 5.18.10-76051810-generic (amd64)
  JDK Version           : 21 (Ubuntu JDK)
  JAVA_HOME             : /usr/lib/jvm/java-21-openjdk-amd64
  Random Testing Seed   : 83483696EBDD363D
  In FIPS 140 mode      : false
=======================================

> Task :compileJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.

> Task :compileTestJava FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':compileTestJava'.
> Could not resolve all files for configuration ':testCompileClasspath'.
   > Could not resolve org.apache.lucene:lucene-test-framework:9.12.0-snapshot-847316d.
     Required by:
         project : > org.opensearch.test:framework:3.0.0-SNAPSHOT:20240627.190338-2549
      > Could not resolve org.apache.lucene:lucene-test-framework:9.12.0-snapshot-847316d.
         > Could not get resource 'https://artifacts.opensearch.org/snapshots/lucene/org/apache/lucene/lucene-test-framework/9.12.0-snapshot-847316d/lucene-test-framework-9.12.0-snapshot-847316d.pom'.
            > Could not GET 'https://artifacts.opensearch.org/snapshots/lucene/org/apache/lucene/lucene-test-framework/9.12.0-snapshot-847316d/lucene-test-framework-9.12.0-snapshot-847316d.pom'. Received status code 403 from server: Forbidden
   > Could not resolve org.apache.lucene:lucene-codecs:9.12.0-snapshot-847316d.
     Required by:
         project : > org.opensearch.test:framework:3.0.0-SNAPSHOT:20240627.190338-2549
      > Could not resolve org.apache.lucene:lucene-codecs:9.12.0-snapshot-847316d.
         > Could not get resource 'https://artifacts.opensearch.org/snapshots/lucene/org/apache/lucene/lucene-codecs/9.12.0-snapshot-847316d/lucene-codecs-9.12.0-snapshot-847316d.pom'.
            > Could not GET 'https://artifacts.opensearch.org/snapshots/lucene/org/apache/lucene/lucene-codecs/9.12.0-snapshot-847316d/lucene-codecs-9.12.0-snapshot-847316d.pom'. Received status code 403 from server: Forbidden

@reta
Copy link
Collaborator Author

reta commented Jun 28, 2024

Sorry for crashing here :) I just found this because I am struggling with resolving locally the artifact lucene-test-framework/9.12.0-snapshot-847316d when compiling tests for the security plugin.

@nibix Hm ... the Apache Lucene snapshots are managed by opensearch.repositories plugin, which sets the repo to https://ci.opensearch.org/ci/dbc/snapshots/lucene/

@nibix
Copy link

nibix commented Jun 28, 2024

@reta I could fix it now by refreshing the snapshots. Thanks for reaching out! :)

akolarkunnu pushed a commit to akolarkunnu/OpenSearch that referenced this pull request Jul 9, 2024
…roject#14581)

(cherry picked from commit 0095fd1)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
bharath-techie added a commit to bharath-techie/OpenSearch that referenced this pull request Jul 10, 2024
* Fix flaky test in range aggregation yaml test (opensearch-project#14486)

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* Use CODECOV_TOKEN (opensearch-project#14536)

Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com>

* [Tiered Caching]  Moving query recomputation logic outside of write lock (opensearch-project#14187)

* Moving query recompute out of write lock

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* [Tiered Caching] Moving query recomputation logic outside of write lock

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Adding java doc for the completable map

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Changes to call future handler only once per key

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Fixing spotless check

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Added changelog

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Addressing comments

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Fixing gradle fail

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Addressing comments to refactor unit test

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* minor UT refactor

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

---------

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>
Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com>
Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com>

* Fix Flaky Test ClusterRerouteIT.testDelayWithALargeAmountOfShards (opensearch-project#14510)

Signed-off-by: kkewwei kkewwei@163.com

Signed-off-by: kkewwei kkewwei@163.com
Signed-off-by: kkewwei <kkewwei@163.com>

* Add doc for debugging rest tests (opensearch-project#14491)

* add doc for debugging rest tests

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* Update TESTING.md

Co-authored-by: Marc Handalian <handalm@amazon.com>
Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* Address comment

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

---------

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
Co-authored-by: Marc Handalian <handalm@amazon.com>

* Fix flaky DefaultCacheStatsHolderTests (opensearch-project#14462)

Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>

* [AUTO] [main] Add bwc version 2.15.1. (opensearch-project#14549)

* Add bwc version 2.15.1

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* Fix auto-generated version

Signed-off-by: Andrew Ross <andrross@amazon.com>

---------

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: Andrew Ross <andrross@amazon.com>
Co-authored-by: opensearch-ci-bot <83309141+opensearch-ci-bot@users.noreply.github.com>
Co-authored-by: Andrew Ross <andrross@amazon.com>

* Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently (opensearch-project#14550)

* Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Addressing comment

Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com>

---------

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>
Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com>
Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com>

* Add allowlist setting for ingest-common processors (opensearch-project#14479)

Add a new static setting that lets an operator choose specific ingest
processors to enable by name. The behavior is as follows:

- If the allowlist setting is not defined, all installed processors are
  enabled. This is the status quo.
- If the allowlist setting is defined as the empty set, then all processors
  are disabled.
- If the allowlist setting contains the names of valid processors, only those
  processors are enabled.
- If the allowlist setting contains a name of a processor that does not exist,
  then the server will fail to start with an IllegalStateException
  listing which processors were defined in the allowlist but are not
  installed.
- If the allowlist setting is changed between server restarts then any
  ingest pipeline using a now-disabled processor will fail. This is the
  same experience if a pipeline used a processor defined by a plugin but
  then that plugin were to be uninstalled across restarts.

Related to opensearch-project#14439

Signed-off-by: Andrew Ross <andrross@amazon.com>

* Fix file cache initialization (opensearch-project#14004)

* fix file cache initialization

Signed-off-by: panguixin <panguixin@bytedance.com>

* changelog

Signed-off-by: panguixin <panguixin@bytedance.com>

* add test

Signed-off-by: panguixin <panguixin@bytedance.com>

---------

Signed-off-by: panguixin <panguixin@bytedance.com>

* Add Ashish Singh as maintainer (opensearch-project#14567)

Signed-off-by: Bukhtawar Khan <bukhtawa@amazon.com>

* Allow @internalapi annotation on classes not meant to be constructed outside of the OpenSearch core (opensearch-project#14575)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Bump com.azure:azure-storage-common from 12.21.2 to 12.25.1 in /plugins/repository-azure (opensearch-project#14517)

* Bump com.azure:azure-storage-common in /plugins/repository-azure

Bumps [com.azure:azure-storage-common](https://github.com/Azure/azure-sdk-for-java) from 12.21.2 to 12.25.1.
- [Release notes](https://github.com/Azure/azure-sdk-for-java/releases)
- [Commits](Azure/azure-sdk-for-java@azure-storage-common_12.21.2...azure-storage-blob_12.25.1)

---
updated-dependencies:
- dependency-name: com.azure:azure-storage-common
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

* Add allowlist setting for search-pipeline-common processors (opensearch-project#14562)

Add a new static setting that lets an operator choose specific search
pipeline processors to enable by name. The behavior is as follows:

- If the allowlist setting is not defined, all installed processors are
  enabled. This is the status quo.
- If the allowlist setting is defined as the empty set, then all processors
  are disabled.
- If the allowlist setting contains the names of valid processors, only those
  processors are enabled.
- If the allowlist setting contains a name of a processor that does not exist,
  then the server will fail to start with an IllegalStateException
  listing which processors were defined in the allowlist but are not
  installed.
- If the allowlist setting is changed between server restarts then any
  ingest pipeline using a now-disabled processor will fail. This is the
  same experience if a pipeline used a processor defined by a plugin but
  then that plugin were to be uninstalled across restarts.

A distinct setting exists for each of request, response, and search phase
results processors.

Related to opensearch-project#14439

Signed-off-by: Andrew Ross <andrross@amazon.com>

* Bump Apache Lucene to 9.11.1 (opensearch-project#14576) (opensearch-project#14581)

(cherry picked from commit 0095fd1)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Add unittests for RemoteClusterStateAttributesManager (opensearch-project#14427)

* Add unittests for RemoteClusterStateAttributesManager

Signed-off-by: Shivansh Arora <hishiv@amazon.com>

* Add Ashish Singh to codeowners (opensearch-project#14592)

Signed-off-by: Ashish Singh <ssashish@amazon.com>

* Add batching processor base type AbstractBatchingProcessor (opensearch-project#14554)

Signed-off-by: Liyun Xiu <xiliyun@amazon.com>

* Add @internalapi annotation to japicmp exclusions (opensearch-project#14597)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix issue 14519:Parsing a GetResult returns NPE if found field is mis… (opensearch-project#14552)

* Fix issue 14519:Parsing a GetResult returns NPE if found field is missing. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com>

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>

* Fix issue 14519:Parsing a GetResult returns NPE if found field is missing. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com>

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>

* Fix issue 14519:Fix wildcart import. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com>

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>

* Fix issue 14519:Fix wildcart import. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com>

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>

* Fix issue 14519:Fix spotless issues. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com>

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>

* Fix issue 14519:update changelog

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>

---------

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Co-authored-by: Daniel Widdis <widdis@gmail.com>

* Star tree mapping changes (opensearch-project#14605)

* Star tree mapping changes with feature flag
---------
Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

* Bump com.microsoft.azure:msal4j from 1.15.1 to 1.16.0 in /plugins/repository-azure (opensearch-project#14610)

* Bump com.microsoft.azure:msal4j in /plugins/repository-azure

Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.15.1 to 1.16.0.
- [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases)
- [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt)
- [Commits](AzureAD/microsoft-authentication-library-for-java@v1.15.1...v1.16.0)

---
updated-dependencies:
- dependency-name: com.microsoft.azure:msal4j
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

* [Bugfix] Fix ICacheKeySerializerTests flakiness (opensearch-project#14564)

* Fix testInvalidInput flakiness

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

* Addressed andrross's comment

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

* rerun security check

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

---------

Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>

* Correct typo in method name (opensearch-project#14621)

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>

* Refactoring FilterPath.parse by using an iterative approach instead of recursion. (opensearch-project#14200)

* Refactor FilterPath parse function (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Implement unit tests for FilterPathTests (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Write warn log if Filter is empty; Add comments (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Add changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unnecessary log statement

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unused logger

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Spotless apply

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove incorrect changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

---------

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Removing String format in RemoteStoreMigrationAllocationDecider to optimise performance(opensearch-project#14612)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata; Correct the check for deciding upload of HashesOfConsistentSettings (opensearch-project#14513)

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata
* Correct the check for deciding upload of hashes of consistent settings

Signed-off-by: Sooraj Sinha <soosinha@amazon.com>

* Improve reroute performance by optimising List.removeAll in LocalShardsBalancer to filter remote search shard from relocation decision (opensearch-project#14613)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>

* Fix assertion failure while deleting remote backed index (opensearch-project#14601)

Signed-off-by: Sachin Kale <kalsac@amazon.com>

* OnHeap Star Tree Implementation

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* addressed nits

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* addressed major nits

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* includes Count Aggregator

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* handling for missing doc values

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* addressing review comments

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* rebasing with main

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* support for empty sequential doc values iterator

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* nits

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* min and max star tree aggregators

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* star tree file formats

* Star tree codec changes

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

* Adding tests

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

* Addressing comments

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

* addressing review comments

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

* Star tree merge changes

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

* fix annotations

* star-tree file formats reader and javadoc fixes

* read for composite index values

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* doc values file format

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

---------

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com>
Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>
Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com>
Signed-off-by: kkewwei kkewwei@163.com
Signed-off-by: kkewwei <kkewwei@163.com>
Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: Andrew Ross <andrross@amazon.com>
Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com>
Signed-off-by: panguixin <panguixin@bytedance.com>
Signed-off-by: Bukhtawar Khan <bukhtawa@amazon.com>
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Signed-off-by: Ashish Singh <ssashish@amazon.com>
Signed-off-by: Liyun Xiu <xiliyun@amazon.com>
Signed-off-by: vatsal <vatsal.v.anand@gmail.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
Signed-off-by: Sachin Kale <kalsac@amazon.com>
Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>
Signed-off-by: Bharathwaj G <bharath78910@gmail.com>
Co-authored-by: bowenlan-amzn <bowenlan23@gmail.com>
Co-authored-by: Prudhvi Godithi <pgodithi@amazon.com>
Co-authored-by: Sagar <99425694+sgup432@users.noreply.github.com>
Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com>
Co-authored-by: kkewwei <kkewwei@163.com>
Co-authored-by: Marc Handalian <handalm@amazon.com>
Co-authored-by: Peter Alfonsi <peter.alfonsi@gmail.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com>
Co-authored-by: opensearch-ci-bot <83309141+opensearch-ci-bot@users.noreply.github.com>
Co-authored-by: Andrew Ross <andrross@amazon.com>
Co-authored-by: panguixin <panguixin@bytedance.com>
Co-authored-by: Bukhtawar Khan <bukhtawa@amazon.com>
Co-authored-by: Andriy Redko <andriy.redko@aiven.io>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Co-authored-by: Shivansh Arora <hishiv@amazon.com>
Co-authored-by: Ashish Singh <ssashish@amazon.com>
Co-authored-by: Liyun Xiu <xiliyun@amazon.com>
Co-authored-by: Vatsal <36672090+imvtsl@users.noreply.github.com>
Co-authored-by: Daniel Widdis <widdis@gmail.com>
Co-authored-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
Co-authored-by: rishavz_sagar <rishavsagar4b1@gmail.com>
Co-authored-by: Sooraj Sinha <81695996+soosinha@users.noreply.github.com>
Co-authored-by: Sachin Kale <sachinpkale@gmail.com>
Co-authored-by: Sarthak Aggarwal <sarthagg@amazon.com>
bharath-techie added a commit to bharath-techie/OpenSearch that referenced this pull request Jul 11, 2024
* Fix flaky test in range aggregation yaml test (opensearch-project#14486)

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* Use CODECOV_TOKEN (opensearch-project#14536)

Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com>

* [Tiered Caching]  Moving query recomputation logic outside of write lock (opensearch-project#14187)

* Moving query recompute out of write lock

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* [Tiered Caching] Moving query recomputation logic outside of write lock

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Adding java doc for the completable map

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Changes to call future handler only once per key

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Fixing spotless check

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Added changelog

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Addressing comments

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Fixing gradle fail

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Addressing comments to refactor unit test

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* minor UT refactor

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

---------

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>
Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com>
Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com>

* Fix Flaky Test ClusterRerouteIT.testDelayWithALargeAmountOfShards (opensearch-project#14510)

Signed-off-by: kkewwei kkewwei@163.com

Signed-off-by: kkewwei kkewwei@163.com
Signed-off-by: kkewwei <kkewwei@163.com>

* Add doc for debugging rest tests (opensearch-project#14491)

* add doc for debugging rest tests

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* Update TESTING.md

Co-authored-by: Marc Handalian <handalm@amazon.com>
Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* Address comment

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

---------

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
Co-authored-by: Marc Handalian <handalm@amazon.com>

* Fix flaky DefaultCacheStatsHolderTests (opensearch-project#14462)

Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>

* [AUTO] [main] Add bwc version 2.15.1. (opensearch-project#14549)

* Add bwc version 2.15.1

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* Fix auto-generated version

Signed-off-by: Andrew Ross <andrross@amazon.com>

---------

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: Andrew Ross <andrross@amazon.com>
Co-authored-by: opensearch-ci-bot <83309141+opensearch-ci-bot@users.noreply.github.com>
Co-authored-by: Andrew Ross <andrross@amazon.com>

* Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently (opensearch-project#14550)

* Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Addressing comment

Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com>

---------

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>
Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com>
Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com>

* Add allowlist setting for ingest-common processors (opensearch-project#14479)

Add a new static setting that lets an operator choose specific ingest
processors to enable by name. The behavior is as follows:

- If the allowlist setting is not defined, all installed processors are
  enabled. This is the status quo.
- If the allowlist setting is defined as the empty set, then all processors
  are disabled.
- If the allowlist setting contains the names of valid processors, only those
  processors are enabled.
- If the allowlist setting contains a name of a processor that does not exist,
  then the server will fail to start with an IllegalStateException
  listing which processors were defined in the allowlist but are not
  installed.
- If the allowlist setting is changed between server restarts then any
  ingest pipeline using a now-disabled processor will fail. This is the
  same experience if a pipeline used a processor defined by a plugin but
  then that plugin were to be uninstalled across restarts.

Related to opensearch-project#14439

Signed-off-by: Andrew Ross <andrross@amazon.com>

* Fix file cache initialization (opensearch-project#14004)

* fix file cache initialization

Signed-off-by: panguixin <panguixin@bytedance.com>

* changelog

Signed-off-by: panguixin <panguixin@bytedance.com>

* add test

Signed-off-by: panguixin <panguixin@bytedance.com>

---------

Signed-off-by: panguixin <panguixin@bytedance.com>

* Add Ashish Singh as maintainer (opensearch-project#14567)

Signed-off-by: Bukhtawar Khan <bukhtawa@amazon.com>

* Allow @internalapi annotation on classes not meant to be constructed outside of the OpenSearch core (opensearch-project#14575)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Bump com.azure:azure-storage-common from 12.21.2 to 12.25.1 in /plugins/repository-azure (opensearch-project#14517)

* Bump com.azure:azure-storage-common in /plugins/repository-azure

Bumps [com.azure:azure-storage-common](https://github.com/Azure/azure-sdk-for-java) from 12.21.2 to 12.25.1.
- [Release notes](https://github.com/Azure/azure-sdk-for-java/releases)
- [Commits](Azure/azure-sdk-for-java@azure-storage-common_12.21.2...azure-storage-blob_12.25.1)

---
updated-dependencies:
- dependency-name: com.azure:azure-storage-common
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

* Add allowlist setting for search-pipeline-common processors (opensearch-project#14562)

Add a new static setting that lets an operator choose specific search
pipeline processors to enable by name. The behavior is as follows:

- If the allowlist setting is not defined, all installed processors are
  enabled. This is the status quo.
- If the allowlist setting is defined as the empty set, then all processors
  are disabled.
- If the allowlist setting contains the names of valid processors, only those
  processors are enabled.
- If the allowlist setting contains a name of a processor that does not exist,
  then the server will fail to start with an IllegalStateException
  listing which processors were defined in the allowlist but are not
  installed.
- If the allowlist setting is changed between server restarts then any
  ingest pipeline using a now-disabled processor will fail. This is the
  same experience if a pipeline used a processor defined by a plugin but
  then that plugin were to be uninstalled across restarts.

A distinct setting exists for each of request, response, and search phase
results processors.

Related to opensearch-project#14439

Signed-off-by: Andrew Ross <andrross@amazon.com>

* Bump Apache Lucene to 9.11.1 (opensearch-project#14576) (opensearch-project#14581)

(cherry picked from commit 0095fd1)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Add unittests for RemoteClusterStateAttributesManager (opensearch-project#14427)

* Add unittests for RemoteClusterStateAttributesManager

Signed-off-by: Shivansh Arora <hishiv@amazon.com>

* Add Ashish Singh to codeowners (opensearch-project#14592)

Signed-off-by: Ashish Singh <ssashish@amazon.com>

* Add batching processor base type AbstractBatchingProcessor (opensearch-project#14554)

Signed-off-by: Liyun Xiu <xiliyun@amazon.com>

* Add @internalapi annotation to japicmp exclusions (opensearch-project#14597)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix issue 14519:Parsing a GetResult returns NPE if found field is mis… (opensearch-project#14552)

* Fix issue 14519:Parsing a GetResult returns NPE if found field is missing. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com>

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>

* Fix issue 14519:Parsing a GetResult returns NPE if found field is missing. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com>

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>

* Fix issue 14519:Fix wildcart import. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com>

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>

* Fix issue 14519:Fix wildcart import. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com>

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>

* Fix issue 14519:Fix spotless issues. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com>

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>

* Fix issue 14519:update changelog

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>

---------

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Co-authored-by: Daniel Widdis <widdis@gmail.com>

* Star tree mapping changes (opensearch-project#14605)

* Star tree mapping changes with feature flag
---------
Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

* Bump com.microsoft.azure:msal4j from 1.15.1 to 1.16.0 in /plugins/repository-azure (opensearch-project#14610)

* Bump com.microsoft.azure:msal4j in /plugins/repository-azure

Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.15.1 to 1.16.0.
- [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases)
- [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt)
- [Commits](AzureAD/microsoft-authentication-library-for-java@v1.15.1...v1.16.0)

---
updated-dependencies:
- dependency-name: com.microsoft.azure:msal4j
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

* [Bugfix] Fix ICacheKeySerializerTests flakiness (opensearch-project#14564)

* Fix testInvalidInput flakiness

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

* Addressed andrross's comment

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

* rerun security check

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

---------

Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>

* Correct typo in method name (opensearch-project#14621)

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>

* Refactoring FilterPath.parse by using an iterative approach instead of recursion. (opensearch-project#14200)

* Refactor FilterPath parse function (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Implement unit tests for FilterPathTests (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Write warn log if Filter is empty; Add comments (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Add changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unnecessary log statement

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unused logger

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Spotless apply

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove incorrect changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

---------

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Removing String format in RemoteStoreMigrationAllocationDecider to optimise performance(opensearch-project#14612)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata; Correct the check for deciding upload of HashesOfConsistentSettings (opensearch-project#14513)

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata
* Correct the check for deciding upload of hashes of consistent settings

Signed-off-by: Sooraj Sinha <soosinha@amazon.com>

* Improve reroute performance by optimising List.removeAll in LocalShardsBalancer to filter remote search shard from relocation decision (opensearch-project#14613)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>

* Fix assertion failure while deleting remote backed index (opensearch-project#14601)

Signed-off-by: Sachin Kale <kalsac@amazon.com>

* OnHeap Star Tree Implementation

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* addressed nits

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* addressed major nits

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* includes Count Aggregator

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* handling for missing doc values

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* addressing review comments

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* rebasing with main

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* support for empty sequential doc values iterator

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* nits

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* min and max star tree aggregators

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* star tree file formats

* Star tree codec changes

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

* Adding tests

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

* Addressing comments

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

* addressing review comments

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

* Star tree merge changes

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

* fix annotations

* star-tree file formats reader and javadoc fixes

* read for composite index values

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* doc values file format

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

---------

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com>
Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>
Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com>
Signed-off-by: kkewwei kkewwei@163.com
Signed-off-by: kkewwei <kkewwei@163.com>
Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: Andrew Ross <andrross@amazon.com>
Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com>
Signed-off-by: panguixin <panguixin@bytedance.com>
Signed-off-by: Bukhtawar Khan <bukhtawa@amazon.com>
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Signed-off-by: Ashish Singh <ssashish@amazon.com>
Signed-off-by: Liyun Xiu <xiliyun@amazon.com>
Signed-off-by: vatsal <vatsal.v.anand@gmail.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
Signed-off-by: Sachin Kale <kalsac@amazon.com>
Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>
Signed-off-by: Bharathwaj G <bharath78910@gmail.com>
Co-authored-by: bowenlan-amzn <bowenlan23@gmail.com>
Co-authored-by: Prudhvi Godithi <pgodithi@amazon.com>
Co-authored-by: Sagar <99425694+sgup432@users.noreply.github.com>
Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com>
Co-authored-by: kkewwei <kkewwei@163.com>
Co-authored-by: Marc Handalian <handalm@amazon.com>
Co-authored-by: Peter Alfonsi <peter.alfonsi@gmail.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com>
Co-authored-by: opensearch-ci-bot <83309141+opensearch-ci-bot@users.noreply.github.com>
Co-authored-by: Andrew Ross <andrross@amazon.com>
Co-authored-by: panguixin <panguixin@bytedance.com>
Co-authored-by: Bukhtawar Khan <bukhtawa@amazon.com>
Co-authored-by: Andriy Redko <andriy.redko@aiven.io>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Co-authored-by: Shivansh Arora <hishiv@amazon.com>
Co-authored-by: Ashish Singh <ssashish@amazon.com>
Co-authored-by: Liyun Xiu <xiliyun@amazon.com>
Co-authored-by: Vatsal <36672090+imvtsl@users.noreply.github.com>
Co-authored-by: Daniel Widdis <widdis@gmail.com>
Co-authored-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
Co-authored-by: rishavz_sagar <rishavsagar4b1@gmail.com>
Co-authored-by: Sooraj Sinha <81695996+soosinha@users.noreply.github.com>
Co-authored-by: Sachin Kale <sachinpkale@gmail.com>
Co-authored-by: Sarthak Aggarwal <sarthagg@amazon.com>
bharath-techie added a commit to bharath-techie/OpenSearch that referenced this pull request Jul 11, 2024
* Fix flaky test in range aggregation yaml test (opensearch-project#14486)

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* Use CODECOV_TOKEN (opensearch-project#14536)

Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com>

* [Tiered Caching]  Moving query recomputation logic outside of write lock (opensearch-project#14187)

* Moving query recompute out of write lock

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* [Tiered Caching] Moving query recomputation logic outside of write lock

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Adding java doc for the completable map

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Changes to call future handler only once per key

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Fixing spotless check

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Added changelog

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Addressing comments

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Fixing gradle fail

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Addressing comments to refactor unit test

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* minor UT refactor

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

---------

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>
Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com>
Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com>

* Fix Flaky Test ClusterRerouteIT.testDelayWithALargeAmountOfShards (opensearch-project#14510)

Signed-off-by: kkewwei kkewwei@163.com

Signed-off-by: kkewwei kkewwei@163.com
Signed-off-by: kkewwei <kkewwei@163.com>

* Add doc for debugging rest tests (opensearch-project#14491)

* add doc for debugging rest tests

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* Update TESTING.md

Co-authored-by: Marc Handalian <handalm@amazon.com>
Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* Address comment

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

---------

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
Co-authored-by: Marc Handalian <handalm@amazon.com>

* Fix flaky DefaultCacheStatsHolderTests (opensearch-project#14462)

Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>

* [AUTO] [main] Add bwc version 2.15.1. (opensearch-project#14549)

* Add bwc version 2.15.1

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* Fix auto-generated version

Signed-off-by: Andrew Ross <andrross@amazon.com>

---------

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: Andrew Ross <andrross@amazon.com>
Co-authored-by: opensearch-ci-bot <83309141+opensearch-ci-bot@users.noreply.github.com>
Co-authored-by: Andrew Ross <andrross@amazon.com>

* Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently (opensearch-project#14550)

* Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Addressing comment

Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com>

---------

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>
Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com>
Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com>

* Add allowlist setting for ingest-common processors (opensearch-project#14479)

Add a new static setting that lets an operator choose specific ingest
processors to enable by name. The behavior is as follows:

- If the allowlist setting is not defined, all installed processors are
  enabled. This is the status quo.
- If the allowlist setting is defined as the empty set, then all processors
  are disabled.
- If the allowlist setting contains the names of valid processors, only those
  processors are enabled.
- If the allowlist setting contains a name of a processor that does not exist,
  then the server will fail to start with an IllegalStateException
  listing which processors were defined in the allowlist but are not
  installed.
- If the allowlist setting is changed between server restarts then any
  ingest pipeline using a now-disabled processor will fail. This is the
  same experience if a pipeline used a processor defined by a plugin but
  then that plugin were to be uninstalled across restarts.

Related to opensearch-project#14439

Signed-off-by: Andrew Ross <andrross@amazon.com>

* Fix file cache initialization (opensearch-project#14004)

* fix file cache initialization

Signed-off-by: panguixin <panguixin@bytedance.com>

* changelog

Signed-off-by: panguixin <panguixin@bytedance.com>

* add test

Signed-off-by: panguixin <panguixin@bytedance.com>

---------

Signed-off-by: panguixin <panguixin@bytedance.com>

* Add Ashish Singh as maintainer (opensearch-project#14567)

Signed-off-by: Bukhtawar Khan <bukhtawa@amazon.com>

* Allow @internalapi annotation on classes not meant to be constructed outside of the OpenSearch core (opensearch-project#14575)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Bump com.azure:azure-storage-common from 12.21.2 to 12.25.1 in /plugins/repository-azure (opensearch-project#14517)

* Bump com.azure:azure-storage-common in /plugins/repository-azure

Bumps [com.azure:azure-storage-common](https://github.com/Azure/azure-sdk-for-java) from 12.21.2 to 12.25.1.
- [Release notes](https://github.com/Azure/azure-sdk-for-java/releases)
- [Commits](Azure/azure-sdk-for-java@azure-storage-common_12.21.2...azure-storage-blob_12.25.1)

---
updated-dependencies:
- dependency-name: com.azure:azure-storage-common
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

* Add allowlist setting for search-pipeline-common processors (opensearch-project#14562)

Add a new static setting that lets an operator choose specific search
pipeline processors to enable by name. The behavior is as follows:

- If the allowlist setting is not defined, all installed processors are
  enabled. This is the status quo.
- If the allowlist setting is defined as the empty set, then all processors
  are disabled.
- If the allowlist setting contains the names of valid processors, only those
  processors are enabled.
- If the allowlist setting contains a name of a processor that does not exist,
  then the server will fail to start with an IllegalStateException
  listing which processors were defined in the allowlist but are not
  installed.
- If the allowlist setting is changed between server restarts then any
  ingest pipeline using a now-disabled processor will fail. This is the
  same experience if a pipeline used a processor defined by a plugin but
  then that plugin were to be uninstalled across restarts.

A distinct setting exists for each of request, response, and search phase
results processors.

Related to opensearch-project#14439

Signed-off-by: Andrew Ross <andrross@amazon.com>

* Bump Apache Lucene to 9.11.1 (opensearch-project#14576) (opensearch-project#14581)

(cherry picked from commit 0095fd1)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Add unittests for RemoteClusterStateAttributesManager (opensearch-project#14427)

* Add unittests for RemoteClusterStateAttributesManager

Signed-off-by: Shivansh Arora <hishiv@amazon.com>

* Add Ashish Singh to codeowners (opensearch-project#14592)

Signed-off-by: Ashish Singh <ssashish@amazon.com>

* Add batching processor base type AbstractBatchingProcessor (opensearch-project#14554)

Signed-off-by: Liyun Xiu <xiliyun@amazon.com>

* Add @internalapi annotation to japicmp exclusions (opensearch-project#14597)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix issue 14519:Parsing a GetResult returns NPE if found field is mis… (opensearch-project#14552)

* Fix issue 14519:Parsing a GetResult returns NPE if found field is missing. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com>

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>

* Fix issue 14519:Parsing a GetResult returns NPE if found field is missing. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com>

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>

* Fix issue 14519:Fix wildcart import. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com>

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>

* Fix issue 14519:Fix wildcart import. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com>

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>

* Fix issue 14519:Fix spotless issues. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com>

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>

* Fix issue 14519:update changelog

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>

---------

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Co-authored-by: Daniel Widdis <widdis@gmail.com>

* Star tree mapping changes (opensearch-project#14605)

* Star tree mapping changes with feature flag
---------
Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

* Bump com.microsoft.azure:msal4j from 1.15.1 to 1.16.0 in /plugins/repository-azure (opensearch-project#14610)

* Bump com.microsoft.azure:msal4j in /plugins/repository-azure

Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.15.1 to 1.16.0.
- [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases)
- [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt)
- [Commits](AzureAD/microsoft-authentication-library-for-java@v1.15.1...v1.16.0)

---
updated-dependencies:
- dependency-name: com.microsoft.azure:msal4j
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

* [Bugfix] Fix ICacheKeySerializerTests flakiness (opensearch-project#14564)

* Fix testInvalidInput flakiness

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

* Addressed andrross's comment

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

* rerun security check

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

---------

Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>

* Correct typo in method name (opensearch-project#14621)

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>

* Refactoring FilterPath.parse by using an iterative approach instead of recursion. (opensearch-project#14200)

* Refactor FilterPath parse function (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Implement unit tests for FilterPathTests (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Write warn log if Filter is empty; Add comments (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Add changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unnecessary log statement

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unused logger

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Spotless apply

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove incorrect changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

---------

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Removing String format in RemoteStoreMigrationAllocationDecider to optimise performance(opensearch-project#14612)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata; Correct the check for deciding upload of HashesOfConsistentSettings (opensearch-project#14513)

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata
* Correct the check for deciding upload of hashes of consistent settings

Signed-off-by: Sooraj Sinha <soosinha@amazon.com>

* Improve reroute performance by optimising List.removeAll in LocalShardsBalancer to filter remote search shard from relocation decision (opensearch-project#14613)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>

* Fix assertion failure while deleting remote backed index (opensearch-project#14601)

Signed-off-by: Sachin Kale <kalsac@amazon.com>

* OnHeap Star Tree Implementation

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* addressed nits

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* addressed major nits

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* includes Count Aggregator

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* handling for missing doc values

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* addressing review comments

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* rebasing with main

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* support for empty sequential doc values iterator

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* nits

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* min and max star tree aggregators

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* star tree file formats

* Star tree codec changes

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

* Adding tests

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

* Addressing comments

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

* addressing review comments

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

* Star tree merge changes

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

* fix annotations

* star-tree file formats reader and javadoc fixes

* read for composite index values

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* doc values file format

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

---------

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com>
Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>
Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com>
Signed-off-by: kkewwei kkewwei@163.com
Signed-off-by: kkewwei <kkewwei@163.com>
Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: Andrew Ross <andrross@amazon.com>
Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com>
Signed-off-by: panguixin <panguixin@bytedance.com>
Signed-off-by: Bukhtawar Khan <bukhtawa@amazon.com>
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Signed-off-by: Ashish Singh <ssashish@amazon.com>
Signed-off-by: Liyun Xiu <xiliyun@amazon.com>
Signed-off-by: vatsal <vatsal.v.anand@gmail.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
Signed-off-by: Sachin Kale <kalsac@amazon.com>
Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>
Signed-off-by: Bharathwaj G <bharath78910@gmail.com>
Co-authored-by: bowenlan-amzn <bowenlan23@gmail.com>
Co-authored-by: Prudhvi Godithi <pgodithi@amazon.com>
Co-authored-by: Sagar <99425694+sgup432@users.noreply.github.com>
Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com>
Co-authored-by: kkewwei <kkewwei@163.com>
Co-authored-by: Marc Handalian <handalm@amazon.com>
Co-authored-by: Peter Alfonsi <peter.alfonsi@gmail.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com>
Co-authored-by: opensearch-ci-bot <83309141+opensearch-ci-bot@users.noreply.github.com>
Co-authored-by: Andrew Ross <andrross@amazon.com>
Co-authored-by: panguixin <panguixin@bytedance.com>
Co-authored-by: Bukhtawar Khan <bukhtawa@amazon.com>
Co-authored-by: Andriy Redko <andriy.redko@aiven.io>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Co-authored-by: Shivansh Arora <hishiv@amazon.com>
Co-authored-by: Ashish Singh <ssashish@amazon.com>
Co-authored-by: Liyun Xiu <xiliyun@amazon.com>
Co-authored-by: Vatsal <36672090+imvtsl@users.noreply.github.com>
Co-authored-by: Daniel Widdis <widdis@gmail.com>
Co-authored-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
Co-authored-by: rishavz_sagar <rishavsagar4b1@gmail.com>
Co-authored-by: Sooraj Sinha <81695996+soosinha@users.noreply.github.com>
Co-authored-by: Sachin Kale <sachinpkale@gmail.com>
Co-authored-by: Sarthak Aggarwal <sarthagg@amazon.com>
harshavamsi pushed a commit to harshavamsi/OpenSearch that referenced this pull request Jul 12, 2024
…roject#14581)

(cherry picked from commit 0095fd1)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
wdongyu pushed a commit to wdongyu/OpenSearch that referenced this pull request Aug 22, 2024
…roject#14581)

(cherry picked from commit 0095fd1)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants