Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RELEASE] Release version 2.3.0 #2447

Closed
40 of 46 tasks
github-actions bot opened this issue Aug 10, 2022 · 31 comments
Closed
40 of 46 tasks

[RELEASE] Release version 2.3.0 #2447

github-actions bot opened this issue Aug 10, 2022 · 31 comments
Assignees
Labels
release v2.3.0 'Issues and PRs related to version v2.3.0'

Comments

@github-actions
Copy link
Contributor

github-actions bot commented Aug 10, 2022

Release OpenSearch and OpenSearch Dashboards 2.3.0

I noticed that a manifest was automatically created in manifests/2.3.0. Please follow the following checklist to make a release.

How to use this issue

This Release Issue

This issue captures the state of the OpenSearch release, its assignee is responsible for driving the release. Please contact them or @mention them on this issue for help. There are linked issues on components of the release where individual components can be tracked. More details are included in the Maintainers Release owner section.

Release Steps

There are several steps to the release process, these steps are completed as the whole release and components that are behind present risk to the release. The release owner completes the tasks in this ticket, whereas component owners resolve tasks on their ticket in their repositories.

Steps have completion dates for coordinating efforts between the components of a release; components can start as soon as they are ready far in advance of a future release.

Component List

To aid in understanding the state of the release there is a table with status indicating each component state. This is updated based on the status of the component issues.

Preparation

  • Assign this issue to a release owner.
  • Declare a pencils down date for new features to be merged.
  • September 02, 2022 is pencils down date for feature freeze.
  • Update the Campaigns section to include monitoring campaigns during this release.
  • Update this issue so all __REPLACE_RELEASE-__ placeholders have actual dates.
  • Document any new quality requirements or changes.
  • Finalize scope and feature set and update the Public Roadmap.
  • Create a release issue in every component repo that links back to this issue, update Components section with these links.
  • Ensure the label is created in each component repo for this new version, and the next minor release. Create a version label
  • Ensure that all release issues created above are assigned to an owner in the component team.

CI/CD (Feature Freeze) - Ends September 07, 2022

  • Create Jenkins workflows that run daily snapshot builds for OpenSearch and OpenSearch Dashboards.
  • Increment each component version to 2.3.0 and ensure working CI in component repositories.
  • Make pull requests to add each component to manifests/2.3.0/opensearch-2.3.0.yml and manifests/2.3.0/opensearch-dashboards-2.3.0.yml with the corresponding checks.
  • OpenSearch / OpenSearch-Dashboards core and components teams finalize their features
  • OpenSearch / OpenSearch-Dashboards core cut branch <MajorVersion>.<MinorVersion> early.

Campaigns

Code Complete - Ends September 07, 2022

Release testing - Ends September 12, 2022

  • All components should have cut branch <MajorVersion>.<MinorVersion> for the release.
  • Declare a release candidate build, and provide the instructions with the release candidates for teams on testing (September 08, 2022).
  • Stop builds for this version of OpenSearch and/or OpenSearch Dashboards in order to avoid accidental commits going in unknowingly. Restart only if necessary else manually run the build workflow and declare new release candidate.
  • Sanity Testing (September 08, 2022 - September 12, 2022): Sanity testing and fixing of critical issues found by teams. Teams test their components within the distribution, ensuring integration, backwards compatibility, and perf tests pass.
  • Publish all test results in the comments of this issue.

Performance testing validation - Ends September 12, 2022

  • Performance tests do not show a regression
  • Longevity tests do not show any issues

Release - Ends September 14 2022

Post Release

  • Create release tags for each component (Jenkins job name: release-tag-creation).
  • Replace refs in manifests/2.3.0 with tags and remove checks.
  • If this is a major or minor version release, stop building previous patch version.
  • Generate distribution release notes reviewed by PM team for opensearch-build repository.
  • Increment version for Helm Charts (sample PR) for the 2.3.0 release.
  • Increment version for Ansible Charts (sample PR) for the 2.3.0 release.
  • Prepare for next patch release by incrementing patch versions for each component.
  • Update this template with any new or missed steps.
  • Create an issue for a retrospective, solicit feedback, and publish a summary.

Components

Replace with links to all component tracking issues.

Component On track Release Notes
opensearch-project/OpenSearch#4357
opensearch-project/OpenSearch-Dashboards#2230
opensearch-project/alerting#534
opensearch-project/anomaly-detection#648
opensearch-project/asynchronous-search#173
opensearch-project/reporting#440
opensearch-project/dashboards-visualizations#110
opensearch-project/index-management#481
opensearch-project/job-scheduler#226
opensearch-project/k-NN#524
opensearch-project/performance-analyzer#260
opensearch-project/security-dashboards-plugin#1089
opensearch-project/security#2063
opensearch-project/sql#795
opensearch-project/observability#978
opensearch-project/alerting-dashboards-plugin#324
opensearch-project/notifications#516
opensearch-project/index-management-dashboards-plugin#242
opensearch-project/anomaly-detection-dashboards-plugin#316
opensearch-project/cross-cluster-replication#535
opensearch-project/common-utils#228
opensearch-project/ml-commons#413
opensearch-project/geospatial#136
opensearch-project/dashboards-maps#39
Legend

Symbol Meaning
🟢 On track with overall release
🟡 Missed last milestone
🔴 Missed multiple milestones

@github-actions github-actions bot added release untriaged Issues that have not yet been triaged v{{ env.VERSION }} labels Aug 10, 2022
@zelinh zelinh added v2.3.0 'Issues and PRs related to version v2.3.0' and removed untriaged Issues that have not yet been triaged v{{ env.VERSION }} labels Aug 16, 2022
@gaiksaya gaiksaya self-assigned this Aug 31, 2022
This was referenced Aug 31, 2022
@navneet1v
Copy link
Contributor

@navneet1v You can install the native plugins using the plugin install command as below (Example)

Block (18 lines)

 ./opensearch-plugin install https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/2.2.1/latest/linux/x64/tar/builds/opensearch/core-plugins/discovery-ec2-2.2.1.zip 
warning: ignoring JAVA_TOOL_OPTIONS=-Dlog4j2.formatMsgNoLookups=true
-> Installing https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/2.2.1/latest/linux/x64/tar/builds/opensearch/core-plugins/discovery-ec2-2.2.1.zip
-> Downloading https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/2.2.1/latest/linux/x64/tar/builds/opensearch/core-plugins/discovery-ec2-2.2.1.zip
[=================================================] 100%   
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@     WARNING: plugin requires additional permissions     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
* java.lang.RuntimePermission accessDeclaredMembers
* java.lang.RuntimePermission getClassLoader
* java.lang.reflect.ReflectPermission suppressAccessChecks
* java.net.SocketPermission * connect,resolve
See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
for descriptions of what these permissions allow and the associated risks.

Continue with installation? [y/N]y
-> Installed discovery-ec2 with folder name discovery-ec2

Yeah sure I will do that. Thanks a lot.

@opensearch-ci-bot
Copy link
Collaborator

opensearch-ci-bot commented Sep 9, 2022

OpenSearch CommitID(after 2022-09-06) & Release Notes info

Repo Branch CommitID Commit Date Release Notes
OpenSearch 2.3 d4a162b 2022-09-09 True
common-utils 2.3 b1009ed 2022-09-12 True
job-scheduler 2.3 99a9567 2022-09-09 True
security 2.3 0aedd61 2022-09-07 True
performance-analyzer 2.3 0ca646f 2022-09-13 False
cross-cluster-replication 2.3 78a6fb7 2022-09-07 True
anomaly-detection 2.3 e775dd3 2022-09-07 True
ml-commons 2.3 37d2d83 2022-09-12 True
k-NN 2.3 78e9383 2022-09-09 True
geospatial 2.3 f9ec0db 2022-09-09 True
opensearch-observability 2.3 add0e26 2022-09-09 True
dashboards-reports 2.3 34f6a09 2022-09-12 True
notifications-core 2.3 74e766d 2022-09-13 True
notifications 2.3 74e766d 2022-09-13 True
asynchronous-search 2.3 0c253c3 2022-09-13 True
sql 2.3 568f9a9 2022-09-13 True
alerting 2.3 34274a0 2022-09-09 True
index-management 2.3 7217b5b 2022-09-12 True

@opensearch-ci-bot
Copy link
Collaborator

opensearch-ci-bot commented Sep 9, 2022

OpenSearch Dashboards CommitID(after 2022-09-06) & Release Notes info

Repo Branch CommitID Commit Date Release Notes
OpenSearch-Dashboards 2.3 d8e4d4d 2022-09-13 True
functionalTestDashboards 2.3 False
securityDashboards 2.3 1d4ee91 2022-09-07 True
anomalyDetectionDashboards 2.3 bc39bb2 2022-09-13 True
observabilityDashboards 2.3 add0e26 2022-09-09 True
ganttChartDashboards 2.3 6acb32e 2022-09-09 True
reportsDashboards 2.3 34f6a09 2022-09-12 True
customImportMapDashboards 2.3 32e985a 2022-09-09 True
notificationsDashboards 2.3 74e766d 2022-09-13 True
queryWorkbenchDashboards 2.3 568f9a9 2022-09-13 True
indexManagementDashboards 2.3 0da978a 2022-09-12 True
alertingDashboards 2.3 a77eabd 2022-09-09 True

@navneet1v
Copy link
Contributor

Added release notes for K-NN, geospatial and customImportMapDashboards

@gaiksaya
Copy link
Member

Performance test runs for this version:
x64: https://build.ci.opensearch.org/job/perf-test/541
arm64: https://build.ci.opensearch.org/job/perf-test/540

@dreamer-89
Copy link
Member

dreamer-89 commented Sep 12, 2022

Performance test runs for this version: x64: https://build.ci.opensearch.org/job/perf-test/541 arm64: https://build.ci.opensearch.org/job/perf-test/540

I compared 2.3.0 with 2.2.1 (baseline). I don’t see any anomaly in results (there are minor ups/downs in latency/throughput. I observe similar numbers comparing older releases 2.2.0 (534 build no) & 2.2.1 (568)). So, I think numbers are fine.

Side notes

Build numbers used
2.2.1 -> 568, 569, 2.3.0 -> 540, 541

X64

Without Security
Minor indexing/search latency improvements, but minor degradation on throughput & search latency on p99/p100
With Security
Same observation as above. But minor search latency uptick across all p50-100. But throughput improvements.

Arm64

With Security
Minor indexing latency (~1.50%) degrade. ~.80% throughput degrade. Query latency improve & throughput increase ~1%
Without Security
Latency improve ~2%, throughput increase ~1%, minor Search query throughput decrease ~1%

@gaiksaya
Copy link
Member

gaiksaya commented Sep 12, 2022

Hi @saratvemulapalli @kotwanikunal @dblock Can you take a look on the above performance test findings?
Thanks!
cc: @CEHENKLE @bbarani

@saratvemulapalli
Copy link
Member

saratvemulapalli commented Sep 12, 2022

@gaiksaya @dreamer-89 trying to look at the runs, is there a human readable format for the results?

@kotwanikunal
Copy link
Member

kotwanikunal commented Sep 12, 2022

@saratvemulapalli

Results attached -

Version Architecture Security Enabled Start Time(PDT) Duration Instance Type Workload Details Index Latency(ms) p50 Index Latency(ms) p90 Index Latency(ms) p99 Index Latency(ms) p100 Index Throughput(docs/s) p0 Index Throughput(docs/s) p50 Index Throughput(docs/s) p100 Index Throughput (docs/s) Ops Count Index Operation Counts Op Error Count Index Operation Counts Op Error Rate Query Latency (ms) p50 Query Latency (ms) p90 Query Latency (ms) p99 Query Latency (ms) p100 Query Throughput (docs/s) p0 Query Throughput (docs/s) p50 Query Throughput (docs/s) p100 Query Operation Counts Ops Count Query Operation Counts Op Error Count Query Operation Counts Op Error Rate CPU (%) p50 CPU (%) p90 CPU (%) p99 CPU (%) p100 Memory (%) p50 Memory (%) p90 Memory (%) p99 Memory (%) p100 GC (ms) Old GC (ms) Young
2.2.1 arm64 with_security 9/11/22, 3:01 AM 6h 35m 33s m6g.xlarge nyc_taxis / 2 warmupIterations / 3 testIterations 1851.8598505373673 2456.7932834712446 4549.877352294918 6659.5146484375 39204.41796875 40484.896675314696 43940.48046875 46459 0 0 360.88779627482097 373.73701316833495 384.26831719970704 390.29581985473635 1.670055627822876 1.7078201830387116 1.7455639243125916 1510 0 0 96 97 97 97 49.2 64 76.96000000000004 91 0 985508
2.2.1 arm64 without_security 9/12/22, 3:01 AM 6h 10m 30s m6g.xlarge nyc_taxis / 2 warmupIterations / 3 testIterations 1646.453753 2210.699582 4342.278781 6122.276367 44029.94922 45598.62776 49929.39063 46013 0 0 362.3607367 374.8823477 378.7677886 391.089801 1.704830134 1.731174356 1.754105937 1510 0 0 96 97 97 97 48 62 83.8 92 0 829901
2.2.1 x64 with_security 9/11/22, 2:58 AM 8h 10m 40s m5.xlarge nyc_taxis / 2 warmupIterations / 3 testIterations 2288.2764957074537 3038.3064399577747 5879.324521689244 9268.974609375 32105.65234375 33927.90692669141 38235.58203125 46894 0 0 286.41736221313477 293.68860984802245 301.12121810913084 318.229997253418 1.7262567162513733 1.740007507801056 1.75261892080307 1510 0 0 97 97 97 97 52 66 87.76999999999992 93 0 1231253
2.2.1 x64 without_security 9/11/22, 2:58 AM 7h 49m 37s m5.xlarge nyc_taxis / 2 warmupIterations / 3 testIterations 2192.6901234475677 2914.758178724321 5700.923922209283 11106.1728515625 33684.34375 35614.496027572684 39991.8125 46757 0 0 295.2953796386719 302.1095615386963 309.9044530029297 322.45588912963865 1.7269416570663452 1.7409352322419487 1.7553181290626525 1510 0 0 97 97 98 98 49 63 84 94 0 1054789
2.3.0 arm64 with_security 9/9/22, 10:41 AM 6h 42m 35s m6g.xlarge nyc_taxis / 2 warmupIterations / 3 testIterations 1884.939796 2498.486831 4672.044931 8184.402832 38684.21484 40013.98915 43568.23438 46488 0 0 351.4363827 359.4697859 368.6659151 437.3339294 1.699884045 1.722742015 1.75187211 1510 0 0 96 97 97 97 52 65 88.98 94 0 979894
2.3.0 arm64 without_security 9/9/22, 10:43 AM 6h 12m 31s m6g.xlarge nyc_taxis / 2 warmupIterations / 3 testIterations 1625.392231 2180.028306 4150.245827 6011.796387 44516.28516 46462.11049 51156.74219 45950 0 0 364.6436938 373.184549 379.6433586 386.4868347 1.685038006 1.709957743 1.753778672 1510 0 0 96 97 97 97 48 64 83.16 91 0 853041
2.3.0 x64 with_security 9/9/22, 10:41 AM 8h 5m 38s m5.xlarge nyc_taxis / 2 warmupIterations / 3 testIterations 2274.482999 3001.790512 5625.219549 9248.959961 32337.31836 33916.93773 37967.18359 46902 0 0 290.8293233 300.3195797 309.3330738 314.5757118 1.725921583 1.739219183 1.752308607 1510 0 0 97 97 98 98 51 65 81.66 87 0 1238934
2.3.0 x64 without_security 9/9/22, 10:41 AM 7h 41m 37s m5.xlarge nyc_taxis / 2 warmupIterations / 3 testIterations 2174.641495 2896.447136 5494.624087 9741.860352 33844.23438 35826.79863 40342.28125 46705 0 0 287.6040042 297.3744589 310.0044941 328.9224983 1.731490076 1.743171066 1.757029057 1510 0 0 97 98 98 98 49 64 86.56 96 0 1033209

Difference (in %) -

Version Architecture Security Enabled Index Latency(ms) p50 Index Latency(ms) p90 Index Latency(ms) p99 Index Latency(ms) p100 Index Throughput(docs/s) p0 Index Throughput(docs/s) p50 Index Throughput(docs/s) p100 Index Throughput (docs/s) Ops Count Index Operation Counts Op Error Count Index Operation Counts Op Error Rate Query Latency (ms) p50 Query Latency (ms) p90 Query Latency (ms) p99 Query Latency (ms) p100 Query Throughput (docs/s) p0 Query Throughput (docs/s) p50 Query Throughput (docs/s) p100 Query Operation Counts Ops Count Query Operation Counts Op Error Count Query Operation Counts Op Error Rate CPU (%) p50 CPU (%) p90 CPU (%) p99 CPU (%) p100 Memory (%) p50 Memory (%) p90 Memory (%) p99 Memory (%) p100 GC (ms) Old GC (ms) Young
2.3 vs 2.2.1 arm64 with_security +1.79% +1.7% +2.69% +22.9% -1.33% -1.16% -0.85% +0.06% -2.62% -3.82% -4.06% +12.05% +1.79% +0.87% +0.36% 0% 0% 0% 0% 0% +5.69% +1.56% +15.62% +3.3% -0.57%
2.3 vs 2.2.1 arm64 without_security -1.28% -1.39% -4.42% -1.8% +1.1% +1.89% +2.46% -0.14% +0.63% -0.45% +0.23% -1.18% -1.16% -1.23% -0.02% 0% 0% 0% 0% 0% 0% +3.23% -0.76% -1.09% +2.79%
2.3 vs 2.2.1 x64 with_security -0.60% -1.20% -4.32% -0.22% 0.72% -0.03% -0.70% 0.02% 1.54% 2.26% 2.73% -1.15% -0.02% -0.05% -0.02% 0% 0% 0% 1.03% 1.03% -1.92% -1.52% -6.96% -6.45% 0.62%
2.3 vs 2.2.1 x64 without_security -0.82% -0.63% -3.62% -12.28% 0.47% 0.60% 0.88% -0.11% -2.60% -1.57% 0.03% 2.01% 0.26% 0.13% 0.10% 0% 0% 1.03% 0% 0% 0% 1.59% 3.05% 2.13% -2.05%

Thanks to Brandon! 🙂

Edit: Results are not anomalous, LGTM!

Edit: Updating the results. Sorry for the bad pasting skills. 🤦

@saratvemulapalli
Copy link
Member

saratvemulapalli commented Sep 12, 2022

Thanks @kotwanikunal, looks like our query latency has improved for arm64 without security by quite a bit (>20% lower latency). Do you guys know if you made any expected changes?

@kotwanikunal
Copy link
Member

Thanks @kotwanikunal, looks like our query latency has improved for arm64 without security by quite a bit (>20% lower latency). Do you guys know if you made any expected changes?

@saratvemulapalli My bad. I swapped the comparisons while pasting. Please have a look again.

@Zhangxunmt
Copy link
Contributor

Added release notes for ml-commons.

@gaiksaya
Copy link
Member

@saratvemulapalli
Copy link
Member

Thanks @kotwanikunal. Looks like the results are very similar. We are good to go!

@navneet1v
Copy link
Contributor

1 more thing while installing the discovery plugin on the cluster downloaded via tar for x86 I am getting this error.
Exception in thread "main" java.io.IOException: Server returned HTTP response code: 403 for URL: https://artifacts.opensearch.org/releases/plugins/discovery-ec2/2.3.0/discovery-ec2-2.3.0.zip
What should be the URL to install the discovery-plugin?

We haven't staged the native plugins yet. We will stage it after we finalize the RC builds. @gaiksaya @peterzhuamazon

Yes until then the plugins can be accessed via url example: https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/2.3.0/latest/linux/x64/tar/builds/opensearch/core-plugins/discovery-ec2-2.3.0.zip

Hi,
Have we staged the native plugins? I am trying to do the custom performance test for the K-NN plugin.

@navneet1v
Copy link
Contributor

Getting exception while installing the OpenSearch via tar.

warning: ignoring JAVA_TOOL_OPTIONS=-Dlog4j2.formatMsgNoLookups=true
[2022-09-13T01:16:38,994][INFO ][o.o.n.Node               ] [88665a4d7d84.ant.amazon.com] version[2.3.0], pid[83775], build[tar/6f6e84ebc54af31a976f53af36a5c69d474a5140/2022-09-09T00:07:12.137133581Z], OS[Mac OS X/12.5.1/x86_64], JVM[Eclipse Adoptium/OpenJDK 64-Bit Server VM/11.0.16/11.0.16+8]
[2022-09-13T01:16:38,997][INFO ][o.o.n.Node               ] [88665a4d7d84.ant.amazon.com] JVM home [/Users/navneev/.sdkman/candidates/java/11.0.16-tem], using bundled JDK [false]
[2022-09-13T01:16:38,998][INFO ][o.o.n.Node               ] [88665a4d7d84.ant.amazon.com] JVM arguments [-Xshare:auto, -Dopensearch.networkaddress.cache.ttl=60, -Dopensearch.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Xms1g, -Xmx1g, -XX:+UseG1GC, -XX:G1ReservePercent=25, -XX:InitiatingHeapOccupancyPercent=30, -Djava.io.tmpdir=/var/folders/rb/dv5bnd5n65vchm4c59_qbvpj1dqcnf/T/opensearch-10526221033923683325, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -XX:MaxDirectMemorySize=536870912, -Dopensearch.path.home=/Users/navneev/Downloads/opensearch-2.3.0, -Dopensearch.path.conf=/Users/navneev/Downloads/opensearch-2.3.0/config, -Dopensearch.distribution.type=tar, -Dopensearch.bundled_jdk=true]
[2022-09-13T01:16:40,004][WARN ][stderr                   ] [88665a4d7d84.ant.amazon.com] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[2022-09-13T01:16:40,005][WARN ][stderr                   ] [88665a4d7d84.ant.amazon.com] SLF4J: Defaulting to no-operation (NOP) logger implementation
[2022-09-13T01:16:40,005][WARN ][stderr                   ] [88665a4d7d84.ant.amazon.com] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[2022-09-13T01:16:40,017][INFO ][o.o.s.s.t.SSLConfig      ] [88665a4d7d84.ant.amazon.com] SSL dual mode is disabled
[2022-09-13T01:16:40,017][INFO ][o.o.s.OpenSearchSecurityPlugin] [88665a4d7d84.ant.amazon.com] OpenSearch Config path is /Users/navneev/Downloads/opensearch-2.3.0/config
[2022-09-13T01:16:40,273][INFO ][o.o.s.s.DefaultSecurityKeyStore] [88665a4d7d84.ant.amazon.com] JVM supports TLSv1.3
[2022-09-13T01:16:40,275][INFO ][o.o.s.s.DefaultSecurityKeyStore] [88665a4d7d84.ant.amazon.com] Config directory is /Users/navneev/Downloads/opensearch-2.3.0/config/, from there the key- and truststore files are resolved relatively
[2022-09-13T01:16:40,344][ERROR][o.o.b.OpenSearchUncaughtExceptionHandler] [88665a4d7d84.ant.amazon.com] uncaught exception in thread [main]
org.opensearch.bootstrap.StartupException: java.lang.IllegalStateException: failed to load plugin class [org.opensearch.security.OpenSearchSecurityPlugin]
	at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:184) ~[opensearch-2.3.0.jar:2.3.0]
	at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:171) ~[opensearch-2.3.0.jar:2.3.0]
	at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:104) ~[opensearch-2.3.0.jar:2.3.0]
	at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138) ~[opensearch-cli-2.3.0.jar:2.3.0]
	at org.opensearch.cli.Command.main(Command.java:101) ~[opensearch-cli-2.3.0.jar:2.3.0]
	at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:137) ~[opensearch-2.3.0.jar:2.3.0]
	at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:103) ~[opensearch-2.3.0.jar:2.3.0]
Caused by: java.lang.IllegalStateException: failed to load plugin class [org.opensearch.security.OpenSearchSecurityPlugin]
	at org.opensearch.plugins.PluginsService.loadPlugin(PluginsService.java:790) ~[opensearch-2.3.0.jar:2.3.0]
	at org.opensearch.plugins.PluginsService.loadBundle(PluginsService.java:730) ~[opensearch-2.3.0.jar:2.3.0]
	at org.opensearch.plugins.PluginsService.loadBundles(PluginsService.java:532) ~[opensearch-2.3.0.jar:2.3.0]
	at org.opensearch.plugins.PluginsService.<init>(PluginsService.java:195) ~[opensearch-2.3.0.jar:2.3.0]
	at org.opensearch.node.Node.<init>(Node.java:420) ~[opensearch-2.3.0.jar:2.3.0]
	at org.opensearch.node.Node.<init>(Node.java:347) ~[opensearch-2.3.0.jar:2.3.0]
	at org.opensearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:242) ~[opensearch-2.3.0.jar:2.3.0]
	at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242) ~[opensearch-2.3.0.jar:2.3.0]
	at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:404) ~[opensearch-2.3.0.jar:2.3.0]
	at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:180) ~[opensearch-2.3.0.jar:2.3.0]
	... 6 more
Caused by: java.lang.reflect.InvocationTargetException
	at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
	at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?]
	at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
	at java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[?:?]
	at org.opensearch.plugins.PluginsService.loadPlugin(PluginsService.java:781) ~[opensearch-2.3.0.jar:2.3.0]
	at org.opensearch.plugins.PluginsService.loadBundle(PluginsService.java:730) ~[opensearch-2.3.0.jar:2.3.0]
	at org.opensearch.plugins.PluginsService.loadBundles(PluginsService.java:532) ~[opensearch-2.3.0.jar:2.3.0]
	at org.opensearch.plugins.PluginsService.<init>(PluginsService.java:195) ~[opensearch-2.3.0.jar:2.3.0]
	at org.opensearch.node.Node.<init>(Node.java:420) ~[opensearch-2.3.0.jar:2.3.0]
	at org.opensearch.node.Node.<init>(Node.java:347) ~[opensearch-2.3.0.jar:2.3.0]
	at org.opensearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:242) ~[opensearch-2.3.0.jar:2.3.0]
	at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242) ~[opensearch-2.3.0.jar:2.3.0]
	at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:404) ~[opensearch-2.3.0.jar:2.3.0]
	at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:180) ~[opensearch-2.3.0.jar:2.3.0]
	... 6 more
Caused by: org.opensearch.OpenSearchException: plugins.security.ssl.transport.keystore_filepath or plugins.security.ssl.transport.server.pemcert_filepath and plugins.security.ssl.transport.client.pemcert_filepath must be set if transport ssl is requested.
	at org.opensearch.security.ssl.DefaultSecurityKeyStore.initTransportSSLConfig(DefaultSecurityKeyStore.java:419) ~[?:?]
	at org.opensearch.security.ssl.DefaultSecurityKeyStore.initSSLConfig(DefaultSecurityKeyStore.java:255) ~[?:?]
	at org.opensearch.security.ssl.DefaultSecurityKeyStore.<init>(DefaultSecurityKeyStore.java:176) ~[?:?]
	at org.opensearch.security.ssl.OpenSearchSecuritySSLPlugin.<init>(OpenSearchSecuritySSLPlugin.java:218) ~[?:?]
	at org.opensearch.security.OpenSearchSecurityPlugin.<init>(OpenSearchSecurityPlugin.java:262) ~[?:?]
	at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
	at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?]
	at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
	at java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[?:?]
	at org.opensearch.plugins.PluginsService.loadPlugin(PluginsService.java:781) ~[opensearch-2.3.0.jar:2.3.0]
	at org.opensearch.plugins.PluginsService.loadBundle(PluginsService.java:730) ~[opensearch-2.3.0.jar:2.3.0]
	at org.opensearch.plugins.PluginsService.loadBundles(PluginsService.java:532) ~[opensearch-2.3.0.jar:2.3.0]
	at org.opensearch.plugins.PluginsService.<init>(PluginsService.java:195) ~[opensearch-2.3.0.jar:2.3.0]
	at org.opensearch.node.Node.<init>(Node.java:420) ~[opensearch-2.3.0.jar:2.3.0]
	at org.opensearch.node.Node.<init>(Node.java:347) ~[opensearch-2.3.0.jar:2.3.0]
	at org.opensearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:242) ~[opensearch-2.3.0.jar:2.3.0]
	at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242) ~[opensearch-2.3.0.jar:2.3.0]
	at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:404) ~[opensearch-2.3.0.jar:2.3.0]
	at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:180) ~[opensearch-2.3.0.jar:2.3.0]
	... 6 more
uncaught exception in thread [main]
java.lang.IllegalStateException: failed to load plugin class [org.opensearch.security.OpenSearchSecurityPlugin]
Likely root cause: OpenSearchException[plugins.security.ssl.transport.keystore_filepath or plugins.security.ssl.transport.server.pemcert_filepath and plugins.security.ssl.transport.client.pemcert_filepath must be set if transport ssl is requested.]
	at org.opensearch.security.ssl.DefaultSecurityKeyStore.initTransportSSLConfig(DefaultSecurityKeyStore.java:419)
	at org.opensearch.security.ssl.DefaultSecurityKeyStore.initSSLConfig(DefaultSecurityKeyStore.java:255)
	at org.opensearch.security.ssl.DefaultSecurityKeyStore.<init>(DefaultSecurityKeyStore.java:176)
	at org.opensearch.security.ssl.OpenSearchSecuritySSLPlugin.<init>(OpenSearchSecuritySSLPlugin.java:218)
	at org.opensearch.security.OpenSearchSecurityPlugin.<init>(OpenSearchSecurityPlugin.java:262)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
	at org.opensearch.plugins.PluginsService.loadPlugin(PluginsService.java:781)
	at org.opensearch.plugins.PluginsService.loadBundle(PluginsService.java:730)
	at org.opensearch.plugins.PluginsService.loadBundles(PluginsService.java:532)
	at org.opensearch.plugins.PluginsService.<init>(PluginsService.java:195)
	at org.opensearch.node.Node.<init>(Node.java:420)
	at org.opensearch.node.Node.<init>(Node.java:347)
	at org.opensearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:242)
	at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242)
	at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:404)
	at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:180)
	at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:171)
	at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:104)
	at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
	at org.opensearch.cli.Command.main(Command.java:101)
	at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:137)
	at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:103)
For complete error details, refer to the log at /Users/navneev/Downloads/opensearch-2.3.0/logs/opensearch.log

@peternied
Copy link
Member

@navneet1v Please create an issue with repro steps for the above issue so we can get someone assigned to look into it

@navneet1v
Copy link
Contributor

@navneet1v Please create an issue with repro steps for the above issue so we can get someone assigned to look into it

Sure will do that.

@gaiksaya
Copy link
Member

1 more thing while installing the discovery plugin on the cluster downloaded via tar for x86 I am getting this error.
Exception in thread "main" java.io.IOException: Server returned HTTP response code: 403 for URL: https://artifacts.opensearch.org/releases/plugins/discovery-ec2/2.3.0/discovery-ec2-2.3.0.zip
What should be the URL to install the discovery-plugin?

We haven't staged the native plugins yet. We will stage it after we finalize the RC builds. @gaiksaya @peterzhuamazon

Yes until then the plugins can be accessed via url example: https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/2.3.0/latest/linux/x64/tar/builds/opensearch/core-plugins/discovery-ec2-2.3.0.zip

Hi, Have we staged the native plugins? I am trying to do the custom performance test for the K-NN plugin.

No we have not. We will to do so today. Artifacts are staged after the teams finish the sanity/smoke testing which was yesterday EOD.
Thanks!

@gaiksaya
Copy link
Member

$ sudo ./opensearch-plugin install discovery-ec2
-> Installing discovery-ec2
-> Downloading discovery-ec2 from opensearch
[=================================================] 100%
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@     WARNING: plugin requires additional permissions     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
* java.lang.RuntimePermission accessDeclaredMembers
* java.lang.RuntimePermission getClassLoader
* java.lang.reflect.ReflectPermission suppressAccessChecks
* java.net.SocketPermission * connect,resolve
See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
for descriptions of what these permissions allow and the associated risks.

Continue with installation? [y/N]y
-> Installed discovery-ec2 with folder name discovery-ec2

@gaiksaya
Copy link
Member

@gaiksaya
Copy link
Member

Retrospective issue to track any missed steps, improvements: #2620

@gaiksaya
Copy link
Member

OpenSearch 2.3.0 has been released. 🎊

@gaiksaya
Copy link
Member

PRs to increment 2.3.0 to 2.3.1 are up for all components. Please review and merge.
Thanks!

@gaiksaya gaiksaya unpinned this issue Sep 20, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release v2.3.0 'Issues and PRs related to version v2.3.0'
Projects
None yet
Development

No branches or pull requests

10 participants