[RELEASE] Release version 2.11.0

[github-actions]

Release OpenSearch and OpenSearch Dashboards 2.11.0

I noticed that a manifest was automatically created in manifests/2.11.0. Please follow the following checklist to make a release.

How to use this issue

This Release Issue

This issue captures the state of the OpenSearch release, its assignee (Release Manager) is responsible for driving the release. Please contact them or @mention them on this issue for help. There are linked issues on components of the release where individual components can be tracked. For more information check the the Release Process OpenSearch Guide.

Please refer to the following link for the release version dates: Release Schedule and Maintenance Policy.

OpenSearch 2.11.0 exit criteria status:

OpenSearch	Status	Description	Comments
Performance tests are run, results are posted to the release ticket and there no unexpected regressions	🟢	The perf test for 2.11.0 RC candidate would be automatically kicked off by this workflow	Performance benchmark runs for this RC will be automatically initiated soon.
Documentation has been fully reviewed and signed off by the documentation community.	🟢
All integration tests are passing	🟢
Release blog is ready	🟢	Release blog has been finalized

OpenSearch dashboards 2.11.0 exit criteria status:

OpenSearch Dashboards	Status	Description	Comments
Performance tests are run, results are posted to the release ticket and there no unexpected regressions	🟢	Performance tests are not executed for OpenSearch Dashboards	Performance tests are not executed for OpenSearch Dashboards
Documentation has been fully reviewed and signed off by the documentation community.	🟢
All integration tests are passing	🟢
Release blog is ready	🟢	Release blog has been finalized

Preparation

• Release manager assigned.
• Existence of label in each component repo. For more information check the release-label section.
• Increase the build frequency.
• Release Issue.

Campaigns

• Component Release Issue.
• Release Campaigns.

Release Branch and Version Increment - Ends Oct 04, 2023 - Owner: [Repository Owner, component release manager]

• Core Release Branch.
• Core Version Increment.
• Components Release Branch.
• Components Version Increment.

Code Complete - Ends Oct 04, 2023 5PM PST- Owner: [Repository Owner, component release manager, build release manager]

• Mark this as done once the Code Complete is reviewed.

Release Candidate Creation and Testing - Starts Oct 05, 2023 - Owner: [Build release manager]

• Raise pull requests to add each component to input manifests/2.11.0/opensearch-2.11.0.yml and manifests/2.11.0/opensearch-dashboards-2.11.0.yml.
• Generate Release Candidate.
• Integ Test TAR.
• Integ Test RPM.
• Docker Build and Scan.
• Backwards Compatibility Tests.
• Windows Integration Test.
• Broadcast and Communication.
• Release Candidate Lock.

Performance testing validation - Starts Oct 05, 2023 - Owner: [Repository Owner, component release manager]

• Post the benchmark-tests
• Longevity tests do not show any issues.

Pre Release - TBD - Owner: [Build release manager]

• Release Labeled Issues.
• Go or No-Go.
• Promote Repos.
• Promote artifacts.
• Release Notes.

Release - TBD - Owner: [Build release manager]

• Maven Promotion.
• Docker Promotion.
• Release Validation.
• Collaboration with the Project Management Team.

Release Checklist.

Release Checklist

Pre-Release activities

• Promote Repos.
  • • OS
  • • OSD
• Promote Artifacts.
  • • Windows
  • • Linux Debian
  • • Linux RPM
  • • Linux TAR
• Consolidated Release Notes.

Release activities

• Docker Promotion.
• Release Validation part 1.
  • • OpenSearch and OpenSearch Dashboard Validation.
  • • Validate the native plugin installation.
• Merge consolidated release notes PR.
• Website and Documentation Changes.
  • • Merge staging website PR.
  • • Promote the website changes to prod.
  • • Add website alert.
• Release Validation part 2.
  • • Validate the artifact download URL's and signatures.
• Release Validation part 3.
  • • Trigger the validation build (Search for Completed validation for <> in the logs).
• Maven Promotion.
• Publish blog posts.
• Advertise on Social Media.
• Post on public slack and Github Release issue.

Post-Release activities

• Release Tags.
• Input Manifest Update.
• Decrease the Build Frequency.
• OpenSearch Build Release notes.
• Retrospective Issue.
• Helm and Ansible Playbook release.
• Upcoming Release Preparation.

Post Release

• Release Tags.
• Input Manifest Update.
• OpenSearch Build Release notes.
• Decrease the Build Frequency.
• Retrospective Issue.
• Helm and Ansible Playbook release.
• Upcoming Release Preparation.

Components

OpenSearch

Component	Issues tagged with 2.11.0
opensearch-project/OpenSearch#8754
opensearch-project/common-utils#496
opensearch-project/k-NN#995
opensearch-project/geospatial#357
opensearch-project/security#3028
opensearch-project/cross-cluster-replication#1080
opensearch-project/ml-commons#1141
opensearch-project/neural-search#224
opensearch-project/notifications#711
opensearch-project/observability#1576
opensearch-project/reporting#729
opensearch-project/sql#1882
opensearch-project/asynchronous-search#316
opensearch-project/anomaly-detection#964
opensearch-project/alerting#1047
opensearch-project/security-analytics#493
opensearch-project/index-management#860
opensearch-project/performance-analyzer#508

OpenSearch Dashboards

Component	Issues tagged with 2.11.0
opensearch-project/OpenSearch-Dashboards#4584
opensearch-project/opensearch-dashboards-functional-test#766
opensearch-project/dashboards-observability#703
opensearch-project/dashboards-reporting#149
opensearch-project/dashboards-visualizations#212
opensearch-project/dashboards-query-workbench#96
opensearch-project/dashboards-maps#443
opensearch-project/anomaly-detection-dashboards-plugin#558
opensearch-project/ml-commons-dashboards#235
opensearch-project/index-management-dashboards-plugin#819
opensearch-project/dashboards-notifications#86
opensearch-project/alerting-dashboards-plugin#655
opensearch-project/security-analytics-dashboards-plugin#661
opensearch-project/security-dashboards-plugin#1519
opensearch-project/dashboards-search-relevance#253

Legend

Symbol	Meaning
🟢	On track with overall release
🟡	Missed last milestone
🔴	Missed multiple milestones

[opensearch-ci-bot]

OpenSearch Dashboards CommitID(after 2023-09-25) & Release Notes info

Repo	Branch	CommitID	Commit Date	Release Notes
OpenSearch-Dashboards	[3c838160fe6b453881ba326c6c8f57e11cf10fdd]	3c83816	2023-10-10	True
functionalTestDashboards	[b0877cfc1a55a5b024493732b2892e22a221ff94]	b0877cf	2023-10-10	False
observabilityDashboards	[3f169c42d71d2dcf5775d7629c9cbbc94d36ed9f]	3f169c4	2023-10-09	False
reportsDashboards	[854efb3fe8c1dcae2648c03451d199fb6275ba2d]	854efb3	2023-10-10	False
ganttChartDashboards	[93092dbf97820c257d9c51a2856321e83a0a4eee]			False
queryWorkbenchDashboards	[aeb4974e45acf4b00ab6ffa66f28e5ff1c3d51b0]	aeb4974	2023-10-05	False
customImportMapDashboards	[ff5fb8027f0263ea04cf991f75febcdce88ce80b]	ff5fb80	2023-10-06	True
anomalyDetectionDashboards	[5f32b78294adb2a5704dc33d9267d580430a1952]	5f32b78	2023-09-26	False
mlCommonsDashboards	[fdc9f446301987eda68cb821b099268566ab6a5a]	fdc9f44	2023-10-05	False
indexManagementDashboards	[58d9170136ab59708e2d72b089080d24cd856171]			False
notificationsDashboards	[9f2f96b5755b92037842d208681cab09900a7c2a]	9f2f96b	2023-10-07	True
alertingDashboards	[7c573cba222c48711cd0fd170ef5e08486d4a131]	7c573cb	2023-10-02	False
securityAnalyticsDashboards	[4b03bee10ee9fc88c2228a4d75e573b643c7c58a]	4b03bee	2023-10-05	False
securityDashboards	[3bb8afea634a364c78ed3431cad20e5263688fdb]	3bb8afe	2023-10-06	False
searchRelevanceDashboards	[a869ba08361b0ce089e7bff3cd6ea09b2c558787]	a869ba0	2023-10-04	False

[dhrubo-os]

I'm trying to spin up the cluster, but I can't login to the dashboard:

I tried to clear my session, but still same.

[bbarani]

I am seeing the same error as @dhrubo-os . This seems to be related to observability dashboards plugin. @anirudha Can you look in to this issue?

[hdhalter]

Documentation status: RED. Out of 16 issues we are tracking, 5 are Done, 6 are in some state of progress, and 5 are in our backlog. Most issues in our backlog (meaning we have not started them) came in on or after 10/2. We have moved 7 issues to 2.12, and for another one, we are waiting for confirmation as to whether it's in 2.11 or 2.12. For a list of issues we are tracking, see https://github.com/opensearch-project/documentation-website/issues?q=is%3Aissue+label%3Av2.11.0+is%3Aopen.

[ps48]

@bbarani @dhrubo-os @rishabh6788, Observability has hard dependency on a dashboards feature. Dashboards-Core backport to 2.11 for this commit went in after the initial branch cut. Should be fixed in next RC.

[rishabh6788]

@ps48 Thank you for the update.
We will generating a new RC candidate at 4pm PST and share the details.

[rishabh6788]

OpenSearch 8607 / OpenSearch Dashboards 6649 is ready for your test.

OpenSearch - Build 8607
OpenSearch Dashboards - Build 6649

• Use the following Docker-Compose to setup a cluster

  docker-compose.yml

    
  version: '3'
  services:
    opensearch-node1:
      image: opensearchstaging/opensearch:2.11.0.8607
      container_name: opensearch-node1
      environment:
        - cluster.name=opensearch-cluster
        - node.name=opensearch-node1
        - discovery.seed_hosts=opensearch-node1,opensearch-node2
        - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
        - bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
        - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
      ulimits:
        memlock:
          soft: -1
          hard: -1
        nofile:
          soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
          hard: 65536
      volumes:
        - opensearch-data1:/usr/share/opensearch/data
      ports:
        - 9200:9200
        - 9600:9600 # required for Performance Analyzer
      networks:
        - opensearch-net
    opensearch-node2:
      image: opensearchstaging/opensearch:2.11.0.8607
      container_name: opensearch-node2
      environment:
        - cluster.name=opensearch-cluster
        - node.name=opensearch-node2
        - discovery.seed_hosts=opensearch-node1,opensearch-node2
        - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
        - bootstrap.memory_lock=true
        - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
      ulimits:
        memlock:
          soft: -1
          hard: -1
        nofile:
          soft: 65536
          hard: 65536
      volumes:
        - opensearch-data2:/usr/share/opensearch/data
      networks:
        - opensearch-net
    opensearch-dashboards:
      image: opensearchstaging/opensearch-dashboards:2.11.0.6649
      container_name: opensearch-dashboards
      ports:
        - 5601:5601
      expose:
        - "5601"
      environment:
        OPENSEARCH_HOSTS: '["https://opensearch-node1:9200","https://opensearch-node2:9200"]'
      networks:
        - opensearch-net
  volumes:
    opensearch-data1:
    opensearch-data2:
  networks:
  
  opensearch-net:
  
  
  
  

  • Download the above docker-compose.yml on your machine.
  • Get latest image versions docker-compose pull.
  • Start the cluster docker-compose up.

• OpenSearch docker 2.11.0.8607

  • Start without security
    • Docker command docker pull opensearchstaging/opensearch:2.11.0.8607 && docker run -it -p 9200:9200 -e "discovery.type=single-node" -e "DISABLE_SECURITY_PLUGIN=true" opensearchstaging/opensearch:2.11.0.8607
    • Connect command curl https://localhost:9200/
  • Start with security
    • Docker command docker pull opensearchstaging/opensearch:2.11.0.8607 && docker run -it -p 9200:9200 -e "discovery.type=single-node" opensearchstaging/opensearch:2.11.0.8607
    • Connect command curl --insecure https://admin:admin@localhost:9200/

• OpenSearch-Dashboards docker 2.11.0.6649

  • Start without security
    • Docker command docker pull opensearchstaging/opensearch-dashboards:2.11.0.6649 && docker run -it --network="host" -e "DISABLE_SECURITY_DASHBOARDS_PLUGIN=true" opensearchstaging/opensearch-dashboards:2.11.0.6649
    • URL https://localhost:5601/
  • Start with security
    • Docker command docker pull opensearchstaging/opensearch-dashboards:2.11.0.6649 && docker run -it --network="host" opensearchstaging/opensearch-dashboards:2.11.0.6649
    • URL https://localhost:5601/

• Use TARs to deploy OpenSearch Manually

  • OpenSearch - Build 8607 (Note: Windows version does not have performance analyzer plugin)

    • arm64 [manifest] [tar] [rpm][deb]
    • x64 [manifest] [tar] [rpm] [deb] [windows]

  • OpenSearch Dashboards - Build 6649

    • arm64 [manifest] [tar][rpm][deb]
    • x64 [manifest] [tar][rpm] [deb] [windows]

Check how to install opensearch and dashboards on different platforms

[rishabh6788]

OpenSearch 8618 / OpenSearch Dashboards 6651 is ready for your test.

OpenSearch - Build 8618
OpenSearch Dashboards - Build 6651

• Use the following Docker-Compose to setup a cluster

  docker-compose.yml

    
  version: '3'
  services:
    opensearch-node1:
      image: opensearchstaging/opensearch:2.11.0.8618
      container_name: opensearch-node1
      environment:
        - cluster.name=opensearch-cluster
        - node.name=opensearch-node1
        - discovery.seed_hosts=opensearch-node1,opensearch-node2
        - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
        - bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
        - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
      ulimits:
        memlock:
          soft: -1
          hard: -1
        nofile:
          soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
          hard: 65536
      volumes:
        - opensearch-data1:/usr/share/opensearch/data
      ports:
        - 9200:9200
        - 9600:9600 # required for Performance Analyzer
      networks:
        - opensearch-net
    opensearch-node2:
      image: opensearchstaging/opensearch:2.11.0.8618
      container_name: opensearch-node2
      environment:
        - cluster.name=opensearch-cluster
        - node.name=opensearch-node2
        - discovery.seed_hosts=opensearch-node1,opensearch-node2
        - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
        - bootstrap.memory_lock=true
        - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
      ulimits:
        memlock:
          soft: -1
          hard: -1
        nofile:
          soft: 65536
          hard: 65536
      volumes:
        - opensearch-data2:/usr/share/opensearch/data
      networks:
        - opensearch-net
    opensearch-dashboards:
      image: opensearchstaging/opensearch-dashboards:2.11.0.6651
      container_name: opensearch-dashboards
      ports:
        - 5601:5601
      expose:
        - "5601"
      environment:
        OPENSEARCH_HOSTS: '["https://opensearch-node1:9200","https://opensearch-node2:9200"]'
      networks:
        - opensearch-net
  volumes:
    opensearch-data1:
    opensearch-data2:
  networks:
  
  opensearch-net:
  
  
  
  

  • Download the above docker-compose.yml on your machine.
  • Get latest image versions docker-compose pull.
  • Start the cluster docker-compose up.

• OpenSearch docker 2.11.0.8618

  • Start without security
    • Docker command docker pull opensearchstaging/opensearch:2.11.0.8618 && docker run -it -p 9200:9200 -e "discovery.type=single-node" -e "DISABLE_SECURITY_PLUGIN=true" opensearchstaging/opensearch:2.11.0.8618
    • Connect command curl https://localhost:9200/
  • Start with security
    • Docker command docker pull opensearchstaging/opensearch:2.11.0.8618 && docker run -it -p 9200:9200 -e "discovery.type=single-node" opensearchstaging/opensearch:2.11.0.8618
    • Connect command curl --insecure https://admin:admin@localhost:9200/

• OpenSearch-Dashboards docker 2.11.0.6651

  • Start without security
    • Docker command docker pull opensearchstaging/opensearch-dashboards:2.11.0.6651 && docker run -it --network="host" -e "DISABLE_SECURITY_DASHBOARDS_PLUGIN=true" opensearchstaging/opensearch-dashboards:2.11.0.6651
    • URL https://localhost:5601/
  • Start with security
    • Docker command docker pull opensearchstaging/opensearch-dashboards:2.11.0.6651 && docker run -it --network="host" opensearchstaging/opensearch-dashboards:2.11.0.6651
    • URL https://localhost:5601/

• Use TARs to deploy OpenSearch Manually

  • OpenSearch - Build 8618 (Note: Windows version does not have performance analyzer plugin)

    • arm64 [manifest] [tar] [rpm][deb]
    • x64 [manifest] [tar] [rpm] [deb] [windows]

  • OpenSearch Dashboards - Build 6651

    • arm64 [manifest] [tar][rpm][deb]
    • x64 [manifest] [tar][rpm] [deb] [windows]

Check how to install opensearch and dashboards on different platforms

[rishabh6788]

Integration Test status with 3rd RC.

All components passing for OpenSearch. See https://build.ci.opensearch.org/blue/organizations/jenkins/integ-test/detail/integ-test/6317/pipeline/104

All Components failing for OpenSearch-Dashboards with Dashboards facing Authentication Exception. See https://build.ci.opensearch.org/blue/organizations/jenkins/integ-test-opensearch-dashboards/detail/integ-test-opensearch-dashboards/4253/pipeline/115.

Attached sample dashboards-log file.
opensearch_dashboards.log
@bbarani @CEHENKLE @seanneumann

[rishabh6788]

RC#4 OpenSearch 8624 / OpenSearch Dashboards 6654 is ready for your test.

OpenSearch - Build 8624
OpenSearch Dashboards - Build 6654

• Use the following Docker-Compose to setup a cluster

  docker-compose.yml

    
  version: '3'
  services:
    opensearch-node1:
      image: opensearchstaging/opensearch:2.11.0.8624
      container_name: opensearch-node1
      environment:
        - cluster.name=opensearch-cluster
        - node.name=opensearch-node1
        - discovery.seed_hosts=opensearch-node1,opensearch-node2
        - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
        - bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
        - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
      ulimits:
        memlock:
          soft: -1
          hard: -1
        nofile:
          soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
          hard: 65536
      volumes:
        - opensearch-data1:/usr/share/opensearch/data
      ports:
        - 9200:9200
        - 9600:9600 # required for Performance Analyzer
      networks:
        - opensearch-net
    opensearch-node2:
      image: opensearchstaging/opensearch:2.11.0.8624
      container_name: opensearch-node2
      environment:
        - cluster.name=opensearch-cluster
        - node.name=opensearch-node2
        - discovery.seed_hosts=opensearch-node1,opensearch-node2
        - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
        - bootstrap.memory_lock=true
        - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
      ulimits:
        memlock:
          soft: -1
          hard: -1
        nofile:
          soft: 65536
          hard: 65536
      volumes:
        - opensearch-data2:/usr/share/opensearch/data
      networks:
        - opensearch-net
    opensearch-dashboards:
      image: opensearchstaging/opensearch-dashboards:2.11.0.6654
      container_name: opensearch-dashboards
      ports:
        - 5601:5601
      expose:
        - "5601"
      environment:
        OPENSEARCH_HOSTS: '["https://opensearch-node1:9200","https://opensearch-node2:9200"]'
      networks:
        - opensearch-net
  volumes:
    opensearch-data1:
    opensearch-data2:
  networks:
  
  opensearch-net:
  
  
  
  

  • Download the above docker-compose.yml on your machine.
  • Get latest image versions docker-compose pull.
  • Start the cluster docker-compose up.

• OpenSearch docker 2.11.0.8624

  • Start without security
    • Docker command docker pull opensearchstaging/opensearch:2.11.0.8624 && docker run -it -p 9200:9200 -e "discovery.type=single-node" -e "DISABLE_SECURITY_PLUGIN=true" opensearchstaging/opensearch:2.11.0.8624
    • Connect command curl https://localhost:9200/
  • Start with security
    • Docker command docker pull opensearchstaging/opensearch:2.11.0.8624 && docker run -it -p 9200:9200 -e "discovery.type=single-node" opensearchstaging/opensearch:2.11.0.8624
    • Connect command curl --insecure https://admin:admin@localhost:9200/

• OpenSearch-Dashboards docker 2.11.0.6654

  • Start without security
    • Docker command docker pull opensearchstaging/opensearch-dashboards:2.11.0.6654 && docker run -it --network="host" -e "DISABLE_SECURITY_DASHBOARDS_PLUGIN=true" opensearchstaging/opensearch-dashboards:2.11.0.6654
    • URL https://localhost:5601/
  • Start with security
    • Docker command docker pull opensearchstaging/opensearch-dashboards:2.11.0.6654 && docker run -it --network="host" opensearchstaging/opensearch-dashboards:2.11.0.6654
    • URL https://localhost:5601/

• Use TARs to deploy OpenSearch Manually

  • OpenSearch - Build 8624 (Note: Windows version does not have performance analyzer plugin)

    • arm64 [manifest] [tar] [rpm][deb]
    • x64 [manifest] [tar] [rpm] [deb] [windows]

  • OpenSearch Dashboards - Build 6654

    • arm64 [manifest] [tar][rpm][deb]
    • x64 [manifest] [tar][rpm] [deb] [windows]

Check how to install opensearch and dashboards on different platforms

[rishabh6788]

Integration Test status with 4th RC.

All components passing for OpenSearch. See https://build.ci.opensearch.org/blue/organizations/jenkins/integ-test/detail/integ-test/6325/pipeline/

Failing Components for OpenSearch-Dashboards:

• alertingDashboards: See logs
• observabilityDashboards: See logs
• reportsDashboards: See logs
• securityAnalyticsDashboards: See logs

[rishabh6788]

Release Notes Readiness

Below components don't have Release Notes created yet for 2.11.0

OpenSearch:

• OpenSearch
• common-utils
• security
• cross-cluster-replication
• ml-commons
• opensearch-observability
• opensearch-reports
• sql
• alerting
• security-analytics
• index-management
• custom-codecs

OpenSearch Dashboards

• OpenSearch-Dashboards
• observabilityDashboards
• reportsDashboards
• ganttChartDashboards
• queryWorkbenchDashboards
• anomalyDetectionDashboards
• mlCommonsDashboards
• indexManagementDashboards
• alertingDashboards
• securityAnalyticsDashboards
• securityDashboards
• searchRelevanceDashboards

[rishabh6788]

RPM Integration Tests for OpenSearch

Failing components:

• neural-search (WITHOUT-SECURITY): See logs
• notifications (WITHOUT-SECURITY): See logs
• security-analytics (WITHOUT-SECURITY): See logs

DEB Integration Tests for OpenSearch

• alerting (WITH-SECURITY): See logs
• notifications (WITHOUT-SECURITY): See logs
• security-analytics (WITHOUT-SECURITY): See logs

[hdhalter]

Documentation status: 6 Done, 8 in various stages of doc/tech review, 3 in backlog or in progress.

[rishabh6788]

RC #5 OpenSearch 8634 / OpenSearch Dashboards 6658 is ready for your test.

OpenSearch - Build 8634
OpenSearch Dashboards - Build 6658

• Use the following Docker-Compose to setup a cluster

  docker-compose.yml

    
  version: '3'
  services:
    opensearch-node1:
      image: opensearchstaging/opensearch:2.11.0.8634
      container_name: opensearch-node1
      environment:
        - cluster.name=opensearch-cluster
        - node.name=opensearch-node1
        - discovery.seed_hosts=opensearch-node1,opensearch-node2
        - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
        - bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
        - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
      ulimits:
        memlock:
          soft: -1
          hard: -1
        nofile:
          soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
          hard: 65536
      volumes:
        - opensearch-data1:/usr/share/opensearch/data
      ports:
        - 9200:9200
        - 9600:9600 # required for Performance Analyzer
      networks:
        - opensearch-net
    opensearch-node2:
      image: opensearchstaging/opensearch:2.11.0.8634
      container_name: opensearch-node2
      environment:
        - cluster.name=opensearch-cluster
        - node.name=opensearch-node2
        - discovery.seed_hosts=opensearch-node1,opensearch-node2
        - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
        - bootstrap.memory_lock=true
        - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
      ulimits:
        memlock:
          soft: -1
          hard: -1
        nofile:
          soft: 65536
          hard: 65536
      volumes:
        - opensearch-data2:/usr/share/opensearch/data
      networks:
        - opensearch-net
    opensearch-dashboards:
      image: opensearchstaging/opensearch-dashboards:2.11.0.6658
      container_name: opensearch-dashboards
      ports:
        - 5601:5601
      expose:
        - "5601"
      environment:
        OPENSEARCH_HOSTS: '["https://opensearch-node1:9200","https://opensearch-node2:9200"]'
      networks:
        - opensearch-net
  volumes:
    opensearch-data1:
    opensearch-data2:
  networks:
  
  opensearch-net:
  
  
  
  

  • Download the above docker-compose.yml on your machine.
  • Get latest image versions docker-compose pull.
  • Start the cluster docker-compose up.

• OpenSearch docker 2.11.0.8634

  • Start without security
    • Docker command docker pull opensearchstaging/opensearch:2.11.0.8634 && docker run -it -p 9200:9200 -e "discovery.type=single-node" -e "DISABLE_SECURITY_PLUGIN=true" opensearchstaging/opensearch:2.11.0.8634
    • Connect command curl https://localhost:9200/
  • Start with security
    • Docker command docker pull opensearchstaging/opensearch:2.11.0.8634 && docker run -it -p 9200:9200 -e "discovery.type=single-node" opensearchstaging/opensearch:2.11.0.8634
    • Connect command curl --insecure https://admin:admin@localhost:9200/

• OpenSearch-Dashboards docker 2.11.0.6658

  • Start without security
    • Docker command docker pull opensearchstaging/opensearch-dashboards:2.11.0.6658 && docker run -it --network="host" -e "DISABLE_SECURITY_DASHBOARDS_PLUGIN=true" opensearchstaging/opensearch-dashboards:2.11.0.6658
    • URL https://localhost:5601/
  • Start with security
    • Docker command docker pull opensearchstaging/opensearch-dashboards:2.11.0.6658 && docker run -it --network="host" opensearchstaging/opensearch-dashboards:2.11.0.6658
    • URL https://localhost:5601/

• Use TARs to deploy OpenSearch Manually

  • OpenSearch - Build 8634 (Note: Windows version does not have performance analyzer plugin)

    • arm64 [manifest] [tar] [rpm][deb]
    • x64 [manifest] [tar] [rpm] [deb] [windows]

  • OpenSearch Dashboards - Build 6658

    • arm64 [manifest] [tar][rpm][deb]
    • x64 [manifest] [tar][rpm] [deb] [windows]

Check how to install opensearch and dashboards on different platforms

[rishabh6788]

2.11.0 Performance comparison with 2.10.0

There is no performance change between 2.11.0 and 2.10.0 on overall indexing and query metrics.

http_logs (x64)

http_logs (arm64)

nyc_taxis (x64)

nyc_taxis (arm64)

[rishabh6788]

RC #6 OpenSearch 8641 / OpenSearch Dashboards 6660 is ready for your test.

OpenSearch - Build 8641
OpenSearch Dashboards - Build 6660

• Use the following Docker-Compose to setup a cluster

  docker-compose.yml

    
  version: '3'
  services:
    opensearch-node1:
      image: opensearchstaging/opensearch:2.11.0.8641
      container_name: opensearch-node1
      environment:
        - cluster.name=opensearch-cluster
        - node.name=opensearch-node1
        - discovery.seed_hosts=opensearch-node1,opensearch-node2
        - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
        - bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
        - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
      ulimits:
        memlock:
          soft: -1
          hard: -1
        nofile:
          soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
          hard: 65536
      volumes:
        - opensearch-data1:/usr/share/opensearch/data
      ports:
        - 9200:9200
        - 9600:9600 # required for Performance Analyzer
      networks:
        - opensearch-net
    opensearch-node2:
      image: opensearchstaging/opensearch:2.11.0.8641
      container_name: opensearch-node2
      environment:
        - cluster.name=opensearch-cluster
        - node.name=opensearch-node2
        - discovery.seed_hosts=opensearch-node1,opensearch-node2
        - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
        - bootstrap.memory_lock=true
        - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
      ulimits:
        memlock:
          soft: -1
          hard: -1
        nofile:
          soft: 65536
          hard: 65536
      volumes:
        - opensearch-data2:/usr/share/opensearch/data
      networks:
        - opensearch-net
    opensearch-dashboards:
      image: opensearchstaging/opensearch-dashboards:2.11.0.6660
      container_name: opensearch-dashboards
      ports:
        - 5601:5601
      expose:
        - "5601"
      environment:
        OPENSEARCH_HOSTS: '["https://opensearch-node1:9200","https://opensearch-node2:9200"]'
      networks:
        - opensearch-net
  volumes:
    opensearch-data1:
    opensearch-data2:
  networks:
  
  opensearch-net:
  
  
  
  

  • Download the above docker-compose.yml on your machine.
  • Get latest image versions docker-compose pull.
  • Start the cluster docker-compose up.

• OpenSearch docker 2.11.0.8641

  • Start without security
    • Docker command docker pull opensearchstaging/opensearch:2.11.0.8641 && docker run -it -p 9200:9200 -e "discovery.type=single-node" -e "DISABLE_SECURITY_PLUGIN=true" opensearchstaging/opensearch:2.11.0.8641
    • Connect command curl https://localhost:9200/
  • Start with security
    • Docker command docker pull opensearchstaging/opensearch:2.11.0.8641 && docker run -it -p 9200:9200 -e "discovery.type=single-node" opensearchstaging/opensearch:2.11.0.8641
    • Connect command curl --insecure https://admin:admin@localhost:9200/

• OpenSearch-Dashboards docker 2.11.0.6660

  • Start without security
    • Docker command docker pull opensearchstaging/opensearch-dashboards:2.11.0.6660 && docker run -it --network="host" -e "DISABLE_SECURITY_DASHBOARDS_PLUGIN=true" opensearchstaging/opensearch-dashboards:2.11.0.6660
    • URL https://localhost:5601/
  • Start with security
    • Docker command docker pull opensearchstaging/opensearch-dashboards:2.11.0.6660 && docker run -it --network="host" opensearchstaging/opensearch-dashboards:2.11.0.6660
    • URL https://localhost:5601/

• Use TARs to deploy OpenSearch Manually

  • OpenSearch - Build 8641 (Note: Windows version does not have performance analyzer plugin)

    • arm64 [manifest] [tar] [rpm][deb]
    • x64 [manifest] [tar] [rpm] [deb] [windows]

  • OpenSearch Dashboards - Build 6660

    • arm64 [manifest] [tar][rpm][deb]
    • x64 [manifest] [tar][rpm] [deb] [windows]

Check how to install opensearch and dashboards on different platforms

[rishabh6788]

Integration Test Update

All components integration tests are passing for OpenSearch. See https://build.ci.opensearch.org/blue/organizations/jenkins/integ-test/detail/integ-test/6345/pipeline

alerting failed due to jenkins error and opensearch-reports failed due to flaky test. Re-ran tests for them and it passed. See https://build.ci.opensearch.org/blue/organizations/jenkins/integ-test/detail/integ-test/6346/pipeline/67/.

For OpenSearch Dashboards below components are failing.

• observabilityDashboards (without-security and with-security)
• reportsDashboards (without-security)

Attached are the log files for teams to take a look and take corrective action.
observabilityDashboards-without-security.txt
observabilityDashboards-with-security.txt
reportsDashboards-without-security.txt

[rishabh6788]

Final RC OpenSearch 8649 / OpenSearch Dashboards 6665 is ready for your test.

OpenSearch - Build 8649
OpenSearch Dashboards - Build 6665

• Use the following Docker-Compose to setup a cluster

  docker-compose.yml

    
  version: '3'
  services:
    opensearch-node1:
      image: opensearchstaging/opensearch:2.11.0.8649
      container_name: opensearch-node1
      environment:
        - cluster.name=opensearch-cluster
        - node.name=opensearch-node1
        - discovery.seed_hosts=opensearch-node1,opensearch-node2
        - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
        - bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
        - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
      ulimits:
        memlock:
          soft: -1
          hard: -1
        nofile:
          soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
          hard: 65536
      volumes:
        - opensearch-data1:/usr/share/opensearch/data
      ports:
        - 9200:9200
        - 9600:9600 # required for Performance Analyzer
      networks:
        - opensearch-net
    opensearch-node2:
      image: opensearchstaging/opensearch:2.11.0.8649
      container_name: opensearch-node2
      environment:
        - cluster.name=opensearch-cluster
        - node.name=opensearch-node2
        - discovery.seed_hosts=opensearch-node1,opensearch-node2
        - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
        - bootstrap.memory_lock=true
        - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
      ulimits:
        memlock:
          soft: -1
          hard: -1
        nofile:
          soft: 65536
          hard: 65536
      volumes:
        - opensearch-data2:/usr/share/opensearch/data
      networks:
        - opensearch-net
    opensearch-dashboards:
      image: opensearchstaging/opensearch-dashboards:2.11.0.6665
      container_name: opensearch-dashboards
      ports:
        - 5601:5601
      expose:
        - "5601"
      environment:
        OPENSEARCH_HOSTS: '["https://opensearch-node1:9200","https://opensearch-node2:9200"]'
      networks:
        - opensearch-net
  volumes:
    opensearch-data1:
    opensearch-data2:
  networks:
  
  opensearch-net:
  
  
  
  

  • Download the above docker-compose.yml on your machine.
  • Get latest image versions docker-compose pull.
  • Start the cluster docker-compose up.

• OpenSearch docker 2.11.0.8649

  • Start without security
    • Docker command docker pull opensearchstaging/opensearch:2.11.0.8649 && docker run -it -p 9200:9200 -e "discovery.type=single-node" -e "DISABLE_SECURITY_PLUGIN=true" opensearchstaging/opensearch:2.11.0.8649
    • Connect command curl https://localhost:9200/
  • Start with security
    • Docker command docker pull opensearchstaging/opensearch:2.11.0.8649 && docker run -it -p 9200:9200 -e "discovery.type=single-node" opensearchstaging/opensearch:2.11.0.8649
    • Connect command curl --insecure https://admin:admin@localhost:9200/

• OpenSearch-Dashboards docker 2.11.0.6665

  • Start without security
    • Docker command docker pull opensearchstaging/opensearch-dashboards:2.11.0.6665 && docker run -it --network="host" -e "DISABLE_SECURITY_DASHBOARDS_PLUGIN=true" opensearchstaging/opensearch-dashboards:2.11.0.6665
    • URL https://localhost:5601/
  • Start with security
    • Docker command docker pull opensearchstaging/opensearch-dashboards:2.11.0.6665 && docker run -it --network="host" opensearchstaging/opensearch-dashboards:2.11.0.6665
    • URL https://localhost:5601/

• Use TARs to deploy OpenSearch Manually

  • OpenSearch - Build 8649 (Note: Windows version does not have performance analyzer plugin)

    • arm64 [manifest] [tar] [rpm][deb]
    • x64 [manifest] [tar] [rpm] [deb] [windows]

  • OpenSearch Dashboards - Build 6665

    • arm64 [manifest] [tar][rpm][deb]
    • x64 [manifest] [tar][rpm] [deb] [windows]

Check how to install opensearch and dashboards on different platforms

[rishabh6788]

Integration Test Results

All components PASSING for OpenSearch. See https://build.ci.opensearch.org/blue/organizations/jenkins/integ-test/detail/integ-test/6364/pipeline/104

All components PASSING for OpenSearch Dashboards. See https://build.ci.opensearch.org/blue/organizations/jenkins/integ-test-opensearch-dashboards/detail/integ-test-opensearch-dashboards/4283/pipeline/111.

OpenSearch Dashboards had a flaky test, passed in second run for OpenSearch Dashboards. See https://build.ci.opensearch.org/blue/organizations/jenkins/integ-test-opensearch-dashboards/detail/integ-test-opensearch-dashboards/4284/pipeline/99/

[bbarani]

We are 🟢 for 2.11.0 release and it is scheduled to be released on Monday (Oct 16 2023).

You can participate in release meeting using this link - https://opensearch.org/events/2023-1018-2-11-release-meetings/

[rishabh6788]

Release checklist

Pre-Release activities

• Promote Repos

• OpenSearch

  • https://build.ci.opensearch.org/job/distribution-promote-repos/74/
  • https://build.ci.opensearch.org/job/distribution-promote-repos/75/

• OpenSearch Dashboards

  • https://build.ci.opensearch.org/job/distribution-promote-repos/76/
  • https://build.ci.opensearch.org/job/distribution-promote-repos/77/

• Promote artifacts

• Windows:

  • https://build.ci.opensearch.org/job/distribution-promote-artifacts/339/
  • https://build.ci.opensearch.org/job/distribution-promote-artifacts/337/

• Linux Debian:

  • https://build.ci.opensearch.org/job/distribution-promote-artifacts/340/
  • https://build.ci.opensearch.org/job/distribution-promote-artifacts/341/
  • https://build.ci.opensearch.org/job/distribution-promote-artifacts/342/
  • https://build.ci.opensearch.org/job/distribution-promote-artifacts/343/

• Linux RPM:

  • https://build.ci.opensearch.org/job/distribution-promote-artifacts/344/
  • https://build.ci.opensearch.org/job/distribution-promote-artifacts/345/
  • https://build.ci.opensearch.org/job/distribution-promote-artifacts/346/
  • https://build.ci.opensearch.org/job/distribution-promote-artifacts/347/

• Linux Tarball:

  • https://build.ci.opensearch.org/job/distribution-promote-artifacts/349/
  • https://build.ci.opensearch.org/job/distribution-promote-artifacts/351/
  • https://build.ci.opensearch.org/job/distribution-promote-artifacts/352/
  • https://build.ci.opensearch.org/job/distribution-promote-artifacts/353/

[rishabh6788]

Created tags for OS and OSD.

Updated 2.11.0 OS and OSD manifests #4133

[rishabh6788]

Maven Release done https://build.ci.opensearch.org/blue/organizations/jenkins/publish-to-maven/detail/publish-to-maven/25/pipeline.

[rishabh6788]

We have successfully released 2.11.0.
Please see #4061 retrospective issue to add your thoughts and feedback/improvements on this release.