[RELEASE] Release version 2.17.0

[github-actions]

Release OpenSearch and OpenSearch Dashboards 2.17.0

I noticed that a manifest was automatically created in manifests/2.17.0. Please follow the following checklist to make a release.

How to use this issue

This Release Issue

This issue captures the state of the OpenSearch release, its assignee (Release Manager) is responsible for driving the release. Please contact them or @mention them on this issue for help. There are linked issues on components of the release where individual components can be tracked. For more information check the the Release Process OpenSearch Guide.

Please refer to the following link for the release version dates: Release Schedule and Maintenance Policy.

Entrance Criteria

Criteria	Status	Description	Comments
Each component release issue has an assigned owner	🟢
Documentation draft PRs are up and in tech review for all component changes	🟢
Sanity testing is done for all components	🟢
Code coverage has not decreased (all new code has tests)
Release notes are ready and available for all components	🟢
Roadmap is up-to-date (information is available to create release highlights)	🟢
Release ticket is cut, and there's a forum post announcing the start of the window	🟢
Any necessary security reviews are complete	🟢

OpenSearch 2.17.0 exit criteria status:

Criteria	Status	Description	Comments
Performance tests are run, results are posted to the release ticket and there no unexpected regressions	🟢
No unpatched vulnerabilities of medium or higher severity that have been publicly known for more than 60 days	🟢
Documentation has been fully reviewed and signed off by the documentation community.	🔴
All integration tests are passing	🟢
Release blog is ready	🔴

OpenSearch-Dashboards 2.17.0 exit criteria status:

Criteria	Status	Description	Comments
Documentation has been fully reviewed and signed off by the documentation community	🔴
No unpatched vulnerabilities of medium or higher severity that have been publicly known for more than 60 days	🟢
All integration tests are passing	🔴
Release blog is ready	🔴

Preparation

• Release manager assigned.
• Update release page on the website with release manager and release issue details. Sample PR
• Existence of label in each component repo. For more information check the release-label section.
• Increase the build frequency.
• Release Issue.

Campaigns

• Component Release Issue.
• Release Campaigns.

Release Branch and Version Increment - Ends Sept 03, 2024

• Core Release Branch.
• Core Version Increment.
• Components Release Branch.
• Components Version Increment.

Feature Freeze - Ends Sept 03, 2024

• OpenSearch / OpenSearch-Dashboards core and components teams finalize their features.

Code Complete - Ends Sept 04, 2024 Sept 05, 2024

• Mark this as done once the Code Complete is reviewed.
• Create/Verify pull requests to add each component to relase input manifests/2.17.0/opensearch-2.17.0.yml and manifests/2.17.0/opensearch-dashboards-2.17.0.yml.

Release Candidate Creation and Testing - Starts Sept 04, 2024 at 4PM PDT

• Generate Release Candidate.
• Integ Test TAR.
• Integ Test RPM.
• Docker Build and Scan.
• Backwards Compatibility Tests.
• Windows Integration Test.
• Broadcast and Communication.
• Release Candidate Lock.

Performance testing validation - Ends Sept 11, 2024

• Post the benchmark-tests
• Longevity tests do not show any issues.

Pre Release - _Ends _Sept 16, 2024

• Release Labeled Issues.
• Go or No-Go.
• Promote Repos.
• Promote artifacts.
• Release Notes.

Release - Ends Sept 17, 2024

• Maven Promotion.
• Docker Promotion.
• Release Validation.
• Collaboration with the Project Management Team.

Release Checklist.

Release Checklist

Pre-Release activities

• Promote Repos.
  • • OS
  • • OSD
• Promote Artifacts.
  • • Windows
  • • Linux Debian
  • • Linux RPM
  • • Linux TAR
• Consolidated Release Notes.

Release activities

• Docker Promotion.
• Release Validation part 1.
  • • OpenSearch and OpenSearch Dashboard Validation.
  • • Validate the native plugin installation.
• Merge consolidated release notes PR.
• Website and Documentation Changes.
  • • Merge staging website PR.
  • • Promote the website changes to prod.
  • • Add website alert.
• Release Validation part 2.
  • • Validate the artifact download URL's and signatures.
• Release Validation part 3.
  • • Trigger the validation build (Search for Completed validation for <> in the logs).
• Maven Promotion.
• Publish blog posts.
• Advertise on Social Media.
• Post on public slack and Github Release issue.

Post-Release activities

• Release Tags.
• Input Manifest Update.
• Decrease the Build Frequency.
• OpenSearch Build Release notes.
• Retrospective Issue.
• Helm and Ansible Playbook release.
• Upcoming Release Preparation.

Post Release

• Release Tags.
• Input Manifest Update.
• OpenSearch Build Release notes.
• Decrease the Build Frequency.
• Retrospective Issue.
• Helm and Ansible Playbook release.
• Upcoming Release Preparation.

Components

Replace with links to all component tracking issues.

Component	Issues tagged with 2.17.0
opensearch-project/OpenSearch#15095
opensearch-project/common-utils#707
opensearch-project/job-scheduler#662
opensearch-project/security#4619
opensearch-project/k-NN#1928
opensearch-project/geospatial#672
opensearch-project/cross-cluster-replication#1426
opensearch-project/ml-commons#2802
opensearch-project/neural-search#858
opensearch-project/notifications#940
opensearch-project/observability#1857
opensearch-project/reporting#1026
opensearch-project/sql#2895
opensearch-project/asynchronous-search#603
opensearch-project/anomaly-detection#1273
opensearch-project/alerting#1636
opensearch-project/security-analytics#1216
opensearch-project/index-management#1224
opensearch-project/performance-analyzer#705
opensearch-project/performance-analyzer-rca#573
opensearch-project/custom-codecs#181
opensearch-project/flow-framework#820
opensearch-project/skills#382

OpenSearch Dashboards

Component	Issues tagged with 2.17.0
opensearch-project/OpenSearch-Dashboards#7617
opensearch-project/opensearch-dashboards-functional-test#1499
opensearch-project/dashboards-observability#2042
opensearch-project/dashboards-reporting#399
opensearch-project/dashboards-visualizations#384
opensearch-project/dashboards-query-workbench#372
opensearch-project/dashboards-maps#648
opensearch-project/anomaly-detection-dashboards-plugin#827
opensearch-project/ml-commons-dashboards#350
opensearch-project/index-management-dashboards-plugin#1104
opensearch-project/dashboards-notifications#232
opensearch-project/alerting-dashboards-plugin#1042
opensearch-project/security-analytics-dashboards-plugin#1106
opensearch-project/security-dashboards-plugin#2077
opensearch-project/dashboards-search-relevance#422
opensearch-project/dashboards-assistant#251

Legend

Symbol	Meaning
🟢	On track with overall release
🟡	Missed last milestone
🔴	Missed multiple milestones

[shiv0408]

PRs approved for v2.17 but pending merge in main due to build failure since yesterday.

• [Remote State] Upload incremental cluster state on master re-election OpenSearch#15145
• [Backport 2.17] Add dedicated string prefix for remote index metadata and remote routing table OpenSearch#15740
• Make Remote Publication a static setting OpenSearch#15478
• [Backport 2.17] Make Remote Publication a static setting (#15478) OpenSearch#15784
• [Backport 2.17] [Remote State] Upload incremental cluster state on master re-election (#15145) OpenSearch#15792

[gargharsh3134]

PR merged in main, but pending backport to 2.17:
opensearch-project/OpenSearch#15292

[getsaurabh02]

Maybe core should freeze a couple of days before plugins?

This is a valid point @msfroh and something we have been considering to have in place while announcing the schedule for next year releases.

[SwethaGuptha]

PR approved but pending merge:
opensearch-project/OpenSearch#15492

[sachinpkale]

PRs merged in main but pending backport to 2.17

• [Backport 2.17] Restore snapshot changes for shallow snapshot V2 (#15462) OpenSearch#15738
• [Backport 2.17] Integrate translog cleanup with snapshot deletion and fix primary term deletion logic OpenSearch#15750

PRs approved but pending merge to main (and pending backport to 2.17)

• [Snapshot V2] Support pinned timestamp in clone flow OpenSearch#15524
• Add path prefix support to hashed prefix snapshots OpenSearch#15664

[gaiksaya]

Updating the release candidate status:

We did try to generate the RC and got a partial one. See
OS: https://build.ci.opensearch.org/blue/organizations/jenkins/distribution-build-opensearch/detail/distribution-build-opensearch/10269/pipeline
OSD: https://build.ci.opensearch.org/blue/organizations/jenkins/distribution-build-opensearch[…]il/distribution-build-opensearch-dashboards/7921/pipeline

Below plugins failed:

• Performance Analyzer
• reportsDashboards

Release candidate creation was blocked until this was resolved. Since the fixes are now in, we have triggered RC again:
OS: https://build.ci.opensearch.org/blue/organizations/jenkins/distribution-build-opensearch/detail/distribution-build-opensearch/10271/pipeline
OSD: https://build.ci.opensearch.org/blue/organizations/jenkins/distribution-build-opensearch[…]il/distribution-build-opensearch-dashboards/7924/pipeline
Thanks!

[gaiksaya]

RC1 was not a valid candidate. Please see recent comments for RC2

Details

RC 1 OpenSearch 10271 / OpenSearch Dashboards 7924 is ready for your test.

OpenSearch - Build 10271
OpenSearch Dashboards - Build 7924

• Use the following Docker-Compose to setup a cluster

  docker-compose.yml

    
  version: '3'
  services:
    opensearch-node1:
      image: opensearchstaging/opensearch:2.17.0.10271
      container_name: opensearch-node1
      environment:
        - cluster.name=opensearch-cluster
        - node.name=opensearch-node1
        - discovery.seed_hosts=opensearch-node1,opensearch-node2
        - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
        - bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
        - OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123!
      ulimits:
        memlock:
          soft: -1
          hard: -1
        nofile:
          soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
          hard: 65536
      volumes:
        - opensearch-data1:/usr/share/opensearch/data
      ports:
        - 9200:9200
        - 9600:9600 # required for Performance Analyzer
      networks:
        - opensearch-net
    opensearch-node2:
      image: opensearchstaging/opensearch:2.17.0.10271
      container_name: opensearch-node2
      environment:
        - cluster.name=opensearch-cluster
        - node.name=opensearch-node2
        - discovery.seed_hosts=opensearch-node1,opensearch-node2
        - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
        - bootstrap.memory_lock=true
        - OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123!
      ulimits:
        memlock:
          soft: -1
          hard: -1
        nofile:
          soft: 65536
          hard: 65536
      volumes:
        - opensearch-data2:/usr/share/opensearch/data
      networks:
        - opensearch-net
    opensearch-dashboards:
      image: opensearchstaging/opensearch-dashboards:2.17.0.7924
      container_name: opensearch-dashboards
      ports:
        - 5601:5601
      expose:
        - "5601"
      environment:
        OPENSEARCH_HOSTS: '["https://opensearch-node1:9200","https://opensearch-node2:9200"]'
      networks:
        - opensearch-net
  volumes:
    opensearch-data1:
    opensearch-data2:
  networks:
  
  opensearch-net:
  
  
  
  

  • Download the above docker-compose.yml on your machine.
  • Get latest image versions docker-compose pull.
  • Start the cluster docker-compose up.

• OpenSearch docker 2.17.0.10271

  • Start without security
    • Docker command docker pull opensearchstaging/opensearch:2.17.0.10271 && docker run -it -p 9200:9200 -e "discovery.type=single-node" -e "DISABLE_SECURITY_PLUGIN=true" opensearchstaging/opensearch:2.17.0.10271
    • Connect command curl http://localhost:9200/
  • Start with security
    • Docker command

    docker pull opensearchstaging/opensearch:2.17.0.10271 && docker run -it -p 9200:9200 -e "discovery.type=single-node" -e "OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123!" opensearchstaging/opensearch:2.17.0.10271
    

    • Connect command curl --insecure 'https://admin:myStrongPassword123!@localhost:9200/'

• OpenSearch-Dashboards docker 2.17.0.7924

  • Start without security
    • Docker command docker pull opensearchstaging/opensearch-dashboards:2.17.0.7924 && docker run -it --network="host" -e "DISABLE_SECURITY_DASHBOARDS_PLUGIN=true" opensearchstaging/opensearch-dashboards:2.17.0.7924
    • URL http://localhost:5601/
  • Start with security
    • Docker command docker pull opensearchstaging/opensearch-dashboards:2.17.0.7924 && docker run -it --network="host" opensearchstaging/opensearch-dashboards:2.17.0.7924
    • URL http://localhost:5601/

• Use TARs to deploy OpenSearch Manually

  • OpenSearch - Build 10271 (Note: Windows version does not have performance analyzer plugin)

    • arm64 [manifest] [tar] [rpm][deb]
    • x64 [manifest] [tar] [rpm] [deb] [windows]

  • OpenSearch Dashboards - Build 7924

    • arm64 [manifest] [tar][rpm][deb]
    • x64 [manifest] [tar][rpm] [deb] [windows]

Check how to install opensearch and dashboards on different platforms

Integration Test Results

• Use the https://metrics.opensearch.org/_dashboards/goto/9ed74dd90eb31c7b83f3542e43328088?security_tenant=global.

• Filter by the distribution_build_number. Use 10271 for OpenSearch and 7924 for OpenSearch Dashboards.
  Example when filtered with 10271 we can see all the passed/failed OpenSearch components.

• Find the list of the created AUTOCUT issues here https://github.com/issues?page=1&q=is%3Aopen+is%3Aissue+user%3Aopensearch-project+label%3Av2.17.0+label%3Aautocut+%5BAUTOCUT%5D+in%3Atitle.

Thank you

[prudhvigodithi]

• Use the https://metrics.opensearch.org/_dashboards/goto/9ed74dd90eb31c7b83f3542e43328088?security_tenant=global.
• Filter by the distribution_build_number. Use 10271 for OpenSearch and 7924 for OpenSearch Dashboards.
  Example when filtered with 10271 we can see all the passed/failed OpenSearch components.

Here is the direct dashboard link filtered with 10271, 7924 to get the and integration test results.

Thank you

[gaiksaya]

RC2 was discarded as it did not pass the exit criteria. Please see RC3 details in the recent comments.

RC2 details

RC 2 OpenSearch 10274 / OpenSearch Dashboards 7926 is ready for your test.

OpenSearch - Build 10274
OpenSearch Dashboards - Build 7926

• Use the following Docker-Compose to setup a cluster

  docker-compose.yml

    
  version: '3'
  services:
    opensearch-node1:
      image: opensearchstaging/opensearch:2.17.0.10274
      container_name: opensearch-node1
      environment:
        - cluster.name=opensearch-cluster
        - node.name=opensearch-node1
        - discovery.seed_hosts=opensearch-node1,opensearch-node2
        - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
        - bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
        - OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123!
      ulimits:
        memlock:
          soft: -1
          hard: -1
        nofile:
          soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
          hard: 65536
      volumes:
        - opensearch-data1:/usr/share/opensearch/data
      ports:
        - 9200:9200
        - 9600:9600 # required for Performance Analyzer
      networks:
        - opensearch-net
    opensearch-node2:
      image: opensearchstaging/opensearch:2.17.0.10274
      container_name: opensearch-node2
      environment:
        - cluster.name=opensearch-cluster
        - node.name=opensearch-node2
        - discovery.seed_hosts=opensearch-node1,opensearch-node2
        - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
        - bootstrap.memory_lock=true
        - OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123!
      ulimits:
        memlock:
          soft: -1
          hard: -1
        nofile:
          soft: 65536
          hard: 65536
      volumes:
        - opensearch-data2:/usr/share/opensearch/data
      networks:
        - opensearch-net
    opensearch-dashboards:
      image: opensearchstaging/opensearch-dashboards:2.17.0.7926
      container_name: opensearch-dashboards
      ports:
        - 5601:5601
      expose:
        - "5601"
      environment:
        OPENSEARCH_HOSTS: '["https://opensearch-node1:9200","https://opensearch-node2:9200"]'
      networks:
        - opensearch-net
  volumes:
    opensearch-data1:
    opensearch-data2:
  networks:
  
  opensearch-net:
  
  
  
  

  • Download the above docker-compose.yml on your machine.
  • Get latest image versions docker-compose pull.
  • Start the cluster docker-compose up.

• OpenSearch docker 2.17.0.10274

  • Start without security
    • Docker command docker pull opensearchstaging/opensearch:2.17.0.10274 && docker run -it -p 9200:9200 -e "discovery.type=single-node" -e "DISABLE_SECURITY_PLUGIN=true" opensearchstaging/opensearch:2.17.0.10274
    • Connect command curl http://localhost:9200/
  • Start with security
    • Docker command

    docker pull opensearchstaging/opensearch:2.17.0.10274 && docker run -it -p 9200:9200 -e "discovery.type=single-node" -e "OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123!" opensearchstaging/opensearch:2.17.0.10274
    

    • Connect command curl --insecure 'https://admin:myStrongPassword123!@localhost:9200/'

• OpenSearch-Dashboards docker 2.17.0.7926

  • Start without security
    • Docker command docker pull opensearchstaging/opensearch-dashboards:2.17.0.7926 && docker run -it --network="host" -e "DISABLE_SECURITY_DASHBOARDS_PLUGIN=true" opensearchstaging/opensearch-dashboards:2.17.0.7926
    • URL http://localhost:5601/
  • Start with security
    • Docker command docker pull opensearchstaging/opensearch-dashboards:2.17.0.7926 && docker run -it --network="host" opensearchstaging/opensearch-dashboards:2.17.0.7926
    • URL http://localhost:5601/

• Use TARs to deploy OpenSearch Manually

  • OpenSearch - Build 10274 (Note: Windows version does not have performance analyzer plugin)

    • arm64 [manifest] [tar] [rpm][deb]
    • x64 [manifest] [tar] [rpm] [deb] [windows]

  • OpenSearch Dashboards - Build 7926

    • arm64 [manifest] [tar][rpm][deb]
    • x64 [manifest] [tar][rpm] [deb] [windows]

Check how to install opensearch and dashboards on different platforms

Integration Test Results

• Use the https://metrics.opensearch.org/_dashboards/goto/9ed74dd90eb31c7b83f3542e43328088?security_tenant=global.

• Filter by the distribution_build_number. Use 10274 for OpenSearch and 7926 for OpenSearch Dashboards.
  Example when filtered with 10274 we can see all the passed/failed OpenSearch components.

• Find the list of the created AUTOCUT issues here https://github.com/issues?page=1&q=is%3Aopen+is%3Aissue+user%3Aopensearch-project+label%3Av2.17.0+label%3Aautocut+%5BAUTOCUT%5D+in%3Atitle.

Thank you

[gaiksaya]

Looking at this workflow run (tar x64) we have below plugins failing the integration tests for

OpenSearch:

• Asynchronous search
• CCR
• Index management
• k-NN
• ml-commons
• Query-insights
• skills

OpenSearch Dashboard components:

1. Dashboards core
2. Index management
3. Observability
4. SecurityAnalytics

[gaiksaya]

RC 3 still had failing tests and did not meet exit criteria.

RC3 details

RC 3 OpenSearch 10284 / OpenSearch Dashboards 7931 is ready for your test.

OpenSearch - Build 10284
OpenSearch Dashboards - Build 7931

• Use the following Docker-Compose to setup a cluster

  docker-compose.yml

    
  version: '3'
  services:
    opensearch-node1:
      image: opensearchstaging/opensearch:2.17.0.10284
      container_name: opensearch-node1
      environment:
        - cluster.name=opensearch-cluster
        - node.name=opensearch-node1
        - discovery.seed_hosts=opensearch-node1,opensearch-node2
        - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
        - bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
        - OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123!
      ulimits:
        memlock:
          soft: -1
          hard: -1
        nofile:
          soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
          hard: 65536
      volumes:
        - opensearch-data1:/usr/share/opensearch/data
      ports:
        - 9200:9200
        - 9600:9600 # required for Performance Analyzer
      networks:
        - opensearch-net
    opensearch-node2:
      image: opensearchstaging/opensearch:2.17.0.10284
      container_name: opensearch-node2
      environment:
        - cluster.name=opensearch-cluster
        - node.name=opensearch-node2
        - discovery.seed_hosts=opensearch-node1,opensearch-node2
        - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
        - bootstrap.memory_lock=true
        - OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123!
      ulimits:
        memlock:
          soft: -1
          hard: -1
        nofile:
          soft: 65536
          hard: 65536
      volumes:
        - opensearch-data2:/usr/share/opensearch/data
      networks:
        - opensearch-net
    opensearch-dashboards:
      image: opensearchstaging/opensearch-dashboards:2.17.0.7931
      container_name: opensearch-dashboards
      ports:
        - 5601:5601
      expose:
        - "5601"
      environment:
        OPENSEARCH_HOSTS: '["https://opensearch-node1:9200","https://opensearch-node2:9200"]'
      networks:
        - opensearch-net
  volumes:
    opensearch-data1:
    opensearch-data2:
  networks:
  
  opensearch-net:
  
  
  
  

  • Download the above docker-compose.yml on your machine.
  • Get latest image versions docker-compose pull.
  • Start the cluster docker-compose up.

• OpenSearch docker 2.17.0.10284

  • Start without security
    • Docker command docker pull opensearchstaging/opensearch:2.17.0.10284 && docker run -it -p 9200:9200 -e "discovery.type=single-node" -e "DISABLE_SECURITY_PLUGIN=true" opensearchstaging/opensearch:2.17.0.10284
    • Connect command curl http://localhost:9200/
  • Start with security
    • Docker command

    docker pull opensearchstaging/opensearch:2.17.0.10284 && docker run -it -p 9200:9200 -e "discovery.type=single-node" -e "OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123!" opensearchstaging/opensearch:2.17.0.10284
    

    • Connect command curl --insecure 'https://admin:myStrongPassword123!@localhost:9200/'

• OpenSearch-Dashboards docker 2.17.0.7931

  • Start without security
    • Docker command docker pull opensearchstaging/opensearch-dashboards:2.17.0.7931 && docker run -it --network="host" -e "DISABLE_SECURITY_DASHBOARDS_PLUGIN=true" opensearchstaging/opensearch-dashboards:2.17.0.7931
    • URL http://localhost:5601/
  • Start with security
    • Docker command docker pull opensearchstaging/opensearch-dashboards:2.17.0.7931 && docker run -it --network="host" opensearchstaging/opensearch-dashboards:2.17.0.7931
    • URL http://localhost:5601/

• Use TARs to deploy OpenSearch Manually

  • OpenSearch - Build 10284 (Note: Windows version does not have performance analyzer plugin)

    • arm64 [manifest] [tar] [rpm][deb]
    • x64 [manifest] [tar] [rpm] [deb] [windows]

  • OpenSearch Dashboards - Build 7931

    • arm64 [manifest] [tar][rpm][deb]
    • x64 [manifest] [tar][rpm] [deb] [windows]

Check how to install opensearch and dashboards on different platforms

Integration Test Results

• Use the https://metrics.opensearch.org/_dashboards/goto/9ed74dd90eb31c7b83f3542e43328088?security_tenant=global.

• Filter by the distribution_build_number. Use 10284 for OpenSearch and 7931 for OpenSearch Dashboards.
  Example when filtered with 10284 we can see all the passed/failed OpenSearch components. Check the metrics here.

• Find the list of the created AUTOCUT issues here https://github.com/issues?page=1&q=is%3Aopen+is%3Aissue+user%3Aopensearch-project+label%3Av2.17.0+label%3Aautocut+%5BAUTOCUT%5D+in%3Atitle.

Thank you

[gaiksaya]

RC 4 did not meet exit criteria. See RC 5 details

RC4 details

## RC 4 OpenSearch 10288 / OpenSearch Dashboards 7935 is ready for your test.

OpenSearch - Build 10288
OpenSearch Dashboards - Build 7935

• Use the following Docker-Compose to setup a cluster

  docker-compose.yml

    
  version: '3'
  services:
    opensearch-node1:
      image: opensearchstaging/opensearch:2.17.0.10288
      container_name: opensearch-node1
      environment:
        - cluster.name=opensearch-cluster
        - node.name=opensearch-node1
        - discovery.seed_hosts=opensearch-node1,opensearch-node2
        - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
        - bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
        - OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123!
      ulimits:
        memlock:
          soft: -1
          hard: -1
        nofile:
          soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
          hard: 65536
      volumes:
        - opensearch-data1:/usr/share/opensearch/data
      ports:
        - 9200:9200
        - 9600:9600 # required for Performance Analyzer
      networks:
        - opensearch-net
    opensearch-node2:
      image: opensearchstaging/opensearch:2.17.0.10288
      container_name: opensearch-node2
      environment:
        - cluster.name=opensearch-cluster
        - node.name=opensearch-node2
        - discovery.seed_hosts=opensearch-node1,opensearch-node2
        - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
        - bootstrap.memory_lock=true
        - OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123!
      ulimits:
        memlock:
          soft: -1
          hard: -1
        nofile:
          soft: 65536
          hard: 65536
      volumes:
        - opensearch-data2:/usr/share/opensearch/data
      networks:
        - opensearch-net
    opensearch-dashboards:
      image: opensearchstaging/opensearch-dashboards:2.17.0.7935
      container_name: opensearch-dashboards
      ports:
        - 5601:5601
      expose:
        - "5601"
      environment:
        OPENSEARCH_HOSTS: '["https://opensearch-node1:9200","https://opensearch-node2:9200"]'
      networks:
        - opensearch-net
  volumes:
    opensearch-data1:
    opensearch-data2:
  networks:
  
  opensearch-net:
  
  
  
  

  • Download the above docker-compose.yml on your machine.
  • Get latest image versions docker-compose pull.
  • Start the cluster docker-compose up.

• OpenSearch docker 2.17.0.10288

  • Start without security
    • Docker command docker pull opensearchstaging/opensearch:2.17.0.10288 && docker run -it -p 9200:9200 -e "discovery.type=single-node" -e "DISABLE_SECURITY_PLUGIN=true" opensearchstaging/opensearch:2.17.0.10288
    • Connect command curl http://localhost:9200/
  • Start with security
    • Docker command

    docker pull opensearchstaging/opensearch:2.17.0.10288 && docker run -it -p 9200:9200 -e "discovery.type=single-node" -e "OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123!" opensearchstaging/opensearch:2.17.0.10288
    

    • Connect command curl --insecure 'https://admin:myStrongPassword123!@localhost:9200/'

• OpenSearch-Dashboards docker 2.17.0.7935

  • Start without security
    • Docker command docker pull opensearchstaging/opensearch-dashboards:2.17.0.7935 && docker run -it --network="host" -e "DISABLE_SECURITY_DASHBOARDS_PLUGIN=true" opensearchstaging/opensearch-dashboards:2.17.0.7935
    • URL http://localhost:5601/
  • Start with security
    • Docker command docker pull opensearchstaging/opensearch-dashboards:2.17.0.7935 && docker run -it --network="host" opensearchstaging/opensearch-dashboards:2.17.0.7935
    • URL http://localhost:5601/

• Use TARs to deploy OpenSearch Manually

  • OpenSearch - Build 10288 (Note: Windows version does not have performance analyzer plugin)

    • arm64 [manifest] [tar] [rpm][deb]
    • x64 [manifest] [tar] [rpm] [deb] [windows]

  • OpenSearch Dashboards - Build 7935

    • arm64 [manifest] [tar][rpm][deb]
    • x64 [manifest] [tar][rpm] [deb] [windows]

Check how to install opensearch and dashboards on different platforms

Integration Test Results

• Use the https://metrics.opensearch.org/_dashboards/goto/9ed74dd90eb31c7b83f3542e43328088?security_tenant=global.

• Filter by the distribution_build_number. Use 10288 for OpenSearch and 7935 for OpenSearch Dashboards.
  Example when filtered with 10288 we can see all the passed/failed OpenSearch components. Check the metrics here.

• Find the list of the created AUTOCUT issues here https://github.com/issues?page=1&q=is%3Aopen+is%3Aissue+user%3Aopensearch-project+label%3Av2.17.0+label%3Aautocut+%5BAUTOCUT%5D+in%3Atitle.

Thank you

[jainankitk]

I think our best bet for the future is to cut the release branch on core as the "freeze" trigger. Maybe core should freeze a couple of days before plugins?

@msfroh - I was discussing something very similar with @andrross offline. IMO, RC1 generation flow should cut the release branch for core and plugins, automatically serving as the "freeze" trigger. After that we can bump the versions for 2.x and main branches to avoid breaking build before the code freeze. But yeah, let us discuss further during the retrospective!

[rishabh6788]

Performance Results for 2.17

No significant performance callouts for 2.17 release compared to 2.16.
No notable regression or improvement detected in 2.17 compared to 2.16.

HTTP_LOGS

NYC_TAXIS

[gaiksaya]

RC5 did not meet the bar for release candidate. See RC6 details below

RC 5 details

RC 5 OpenSearch 10289 / OpenSearch Dashboards 7936 is ready for your test.

OpenSearch - Build 10289
OpenSearch Dashboards - Build 7936

• Use the following Docker-Compose to setup a cluster

  docker-compose.yml

    
  version: '3'
  services:
    opensearch-node1:
      image: opensearchstaging/opensearch:2.17.0.10289
      container_name: opensearch-node1
      environment:
        - cluster.name=opensearch-cluster
        - node.name=opensearch-node1
        - discovery.seed_hosts=opensearch-node1,opensearch-node2
        - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
        - bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
        - OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123!
      ulimits:
        memlock:
          soft: -1
          hard: -1
        nofile:
          soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
          hard: 65536
      volumes:
        - opensearch-data1:/usr/share/opensearch/data
      ports:
        - 9200:9200
        - 9600:9600 # required for Performance Analyzer
      networks:
        - opensearch-net
    opensearch-node2:
      image: opensearchstaging/opensearch:2.17.0.10289
      container_name: opensearch-node2
      environment:
        - cluster.name=opensearch-cluster
        - node.name=opensearch-node2
        - discovery.seed_hosts=opensearch-node1,opensearch-node2
        - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
        - bootstrap.memory_lock=true
        - OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123!
      ulimits:
        memlock:
          soft: -1
          hard: -1
        nofile:
          soft: 65536
          hard: 65536
      volumes:
        - opensearch-data2:/usr/share/opensearch/data
      networks:
        - opensearch-net
    opensearch-dashboards:
      image: opensearchstaging/opensearch-dashboards:2.17.0.7936
      container_name: opensearch-dashboards
      ports:
        - 5601:5601
      expose:
        - "5601"
      environment:
        OPENSEARCH_HOSTS: '["https://opensearch-node1:9200","https://opensearch-node2:9200"]'
      networks:
        - opensearch-net
  volumes:
    opensearch-data1:
    opensearch-data2:
  networks:
  
  opensearch-net:
  
  
  
  

  • Download the above docker-compose.yml on your machine.
  • Get latest image versions docker-compose pull.
  • Start the cluster docker-compose up.

• OpenSearch docker 2.17.0.10289

  • Start without security
    • Docker command docker pull opensearchstaging/opensearch:2.17.0.10289 && docker run -it -p 9200:9200 -e "discovery.type=single-node" -e "DISABLE_SECURITY_PLUGIN=true" opensearchstaging/opensearch:2.17.0.10289
    • Connect command curl http://localhost:9200/
  • Start with security
    • Docker command

    docker pull opensearchstaging/opensearch:2.17.0.10289 && docker run -it -p 9200:9200 -e "discovery.type=single-node" -e "OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123!" opensearchstaging/opensearch:2.17.0.10289
    

    • Connect command curl --insecure 'https://admin:myStrongPassword123!@localhost:9200/'

• OpenSearch-Dashboards docker 2.17.0.7936

  • Start without security
    • Docker command docker pull opensearchstaging/opensearch-dashboards:2.17.0.7936 && docker run -it --network="host" -e "DISABLE_SECURITY_DASHBOARDS_PLUGIN=true" opensearchstaging/opensearch-dashboards:2.17.0.7936
    • URL http://localhost:5601/
  • Start with security
    • Docker command docker pull opensearchstaging/opensearch-dashboards:2.17.0.7936 && docker run -it --network="host" opensearchstaging/opensearch-dashboards:2.17.0.7936
    • URL http://localhost:5601/

• Use TARs to deploy OpenSearch Manually

  • OpenSearch - Build 10289 (Note: Windows version does not have performance analyzer plugin)

    • arm64 [manifest] [tar] [rpm][deb]
    • x64 [manifest] [tar] [rpm] [deb] [windows]

  • OpenSearch Dashboards - Build 7936

    • arm64 [manifest] [tar][rpm][deb]
    • x64 [manifest] [tar][rpm] [deb] [windows]

Check how to install opensearch and dashboards on different platforms

Integration Test Results

• Use the https://metrics.opensearch.org/_dashboards/goto/9ed74dd90eb31c7b83f3542e43328088?security_tenant=global.

• Filter by the distribution_build_number. Use 10289 for OpenSearch and 7936 for OpenSearch Dashboards.
  Example when filtered with 10289 we can see all the passed/failed OpenSearch components.
  See results here

• Find the list of the created AUTOCUT issues here https://github.com/issues?page=1&q=is%3Aopen+is%3Aissue+user%3Aopensearch-project+label%3Av2.17.0+label%3Aautocut+%5BAUTOCUT%5D+in%3Atitle.

Thank you

[gaiksaya]

RC 6 OpenSearch 10292 / OpenSearch Dashboards 7942 is ready for your test.

OpenSearch - Build 10292
OpenSearch Dashboards - Build 7942

• Use the following Docker-Compose to setup a cluster

  docker-compose.yml

    
  version: '3'
  services:
    opensearch-node1:
      image: opensearchstaging/opensearch:2.17.0.10292
      container_name: opensearch-node1
      environment:
        - cluster.name=opensearch-cluster
        - node.name=opensearch-node1
        - discovery.seed_hosts=opensearch-node1,opensearch-node2
        - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
        - bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
        - OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123!
      ulimits:
        memlock:
          soft: -1
          hard: -1
        nofile:
          soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
          hard: 65536
      volumes:
        - opensearch-data1:/usr/share/opensearch/data
      ports:
        - 9200:9200
        - 9600:9600 # required for Performance Analyzer
      networks:
        - opensearch-net
    opensearch-node2:
      image: opensearchstaging/opensearch:2.17.0.10292
      container_name: opensearch-node2
      environment:
        - cluster.name=opensearch-cluster
        - node.name=opensearch-node2
        - discovery.seed_hosts=opensearch-node1,opensearch-node2
        - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
        - bootstrap.memory_lock=true
        - OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123!
      ulimits:
        memlock:
          soft: -1
          hard: -1
        nofile:
          soft: 65536
          hard: 65536
      volumes:
        - opensearch-data2:/usr/share/opensearch/data
      networks:
        - opensearch-net
    opensearch-dashboards:
      image: opensearchstaging/opensearch-dashboards:2.17.0.7942
      container_name: opensearch-dashboards
      ports:
        - 5601:5601
      expose:
        - "5601"
      environment:
        OPENSEARCH_HOSTS: '["https://opensearch-node1:9200","https://opensearch-node2:9200"]'
      networks:
        - opensearch-net
  volumes:
    opensearch-data1:
    opensearch-data2:
  networks:
  
  opensearch-net:
  
  
  
  

  • Download the above docker-compose.yml on your machine.
  • Get latest image versions docker-compose pull.
  • Start the cluster docker-compose up.

• OpenSearch docker 2.17.0.10292

  • Start without security
    • Docker command docker pull opensearchstaging/opensearch:2.17.0.10292 && docker run -it -p 9200:9200 -e "discovery.type=single-node" -e "DISABLE_SECURITY_PLUGIN=true" opensearchstaging/opensearch:2.17.0.10292
    • Connect command curl http://localhost:9200/
  • Start with security
    • Docker command

    docker pull opensearchstaging/opensearch:2.17.0.10292 && docker run -it -p 9200:9200 -e "discovery.type=single-node" -e "OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123!" opensearchstaging/opensearch:2.17.0.10292
    

    • Connect command curl --insecure 'https://admin:myStrongPassword123!@localhost:9200/'

• OpenSearch-Dashboards docker 2.17.0.7942

  • Start without security
    • Docker command docker pull opensearchstaging/opensearch-dashboards:2.17.0.7942 && docker run -it --network="host" -e "DISABLE_SECURITY_DASHBOARDS_PLUGIN=true" opensearchstaging/opensearch-dashboards:2.17.0.7942
    • URL http://localhost:5601/
  • Start with security
    • Docker command docker pull opensearchstaging/opensearch-dashboards:2.17.0.7942 && docker run -it --network="host" opensearchstaging/opensearch-dashboards:2.17.0.7942
    • URL http://localhost:5601/

• Use TARs to deploy OpenSearch Manually

  • OpenSearch - Build 10292 (Note: Windows version does not have performance analyzer plugin)

    • arm64 [manifest] [tar] [rpm][deb]
    • x64 [manifest] [tar] [rpm] [deb] [windows]

  • OpenSearch Dashboards - Build 7942

    • arm64 [manifest] [tar][rpm][deb]
    • x64 [manifest] [tar][rpm] [deb] [windows]

Check how to install opensearch and dashboards on different platforms

Integration Test Results

• Use the https://metrics.opensearch.org/_dashboards/goto/9ed74dd90eb31c7b83f3542e43328088?security_tenant=global.

• Filter by the distribution_build_number. Use 10292 for OpenSearch and 7942 for OpenSearch Dashboards.
  Example when filtered with 10292 we can see all the passed/failed OpenSearch components.
  See results here

• Find the list of the created AUTOCUT issues here https://github.com/issues?page=1&q=is%3Aopen+is%3Aissue+user%3Aopensearch-project+label%3Av2.17.0+label%3Aautocut+%5BAUTOCUT%5D+in%3Atitle.

Thank you

[gaiksaya]

Updating the status of 2.17.0 as per today Go/No-Go call:
The decision for this call was a NO-GO based on below status:

• Integration tests are failing for IndexManagementDashboards and OpenSearch-Dashboards core. The tests are flaky and need to be re-run. We did try running them twice yesterday however, they are still failing. We will continue to run them again and see if they go through. Flaky tests have been a hurdle for a long time and block the release process. Component owners please take action to fix in if possible, in this release or else next. We will be creating issues with respective component owners. You can check the metrics here.
• We have 10 documentation PRs up are different stages of review. However, the status is RED as they still need to be reviewed and not ready to merge yet.
• We do not have release blog ready yet. @getsaurabh02 will be coordinating to get that in.
• Regarding CVEs, we discovered a high CVE in docker scan which missed the recent release candidate for OpenSearch-Dashboards. We will generating a new RC today only for OpenSearch-Dashboards to get it in.
  We have our next Go/No-Go call scheduled for Monday at 9AM PST.

[gaiksaya]

RC 7

We have next RC which only contains OpenSearch Dashboards: https://build.ci.opensearch.org/blue/organizations/jenkins/distribution-build-opensearch[…]il/distribution-build-opensearch-dashboards/7944/pipeline

We are still seeing the CVE in the scan. Working with the core team to see if is actually an issue or false positive.

[gaiksaya]

RC 8

2 components had to patch the software to resolve high CVEs.
We have generated new RC with those changes.
https://build.ci.opensearch.org/blue/organizations/jenkins/distribution-build-opensearch-dashboards/detail/distribution-build-opensearch-dashboards/7946/pipeline/1392/

We will continue to run tests and try to meet the exit criteria.
Thanks!