[RELEASE] Release version 2.11.1 #4161

github-actions · 2023-10-20T00:17:08Z

Release OpenSearch and OpenSearch Dashboards 2.11.1

I noticed that a manifest was automatically created in manifests/2.11.1. Please follow the following checklist to make a release.

How to use this issue

This Release Issue

This issue captures the state of the OpenSearch release, its assignee (Release Manager) is responsible for driving the release. Please contact them or @mention them on this issue for help. There are linked issues on components of the release where individual components can be tracked. For more information check the the Release Process OpenSearch Guide.

Please refer to the following link for the release version dates: Release Schedule and Maintenance Policy.

Preparation

Release manager assigned.
Existence of label in each component repo. For more information check the release-label section.
Increase the build frequency.
Release Issue.

Campaigns

Release Branch and Version Increment - Ends Nov 14 2023

Feature Freeze

OpenSearch / OpenSearch-Dashboards core and components teams finalize their features.

Code Complete - Ends date

Mark this as done once the Code Complete is reviewed.
Create/Verify pull requests to add each component to relase input manifests/2.11.1/opensearch-2.11.1.yml and manifests/2.11.1/opensearch-dashboards-2.11.1.yml.

Release Candidate Creation and Testing - Ends Nov 21 2023

Pre Release - Ends Nov 24 2023

Release - Ends Nov 27 2023

Release Checklist.

Release Checklist

Pre-Release activities

Release activities

Post-Release activities

Post Release

Components

Replace with links to all component tracking issues.

Component	On track	Release Notes
{COMPONENT_ISSUE_LINK}	{INDICATOR}}	{STATUS}

Legend

Symbol	Meaning
🟢	On track with overall release
🟡	Missed last milestone
🔴	Missed multiple milestones

The text was updated successfully, but these errors were encountered:

bbarani · 2023-10-27T17:34:05Z

Bug fixes targeted for 2.11.1 release:

OpenSearch:

[Regression] Request decompression was broken in 2.11 OpenSearch#10802

peternied · 2023-10-27T17:50:59Z

For reference the fix for that issue was made inside the security plugin, it is in the 2.11 branch:

[Backport 2.11] Use new instance of Decompressor on channel initialization security#3599

Having the ref for the security plugin point to 27dee2 to be sure the fix is included in this patch release.

Thank you @Divyaasm

bbarani · 2023-10-31T21:53:44Z

List of repos that have changes merged in to 2.11 branch post 2.11 release.

Repo	Branch	CommitID	Commit Date	PRs that went in after 2.11.0
OpenSearch	[2.11]	ac4de44	2023-10-30	opensearch-project/OpenSearch#10677, opensearch-project/OpenSearch#10719, opensearch-project/OpenSearch#11005
common-utils	[2.11]	dcf288d	2023-10-24	opensearch-project/common-utils#551, opensearch-project/common-utils#555
job-scheduler	[2.11]
k-NN	[2.11]
geospatial	[2.11]	07fd490	2023-10-27	opensearch-project/geospatial#576
security	[2.11]	f27dee2	2023-10-25	opensearch-project/security#3599, opensearch-project/security#3486, opensearch-project/security#3480
cross-cluster-replication	[2.11]	cf633fa	2023-10-18	opensearch-project/cross-cluster-replication#1276
ml-commons	[2.11]	1a31c5f	2023-10-30	opensearch-project/ml-commons#1524
neural-search	[2.11]
notifications-core	[2.11]
notifications	[2.11]
opensearch-observability	[2.11]	3b6d40e	2023-10-16
opensearch-reports	[2.11]	ab8d86c	2023-10-16	opensearch-project/reporting#915
sql	[2.11]	6e17ae6	2023-10-30	22 PRs (including multiple features)
asynchronous-search	[2.11]
anomaly-detection	[2.11]
alerting	[2.11]	c00c86d	2023-10-24	opensearch-project/alerting#1286
security-analytics	[2.11]	5b4ab6c	2023-10-31	7 commits
index-management	[2.11]	d1f9bce	2023-10-13	opensearch-project/index-management#1013, opensearch-project/index-management#1012
performance-analyzer	[2.11]
custom-codecs	[2.11]

Repo	Branch	CommitID	Commit Date	PRs that went in after 2.11.0
OpenSearch-Dashboards	[2.11]	227a412	2023-10-27	opensearch-project/OpenSearch-Dashboards#5288, opensearch-project/OpenSearch-Dashboards#5301, opensearch-project/OpenSearch-Dashboards#5330, opensearch-project/OpenSearch-Dashboards#5390
functionalTestDashboards	[2.11]	668a8b5	2023-10-30	opensearch-project/opensearch-dashboards-functional-test#929, opensearch-project/opensearch-dashboards-functional-test#933, opensearch-project/opensearch-dashboards-functional-test#938, opensearch-project/opensearch-dashboards-functional-test#945
observabilityDashboards	[2.11]	e0a509c	2023-10-30	26 commits
reportsDashboards	[2.11]	82d154a	2023-10-19	opensearch-project/dashboards-reporting#229, opensearch-project/dashboards-reporting#223
ganttChartDashboards	[2.11]
queryWorkbenchDashboards	[2.11]	94805c5	2023-10-30	12 commits
customImportMapDashboards	[2.11]
anomalyDetectionDashboards	[2.11]
mlCommonsDashboards	[2.11]
indexManagementDashboards	[2.11]
notificationsDashboards	[2.11]
alertingDashboards	[2.11]	8a55e43	2023-10-25	opensearch-project/alerting-dashboards-plugin#767,
securityAnalyticsDashboards	[2.11]	de6001a	2023-10-25	opensearch-project/security-analytics-dashboards-plugin#770, opensearch-project/security-analytics-dashboards-plugin#762, opensearch-project/security-analytics-dashboards-plugin#766, opensearch-project/security-analytics-dashboards-plugin#752
securityDashboards	[2.11]	ffc793f	2023-10-23	opensearch-project/security-dashboards-plugin#1626
searchRelevanceDashboards	[2.11]

CEHENKLE · 2023-11-01T22:15:46Z

So we need to make a decision about 2.11.1.

A little background first:

1/ We need this release because of a critical breaking bug in the security plug-in. This bug impacts everyone using logstash and the only work around is to turn off compression (which is expensive)

2/ As a refresher, our normal process is to commit changes to main, then backport to 2.x.. Since the 1.0 release, we only backport bug fixes and security changes to the current 2.x branch (in this case 2.11) so that if we need to we can quickly do a patch release we can.

3/ There are two reason we don’t back port features to the current release branch (right now 2.11):
a) Because we want patch release to be as thin as possible, and the more changes you make, the more verification we need to do
b) Adding features in a patch breaks semver.

However, looking at the list of changes for 2.11, we currently have a mix of features that have been merged in.

Here are the ways forward that we’ve thought of (feel free to suggest another way):

Ship nothing, wait until Jan 2.12

Pros:
-- Least disruptive
-- Follows SemVer
Cons:
-- We are broken with logstash, and the only work around in place is not scalable (turn compression off) and costs more money
-- The high severity fixes will not be released quickly

Ship 2.11.1 as-is with features using 2.11 branch

Pros:
-- This is a quick fix with limited change to build infrastructure and workflows
-- We will be able to follow the normal patch release process
-- The high severity fixes will be released quickly
Cons:
-- This is against SemVar guidelines as we are introducing features in patch release
-- Features might get released without proper testing

Create 2.11.1 branch and use commits from 2.11 release ( and cherry pick only necessary bug fixes in to 2.11.1 branch)

Pros:
-- This is a quick fix with limited blast radius.
-- We will be able to adhere to SemVar guidelines
-- Patch release will be lean with minimal changes (as expected out of patch release)
-- Limited participation required from all the teams
-- The high severity fixes will be released quickly
Cons:
-- This requires manual creation of branches followed by cherry picking the code commits
-- This might require additional changes in build scripts to use 2.11.1 patch branch (3 digit branches were never used before)
-- We will deviate from our normal patch release process, which adds risks
-- We would need to tag 2.11.1 in 2.11.1 branch instead of 2.11 branch (and this might cause confusion)

Release 2.12 version instead of 2.11.1 version to accommodate new features

Pros:
-- We will be able to adhere to SemVar guidelines
-- This will be a full blown release with comprehensive testing so we will have high confidence in the features
-- We will be able to release additional features in this release
-- Community trust and transparency will not be impacted
Cons:
-- This requires a full blown release so would take more time compared to a patch release
-- The high severity fixes has to wait until we release minor version
-- This might delay the release planning to accommodate any feature that is not ready at this point in time

Revert all feature related changes from 2.11 branch

Pros:
-- We will be able to adhere to SemVar guidelines
-- We will not deviate from our normal patch release process
-- Patch release will be lean with minimal changes (as expected out of patch release)
-- Limited participation required from all plug-ins
-- The high severity fixes will be released quickly
Cons:
-- This requires manual revert of features from 2.11 release branch ( so there is a decent effort required from all plugin / core owners)
-- Downstream teams that picked up 2.11 and built from it will be left in a weird state
-- The reverts has be coordinated across multiple teams to avoid introducing additional issues.

As you can see, there’s no solution without cons, but given all the trade-offs, I’m leaning towards 1. WDYT?

Thanks,

PS, because this is impacting people today, I’d like to get this release out ASAP. So please put your comments up in the next 48 hours so we can prep a release for early next week (if that's what we're going to do). I also want to better understand why features were backported to the release branch, and what we should do about this in the future, but I’d like to tackle that after 2.11.1 is out.

reta · 2023-11-01T22:22:59Z

Thanks @CEHENKLE

Ship 2.11.1 as-is with features using 2.11 branch

+1 to this option, for OpenSearch core at least, out of 3 merged pull requests, 2 are non-functional changes and 1 is a bugfix (with accordance to semver and signed off by the team before the merge). For a plugins though (except security one) would be good to hear maintainers.

smortex · 2023-11-02T02:08:05Z

Regarding back-ported features in release branches

However, looking at the list of changes for 2.11, we currently have a mix of features that have been merged in.

😶 That's unexpected. It feels like a lot of these backports PR where opened with some automation due to improper labels being put on the initial PR (i.e. a feature should only have been labeled backport-2.x and a bugfix should have been labeled backport-2.x and backport-2.10). Maybe ensuring each PR is labelled as a backwards-incompatible, enhancement or bugfix can be used with this automation to detect inconsistencies and avoid this situation in the future? Is there a PR to discuss this?

Regarding what to do with the next release

#	Scenario	My feeling
0	Ship nothing, wait until Jan 2.12	LGTM but does not looks acceptable for some users 🙃
1	Ship 2.11.1 as-is with features using 2.11 branch	Strong no go: if we do this on purpose we should really stop pretending we are doing something that connects even marginally with SemVer. When this happen by accident, SemVer has a FAQ entry to describe how it should be handled (basically scenario 5)
2	Create 2.11.1 branch and use commits from 2.11 release ( and cherry pick only necessary bug fixes in to 2.11.1 branch)	That is basically having a `2.11.1` branch that contains what `2.11` was supposed to have. I wonder if keeping the `2.11` branch at it is makes sense in this case, because a `2.11.2` version would be branched out of the `2.11.1` branch is another patch release is to be issued
3	Release 2.12 version instead of 2.11.1 version to accommodate new features	LGTM
4	Revert all feature related changes from 2.11 branch	LGTM

Given the situation, 0 & 3 seems the most practical (bump minor version and release), 4 is probably the cleanest thing we can do but require more efforts (allows us to bump the patch version and release).

Branching 2.12 from 2.11 instead of 2.x and releasing the 2.12.0 version from there seems to require the minimum efforts and would have my preference because it looks to require reasonable effort (scenario 3).

If the cost regarding community trust of releasing 2.12 earlier than announced and without the announced features of 2.12 seems problematic, scenario 4 is probably what must be done IMHO.

If both are too much pain, as a last resort scenario 0 is acceptable.

peternied · 2023-11-02T14:31:28Z

Chiming in for Security, I would advocate for option 1, as its the smallest deviation from our existing process and I trust that maintainers are making the right choices.

For me the most important trait of following Semver is signaling breaking changes on major version changes, secondarily is signaling potential risk between minor/patch versions. The sematic argument of 'is it a feature or a bug' is hard to judge on a product of this scale.

Changing the process mid-flight is risky - I'd advocate for doing a retro afterwards to drill into misaligned expectations around branches / release.

macohen · 2023-11-02T16:43:27Z

I think that this should be the 2.12 release. We know there are features and bugs in this release. We need to be careful with testing and make sure we're confident in the release. It's a slippery slope to make exceptions on semver, but continue to say that we follow it. We should also have a retro meeting after the fact that involves representatives from interested parties who take these releases into their products/services.

joshuarrrr · 2023-11-02T16:59:53Z

In terms of level of effort, options 2 and 4 seem comparable/equivalent, but 4 gets us to a proper/desired state where we'd prefer to be, whereas option 2 deviates our process quite a bit.

Option 0 - Prolongs user pain unnecessarily
Option 1 - I think this is wrong and damages trust
Option 2 - I think this is fine, but worse than option 4, which is similar level of effort
Option 3 - I also think this is preferable to options 0 and 1, but will mean a slower fix for the critical bug
Option 4 - My preferred option to solve pain while restoring us to a correct state

I trust that maintainers are making the right choices.

But we have evidence that's not the case - maintainers are responsible for adding the labels which triggers the backport automation and have (intentionally or not) violated semver by improperly backporting features to the 2.11 branch.

dblock · 2023-11-02T17:03:57Z

I recommend rewinding all 2.11 to the 2.11 release, then applying only security fixes on top, aka reverting anything that's not a security fix from 2.11. If there were a critical security vulnerability right now we would have to do that. If we ship 2.11.1 with the additional commits we won't be able to do a 2.11.2 just with security fixes, forcing these features onto users who have standard practices around semver.

At the same time, we can ship 2.12 with these features as always.

reta · 2023-11-02T17:18:13Z

I recommend rewinding all 2.11 to the 2.11 release, then applying only security fixes on top, aka reverting anything that's not a security fix from 2.11.

.. and bugfixes (this fits semver), for OpenSearch core there was deliberate decision to bring the bugfix to 2.11.1 (see please Slack discussion in #release channel).

austintlee · 2023-11-03T00:21:44Z

@dblock you're endorsing option 4, correct?

That is the best option in my opinion. If we ever have to do a 2.11.2, you would want 2.11.1 to be in the state that db is describing.

krisfreedain · 2023-11-03T07:08:07Z

I would agree - if we do a 2.11.1, it should follow Semver and only be bug fixes. Vote for option 4 - then we can move any feature-related work to 2.12

stephen-crawford · 2023-11-03T17:31:45Z

I also vote for option #4. I don't think that tossing semver is a good call and some of the changes seem pretty substantial. For example:

opensearch-project/sql@6e17ae6

opensearch-project/security-analytics@658aa99

opensearch-project/security-analytics@559d97e

None of these seem like bug fixes to me and we should not say we are semver if we are not.

That leaves releasing 2.12 now or reverting changes.

I don't think we should release now, since I doubt that will be easier than reverting and don't love having such a thin/rushed version.

That leaves reverting as the best choice in my opinion.

CEHENKLE · 2023-11-04T00:10:01Z

Okay, seems like we're coalescing around number 4 (Revert all feature related changes from 2.11 branch, release 2.11.1 from there). We'll put a meta issue up (linked here) with sub issues for each feature added, and track the removal here. When they're all closed, we'll cut the release.

In the spirit of many eyes make shallow bugs, everyone feel free to cross check the issues ID'd in the meta issue once it's up to make sure we didn't miss anything.

Thanks,

Divyaasm · 2023-11-17T19:09:09Z

Release Candidate details for OpenSearch - 8873 and OpenSearch Dashboards - 6816

OpenSearch - Build 8873
OpenSearch Dashboards - Build 6816

Use the following Docker-Compose to setup a cluster

docker-compose.yml

version: '3' services: opensearch-node1: image: opensearchstaging/opensearch:2.11.1.8873 container_name: opensearch-node1 environment: - cluster.name=opensearch-cluster - node.name=opensearch-node1 - discovery.seed_hosts=opensearch-node1,opensearch-node2 - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2 - bootstrap.memory_lock=true # along with the memlock settings below, disables swapping - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM ulimits: memlock: soft: -1 hard: -1 nofile: soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems hard: 65536 volumes: - opensearch-data1:/usr/share/opensearch/data ports: - 9200:9200 - 9600:9600 # required for Performance Analyzer networks: - opensearch-net opensearch-node2: image: opensearchstaging/opensearch:2.11.1.8873 container_name: opensearch-node2 environment: - cluster.name=opensearch-cluster - node.name=opensearch-node2 - discovery.seed_hosts=opensearch-node1,opensearch-node2 - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2 - bootstrap.memory_lock=true - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" ulimits: memlock: soft: -1 hard: -1 nofile: soft: 65536 hard: 65536 volumes: - opensearch-data2:/usr/share/opensearch/data networks: - opensearch-net opensearch-dashboards: image: opensearchstaging/opensearch-dashboards:2.11.1.6816 container_name: opensearch-dashboards ports: - 5601:5601 expose: - "5601" environment: OPENSEARCH_HOSTS: '["https://opensearch-node1:9200","https://opensearch-node2:9200"]' networks: - opensearch-net volumes: opensearch-data1: opensearch-data2:

networks: opensearch-net:

Download the above docker-compose.yml on your machine.
Get latest image versions docker-compose pull.
Start the cluster docker-compose up.

OpenSearch docker 2.11.1.8873
- Start without security
  - Docker command docker pull opensearchstaging/opensearch:2.11.1.8873 && docker run -it -p 9200:9200 -e "discovery.type=single-node" -e "DISABLE_SECURITY_PLUGIN=true" opensearchstaging/opensearch:2.11.1.8873
  - Connect command curl https://localhost:9200/
- Start with security
  - Docker command docker pull opensearchstaging/opensearch:2.11.1.8873 && docker run -it -p 9200:9200 -e "discovery.type=single-node" opensearchstaging/opensearch:2.11.1.8873
  - Connect command curl --insecure https://admin:admin@localhost:9200/
OpenSearch-Dashboards docker 2.11.1.6816
- Start without security
  - Docker command docker pull opensearchstaging/opensearch-dashboards:2.11.1.6816 && docker run -it --network="host" -e "DISABLE_SECURITY_DASHBOARDS_PLUGIN=true" opensearchstaging/opensearch-dashboards:2.11.1.6816
  - URL https://localhost:5601/
- Start with security
  - Docker command docker pull opensearchstaging/opensearch-dashboards:2.11.1.6816 && docker run -it --network="host" opensearchstaging/opensearch-dashboards:2.11.1.6816
  - URL https://localhost:5601/
Use TARs to deploy OpenSearch Manually
- OpenSearch - Build 8873 (Note: Windows version does not have performance analyzer plugin)
  - arm64 [manifest] [tar] [rpm][deb]
  - x64 [manifest] [tar] [rpm] [deb] [windows]
- OpenSearch Dashboards - Build 6816
  - arm64 [manifest] [tar][rpm][deb]
  - x64 [manifest] [tar][rpm] [deb] [windows]

Check how to install opensearch and dashboards on different platforms

Divyaasm · 2023-11-18T01:48:45Z

Final Release Candidate details for OpenSearch - 8875 and OpenSearch Dashboards - 6816

OpenSearch - Build 8875
OpenSearch Dashboards - Build 6816

Use the following Docker-Compose to setup a cluster

docker-compose.yml

version: '3' services: opensearch-node1: image: opensearchstaging/opensearch:2.11.1.8875 container_name: opensearch-node1 environment: - cluster.name=opensearch-cluster - node.name=opensearch-node1 - discovery.seed_hosts=opensearch-node1,opensearch-node2 - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2 - bootstrap.memory_lock=true # along with the memlock settings below, disables swapping - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM ulimits: memlock: soft: -1 hard: -1 nofile: soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems hard: 65536 volumes: - opensearch-data1:/usr/share/opensearch/data ports: - 9200:9200 - 9600:9600 # required for Performance Analyzer networks: - opensearch-net opensearch-node2: image: opensearchstaging/opensearch:2.11.1.8875 container_name: opensearch-node2 environment: - cluster.name=opensearch-cluster - node.name=opensearch-node2 - discovery.seed_hosts=opensearch-node1,opensearch-node2 - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2 - bootstrap.memory_lock=true - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" ulimits: memlock: soft: -1 hard: -1 nofile: soft: 65536 hard: 65536 volumes: - opensearch-data2:/usr/share/opensearch/data networks: - opensearch-net opensearch-dashboards: image: opensearchstaging/opensearch-dashboards:2.11.1.6816 container_name: opensearch-dashboards ports: - 5601:5601 expose: - "5601" environment: OPENSEARCH_HOSTS: '["https://opensearch-node1:9200","https://opensearch-node2:9200"]' networks: - opensearch-net volumes: opensearch-data1: opensearch-data2:

networks: opensearch-net:

Download the above docker-compose.yml on your machine.
Get latest image versions docker-compose pull.
Start the cluster docker-compose up.

OpenSearch docker 2.11.1.8875
- Start without security
  - Docker command docker pull opensearchstaging/opensearch:2.11.1.8875 && docker run -it -p 9200:9200 -e "discovery.type=single-node" -e "DISABLE_SECURITY_PLUGIN=true" opensearchstaging/opensearch:2.11.1.8875
  - Connect command curl https://localhost:9200/
- Start with security
  - Docker command docker pull opensearchstaging/opensearch:2.11.1.8875 && docker run -it -p 9200:9200 -e "discovery.type=single-node" opensearchstaging/opensearch:2.11.1.8875
  - Connect command curl --insecure https://admin:admin@localhost:9200/
OpenSearch-Dashboards docker 2.11.1.6816
- Start without security
  - Docker command docker pull opensearchstaging/opensearch-dashboards:2.11.1.6816 && docker run -it --network="host" -e "DISABLE_SECURITY_DASHBOARDS_PLUGIN=true" opensearchstaging/opensearch-dashboards:2.11.1.6816
  - URL https://localhost:5601/
- Start with security
  - Docker command docker pull opensearchstaging/opensearch-dashboards:2.11.1.6816 && docker run -it --network="host" opensearchstaging/opensearch-dashboards:2.11.1.6816
  - URL https://localhost:5601/
Use TARs to deploy OpenSearch Manually
- OpenSearch - Build 8875 (Note: Windows version does not have performance analyzer plugin)
  - arm64 [manifest] [tar] [rpm][deb]
  - x64 [manifest] [tar] [rpm] [deb] [windows]
- OpenSearch Dashboards - Build 6816
  - arm64 [manifest] [tar][rpm][deb]
  - x64 [manifest] [tar][rpm] [deb] [windows]

Check how to install opensearch and dashboards on different platforms

Divyaasm · 2023-11-20T18:39:22Z

Integration Test results for 2.11.1:

For OpenSearch:

x64 Tarball: https://build.ci.opensearch.org/blue/organizations/jenkins/integ-test/detail/integ-test/6514/pipeline

Failed plugins:

| alerting | with-security | FAIL |

| security-analytics | with-security | FAIL |

arm64 Tarball: https://build.ci.opensearch.org/blue/organizations/jenkins/integ-test/detail/integ-test/6513/pipeline

Failed plugins:

| alerting | with-security | FAIL |

For OpenSearchDashBoards:

x64 Tarball: https://build.ci.opensearch.org/blue/organizations/jenkins/integ-test-opensearch-dashboards/detail/integ-test-opensearch-dashboards/4590/pipeline

Failed plugins:

| OpenSearch-Dashboards | with-security        | FAIL  |
| OpenSearch-Dashboards | without-security     | FAIL  |

| alertingDashboards   | with-security        | FAIL  |
| alertingDashboards   | without-security     | FAIL  |

| observabilityDashboards | with-security | FAIL |

arm64 Tarball: https://build.ci.opensearch.org/blue/organizations/jenkins/integ-test-opensearch-dashboards/detail/integ-test-opensearch-dashboards/4591/pipeline

Failed plugins:

| OpenSearch-Dashboards | with-security        | FAIL  |
| OpenSearch-Dashboards | without-security     | FAIL  |

| alertingDashboards   | with-security        | FAIL  |
| alertingDashboards   | without-security     | FAIL  |

bbarani · 2023-11-22T20:52:40Z

We couldn't finalize a release candidate build for 2.11.1 release due to last minute issues in OSD along with missing release notes in multiple repositories. We have moved the release date for 2.11.1 to Nov 27 2023 to accommodate the fixes. Please let us know if you have any questions.

pjfitzgibbons · 2023-11-27T20:04:33Z

I signoff dashboardsObservability plugin "with security" integ test - we have test manually with success.

kavilla · 2023-11-27T20:22:15Z

I signoff OpenSearch Dashboards integ tests - we have tested those manually and so success. We will do a quick follow to ensure test success on the official job.

cc: @manasvinibs

Divyaasm · 2023-11-27T21:44:07Z

bbarani · 2023-11-27T22:33:03Z

All, Maven publishing part is delayed due to this issue. We will publish it to Maven as soon as the issue is fixed

bbarani · 2023-11-27T23:16:09Z

Hello all, we discovered that few enhancements were merged in to the final release candidate of 2.11.1 version deviating from SemVar guidelines. We have requested the maintainers of the repo to revert the changes. We have moved the 2.11.1 release to Nov 30 2023 to accommodate the changes.

Divyaasm · 2023-11-28T17:37:24Z

Awaiting for the merge of the revert PR's with features and enhancements before moving forward with the 2.11.1 release

Divyaasm · 2023-11-29T17:20:22Z

Final Release Candidate details for OpenSearch - 8933 and OpenSearch Dashboards - 6867

OpenSearch - Build 8933
OpenSearch Dashboards - Build 6867

Use the following Docker-Compose to setup a cluster

docker-compose.yml

version: '3' services: opensearch-node1: image: opensearchstaging/opensearch:2.11.1.8933 container_name: opensearch-node1 environment: - cluster.name=opensearch-cluster - node.name=opensearch-node1 - discovery.seed_hosts=opensearch-node1,opensearch-node2 - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2 - bootstrap.memory_lock=true # along with the memlock settings below, disables swapping - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM ulimits: memlock: soft: -1 hard: -1 nofile: soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems hard: 65536 volumes: - opensearch-data1:/usr/share/opensearch/data ports: - 9200:9200 - 9600:9600 # required for Performance Analyzer networks: - opensearch-net opensearch-node2: image: opensearchstaging/opensearch:2.11.1.8933 container_name: opensearch-node2 environment: - cluster.name=opensearch-cluster - node.name=opensearch-node2 - discovery.seed_hosts=opensearch-node1,opensearch-node2 - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2 - bootstrap.memory_lock=true - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" ulimits: memlock: soft: -1 hard: -1 nofile: soft: 65536 hard: 65536 volumes: - opensearch-data2:/usr/share/opensearch/data networks: - opensearch-net opensearch-dashboards: image: opensearchstaging/opensearch-dashboards:2.11.1.6867 container_name: opensearch-dashboards ports: - 5601:5601 expose: - "5601" environment: OPENSEARCH_HOSTS: '["https://opensearch-node1:9200","https://opensearch-node2:9200"]' networks: - opensearch-net volumes: opensearch-data1: opensearch-data2:

networks: opensearch-net:

Download the above docker-compose.yml on your machine.
Get latest image versions docker-compose pull.
Start the cluster docker-compose up.

OpenSearch docker 2.11.1.8933
- Start without security
  - Docker command docker pull opensearchstaging/opensearch:2.11.1.8933 && docker run -it -p 9200:9200 -e "discovery.type=single-node" -e "DISABLE_SECURITY_PLUGIN=true" opensearchstaging/opensearch:2.11.1.8933
  - Connect command curl http://localhost:9200/
- Start with security
  - Docker command docker pull opensearchstaging/opensearch:2.11.1.8933 && docker run -it -p 9200:9200 -e "discovery.type=single-node" opensearchstaging/opensearch:2.11.1.8933
  - Connect command curl --insecure https://admin:admin@localhost:9200/
OpenSearch-Dashboards docker 2.11.1.6867
- Start without security
  - Docker command docker pull opensearchstaging/opensearch-dashboards:2.11.1.6867 && docker run -it --network="host" -e "DISABLE_SECURITY_DASHBOARDS_PLUGIN=true" opensearchstaging/opensearch-dashboards:2.11.1.6867
  - URL http://localhost:5601/
- Start with security
  - Docker command docker pull opensearchstaging/opensearch-dashboards:2.11.1.6867 && docker run -it --network="host" opensearchstaging/opensearch-dashboards:2.11.1.6867
  - URL https://localhost:5601/
Use TARs to deploy OpenSearch Manually
- OpenSearch - Build 8933 (Note: Windows version does not have performance analyzer plugin)
  - arm64 [manifest] [tar] [rpm][deb]
  - x64 [manifest] [tar] [rpm] [deb] [windows]
- OpenSearch Dashboards - Build 6867
  - arm64 [manifest] [tar][rpm][deb]
  - x64 [manifest] [tar][rpm] [deb] [windows]

Check how to install opensearch and dashboards on different platforms

Divyaasm · 2023-11-29T17:43:09Z

Integration Test results for 2.11.1:

For OpenSearch - 8933:

x64 Tarball: https://build.ci.opensearch.org/blue/organizations/jenkins/integ-test/detail/integ-test/6515/pipeline

arm64 Tarball: https://build.ci.opensearch.org/blue/organizations/jenkins/integ-test/detail/integ-test/6516/pipeline

All components integration tests are passing for OpenSearch

For OpenSearchDashBoards - 6867:

x64 Tarball: https://build.ci.opensearch.org/blue/organizations/jenkins/integ-test-opensearch-dashboards/detail/integ-test-opensearch-dashboards/4648/pipeline

Failed plugins:

| OpenSearch-Dashboards | with-security        | FAIL  |
| OpenSearch-Dashboards | without-security     | FAIL  |

Check the test-report manifest in the issue for logs related to OSD integ-test failure

arm64 Tarball: https://build.ci.opensearch.org/blue/organizations/jenkins/integ-test-opensearch-dashboards/detail/integ-test-opensearch-dashboards/4649/pipeline

Failed plugins:

| OpenSearch-Dashboards | with-security        | FAIL  |
| OpenSearch-Dashboards | without-security     | FAIL  |

Check the test-report manifest in the issue for logs related to OSD integ-test failure

kavilla · 2023-11-29T18:19:58Z

Manual sign off on those tests. Currently working on it: opensearch-project/OpenSearch-Dashboards#5506

Divyaasm · 2023-11-30T02:06:22Z

Final Release Candidate details for OpenSearch - 8942 and OpenSearch Dashboards - 6867

OpenSearch - Build 8942
OpenSearch Dashboards - Build 6867

Use the following Docker-Compose to setup a cluster

docker-compose.yml

version: '3' services: opensearch-node1: image: opensearchstaging/opensearch:2.11.1.8942 container_name: opensearch-node1 environment: - cluster.name=opensearch-cluster - node.name=opensearch-node1 - discovery.seed_hosts=opensearch-node1,opensearch-node2 - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2 - bootstrap.memory_lock=true # along with the memlock settings below, disables swapping - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM ulimits: memlock: soft: -1 hard: -1 nofile: soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems hard: 65536 volumes: - opensearch-data1:/usr/share/opensearch/data ports: - 9200:9200 - 9600:9600 # required for Performance Analyzer networks: - opensearch-net opensearch-node2: image: opensearchstaging/opensearch:2.11.1.8942 container_name: opensearch-node2 environment: - cluster.name=opensearch-cluster - node.name=opensearch-node2 - discovery.seed_hosts=opensearch-node1,opensearch-node2 - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2 - bootstrap.memory_lock=true - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" ulimits: memlock: soft: -1 hard: -1 nofile: soft: 65536 hard: 65536 volumes: - opensearch-data2:/usr/share/opensearch/data networks: - opensearch-net opensearch-dashboards: image: opensearchstaging/opensearch-dashboards:2.11.1.6867 container_name: opensearch-dashboards ports: - 5601:5601 expose: - "5601" environment: OPENSEARCH_HOSTS: '["https://opensearch-node1:9200","https://opensearch-node2:9200"]' networks: - opensearch-net volumes: opensearch-data1: opensearch-data2:

networks: opensearch-net:

Download the above docker-compose.yml on your machine.
Get latest image versions docker-compose pull.
Start the cluster docker-compose up.

OpenSearch docker 2.11.1.8942
- Start without security
  - Docker command docker pull opensearchstaging/opensearch:2.11.1.8942 && docker run -it -p 9200:9200 -e "discovery.type=single-node" -e "DISABLE_SECURITY_PLUGIN=true" opensearchstaging/opensearch:2.11.1.8942
  - Connect command curl http://localhost:9200/
- Start with security
  - Docker command docker pull opensearchstaging/opensearch:2.11.1.8942 && docker run -it -p 9200:9200 -e "discovery.type=single-node" opensearchstaging/opensearch:2.11.1.8942
  - Connect command curl --insecure https://admin:admin@localhost:9200/
OpenSearch-Dashboards docker 2.11.1.6867
- Start without security
  - Docker command docker pull opensearchstaging/opensearch-dashboards:2.11.1.6867 && docker run -it --network="host" -e "DISABLE_SECURITY_DASHBOARDS_PLUGIN=true" opensearchstaging/opensearch-dashboards:2.11.1.6867
  - URL http://localhost:5601/
- Start with security
  - Docker command docker pull opensearchstaging/opensearch-dashboards:2.11.1.6867 && docker run -it --network="host" opensearchstaging/opensearch-dashboards:2.11.1.6867
  - URL https://localhost:5601/
Use TARs to deploy OpenSearch Manually
- OpenSearch - Build 8942 (Note: Windows version does not have performance analyzer plugin)
  - arm64 [manifest] [tar] [rpm][deb]
  - x64 [manifest] [tar] [rpm] [deb] [windows]
- OpenSearch Dashboards - Build 6867
  - arm64 [manifest] [tar][rpm][deb]
  - x64 [manifest] [tar][rpm] [deb] [windows]

Check how to install opensearch and dashboards on different platforms

Divyaasm · 2023-11-30T02:38:10Z

Integration Test results for 2.11.1:

For OpenSearch - 8942:

x64 Tarball: https://build.ci.opensearch.org/blue/organizations/jenkins/integ-test/detail/integ-test/6524/pipeline
arm64 Tarball: https://build.ci.opensearch.org/blue/organizations/jenkins/integ-test/detail/integ-test/6526/pipeline

All components integration tests are passing for OpenSearch

For OpenSearchDashBoards - 6867:

x64 Tarball: https://build.ci.opensearch.org/blue/organizations/jenkins/integ-test-opensearch-dashboards/detail/integ-test-opensearch-dashboards/4648/pipeline

Failed plugins:

| OpenSearch-Dashboards | with-security        | FAIL  |
| OpenSearch-Dashboards | without-security     | FAIL  |

Check the test-report manifest in the issue for logs related to OSD integ-test failure

arm64 Tarball: https://build.ci.opensearch.org/blue/organizations/jenkins/integ-test-opensearch-dashboards/detail/integ-test-opensearch-dashboards/4649/pipeline

Failed plugins:

| OpenSearch-Dashboards | with-security        | FAIL  |
| OpenSearch-Dashboards | without-security     | FAIL  |

Check the test-report manifest in the issue for logs related to OSD integ-test failure

bbarani · 2023-11-30T22:58:47Z

Validated signatures:

gpg --verify opensearch-2.11.1-linux-x64.tar.gz.sig  opensearch-2.11.1-linux-x64.tar.gz    
gpg: Signature made Thu Nov 30 11:34:31 2023 PST
gpg:                using RSA key C2EE2AF6542C03B4
gpg: Good signature from "OpenSearch project <opensearch@amazon.com>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: C5B7 4989 65EF D1C2 924B  A9D5 39D3 1987 9310 D3FC
     Subkey fingerprint: 2187 3199 B103 0FCD 49DA  83F8 C2EE 2AF6 542C 03B4

Divyaasm · 2023-11-30T23:10:44Z

Native Plugin Installation:

 ./opensearch-plugin install repository-s3
-> Installing repository-s3
-> Downloading repository-s3 from opensearch
[=================================================] 100%   
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@     WARNING: plugin requires additional permissions     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
* java.io.FilePermission config#plus read
* java.lang.RuntimePermission accessDeclaredMembers
* java.lang.RuntimePermission getClassLoader
* java.lang.RuntimePermission setContextClassLoader
* java.lang.reflect.ReflectPermission suppressAccessChecks
* java.net.NetPermission setDefaultAuthenticator
* java.net.SocketPermission * connect,resolve
* java.util.PropertyPermission aws.configFile read,write
* java.util.PropertyPermission aws.sharedCredentialsFile read,write
* java.util.PropertyPermission opensearch.allow_insecure_settings read,write
* java.util.PropertyPermission opensearch.path.conf read,write
See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
for descriptions of what these permissions allow and the associated risks.

Continue with installation? [y/N]y
-> Installed repository-s3 with folder name repository-s3

bbarani · 2023-11-30T23:11:44Z

OpenSearch 2.11.1 version has been released 🎉 !

github-actions bot added release untriaged Issues that have not yet been triaged v2.11.1 labels Oct 20, 2023

peterzhuamazon removed the untriaged Issues that have not yet been triaged label Oct 24, 2023

bbarani assigned Divyaasm Oct 27, 2023

cwperks mentioned this issue Oct 27, 2023

Use new instance of Decompressor on channel initialization opensearch-project/security#3583

Merged

3 tasks

Divyaasm pinned this issue Oct 30, 2023

prudhvigodithi mentioned this issue Nov 17, 2023

[BUG] Missing updateVersion gradle task opensearch-project/custom-codecs#84

Closed

bbarani mentioned this issue Nov 18, 2023

[Enhancement] Block features from getting merged in to release branch after a release #4223

Open

cwperks mentioned this issue Nov 20, 2023

Add release notes for 2.11.1 opensearch-project/security#3749

Merged

3 tasks

mmguero mentioned this issue Nov 21, 2023

update OpenSearch to v2.11.1 idaholab/Malcolm#298

Closed

monusingh-1 mentioned this issue Nov 22, 2023

Add 2.11.1.0 release notes opensearch-project/cross-cluster-replication#1285

Merged

5 tasks

prudhvigodithi mentioned this issue Nov 28, 2023

Allow lang-painless maven package to add compile dependencies opensearch-project/OpenSearch#11356

Closed

1 task

This was referenced Nov 30, 2023

Increment OpenSearch and Dashboards to 2.11.1 opensearch-project/ansible-playbook#148

Merged

Upgrade appVersions to 2.11.1 opensearch-project/helm-charts#500

Merged

[Retrospective] Release Version 2.11.1 #4259

Closed

vooon mentioned this issue Dec 4, 2023

Release 2.11.1.0 Aiven-Open/prometheus-exporter-plugin-for-opensearch#238

Closed

5 tasks

gaiksaya closed this as completed Dec 8, 2023

gaiksaya unpinned this issue Dec 8, 2023

[RELEASE] Release version 2.11.1 #4161

[RELEASE] Release version 2.11.1 #4161

Comments

github-actions bot commented Oct 20, 2023 • edited by Divyaasm Loading

Release OpenSearch and OpenSearch Dashboards 2.11.1

This Release Issue

Preparation

Campaigns

Release Branch and Version Increment - Ends Nov 14 2023

Feature Freeze

Code Complete - Ends date

Release Candidate Creation and Testing - Ends Nov 21 2023

Pre Release - Ends Nov 24 2023

Release - Ends Nov 27 2023

Release Checklist.

Pre-Release activities

Release activities

Post-Release activities

Post Release

Components

bbarani commented Oct 27, 2023

Bug fixes targeted for 2.11.1 release:

peternied commented Oct 27, 2023

bbarani commented Oct 31, 2023 • edited Loading

CEHENKLE commented Nov 1, 2023 • edited Loading

reta commented Nov 1, 2023

smortex commented Nov 2, 2023

Regarding back-ported features in release branches

Regarding what to do with the next release

peternied commented Nov 2, 2023

macohen commented Nov 2, 2023 • edited Loading

joshuarrrr commented Nov 2, 2023

dblock commented Nov 2, 2023

reta commented Nov 2, 2023

austintlee commented Nov 3, 2023

krisfreedain commented Nov 3, 2023

stephen-crawford commented Nov 3, 2023

CEHENKLE commented Nov 4, 2023

Divyaasm commented Nov 17, 2023

Release Candidate details for OpenSearch - 8873 and OpenSearch Dashboards - 6816

Divyaasm commented Nov 18, 2023

Final Release Candidate details for OpenSearch - 8875 and OpenSearch Dashboards - 6816

Divyaasm commented Nov 20, 2023 • edited Loading

Integration Test results for 2.11.1:

For OpenSearch:

For OpenSearchDashBoards:

bbarani commented Nov 22, 2023 • edited Loading

pjfitzgibbons commented Nov 27, 2023

kavilla commented Nov 27, 2023 • edited Loading

Divyaasm commented Nov 27, 2023 • edited Loading

OpenSearch 2.11.1 Release Checklist

Pre-Release activities

Release activities

Post-Release activities

bbarani commented Nov 27, 2023

bbarani commented Nov 27, 2023 • edited Loading

Divyaasm commented Nov 28, 2023

Divyaasm commented Nov 29, 2023 • edited Loading

Final Release Candidate details for OpenSearch - 8933 and OpenSearch Dashboards - 6867

Divyaasm commented Nov 29, 2023

Integration Test results for 2.11.1:

For OpenSearch - 8933:

For OpenSearchDashBoards - 6867:

kavilla commented Nov 29, 2023

Divyaasm commented Nov 30, 2023 • edited Loading

Final Release Candidate details for OpenSearch - 8942 and OpenSearch Dashboards - 6867

Divyaasm commented Nov 30, 2023

Integration Test results for 2.11.1:

For OpenSearch - 8942:

For OpenSearchDashBoards - 6867:

bbarani commented Nov 30, 2023 • edited Loading

Divyaasm commented Nov 30, 2023

bbarani commented Nov 30, 2023

github-actions bot commented Oct 20, 2023 •

edited by Divyaasm

Loading

bbarani commented Oct 31, 2023 •

edited

Loading

CEHENKLE commented Nov 1, 2023 •

edited

Loading

macohen commented Nov 2, 2023 •

edited

Loading

Divyaasm commented Nov 20, 2023 •

edited

Loading

bbarani commented Nov 22, 2023 •

edited

Loading

kavilla commented Nov 27, 2023 •

edited

Loading

Divyaasm commented Nov 27, 2023 •

edited

Loading

bbarani commented Nov 27, 2023 •

edited

Loading

Divyaasm commented Nov 29, 2023 •

edited

Loading

Divyaasm commented Nov 30, 2023 •

edited

Loading

bbarani commented Nov 30, 2023 •

edited

Loading