Skip to content

Commit

Permalink
[Dev] enable opensearch snapshot for Darwin (#3537) (#3548)
Browse files Browse the repository at this point in the history
Enable the downloading of Darwin for running the command
`yarn opensearch snapshot`.

Darwin is not officially supported but snapshots are being
built here:
https://build.ci.opensearch.org/job/distribution-build-opensearch/

Issue resolved:
#2944

Signed-off-by: Kawika Avilla <kavilla414@gmail.com>
Co-authored-by: Josh Romero <rmerqg@amazon.com>
  • Loading branch information
kavilla and joshuarrrr authored Mar 8, 2023
1 parent ef7caaa commit b90fcde
Show file tree
Hide file tree
Showing 4 changed files with 48 additions and 8 deletions.
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,9 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
- [Data] Make the newly created configurations get added to beginning of the `aggConfig` array when using `createAggConfig` ([#3160](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/3160))
- [Optimizer] Increase the amount of time an optimizer worker is provided to exit before throwing an error ([#3193](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/3193))
- [Multiple DataSource] Add support for SigV4 authentication ([#3058](https://github.com/opensearch-project/OpenSearch-Dashboards/issues/3058))
- Make build scripts find and use the latest version of Node.js that satisfies `engines.node` ([#3467](https://github.com/opensearch-project/OpenSearch-Dashboards/issues/3467))
- [Multiple DataSource] Refactor test connection to support SigV4 auth type ([#3456](https://github.com/opensearch-project/OpenSearch-Dashboards/issues/3456))
- [Darwin] Add support for Darwin for running OpenSearch snapshots with `yarn opensearch snapshot` ([#3537](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/3537))

### 🐛 Bug Fixes

Expand Down
6 changes: 3 additions & 3 deletions DEVELOPER_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ $ yarn osd clean

OpenSearch Dashboards requires a running version of OpenSearch to connect to. In a separate terminal you can run the latest snapshot built using:

_(Linux and Windows only - for MacOS, you'll need to [run OpenSearch from a tarball](#alternative---run-opensearch-from-tarball) instead)_
_(Linux, Windows, Darwin (MacOS) only - for others, you'll need to [run OpenSearch from a tarball](#alternative---run-opensearch-from-tarball) instead)_
```bash
$ yarn opensearch snapshot
```
Expand Down Expand Up @@ -184,7 +184,7 @@ By default, the snapshot command will run [a minimal distribution of OpenSearch]

If you would like to run OpenSearch with a particular plugin installed on the cluster snapshot, pass the `--P` flag after `yarn opensearch snapshot`. You can use the flag multiple times to install multiple plugins. The argument value can be a URL to the plugin's zip file, maven coordinates of the plugin, or a local zip file path (use `file://` followed by the absolute or relative path, in that case). For example:

_(Linux and Windows only - for MacOS, you'll need to [run OpenSearch from a tarball](#alternative---run-opensearch-from-tarball) instead)_
_(Linux, Windows, Darwin (MacOS) only - for others, you'll need to [run OpenSearch from a tarball](#alternative---run-opensearch-from-tarball) instead)_
```bash
$ yarn opensearch snapshot --P https://repo1.maven.org/maven2/org/opensearch/plugin/opensearch-test-plugin/2.4.0.0/opensearch-test-plugin-2.4.0.0.zip
```
Expand Down Expand Up @@ -214,7 +214,7 @@ $ yarn opensearch snapshot --version 2.2.0 -E cluster.name=test -E path.data=/tm

### Alternative - Run OpenSearch from tarball

OpenSearch does not yet create artifacts for MacOS, so you'll need to download, install, and run the tarball instead. (You'll also need Java installed and the `JAVA_HOME` environmental variable set - see [OpenSearch developer guide](https://github.com/opensearch-project/OpenSearch/blob/main/DEVELOPER_GUIDE.md#install-prerequisites) for details).
If you would like to run OpenSearch from the tarball, you'll need to download the minimal distribution, install it, and then run the executable. (You'll also need Java installed and the `JAVA_HOME` environmental variable set - see [OpenSearch developer guide](https://github.com/opensearch-project/OpenSearch/blob/main/DEVELOPER_GUIDE.md#install-prerequisites) for details).

1. Download the latest minimal distribution of OpenSearch from [the downloads page](https://opensearch.org/downloads.html#minimal). Note the version and replace in commands below.
2. Unzip the `tar.gz` file: `tar -xvf opensearch-<OpenSearch-version>-linux-x64.tar.gz`
Expand Down
5 changes: 3 additions & 2 deletions packages/osd-opensearch/src/artifact.js
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ const { createHash } = require('crypto');
const path = require('path');

const asyncPipeline = promisify(pipeline);
const SUPPORTED_PLATFORMS = ['linux', 'windows', 'darwin'];
const DAILY_SNAPSHOTS_BASE_URL = 'https://artifacts.opensearch.org/snapshots/core/opensearch';
// TODO: [RENAMEME] currently do not have an existing replacement
// issue: https://github.com/opensearch-project/OpenSearch-Dashboards/issues/475
Expand Down Expand Up @@ -188,8 +189,8 @@ async function getArtifactSpecForSnapshotFromUrl(urlVersion, log) {
const arch = process.arch === 'arm64' ? 'arm64' : 'x64';
const extension = process.platform === 'win32' ? 'zip' : 'tar.gz';

if (platform !== 'linux' && platform !== 'windows') {
throw createCliError(`Snapshots are only available for Linux and Windows`);
if (!SUPPORTED_PLATFORMS.includes(platform)) {
throw createCliError(`Snapshots are only available for Linux, Windows, and Darwin`);
}

const latestUrl = `${DAILY_SNAPSHOTS_BASE_URL}/${desiredVersion}-SNAPSHOT`;
Expand Down
42 changes: 39 additions & 3 deletions packages/osd-opensearch/src/artifact.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -163,17 +163,17 @@ describe('Artifact', () => {
});
});

it('should throw when on a non-Linux or non-Windows platform', async () => {
it('should throw when on a non-Linux, non-Windows, non-Darwin platform', async () => {
Object.defineProperties(process, {
platform: {
value: 'darwin',
value: 'android',
},
arch: {
value: ORIGINAL_ARCHITECTURE,
},
});
await expect(Artifact.getSnapshot('default', 'INVALID_PLATFORM', log)).rejects.toThrow(
'Snapshots are only available for Linux'
'Snapshots are only available for Linux, Windows, and Darwin'
);
});

Expand All @@ -189,6 +189,42 @@ describe('Artifact', () => {
mockFetch(MOCKS.multipleArch[0]);
artifactTest();
});

it('should not throw when on a Linux platform', async () => {
Object.defineProperties(process, {
platform: {
value: 'linux',
},
arch: {
value: 'x64',
},
});
artifactTest();
});

it('should not throw when on a Windows platform', async () => {
Object.defineProperties(process, {
platform: {
value: 'win32',
},
arch: {
value: 'x64',
},
});
artifactTest();
});

it('should not throw when on a Darwin platform', async () => {
Object.defineProperties(process, {
platform: {
value: 'darwin',
},
arch: {
value: 'x64',
},
});
artifactTest();
});
});

describe('with custom snapshot manifest URL', () => {
Expand Down

0 comments on commit b90fcde

Please sign in to comment.