Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add install_demo_configuration Batch script for Windows #2161

Merged
merged 12 commits into from
Oct 27, 2022
Merged

Add install_demo_configuration Batch script for Windows #2161

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
91fa9bf
Patch bump for scala dependency
stephen-crawford Oct 12, 2022
a2d2e11
Merge branch 'opensearch-project:main' into main
stephen-crawford Oct 20, 2022
a593d98
Merge branch 'opensearch-project:main' into main
stephen-crawford Oct 24, 2022
b7b6d23
Move changes to main based
stephen-crawford Oct 24, 2022
f4e52dc
Create batch script for install_demo_configuration
cwperks Oct 12, 2022
c2b58a7
Finish implementation of install_demo_configuration.bat
cwperks Oct 12, 2022
a1ad72f
Surround set commands with quotes
cwperks Oct 13, 2022
53839ae
Avoid hidden carriage return on footer line
cwperks Oct 24, 2022
6d5d4ba
Merge pull request #2 from scrawfor99/windows-plugin-install
cwperks Oct 24, 2022
0754c59
Version 3.0.0 numbering change
stephen-crawford Oct 24, 2022
497912f
Disable windows-install on 3.0.0 until stable distro is available
cwperks Oct 24, 2022
f4fb6cf
Re-enable windows plugin install
cwperks Oct 27, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
173 changes: 115 additions & 58 deletions .github/workflows/plugin_install.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,63 +3,120 @@ name: Plugin Install
on: [push, pull_request, workflow_dispatch]

jobs:
plugin_install:
name: Plugin Install

linux-install:

runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v2

- name: Set up JDK
uses: actions/setup-java@v1
with:
java-version: 11

- name: Build
run: ./gradlew clean assemble -Dbuild.snapshot=false

- name: Download OpenSearch Core
run: |
opensearch_version=`./gradlew properties -q | grep "opensearch_version:" | awk '{print $2}' | sed 's/-SNAPSHOT//g'`
wget https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/$opensearch_version/latest/linux/x64/tar/builds/opensearch/dist/opensearch-min-$opensearch_version-linux-x64.tar.gz
tar -xzf opensearch-*.tar.gz
rm -f opensearch-*.tar.gz

- name: Move and rename security plugin for installation
run: mv build/distributions/opensearch-security-*.zip opensearch-security.zip

- name: Run OpenSearch with plugin
run: |
cat > os-ep.sh <<EOF
yes | opensearch-plugin install file:///docker-host/security-plugin.zip
chmod +x plugins/opensearch-security/tools/install_demo_configuration.sh
yes | plugins/opensearch-security/tools/install_demo_configuration.sh
chown 1001:1001 -R /opensearch
su -c "/opensearch/bin/opensearch" -s /bin/bash opensearch
EOF
docker build -t opensearch-test:latest -f- . <<EOF
FROM ubuntu:latest
COPY --chown=1001:1001 os-ep.sh /docker-host/
COPY --chown=1001:1001 opensearch-security.zip /docker-host/security-plugin.zip
COPY --chown=1001:1001 opensearch* /opensearch/
RUN chmod +x /docker-host/os-ep.sh
RUN useradd -u 1001 -s /sbin/nologin opensearch
ENV PATH="/opensearch/bin:${PATH}"
WORKDIR /opensearch/
ENTRYPOINT /docker-host/os-ep.sh
EOF
docker run --name ops -d -p 9200:9200 -p 9600:9600 -i opensearch-test:latest

- name: Sleep while OpenSearch finishes starting up
uses: whatnick/wait-action@v0.1.2
with:
time: '30s'

- name: Check OpenSearch Running
run: curl -XGET https://localhost:9200 -u 'admin:admin' -k -v

- name: Get Docker Logs
if: always()
run: docker logs ops

- name: Run sanity tests
run: ./gradlew integTestRemote -Dtests.rest.cluster=localhost:9200 -Dtests.cluster=localhost:9200 -Dtests.clustername="opensearch" -Dhttps=true -Duser=admin -Dpassword=admin
- uses: actions/checkout@v2

- name: Set up JDK
uses: actions/setup-java@v1
with:
java-version: 11

- name: Build
run: ./gradlew clean assemble -Dbuild.snapshot=false

- name: Download OpenSearch Core
run: |
opensearch_version=`./gradlew properties -q | grep "opensearch_version:" | awk '{print $2}' | sed 's/-SNAPSHOT//g'`
wget https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/$opensearch_version/latest/linux/x64/tar/builds/opensearch/dist/opensearch-min-$opensearch_version-linux-x64.tar.gz
tar -xzf opensearch-*.tar.gz
rm -f opensearch-*.tar.gz

- name: Move and rename security plugin for installation
run: mv build/distributions/opensearch-security-*.zip opensearch-security.zip

- name: Run OpenSearch with plugin
run: |
cat > os-ep.sh <<EOF
yes | opensearch-plugin install file:///docker-host/security-plugin.zip
chmod +x plugins/opensearch-security/tools/install_demo_configuration.sh
yes | plugins/opensearch-security/tools/install_demo_configuration.sh
chown 1001:1001 -R /opensearch
su -c "/opensearch/bin/opensearch" -s /bin/bash opensearch
EOF
docker build -t opensearch-test:latest -f- . <<EOF
FROM ubuntu:latest
COPY --chown=1001:1001 os-ep.sh /docker-host/
COPY --chown=1001:1001 opensearch-security.zip /docker-host/security-plugin.zip
COPY --chown=1001:1001 opensearch* /opensearch/
RUN chmod +x /docker-host/os-ep.sh
RUN useradd -u 1001 -s /sbin/nologin opensearch
ENV PATH="/opensearch/bin:${PATH}"
WORKDIR /opensearch/
ENTRYPOINT /docker-host/os-ep.sh
EOF
docker run --name ops -d -p 9200:9200 -p 9600:9600 -i opensearch-test:latest

- name: Sleep while OpenSearch finishes starting up
uses: whatnick/wait-action@v0.1.2
with:
time: '30s'

- name: Check OpenSearch Running
run: curl -XGET https://localhost:9200 -u 'admin:admin' -k -v

- name: Get Docker Logs
if: always()
run: docker logs ops

- name: Run sanity tests
run: ./gradlew integTestRemote -Dtests.rest.cluster=localhost:9200 -Dtests.cluster=localhost:9200 -Dtests.clustername="opensearch" -Dhttps=true -Duser=admin -Dpassword=admin

windows-install:

runs-on: windows-latest

steps:
- name: Enable longer filenames for Windows
run: git config --system core.longpaths true

- name: Checkout Plugin
uses: actions/checkout@v2

- name: Set up JDK
uses: actions/setup-java@v1
with:
java-version: 11

- name: Download OpenSearch Core
run: |
cd ..
Invoke-WebRequest https://artifacts.opensearch.org/snapshots/core/opensearch/3.0.0-SNAPSHOT/opensearch-min-3.0.0-SNAPSHOT-windows-x64-latest.zip -Outfile opensearch-3.0.0.zip
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we replace all the references to 3.0.0 with something like ${{ env.OPENSEARCH_VERSION }}, see how this can be set for the job with the env [1]

[1] https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#env

tar -xzf opensearch-3.0.0.zip
del opensearch-3.0.0.zip
Rename-Item opensearch-* Opensearch

- name: Build Security Plugin with Gradle
run: |
.\gradlew.bat assemble
env:
_JAVA_OPTIONS: -Xmx4096M

- name: Move and rename security plugin
run: move build\distributions\opensearch-security-*.zip ..\opensearch-security.zip

- name: Install the plugin
run: |
cd ..
'y' | .\Opensearch\bin\opensearch-plugin.bat install file:\a\security\opensearch-security.zip
'y', 'y', 'N' | .\Opensearch\plugins\opensearch-security\tools\install_demo_configuration.bat

- name: Run OpenSearch with plugin
run: |
cd ..
start .\Opensearch\bin\opensearch.bat
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should have a way to get the logs from the process in cases it doesn't come up like in the linux version

https://github.com/opensearch-project/security/blob/main/.github/workflows/plugin_install.yml#L60-L62

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note; this is a good candidate for a follow up issue


- name: Sleep while OpenSearch finishes starting up
run: Start-Sleep -s 30

- name: Check OpenSearch Running
run: |
$credentialBytes = [Text.Encoding]::ASCII.GetBytes("admin:admin")
$encodedCredentials = [Convert]::ToBase64String($credentialBytes)
$baseCredentials = "Basic $encodedCredentials"
$Headers = @{ Authorization = $baseCredentials }
Invoke-WebRequest -SkipCertificateCheck -Uri 'https://localhost:9200' -Headers $Headers
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Save some code and rewrite as Invoke-WebRequest -SkipCertificateCheck -Uri 'https://admin:admin@localhost:9200' , worked when I tried it locally. More details on this format [1]

[1] https://serverfault.com/questions/371907/can-you-pass-user-pass-for-http-basic-authentication-in-url-parameters

Loading