Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build OpenSearch in CD workflow in order to build security plugin #1364

Merged
merged 3 commits into from
Aug 7, 2021
Merged

Build OpenSearch in CD workflow in order to build security plugin #1364

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
bfc1d2b
Build OpenSearch in CD workflow in order to build security plugin
cliu123 Jul 30, 2021
e0d8e5a
Remove the unnecessary configurations that were for uploading artifac…
cliu123 Aug 2, 2021
03a261d
Move Build OpenSearch step after Cache
cliu123 Aug 3, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
88 changes: 50 additions & 38 deletions .github/workflows/cd.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,25 @@ jobs:

steps:

- name: Set up JDK 11
- name: Set up JDK 14
uses: actions/setup-java@v1
with:
java-version: 11.0.x
java-version: 14.0.x
server-id: sonatype-nexus-staging
cliu123 marked this conversation as resolved.
Show resolved Hide resolved
server-username: SONATYPE_USER
server-password: SONATYPE_PASSWORD

- name: Checkout OpenSearch
uses: actions/checkout@v2
with:
repository: 'opensearch-project/OpenSearch'
path: OpenSearch
ref: '1.0'

- name: Build OpenSearch
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this step be after Cache Maven packages step?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't OpenSearch artifact built be cached?
If moving Build OpenSearch step and putting it after Cache Maven packages step, both of Build OpenSearch and Build steps will be after Cache Maven packages step. Shouldn't the build outputs be cached?

Copy link
Contributor

@vrozov vrozov Aug 3, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the point of caching OpenSearch maven packages when they are rebuilt anyway?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree. In this case, only downloaded dependencies should be cached. There is no point of caching build outputs.
Moved. Thanks!

working-directory: ./OpenSearch
run: ./gradlew publishToMavenLocal -Dbuild.snapshot=false

- name: Checkout security
uses: actions/checkout@v2

Expand All @@ -41,42 +52,43 @@ jobs:
zip -r artifacts.zip artifacts
echo "TAG_VERSION=${GITHUB_REF/refs\/tags\//}" >> $GITHUB_ENV

- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_STAGING_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_STAGING_SECRET_ACCESS_KEY }}
aws-region: us-west-2

- name: Upload Artifacts to S3
run: |
zip=`ls artifacts/*.zip`
rpm=`ls artifacts/*.rpm`
deb=`ls artifacts/*.deb`

# Inject the build number before the suffix
zip_outfile=`basename ${zip%.zip}-build-${GITHUB_RUN_NUMBER}.zip`
rpm_outfile=`basename ${rpm%.rpm}-build-${GITHUB_RUN_NUMBER}.rpm`
deb_outfile=`basename ${deb%.deb}-build-${GITHUB_RUN_NUMBER}.deb`

s3_prefix="s3://staging.artifacts.opendistroforelasticsearch.amazon.com/snapshots/elasticsearch-plugins/security/"

echo "Copying ${zip} to ${s3_prefix}${zip_outfile}"
aws s3 cp --quiet $zip ${s3_prefix}${zip_outfile}

echo "Copying ${rpm} to ${s3_prefix}${rpm_outfile}"
aws s3 cp --quiet $rpm ${s3_prefix}${rpm_outfile}

echo "Copying ${deb} to ${s3_prefix}${deb_outfile}"
aws s3 cp --quiet $deb ${s3_prefix}${deb_outfile}

- name: Upload Artifacts to Maven Central
env:
SONATYPE_USER: ${{ secrets.SONATYPE_USER }}
SONATYPE_PASSWORD: ${{ secrets.SONATYPE_PASSWORD }}
run: |
gpg --batch --import --no-tty <(echo -e "${{ secrets.PGP_PRIVATE_KEY }}")
mvn -B deploy -Padvanced -Prelease -DskipTests -Dgpg.passphrase=${{ secrets.PGP_PASSPHRASE }}
# TODO: Update the following configuration commented out once the new CI/CD process of OpenSearch is confirmed.
cliu123 marked this conversation as resolved.
Show resolved Hide resolved
# - name: Configure AWS Credentials
# uses: aws-actions/configure-aws-credentials@v1
# with:
# aws-access-key-id: ${{ secrets.AWS_STAGING_ACCESS_KEY_ID }}
# aws-secret-access-key: ${{ secrets.AWS_STAGING_SECRET_ACCESS_KEY }}
# aws-region: us-west-2
#
# - name: Upload Artifacts to S3
# run: |
# zip=`ls artifacts/*.zip`
# rpm=`ls artifacts/*.rpm`
# deb=`ls artifacts/*.deb`
#
# # Inject the build number before the suffix
# zip_outfile=`basename ${zip%.zip}-build-${GITHUB_RUN_NUMBER}.zip`
# rpm_outfile=`basename ${rpm%.rpm}-build-${GITHUB_RUN_NUMBER}.rpm`
# deb_outfile=`basename ${deb%.deb}-build-${GITHUB_RUN_NUMBER}.deb`
#
# s3_prefix="s3://staging.artifacts.opendistroforelasticsearch.amazon.com/snapshots/elasticsearch-plugins/security/"
#
# echo "Copying ${zip} to ${s3_prefix}${zip_outfile}"
# aws s3 cp --quiet $zip ${s3_prefix}${zip_outfile}
#
# echo "Copying ${rpm} to ${s3_prefix}${rpm_outfile}"
# aws s3 cp --quiet $rpm ${s3_prefix}${rpm_outfile}
#
# echo "Copying ${deb} to ${s3_prefix}${deb_outfile}"
# aws s3 cp --quiet $deb ${s3_prefix}${deb_outfile}
#
# - name: Upload Artifacts to Maven Central
# env:
# SONATYPE_USER: ${{ secrets.SONATYPE_USER }}
# SONATYPE_PASSWORD: ${{ secrets.SONATYPE_PASSWORD }}
# run: |
# gpg --batch --import --no-tty <(echo -e "${{ secrets.PGP_PRIVATE_KEY }}")
# mvn -B deploy -Padvanced -Prelease -DskipTests -Dgpg.passphrase=${{ secrets.PGP_PASSPHRASE }}

- name: Create Github Draft Release
id: create_release
Expand Down