Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Setup GitHub actions #617

Merged
merged 38 commits into from
Jan 25, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
44eb576
intial commit to test one job for Github actions
tijcolem Jan 8, 2021
6540fa5
fix envs
tijcolem Jan 8, 2021
7a1237d
exclude this branch while testing github actions
tijcolem Jan 8, 2021
db22467
Copy the travis CI scripts and re-use for github actions
tijcolem Jan 11, 2021
5fb3ae8
Overwite ruby with custom version. Add new checks in bash
tijcolem Jan 11, 2021
c4fe2b3
run the unit tests now that setup is complete
tijcolem Jan 11, 2021
b6f2dd2
fix old travis envs
tijcolem Jan 11, 2021
d38eb00
add in step to print the logs
tijcolem Jan 11, 2021
923a70c
print logs on failure
tijcolem Jan 11, 2021
5d86045
Change the env BUILD_TYPE to GITHUB_JOB
tijcolem Jan 11, 2021
90d1e5a
Add in the linux integration steps and use the BUILD_TYPE vs the GITH…
tijcolem Jan 11, 2021
0f4f5cd
use proper syntax for assigning envs via github actions
tijcolem Jan 11, 2021
8f6617c
add docker integration tests
tijcolem Jan 12, 2021
ba61a78
add and enable docker-compose COMPOSE_INTERACTIVE_NO_CLI for CI
tijcolem Jan 12, 2021
91a298e
Add the docker-compose setup script as the built-on github action doc…
tijcolem Jan 12, 2021
82a0fcb
fix bash check for platform and build_type
tijcolem Jan 12, 2021
2596a93
add option to docker-compose to hopefully remove error on Github actions
tijcolem Jan 12, 2021
847f2aa
Add and test out macos builds. Comment out linux builds for now as th…
tijcolem Jan 13, 2021
a9d1ac9
For mac builds, use the tar.gz openstudio files as mac 10.15 is havin…
tijcolem Jan 13, 2021
d0e4b30
install core utils for macos for building nokogiri
tijcolem Jan 14, 2021
967c12d
fix the tar
tijcolem Jan 14, 2021
74b3f08
fix the curl output file name of OpenStudio
tijcolem Jan 14, 2021
f398f4b
almost there. adding suffix
tijcolem Jan 14, 2021
ec414f0
Setup is now working for osx. update the test.sh for osx
tijcolem Jan 14, 2021
4faa85b
adding integration tests for macos
tijcolem Jan 15, 2021
55f01ae
Use one job to string together the build, unit, integration and final…
tijcolem Jan 15, 2021
2c2b45c
fix yml
tijcolem Jan 15, 2021
8f930a7
change -uses to uses
tijcolem Jan 15, 2021
691c8e3
fixup mac export script
tijcolem Jan 15, 2021
052d003
export to non repo dir and then cp back to repo dir
tijcolem Jan 15, 2021
16abb9f
forgot to wrap var using bash
tijcolem Jan 15, 2021
04d8041
Adding in all final steps for docker, linux and mac builds and publis…
tijcolem Jan 19, 2021
0cce6b0
update push to docker hub
tijcolem Jan 19, 2021
d6b61be
clean up and another condition for docker hub uploads for good measur…
tijcolem Jan 19, 2021
156181f
rename the github action file and add in checks for docker upload
tijcolem Jan 19, 2021
bd5aa82
Run on push and pull_request events
tijcolem Jan 19, 2021
ed36e92
Restore the travis and appveyor CI jobs and fix export of osx on travis
tijcolem Jan 21, 2021
da15e12
remove all travis scripts and travis.yaml. Disabled project on travis…
tijcolem Jan 25, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
129 changes: 129 additions & 0 deletions .github/workflows/openstudio-server-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,129 @@
name: openstudio-server

on: [push, pull_request]

# example of how to restrict to one branch and push event
#on:
# push:
# branches:
# - test_branch

env:
USE_TESTING_TIMEOUTS: "true"
OPENSTUDIO_VERSION: 3.1.0
OPENSTUDIO_VERSION_SHA: e165090621
OPENSTUDIO_VERSION_EXT: ""
DOCKER_COMPOSE_VERSION: 1.21.1
BUNDLE_WITHOUT: native_ext


jobs:
linux-test:
runs-on: ubuntu-18.04
steps:
- name: Check out repository
uses: actions/checkout@v2
- name: setup
shell: bash
run: ./ci/github-actions/setup.sh
- name: unit-test
shell: bash
run: ./ci/github-actions/test.sh
env:
BUILD_TYPE: test
- name: integration
shell: bash
run: ./ci/github-actions/test.sh
env:
BUILD_TYPE: integration
- name: logs
shell: bash
run: ./ci/github-actions/print_logs.sh
- name: build gem package
if: ${{ success() }}
shell: bash
run: ./ci/github-actions/export_build_linux.sh
- name: upload gem package
if: ${{ success() }}
uses: actions/upload-artifact@v2
with:
name: openstudio-server-gems-linux
path: build/NREL/export/*.tar.gz
macos-test:
runs-on: macos-10.15
steps:
- name: Check out repository
uses: actions/checkout@v2
- name: setup
shell: bash
run: ./ci/github-actions/setup.sh
- name: unit-test
shell: bash
run: ./ci/github-actions/test.sh
env:
BUILD_TYPE: test
- name: integration
shell: bash
run: ./ci/github-actions/test.sh
env:
BUILD_TYPE: integration
- name: logs
shell: bash
run: ./ci/github-actions/print_logs.sh
- name: build gem package
if: ${{ success() }}
shell: bash
run: ./ci/github-actions/export_build_osx.sh
- name: upload gem package
if: ${{ success() }}
uses: actions/upload-artifact@v2
with:
name: openstudio-server-gems-darwin
path: build/NREL/export/*.tar.gz
docker:
runs-on: ubuntu-18.04
steps:
- name: Check out repository
uses: actions/checkout@v2
- name: setup
shell: bash
run: ./ci/github-actions/setup.sh
env:
BUILD_TYPE: docker
- name: docker
shell: bash
run: |
export OPENSTUDIO_TAG=develop
sed -i -E "s/.git//g" .dockerignore
docker volume create --name=osdata
docker images --all
docker --version
docker-compose --version
docker-compose -f docker-compose.test.yml pull
docker-compose -f docker-compose.test.yml build --build-arg OPENSTUDIO_VERSION=$OPENSTUDIO_TAG
docker-compose -f docker-compose.test.yml up -d
docker-compose exec -T web /usr/local/bin/run-server-tests
docker-compose stop
git checkout -- .dockerignore && git checkout -- Dockerfile
env:
CI: true
OS_SERVER_NUMBER_OF_WORKERS: 4
BUILD_TYPE: docker
COMPOSE_INTERACTIVE_NO_CLI: 1
- name: logs
shell: bash
run: ./ci/github-actions/print_logs.sh
- name: docker-upload
if: |
github.ref == 'refs/heads/master' ||
github.ref == 'refs/heads/develop' ||
github.ref == 'refs/heads/setup_github_actions'
shell: bash
run: ./docker/deployment/scripts/deploy_docker_github_actions.sh
env:
CI: true
BUILD_TYPE: docker
# On forked Pull Requests secrets will not be sent to the runner. See more info here:
# https://docs.github.com/en/actions/reference/encrypted-secrets
DOCKER_PASS: ${{ secrets.DOCKER_PASS }}
DOCKER_USER: ${{ secrets.DOCKER_USER }}
125 changes: 0 additions & 125 deletions .travis.yml

This file was deleted.

2 changes: 1 addition & 1 deletion bin/resources/local/rspec_command
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ rspec_path = File.absolute_path(File.join(__FILE__, './../../../../gems/bin/rspe
::ENV['OS_SERVER_LOG_PATH'] = File.join(output_dir, 'logs', 'rails.log')
::ENV['RAILS_ENV'] = 'local-test'

#If this is a travis build and it's BUILD_TYPE=test we want to omit the openstudio_algo_spec.rb tests
#If this is a CI build (travis, github actions, etc..) and the BUILD_TYPE=test omit the openstudio_algo_spec.rb tests
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

if ENV['BUILD_TYPE'] == 'test'
sys_call = " \"#{ruby_exe}\" \"#{bundle_path}\" exec \"#{ruby_exe}\" \"#{rspec_path}\" --tag ~depends_r --tag ~depends_gecko --tag ~depends_resque --exclude-pattern "\
"\"**/models/cluster_spec.rb\" --exclude-pattern \"**/features/openstudio_algo_spec.rb\""
Expand Down
File renamed without changes.
20 changes: 20 additions & 0 deletions ci/github-actions/export_build_linux.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# empty dir for export
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sweet! finally building package from the CI!

mkdir -p $GITHUB_WORKSPACE/build/NREL/export
mkdir -p $HOME/build/NREL/export

export PATH="/usr/loca/bin/ruby:/usr/local/openstudio-${OPENSTUDIO_VERSION}${OPENSTUDIO_VERSION_EXT}/bin:${PATH}"
export GEM_HOME="$GITHUB_WORKSPACE/gems"
export GEM_PATH="$GITHUB_WORKSPACE/gems:$GITHUB_WORKSPACE/gems/bundler/gems"
# Dir containing openstudio
export RUBYLIB="/usr/local/openstudio-${OPENSTUDIO_VERSION}${OPENSTUDIO_VERSION_EXT}/Ruby"
export OPENSTUDIO_TEST_EXE="/usr/local/openstudio-${OPENSTUDIO_VERSION}${OPENSTUDIO_VERSION_EXT}/bin/openstudio"
mongo_dir="/usr/bin"
/usr/local/ruby/bin/ruby "${GITHUB_WORKSPACE}/bin/openstudio_meta" install_gems --export="${HOME}/build/NREL/export"
oss_filename="OpenStudio-server-$(git -C "${GITHUB_WORKSPACE}" rev-parse --short=10 HEAD)-linux.tar.gz"
ls -al $HOME/build/NREL/export/
# Build the gems and output to tar gz file. You need to specify a dir outside of the repo to export or
# openstudio_meta will error. Then for sake of using relative dirs with GitHub upload-artifacts, cp the file to
# the repo working directory.
cp $HOME/build/NREL/export/$oss_filename $GITHUB_WORKSPACE/build/NREL/export/$oss_filename


17 changes: 17 additions & 0 deletions ci/github-actions/export_build_osx.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
#empty dir for export
mkdir -p $GITHUB_WORKSPACE/build/NREL/export
mkdir -p $HOME/build/NREL/export


export OS_NAME_WITH_PLUS=OpenStudio-${OPENSTUDIO_VERSION}${OPENSTUDIO_VERSION_EXT}+${OPENSTUDIO_VERSION_SHA}-Darwin
export PATH="$GITHUB_WORKSPACE/gems/bin:/usr/local/ruby/bin:$HOME/$OS_NAME_WITH_PLUS/bin:$PATH"
export RUBYLIB="$HOME/$OS_NAME_WITH_PLUS/Ruby"
export GEM_HOME="$GITHUB_WORKSPACE/gems"
export GEM_PATH="$GITHUB_WORKSPACE/gems:$GITHUB_WORKSPACE/gems/bundler/gems"
oss_filename="OpenStudio-server-$(git -C "${GITHUB_WORKSPACE}" rev-parse --short=10 HEAD)-darwin.tar.gz"

# Build the gems and output to tar gz file. You need to specify a dir outside of the repo to export or
# openstudio_meta will error. Then for sake of using relative dirs with GitHub upload-artifacts, cp the file to
# the repo working directory.
/usr/local/ruby/bin/ruby "${GITHUB_WORKSPACE}/bin/openstudio_meta" install_gems --export="${HOME}/build/NREL/export"
cp $HOME/build/NREL/export/$oss_filename $GITHUB_WORKSPACE/build/NREL/export/$oss_filename
File renamed without changes.
16 changes: 8 additions & 8 deletions ci/travis/print_logs.sh → ci/github-actions/print_logs.sh
Original file line number Diff line number Diff line change
@@ -1,42 +1,42 @@
#!/usr/bin/env bash

echo "Current directory is $(pwd)"
# echo "tree: ${TRAVIS_BUILD_DIR}/spec"
# tree "${TRAVIS_BUILD_DIR}/spec"
# echo "tree: ${GITHUB_WORKSPACE}/spec"
# tree "${GITHUB_WORKSPACE}/spec"
echo "=== PRINTING ERROR LOG REPORTS ==="

shopt -s nullglob

echo "=== PRINTING spec/files/logs/* ==="
for F in "${TRAVIS_BUILD_DIR}/spec/files/logs/"*
for F in "${GITHUB_WORKSPACE}/spec/files/logs/"*
do
echo '======================================================'
echo $F
echo '======================================================'
# Limit the rate of printing the log (with pv) to keep travis happy. https://github.com/travis-ci/travis-ci/issues/6018
# Limit the rate of printing the log (with pv)
cat $F | pv -q -L 3k
echo
done

echo "=== PRINTING /spec/unit-test/logs/* ==="
for F in "${TRAVIS_BUILD_DIR}/spec/unit-test/logs/"*
for F in "${GITHUB_WORKSPACE}/spec/unit-test/logs/"*
do
echo '======================================================'
echo $F
echo '======================================================'
# Limit the rate of printing the log (with pv) to keep travis happy. https://github.com/travis-ci/travis-ci/issues/6018
# Limit the rate of printing the log (with pv)
cat $F | pv -q -L 3k
echo
done


echo "=== PRINTING /spec/unit-test/logs/rails.log/* ==="
for F in "${TRAVIS_BUILD_DIR}/spec/unit-test/logs/rails.log/"*
for F in "${GITHUB_WORKSPACE}/spec/unit-test/logs/rails.log/"*
do
echo '======================================================'
echo $F
echo '======================================================'
# Limit the rate of printing the log (with pv) to keep travis happy. https://github.com/travis-ci/travis-ci/issues/6018
# Limit the rate of printing the log (with pv)
cat $F | pv -q -L 3k
echo
done
Loading