Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

2.7.1 unexpected return #443

Merged
merged 6 commits into from
Dec 5, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ after_failure: ./ci/travis/print_logs.sh

# Services for linux -- all instances
services:
- mongod
- mongodb
- redis-server
- docker

Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,11 +81,11 @@ docker-compose rm -f
docker volume rm osdata dbdata
docker volume create --name=osdata
docker volume create --name=dbdata
docker-compose up
OS_SERVER_NUMBER_OF_WORKERS=N docker-compose up
docker-compose service scale worker=N

# Or one line
docker-compose rm -f && docker-compose build && docker volume rm osdata dbdata && docker volume create --name=osdata && docker volume create --name=dbdata && OS_SERVER_NUMBER_OF_WORKERS=5 docker-compose up && docker-compose service scale worker=N
docker-compose rm -f && docker-compose build && docker volume rm osdata dbdata && docker volume create --name=osdata && docker volume create --name=dbdata && OS_SERVER_NUMBER_OF_WORKERS=N docker-compose up && docker-compose service scale worker=N
```

Congratulations! Visit `http://localhost:8080` to see the OpenStudio Server Management Console.
Expand Down
14 changes: 10 additions & 4 deletions ci/travis/setup.sh
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,18 @@ if [ "${BUILD_TYPE}" == "docker" ]; then
else
if [ "${TRAVIS_OS_NAME}" == "osx" ]; then
brew update > /Users/travis/build/NREL/OpenStudio-server/spec/files/logs/brew-update.log
# AP: do we need mongo install here ? seems to be handled by service defined in travis yml
brew install mongodb@3.4 pv tree
ln -s /usr/local/opt/mongodb@3.4/bin/* /usr/local/bin
# AP: do we need mongo install here ? seems to be handled by service defined in travis yml.
# NL: Services are not handled in osx
brew install pv tree

# Install mongodb from a download. Brew is hanging and requires building mongo. This also speeds up the builds.
curl -SLO https://fastdl.mongodb.org/osx/mongodb-osx-ssl-x86_64-3.4.18.tgz
tar xvzf mongodb-osx-ssl-x86_64-3.4.18.tgz
cp mongodb-osx-x86_64-3.4.18/bin/* /usr/local/bin/

# Install openstudio -- Use the install script that is in this repo now, the one on OpenStudio/develop has changed
curl -SLO --insecure https://s3.amazonaws.com/openstudio-builds/$OPENSTUDIO_VERSION/OpenStudio-$OPENSTUDIO_VERSION.$OPENSTUDIO_VERSION_SHA-Darwin.zip
unzip OpenStudio-$OPENSTUDIO_VERSION.$OPENSTUDIO_VERSION_SHA-Darwin.zip
# Use the install script that is in this repo now, the one on OpenStudio/develop has changed
sed -i -e "s|REPLACEME|$HOME/openstudio|" ci/travis/install-mac.qs
rm -rf $HOME/openstudio
# Will install into $HOME/openstudio and RUBYLIB will be $HOME/openstudio/Ruby
Expand Down
3 changes: 2 additions & 1 deletion docker/server/start-server.sh
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,8 @@ cd /opt/openstudio/server && bundle exec rake db:mongoid:create_indexes
# https://github.com/NREL/OpenStudio-server/issues/348
ruby /usr/local/lib/memfix-controller.rb start

# AP hack for now to ensure both resque and nginx/rails are running on web
# AP hack for now to ensure both resque and nginx/rails are running on web.
# Note that this will only run the analysis background queue as it is set in the docker-compose env file.
bundle exec rake environment resque:work &

/opt/nginx/sbin/nginx
3 changes: 2 additions & 1 deletion docker/server/start-workers.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,5 @@ echo "Waiting for Mongo to start"
echo "Waiting for Redis to start"
/usr/local/bin/wait-for-it --strict -t 0 queue:6379

cd /opt/openstudio/server && bundle exec rake environment resque:workers
# Only start a single worker when calling this script (do not use resque:workers).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if this is appropriate, we should rename the script (which is start-workers.sh). but is it used for for worker node setup as well as web-background?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, start-workers.sh only starts a single worker instance and registers it with redis. This was probably causing an issue because in docker-compose.yml (for local use) had COUNT=1 set then called resque:workers. Let me know if you want me to rename before merging.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

preference for renaming but not required.

cd /opt/openstudio/server && bundle exec rake environment resque:work
2 changes: 1 addition & 1 deletion local_setup_scripts/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ services:
environment:
- QUEUES=simulations
- COUNT=1
command: bundle exec rake environment resque:workers
command: bundle exec rake environment resque:work
volumes:
- /mnt/openstudio
depends_on:
Expand Down
51 changes: 10 additions & 41 deletions server/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,15 @@ COUNT=4 QUEUES=simulations bundle exec rake environment resque:workers
QUEUES=analysis_wrappers bundle exec rake environment resque:work
```

### Running Simulations for development in the foreground

OpenStudio Server now runs the simulations through the OpenStudio CLI. In order for this to work on local development,
the gems need to be installed in the OpenStudio installation directory. Run the following for OSX:

```bash
cd /Applications/OpenStudio-x.y.z/Ruby
bundle install --path ./gems
```

## Starting Rserve for development in the foreground

Expand All @@ -69,21 +78,9 @@ gem install nokogiri -- --use-system-libraries --with-xml2-include=/usr/include/
gem install libxml-ruby -- --with-xml2-include=/usr/include/libxml2 --with-xml2-lib=/usr/lib/
```

# Testing using Docker-compose

```
docker volume create --name=osdata
docker-compose -f docker-compose.test.yml build
docker-compose -f docker-compose.test.yml up

# One line
docker-compose rm -f && docker-compose -f docker-compose.test.yml build && docker-compose -f docker-compose.test.yml up
```

# Testing

If running the tests on a local machine, then make sure to install
geckodriver to run the webpage.
If running the tests on a local machine, then make sure to install geckodriver to run the webpage.

```
bundle install
Expand All @@ -108,31 +105,3 @@ publishing scripts are run on TravisCI. These are only run the develop and maste
* Write tests for each analysis (expand existing SPEA test)
* Add CLI path to config.yml
* Add tests for embedded files on analysis model. Test result of R code that pushed to analysis model (i.e. best_point.json)

# AWS Elastic Container Service

It is possible to use Amazon's Elastic Container Service but it will
be limited to only running one machine and it is not possible to add
more worker nodes. The preferred approach is to use docker-machine and
docker-swarm or to custom deploy.

```
ecs-cli configure --cluster openstudio
ecs-cli up --keypair <key-pair-name> --capability-iam --size 1 --instance-type t2.medium --port 8080
ecs-cli compose -f docker-compose.deploy.yml up
# Get the IP address from the console `ecs-cli ps`
ecs-cli down --force
```

# Docker-Machine

## AWS

The easiest approach to using Docker-Machine is to export your keys as
environment variables.

```
export $AWS_ACCESS_KEY_ID=<your-access-key>
export $AWS_SECRET_ACCESS_KEY=<your-secret-access-key>
```

6 changes: 6 additions & 0 deletions server/config/environments/development.rb
Original file line number Diff line number Diff line change
Expand Up @@ -76,4 +76,10 @@

# Use a different logger for distributed setups.
# config.logger = ActiveSupport::TaggedLogging.new(SyslogLogger.new)

# If you need to use custom gems in your development environment, then uncomment the line below. You will
# also need to run `bundle install` in your local openstudio Gemfile path. For example:
# cd /Applications/OpenStudio-x.y.z/Ruby
# bundle install --path ./gems
# config.os_gemfile_path = File.expand_path(File.join(File.dirname(`which openstudio`), '../Ruby'))
end
1 change: 0 additions & 1 deletion server/config/environments/docker-dev.rb
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,5 @@
# config.logger = ActiveSupport::TaggedLogging.new(SyslogLogger.new)

# detect os version build path to gemfile. this is passed to cli as --bundle
os_version = `openstudio openstudio_version`
config.os_gemfile_path = "/var/oscli"
end
1 change: 0 additions & 1 deletion server/config/environments/docker-test.rb
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,5 @@
# config.logger = ActiveSupport::TaggedLogging.new(SyslogLogger.new)

# detect os version build path to gemfile. this is passed to cli as --bundle
os_version = `openstudio openstudio_version`
config.os_gemfile_path = "/var/oscli"
end
1 change: 0 additions & 1 deletion server/config/environments/docker.rb
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,5 @@
config.active_support.deprecation = :notify

# detect os version build path to gemfile. this is passed to cli as --bundle
os_version = `openstudio openstudio_version`
config.os_gemfile_path = "/var/oscli"
end