Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

2.7.1 unexpected return #443

Merged
merged 6 commits into from
Dec 5, 2018
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ gemfile:

# Set the default scripts that are used for the majority of the tests in the matrix
before_install:
- ./ci/travis/setup.sh
- travis_wait ./ci/travis/setup.sh
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

before_script:
- sudo Xvfb :99 -ac -screen 0 1024x768x8 &
- sleep 3
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,11 +81,11 @@ docker-compose rm -f
docker volume rm osdata dbdata
docker volume create --name=osdata
docker volume create --name=dbdata
docker-compose up
OS_SERVER_NUMBER_OF_WORKERS=N docker-compose up
docker-compose service scale worker=N

# Or one line
docker-compose rm -f && docker-compose build && docker volume rm osdata dbdata && docker volume create --name=osdata && docker volume create --name=dbdata && OS_SERVER_NUMBER_OF_WORKERS=5 docker-compose up && docker-compose service scale worker=N
docker-compose rm -f && docker-compose build && docker volume rm osdata dbdata && docker volume create --name=osdata && docker volume create --name=dbdata && OS_SERVER_NUMBER_OF_WORKERS=N docker-compose up && docker-compose service scale worker=N
```

Congratulations! Visit `http://localhost:8080` to see the OpenStudio Server Management Console.
Expand Down
3 changes: 2 additions & 1 deletion docker/server/start-server.sh
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,8 @@ cd /opt/openstudio/server && bundle exec rake db:mongoid:create_indexes
# https://github.com/NREL/OpenStudio-server/issues/348
ruby /usr/local/lib/memfix-controller.rb start

# AP hack for now to ensure both resque and nginx/rails are running on web
# AP hack for now to ensure both resque and nginx/rails are running on web.
# Note that this will only run the analysis background queue as it is set in the docker-compose env file.
bundle exec rake environment resque:work &

/opt/nginx/sbin/nginx
3 changes: 2 additions & 1 deletion docker/server/start-workers.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,5 @@ echo "Waiting for Mongo to start"
echo "Waiting for Redis to start"
/usr/local/bin/wait-for-it --strict -t 0 queue:6379

cd /opt/openstudio/server && bundle exec rake environment resque:workers
# Only start a single worker when calling this script (do not use resque:workers).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if this is appropriate, we should rename the script (which is start-workers.sh). but is it used for for worker node setup as well as web-background?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, start-workers.sh only starts a single worker instance and registers it with redis. This was probably causing an issue because in docker-compose.yml (for local use) had COUNT=1 set then called resque:workers. Let me know if you want me to rename before merging.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

preference for renaming but not required.

cd /opt/openstudio/server && bundle exec rake environment resque:work
2 changes: 1 addition & 1 deletion local_setup_scripts/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ services:
environment:
- QUEUES=simulations
- COUNT=1
command: bundle exec rake environment resque:workers
command: bundle exec rake environment resque:work
volumes:
- /mnt/openstudio
depends_on:
Expand Down
51 changes: 10 additions & 41 deletions server/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,15 @@ COUNT=4 QUEUES=simulations bundle exec rake environment resque:workers
QUEUES=analysis_wrappers bundle exec rake environment resque:work
```

### Running Simulations for development in the foreground

OpenStudio Server now runs the simulations through the OpenStudio CLI. In order for this to work on local development,
the gems need to be installed in the OpenStudio installation directory. Run the following for OSX:

```bash
cd /Applications/OpenStudio-x.y.z/Ruby
bundle install --path ./gems
```

## Starting Rserve for development in the foreground

Expand All @@ -69,21 +78,9 @@ gem install nokogiri -- --use-system-libraries --with-xml2-include=/usr/include/
gem install libxml-ruby -- --with-xml2-include=/usr/include/libxml2 --with-xml2-lib=/usr/lib/
```

# Testing using Docker-compose

```
docker volume create --name=osdata
docker-compose -f docker-compose.test.yml build
docker-compose -f docker-compose.test.yml up

# One line
docker-compose rm -f && docker-compose -f docker-compose.test.yml build && docker-compose -f docker-compose.test.yml up
```

# Testing

If running the tests on a local machine, then make sure to install
geckodriver to run the webpage.
If running the tests on a local machine, then make sure to install geckodriver to run the webpage.

```
bundle install
Expand All @@ -108,31 +105,3 @@ publishing scripts are run on TravisCI. These are only run the develop and maste
* Write tests for each analysis (expand existing SPEA test)
* Add CLI path to config.yml
* Add tests for embedded files on analysis model. Test result of R code that pushed to analysis model (i.e. best_point.json)

# AWS Elastic Container Service

It is possible to use Amazon's Elastic Container Service but it will
be limited to only running one machine and it is not possible to add
more worker nodes. The preferred approach is to use docker-machine and
docker-swarm or to custom deploy.

```
ecs-cli configure --cluster openstudio
ecs-cli up --keypair <key-pair-name> --capability-iam --size 1 --instance-type t2.medium --port 8080
ecs-cli compose -f docker-compose.deploy.yml up
# Get the IP address from the console `ecs-cli ps`
ecs-cli down --force
```

# Docker-Machine

## AWS

The easiest approach to using Docker-Machine is to export your keys as
environment variables.

```
export $AWS_ACCESS_KEY_ID=<your-access-key>
export $AWS_SECRET_ACCESS_KEY=<your-secret-access-key>
```

3 changes: 3 additions & 0 deletions server/config/environments/development.rb
Original file line number Diff line number Diff line change
Expand Up @@ -76,4 +76,7 @@

# Use a different logger for distributed setups.
# config.logger = ActiveSupport::TaggedLogging.new(SyslogLogger.new)

# TODO: Remove assumption that we are developing on a machine that has which installed! Make cross platform...
config.os_gemfile_path = File.expand_path(File.join(File.dirname(`which openstudio`), '../Ruby'))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think this will fail on windows. also, it will trigger the --bundle options to be passed with openstudio cli call, which should only happen if the bundle has been installed locally and we do not want to use what comes packaged with openstudio

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will fail on windows, but I'm assuming that no one is developing on windows (or running the development.rb environment. That was the essence of my TODO comment.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess i'm more concerned with introduction of requirement that bundle install be run locally for the openstudio install. what is this getting us?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i have approved this pr, but for the record i think that at a minimum a very explicit comment about the bundle install requirement will help us in the future. or commit with this commented out and the explanation to uncomment if you want to run with a locally installed openstudio bundle.

end
1 change: 0 additions & 1 deletion server/config/environments/docker-dev.rb
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,5 @@
# config.logger = ActiveSupport::TaggedLogging.new(SyslogLogger.new)

# detect os version build path to gemfile. this is passed to cli as --bundle
os_version = `openstudio openstudio_version`
config.os_gemfile_path = "/var/oscli"
end
1 change: 0 additions & 1 deletion server/config/environments/docker-test.rb
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,5 @@
# config.logger = ActiveSupport::TaggedLogging.new(SyslogLogger.new)

# detect os version build path to gemfile. this is passed to cli as --bundle
os_version = `openstudio openstudio_version`
config.os_gemfile_path = "/var/oscli"
end
1 change: 0 additions & 1 deletion server/config/environments/docker.rb
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,5 @@
config.active_support.deprecation = :notify

# detect os version build path to gemfile. this is passed to cli as --bundle
os_version = `openstudio openstudio_version`
config.os_gemfile_path = "/var/oscli"
end