Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make downloading files back to server optional #571

Merged
merged 1 commit into from
Jun 22, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Empty file modified local_setup_scripts/docker_logs.sh
100644 → 100755
Empty file.
Empty file modified local_setup_scripts/docker_logs2.sh
100644 → 100755
Empty file.
Empty file modified local_setup_scripts/nuke.sh
100644 → 100755
Empty file.
12 changes: 6 additions & 6 deletions local_setup_scripts/rebuild_sr.sh
100644 → 100755
Original file line number Diff line number Diff line change
Expand Up @@ -6,23 +6,23 @@ docker volume rm osdata -f || true
docker volume rm dbdata -f || true

while [ $(docker ps -q | wc -l) != 1 ]; do sleep 5; done
docker image rm 127.0.0.1:5000/openstudio-server -f
docker build . -t="127.0.0.1:5000/openstudio-server" --build-arg OPENSTUDIO_VERSION=3.0.0-beta
#docker image rm 127.0.0.1:5000/openstudio-server -f
brianlball marked this conversation as resolved.
Show resolved Hide resolved
docker build . -t="127.0.0.1:5000/openstudio-server" --build-arg OPENSTUDIO_VERSION=3.0.1-rc1
docker push 127.0.0.1:5000/openstudio-server
cd docker/R/
docker image rm 127.0.0.1:5000/openstudio-rserve -f
#docker image rm 127.0.0.1:5000/openstudio-rserve -f
docker build . -t="127.0.0.1:5000/openstudio-rserve"
docker push 127.0.0.1:5000/openstudio-rserve
docker pull mongo:3.4.10
docker tag mongo 127.0.0.1:5000/mongo
docker tag mongo:3.4.10 127.0.0.1:5000/mongo
docker push 127.0.0.1:5000/mongo
docker image rm mongo || true
docker image rm mongo:3.4.10 || true
docker pull redis:4.0.6
docker tag redis:4.0.6 127.0.0.1:5000/redis
docker push 127.0.0.1:5000/redis
cd ../../local_setup_scripts
docker stack deploy osserver --compose-file=./docker-compose.yml
while ( nc -zv 127.0.0.1 80 3>&1 1>&2- 2>&3- ) | awk -F ":" '$3 != " Connection refused" {exit 1}'; do sleep 5; done
docker service scale osserver_worker=42
docker service scale osserver_worker=1
echo 'osserver stack rebuilt and redeployed'

Empty file modified local_setup_scripts/redeploy.sh
100644 → 100755
Empty file.
40 changes: 27 additions & 13 deletions server/app/jobs/dj_jobs/run_simulate_data_point.rb
Original file line number Diff line number Diff line change
Expand Up @@ -284,21 +284,35 @@ def perform

# Post the reports back to the server
uploads_successful = []

Dir["#{simulation_dir}/reports/*.{html,json,csv}"].each { |rep| uploads_successful << upload_file(rep, 'Report') }

if @data_point.analysis.download_reports
brianlball marked this conversation as resolved.
Show resolved Hide resolved
@sim_logger.info "downloading reports/*.{html,json,csv}"
Dir["#{simulation_dir}/reports/*.{html,json,csv}"].each { |rep| uploads_successful << upload_file(rep, 'Report') }
else
@sim_logger.info "NOT downloading /reports/*.{html,json,csv} since download_reports value is: #{@data_point.analysis.download_reports}"
end
report_file = "#{run_dir}/objectives.json"
uploads_successful << upload_file(report_file, 'Report', 'objectives', 'application/json') if File.exist?(report_file)

report_file = "#{simulation_dir}/out.osw"
uploads_successful << upload_file(report_file, 'Report', 'Final OSW File', 'application/json') if File.exist?(report_file)

report_file = "#{simulation_dir}/in.osm"
uploads_successful << upload_file(report_file, 'OpenStudio Model', 'model', 'application/osm') if File.exist?(report_file)

report_file = "#{run_dir}/data_point.zip"
uploads_successful << upload_file(report_file, 'Data Point', 'Zip File', 'application/zip') if File.exist?(report_file)

if @data_point.analysis.download_osw
@sim_logger.info "downloading out.OSW"
report_file = "#{simulation_dir}/out.osw"
uploads_successful << upload_file(report_file, 'Report', 'Final OSW File', 'application/json') if File.exist?(report_file)
else
@sim_logger.info "NOT downloading out.OSW since download_osw value is: #{@data_point.analysis.download_osw}"
end
if @data_point.analysis.download_osm
@sim_logger.info "downloading in.OSM"
report_file = "#{simulation_dir}/in.osm"
uploads_successful << upload_file(report_file, 'OpenStudio Model', 'model', 'application/osm') if File.exist?(report_file)
else
@sim_logger.info "NOT downloading in.OSM since download_osm value is: #{@data_point.analysis.download_osm}"
end
if @data_point.analysis.download_zip
@sim_logger.info "downloading datapoint.ZIP"
report_file = "#{run_dir}/data_point.zip"
uploads_successful << upload_file(report_file, 'Data Point', 'Zip File', 'application/zip') if File.exist?(report_file)
else
@sim_logger.info "NOT downloading datapoint.zip since download_zip value is: #{@data_point.analysis.download_zip}"
end
run_result = :errored unless uploads_successful.all?
end

Expand Down
6 changes: 5 additions & 1 deletion server/app/models/analysis.rb
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,11 @@ class Analysis
field :initialize_worker_timeout, type: Integer, default: 28800 # set default to 8 hrs
field :upload_results_timeout, type: Integer, default: 28800 # set default to 8 hrs
field :run_workflow_timeout, type: Integer, default: 28800 # set default to 8 hrs

field :download_zip, type: Boolean, default: true
brianlball marked this conversation as resolved.
Show resolved Hide resolved
field :download_osm, type: Boolean, default: true
field :download_osw, type: Boolean, default: true
field :download_reports, type: Boolean, default: true

# Hash of the jobs to run for the analysis
# field :jobs, type: Array, default: [] # very specific format
# move the results into the jobs array
Expand Down