Merge pull request #149 from bird-house/update-raven-and-jupyter-env-…

…for-raven-demo

Update Raven and Jupyter env for Raven demo

Raven release notes PR Ouranosinc/raven#374 + Ouranosinc/raven#382

Jupyter env update PR Ouranosinc/PAVICS-e2e-workflow-tests#71

Other fixes:
* Fix intermittent Jupyter spawning error by doubling various timeouts config (it's intermittent so hard to test so we are not sure which ones of timeout fixed it)
* Fix Finch and Raven "Broken pipe" error when the request size is larger than default 3mb (bumped to 100mb) (fixes Ouranosinc/raven#361 and Finch related comment bird-house/finch#98 (comment))
* Lower chance to have "Max connection" error for Finch and Raven (bump parallelprocesses from 2 to 10). In prod, the server has the CPU needed to run 10 concurrent requests if needed so this prevent users having to "wait" after each other.

birdhouse/config/finch/wps.cfg.template

@@ -2,6 +2,12 @@
 outputurl = https://${PAVICS_FQDN_PUBLIC}/wpsoutputs
 outputpath = /data/wpsoutputs
 
+# default 3mb, fix "Broken pipe" between the proxy and the wps service
+maxrequestsize = 100mb
+
+# default 2, too low for a production server with capable CPUs
+parallelprocesses = 10
+
 [logging]
 level = INFO
 database=postgresql://${POSTGRES_PAVICS_USERNAME}:${POSTGRES_PAVICS_PASSWORD}@postgres/finch

birdhouse/config/jupyterhub/jupyterhub_config.py.template

@@ -92,6 +92,15 @@ c.JupyterHub.log_level = logging.DEBUG
 
 c.Spawner.debug = True
 
+## Timeout (in seconds) to wait for spawners to initialize
+c.JupyterHub.init_spawners_timeout = 20  # default 10
+
+## Timeout (in seconds) before giving up on a spawned HTTP server
+c.Spawner.http_timeout = 60  # default 30
+
+## Timeout (in seconds) before giving up on starting of single-user server.
+c.Spawner.start_timeout = 120  # default 60
+
 ## Extra arguments to be passed to the single-user server.
 c.Spawner.args = ["--NotebookApp.terminals_enabled=False"]
 

birdhouse/config/raven/wps.cfg.template

@@ -2,6 +2,12 @@
 outputurl = https://${PAVICS_FQDN_PUBLIC}/wpsoutputs
 outputpath = /data/wpsoutputs
 
+# default 3mb, fix "Broken pipe" between the proxy and the wps service
+maxrequestsize = 100mb
+
+# default 2, too low for a production server with capable CPUs
+parallelprocesses = 10
+
 [logging]
 level = INFO
 database=postgresql://${POSTGRES_PAVICS_USERNAME}:${POSTGRES_PAVICS_PASSWORD}@postgres/raven

birdhouse/default.env

@@ -2,7 +2,7 @@
 # All env in this default.env must not depend on any env in env.local.
 
 # Jupyter single-user server images, can be overriden in env.local to have a space separated list of multiple images
-export DOCKER_NOTEBOOK_IMAGES="pavics/workflow-tests:210414"
+export DOCKER_NOTEBOOK_IMAGES="pavics/workflow-tests:210415"
 
 export FINCH_IMAGE="birdhouse/finch:version-0.7.3"
 

birdhouse/docker-compose.yml

@@ -172,7 +172,7 @@ services:
     restart: always
 
   raven:
-    image: pavics/raven:0.11.1
+    image: pavics/raven:0.12.1
     container_name: raven
     ports:
       - "8096:9099"