From 8d398e320552880d743d27ab9d1a5c7e2af3114b Mon Sep 17 00:00:00 2001 From: Long Vu Date: Wed, 3 Feb 2021 16:31:40 -0500 Subject: [PATCH 1/4] proxy: increase timeout since raven can take lots of time to process requests Fix this timeout error in some Raven notebooks: ``` HTTPError: 504 Server Error: Gateway Time-out for url: https://pavics.ouranos.ca/twitcher/ows/proxy/raven/wps ``` This is just a work-around since something is very wrong on our production host Boreas (a physical host with 128G ram and 48 logical cpu). During the test, Boreas was having this "load average: 6.35, 5.90, 4.33". For its hardware specs, it is basically idle. The increased timeout was not needed for my test VM (10G ram, 2 cpu), medus.ouranos.ca (physical host with 16G ram, 16 logical cpu) and hirondelle.crim.ca (VM with 32G ram, 8 cpu). --- birdhouse/config/proxy/nginx.conf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/birdhouse/config/proxy/nginx.conf b/birdhouse/config/proxy/nginx.conf index 0f1e42343..d44d4d8b0 100755 --- a/birdhouse/config/proxy/nginx.conf +++ b/birdhouse/config/proxy/nginx.conf @@ -31,7 +31,7 @@ http { client_body_timeout 600s; # timeout for reading a response from the proxied server - proxy_read_timeout 120s; # default 60s + proxy_read_timeout 240s; # default 60s include /etc/nginx/conf.d/*.conf; From d89ae8133274f130b6cb220f197201faa45c8a8b Mon Sep 17 00:00:00 2001 From: Long Vu Date: Wed, 3 Feb 2021 20:16:29 -0500 Subject: [PATCH 2/4] document timeout limit for WPS process --- birdhouse/README.rst | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/birdhouse/README.rst b/birdhouse/README.rst index 01da36481..a92491ecb 100644 --- a/birdhouse/README.rst +++ b/birdhouse/README.rst @@ -67,6 +67,15 @@ To launch all the containers, use the following command: If you get a ``'No applicable error code, please check error log'`` error from the WPS processes, please make sure that the WPS databases exists in the postgres instance. See `create-wps-pgsql-databases.sh `_ (:download:`download `). + +Note +---- + +* All WPS requests should be completed within ``proxy_read_timeout`` of the + Nginx proxy, see `nginx.conf`_ (:download:`download `). + Any WPS requests that will take longer should use the async mode. + + Manual steps post deployment ---------------------------- @@ -243,3 +252,6 @@ Given a version number MAJOR.MINOR.PATCH, increment the: PATCH version when we update existing components without impact on other existing components and the change is a minor change for the existing component. + + +.. _nginx.conf: ./config/proxy/nginx.conf From 2507c6f4e588ae1adf21283e8e8026a3ed0b75a2 Mon Sep 17 00:00:00 2001 From: Long Vu Date: Wed, 3 Feb 2021 20:19:32 -0500 Subject: [PATCH 3/4] docs: add automated TOC generation --- birdhouse/README.rst | 3 +++ 1 file changed, 3 insertions(+) diff --git a/birdhouse/README.rst b/birdhouse/README.rst index a92491ecb..6c4af4fe7 100644 --- a/birdhouse/README.rst +++ b/birdhouse/README.rst @@ -1,3 +1,6 @@ +.. contents:: + + Docker instructions ------------------- From 4a6b5a47328dc04e6d6ff8f2f7621b3b48b481cf Mon Sep 17 00:00:00 2001 From: Long Vu Date: Wed, 3 Feb 2021 20:32:19 -0500 Subject: [PATCH 4/4] docs: remove useless line break --- birdhouse/README.rst | 55 +++++++++++++++----------------------------- 1 file changed, 19 insertions(+), 36 deletions(-) diff --git a/birdhouse/README.rst b/birdhouse/README.rst index 6c4af4fe7..26a41dc80 100644 --- a/birdhouse/README.rst +++ b/birdhouse/README.rst @@ -6,20 +6,15 @@ Docker instructions Requirements: +* Centos 7 or Ubuntu Bionic (18.04), other distros untested. -* - Centos 7 or Ubuntu Bionic (18.04), other distros untested. - -* - Hostname of Docker host must exist on the network. Must use bridge +* Hostname of Docker host must exist on the network. Must use bridge networking if Docker host is a Virtual Machine. -* - User running ``pavics-compose.sh`` below must not be ``root`` but a regular user +* User running ``pavics-compose.sh`` below must not be ``root`` but a regular user belonging to the ``docker`` group. -* - Install latest docker-ce and docker-compose for the chosen distro (not the +* Install latest docker-ce and docker-compose for the chosen distro (not the version from the distro). To run ``docker-compose`` for PAVICS, the `pavics-compose.sh `_ (:download:`download `) wrapper script must be used. @@ -85,25 +80,18 @@ Manual steps post deployment Change geoserver default admin password ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -* - Go to +* Go to ``https:///geoserver/web/wicket/bookmarkable/org.geoserver.security.web.UserGroupRoleServicesPage`` (Security -> Users, Groups, and Roles) -* - Login using the default username ``admin`` and default password ``geoserver``. +* Login using the default username ``admin`` and default password ``geoserver``. -* - Click on tab "Users/Groups". +* Click on tab "Users/Groups". -* - Click on user "admin". +* Click on user "admin". -* - Change the password. +* Change the password. -* - Click "Save". +* Click "Save". Create public demo user in Magpie for JupyterHub login ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -123,23 +111,19 @@ instructions below. Manual instructions: -* - Go to +* Go to ``https:///magpie/ui/login`` and login with the ``admin`` user. The password should be in ``env.local``. -* - Then go to ``https:///magpie/ui/users/add``. +* Then go to ``https:///magpie/ui/users/add``. -* - Fill in: +* Fill in: * User name: * Email: < anything is fine > * Password: < you decide > * User group: ``anonymous`` -* - Click "Add User". +* Click "Add User". Optional: prepare instance to run automated end-to-end test suite ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -166,6 +150,7 @@ The canarie monitoring link instance is ready to run the automated end-to-end test suite. That link should return the HTTP response code ``200``. + Vagrant instructions -------------------- @@ -227,6 +212,7 @@ Starting and managing the lifecycle of the VM: # not needed normally during tight development loop vagrant provision + Tagging policy -------------- @@ -239,20 +225,17 @@ together. So we need a slight modification to the definition of the standard. Given a version number MAJOR.MINOR.PATCH, increment the: -#. - MAJOR version when the API or user facing UI changes that requires +#. MAJOR version when the API or user facing UI changes that requires significant documentation update and/or re-training of the users. Also valid when a big milestone has been reached (ex: DACCS is released). -#. - MINOR version when we add new components or update existing components +#. MINOR version when we add new components or update existing components that also require change to other existing components (ex: new Magpie that also force Twitcher and/or Frontend update) or the change to the existing component is a major one (ex: major refactoring of Twitcher, big merge with corresponding upstream component from birdhouse project). -#. - PATCH version when we update existing components without impact on other +#. PATCH version when we update existing components without impact on other existing components and the change is a minor change for the existing component.