-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bump canarie api 1.0.0 #452
Changes from all commits
3c5896a
dc2dfd2
3bdb2e2
be5150c
4135b03
ebc9d43
e76fcd6
8002c5e
6c6f08c
bd7e9b8
b2631b2
d478e95
8ecfbf5
5a63d49
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1 +1 @@ | ||
2.2.2 2024-05-11T03:52:01Z | ||
2.3.0 2024-05-14T13:43:50Z |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,45 @@ | ||
version: "3.4" | ||
|
||
x-logging: | ||
&default-logging | ||
driver: "json-file" | ||
options: | ||
max-size: "50m" | ||
max-file: "10" | ||
|
||
services: | ||
canarie-api: | ||
image: ${CANARIE_IMAGE} | ||
container_name: canarie-api | ||
volumes: | ||
- canarie-data:/data/ | ||
- ./components/canarie-api/docker_configuration.py:/config/docker_configuration.py:ro | ||
- ./components/canarie-api/entrypoint:/entrypoint:ro | ||
environment: | ||
CANARIE_API_CONFIG_FN: /config/docker_configuration.py | ||
CANARIE_MONITORING_EXTRA_CONF_DIR: ${CANARIE_MONITORING_EXTRA_CONF_DIR} | ||
restart: always | ||
entrypoint: /entrypoint | ||
logging: *default-logging | ||
|
||
canarie-api-cron: | ||
image: ${CANARIE_IMAGE} | ||
container_name: canarie-api-cron | ||
depends_on: | ||
- canarie-api | ||
# The volumes and environment should be the same as the canarie-api container except for the proxy-logs volume | ||
volumes: | ||
- canarie-data:/data/ | ||
- ./components/canarie-api/docker_configuration.py:/config/docker_configuration.py:ro | ||
- ./components/canarie-api/entrypoint-cron:/entrypoint:ro | ||
- proxy-logs:/logs/:ro # aligns with the DATABASE["access_log"] value in docker_configuration.py.template | ||
environment: | ||
CANARIE_API_CONFIG_FN: /config/docker_configuration.py | ||
CANARIE_MONITORING_EXTRA_CONF_DIR: ${CANARIE_MONITORING_EXTRA_CONF_DIR} | ||
restart: always | ||
entrypoint: /entrypoint | ||
logging: *default-logging | ||
|
||
volumes: | ||
canarie-data: | ||
proxy-logs: | ||
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,16 +1,7 @@ | ||
#!/bin/sh -x | ||
|
||
# hack needed for Boreas and Medus only else cronjob do not fire | ||
# this hack should still work for other deployments | ||
# remove if cause problem with newer version of canarieapi | ||
touch /etc/cron.d/canarieapi-cron | ||
|
||
pip install requests-cache | ||
|
||
# chain existing cmd | ||
exec /bin/sh -c "\ | ||
env >> /etc/environment && \ | ||
cron && \ | ||
nginx && \ | ||
gunicorn -b 0.0.0.0:2000 --workers 1 --log-level=DEBUG --timeout 30 -k gevent canarieapi.wsgi \ | ||
" |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
#!/bin/sh -x | ||
|
||
# hack needed for Boreas and Medus only else cronjob do not fire | ||
# this hack should still work for other deployments | ||
# remove if cause problem with newer version of canarieapi | ||
touch /etc/cron.d/canarieapi-cron | ||
|
||
pip install requests-cache | ||
|
||
# chain existing cmd | ||
exec /bin/sh -c "\ | ||
env >> /etc/environment && \ | ||
cron -f | ||
" |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,12 +1,13 @@ | ||
version: "3.4" | ||
|
||
services: | ||
proxy: | ||
canarie-api: &canarie-volumes | ||
volumes: | ||
# NOTE: | ||
# Prefix '0_' to the mounted file name to ensure it is loaded first by 'birdhouse/components/canarie-api/docker_configuration.py' | ||
# This ensures that the alphabetical loading order it defines will first load the default configs, then will load | ||
# the Cowbird config, and then all python configuration within 'optional-components/canarie-api-full-monitoring'. | ||
# It is important to load Cowbird before, as the full monitoring need to know if 'Cowbird' service is added to the | ||
# stack in order to apply the corresponding public endpoint conditionally. | ||
- ./components/cowbird/config/proxy/canarie_api_monitoring.py:${CANARIE_MONITORING_EXTRA_CONF_DIR}/0_cowbird_config.py:ro | ||
- ./components/cowbird/config/canarie-api/canarie_api_monitoring.py:${CANARIE_MONITORING_EXTRA_CONF_DIR}/0_cowbird_config.py:ro | ||
canarie-api-cron: *canarie-volumes |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,6 @@ | ||
version: "3.4" | ||
services: | ||
proxy: | ||
canarie-api: &canarie-volumes | ||
volumes: | ||
- ./components/finch/config/canarie-api/canarie_api_monitoring.py:${CANARIE_MONITORING_EXTRA_CONF_DIR}/finch_canarie_api_monitoring.py:ro | ||
canarie-api-cron: *canarie-volumes |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,6 @@ | ||
version: "3.4" | ||
services: | ||
proxy: | ||
canarie-api: &canarie-volumes | ||
volumes: | ||
- ./components/geoserver/config/canarie-api/canarie_api_monitoring.py:${CANARIE_MONITORING_EXTRA_CONF_DIR}/geoserver_canarie_api_monitoring.py:ro | ||
canarie-api-cron: *canarie-volumes |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,6 @@ | ||
version: "3.4" | ||
services: | ||
proxy: | ||
canarie-api: &canarie-volumes | ||
volumes: | ||
- ./components/hummingbird/config/canarie-api/canarie_api_monitoring.py:${CANARIE_MONITORING_EXTRA_CONF_DIR}/hummingbird_canarie_api_monitoring.py:ro | ||
canarie-api-cron: *canarie-volumes |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,6 @@ | ||
version: "3.4" | ||
services: | ||
proxy: | ||
canarie-api: &canarie-volumes | ||
volumes: | ||
- ./components/jupyterhub/config/canarie-api/canarie_api_monitoring.py:${CANARIE_MONITORING_EXTRA_CONF_DIR}/jupyterhub_canarie_api_monitoring.py:ro | ||
canarie-api-cron: *canarie-volumes |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,6 @@ | ||
version: "3.4" | ||
services: | ||
proxy: | ||
canarie-api: &canarie-volumes | ||
volumes: | ||
- ./components/magpie/config/canarie-api/canarie_api_monitoring.py:${CANARIE_MONITORING_EXTRA_CONF_DIR}/magpie_canarie_api_monitoring.py:ro | ||
canarie-api-cron: *canarie-volumes |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,6 @@ | ||
version: "3.4" | ||
services: | ||
proxy: | ||
canarie-api: &canarie-volumes | ||
volumes: | ||
- ./components/raven/config/canarie-api/canarie_api_monitoring.py:${CANARIE_MONITORING_EXTRA_CONF_DIR}/raven_canarie_api_monitoring.py:ro | ||
canarie-api-cron: *canarie-volumes |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,6 @@ | ||
version: "3.4" | ||
services: | ||
proxy: | ||
canarie-api: &canarie-volumes | ||
volumes: | ||
- ./components/stac/config/canarie-api/canarie_api_monitoring.py:${CANARIE_MONITORING_EXTRA_CONF_DIR}/stac_canarie_api_monitoring.py:ro | ||
canarie-api-cron: *canarie-volumes |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,6 @@ | ||
version: "3.4" | ||
services: | ||
proxy: | ||
canarie-api: &canarie-volumes | ||
volumes: | ||
- ./components/thredds/config/canarie-api/canarie_api_monitoring.py:${CANARIE_MONITORING_EXTRA_CONF_DIR}/thredds_canarie_api_monitoring.py:ro | ||
canarie-api-cron: *canarie-volumes |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,6 @@ | ||
version: "3.4" | ||
services: | ||
proxy: | ||
canarie-api: &canarie-volumes | ||
volumes: | ||
- ./components/twitcher/config/canarie-api/canarie_api_monitoring.py:${CANARIE_MONITORING_EXTRA_CONF_DIR}/twitcher_canarie_api_monitoring.py:ro | ||
canarie-api-cron: *canarie-volumes | ||
Comment on lines
+3
to
+6
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Does this anchor work when all components are combined into one big compose file? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes it works, docker compose parses yaml file individually first and then combines them into one big definition. Yaml anchors are applied at the file level. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,6 @@ | ||
version: "3.4" | ||
services: | ||
proxy: | ||
canarie-api: &canarie-volumes | ||
volumes: | ||
- ./components/wps_outputs-volume/config/canarie-api/canarie_api_monitoring.py:${CANARIE_MONITORING_EXTRA_CONF_DIR}/wps_outputs-volume_canarie_api_monitoring.py:ro | ||
canarie-api-cron: *canarie-volumes |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,6 @@ | ||
version: "3.4" | ||
services: | ||
proxy: | ||
canarie-api: &canarie-volumes | ||
volumes: | ||
- ./deprecated-components/catalog/config/canarie-api/canarie_api_monitoring.py:${CANARIE_MONITORING_EXTRA_CONF_DIR}/catalog_canarie_api_monitoring.py:ro | ||
canarie-api-cron: *canarie-volumes |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,6 @@ | ||
version: "3.4" | ||
services: | ||
proxy: | ||
canarie-api: &canarie-volumes | ||
volumes: | ||
- ./deprecated-components/flyingpigeon/config/canarie-api/canarie_api_monitoring.py:${CANARIE_MONITORING_EXTRA_CONF_DIR}/flyingpigeon_canarie_api_monitoring.py:ro | ||
canarie-api-cron: *canarie-volumes |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,6 @@ | ||
version: "3.4" | ||
services: | ||
proxy: | ||
canarie-api: &canarie-volumes | ||
volumes: | ||
- ./deprecated-components/frontend/config/canarie-api/canarie_api_monitoring.py:${CANARIE_MONITORING_EXTRA_CONF_DIR}/frontend_canarie_api_monitoring.py:ro | ||
canarie-api-cron: *canarie-volumes |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,6 @@ | ||
version: "3.4" | ||
services: | ||
proxy: | ||
canarie-api: &canarie-volumes | ||
volumes: | ||
- ./deprecated-components/malleefowl/config/canarie-api/canarie_api_monitoring.py:${CANARIE_MONITORING_EXTRA_CONF_DIR}/malleefowl_canarie_api_monitoring.py:ro | ||
canarie-api-cron: *canarie-volumes |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,6 @@ | ||
version: "3.4" | ||
services: | ||
proxy: | ||
canarie-api: &canarie-volumes | ||
volumes: | ||
- ./deprecated-components/phoenix/config/canarie-api/canarie_api_monitoring.py:${CANARIE_MONITORING_EXTRA_CONF_DIR}/phoenix_canarie_api_monitoring.py:ro | ||
canarie-api-cron: *canarie-volumes |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Very much like this PR. It's much cleaner this way that the canarie-api monitoring is separate from the proxy container.
I just wonder why are you using data-volume and not direct persistence on disk. And if you really need to use data-volume, the current way we will lose the data on
./pavics-compose.sh down -v
.To avoid losing data on
./pavics-compose.sh down -v
, use external data-volume like the monitoring component.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A few reasons...
No other component is storing logs in a persistent manner. I'm not opposed to this but this could be a larger discussion about log retention that isn't related to this PR
Sure, but are you bringing the stack down with the
-v
flag often? Personally, I never ever use the-v
flag in production for that reason, and I'll delete a volume if needed manually.I actually find it annoying to have to delete the externally created jupyterhub, prometheus, grafana, etc. volumes when I'm testing and want to start with a blank slate. I personally would actually like to see the externally created volumes become non-external (or a bind-mount from disk) for this reason.
Like I mentioned above, we don't currently have a defined log retention mechanism/policy. I'd love to add one, but maybe we can defer the discussion of how to manage logs when we discuss that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the way it is defined currently is the right way.
If a platform maintainer wants to persist the logs at a specific location/drive/bind-mount, they simply need to add the relevant overrides to define the volume.
The flexibility is left up to the maintainer.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I actually do use
-v
from time to time. Otherwise, all the anonymous volume are not delete and nextup
new anonymous volumes will be created and the old anonymous volumes are only deleted withdocker volume prune --all
.Here is an example of anonymous volume:
birdhouse-deploy/birdhouse/components/finch/docker-compose-extra.yml
Line 21 in 8038856