Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add admin dashboard #108

Merged
merged 26 commits into from
Feb 28, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
0a13697
Add homepage service
hmakelin Feb 24, 2024
f05e43b
Improve home page layout
hmakelin Feb 24, 2024
bd5d1a4
Simplify service descriptions
hmakelin Feb 24, 2024
a57f6ca
Remove homepage to maps-volume connection since it does not exist (yet?)
hmakelin Feb 24, 2024
134e4c1
Provide more context on different services on home page
hmakelin Feb 24, 2024
6dcf09c
Update system external interfaces diagram to include admin services
hmakelin Feb 24, 2024
5f0a06f
Add TODO to use a more secure solution for monitorin docker services
hmakelin Feb 24, 2024
e472f40
Expose gisnav ROS launch param files to file manager for easier confi…
hmakelin Feb 25, 2024
aadba80
Add notice that static docs integration does not exist yet between gi…
hmakelin Feb 25, 2024
7388729
WIP: Add system monitoring service
hmakelin Feb 25, 2024
4ed14e5
Move system monitoring info to header to declutter dashboard
hmakelin Feb 25, 2024
8b3d500
Update interfaces diagram
hmakelin Feb 25, 2024
6a9a087
Update monitoring service label in diagram
hmakelin Feb 25, 2024
496df5b
Add missing gscam volume and restructure interfaces diagram
hmakelin Feb 25, 2024
73e48c9
Move interfaces diagram closer to code
hmakelin Feb 25, 2024
cbb2464
Update monitoring service label in diagram for second time [skip ci]
hmakelin Feb 25, 2024
3764dcd
Fix gscam volume name in diagram [skip ci]
hmakelin Feb 25, 2024
c611381
Fix ArduPilot capitalization [skip ci]
hmakelin Feb 25, 2024
fdd0189
Make diagram leaner [skip ci]
hmakelin Feb 25, 2024
4c3fbcb
Minor diagram layout improvement [skip ci]
hmakelin Feb 25, 2024
22f8197
Make gisnav entrypoint script work when not built with Docker Compose…
hmakelin Feb 25, 2024
32ea777
Fix gisnav-volume mapping
hmakelin Feb 26, 2024
f224946
Ensure fileserver can edit files on shared volume [skip ci]
hmakelin Feb 26, 2024
da178fb
Do not attempt to move gisnav config files if they already exist on t…
hmakelin Feb 27, 2024
4e79436
Fix docker socket exposure
hmakelin Feb 28, 2024
2ab198e
Fix entrypoint file move exception handling
hmakelin Feb 28, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
100 changes: 99 additions & 1 deletion docker/DOCKER.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
Docker build contexts
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

In the ``docker/`` folder you can find a collection of directories roughly
corresponding to :term:`Docker` build contexts, and a number of
Expand All @@ -8,3 +8,101 @@ case of multi-stage builds multiple service images can be created from the same
build context. The ``docker/Makefile`` may define additional phony targets for
commonly needed tasks such as exposing the X server to containers that have a
:term:`GUI` component.

External interfaces
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The |Docker Compose file|_ defines all services used to support GISNav deployments.
The diagram below describes the system architecture through the external
interfaces and Docker bridge networks shared between the Docker Compose services.

The Docker bridge networks have in-built DNS which means the container names
resolve to their respective IP addresses. The container name will equal the
service name prefixed with ``gisnav-`` and suffixed with ``-1``. So for example
deploying the ``mapserver`` service using the Compose file will start a
Docker container with the hostname ``gisnav-mapserver-1``.

.. todo::
* Split mavlink network into mavlink and ROS networks. For ROS the intention
is to use the shared memory device instead of serializing and going through
the network stack since we will be passing a lot of images around.
* Build and expose static docs to home page - possibly no need for a server

.. note::
The application services have access to both ``gis`` and ``mavlink`` networks.

.. mermaid::

graph TD
subgraph mavlink ["mavlink"]
mavlink_qgc[qgc]
subgraph simulation ["Simulation Services"]
simulation_px4[px4]
simulation_ardupilot[ardupilot]
end
subgraph middleware ["Middleware Services"]
middleware_mavros[mavros]
middleware_micro_ros_agent[micro-ros-agent]
middleware_gscam[gscam]
end
end

subgraph gis_mavlink ["gis & mavlink"]
subgraph application ["Application Services"]
application_gisnav[gisnav]
application_autoheal[autoheal]
end
end
subgraph gis ["gis"]
subgraph gis_services ["GIS Services"]
gis_mapserver[mapserver]
gis_qgis[qgis]
end
subgraph data_services ["Data Services"]
gis_postgres[postgres]
end
end

subgraph admin ["admin"]
homepage[homepage]
fileserver[fileserver]
end

subgraph volumes ["User managed\nshared volumes"]
gscam_volume[gscam-volume]
gis_maps_volume[maps-volume]
application_gisnav_volume[gisnav-volume]
end
application_docs_volume[docs-volume]

mavlink_qgc -->|14550/udp\nMAVLink| simulation_px4
simulation_px4 -->|14540/udp\nMAVLink| middleware_mavros
simulation_px4 -->|8888/udp\nDDS-XRCE | middleware_micro_ros_agent
simulation_px4 -->|5600/udp\nRTP H.264 Video| middleware_gscam
middleware_mavros -->|/dev/shm\nROS 2 Fast DDS| application_gisnav
middleware_micro_ros_agent -->|/dev/shm\nROS 2 Fast DDS| application_gisnav
middleware_gscam -->|/dev/shm\nROS 2 Fast DDS| application_gisnav
application_gisnav -->|5432/tcp| gis_postgres

application_gisnav -->|80/tcp\nHTTP WMS| gis_mapserver
gis_mapserver -->|80/tcp\nHTTP WMS| gis_qgis
gis_qgis -->|5432/tcp| gis_postgres
gis_mapserver ---|/etc/mapserver| gis_maps_volume
application_gisnav_volume ---|/etc/gisnav| application_gisnav
application_docs_volume ---|/path/to/built/docs| application_gisnav
homepage ---|3000/tcp| fileserver
fileserver ---|"/var/www/filegator/"| volumes
gscam_volume ---|/etc/gscam| middleware_gscam

application_docs_volume ---|/path/to/docs:ro| homepage

subgraph host ["host"]
monitoring["monitoring"]
docker_host["docker host"]
end

monitoring ---|61208/tcp| homepage
docker_host ---|/var/run/docker.sock| monitoring

classDef network fill:transparent,stroke-dasharray:5 5;
class mavlink,gis,gis_mavlink,admin,admin_gis,host network
3 changes: 3 additions & 0 deletions docker/apache/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
FROM ubuntu:jammy AS apache

LABEL maintainer="Harri Makelin <hmakelin@protonmail.com>"

# Update apt repository and install necessary packages
RUN apt-get update && apt-get install -y \
cgi-mapserver \
Expand Down Expand Up @@ -125,6 +127,7 @@ RUN mkdir -p /var/www/ && \
cp configuration_sample.php configuration.php && \
sed -i "s/'app_name' => 'FileGator'/'app_name' => 'GISNav'/g" configuration.php && \
sed -i "s|'logo' => 'https://filegator.io/filegator_logo.svg'|'logo' => 'img/logo-no-background.svg'|g" configuration.php && \
sed -i "/'editable' => \[/ s/]/, '.yml', '.yaml'&/" configuration.php && \
sed -i 's/100 \* 1024 \* 1024/2 \* 1024 \* 1024 \* 1024/g' configuration.php && \
mkdir -p dist/img && \
cp /logo-no-background.svg dist/img && \
Expand Down
4 changes: 4 additions & 0 deletions docker/apache/fileserver/entrypoint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,8 @@ rm -f /var/run/apache2/cgisock.*

apache2ctl -D FOREGROUND &

# Make sure we can edit files on mapped volumes
chown -R www-data:www-data /var/www/filegator/repository
chmod -R 775 /var/www/filegator/repository

exec "$@"
6 changes: 4 additions & 2 deletions docker/apache/mapserver/entrypoint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,8 @@ process_directory_change() {
fi
fi

sleep 0.5

if [[ "$BASENAME" =~ \.(tif|tiff|jp2|ecw|img)$ ]]; then
echo "Raster file detected, regenerating VRT."
cd /etc/mapserver && gdalbuildvrt "$VRT_FILE" "$DIR_PATH"/*.tif "$DIR_PATH"/*.tiff "$DIR_PATH"/*.jp2 "$DIR_PATH"/*.ecw "$DIR_PATH"/*.img
Expand All @@ -69,7 +71,7 @@ done &

# Move demo imagery and DEM over to shared volume if still on image
# This should trigger the inotify script
mv /etc/mapserver/$NAIP_ZIP_FILENAME $IMAGERY_DIR | echo "NAIP imagery not found on container - likely already moved to shared volume"
mv /etc/mapserver/$DEM_FILENAME $DEM_DIR | echo "USGS DEM not found on container - likely already moved to shared volume"
mv /etc/mapserver/$NAIP_ZIP_FILENAME $IMAGERY_DIR || echo "NAIP imagery not found on container - likely already moved to shared volume"
mv /etc/mapserver/$DEM_FILENAME $DEM_DIR || echo "USGS DEM not found on container - likely already moved to shared volume"

exec "$@"
Loading
Loading