Ux for new developers (#1071)

.gitignore

@@ -142,3 +142,6 @@ cython_debug/
 .vscode/
 .idea/
 .vim/
+
+# Development Docker Compose file
+docker-compose.dev.yaml

README.md

@@ -16,6 +16,8 @@ and [building easy-to-read messages ready to be sent to the scanned organization
 
 If you want to use additional modules that weren't included here due to non-BSD-compatible licenses, browse to the [Artemis-modules-extra](https://github.com/CERT-Polska/Artemis-modules-extra) repository.
 
+If you want to modify/develop Artemis, read **[Development](#development)** first.
+
 **Artemis is experimental software, under active development - use at your own risk.**
 
 To chat about Artemis, join the Discord server:
@@ -43,9 +45,15 @@ The possibility to automatically prepare such reports enabled us to notify entit
 ## Screenshots
 ![Artemis - scan](.github/screenshots/scan.png)
 
+<a name="development"></a>
 ## Development
-If you want to develop Artemis, remember to change the [Docker Compose file](https://github.com/CERT-Polska/Artemis/blob/main/docker-compose.yaml#L7) so that
-the Artemis image is built locally (from the code you are changing) not downloaded from Docker Hub.
+To start a locally modified version of Artemis, run:
+```commandline
+ cp env.example .env  # after doing that, configure the settings by changing the user-agent and any other settings you want to change
+ ./scripts/start_dev
+```
+The Artemis image is then built locally (from the code you are developing) not downloaded from Docker Hub.
+For `web`, you will also be able to see the results of code modifications on the page without reloading the entire container.
 
 ### Tests
 To run the tests, use:

docker-compose.yaml

@@ -1,12 +1,6 @@
 x-artemis-build-or-image:
   &artemis-build-or-image
   image: certpl/artemis:latest
-  # Replace with the following to use local Artemis source code to build the container,
-  # not one downloaded from Docker Hub:
-  #
-  # build:
-  #    context: .
-  #    dockerfile: docker/Dockerfile
 
 services:
   redis:
@@ -49,7 +43,7 @@ services:
   # This is a legacy entry to facillitate migration - MongoDB is not used by Artemis anymore.
   db:
     # Even in the most quiet configuration, MongoDB produces a significant amount of logs.
-    # Therefore mongoDB logs are disabled by default (although they can be enabled).
+    # Therefore, mongoDB logs are disabled by default (although they can be enabled).
     command: ["bash", "-c", "/usr/local/bin/docker-entrypoint.sh mongod --wiredTigerCacheSizeGB 2 --auth --bind_ip_all 2>&1 >/dev/null"]
     environment:
      - MONGO_INITDB_ROOT_USERNAME=root

scripts/create_development_docker_compose.py

@@ -0,0 +1,77 @@
+from abc import ABC, abstractmethod
+from typing import Any
+
+import yaml
+
+
+class YamlProcessor(ABC):
+    @abstractmethod
+    def process(self, data: Any) -> Any:
+        pass
+
+
+class LocalBuildStrategy(YamlProcessor):
+    def process(self, data: Any) -> Any:
+        if data["x-artemis-build-or-image"].get("image"):
+            del data["x-artemis-build-or-image"]["image"]
+            data["x-artemis-build-or-image"]["build"] = {"context": ".", "dockerfile": "docker/Dockerfile"}
+            return data
+        return data
+
+
+class WebCommandStrategy(YamlProcessor):
+    def process(self, data: Any) -> Any:
+        for service in data["services"]:
+            if service == "web":
+                data["services"][service]["command"] = "uvicorn artemis.main:app --host 0.0.0.0 --port 5000 --reload"
+        return data
+
+
+class VolumeDevelopStrategy(YamlProcessor):
+    def process(self, data: Any) -> Any:
+        for service in data["services"]:
+            if service == "web" and "./:/opt" not in data["services"][service]["volumes"]:
+                data["services"][service]["volumes"].append("./:/opt")
+        return data
+
+
+class LocalBuildContainersStrategy(YamlProcessor):
+    def process(self, data: Any) -> Any:
+        for service in data["services"]:
+            if data["services"][service]["image"] == "certpl/artemis:latest":
+                del data["services"][service]["image"]
+                data["services"][service]["build"] = {"context": ".", "dockerfile": "docker/Dockerfile"}
+        return data
+
+
+class FileProcessor:
+    def __init__(self, input_file: str, output_file: str) -> None:
+        self.docker_compose_data = None
+        self.input_file = input_file
+        self.output_file = output_file
+
+    def set_data(self) -> None:
+        self.docker_compose_data = yaml.safe_load(open(self.input_file))
+
+    def process_file(self, strategy: YamlProcessor) -> None:
+        self.docker_compose_data = strategy.process(self.docker_compose_data)
+
+        with open(self.output_file, "w") as file:
+            yaml.dump(self.docker_compose_data, file)
+
+
+if __name__ == "__main__":
+
+    input_yaml_file = "docker-compose.yaml"
+    output_yaml_file = "docker-compose.dev.yaml"
+
+    processor = FileProcessor(input_yaml_file, output_yaml_file)
+    processor.set_data()
+
+    processor.process_file(LocalBuildStrategy())
+
+    processor.process_file(WebCommandStrategy())
+
+    processor.process_file(VolumeDevelopStrategy())
+
+    processor.process_file(LocalBuildContainersStrategy())

scripts/run_docker_compose

@@ -12,6 +12,7 @@ exists() {
     [ -e "$1" ]
 }
 
+# This includes Docker Compose files from subdirectories, if e.g. the user cloned https://github.com/cert-polska/artemis-modules-extra
 if exists */docker-compose.yml; then
     for DOCKER_COMPOSE_FILE in */docker-compose.yml; do
         FILE_OPTIONS="$FILE_OPTIONS -f $DOCKER_COMPOSE_FILE"

scripts/start_dev

@@ -0,0 +1,6 @@
+#!/bin/bash
+
+cd $(dirname $0)/..
+python3 scripts/create_development_docker_compose.py
+
+ADDITIONAL_DOCKER_COMPOSE_OPTIONS="-f docker-compose.dev.yaml" ./scripts/start