Skip to content

Commit

Permalink
Ux for new developers (#1071)
Browse files Browse the repository at this point in the history
  • Loading branch information
michalkrzem authored Jun 17, 2024
1 parent 0e0183c commit 9419e4e
Show file tree
Hide file tree
Showing 6 changed files with 98 additions and 9 deletions.
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -142,3 +142,6 @@ cython_debug/
.vscode/
.idea/
.vim/

# Development Docker Compose file
docker-compose.dev.yaml
12 changes: 10 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@ and [building easy-to-read messages ready to be sent to the scanned organization

If you want to use additional modules that weren't included here due to non-BSD-compatible licenses, browse to the [Artemis-modules-extra](https://github.com/CERT-Polska/Artemis-modules-extra) repository.

If you want to modify/develop Artemis, read **[Development](#development)** first.

**Artemis is experimental software, under active development - use at your own risk.**

To chat about Artemis, join the Discord server:
Expand Down Expand Up @@ -43,9 +45,15 @@ The possibility to automatically prepare such reports enabled us to notify entit
## Screenshots
![Artemis - scan](.github/screenshots/scan.png)

<a name="development"></a>
## Development
If you want to develop Artemis, remember to change the [Docker Compose file](https://github.com/CERT-Polska/Artemis/blob/main/docker-compose.yaml#L7) so that
the Artemis image is built locally (from the code you are changing) not downloaded from Docker Hub.
To start a locally modified version of Artemis, run:
```commandline
cp env.example .env # after doing that, configure the settings by changing the user-agent and any other settings you want to change
./scripts/start_dev
```
The Artemis image is then built locally (from the code you are developing) not downloaded from Docker Hub.
For `web`, you will also be able to see the results of code modifications on the page without reloading the entire container.

### Tests
To run the tests, use:
Expand Down
8 changes: 1 addition & 7 deletions docker-compose.yaml
Original file line number Diff line number Diff line change
@@ -1,12 +1,6 @@
x-artemis-build-or-image:
&artemis-build-or-image
image: certpl/artemis:latest
# Replace with the following to use local Artemis source code to build the container,
# not one downloaded from Docker Hub:
#
# build:
# context: .
# dockerfile: docker/Dockerfile

services:
redis:
Expand Down Expand Up @@ -49,7 +43,7 @@ services:
# This is a legacy entry to facillitate migration - MongoDB is not used by Artemis anymore.
db:
# Even in the most quiet configuration, MongoDB produces a significant amount of logs.
# Therefore mongoDB logs are disabled by default (although they can be enabled).
# Therefore, mongoDB logs are disabled by default (although they can be enabled).
command: ["bash", "-c", "/usr/local/bin/docker-entrypoint.sh mongod --wiredTigerCacheSizeGB 2 --auth --bind_ip_all 2>&1 >/dev/null"]
environment:
- MONGO_INITDB_ROOT_USERNAME=root
Expand Down
77 changes: 77 additions & 0 deletions scripts/create_development_docker_compose.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
from abc import ABC, abstractmethod
from typing import Any

import yaml


class YamlProcessor(ABC):
@abstractmethod
def process(self, data: Any) -> Any:
pass


class LocalBuildStrategy(YamlProcessor):
def process(self, data: Any) -> Any:
if data["x-artemis-build-or-image"].get("image"):
del data["x-artemis-build-or-image"]["image"]
data["x-artemis-build-or-image"]["build"] = {"context": ".", "dockerfile": "docker/Dockerfile"}
return data
return data


class WebCommandStrategy(YamlProcessor):
def process(self, data: Any) -> Any:
for service in data["services"]:
if service == "web":
data["services"][service]["command"] = "uvicorn artemis.main:app --host 0.0.0.0 --port 5000 --reload"
return data


class VolumeDevelopStrategy(YamlProcessor):
def process(self, data: Any) -> Any:
for service in data["services"]:
if service == "web" and "./:/opt" not in data["services"][service]["volumes"]:
data["services"][service]["volumes"].append("./:/opt")
return data


class LocalBuildContainersStrategy(YamlProcessor):
def process(self, data: Any) -> Any:
for service in data["services"]:
if data["services"][service]["image"] == "certpl/artemis:latest":
del data["services"][service]["image"]
data["services"][service]["build"] = {"context": ".", "dockerfile": "docker/Dockerfile"}
return data


class FileProcessor:
def __init__(self, input_file: str, output_file: str) -> None:
self.docker_compose_data = None
self.input_file = input_file
self.output_file = output_file

def set_data(self) -> None:
self.docker_compose_data = yaml.safe_load(open(self.input_file))

def process_file(self, strategy: YamlProcessor) -> None:
self.docker_compose_data = strategy.process(self.docker_compose_data)

with open(self.output_file, "w") as file:
yaml.dump(self.docker_compose_data, file)


if __name__ == "__main__":

input_yaml_file = "docker-compose.yaml"
output_yaml_file = "docker-compose.dev.yaml"

processor = FileProcessor(input_yaml_file, output_yaml_file)
processor.set_data()

processor.process_file(LocalBuildStrategy())

processor.process_file(WebCommandStrategy())

processor.process_file(VolumeDevelopStrategy())

processor.process_file(LocalBuildContainersStrategy())
1 change: 1 addition & 0 deletions scripts/run_docker_compose
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ exists() {
[ -e "$1" ]
}

# This includes Docker Compose files from subdirectories, if e.g. the user cloned https://github.com/cert-polska/artemis-modules-extra
if exists */docker-compose.yml; then
for DOCKER_COMPOSE_FILE in */docker-compose.yml; do
FILE_OPTIONS="$FILE_OPTIONS -f $DOCKER_COMPOSE_FILE"
Expand Down
6 changes: 6 additions & 0 deletions scripts/start_dev
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
#!/bin/bash

cd $(dirname $0)/..
python3 scripts/create_development_docker_compose.py

ADDITIONAL_DOCKER_COMPOSE_OPTIONS="-f docker-compose.dev.yaml" ./scripts/start

0 comments on commit 9419e4e

Please sign in to comment.