Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ux for new developers #1071

Merged
merged 14 commits into from
Jun 17, 2024
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -142,3 +142,6 @@ cython_debug/
.vscode/
.idea/
.vim/

# Development Docker Compose file
docker-compose.dev.yaml
12 changes: 10 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@ and [building easy-to-read messages ready to be sent to the scanned organization

If you want to use additional modules that weren't included here due to non-BSD-compatible licenses, browse to the [Artemis-modules-extra](https://github.com/CERT-Polska/Artemis-modules-extra) repository.

If you want to modify/develop Artemis, read **[Development](#development)** first.

**Artemis is experimental software, under active development - use at your own risk.**

To chat about Artemis, join the Discord server:
Expand Down Expand Up @@ -43,9 +45,15 @@ The possibility to automatically prepare such reports enabled us to notify entit
## Screenshots
![Artemis - scan](.github/screenshots/scan.png)

<a name="development"></a>
## Development
If you want to develop Artemis, remember to change the [Docker Compose file](https://github.com/CERT-Polska/Artemis/blob/main/docker-compose.yaml#L7) so that
the Artemis image is built locally (from the code you are changing) not downloaded from Docker Hub.
To start a locally modified version of Artemis, run:
```commandline
cp env.example .env # after doing that, configure the settings by changing the user-agent and any other settings you want to change
./scripts/start_dev
```
The Artemis image is then built locally (from the code you are developing) not downloaded from Docker Hub.
For `web`, you will also be able to see the results of code modifications on the page without reloading the entire container.

### Tests
To run the tests, use:
Expand Down
8 changes: 1 addition & 7 deletions docker-compose.yaml
Original file line number Diff line number Diff line change
@@ -1,12 +1,6 @@
x-artemis-build-or-image:
&artemis-build-or-image
image: certpl/artemis:latest
# Replace with the following to use local Artemis source code to build the container,
# not one downloaded from Docker Hub:
#
# build:
# context: .
# dockerfile: docker/Dockerfile

services:
redis:
Expand Down Expand Up @@ -49,7 +43,7 @@ services:
# This is a legacy entry to facillitate migration - MongoDB is not used by Artemis anymore.
db:
# Even in the most quiet configuration, MongoDB produces a significant amount of logs.
# Therefore mongoDB logs are disabled by default (although they can be enabled).
# Therefore, mongoDB logs are disabled by default (although they can be enabled).
command: ["bash", "-c", "/usr/local/bin/docker-entrypoint.sh mongod --wiredTigerCacheSizeGB 2 --auth --bind_ip_all 2>&1 >/dev/null"]
environment:
- MONGO_INITDB_ROOT_USERNAME=root
Expand Down
77 changes: 77 additions & 0 deletions scripts/create_development_docker_compose.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
from abc import ABC, abstractmethod
from typing import Any

import yaml


class YamlProcessor(ABC):
@abstractmethod
def process(self, data: Any) -> Any:
pass


class LocalBuildStrategy(YamlProcessor):
def process(self, data: Any) -> Any:
if data["x-artemis-build-or-image"].get("image"):
del data["x-artemis-build-or-image"]["image"]
data["x-artemis-build-or-image"]["build"] = {"context": ".", "dockerfile": "docker/Dockerfile"}
return data
return data


class WebCommandStrategy(YamlProcessor):
def process(self, data: Any) -> Any:
for service in data["services"]:
if service == "web":
data["services"][service]["command"] = "uvicorn artemis.main:app --host 0.0.0.0 --port 5000 --reload"
return data


class VolumeDevelopStrategy(YamlProcessor):
def process(self, data: Any) -> Any:
for service in data["services"]:
if service == "web" and "./:/opt" not in data["services"][service]["volumes"]:
data["services"][service]["volumes"].append("./:/opt")
return data


class LocalBuildContainersStrategy(YamlProcessor):
def process(self, data: Any) -> Any:
for service in data["services"]:
if data["services"][service]["image"] == "certpl/artemis:latest":
del data["services"][service]["image"]
data["services"][service]["build"] = {"context": ".", "dockerfile": "docker/Dockerfile"}
return data


class FileProcessor:
def __init__(self, input_file: str, output_file: str) -> None:
self.docker_compose_data = None
self.input_file = input_file
self.output_file = output_file

def set_data(self) -> None:
self.docker_compose_data = yaml.safe_load(open(self.input_file))

def process_file(self, strategy: YamlProcessor) -> None:
self.docker_compose_data = strategy.process(self.docker_compose_data)

with open(self.output_file, "w") as file:
yaml.dump(self.docker_compose_data, file)


if __name__ == "__main__":

input_yaml_file = "docker-compose.yaml"
output_yaml_file = "docker-compose.dev.yaml"

processor = FileProcessor(input_yaml_file, output_yaml_file)
processor.set_data()

processor.process_file(LocalBuildStrategy())

processor.process_file(WebCommandStrategy())

processor.process_file(VolumeDevelopStrategy())

processor.process_file(LocalBuildContainersStrategy())
1 change: 1 addition & 0 deletions scripts/run_docker_compose
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ exists() {
[ -e "$1" ]
}

# This includes Docker Compose files from subdirectories, if e.g. the user cloned https://github.com/cert-polska/artemis-modules-extra
if exists */docker-compose.yml; then
for DOCKER_COMPOSE_FILE in */docker-compose.yml; do
FILE_OPTIONS="$FILE_OPTIONS -f $DOCKER_COMPOSE_FILE"
Expand Down
6 changes: 6 additions & 0 deletions scripts/start_dev
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
#!/bin/bash

cd $(dirname $0)/..
python3 scripts/create_development_docker_compose.py

ADDITIONAL_DOCKER_COMPOSE_OPTIONS="-f docker-compose.dev.yaml" ./scripts/start
Loading