Getting Started

Pre-requirements

• Docker
• docker-compose
• Amazon Web Service S3 (Access Key, Secret Access Key from AWS IAM)

1. Clone the repository

git clone https://github.com/bunseokbot/darklight.git

2. Compose database

Setting your MariaDB account

Set database account credential at environment variable

export MYSQL_USER=<YOUR_MYSQL_USERNAME>
export MYSQL_PASSWORD=<YOUR_MYSQL_PASSWORD>
export MYSQL_DATABASE=<YOUR_MYSQL_DATABASE>

Run docker-compose for composing a database

docker-compose -f docker-compose-database.yml up -d

This script will run MariaDB, Elasticsearch, Kibana

3. Setting configuration

Editing configuration file as you want

Edit configuration options on your environment.

[DATABASE]
URL=mysql+pymysql://darklight:darklight@mariadb:3306/darklight
DEBUG=false

[ELASTICSEARCH]
HOST=elasticsearch
PORT=9200
USERNAME=
PASSWORD=

[HEADLESS]
PATH=/usr/bin/phantomjs

[TOR]
PROTOCOL=socks5h
HOST=torproxy
PORT=9050

[CELERY]
BROKER_URL=redis://redis:6379/0
RESULT_BACKEND=redis://redis:6379/1

[STORAGE]
AWS_ACCESS_KEY_ID=<YOUR_AWS_ACCESS_KEY_ID>
AWS_SECRET_ACCESS_KEY=<YOUR_AWS_SECRET_ACCESS_KEY>
REGION_NAME=<YOUR_REGION_NAME>
BUCKET_NAME=darklight

If you want to save the screenshot on local. Keep BUCKET_NAME as blank then it'll be saved at the screenshot directory on worker container. You can pull it from the worker container or use volume sharing /usr/src/app/screenshot if you want to get files automatically at the host.

Setting configuration file path

Set the config.ini file path into environment variable.

env CONFIG_FILE=<CONFIG_FILE_PATH>

4. Run core engine

Compose all core engine code and check all the containers running normally after you compose it.

docker-compose build worker // use this command if you edited the code or initial run.
docker-compose up -d