-
Notifications
You must be signed in to change notification settings - Fork 0
Config
Sara Veldhoen edited this page Feb 13, 2024
·
1 revision
The configuration of DANE components is done through the DANE.config module, which builds on top of YACS. You can read about it here
The config file in ./config/config.yml is a first that you can alter for any worker's specific needs. It consists of various sections:
-
RABBITMQ
holds settings for running the worker in the DANE pipeline. You don't need to bother about this when developing locally. -
ELASTICSEARCH
holds settings for running the worker in the DANE pipeline. You don't need to bother about this when developing locally. -
FILE_SYSTEM
specifies where files can be read/written. TheBASE_MOUNT
should refer to the root of the file system: mostlydata
when running locally, and/data
when running in a container. -
INPUT
specifies where the worker's input can be downloaded from. Currently, we use S3 for I/O. Not strictly necessary for local development. -
OUTPUT
specifies where the worker's output can be written to. Currently, we use S3 for I/O. Not strictly necessary for local development. -
WORKER-SETTINGS
may hold any configuration that is specific to this worker, like (hyper) parameter settings. -
DANE_DEPENDENCIES
lists all the other DANE workers that this worker is dependant on.