dora
goal is to be a low latency, composable, and distributed data flow.
This project is in early development, and many features have yet to be implemented with breaking changes. Please don't take for granted the current design.
dora
primary support is with Linux
( Ubuntu 20.04 and Ubuntu 22.04 ) as it is the primary OS for both Cloud and small computers. If you wish to use dora
with another OS, please compile from source.
The documentation can be found here: https://dora-rs.github.io/dora/
- Install
dora
binaries from GitHub releases
For linux
wget https://github.com/dora-rs/dora/releases/download/<version>/dora-<version>-x86_64-Linux.zip
unzip dora-<version>-x86_64-Linux.zip
python3 -m pip install dora-rs==<version>
PATH=$PATH:$(pwd):$(pwd)/iceoryx
dora --help
This is
x86_64
only for the moment.
- Create a new dataflow
dora new abc_project --lang python
cd abc_project
This creates the following abc_project
directory
.
├── dataflow.yml
├── node_1
│ └── node_1.py
├── op_1
│ └── op_1.py
└── op_2
└── op_2.py
- Start
dora-coordinator
in a separate terminal window
# New terminal window
dora up
- Start your dataflow
# Other window
dora start dataflow.yml
# Output: c95d118b-cded-4531-a0e4-cd85b7c3916c
The output is the unique ID of the dataflow instance, which can be used to control it through the dora
CLI.
- You will see in your
dora-coordinator
window operators receiving ticks.
Received input tick, with data: b''
Received input tick, with data: b''
Received input tick, with data: b''
...
- Stop your dataflow
dora stop c95d118b-cded-4531-a0e4-cd85b7c3916c
(Pass the ID returned by dora start
here.)
- You can then add or modify operators or nodes. For adding nodes easily, you can use the
dora
CLI again:
- Run
dora new --kind operator --lang Rust <name>
to create a new Rust operator named<name>
. - Run
dora new --kind custom-node --lang Rust <name>
to create a new custom node named<name>
.
You need to add the created operators/nodes to your dataflow YAML file.
- You can also download already implemented operators by putting links in the dataflow. This example will launch a webcam plot stream.
communication:
zenoh:
prefix: /abc_project
nodes:
- id: op_1
operator:
python: https://raw.githubusercontent.com/dora-rs/dora-drives/main/operators/webcam.py
inputs:
tick: dora/timer/millis/100
outputs:
- image
- id: op_2
operator:
python: https://raw.githubusercontent.com/dora-rs/dora-drives/main/physicals/plot.py
inputs:
image: op_1/image
Make sure to have a webcam and cv2 install via:
pip install opencv-python
Composability as:
-
YAML
declarative programming - language-agnostic:
- Rust
- C
- C++
- Python
- Isolated operators and custom nodes that can be reused.
Low latency as:
- written in ...Cough...blazingly fast ...Cough... Rust.
- PubSub communication with
iceoryx
Distributed as:
- PubSub communication with
zenoh
- Distributed telemetry with
opentelemetry
- Check out dora-drives for a template of an autonomous vehicle within a simulation.
This project is licensed under Apache-2.0. Check out NOTICE.md for more information.