Generics for implementing new sensors that interact with the OpenSensor Infrastructure as Code.
- rust: Minimum Supported Rust Version 1.65
- lld linker: for faster Rust builds.
- See .cargo/config.toml for platform-specific installation instructions and the Rust Performance Book for the reasons for using lld.
- docker: Container engine
- docker compose: Multi-container orchestration. NOTE:
docker-compose
is now deprecated and the compose functionality is integrated into thedocker compose
command. To install alongside an existing docker installation, runsudo apt-get install docker-compose-plugin
. ref.
For Debian-based Linux distros, you can install opensensor-rs
's dependencies (except Docker, that require special repository configuration documented above) with the following command:
apt-get install clang build-essential lld clang zstd libzstd-dev make cmake pkg-config libssl-dev
opensensor-rs
is tested on Ubuntu 22.04 LTS, but welcomes pull requests to fix Windows or MacOS issues.
- Clone repo:
git clone https://github.com/opensensordotdev/opensensor-rs.git
- Ensure all requirements have been installed, especially the lld linker! Otherwise
opensensor-rs
won't build!
./bootstrap_cluster.sh
: Start the testing Redpanda, MinIO, and monitoring stack.cargo test
: Verify all cargo tests pass
Measurement
: The physical data structure representing the discrete unit of data produced by the sensor. EachSensor
can produce multiple kinds ofMeasurements
.Transducer
: The interface with sensor hardware, producing a stream ofMeasurements
.Sensor
: The abstraction between multipleTransducers
that produce the same kind ofMeasurements
, grouping common validation logic together. Consumes from the mpsc channel that theTransducer
sendsMeasurements
on, validates them, and produces to Redpanda.
Measurements
are notionally serialized using Google flatbuffers. For a sample Measurement
, Transducer
, and Sensor
implementation, see the sensor-simple
crate in the opensensor repository.
In order to make implementing new sensors as straightforward as possible, opensensor-rs
seeks to provide automatic archiving of Measurement
implementers to Parquet through Rust's Arrow bindings. Experiments for archiving arbitrary Rust structs to arrow and then parquet are documented in the archiver
directory and in arrow.rs
. Ideally, this functionality would be derivable or implementable through traits to allow arbitrary measurements to be serialized to/from parquet.
Filed this issue on arrow2, but even though their resulting PR fixed the Rust code (the tests in test_arrow.rs
now pass), the resulting parquet for any nested arrays or structs still can't be deserialized in pyarrow.
The test cases in test_arrow.rs
are based on the following examples: