Measure smarter, deploy greener: A tool for inferring and optimising the power consumption of Convolutional Neural Networks (CNN's) on edge devices.
EdgeProfiler helps you to understand and minimise your model's power consumption and runtime, allowing you to gauge your deployment's environmental impact during the training process to help you to make smarter training decisions.
What's inside:
- Inference: Determine power consumption and runtime for different layers in a CNN model on an Nvidia Jetson edge device using our custom models.
- Training: Build your own power consumption and runtime models using DagsHub for data versioning, Scikit-Learn for model training and MLFlow for experiment tracking.
- Data Collection: Record measurements of a model's power consumption and runtime storing all data versions in DagsHub.
❓ How it works
The approach we take is similar to that of the NeuralPower paper. We use the same methodology focusing on data collection, model training, and power prediction for TensorRT models on edge devices.
Tip: More information on NeuralPower is documented here and TensorRT is documented here.
What we have done until now:
-
We have collected power and runtime measurements on a Jetson Orin device for 21 models. The dataset can be found on the DagsHub repository.
-
We have trained power and runtime prediction models for 3 different layer types of CNN models. The experiments can be viewed in the DagsHub MLFlow UI.
High-level workflow:
- Collect power and runtime measurements from a Jetson Orin using the
measure_idling_power.pyandmeasure_inference_power.pyscripts and then upload to DagsHub with thedata_version.pyscript. - Pull the raw dataset from DagsHub onto your development machine using the
data_version.pyscript. - Run the data pipeline to create the training dataset using the
create_dataset.shscript. - Run the
run.pymodel training script storing logs and models with MLFlow.
📌 Important
DagsHub repository: https://dagshub.com/fuzzylabs/edge-vision-power-estimation
MLFlow UI: https://dagshub.com/fuzzylabs/edge-vision-power-estimation.mlflow/
Learn more about how to get started to train power and runtime prediction models in the Model Training section.
To get started, set up your python environment. We really like using uv for package and project management - so if you don't have it go ahead and follow their installation guide from here.
Once you have it installed move into the model_training directory:
cd model_training
Then run the following to activate the environment and sync dependencies with the environment:
uv venv --python 3.12
source .venv/bin/activate
uv syncNote
The commands to perform inference and training require communicating with the FuzzyLabs DagsHub repository to pull data and models. To be able to do this you will be prompted to create an account or log in if your computer is not authenticated.
Performing inference can be done with a single command - if you are not already in the model_training directory then move into it:
cd model_training
Then run the following inference script to predict the power consumption and runtime for a resnet18 model on a Jetson Orin device using our custom model:
python inference.py \
--trt-engine-path sample_data/resnet18_trt_engine_info.json \
--result-csv-path results/resnet18_predictions.csv
The sample_data directory contains a handful of TRT Engine files for you to try. These files contain the features used to generate predictions for different CNN models.
The FuzzyLabs DagsHub contains trt_engine_info.json files for 21 popular CNN models. You can run inference over these files yourself by downloading and passing the path to the same inference script above with the --trt-engine-path flag.
Coming soon...
Tip: For more details on running inference see the Inference section of the
model_trainingREADME.
You can reproduce our power consumption and runtime models with a few simple steps. First you need to pull our training data from DagsHub and then run the training script.
If you are not already in the model_training directory then move into it:
cd model_training
Then run the following command to retrieve the training dataset from DagsHub:
python data_version.py \
--owner fuzzylabs \
--name edge-vision-power-estimation \
--local-dir-path training_data \
--remote-dir-path training_data \
--branch main \
--download
Once you have access to the training dataset you can train your own model with our training script:
python run.py
Tip: For more details on training your own model see the Run Training Script section of the
model_trainingREADME.
In the Model Training section above we showed you how you can train your own model with our training dataset.
However, if you do have access to the Jetson device, feel free to follow the step by step guide outlined in the getting started section of the jetson/power_logging README to collect your own measurements.
For more details on the background of the project please refer to the project's Notion site:
Contributions are welcome! Please read the Contributing Guide to get started.
- 💡 Contributing Guide : Learn about our contribution process and coding standards.
- 🐛 Report an Issue : Found a bug? Let us know!
- 💬 Start a Discussion : Have ideas or suggestions? We'd love to hear from you.
A GitHub workflow is set up to automatically deploy the MKDocs website whenever a new release is published in the repository.
Running MKDocs Locally If you're working on the docs and want to preview changes locally, you'll need to set up a virtual environment and install MKDocs:
python3 -m venv .venv
source .venv/bin/activate
pip install mkdocsOnce that's done, you can start the local server with:
mkdocs serveThis will launch the MKDocs site, which you can view at: http://127.0.0.1:8000/
The following resources have served as an inspiration for this project:
- NeuralPower paper authors
- Profiling Energy Consumption of Deep Neural Networks on NVIDIA Jetson Nano authors
Copyright © 2024 Fuzzy Labs.
Released under the Apache 2.0