- Clone the project from GitHub: RealsenseFuelcap
- Open the VSCode terminal in the project directory.
- Build the Cuda docker container using the following command:
docker buildx build -t mines_cuda -f Dockerfile.CUDA .
- Visit the Releases page and download the latest models.zip file
- Inside the workspace create a folder named "models/" if there is not one already
- Extract "bbox_model.pth" and "bbox_model.pth" into the models folder
For information on Unity Data Collection, please refer to the following GitHub repository: KeypointSimulation.
- CUDA
- VSCode
- DevContainers Extension
- Docker Engine
- Nvidia Container Toolkit
- See the Dockerfile.CUDA for a full list of the dependencies used in the container
Once all dependencies are met, you can run the code using the following procedure
-
Run the Docker Container. To do so execute the following command (ensure you change "path/to/repositiory" to your working directory):
xhost +local:root docker run -it --runtime=nvidia --net host --ipc host -e "DISPLAY=$DISPLAY" \ -v "$HOME/.Xauthority:/root/.Xauthority:ro" --privileged \ -v $HOME/path/to/repository:/home/mines/mines_ws \ --rm --name cuda_container mines_cuda
-
source the bashrc file
source .bashrc
-
Build the ROS workspace by running the alias:
ros_build
The bashrc file includes aliases for several functonalities within the code, including starting the realsense camera, data collection, and fuelcap detection nodes, calling the capture image service, and displaying the fuelcap detection info message
Follow the steps for building ROS, after building ros, in any unused terminal within the container execute the command below, then start your node:
source .bashrc
For the following nodes, execute the corresponding alias
- Realsense Node
start_realsense
- Data Collection Node
data_collection_launch
- Fuelcap Detection Node
fuelcap_detection_launch
- Display Detection Info
detection_info_launch
To call the capture image service, run the following command:
capture_image
Machine Learning Models are trained and evaluated using jupiter notebooks. In order to run the notebooks, you must be inside of the docker container from a devcontainers window.
-
To do this, start the docker contianer with the following command inside a VSCode terminal (ensure you change "path/to/repositiory" to your working directory)
xhost +local:root docker run -it --runtime=nvidia --net host --ipc host -e "DISPLAY=$DISPLAY" \ -v "$HOME/.Xauthority:/root/.Xauthority:ro" --privileged \ -v $HOME/path/to/repository:/home/mines/mines_ws \ --rm --name cuda_container mines_cuda
-
While the container is running, attach a new window to a devcontainer window.
- type ctrl+p and type ">Dev Containser: Attach to Running Container..."
- then select the containter (/cuda_container)
- After the window appears, you will be able to run the notebooks. Notebook cell's should be run sequentially unless otherwise mentioned. There are multiple notebooks including ones for
- Training Bounding Box Model
- Training Keypoints Model
- Generating Bounding Box Data
- Testing Bounding Box Model
- Testing Keypoints Model
- Bounding Box Model Statistics
- Pose Estimation (visualize)
- Prediction Results (based on ground truth data)
- Keenan Buckley - https://github.com/keenanbuckley
- David Munro - https://github.com/damunro
- Jaxon Schauer - https://github.com/jschauer1
- Xavier Cotton - https://github.com/Eldarch
This project uses a permissive BSD Zero-Clause License. For more information, see the accompanying LICENSE file.