3D multimodal medical imaging technique based on freehand ultrasound, marker tracking, and 3D reconstruction
This is the main repository for the Python3-based 3D multimodal imaging technique using freehand ultrasound and marker tracking. The project is a part of the Neuro-Mechatronics Interfaces group at Carnegie Mellon University
With this proposed technique, we can obtain the internal structure with 3D freehand ultrasound to complement the design process of an HD-EMG sleeve with optimized electrode positioning. Our system is composed of 8 Vicon tracking cameras and an ultrasound machine.
This project is inspired by Jhacson Meza's work on 3D ultrasound reconstruction
Note: This process assumes that the motion tracking data contains at least 3 markers on a plane to be able to reconstruct the pose of the ultrasound probe. The motion tracking data should be in the form of a CSV file with the following format:
The following demonstrates the structure as well as useage of the folders and files:
dataset/
- The primary directory containing the generated datamodels/
- The directory containing STL files used in visualization1_process_us_video.py
- The script to process the ultrasound video2_pose_estimation.py
- Loads motion tracking data and estimates the pose of the ultrasound probe3_multimodal_alignment.py
- Aligns the ultrasound and motion tracking data using correlation tecnique4_reconstruction.py
- Generates the 3D reconstruction of the ultrasound data
To get started, clone the repository and navigate to the package directory:
git clone https://github.com/Neuro-Mechatronics-Interfaces/Python_Ultrasound_Reconstruction.git
cd Python_Ultrasound_Recunstruction
- (Optional) Installing packages locally in a virtual environment instead of the PC is optional but recommended:
python -m venv py_us
- Activate the new environment:
# In cmd.exe py_us\Scripts\activate.bat # In PowerShell py_us\Scripts\Activate.ps1 # On Linux and MacOS source py_us/bin/activate
- When you're done, to deactivate the virtual environment, you can run the
deactivate
command:deactivate
Install the python packages required for the project in the environment:
pip install -r requirements.txt
- If it fails, try upgrading your pip version:
python -m pip install --upgrade pip
Set the path to the dataset in the 1_process_us_video.py
script and run the script to process the ultrasound video:
python 1_process_us_video.py
Set the path to the dataset in the 2_pose_estimation.py
script and run the script to estimate the pose of the ultrasound probe:
python 2_pose_estimation.py
Set the path to the dataset in the 3_multimodal_alignment.py
script and run the script to align the ultrasound and motion tracking data:
python 3_multimodal_alignment.py
Set the path to the dataset in the 4_reconstruction.py
script and run the script to generate the 3D reconstruction of the ultrasound data:
python 4_reconstruction.py