Gesture-Based Interaction with Feedforward

Introduction

Within the scope of the User Interface Technologies (UIT) course, we have embarked on the development of a 3D feed- forward system, aimed at gestural interaction via webcam to allow users to interface with the computer through gestures, receiving visual feedback in real time to facilitate the learning and execution of movement sequences.

V Arrows

V GIFs

Getting Started (Development)

1. Clone the repository and cd into it:

   git clone https://github.com/tkachenko0/3d-Gesture-Feedbacks-and-Feeedforwards.git
   # cd into the project directory
   cd 3D-Gesture-Feedbacks-and-Feeedforwards

2. Create a virtual environment:

   python3.9 -m venv <YOUR-VENV-NAME>

3. Activate the virtual environment:

   source <YOUR-VENV-NAME>/bin/activate

4. Install the dependencies:

   pip3 install -r requirements.txt

5. Setup your environment variables in the .properties file:

   touch .properties
   

   # Example of the .properties file
   CAMERA_INDEX=1 # the index of the camera to use

Note: If you install new dependencies, make sure to update the requirements.txt file:

pip3 freeze > requirements.txt

References

• Mediapipe
• OpenCV
• Tensorflow