Overview
DeepASL utilizes webcam video feed and some Python code to interpret American Sign Language hand gestures in real time! My goal with this project was to learn the very basics of what makes up Convolutional Neural Networks and how Computer Vision can make them interactively useful in the real world.
Installation and Usage
- Clone this repo:
git clone https://github.com/cesarealmendarez/DeepASL.git - Navigate to project:
cd DeepASL - Install required packages:
pip3 install opencv-python mediapipe numpy - Run DeepASL:
python3 app.py
What's on my Screen?
Once you run DeepASL, two windows will appear, the Analytics window displays the raw video along with the extracted data points used in interpreting hand landmarks/steadiness, depth perception, output confidence, and finally triggering the snapshot. The Hand Segmentation window shows what the network will break down into a pattern of 1's and 0's prompting it's best attempt to guess what ASL letter you're showing!
Resources Used
- MediaPipe: Used to perceive the shape of the hand and create a skeleton-like outline, enabling segmentation of useful classification features.
- MNIST Handwritten Digits Classification using a Convolutional Neural Network (CNN)
- A Comprehensive Guide to Convolutional Neural Networks — the ELI5 way
- Simple Introduction to Convolutional Neural Networks