Skip to content

Latest commit

 

History

History
52 lines (43 loc) · 3.15 KB

README.md

File metadata and controls

52 lines (43 loc) · 3.15 KB
Raw

American Sign-language to Audio 😎

This repo hosts the code for a system that can convert American sign language to audio. As the title suggests, this system aims to establish efficient communication between normal language and sign language.

Technologies used 💖

  • OpenCV
  • TensorFlow
  • gTTS

About the system 🎶

  • The core of the system is a machine learning model (find the notebook 🌟 here 🌟 ).
  • An example dataset for the ML model can be found here. The images undergoes the following transformations before being used for training:
    👇👇👇
  gray = cv.cvtColor(image, cv.COLOR_BGR2GRAY)
  blur = cv.GaussianBlur(gray,(5,5),2)
  th3 = cv.adaptiveThreshold(blur, 255, cv.ADAPTIVE_THRESH_GAUSSIAN_C, cv.THRESH_BINARY_INV, 11, 2)
  ret, res = cv.threshold(th3, 70, 255, cv.THRESH_BINARY_INV + cv.THRESH_OTSU)
  res = cv.resize(res, (128,128))
  • After training the model to the desired accuracy, you can save it in Custom_models folder. Save it under the name: base_model.h5 👏 👏
  • After the initial training procedure, the model can further be trained by transfer learning on the base model on the custom dataset of a person using detect_and_train.py. You can run the python file using the command:
root@Gokuls-Laptop:~/Sign-language# python self_train/detect_and_train.py -n [NAME]

where [NAME] is the name of the user whose database was just created. This python file will capture 100 images of each of the letters done the the person's hands 💦 💦 . This database will be then used for transfer learning and the model will be saved with the persons name in Custom_models.

  • After training for the profile, run the python file hand_track.py using the following command:
root@Gokuls-Laptop:~/Sign-language# python hand_track.py -n [NAME]

where [NAME] is the name of the person whose database already exists. To use the base model itself, the arguments can be skipped 💔 and simply run:

root@Gokuls-Laptop:~/Sign-language# python hand_track.py

Find the notebook for the base model here. 💞

Model Details

  • Framework: TensorFlow
  • Architecture: Convolutional Neural Network.
  • Recommends training till an accuracy of 80% on the training data atleast (can be improved with detect_and_train.py)


Gokul G Menon