Skip to content

This repo hosts the code for a system that can convert American sign language to audio. As the title suggests, this system aims to establish efficient communication between normal language and sign language.

License

Notifications You must be signed in to change notification settings

Gokul-GMenon/Sign-language

Repository files navigation

American Sign-language to Audio 😎

This repo hosts the code for a system that can convert American sign language to audio. As the title suggests, this system aims to establish efficient communication between normal language and sign language.

Technologies used 💖

  • OpenCV
  • TensorFlow
  • gTTS

About the system 🎶

  • The core of the system is a machine learning model (find the notebook 🌟 here 🌟 ).
  • An example dataset for the ML model can be found here. The images undergoes the following transformations before being used for training:
    👇👇👇
  gray = cv.cvtColor(image, cv.COLOR_BGR2GRAY)
  blur = cv.GaussianBlur(gray,(5,5),2)
  th3 = cv.adaptiveThreshold(blur, 255, cv.ADAPTIVE_THRESH_GAUSSIAN_C, cv.THRESH_BINARY_INV, 11, 2)
  ret, res = cv.threshold(th3, 70, 255, cv.THRESH_BINARY_INV + cv.THRESH_OTSU)
  res = cv.resize(res, (128,128))
  • After training the model to the desired accuracy, you can save it in Custom_models folder. Save it under the name: base_model.h5 👏 👏
  • After the initial training procedure, the model can further be trained by transfer learning on the base model on the custom dataset of a person using detect_and_train.py. You can run the python file using the command:
root@Gokuls-Laptop:~/Sign-language# python self_train/detect_and_train.py -n [NAME]

where [NAME] is the name of the user whose database was just created. This python file will capture 100 images of each of the letters done the the person's hands 💦 💦 . This database will be then used for transfer learning and the model will be saved with the persons name in Custom_models.

  • After training for the profile, run the python file hand_track.py using the following command:
root@Gokuls-Laptop:~/Sign-language# python hand_track.py -n [NAME]

where [NAME] is the name of the person whose database already exists. To use the base model itself, the arguments can be skipped 💔 and simply run:

root@Gokuls-Laptop:~/Sign-language# python hand_track.py

Find the notebook for the base model here. 💞

Model Details

  • Framework: TensorFlow
  • Architecture: Convolutional Neural Network.
  • Recommends training till an accuracy of 80% on the training data atleast (can be improved with detect_and_train.py)


Gokul G Menon

About

This repo hosts the code for a system that can convert American sign language to audio. As the title suggests, this system aims to establish efficient communication between normal language and sign language.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published