Problem Statement

Building Bridges with NLP: Communication Beyond Speech

Imagine a world where people with speech disorders can express themselves freely. Design an NLP-powered system that bridges the communication gap. Think beyond traditional speech recognition - how can NLP empower individuals with speech difficulties, improve therapy, and create a more inclusive environment?

What is this project about?

Our projects aims to create an application that is tailored for individuals having speech disorders, providing personalized therapy and communication assistance to enhace their ability to express themselves and engage with others effectively. Also we plan to foster a community around this application to meet the evolving needs of our users. Furthermore we plan to incorporate feedback from the them and to inform the development of new features and functionality, that will make a positive impact and empower individuals with speech disorders to express themselves and engage with the world around them.

Tech Stack Used

• Frontend Development: React Native
• User Interface Design: Figma and React Styles
• Database Management: Firebase
• Backend Development: FastAPI
• Deployment: Expo
• ML/NLP: Mediapipe, Open AI's Whisper V3 Large Model and Talkify
• Training Dataset: Kaggle

Features

• Communify: Our flagship feature that helps to translate broken speech into text.
• Signify: Users can use inbuilt custom sign model, ASL (American Sign Language) or ISL (Indian Sign Language) which translates sign language into text in real-time.
• Practice: Enables users to practice their speech through personalized learning plan.
• User Interface: Offers an intuitive platform for interaction and speech therapy.

Team Members

• Anwin Sharon
• Saket Gudimella
• Kushargra Purohith

Developed by Team ByteTribe at Hack2TechSustain'24