⭐ nice, YuRide Challenge Winner
Focus is a web-based utility for preventing fatigue-based car accidents. It detects and alerts both drivers and vehicle systems to potential drowsiness - helping to prevent accidents. This software is built to be easily ported to older vehicle independent of a "central compute unit" or a newer vehicle with one
When you are rolling down the highway in a two-ton death machine, the last thing you wanna do is take your eyes off the road or fall asleep. This program prevents and alerts drivers/vehicles to possible fatigue/drowsiness, preventing possible accidents.
We use GatsbyJS with theme-ui for our frontend visuals. We also combine framer-motion, for fancy animations
For face landmark detection, we use @tfjs/face-landmark-detection. We chose google's library over others since it has a relatively accurate, small, and therefore quick model, making it perfect for low-cost hardware. We've also chose the WASM backend for tf.js considering that it performs better than WebGL on low-end devices, and about the same on medium-higher performance hardware.
We take the ratio of the height of one's eyes, compared to the length. Because for most people, eyes are longer than the height, the resulting ratio is 99% of the time a value from 0-1.
Here's a diagram and equation describing the calculation of the ratio, which is called EAR.
Our ratio is similar to the one from here: Real-Time Eye Blink Detection using Facial Landmarks. However, the main difference for ours is that we only use one vertical line for our aspect ratio. This is fine since the tf.js model is incredibly stable, and it very slightly improves perf (but it's probably not noticable at all).
You may worry that the drowsiness detection is extremely volative, but don't worry. From our testing, Google's face landmarks models are surprisingly extremely stable, even accurate across a large range of head angles.
This is a hackathon, we got 24 hours (some of which I'd like to use for sleep), and gathering training data, waiting for it to train, and then building the necessary infrastructure to make it run is a huge PITA.
- Training a model to automatically predict the EAR threshold would be pretty nice. This probably wouldn't be too hard, but I'm sleepy.
- Gaze detection to make sure people are actually look at the road - apparently google's tf.js model for that is in development.
- Proper authentication/alternative methods to communicate with vehicles. Unsecured GET requests to shut down your car is a bit... sketchy. We could probably solve this problem with some rotating crypto style-stuff (kinda like U2F), and NOT communicating over web requests (we only did this for demo/concept purposes).
- Better performance. Not running ML predictions on a separate WebWorker is... yikes.
