Lightweight realtime mobile-based computer vision Sudoku solver
![](https://private-user-images.githubusercontent.com/30610197/311962392-98b3c43f-e0bd-4cd1-981c-8b0eb022ff97.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjYxMzEyMjQsIm5iZiI6MTcyNjEzMDkyNCwicGF0aCI6Ii8zMDYxMDE5Ny8zMTE5NjIzOTItOThiM2M0M2YtZTBiZC00Y2QxLTk4MWMtOGIwZWIwMjJmZjk3LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA5MTIlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwOTEyVDA4NDg0NFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTNiOTIyZWVjMDU0OWFjZDBjYTIxZDVmMmQ1MDMyMjVmM2VhYTQ4ODE2NTc0ODBkY2NhNWUyNjIyOWVmMDgxN2QmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.WU6CvNbgQ1nOJ0-j_o4kcy62XNrxjyEfUe7NJG7b2LI)
Deployed at https://knosmos.github.io/sudontku/
Starting with the raw camera feed, Otsu's adaptive thresholding gives us a binary black-and-white image. After some more preprocessing to clean up the image, we run contour detection and polygon approximation to find candidate board boundary points. We look for the largest square-shaped contour (four edges that are roughly equal in length), and run a homography to correct the perspective distortion. From here, we can split the board into its 81 cells and individually run digit recognition on each of them; this is implemented with template matching. The last step is the easiest: a backtracking algorithm takes the detected board and solves the Sudoku.
My friend is too good at Sudoku. She beat me ten times in a row, so I figured I would try to level the playing field :)
- Camera image capture
- Image preprocessing
- Board detection and perspective transformation
- Template-matching digit recognition
- Sudoku solver
- Clean UI
- Image upload
- Board detection confidence and prompting for manual corner selection