- An object/segmentation detection app created using streamlit. Refer below for all the features of the app
- Model used: https://github.com/ultralytics/ultralytics
- App link: https://yolov8-app.streamlit.app/
- Link to EasyOCR: https://github.com/JaidedAI/EasyOCR
- NOTE: The streamlit app might not work if the GitLFS's free bandwidth exceeds 1 gb.
-
Object detection/segmentation using pre-trained yoloV8 model (trained on Open Images V7 dataset with 600 distinct classes) , refer to openimages.txt for the list of objects detectable using the base model
-
Custom-trained yolov8 model for detecting potholes
-
Custom-trained yolov8 model for detecting car license plates
-
Integrated license plate detector with EasyOCR for reading license plates. Additionally, added an image preprocessing function to handle images with brightness and Image glare issues
-
Custom-trained yolov8 model to detect PPE (7 classes: ['Protective Helm', 'Shield', 'Jacket', 'Dust Mask', 'Eye Wear, 'Glove, 'Protective Boots')
-
To use your custom trained model, just add your .pt files into the weights and make some minor changes to the settings.py and app.py files (Note: If your model's weights are >25mb, you will need to us Git LFS to upload your files)
Custom-trained models | mAP50 | mAP50-95 |
---|---|---|
Potholes detection (yolov8m) | 0.721 | 0.407 |
Car License plate detection (yolov8m) | 0.995 | 0.828 |
PPE Detection (yolov8m) | 0.991 | 0.738 |
- mAP50 and mAP50-95 are metrics used to evaluate object detection models.
- mAP50 measures the average precision at an Intersection over Union (IoU) threshold of 0.5, while mAP50-95 considers the average precision across IoU thresholds from 0.5 to 0.95.
- Higher values indicate better accuracy and robustness in detecting objects across different IoU levels.
- IOU is the ratio of the area of overlap between the predicted and actual bounding boxes to the area of their union
- EasyOCR is an open-source Python library for Optical Character Recognition (OCR). Its primary purpose is to recognize and extract text from images, enabling applications to convert images containing printed or handwritten text into machine-readable and editable text.
-
The fine-tuned yolov8 model is used for the license plate detection in an image, accurately locating the license plate's position.
-
The detected license plate region is cropped from the original image to isolate the license plate.
-
The cropped license plate image is converted to black and white, simplifying the image and emphasizing text features.
-
OpenCV (cv2) is used to enhance the contrast of the black and white license plate image, making the text more distinct and improving OCR accuracy.
-
EasyOCR is applied to the preprocessed license plate image to perform Optical Character Recognition (OCR), extracting and reading the text content from the license plate.
- Step 1: Select a task (Detection, segmentation, Potholes detection etc.)
- Step 2: Adjust model confidence if needed
- Step 3: Select source type (image/video)
- Step 4: Upload the image/video
- After uploading an image, click on the Detect objects button on the sidebar
- Improve the accuracy of the custom-trained models (train on higher quality data, Data augmentation, Hyperparameter tuning(computationally intensive, time-consuming)
- Integrate car license plate detector with SORT/DeepSORT which keep tracks of the car's information. (For real-world use case)
- Experiment with using different size yolov8 models (smaller models offers faster inference but less accuracy), smaller size models may be more suitable if you're deploying your app on Streamlit's Community Cloud
- Currently webcam feature isn't working after deploying to streamlit cloud but it works locally.
- App is currently deployed at Streamlit's Community Cloud which has limited resource, which may crash the app if the resources are exceeded.
- Video processing are slow running on Streamlit Cloud (Deploying the app on a paid-cloud service would help with the processing speed)
Step 6: Fine-tune the yolov8 model using the annotated images (fine-tune using personal gpu or use a gpu from google colab)
RoboFlow will generate a .yaml file automatically, verify it if needed and then proceed with training the model
!yolo task=detect mode=train model=yolov8m.pt data=/content/drive/MyDrive/carplate/car-plate-detection-1/data.yaml epochs=70 imgsz=640
!yolo task=detect mode=val model=/content/drive/MyDrive/carplate/runs/detect/train/weights/best.pt data=/content/drive/MyDrive/carplate/car-plate-detection-1/data.yaml
!yolo task=detect mode=predict model=/content/drive/MyDrive/carplate/runs/detect/train/weights/best.pt conf=0.25 source=/content/drive/MyDrive/carplate/car-plate-detection-1/test/images