This project focuses on enhancing infrastructure inspection through the integration of deep learning models and generative AI technologies for crack detection and report generation. The inspection process primarily involves two main components: crack detection and report generation.
Crack detection is achieved through a deep learning model inspired by GoogleNet architecture. This model is trained on a dataset of annotated crack images to accurately identify and classify cracks present in the structures. The trained model is capable of detecting cracks with high accuracy, aiding in the early identification of potential structural issues.
Dataset link: https://universe.roboflow.com/dronerangers/crack-detection-kjeab
The report generation process utilizes various technologies to create comprehensive inspection reports. Firstly, a Visual Question Answering (VQA) model is employed to gather answers related to detected cracks and other visual aspects from the inspection data. Additionally, depth camera data is utilized to provide further insights into the structural condition. Finally, a Language Model (LM) is used to generate detailed inspection reports in Markdown format. The generated reports are presented to users through a Streamlit web page interface, offering an intuitive and accessible platform for viewing inspection findings and recommendations.
To provide a visual overview of our project's capabilities and functionality, we have prepared a video demonstration showcasing the crack detection process, report generation, and the user interface for accessing inspection findings and recommendations.
Crack-Genius.Demonstration.Video.mp4
We have won 6th place in the IEEE IES Gen AI Challenge 2024 and are invited to present our paper at IECON 2024, Chicago 🥳🥳
Watch the Video Demonstration on YouTube
In this video, you will see our deep learning model in action as it accurately detects cracks in infrastructure images. Following crack detection, you'll witness how our system generates detailed inspection reports, incorporating insights from both visual data and depth camera information. Finally, we'll walk you through the user-friendly Streamlit web page interface, where inspection reports are presented for easy access and interpretation by users.
-
Amritha Nandini
GitHub: amri-tah -
Sanjay Kumar
GitHub: Sanjay-saturn -
Saran Dharshan
GitHub: SaranDharshanSP