title | emoji | colorFrom | colorTo | sdk | sdk_version | app_file | license |
---|---|---|---|---|---|---|---|
Diabetic Retinopathy Detection App |
🐢 |
blue |
purple |
gradio |
4.22.0 |
app.py |
mit |
Diabetic Retinopathy Detection: Utilizing Multiprocessing for Processing Large Datasets and Transfer Learning to Fine-Tune Deep Learning Models
Efficiently process large datasets & develop advanced model pipelines for diabetic retinopathy detection. Streamlining diagnosis.
In this project, large datasets are efficiently handled by downloading, extracting, and preparing them for analysis. Utilizing PyTorch Lightning, a robust system for diabetic retinopathy detection is developed, categorizing images into distinct disease stages. The model pipeline is enriched with various pretrained backbone models, with progress tracked using TensorBoard. Furthermore, a user-friendly web app is created to showcase the model's capabilities. The approach pursued aims to streamline both data processing and model development, facilitating accurate and accessible diabetic retinopathy diagnosis.
Introduction: Diabetic retinopathy (DR) remains a significant global health concern, with early detection playing a critical role in preventing vision loss. For those eager to contribute to this vital area of research, a comprehensive project studio is readily available. This studio has already tackled many essential tasks involved in DR detection, providing researchers and enthusiasts with a ready-to-use platform for experimentation.
Get Started with the Project Studio: Researchers and enthusiasts alike can access the necessary tools and resources by duplicating this project studio. This streamlined solution offers an immediate starting point for experimentation on the Diabetic Retinopathy Dataset.
What the Studio Offers:
- Efficient Handling of Large Datasets: The studio automates the management of large datasets, including downloading, extracting, and data preparation.
- Advanced Model Development: Utilizing PyTorch Lightning, the studio facilitates the development of a sophisticated system for DR detection, categorizing images into different disease stages.
- Integration of Pretrained Backbone Models: Various pretrained backbone models are integrated into the pipeline, allowing for experimentation with different architectures.
- Progress Tracking with TensorBoard: Researchers can monitor progress seamlessly with TensorBoard integration, tracking metrics and visualizing model performance.
- User-Friendly Web Application: A user-friendly web application is provided for showcasing model capabilities and sharing findings effortlessly.
Here's a more structured and standardized version of the steps in a blog format:
Note: You can skip this entire step, as this studio already has it done for you.
In this step, we'll walk through the process of downloading and preprocessing the Diabetic Retinopathy Detection dataset. This dataset is commonly used for developing algorithms to identify diabetic retinopathy in eye images.
Before we begin, ensure you have the following prerequisites:
- Kaggle API key (Get one here)
kaggle
library installed (pip install kaggle
)
Note: Before proceeding with the steps below, make sure to change your current directory to dr-detection
and install the required dependencies by running the following commands:
cd dr-detection
pip install -r requirements.txt
There are two ways to download the dataset:
kaggle competitions download -c diabetic-retinopathy-detection
# Extract
unzip diabetic-retinopathy-detection.zip -d data/diabetic-retinopathy-detection
rm diabetic-retinopathy-detection.zip
./scripts/download-dr-dataset.sh
# Merge and extract the parts
./scripts/merge_and_extract.sh
Once the dataset is downloaded, preprocess the images to crop and resize them.
python scripts/crop_and_resize.py --src data/diabetic-retinopathy-dataset/train data/diabetic-retinopathy-dataset/resized/train
python scripts/crop_and_resize.py --src data/diabetic-retinopathy-dataset/test data/diabetic-retinopathy-dataset/resized/test
Finally, split the data into train and validation sets and save them to CSV files.
python scripts/split_dataset.py
In the previous section, we covered how to set up your dataset and configure your training pipeline using a Config
class. Now, let's dive into training your model and monitoring its progress using TensorBoard.
If you're looking for examples of data transformations and augmentations, you can explore the provided notebook.ipynb
file. This notebook contains various examples of data preprocessing techniques, such as resizing, cropping, rotation, and more.
To open and explore the notebook:
- Navigate to the directory containing the
notebook.ipynb
file. - Open the notebook and run the cells to see different transformation and augmentation examples.
To train your model, you can use the provided train.py
script. Make sure you have set up your environment correctly and installed all dependencies as mentioned earlier. Here's how you can run the training pipeline:
- Open your terminal or command prompt.
- Navigate to the directory containing the
train.py
script. - Run the following command:
python train.py
This command will execute the training script and start training your model based on the parameters specified in your Config
class.
TensorBoard is a powerful tool for visualizing and monitoring the training process. You can use it to track metrics such as loss, accuracy, and learning rate over time, as well as visualize model graphs and embeddings.
To load TensorBoard logs and monitor your training progress:
- Ensure you have TensorBoard installed. You can install it via pip:
pip install tensorboard
- Once your model starts training, TensorBoard logs will be generated in the specified directory (e.g.,
"logs/"
). You can launch TensorBoard using the following command:
tensorboard --logdir=logs/
This command will start a TensorBoard server locally, allowing you to view your training metrics and visualizations in your web browser.
Welcome to Diabetic Retinopathy Detection App! This app utilizes deep learning models to detect diabetic retinopathy in retinal images. Diabetic retinopathy is a common complication of diabetes and early detection is crucial for effective treatment.
Use the interactive interface below to upload retinal images and get predictions on diabetic retinopathy severity.
Open Diabetic Retinopathy Detection App
- Click on the "Open Diabetic Retinopathy Detection App" button above.
- Upload a retinal image by clicking on the "Upload Image" button.
- Once the image is uploaded, the model will process it and provide predictions on the severity of diabetic retinopathy.
- Interpret the results provided by the model.