A project by Planet-Earth (#sg_planetearth) study group in Facebook Secure and Private AI Scholarship Challenge 2019.
Check out our web-app we have deployed at https://smog4000.onrender.com/. At this site, you can upload your street/highway images and test the accuracy of our classification system. Although it works on all images, you can get the best results on highway images.
Smog Detection project has been created as a collaborative team effort between
Facebook Secure and Private AI Scholars on #sg_planetearth. The main focus of the group is to provide Deep Learning solutions to solve most concerning real-world problems, such as Climate Change, Food Security, Plastic Pollution, Deep Fake, Oceanic Pollution, Industrial Nuclear Waste, Clean Energy and more(using AI for social good). Currently, our team is focusing on Smog and Fog detection on highways.
The sudden appearance of smog and/or fog on the highway more often than not causes serious and sometimes fatal accidents. Smog is the main agent for severe air pollution. It can aggravate health problems including problems with breathing and sleeping, as well as it can inversely damage plants and forest cover.
Monitoring, early smog detection, and preventive action. By using traffic cameras and train a model to recognize the smog/fog patterns, we can automate the alert and send a notification promptly. When smog/fog appears, the system notifies drivers who are within specified range about an upcoming “obstacle”.
In any practical situation, other components need to be taken into consideration as well. For example, the presence of flying birds and any type of material that will block the camera view. To identify the range of vision sensors need to be added to the solution.
There are plenty of existing solutions and advancements in computer vision. Our approach is to use Machine Learning Image Detection and train a CNN model. Smog Detection is a smog classification project, where images are arranged into two main categories: "Smog/Fog/NotClearlyVisible Highways" and "Clearly Visible Highways".
Smog is a byproduct of the global climate change scenario. As we move further into the industrialized age, Smog continues to pollute our air, reduce visibility on roads and is a leading cause of accidents on roads.
The primary goal of the project is to avoid and help reduce the rate of accidents in self-driving vehicles. That can be achieved by using this classifier model as one of the key components attached to traffic cameras. This way vehicles can automatically be alerted about smog on the roads, streets or highways, no matter if it's heavy, medium or low traffic to adjust its dynamics like speed, steering rotation, lanes, etc. We can use the prediction output labels from the model to alert vehicles and drivers.
Since inception, people in the group have taken their task most seriously. They contributed towards building a unique dataset which now has more than 4,000 images divided evenly between the two groups mentioned above. Our collection consists of:
- Images need to have a view of highways or streets;
- Avoid a large number of people;
- Avoid traffic accidents;
- Avoid night pictures;
- Avoid bird's-eye view;
- Avoid images that are copyrighted or have a watermark;
- Acceptable image formats: jpg, jpeg, png, jfif, webp.
Notes:
- Synthetic images we used were taken from SFSU synthetic dataset
- To detect and remove duplicates (regardless of format and resolution), we were using DupeGuru. It is an open-source tool for duplicate image detection that works across different platforms (Linux, OS X, Windows).
| No | Name | Slack Handle |
|---|---|---|
| 1 | Shudipto Trafder | @Shudipto Trafder |
| 2 | Berenice Terwey | @Berenice |
| 3 | Agata Gruza | @Agata [OR, USA] |
| 4 | Ingus Terbets | @Ingus Terbets |
| 5 | Akash Antony | @Akash Antony |
| 6 | Alexander Villasoto | @Alexander Villasoto |
| 7 | Pooja Vinod | @Pooja Vinod |
| 8 | Ramkrishna Acharya | @Viper |
| 9 | Sourav Kumar | @sourav kumar |
| 10 | George Christopoulos | @George Christopoulos |
| 11 | Sayed Maheen Basheer | @Sayed Maheen Basheer |
| 12 | Abhishek Lalwani | @Abhishek Lalwani |
| 13 | Laura Truncellito | @LauraT |
'Smog' Detection
'Clear' Detection
This is a descriptive summary of the Model Architecture we have implemented, and the Data Augmentations we have applied.
Find the notebook for this implementation here.
Input to model:
Images captured by traffic cameras.
Output from model :
Prediction as label '0'(clear view) or '1'(smog detected).
- transforms.RandomRotation(30),
- transforms.RandomHorizontalFlip(),
- transforms.Resize(256),
- transforms.ColorJitter(0.1),
- transforms.CenterCrop(224),
- transforms.ToTensor(),
- transforms.Normalize([0.485, 0.456, 0.406],[0.229, 0.224, 0.225])
Five modules contain deeper sublayers. Let's go through them one by one:
- CNN layers : 7
- Linear layers : 2
- Pooling layers : 4
- Batch normalization layers : 7
conv1 :
- CNN layers : Conv2d(3, 32, 3, padding=1), Conv2d(32, 32, 3, stride=2, padding=1)
- Pooling layer : MaxPool2d(2, 2)
- Batch normalization layer : BatchNorm2d(32), nn.BatchNorm2d(32),
conv2 :
- CNN layers : Conv2d(32, 64, 3, padding=1), Conv2d(64, 64, 3, stride=2, padding=1)
- Pooling layer : MaxPool2d(2, 2)
- Batch normalization layer : BatchNorm2d(32), nn.BatchNorm2d(64),
conv3 :
- CNN layers : Conv2d(64, 128, 3, padding=1), Conv2d(128, 128, 3, stride=2, padding=1)
- Pooling layer : MaxPool2d(2, 2)
- Batch normalization layer : BatchNorm2d(128), BatchNorm2d(128)
conv4 :
- CNN layers : Conv2d(128, 256, 3, padding=1)
- Pooling layer : MaxPool2d(2, 2)
- Batch normalization layer : BatchNorm2d(256)
fc :
- Linear layers : Linear(256, 128), Linear(128, 2)
Dropout has been applied in conv4 and linear layer where probability = 0.35 in conv4 and 0.5 in fc.
Mish activation has been used.
Mila activation has been used.
Adam optimizer is used here.
CrossEntropyLoss is used.
- lr schedular = StepLR(optimizer, step_size=10, gamma=0.5)
- lr = 0.001
epoch : 50
Test Loss: 36.734721
Accuracy: 99.0000
- Test Accuracy of 0: 98% (389/393)
- Test Accuracy of 1: 98% (378/382)
Test Accuracy (Overall): 98% (767/775)
We decided to create a user-friendly web app so everyone can test and experiment with our project. We hope that this will be useful for any road-alerts related project that may require such a smog detection/classification facility.
- Libraries used: keras 2.2.4, numpy, matplotlib
- Trained: on google Colab
- Dataset Used: Smog4000
- Find the notebook for this implementation here.
Smog4000 dataset was created by our team.
- Rescale range: 1./255
- shear and zoom: 0.2
- color: RGB
- Outshape: 224 X 224
Custom CNN model is used here. Complete summary of model architecture is given below:
The Sequential model has 3 blocks of CNN layers and one final linear layer. In each CNN block we have:
- Convolution layers of same out filters(32, 32, 64, 64, 128, 128)
- Filter shape: (3, 3)
- Activation Function: ReLU(Rectified Linear Unit)
- MaxPooling: pool size(3, 3)
- BatchNormalization
- Dropout: 0.25
The Final block is for linear layers. It has:
- Flatten
- Dense of out 256 and ReLU
- BatchNormalization
- Dropout of 0.5
- Classification layer with sigmoid
Total parameters: 1,469,346
- Optimizer: Adam(Adaptive Momentum) Optimizer is used on this model.
- Learning rate: default(0.001)
- Loss function: Categorical_crossentropy (categories: clear, smog)
- Epochs: 10
- Batch Size: 32
- Train Accuracy: 0.9968
- Validation Accuracy: 0.9921
- Test Accuracy: 0.985
- Train time: 750 seconds per epoch
- Using fewer parameters
- Test with different optimizer and loss functions
The images collected in this project are carefully chosen to avoid any copyright issues. For images that contain intellectual property issues or private images, we can consider incorporating federated learning and secure prediction into our classification models. This will allow for training in a private cloud while minimizing the risk of leaking intellectual property or private training data.