README.md

Deep learning track

Four week deep learning course + coding assignment --> Kaggle competition
❗= Compulsory materials and assignments

Week 1 - Deep learning basics & Image classification

Course material:

• Theory:
  • MIT Introduction to Deep Learning
  • MIT Computer Vision with Convolution Neural Network
  • Stanford Convolutional Neural Networks
• Practice:
  • ❗Fastai Image Classification
  • ❗Fastai Data Cleaning and Production & Gradient Descent

Coding assignment: Implement an image classifier with any dataset

1. ❗ Build a Notebook from a cloud server
   • Lecture dataset: The Oxford-IIIT Pet Dataset
     • Other interesting image datasets:
       • Stanford Dogs Dataset
       • Dogs vs Cats
       • Fruits dataset
       • many more ...
2. ❗ Minimum requirement: Implement using Fastai library
   =>  Beyond minimum requirement:
   • Implement with Keras, Pytorch, Tensorflow etc.
   • Explanation / visualization of concepts
3. Send the link of your notebook with this Google form (all notebook links will be posted here)
4. Present your code in next workshop

Notebooks

Name	Notebook	Code
Aditya	Lecture 1	fastai
Rong	Image classification: clothing	fastai, opencv
Ruben	Pets dataset Keras implementation	Keras
Qingli	Lecture 1	fastai

Workshop:

• Presentation of codes and concepts
• Coding challenge - to be decided

Week 2 - Deep learning Image classification

Course material:

• Theory:
  • (Optional) Papers that explain finding learning rate Paper1 Paper 2
  • (Optional) Bag of tricks for training CNNs
  • (Optional and very helpful) Troubleshooting Deep Neural Networks
• Practice:
  • ❗Fastai Image Classification Lecture 2 Lecture 3
  • ❗Fastai NLP Lecture 4 Interesting ideas i.e. learn.freeze_to()

Coding assignment: Implement an image classifier (continued from week 1)

1. Another interesting dataset for image classification task
   • Mushrooms dataset
2. Send the link of your notebook with this Google form (all notebook links will be posted here)
3. Present your code in next workshop

Workshop:

• Presentation of codes and concepts
• Coding challenge
  • (Option 1) Classification of mushroom images

Notebooks

Name	Notebook	Code	Accuracy
Aditya	Image classification: Mushrooms	fastai	87
Salih	Image classification: Mushrooms	keras	89

Week 3 - Backprop and CNN deep dive

Course material:

• Theory:
  • (Optional) Heatmaps generated by interp.plot_top_losses Grad-CAM heatmaps
  • (Still very helpful) Troubleshooting Deep Neural Networks
  • (Optional)CNN visualization repo, Deep visualization repo
  • (Optional)CNN visualization paper
• Practice:
  • ❗Fastai Lectures Lecture 5 Lecture 6

Coding assignment: Improve your models

1. Ideas from the session
   • Use custom FeedForwardNet to replace the simple one provided by Fastai
   • Experiment with learn.fit which uses Cosine learning rates
   • More experiments: unfreeze more last layers, lower weight decay wd (allows for higher learning rate) and bayesian NN
2. Send the link of your notebook with this Google form (all notebook links will be posted here)

Notebooks

Name	Notebook
Aditya	Understanding grads and hooks
Rong	Understanding Pytorch Hooks
Rong	Visualizating CNN slides

3. Present your code in next workshop

Week 4 - Kaggle competition kick-off and team formation

❗ Join teams in this Google form

Final project - Earthquake Kaggle competition

Select 8 most active and contributive members near the end of the course. May divide into teams if there are more than 8 participants.

Q & A:

Q: Why Fastai?

A: Fastai is a high level library that implements cutting edge deep learning algorithms. It allows anyone to start implementing deep learning quickly with world-class performance and it's being used frequently in top Kaggle submissions. Fastai has excellent automation for data loading and data preprocessing. Even you wish to use other deep learning libraries to implement the models, these helper functions will still help with the boring tasks.

All in all, it's great place for beginners to start, and a valuable tool for experienced data scientist.

Q: I want to learn how Deep Learning models are implemented from scratch but Fastai makes models appear as block boxes. Why don't we learn Tensorflow or Pytorch instead?

A: It could take months to study deep learning theories and implement everything with Tensorflow or Pytorch from scratch. Also, not everyone is interested in theory and implementation details. Fastai offers a minimum requirement to understand and implement deep learning models, which is a good point to jump start.

If you are curious to learn more, you could study on your own from our recommended resources and numerous online resources. You are also encouraged to implement with other deep learning frameworks and share your codes in the weekly workshop.