Neural Networks and DeepLearning

Syllabus

• Definitions: Data Science, Artificial Intelligence, Machine Learning, Neural Networks and Deep Learning
• Supervised Machine Learning: Concepts and Process Workflow
• Applied Math and Machine Learning Basics
  • Linear Algebra
  • Probability and Information Theory
  • Numerical Computation
  • Machine Learning Basics
• Introduction: Logistic Regression and Neural Networks
• Development Frameworks: PyTorch / Tensorflow
• Types of Neural networks
  • Perceptron (P), FeedForward (FF), Deep Feed Forward (DFF), Extreme Learning Machines (ELM)
  • Recurrent Neural Network (RNN), Long / Short Term Memory (LSTM), Gated Recurrent Unit (GRU), Deep Residual Networks (DRN), Transformer Neural Networks (Transformers)
  • Auto Encoders (AE, VAE, DAE, SAE)
  • Convolution Neural Networks (CNN), Deconvolution Neural Networks (DNN), DCIGN
  • Generative Adversarial Network (GAN)
  • Other Types (LSM, KN, NTM, SVM)
• NN Topics: Representation Learning, Reinforcement Learning, Transfer Learning

Books

• Ian Goodfellow, Yoshua Bengio, Aaron Courville. Deep Learning. 2016
• Hastie, Tibshirani and Friedman. The Elements of Statistical Learning. 2nd edition, 2009
• Molnar, Christoph. Interpretable Machine Learning. Lulu. com, 2020.
• Strang, Gilbert. Linear algebra and learning from data. Wellesley-Cambridge Press, 2019.

Papers

• 100 best NLP Papers
• Papers on Text Style Transfer

References

• Neural Networks and Deep Learning ref1, ref2, ref3, ref4, ref5, ref6, ref7, ref8, ref9, ref10, simple implementation, book, viz, video, meme
• Deep Learning and NLP ref1
• Neural Network concepts
  • General Math
  • Linear and dense layers
  • Weight Initialization ref1, ref2
  • Weight Averaging ref1
  • Hyperparameter Tuning ref1, ref2
  • Gradient Descent ref1, ref2, ref3, video
  • Backpropagation ref1
  • Loss Functions ref1
• Convolutional Neural Networks ref1, ref2, ref3, ref4, ref5, ref6, ref6, ref7, ref8, ref9, ref10, architectures, viz
  • CNNs for NLP ref1, ref2
  • CNNs for object detection ref1
  • CNNs for Time Series ref1
  • Capsule Networks ref1
• RNNs (Sequence Models) ref1, ref2, ref3
  • Attention Models ref1
  • LSTMs and GRUs ref1, ref2, ref3, ref4, ref5, ref6
  • Word Embeedings ref1, ref2, ref3, ref4, ref5, ref6, ref6
    • Word2vec ref1, ref2, ref3, ref4, ref5, ref6, ref7, ref, video, en-us trained models, pt-br trained models, ge-de trained models, pre-trained models
    • Char2vec ref1
    • Sentence Embeddings ref1, ref2,
    • Doc2vec ref1, ref2, ref3
    • Beyond Word Embeddings ref1, ref2, ref3, ref4, ref5, ref6, ref7
  • Glove ref1, ref2, ref3
  • FastText ref1, ref2
  • Misspelling Oblivious Word Embeddings (MOE) ref1, ref2
• Transformers ref1, ref2, ref3, ref4, ref5
  • BERT ref1, ref2, ref3, ref4, ref5, ref6, ref7, ref8, ref9, ref10, ref11
  • ALBERT ref1, ref2
  • XLNet ref1
  • ULMFiT ref1
  • GPT-3 ref1
  • Reinforcement Learning ref1, ref2, ref3, ref4, ref5, ref6, programming resource
• Transfer Learning ref1, ref2, ref3, ref4, ref5, ref6, ref6, ref7
• Autoencoders ref1, ref2, ref3, ref4, ref5
• Generative Adversarial Networks ref1, ref2, ref3, ref4, ref5, ref6, ref7, ref8, GANS and Deepfakes, Colab Notebooks

** Software **

• Anaconda - Jupyter - Python(Python Environment)
• Jupyter ref1, ref2, ref3, ref4, ref5, ref6, ref7, ref8
• Numpy ref1, ref2, ref3
• Pandas ref1, ref2, ref3, ref4, styling
• Pandas Machine Learning
• Machine Learning Tools
• Tensor Flow ref1, ref2, ref3, ref4, ref5
• TF.Text ref1
• TF Datasets ref1
• Keras ref1, ref2, ref3
• PyTorch ref1, Comparison Tensorflow vs PyTorch, Comparison Tensorflow vs PyTorch 2
• Hugging Face ref1
• Gensim