Deep Learning Lecture Notes and Experiments

2023-2024 Version

Revised and expanded

Theory

Topic	Note	Video	Code
Overview	PDF	-	-
Supervised Learning	PDF	-	-
Multilayer Perceptron (MLP)	PDF	-	Notebook
Convolutional Neural Network (CNN)	PDF	-	Notebook
Recurrent Neural Network (RNN)	PDF	-	Notebook
Transformer	PDF	-	Notebook
Mamba	PDF	-	SimpleMamba Mamba2
Optimization	PDF	-	-
Regularization	PDF	-	-
Detection	PDF	-	-
Segmentation	PDF	-	SAM2
Autoencoder (AE)	PDF	-	AE & Denoising AE Colorization AE
Variational Autoencoder (VAE)	PDF	-	VAE and CVAE
Generative Adversarial Network (GAN)	PDF	-	DCGAN and CGAN
Intro to Large Language Models (LLMs)	PDF	-	GPT2-TS-train, GPT2-TS-val
LLM Data and Model	PDF	-	GPT2-TS-ft, GPT2-TS-ft-val

Practice

Topic	Note	Video	Code
Development Environment	PDF	-	-
Python	PDF	-	-
Numpy	PDF	-	-
Einsum	PDF	-	Notebook
Einops	PDF	-	Notebook
PyTorch	PDF	-	-
Gradio	PDF	-	Notebook Llama Chat
Efficiency	PDF	-	Code
PyTorch Lightning	PDF	-	Notebook
Model Packaging & Serving	PDF	-	ONNX Export ONNX Runtime TorchScript & TensorRT PyTriton Yolo Client PyTriton Yolo Server
Docker	PDF	=	-
HuggingFcae	PDF	-	-

Install

Assuming you already have anaconda or venv, install the required python packages to run the experiments in this version.

pip install -r requirements.txt --upgrade

2022

Coverage:

AI, ML and Deep Learning	Note	Video	Code
Overview	PDF	YouTube	-
Toolkit
Development Environment     and Code Editor	PDF	YouTube	-
Python	PDF	YouTube	-
Numpy	PDF	YouTube	Jupyter
Einsum	PDF	YouTube	Jupyter
Einops	PDF	YouTube	Jupyter & Jupyter (Audio)
PyTorch & Timm	PDF	YouTube	PyTorch/Timm & Input Jupyter
Gradio & Hugging Face	PDF	YouTube	Jupyter
Weights and Biases	PDF	YouTube	Jupyter
Hugging Face Accelerator	Same as W&B	Same as W&B	Jupyter & Python
Datasets & Dataloaders	PDF	YouTube	Jupyter
Supervised Learning	PDF	YouTube
PyTorch Lightning	PDF	YouTube	MNIST & KWS
Keyword Spotting App			cd versions/2022/supervised/python && python3 kws-infer.py --gui
Building blocks: MLPs, CNNs, RNNs, Transformers
MLP	PDF	YouTube	MLP on CIFAR10
CNN	PDF	YouTube	CNN on CIFAR10
Transformer	PDF	YouTube	Transformer on CIFAR10
Backpropagation	PDF
Optimization	PDF
Regularization	PDF
Unsupervised Learning	Soon
AutoEncoders	PDF	YouTube	AE MNIST Colorization CIFAR10

Star, Fork, Cite

If you find this work useful, please give it a star, fork, or cite:

@misc{atienza2020dl,
  title={Deep Learning Lecture Notes},
  author={Atienza, Rowel},
  year={2020},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/roatienza/Deep-Learning-Experiments}},
}