This repository contains implementation of basic ML algorithms Models in Python without use of any frameworks.
Supervised learning using a Neural Network to predict a function that classisfies MNIST digits.
Neural network structure: The neural network takes in (28x28) 784 inputs, one hidden layer with n hidden units (where n is a parameter that can be changed), and 10 output units(0-9). The hidden and output units uses sigmoid activation function. The network is fully connected —that is, every input unit connects to every hidden unit, and every hidden unit connects to every output unit. Every hidden and output unit also has a weighted connection from a bias unit, whose value is set to 1.
Number of neurons in the hidden layer - n = 100
Number of epochs - epochs = 50
Learining rate - eta = 0.1
Momentum - alpha = 0.9
| η | α | n | Training Accuracy | Test Accuracy |
|---|---|---|---|---|
| 0.1 | 0.9 | 20 | 96% | 93% |
| 0.1 | 0.9 | 50 | 98% | 96.8% |
| 0.1 | 0.9 | 100 | 99.6% | 96.85% |
| 0.1 | 0 | 100 | 99.6% | 97.7% |
| 0.1 | 0.25 | 100 | 99.6% | 97.7% |
| 0.1 | 0.5 | 100 | 99.6% | 97.6% |
Unsupervised learning with k-Means clustering using EM algorithm
k - number of clusters
Input is 1500 data points(2d) taken from random Gaussian distribution. Although it can accept any 2d data as input.
Plots and log-likelihood for different values of k can be found here
k = 2
k = 3
k = 4
k = 5
3.
Unsupervised learning with GMM using EM algorithm
k - number of clusters
Input is 1500 data points(2d) taken from random Gaussian distribution. Although it can accept any 2d data as input.
Plots and log-likelihood for different values of k can be found here. It makes use of k-Means Clustering to get k clusters and then run Gaussian model and predicts a Gaussian around the data.
k = 2
k = 3
k = 4
k = 5
- Python
- Matplotlib