"Keras is an open source neural network library written in Python and capable of running on top of either TensorFlow, CNTK or Theano.
Use Keras if you need a deep learning libraty that:
- Allows for easy and fast prototyping
- Supports both convolutional networks and recurrent networks, as well as combinations of the two
- Runs seamlessly on CPU and GPU
Keras is compatible with Python 2.7-3.5"[1].
Since Semptember 2016, Keras is the second-fastest growing Deep Learning framework after Google's Tensorflow, and the third largest after Tensorflow and Caffe[2].
"Deep Learning is the application to learning tasks of artificial neural networks(ANNs) that contain more than one hidden layer. Deep learning is part of Machine Learning methods based on learning data representations. Learning can be supervised, parially supervised or unsupervised[3]."
Simple Youtube presentation what type of visualization is generated:
You will learn:
- What is Keras library and how to use it
- What is Deep Learning
- How to use ready datasets
- What is Convolutional Neural Networks(CNN)
- How to build step by step Convolutional Neural Networks(CNN)
- What are differences in model results
- What is supervised and unsupervised learning
- Basics of Machine Learning
- Introduction to Artificial Intelligence(AI)
- 4layerCNN.py - 4-layer Keras model
- 6layerCNN.py - 6-layer Keras model
- README.md - project description step by step
OPERATION DATA DIMENSIONS WEIGHTS(N) WEIGHTS(%)
Input ##### 3 32 32
Conv2D \|/ ------------------- 896 0.0%
relu ##### 32 32 32
Dropout | || ------------------- 0 0.0%
##### 32 32 32
Conv2D \|/ ------------------- 9248 0.4%
relu ##### 32 32 32
MaxPooling2D Y max ------------------- 0 0.0%
##### 32 16 16
Conv2D \|/ ------------------- 18496 0.8%
relu ##### 64 16 16
Dropout | || ------------------- 0 0.0%
##### 64 16 16
Conv2D \|/ ------------------- 36928 1.5%
relu ##### 64 16 16
MaxPooling2D Y max ------------------- 0 0.0%
##### 64 8 8
Conv2D \|/ ------------------- 73856 3.1%
relu ##### 128 8 8
Dropout | || ------------------- 0 0.0%
##### 128 8 8
Conv2D \|/ ------------------- 147584 6.2%
relu ##### 128 8 8
MaxPooling2D Y max ------------------- 0 0.0%
##### 128 4 4
Flatten ||||| ------------------- 0 0.0%
##### 2048
Dropout | || ------------------- 0 0.0%
##### 2048
Dense XXXXX ------------------- 2098176 87.6%
relu ##### 1024
Dropout | || ------------------- 0 0.0%
##### 1024
Dense XXXXX ------------------- 10250 0.4%
softmax ##### 10
model = Sequential()
model.add(Conv2D(32, (3, 3), padding='same', activation='relu', input_shape=x_train.shape[1:]))
model.add(Dropout(0.2))
model.add(Conv2D(32,(3,3),padding='same', activation='relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Conv2D(64,(3,3),padding='same',activation='relu'))
model.add(Dropout(0.2))
model.add(Conv2D(64,(3,3),padding='same',activation='relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Conv2D(128,(3,3),padding='same',activation='relu'))
model.add(Dropout(0.2))
model.add(Conv2D(128,(3,3),padding='same',activation='relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Flatten())
model.add(Dropout(0.2))
model.add(Dense(1024,activation='relu',kernel_constraint=maxnorm(3)))
model.add(Dropout(0.2))
model.add(Dense(num_classes, activation='softmax'))
sgd = SGD(lr=0.01, momentum=0.9, decay=1e-6, nesterov=False)
Train model:
model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])
return model
cnn_n = base_model()
cnn_n.summary()
Fit model:
cnn = cnn_n.fit(x_train, y_train, batch_size=batch_size, epochs=epochs, validation_data=(x_test,y_test),shuffle=True)
All results are for 50k iteration, learning rate=0.01. Neural networks have been trained at 16 cores and 16GB RAM on plon.io
- epochs = 10 accuracy=75.61%
Confusion matrix result:
[[806 9 39 13 28 4 7 9 61 24]
[ 14 870 4 10 3 4 7 0 28 60]
[ 69 1 628 64 122 36 44 19 13 4]
[ 19 5 52 582 109 99 76 29 14 15]
[ 13 2 44 46 761 27 38 62 6 1]
[ 15 1 50 189 69 588 31 48 7 2]
[ 8 3 39 53 52 14 814 4 10 3]
[ 15 3 31 45 63 29 5 795 2 12]
[ 61 13 8 10 17 1 4 4 875 7]
[ 23 52 11 10 7 7 5 12 31 842]]
Confusion matrix vizualizing
Time of learning process: 1h 45min
- epochs = 20 accuracy=75.31%
Confusion matrix result:
[[810 5 30 22 14 2 9 10 60 38]
[ 13 862 7 8 3 6 4 7 20 70]
[ 85 2 626 67 84 44 44 27 12 9]
[ 39 6 47 581 73 137 50 38 17 12]
[ 22 1 52 87 744 34 22 64 2 2]
[ 20 3 40 178 44 639 21 48 2 5]
[ 12 3 42 55 67 16 782 10 7 6]
[ 15 2 24 38 59 37 3 810 5 7]
[ 79 14 10 19 6 4 8 5 827 28]
[ 25 60 8 9 8 5 2 12 21 850]]
Confusion matrix vizualizing
Time of learning process: 3h 40min
- epochs = 50 accuracy=69.93%
Confusion matrix result:
[[760 5 72 32 11 6 12 7 67 28]
[ 12 862 10 16 3 2 18 4 30 43]
[ 55 1 712 67 44 35 47 20 11 8]
[ 37 7 126 554 63 81 69 45 11 7]
[ 23 2 125 86 622 27 36 69 8 2]
[ 20 2 121 201 48 488 56 50 7 7]
[ 16 7 101 65 28 27 734 8 10 4]
[ 16 4 59 60 57 36 9 749 5 5]
[107 13 30 32 3 10 8 6 770 21]
[ 42 100 8 26 8 7 4 21 42 742]]
Confusion matrix vizualizing
Time of learning process: 8h 10min
- epochs = 100 accuracy=68.66%
Confusion matrix result:
[[736 11 54 45 30 14 15 9 61 25]
[ 10 839 6 38 3 13 7 5 22 57]
[ 47 2 566 96 145 65 51 17 7 4]
[ 23 6 56 570 97 140 57 29 12 10]
[ 16 2 52 80 700 55 25 64 3 3]
[ 10 1 64 211 59 582 24 39 6 4]
[ 4 3 42 114 121 40 650 13 5 8]
[ 14 1 40 57 69 68 11 723 3 14]
[ 93 32 26 37 16 15 6 2 752 21]
[ 34 83 8 42 12 21 6 21 25 748]]
Confusion matrix vizualizing
Time of learning process: 17h 10min
OPERATION DATA DIMENSIONS WEIGHTS(N) WEIGHTS(%)
Input ##### 3 32 32
Conv2D \|/ ------------------- 896 0.1%
relu ##### 32 32 32
Conv2D \|/ ------------------- 9248 0.7%
relu ##### 32 30 30
MaxPooling2D Y max ------------------- 0 0.0%
##### 32 15 15
Dropout | || ------------------- 0 0.0%
##### 32 15 15
Conv2D \|/ ------------------- 18496 1.5%
relu ##### 64 15 15
Conv2D \|/ ------------------- 36928 3.0%
relu ##### 64 13 13
MaxPooling2D Y max ------------------- 0 0.0%
##### 64 6 6
Dropout | || ------------------- 0 0.0%
##### 64 6 6
Flatten ||||| ------------------- 0 0.0%
##### 2304
Dense XXXXX ------------------- 1180160 94.3%
relu ##### 512
Dropout | || ------------------- 0 0.0%
##### 512
Dense XXXXX ------------------- 5130 0.4%
softmax ##### 10
model = Sequential()
model.add(Conv2D(32, (3, 3), padding='same', input_shape=x_train.shape[1:]))
model.add(Activation('relu'))
model.add(Conv2D(32,(3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(64, (3, 3), padding='same'))
model.add(Activation('relu'))
model.add(Conv2D(64, (3,3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(512))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(num_classes))
model.add(Activation('softmax'))
sgd = SGD(lr = 0.1, decay=1e-6, nesterov=True)
Train model:
model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])
return model
cnn_n = base_model()
cnn_n.summary()
Fit model:
cnn = cnn_n.fit(x_train, y_train, batch_size=batch_size, epochs=epochs, validation_data=(x_test,y_test),shuffle=True)
All results are for 50k iteration, learning rate=0.1. Neural networks have been trained at 16 cores and 16GB RAM on plon.io
- epochs = 10 accuracy=71.29%
Confusion matrix result:
[[772 5 24 19 17 6 18 10 66 63]
[ 14 637 1 4 3 7 19 2 38 275]
[ 81 0 538 50 88 86 93 27 19 18]
[ 20 1 52 468 60 180 143 33 13 30]
[ 19 1 51 59 662 33 91 66 16 2]
[ 12 0 34 135 37 664 53 41 11 13]
[ 7 0 23 29 26 13 885 2 10 5]
[ 10 0 24 45 48 69 20 756 5 23]
[ 74 4 3 9 4 6 8 4 854 34]
[ 18 8 5 13 9 4 10 6 34 893]]
Confusion matrix vizualizing
Time of learning process: 1h 10min
- epochs = 20 accuracy=74.57%
Confusion matrix result:
[[729 11 56 36 52 5 12 11 58 30]
[ 8 883 1 12 5 1 19 0 11 60]
[ 40 3 545 88 152 54 71 35 7 5]
[ 10 8 30 583 128 106 75 38 10 12]
[ 7 1 15 37 806 9 43 77 5 0]
[ 6 5 18 214 76 586 32 59 2 2]
[ 3 3 23 58 65 7 825 11 4 1]
[ 5 2 12 56 73 24 11 811 0 6]
[ 39 30 11 18 18 5 11 5 847 16]
[ 30 61 3 22 9 2 7 8 16 842]]
Confusion matrix vizualizing
Time of learning process: 2h 15min
- epochs = 50 accuracy=75.32%
Confusion matrix result:
[[727 8 53 25 39 5 4 13 82 44]
[ 7 821 6 10 5 2 4 2 22 121]
[ 46 0 652 66 107 43 40 30 9 7]
[ 14 2 61 577 113 125 40 32 13 23]
[ 7 0 46 39 812 15 20 49 6 6]
[ 3 1 47 150 66 649 17 49 4 14]
[ 1 3 49 69 80 27 751 2 9 9]
[ 9 0 22 47 65 38 8 791 3 17]
[ 32 19 8 21 15 1 6 8 859 31]
[ 19 30 8 14 5 0 2 9 20 893]]
Confusion matrix vizualizing
Time of learning process: 5h 45min
- epochs = 100 accuracy=67.06%
Time of learning process: 11h 10min
Confusion matrix result:
[[599 5 74 98 55 14 12 9 117 17]
[ 16 738 12 65 9 26 7 6 40 81]
[ 31 0 523 168 136 86 33 14 9 0]
[ 10 1 31 652 90 175 19 15 5 2]
[ 6 0 34 132 717 55 16 31 9 0]
[ 5 1 17 233 53 661 10 15 4 1]
[ 2 1 39 157 105 48 637 3 7 1]
[ 6 0 14 97 103 96 5 637 5 1]
[ 41 7 28 84 19 18 6 4 783 10]
[ 25 28 8 77 29 27 5 19 59 723]]
Confusion matrix vizualizing
- Official Keras Documentation
- About Keras on Wikipedia
- About Deep Learning on Wikipedia
- Tutorial by Dr. Jason Brownlee
- Tutorial by Parneet Kaur
- Tutorial by Giuseppe Bonaccorso
- Open Source on GitHub
- You can download code from GitHub
- You can run the project in your browser