- An interview test of Ideta company.
- Performed by Anh-Thi DINH.
- Date: 13-Jan-2021.
- 2 Classes: "moutain" (and its related: mainland, forest, moutain path,...) and "beach" (and its related: sea, ocean, beach house,...)
- Dataset: extracted (~369MB) from MIT's places dataset. It contains ~15K images for each category.
- Method: using transfer learning and CNN.
- Deploy: a web-app at dinhanhthi.github.io/mountain-vs-beach using TensorFlow.js.
- Jupyter notebook: view in HTML format.
- Result: 94.29% accuracy.
- Overview the task: Number of steps? Possible approaches? Available frameworks and tools? Data?
- Find and get dataset: extracted (~369MB) from MIT's places dataset (26GB). It contains ~15K images for each category. Remark: There are categories
beach
andmountain
in the original dataset but there are only ~5K images for each. That's why I use also other "related classes" to get more data, i.e.- "beach related" = "beach" + "beach_house" + "ocean".
- "mountain related" = "mountain" + "mountain_path" + "mountain_snowy".
- Choose appropriate methods & tools:
-
Framework: TensorFlow
-
Final model: use transfer learning from a pretrained InceptionV3 model with "imagenet" weights + a few more layers.
# WARNING: check the notebook for full parameters # Pretrained InceptionV3 # up to layer "mixed7" ((None, 10, 10, 768)) # We don't train these layers, just use the last output. pre_trained_model = InceptionV3(weights = "imagenet") last_output = pre_trained_model.get_layer('mixed7').output # Add more layers x = layers.Flatten()(last_output) x = layers.Dense(512, activation='relu')(x) x = layers.Dropout(0.35)(x) # prevent "overfitting" x = layers.Dense (1, activation='sigmoid')(x) # Model model = Model(pre_trained_model.input, x) model.compile( optimizer = RMSprop(lr=1e-5), loss = 'binary_crossentropy', metrics = ['accuracy']) # Train with epochs=100, batch_size=50
-
Machine: try both Google Colab and personal laptop (Dell XPS 15 7950 with 32GB RAM, 4GB GPU and Intel® Core™ i7-9750H CPU @ 2.60GHz × 12).
-
Final score: 94.29% accuracy.
-
- Already tried approaches:
- Use directly data: the "right" images of beach and mountain. However, there are not many data (5K for each category). Only 90% accuracy or underfitting/overfitting sometimes. That's why I use "other related" images (15K for each category).
- Smaller input size (
150x150
, for quicker training): smaller accuracy. - Using VGG16 places365 pretrained weights (instead of InceptionV3 + "imagenet"): only 88% accuracy.
- Using smaller dropout (
0.2
instead of0.35
): smaller accuracy. - Using bigger input size (
224x224
), smaller learning rate or biggerbatch_size
: not enough GPU memory or time to train the model.
- Deployment:
- Result: dinhanhthi.github.io/mountain-vs-beach.
- Chosen approach (serverless): using TensorFlow.js to load weights + trained model from
.json
and.bin
files + static site + hosted using Github Pages (free).- Weakness: trained model is big (193MB), it takes too much time for users to load the site.
- Solution: Using quantization technique in tfjs-converter to reduce the model from 193MB to 48MB (it's also big but better).
- Quantization technique: compress our model parameters from Float32s (4 bytes) to Uint8s (single bytes) by mapping each tensors’ value from the range [min value, max value] represented by 255⁴ bits to [0, 255] represented by 255 bits.
- This converter will convert a saved model (exported by TensorFlow Core) to a format of TensorFlow.js. The converted format contains splitted 4MB
.bin
files - that way they can be cached by browsers. That's why the users have to wait only once when load the page at the first time, other times, it's quick! - Weakness: we lose accuracy (may be not much, may be much).
- Other options (server side):
- Frontend: Bootstrap + HTML + CSS + Vanilla JS.
- 3 ways to input an image:
- Upload from local (recommended).
- Choose example images right on the page.
Paste from URL of image (must follow a good format, otherwise, it will not work!). Acceptable URLs are.jpeg
,.png
,.jpg
orbase64
urls.
- Technical problem with URL option: TFJS's
fromPixels()
method is causing a CORS error (we cannot fetch the image via an intermediate site like our web-app). - The meaning of results: because our problem is a binary classification. I use
sigmoid
to get the probability of thescore
. Thisscore
is between0
(beach) and1
(mountain).- It's a mountain if
score > 0.65
and we outputscore*100
%. - It's a beach if
score < 0.35
and we output(1-score)*100
%. - Not sure result:
score
between[0.35, 0.65]
. It may come from your input image (not good format, cannot fetch URL, there are both beach and mountain in the image, there aren't either mountain or beach in the image,...).
- It's a mountain if
- 3 ways to input an image: