Brief Introduction
Prerequisite
Starting the project
Getting the data
Training
Predicting
More Details
Development with docker on Windows
Troubleshoot
This sample project will look into:
- generating images of certain shapes with labels
- create and train a simple/convoluted neural network model using tensorflow
- predicting a shape drawn by a user using the trained model.
- Docker
If you don't want to use Docker containers:
- Node / NPM
- Python 3
- Tensorflow
But the instructions here will be using Docker. Additional code changes or configuration may be required if you are not using Docker.
If you are using Windows 10, check below for additional set up.
First, clone the project.
git clone https://github.com/dakotaJang/shapes.git
Then run docker-compose in project directory:
cd shapes
docker-compose up
This will start 4 services.
- shapes-generator
- shapes-trainer
- shapes-predictor
- shapes-model
-
Open web browser (Chrome) and go to http://localhost:8081
Here we can simulate and download shapes using 100px by 100px canvas. Each canvas generates different shapes and you can specify how many simulations you would like to download.
You go to
shapes-generator
to view the polymer based components and modify to create shapes that you desire. -
Click "Download" button(s) to simulate and generate images to download.
-
After you download, place the zip files in the
shapes-data
folder.
-
Open a new tab on web browser and go to http://localhost:8082
Here we will see the contents of
/shapes-trainer
. -
Open the
train.ipynb
notebook file.Follow the instructions in the notebook.
You can run the commands by clicking the code block and clicking the
Run
button near the top of the page.
-
Enter following command to get in to container's terminal
docker exec -it shapes-trainer /bin/bash
-
Follow the instruction found in the note book
By the end of the notebook, you should have the trained model(s) created in the
shapes-model
folder.
-
Open a new tab on web browser and go to http://localhost:8084
-
Load model by entering the first input with folder name that contains the model.json in the shapes-model folder.
-
Click
Load Model
button to load the model. -
Once the model have been loaded successfully, you can start drawing on canvas to see if the model can predict accurately.
- Polymer CLI
- Installed polymer-cli on node:slim to keep the image size to a minimum.
- Links: Docker Hub, GitHub
- Tensorflow
- Installed TensorflowJS on top of Tensorflow image for web compatibility
- Links: Docker Hub, GitHub
Docker is more suited to work with linux environment. So using Docker on windows can be troublesome.
-
had problem using the bind mount for development on WINDOWS 10.
- fixed it by one of two things:
- Open Docker Settings > Shared Drives > Reset credentials...
- seems like changing my password caused the problem
- probably need to reset credentials again if I changed the password in the future
- May need to set environment variable
SETX COMPOSE_CONVERT_WINDOWS_PATHS 1
- Open Docker Settings > Shared Drives > Reset credentials...
- fixed it by one of two things:
-
To do some development with docker on windows we need to start
docker-volume-watcher
First install the toolpip install docker-windows-volume-watcher
Run the command with target container
docker-volume-watcher shapes-generator
docker-volume-watcher shapes-predictor
-
If the docker fails to run
- Open Hyper-V Manager
- Select your virtual machine (if not selected)
- Click "Stop Service" under Actions tab
- Click "Turn Off" on the prompt window
- Click "Start Service" under Actions tab
- Start Docker
- If this doesn't work, try restarting the your machine
-
No model created
- If there are no folder(s) with
model.json
inshapes-model
folder, but there is a.h5
file inshapes-model
folder then training was successful but couldn't create a web friendly version of the model. Probably due to not enough memory. We need to convert the keras model manually.
tensorflowjs_converter \ --input_format=keras \ /shapes-model/my_keras_model.h5 \ /shapes-model/my_tfjs_model
- If there are nothing created in
shapes-model
folder then training probably failed.- check if training had error
- if process was
Killed
- improve memory management
- increase memory
- train model on the host machine
- If there are no folder(s) with
-
If you get a error similar to:
shapes-predictor | npm ERR! enoent ENOENT: no such file or directory, open '/shapes-predictor/package.json'
- This is probably because the Volume is not setup correctly. Maybe due to changed password. Rest credentials in Docker > Settings > Shared Drives
- Docker: https://www.docker.com
- Polymer: https://www.polymer-project.org
- Tensorflow: https://www.tensorflow.org
- TensorflowJS: https://js.tensorflow.org
- Docker Windows Volume Watcher: https://github.com/merofeev/docker-windows-volume-watcher