Makers Final Project:
Our aim was to build a neural net that would learn to play the classic video game Pong! We then implemented this neural net in the backend of our program and connected it to our own version of Pong - built in Javascript!
See it live on Heroku
The Team: Asia Ellis, David Papamichael, Jake Phillips, Nicolas Raffray, Tom Ratcliffe
Clone this repository and in the command line type:
install pip3
pip3 install -r requirements.txt
From the terminal navigate to the root of the directory:
cd Net-Positive-Makers
And run:
python3 net_positive/manage.py runserver
Then open a browser and visit this link localhost::8000
The neural networks we used were heavily based on the work done by Andrej Karpathy in his Reinforcement Learning Blog. Using this neural network training approach we arrived at our most skilled neural net based Pong bot. His work also provided strong guidance for how a net should interact with the pong game and what values it should receive during training. We added additional logging and checkpointing capabilities to the training code to allow for better visibility during the training process.
This team tried different methods of including biases in the net, making it deeper and using Adam optimization instead of RMSprop. We found that the simple single layer without biases yielded far superior results when compared to the combination of the aforementioned specifications. Nevertheless, our version of the neural net is represented in the game as Bjorn Cyborg and its code can be seen in the models section of the project along with Andrej's.
The nets were trained over around 15,000 games of Pong using the OpenAI Gym Python library. After this training their forward propagation code and final network weights were ported into our python backend. This allowed the nets to then play our version of the game which was made to mimic the OpenAI Gym version as far as possible.
Training runs were completed on AWS EC2 instances, we found a C5.2xlarge instance provided a good balance between training speed and cost.
We also added the capability for the neural networks to be trained directly on our own JavaScript version of the game. Although this was found to be around 10 times slower due to the additional latency involved in the websocket communication between the frontend game and backend training code (compared to the approach where the Python training code directly invoked the OpenAI Gym Python library during training).
For Python tests, navigate to the root of the directory:
cd Net-Positive-Makers
And run:
PYTHONPATH=net_positive/ pytest --ds=net_positive.settings
For JavaScript tests, navigate to the root of the directory. After running the following command tests will be visible at localhost:8888
jasmine server
The below diagram shows an example interaction between application components on Arcade mode, where one of the paddles is moved by a bot. On Training mode there would be WebSocket calls made for the second bot also.
