Virtual ML.NET 2021 Hackathon Submission

Virtual on November 12-19, 2021

Team members

• Jim SEOW
• Shehroze Malik
• Praveen Raghuvanshi

What we have done:

Deliverable 1:

A workflow to train PyTorch OpenAI Gym (stable-baselines3 or SB3) model using Godot environment on Windows

The provided GymGodot is being tested ONLY on Linux & Godot 3.3

• We hacked the provided python script solution to make them work on Windows

Deliverable 2:

A working example how to export SB3 ONNX

• We learned that it was not trivial to simply export PyTorch model to ONNX due to no broadcast_tensors opperator support discussed here
• We follow the instruction and added extra pytorch codes and attached model_to_onnx.py to export the trained pytorch model to StarShipLander.ONNX

Deliverable 3:

A .NET WebSocket Server serving RL inference using SB3 ONNX applicable for e.g. Unity3D

• We replace the provided python Gym_server with godot_net_server
• This removed dependency on python for RL inference.

Deliverable 4:

An example how to integrate ML.NET in Godot

• We removed the need of having two applications: a UI client and a (python/.NET) server serving RL inference by implementing a single application solution: GymRogueOne

Reference: the provided GymGodot's rocket model is replaced with a SpaceX Starship model by MartianDays under Creative Commons Attribution

Deliverable 5:

A workflow to possibly make SEVEN OpenAI Gym RL algorithms accessible throgh ML.NET

For more information: mlnet_findings.pptx

• A2C: Asynchronous Advantage Actor Critic.
• DDPG: Deep Deterministic Policy Gradient.
• DQN: Deep Q Network.
• HER: Hindsight Experience Replay.
• PPO: Proximal Policy Optimization. e.g. GymRogueOne
• SAC: Soft Actor Critic. - tested
• TD3: Twin Delayed DDPG.

How to run the GymRogueOne solution? (Quick Start)

Step 1 - Git clone and download the missing large size file

git clone https://github.com/JimFFM/ml-hackathon-2021
cd ml-hackathon-2021/GymRogueOne/StarShip/

Download the missing large size file: Starship.material into the StarShip folder, according to the link provided here

wget https://drive.google.com/file/d/1pekt-xm665w6h77tOVHtmUjPeRDq5jF4/view?usp=sharing

Step 2 - Download Godot

• Godot download page
• In our case, we downloaded the Godot Mono x86-64 bit zip file
• Unzip the Godot application (~81 MB)

Step 3 - Select the project.godot file and click run

• A window shows up to allow you to import the Godot project file

• Navigate to the folder ml-hackathon-2021/GymRogueOne/ and select project.godot
• Go to top right corner and click play

Step 4 Play with these parameters to achieve soft landing

More: Longer starship landing video