Capture that flag Pacman

Authors

• Johan Hammarstedt, jhammarstedt
• Lukas Vordemann, vordemann

Task

To build a team to play CTF- Pacman, created by the competition hosted at Berkley. This was built as a school assignment for the class DD2438 at KTH with an inter-class tournament, which we ended up winning.

Initial idea

Our orignal idea was to train a DQL agent to play capture the flag pacman using CNN + fully connected layers. However, the setup for the agent is there, but due to the short deadline we didn't have time to train it sufficiently and as a result very little learning was done.

Final team

As a backup to the DQL agent we also added a heuristic based team, that ended up performing really well and won the final tournament. Setup:

• Defensive Agent:
  • Uses A* to get all paths
  • Tracks enemy movement by checking where food disapears (since you don't have vision of enemies if they're too far away)
  • Rush to capsules at beginning to prevent it from being stolen
• Offensive Agent:
  • Find closest pill with A*
  • "Risk Assesment", will only eat aroud 5 foods at the time to avoid overextending
  • If chased it will compute the best path to base using A*, or check for a closer capsule while avoiding enemies
  • A flanking manouver to avoid getting stuck with enemies

Run

Use the following command to run the code for the heuristic vs a baseline team $ python capture.py -r heuristicTeam -b baseline -l RANDOM50

To train the DQN agent: $ python capture.py -r DQN_agent -b baseline -n 5000 -x 4000 -l RANDOM99

• -n is number of episodes
• -x is number of trainings
• -l choose a random grid

Game enviroment

Credits to cshelton for rewriting the enviroment to support python 3, you can find the repository here

Useful resources:

• CFT Pacman
• starcraft RL
• RL in pytorch: Mario, Flappy Bird
• Keras RL
• MARL on github
• Pacman DQL, repo