This repository is for learning reinforcement learning using PyTorch.

how to use

Set this root dir as PYTHONPATH.

export PYTHONPATH=$(pwd)

Contents

predl

predl contains some algorithms without Deep Learning. Here, PyTorch is just numpy-alternative.

• lookup table

  • Q-Learning
  • SARSA
    • n-step SARSA
    • SARSA(λ)
  • DynaQ (Model Free Approach)

• function approximation

  • Q-learning
  • SARSA

• policy gradient

  • REINFORCE
  • REINFORCE with baseline
  • actor-critic

dl

dl contains some algorithms using DL.

• Deep Q-Network

  cd dl/dqn
  python exec.py [--env Pong ...] # `python exec.py -h` for help
  ---
  tensorboard log_dir runs
  

Dependencies

• gym
  • gym requires a lot depends on your environment thus read the official document before using pip.
• PyTorch
• matplotlib
• tensorboard-pytorch
  • pip install tensorboardX

References

• UCL Course on RL by David Silver, DeepMind.

  • I mainly learned RL in this lecture.
  • Lecture videos are available on YouTube

• Richard S. Sutton and Andrew G. Barto Reinforcement Learning: An Introduction. 2017.

  • online draft is available

• Introduction to reinforcement learning and OpenAI Gym by Justin Francis

  • Good introduction to OpenAI Gym

• 牧野貴樹ほか.これからの強化学習.2016.

  • 基礎から応用まで扱っています