Skip to content

Reinforcement learning with unsupervised auxiliary tasks

License

Notifications You must be signed in to change notification settings

miyosuda/unreal

Repository files navigation

UNREAL

CircleCI

About

Replicating UNREAL algorithm described in Google Deep Mind's paper "Reinforcement learning with unsupervised auxiliary tasks."

https://arxiv.org/pdf/1611.05397.pdf

Implemented with TensorFlow and DeepMind Lab environment.

Preview

seekavoid_arena_01

seekavoid_arena_01

stairway_to_melon

stairway_to_melon

nav_maze_static_01

nav_maze_static_01

Network

Network

All weights of convolution layers and LSTM layer are shared.

Requirements

  • TensorFlow (Tested with r1.0)
  • DeepMind Lab
  • numpy
  • cv2
  • pygame
  • matplotlib

Result

"seekavoid_arena_01" Level

seekavoid_01_score

"nav_maze_static_01" Level

nav_maze_static_01_score

How to train

First, download and install DeepMind Lab

$ git clone https://github.com/deepmind/lab.git

Then build it following the build instruction. https://github.com/deepmind/lab/blob/master/docs/build.md

Clone this repo in lab directory.

$ cd lab
$ git clone https://github.com/miyosuda/unreal.git

Add this bazel instruction at the end of lab/BUILD file

package(default_visibility = ["//visibility:public"])

Then run bazel command to run training.

bazel run //unreal:train --define headless=glx

--define headlesss=glx uses GPU rendering and it requires display not to sleep. (We need to disable display sleep.)

If you have any trouble with GPU rendering, please use software rendering with --define headless=osmesa option.

How to show result

To show result after training, run this command.

bazel run //unreal:display --define headless=glx

About

Reinforcement learning with unsupervised auxiliary tasks

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published