Skip to content

Sawyer environments for reinforcement learning using the OpenAI Gym interface (EXPERIMENTAL)

License

Notifications You must be signed in to change notification settings

rlworkgroup/gym-sawyer

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

15 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

gym-sawyer

Sawyer environments for reinforcement learning that use the OpenAI Gym interface, as well as Dockerfiles with ROS to communicate with the real robot or a simulated one with Gazebo.

This repository is under development, so all code is still experimental.

Docker containers

Sawyer Simulation

We use Gazebo to simulate Sawyer, so a dedicated GPU is required (see System Requirements). Currently only NVIDIA GPUs are supported.

NVIDIA GPU

This section contains instructions to build the docker image and run the docker container for the simulated Sawyer in the ROS environment using an NVIDIA GPU.

Prerequisites

Tested on Ubuntu 16.04

Instructions
  1. In the root folder of your cloned repository build the image by running:
$ make build-nvidia-sawyer-sim
  1. After the image is built, run the container:
$ make run-nvidia-sawyer-sim
  1. Gazebo and MoveIt! should open with Sawyer in them

  2. To exit the container, type sudo docker stop sawyer-sim in a new terminal.

Control Sim Sawyer
  1. Open a new terminal in the container by running:
$ docker exec -it sawyer-sim bash
  1. Once inside the terminal, run the following commands to execute a keyboard controller:
$ cd ~/ros_ws
$ ./intera.sh sim
$ rosrun intera_examples joint_position_keyboard.py
  1. The following message should appear:
Initializing node...
Getting robot state...
[INFO] [1544554222.728405, 33.526000]: Enabling robot...
[INFO] [1544554222.729527, 33.527000]: Robot Enabled
Controlling joints. Press ? for help, Esc to quit.
  1. Type ? to get the keys that control sawyer.
View the header camera in Sim Sawyer
  1. Perform the first step from the previous section
  2. Once in the container shell, run the following command:
rosrun image_view image_view image:=/io/internal_camera/head_camera/image_raw

Sawyer Robot

A dedicated GPU is recommended for rviz and other visualization tools. Currently only NVIDIA GPUs are supported.

NVIDIA GPU

This section contains instructions to build the docker image and run the docker container for the Sawyer robot in the ROS environment using an NVIDIA GPU.

Prerequisites
  • Same as sawyer simulation.
Instructions
  1. Export sawyer hostname, sawyer ip address, and workstation ip address.
$ export SAWYER_HOSTNAME=__sawyerhostname__  
$ export SAWYER_IP=__sawyerip__ 
$ export WORKSTATION_IP=__workip__ 
  1. In the root folder of your cloned repository build the image by running:
$ make build-nvidia-sawyer-robot 
  1. After the image is built, run the container:
$ make run-nvidia-sawyer-robot
  1. Rviz should open with Sawyer in it. Now you can plan and execute trajectories through rviz.

  2. To exit the container, type sudo docker stop sawyer-robot in a new terminal.

Garage-ROS-Intera

To run Reinforcement Learning algorithms along with the Sawyer robot, we use the garage docker images that include an extensive library of utilities and primitives for RL experiments.

On top of the garage images, we add the layers for ROS and Intera that work with Python3 (garage runs with Python3), so we're able to communicate with Sawyer through ROS communication using the convenient libraries for Python rospy, intera_interface, moveit_commander and moveit_msgs.

Under this schema, two docker containers are needed: one for the simulated or real sawyer, and another with garage-ros-intera. The former creates the ROS master while the latter subscribes to the ROS topics to control and visualize what Sawyer is doing through Reinforcement Learning algorithms.

Prerequisites

Tested on Ubuntu 16.04. It's recommended to use the versions indicated above for docker-ce and docker-compose.

Instructions

In the root of the gym-sawyer repository execute:

$ make run-nvidia-sawyer-<type>

Where type can be sim (run simulated Sawyer on Gazebo) or robot (you're connected to a real Sawyer).

Once the container is up and running (make sure Gazebo is fully initialized if running simulated Sawyer). Then run:

$ make run-garage-<type>-ros RUN_CMD="examples/hello_world_sawyer.py"

Where type can be:

  • headless: garage without environment visualization.
  • nvidia: garage with environment visualization using an NVIDIA graphics card. If your computer has an NVIDIA GPU, use this image to render the environments in garage, and the pre-requisites are the same as for the Sawyer Simulation image.

You should see Sawyer moving to neutral position and then waving its arm three times.

The command to execute in the image is specified in the variable RUN_CMD.

Run your local repository of garage with ros-intera

If you're working with garage in your local repository and would like to include your latests changes, follow these instructions.

  1. Make sure to run build your docker image for garage first. For further information, visit garage.
  2. Then rebuild and run the garage-ros image:
$ make run-garage-<type>-ros RUN_CMD="..."

About

Sawyer environments for reinforcement learning using the OpenAI Gym interface (EXPERIMENTAL)

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published