openai /baselines
OpenAI Baselines: high-quality implementations of reinforcement learning algorithms
★ 16,780⑂ 4,916👁 621⑃ 376● 850▤ 0
+0
Daily
16,780
stars
4,916
forks
621
watchers
376
prs
850
issues
0
releases
README
Status: Maintenance (expect bug fixes and minor updates)
Baselines
OpenAI Baselines is a set of high-quality implementations of reinforcement learning algorithms.
These algorithms will make it easier for the research community to replicate, refine, and identify new ideas, and will create good baselines to build research on top of. Our DQN implementation and its variants are roughly on par with the scores in published papers. We expect they will be used as a base around which new ideas can be added, and as a tool for comparing a new approach against existing ones.
Prerequisites
Baselines requires python3 (=3.5) with the development headers. You'll also need system packages CMake, OpenMPI and zlib. Those can be installed as follows
Ubuntu
bash
sudo apt-get update && sudo apt-get install cmake libopenmpi-dev python3-dev zlib1g-dev
Mac OS X
Installation of system packages on Mac requires Homebrew. With Homebrew installed, run the following:
bash
brew install cmake openmpi
Virtual environment
From the general python package sanity perspective, it is a good idea to use virtual environments (virtualenvs) to make sure packages from different projects do not interfere with each other. You can install virtualenv (which is itself a pip package) via
bash
pip install virtualenv
Virtualenvs are essentially folders that have copies of python executable and all python packages.
To create a virtualenv called venv with python3, one runs
bash
virtualenv /path/to/venv --python=python3
To activate a virtualenv:
. /path/to/venv/bin/activate
More thorough tutorial on virtualenvs and options can be found here
Tensorflow versions
The master branch supports Tensorflow from version 1.4 to 1.14. For Tensorflow 2.0 support, please use tf2 branch.
Installation
- Clone the repo and cd into it:
bash
git clone https://github.com/openai/baselines.git
cd baselines
- If you don't have TensorFlow installed already, install your favourite flavor of TensorFlow. In most cases, you may use
bash
pip install tensorflow-gpu==1.14 if you have a CUDA-compatible gpu and proper drivers
or
bash
pip install tensorflow==1.14
to install Tensorflow 1.14, which is the latest version of Tensorflow supported by the master branch. Refer to TensorFlow installation guide
for more details.
- Install baselines package
bash
pip install -e .
MuJoCo
Some of the baselines examples use MuJoCo (multi-joint dynamics in contact) physics simulator, which is proprietary and requires binaries and a license (temporary 30-day license can be obtained from www.mujoco.org). Instructions on setting up MuJoCo can be found here
Testing the installation
All unit tests in baselines can be run using pytest runner:
pip install pytest
pytest
Training models
Most of the algorithms in baselines repo are used as follows:
bash
python -m baselines.run --alg= --env= [additional arguments]
Example 1. PPO with MuTop issues
#121Taking action limits into account in PPO/TRPO/ACKTR.37 comments · 2017-08-29#431Default hyperparamters for `run_atari.py` (using P-DDDQN) fail with Pong and Breakout (log files attached)34 comments · 2018-06-08#162Saving and restoring DDPG agent26 comments · 2017-10-09#938DDPG implementation fails to learn well on at least five MuJoCo-v2 envs for all three noise types. I report steps to reproduce and learning curve plots [and show that PPO2 seems to work fine].21 comments · 2019-06-20#596About Tensorboard21 comments · 2018-09-17#176Cannot reproduce Breakout benchmark using Double DQN15 comments · 2017-10-20#608`SubprocVecEnv` speedup does not scale linearly compared with `DummyVecEnv`14 comments · 2018-09-21#240Reference for DeepMind using FireResetEnv wrapper's functionality14 comments · 2017-12-30