+0
Daily
5,938
stars
1,206
forks
102
watchers
20
prs
71
issues
0
releases
README
Deep Reinforcement Learning Algorithms with PyTorch
!Travis CI
!RL !PyTorch
This repository contains PyTorch implementations of deep reinforcement learning algorithms and environments.
(To help you remember things you learn about machine learning in general write them in Gizmo)
Algorithms Implemented
1. Deep Q Learning (DQN) (Mnih et al. 2013)
1. DQN with Fixed Q Targets (Mnih et al. 2013)
1. Double DQN (DDQN) (Hado van Hasselt et al. 2015)
1. DDQN with Prioritised Experience Replay (Schaul et al. 2016)
1. Dueling DDQN (Wang et al. 2016)
1. REINFORCE (Williams et al. 1992)
1. Deep Deterministic Policy Gradients (DDPG) (Lillicrap et al. 2016 )
1. Twin Delayed Deep Deterministic Policy Gradients (TD3) (Fujimoto et al. 2018)
1. Soft Actor-Critic (SAC) (Haarnoja et al. 2018)
1. Soft Actor-Critic for Discrete Actions (SAC-Discrete) (Christodoulou 2019)
1. Asynchronous Advantage Actor Critic (A3C) (Mnih et al. 2016)
1. Syncrhonous Advantage Actor Critic (A2C)
1. Proximal Policy Optimisation (PPO) (Schulman et al. 2017)
1. DQN with Hindsight Experience Replay (DQN-HER) (Andrychowicz et al. 2018)
1. DDPG with Hindsight Experience Replay (DDPG-HER) (Andrychowicz et al. 2018 )
1. Hierarchical-DQN (h-DQN) (Kulkarni et al. 2016)
1. Stochastic NNs for Hierarchical Reinforcement Learning (SNN-HRL) (Florensa et al. 2017)
1. Diversity Is All You Need (DIAYN) (Eyensbach et al. 2018)
All implementations are able to quickly solve Cart Pole (discrete actions), Mountain Car Continuous (continuous actions),
Bit Flipping (discrete actions with dynamic goals) or Fetch Reach (continuous actions with dynamic goals). I plan to add more hierarchical RL algorithms soon.
Environments Implemented
1. Bit Flipping Game (as described in Andrychowicz et al. 2018)
1. Four Rooms Game (as described in Sutton et al. 1998)
1. Long Corridor Game (as described in Kulkarni et al. 2016)
1. Ant-{Maze, Push, Fall} (as desribed in Nachum et al. 2018 and their accompanying code)
Results
1. Cart Pole and Mountain Car
Below shows various RL algorithms successfully learning discrete action game Cart Pole
or continuous action game Mountain Car. The mean result from running the algorithms
with 3 random seeds is shown with the shaded area representing plus and minus 1 standard deviation. Hyperparameters
used can be found in files results/Cart_Pole.py and results/Mountain_Car.py.
!Cart Pole and Mountain Car Results
2. Hindsight Experience Replay (HER) Experiements
Below shows the performance of DQN and DDPG with and without Hindsight Experience Replay (HER) in the Bit Flipping (14 bits)
and Fetch Reach environments described in the papers Hindsight Experience Replay 2018
and Multi-Goal Reinforcement Learning 2018. The results replicate the results found in
the papers and show how adding HER can allow an agent to solve problems that it otherwise would not be able to solve at all. Note that the same hyperparameters wereTop issues
#31RuntimeError: Expected object of type torch.FloatTensor but found type torch.cuda.FloatTensor for argument #4 'mat1'9 comments · 2019-10-25#85A question about critic-loss in discrete sac?5 comments · 2022-10-08#10 add PPO-HER5 comments · 2019-04-09#80torch版本问题4 comments · 2022-05-04#52SpaceInvaders(SAC_Discrete) : Error4 comments · 2020-04-29#77terminate called after throwing an instance of 'at::Error' what(): CUDA error (3): initialization error (check_status at /pytorch/aten/src/ATen/cuda/detail/CUDAHooks.cpp:36)3 comments · 2021-11-08#29how to visualize the enviroment3 comments · 2019-10-24#15Cart_Pole.py fails when running cuda9.0 gpu3 comments · 2019-04-21