This video explains the pain points associated with single-machine training for RL environments. It contains a detailed walkthrough of an experiment conducted on Azure ML in a distributed fashion using the 'APEX' algorithm on a custom GYM environment using Ray's RLLIB.
Source Code: github.com/sriksmachi/supercabs
Article on distributed RL algorithms: towardsdatascience.com/accelerate-training-in-rl-using-distributed-reinforcement-learning-architectures-b5a726b49826