Continuous Action Deep Reinforcement Learning for Dexterous Robotic Manipulation

Learning to execute a plethora of complex, dexterous manipulation tasks efficiently has been a challenging and still unsolved problem in robotics. Many studies have tried to solve this problem; however, their solutions rely on simulations that are hard to obtain and difficult to scale into real-world applications. In this paper, we propose an open-source robotic manipulation testbed that consists of a fully-actuated, 4-DoF, two-fingered robot gripper and a series of sensorized objects as well as a reinforcement learning methodology that utilizes the testbed to learn the execution of dexterous manipulation tasks. In particular, two state-of-the-art deep reinforcement learning algorithms DDPG and TD3, are adapted and tested to understand their limitations in operating with multiple continuous actions. Experimental results using two different state-space representations show that the proposed methodology can be employed to execute a series of dexterous manipulation tasks efficiently. The source code and the CAD files of the robotic manipulation testbed are provided to facilitate replication of the results and further experimentation by other research groups.