End-To-End Learning from Demonstation for Object Manipulation of Robotis-Op3 Humanoid Robot

Abstract

Humanoid robots are deployed ranging from houses and hotels to healthcare and industry environments to help people. Robots can be easily programed by users to predefined tasks such as walking, grasping, stand-up, and shake-up. However, in these days, all robots are expected to learn itself from the obtained experience by watching the environment and people in there. In this study, it is aimed for Robotis-Op3 humanoid robot to grasp the objects by learning from demonstrations based on vision. A new algorithm is proposed for this purpose. Firstly, the robot is manipulated from user commands and the raw images from the camera of Robotis-Op3 are collected. Secondly, a semantic segmentation algorithm is applied to detect and recognize the objects. A new model using Convolutional Neural Networks (CNNs) and Long Short-Term Memory Networks (LSTMs) is then proposed to learn the user demonstrations. The results were compared in terms of training time, performance, and model complexity. Simulation results showed that new models produced a high performance for object manipulation. © 2020 IEEE.