Back to Search

ACNMP: skill transfer and task extrapolation through learning from demonstration and reinforcement learning via representation sharing

Title	ACNMP: skill transfer and task extrapolation through learning from demonstration and reinforcement learning via representation sharing
Author	Akbulut, M. T., Öztop, Erhan, Xue, H., Tekden, A. E., Şeker, M. Y., Uğur, E.
Publication Date:	2020
Publication Place	- ML Research Press
Subject	Deep learning, Learning from demonstration, Reinforcement learning, Representation learning
Type	Document
Language	English
Digital	Yes
Manuscript	No
Library:	Özyeğin University
Library Asset ID	2640-3498
Record ID	bf595557-5612-421f-b1d6-80b31ea0b084
Library Location	Computer Science
Date	2020
Notes	Horizon 2020 Framework Programme ; Core Research for Evolutional Science and Technology ; Osaka University ; TÜBİTAK
Sample Text	To equip robots with dexterous skills, an effective approach is to first transfer the desired skill via Learning from Demonstration (LfD), then let the robot improve it by self-exploration via Reinforcement Learning (RL). In this paper, we propose a novel LfD+RL framework, namely Adaptive Conditional Neural Movement Primitives (ACNMP), that allows efficient policy improvement in novel environments and effective skill transfer between different agents. This is achieved through exploiting the latent representation learned by the underlying Conditional Neural Process (CNP) model, and simultaneous training of the model with supervised learning (SL) for acquiring the demonstrated trajectories and via RL for new trajectory discovery. Through simulation experiments, we show that (i) ACNMP enables the system to extrapolate to situations where pure LfD fails; (ii) Simultaneous training of the system through SL and RL preserves the shape of demonstrations while adapting to novel situations due to the shared representations used by both learners; (iii) ACNMP enables order-of-magnitude sample-efficient RL in extrapolation of reaching tasks compared to the existing approaches; (iv) ACNMPs can be used to implement skill transfer between robots having different morphology, with competitive learning speeds and importantly with less number of assumptions compared to the state-of-the-art approaches. Finally, we show the real-world suitability of ACNMPs through real robot experiments that involve obstacle avoidance, pick and place and pouring actions.
Cilt	155

View in source Özyeğin University Özyeğin University - Historical works, archives, and periodicals search engine

Özyeğin University

ACNMP: skill transfer and task extrapolation through learning from demonstration and reinforcement learning via representation sharing

Author Akbulut, M. T., Öztop, Erhan, Xue, H., Tekden, A. E., Şeker, M. Y., Uğur, E.

Publication Date 2020

Publication Place - ML Research Press

Subject Deep learning, Learning from demonstration, Reinforcement learning, Representation learning

Type Document

Language English

Digital Yes

Manuscript No

Library Özyeğin University

Library Asset ID 2640-3498

Record ID bf595557-5612-421f-b1d6-80b31ea0b084

Library Location Computer Science

Date 2020

Notes Horizon 2020 Framework Programme ; Core Research for Evolutional Science and Technology ; Osaka University ; TÜBİTAK

Sample Text To equip robots with dexterous skills, an effective approach is to first transfer the desired skill via Learning from Demonstration (LfD), then let the robot improve it by self-exploration via Reinforcement Learning (RL). In this paper, we propose a novel LfD+RL framework, namely Adaptive Conditional Neural Movement Primitives (ACNMP), that allows efficient policy improvement in novel environments and effective skill transfer between different agents. This is achieved through exploiting the latent representation learned by the underlying Conditional Neural Process (CNP) model, and simultaneous training of the model with supervised learning (SL) for acquiring the demonstrated trajectories and via RL for new trajectory discovery. Through simulation experiments, we show that (i) ACNMP enables the system to extrapolate to situations where pure LfD fails; (ii) Simultaneous training of the system through SL and RL preserves the shape of demonstrations while adapting to novel situations due to the shared representations used by both learners; (iii) ACNMP enables order-of-magnitude sample-efficient RL in extrapolation of reaching tasks compared to the existing approaches; (iv) ACNMPs can be used to implement skill transfer between robots having different morphology, with competitive learning speeds and importantly with less number of assumptions compared to the state-of-the-art approaches. Finally, we show the real-world suitability of ACNMPs through real robot experiments that involve obstacle avoidance, pick and place and pouring actions.

Cilt 155