Towards a Robust Interactive and Learning Social Robot

July 2018

Towards a Robust Interactive and Learning Social Robot

Authors:

Michiel De Jong, Kevin Zhang, Aaron Roth, Travers Rhodes, Robin Schmucker, Chenghui Zhou, Sofia Ferreira, Joao Cartucho, and Manuela Veloso

Abstract:

Pepper is a humanoid robot, specifically designed for social interaction, that has been deployed in a variety of public environments. A programmable version of Pepper is also available, enabling our focused research on perception and behavior robustness and capabilities of an interactive social robot. We address Pepper perception by integrating state-of-the-art vision and speech recognition systems and experimentally analyzing their effectiveness. As we recognize limitations of the individual perceptual modalities, we introduce a multi-modality approach to increase the robustness of human social interaction with the robot. We combine vision, gesture, speech, and input from an onboard tablet, a remote mobile phone, and external microphones. Our approach includes the proactive seeking of input from a different modality, adding robustness to the failures of the separate components. We also introduce a learning algorithm to improve communication capabilities over time, updating speech recognition through social interactions. Finally, we realize the rich robot body-sensory data and introduce both a nearest-neighbor and a deep learning approach to enable Pepper to classify and speak up a variety of its own body motions. We view the contributions of our work to be relevant both to Pepper specifically and to other general social robots.
@conference{de Jong-2018-110965,
author = {Michiel De Jong And Kevin Zhang And Aaron Roth And Travers Rhodes And Robin Schmucker And Chenghui Zhou And Sofia Ferreira And Joao Cartucho And Manuela Veloso},
title = {Towards a Robust Interactive and Learning Social Robot},
booktitle = {Proceedings of 17th International Conference on Autonomous Agents and MultiAgent Systems (AAMAS '18)},
year = {2018},
month = {July},
editor = {M. Dastani, G. Sukthankar, E. André, S. Koenig},
pages = {883 - 891},
publisher = {International Foundation for Autonomous Agents and Multiagent Systems (www.ifaamas.org)},
keywords = {Human-Robot Interaction, Robot Autonomy, Robot Machine Learning, Pepper, Service Robot, Social Robot},
}
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.