The feeling of success: Does touch sensing help predict grasp outcomes?

November 2017

The feeling of success: Does touch sensing help predict grasp outcomes?

Authors:

Roberto Calandra, Andrew Owens, Manu Upadhyaya, Wenzhen Yuan, Justin Lin, Edward Adelson, and Sergey Levine
Conference Paper
Proceedings of (CoRL) Conference on Robot Learning

Abstract:

A successful grasp requires careful balancing of the contact forces. Deducing whether a particular grasp will be successful from indirect measurements, such as vision, is therefore quite challenging, and direct sensing of contacts through touch sensing provides an appealing avenue toward more successful and consistent robotic grasping. However, in order to fully evaluate the value of touch sensing for grasp outcome prediction, we must understand how touch sensing can influence outcome prediction accuracy when combined with other modalities. Doing so using conventional model-based techniques is exceptionally difficult. In this work, we investigate the question of whether touch sensing aids in predicting grasp outcomes within a multimodal sensing framework that combines vision and touch. To that end, we collected more than 9,000 grasping trials using a two-finger gripper equipped with GelSight high-resolution tactile sensors on each finger, and evaluated visuo-tactile deep neural network models to directly predict grasp outcomes from either modality individually, and from both modalities together. Our experimental results indicate that incorporating tactile readings substantially improve grasping performance.
@conference{Calandra-2017-119924,
author = {Roberto Calandra And Andrew Owens And Manu Upadhyaya And Wenzhen Yuan And Justin Lin And Edward Adelson And Sergey Levine},
title = {The feeling of success: Does touch sensing help predict grasp outcomes?},
booktitle = {Proceedings of (CoRL) Conference on Robot Learning},
year = {2017},
month = {November},
pages = {314 - 323},
}
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.