DexPilot: Vision-Based Teleoperation of Dexterous Robotic Hand-Arm System

May 2020

DexPilot: Vision-Based Teleoperation of Dexterous Robotic Hand-Arm System

Authors:

Ankur Handa, Karl Van Wyk, Wei Yang, Jacky Liang, Yu-Wei Chao, Qian Wan, Stan Birchfield, Nathan D. Ratliff, and Dieter Fox
Conference Paper
Proceedings of (ICRA) International Conference on Robotics and Automation

Abstract:

Teleoperation offers the possibility of imparting robotic systems with sophisticated reasoning skills, intuition, and creativity to perform tasks. However, teleoperation solutions for high degree-of-actuation (DoA), multi-fingered robots are generally cost-prohibitive, while low-cost offerings usually offer reduced degrees of control. Herein, a low-cost, depth-based teleoperation system, DexPilot, was developed that allows for complete control over the full 23 DoA robotic system by merely observing the bare human hand. DexPilot enabled operators to solve a variety of complex manipulation tasks that go beyond simple pick-and-place operations and performance was measured through speed and reliability metrics. DexPilot cost-effectively enables the production of high dimensional, multi-modality, state-action data that can be leveraged in the future to learn sensorimotor policies for challenging manipulation tasks. The videos of the experiments can be found at:
https://sites.google.com/view/dex-pilot.
@conference{Handa-2020-126758,
author = {Ankur Handa And Karl Van Wyk And Wei Yang And Jacky Liang And Yu-Wei Chao And Qian Wan And Stan Birchfield And Nathan D. Ratliff And Dieter Fox},
title = {DexPilot: Vision-Based Teleoperation of Dexterous Robotic Hand-Arm System},
booktitle = {Proceedings of (ICRA) International Conference on Robotics and Automation},
year = {2020},
month = {May},
pages = {9164 - 9170},
}
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.