Vision-based Teleoperation of Shadow Dexterous Hand using End-to-End Deep Neural Network
In this paper, we present TeachNet, a novel neural network architecture for intuitive and markerless vision-based teleoperation of dexterous robotic hands. Robot joint angles are directly generated from depth images of the human hand that produce visually similar robot hand poses in an end-to-end fashion.
Evidence notes
- TeachNet generates robot joint angles directly from human-hand depth images and uses a consistency loss to bridge anatomical and visual differences between human and robotic hands.
- Training uses 400,000 paired human and simulated robot depth images with joint angles for a Shadow C6 hand.
- Imitation and grasping trials with novice users show faster and more reliable teleoperation than the compared vision-based method.
Company context
Shadow Robot develops dexterous robotic hands, tactile sensing and teleoperation systems for manipulation research, embodied-AI labs and advanced robot-control work.