Elly: A Haptic Learning and Recovery System for Robust Manipulation
IROS 2022
Paper
Code
Data
Even the most robust autonomous behaviors can fail. The goal of this research is to both recover and learn from failures so they can be prevented in the future. We propose haptic intervention for real-time failure recovery and data collection. Elly is a system that allows for seamless transitions between autonomous robot behaviors and human intervention while collecting the necessary sensory information to learn from the human’s recovery strategy. The system and our design choices were experimentally validated on a single arm task -- installing a lightbulb in a socket -- and a bimanual task -- screwing a cap on a bottle -- using two 7-DOF manipulators equipped 4-finger grippers. In these examples, Elly achieved over 80\% task completion during a total of 40 runs.
System Diagram: We collect a small set of human demonstrations of a complete task. The demonstrations are then segmented into a sequence of primitives and used to parametrize the autonomous model-based controllers. In case of failure, we switch to haptic control which enables task completion while providing the operator with real-time contact feedback. Data from the failure recovery strategies is collected and added to the original data set to improve the robot's knowledge of the task. The yellow highlight outlines the interactive haptic learning loop.
Tracking
Contact alignment
Haptics
The development of the controller relied on Simulation and Active Interfaces:
Human Data Labelling and Segmentation:
@article{galbally2022skills,
author = {Galbally, Elena and Piedra, Adrian and Brosque, Cynthia and Chen, Yuxiao and Khatib, Oussama},
title = {Learning From Failures},
conference = {IROS 2022},
year = {2022},
publisher = {IROS}}