Comparing Human and Robot Performance in the Execution of Kitchen Tasks: Evaluating Grasping and Dexterous Manipulation Skills

Over the last decades there has been a lot of research effort focusing on the development of household robots. Such robots need to execute a plethora of complex tasks that require significant dexterity and that need to be employed in dynamic and unstructured environments (e.g., a Kitchen environment). In this work, we focus on comparing human and robot performance in the execution of complex kitchen tasks, assessing the grasping and dexterous manipulation skills that are required. In particular, the study is based on a comprehensive collection of grasping and manipulation strategies that are employed by humans and humans directly operating robots. An unprecedented dataset is created containing more than 2000 activities that are typically executed in a kitchen environment and a total of more than two hours of data. Based on the analysis of this dataset, we propose a taxonomy that classifies the attributes of kitchen specific grasping and manipulation strategies, as well as appropriate benchmarks to compare the performance of robotic grippers against human counterparts using what we call dexterity/capability map. The color-coded maps enable us to visualize the current capabilities and limitations of robotic grippers in the execution of specific tasks. These insights can be used for the development of new classes of grippers and hands capable of performing on par with human hands.

Data and Annotations File

The data and the annotations files can be found at the following URL:

Kitchen Data Google Drive Folder

You can read the papers here and here. If you want to cite the research you can use the following:

@inproceedings{elangovan2023human,
title={On Human Grasping and Manipulation in Kitchens: Automated Annotation, Insights, and Metrics for Effective Data Collection},
author={Elangovan, Nathan and Godoy, Ricardo V and Sanches, Felipe and Wang, Ke and White, Tom and Jarvis, Patrick and Liarokapis, Minas},
booktitle={2023 IEEE International Conference on Robotics and Automation (ICRA)},
pages={11329–11335},
year={2023}}

@inproceedings{elangovan2022comparing,
title={Comparing Human and Robot Performance in the Execution of Kitchen Tasks: Evaluating Grasping and Dexterous Manipulation Skills},
author={Elangovan, Nathan and Chang, Che-Ming and Godoy, Ricardo V and Sanches, Felipe and Wang, Ke and Jarvis, Patrick and Liarokapis, Minas},
booktitle={2022 IEEE-RAS 21st International Conference on Humanoid Robots (Humanoids)},
pages={518–525},
year={2022}}