Bimanual Grasp Synthesis for Dexterous Robot Hands

Yanming Shao, Chenxi Xiao*,
ShanghaiTech University
RA-L 24' | Transferred to ICRA 25'
*Corresponding Author
Paper Arxiv Dataset & Visualization

Teaser Figure

input

Bimanual manipulation is necessary for handling large and heavy objects (e.g., basins, kitchen appliances). These objects would otherwise be difficult to handle using single-handed manipulation due to imbalanced contact forces and torques.


Abstract

Humans naturally perform bimanual skills to handle large and heavy objects. To enhance a robot's object manipulation capabilities, generating effective bimanual grasp poses is essential. Nevertheless, bimanual grasp synthesis for dexterous hand manipulators remains underexplored. To bridge this gap, we propose the BimanGrasp algorithm for synthesizing bimanual grasps on 3D objects. The BimanGrasp algorithm generates grasp poses by optimizing an energy function that considers grasp stability and feasibility. Furthermore, the quality of the synthesized grasps is verified using the Isaac Gym physics simulation engine. These verified grasp poses form the BimanGrasp-Dataset, which is the first synthesized bimanual dexterous hand grasp pose dataset to our knowledge. The dataset comprises over 150k verified grasps on 900 objects, facilitating the synthesis of bimanual grasps through a data-driven approach. Last, we propose a diffusion model (BimanGrasp-DDPM) trained on the BimanGrasp-Dataset. This model achieved a grasp synthesis success rate of 69.87% and significant acceleration in computational speed compared to BimanGrasp algorithm.


Pipeline

input

Our pipeline for synthesizing stable bimanual grasps, which includes: (A) generating grasp poses by initializing the bimanual grasp poses around the objects and then improving their quality through optimization; (B) verifying the grasp poses based on shape penetration and physics simulation using Isaac Gym; (C) utilizing the verified grasps (BimanGrasp-Dataset) to train a generative model (BimanGrasp-DDPM), with post-processing techniques to remove penetrations.


Optimization Steps

input

Visualization Environment

input

Qualitative results 1: Geometric Primitives

input

Qualitative results 2: Daily Objects

input

Qualitative results 3: Objects from GSO Dataset.

input

Qualitative results 4: Objects from DDG, YCB, and ContactDB.

input

Qualitative results 5: 3D Visualizations (6 instances).


Quantitive results 1: The average grasp success rate of bimanual grasping and unimanual grasping.

input

Quantitive results 2: Success rate (%) of BimanGrasp in IsaacGym under different friction coefficient settings. Object density is at ρ = 2500 kg · m−3

input

BibTeX

@article{shao2024bimanual,
        author={Shao, Yanming and Xiao, Chenxi},
        journal={IEEE Robotics and Automation Letters}, 
        title={Bimanual Grasp Synthesis for Dexterous Robot Hands}, 
        year={2024},
        volume={9},
        number={12},
        pages={11377-11384},
        doi={10.1109/LRA.2024.3490393}}