Dynamics-Guided Diffusion Model
for Sensor-less Robot Manipulator Design

Conference on Robot Learning (CoRL 2024)

Best Machine Learning Paper at the Morphology-Aware Policy and Design Learning Workshop

Shift Down
Shift Right
Rotate Counterclockwise
Pose Convergence

Task-specific Designs without Task-specific Training


We present Dynamics-Guided Diffusion Model (DGDM), a data-driven framework for generating task-specific manipulator designs without task-specific training. Given object shapes and task specifications, DGDM generates sensor-less manipulator designs that can blindly manipulate objects towards desired motions and poses using an open-loop parallel motion. This framework 1) flexibly represents manipulation tasks as interaction profiles, 2) represents the design space using a geometric diffusion model, and 3) efficiently searches this design space using the gradients provided by a dynamics network trained without any task information. We evaluate DGDM on various manipulation tasks ranging from shifting/rotating objects to converging objects to a specific pose. Our generated designs outperform optimization-based and unguided diffusion baselines relatively by 31.5% and 45.3% on average success rate. With the ability to generate a new design within 0.8s, DGDM facilitates rapid design iteration and enhances the adoption of data-driven approaches for robot mechanism design.

Paper

Latest version: arXiv or here.

Code

Codebase

Team

Xiaomeng Xu1
Huy Ha1,2
Shuran Song1,2

1 Stanford University           2 Columbia University          

Technical Summary Video (with audio)

Results

Pose Convergence
The goal of pose convergence is to design fingers that always reorient a target object to a specified orientation when closing the gripper in parallel. As you can imagine, this manipulator can be quite useful in industrial settings such as assembly lines. When objects are fed in with different poses we can automatically align them to the same pose. Then the following robot can just blindly manipulate the object.

More examples of pose convergence manipulators in the real world:

Rotate
Counterclockwise
Clockwise

Shift Up/Down
Up
Down

Shift Left/Right
Left
Right

Multi-object Results

Our framework also allows designing manipulators for a set of objects to achieve a task.
Rotate Clockwise

BibTeX

@article{xu2024dynamics,
	title={Dynamics-Guided Diffusion Model for Robot Manipulator Design},
	author={Xu, Xiaomeng and Ha, Huy and Song, Shuran},
	journal={arXiv preprint arXiv:2402.15038},
	year={2024}
}

Contact

If you have any questions, please feel free to contact Xiaomeng Xu.