Dynamics-Guided Diffusion Model
for Robot Manipulator Design

Shift Down
Shift Right
Rotate Counterclockwise
Pose Convergence

Task-specific Designs without Task-specific Training


We present Dynamics-Guided Diffusion Model, a data-driven framework for generating manipulator geometry designs for a given manipulation task. Instead of training different design models for each task, our approach employs a learned dynamics network shared across tasks. For a new manipulation task, we first decompose it into a collection of individual motion targets which we call target interaction profile, where each individual motion can be modeled by the shared dynamics network. The design objective constructed from the target and predicted interaction profiles provides a gradient to guide the refinement of finger geometry for the task. This refinement process is executed as a classifier-guided diffusion process, where the design objective acts as the classifier guidance. We evaluate our framework on various manipulation tasks, under the sensor-less setting using only an open-loop parallel jaw motion. Our generated designs outperform optimization-based and unguided diffusion baselines relatively by 31.5% and 45.3% on average manipulation success rate. With the ability to generate a design within 0.8 seconds, our framework could facilitate rapid design iteration and enhance the adoption of data-driven approaches for robotic mechanism design.

Paper

Latest version: arXiv or here.

Code

Codebase

Team

Xiaomeng Xu1
Huy Ha1,2
Shuran Song1,2

1 Stanford University           2 Columbia University          

Technical Summary Video (with audio)

Results

Pose Convergence
The goal of pose convergence is to design fingers that always reorient a target object to a specified orientation when closing the gripper in parallel. As you can imagine, this manipulator can be quite useful in industrial settings such as assembly lines. When objects are fed in with different poses we can automatically align them to the same pose. Then the following robot can just blindly manipulate the object.

More examples of pose convergence manipulators in the real world:

Rotate
Counterclockwise
Clockwise

Shift Up/Down
Up
Down

Shift Left/Right
Left
Right

Multi-object Results

Our framework also allows designing manipulators for a set of objects to achieve a task.
Rotate Clockwise

BibTeX

@article{xu2024dynamics,
	title={Dynamics-Guided Diffusion Model for Robot Manipulator Design},
	author={Xu, Xiaomeng and Ha, Huy and Song, Shuran},
	journal={arXiv preprint arXiv:2402.15038},
	year={2024}
}

Contact

If you have any questions, please feel free to contact Xiaomeng Xu.